Occasional build freeze

[petrhlavka]

Hello,

I've using Visual Build for two years without any problems. However in last month or two, some of my builds run through VisBuildCmd.exe start freezing. The freezes occur in various steps. The last step in the log file is usually running external command through cmd.exe, building with Visual Studio .NET 2003 SP1 or 'Set Macro' step.

Didn't somebody encounter similar problems? I'm trying to find what is causing this. I have the same problems on two computers, where no major OS/installation changes/updates weren't done for half a year.

Thanks for the replies.

Visual Build Professional 6.6
Registered to: GRISOFT - Brno, Czech Republic (1-site license)
Windows Version: 5.2.3790.2.0
Install path: C:\Program Files (x86)\VisBuildPro6
SftTree_IX86_U_50.dll version 5.06
unins000.exe version 51.47.0.0
VisBuildCmd.exe version 6.6.0.0
VisBuildPro.exe version 6.6.0.1
VisBuildBld.dll version 6.6.0.1
VisBuildBurn.dll version 6.6.0.0
VisBuildCore.dll version 6.6.0.0
VisBuildDotNET.dll version 6.6.0.0
VisBuildExt.dll version 6.6.0.0
VisBuildLog.dll version 6.6.0.0
VisBuildMisc.dll version 6.6.0.0
VisBuildMS.dll version 6.6.0.0
VisBuildMS2.dll version 6.6.0.0
VisBuildNet.dll version 6.6.0.0
VisBuildSvr.dll version 6.6.0.0
VisBuildSvr.Interop.dll version 1.0.0.0
VisBuildVCS.dll version 6.6.0.0

[kinook]

Is VisBuildCmd.exe definitely hung (can it be aborted via Ctrl+C or Ctrl+Break?)? Is it using any CPU cycles? Is the process that it launched for the current step (devenv for VS.NET) still running, and if so, does killing it allow VBP to continue? Are there any recent errors reported by VBP in the Windows Event Viewer? Could you try re-downloading and installing again from http://www.kinook.com/Download/VisBuildProEval.exe?

Please ZIP and send to support@kinook.com:
1) The .bld file
2) A VS solution that manifests the problem
3) A build log file

Thanks.

[petrhlavka]

VisBuildCmd.exe seems to me to be hung, it's not using any CPU cycles only single thread is being switched into context. The started process is always finished. The build is started through sheduler, so I don't know how to break it. I can only kill it. There is nothing suspicious in event log. The builds have over 200 steps, if I reorder some of them or remove any, the build starts freezing at another step so unfortunately it's hard to narrow the problem.

I will try to reinstall the VisualBuild, as the freezes occur only occasionally (however, on the same steps of build) and recently, it seems to me that some other software or OS updates can be casing this problem.

[NathanGaida]

I've been experiencing the same thing. OS is Server2003. After upgrading to VBP6.6 our builds have been locking up randomly. Generally we run them w/ GUI, and what we see when we RDP to the machine where the build is locked is that VBP is up and running, but the program window is white and does not respond.

Ironically we have 5 of these machines, and only 2 of them are experiencing the lockup issue (randomly), the other 3 have not.

We have seen a similar issue in the past, and that was resolved by a "stealth" update Kinook did. Ie the version was still 6.4, but they updated one of the DLLs to 6.4.0.1.

[petrhlavka]

Hmmm. It seems that there is a new 6.6, release December 31, 2007. The previous one came from September, there is no Changelog/Annoucement in the forum. I will try it.

[NathanGaida]

I think I've been using the latest 6.6. I did a compare between one of the machines that is working and the one that isn't. I thought I had updated all of my machines to 6.6, but apparently that isn't true. The working machines are running 6.4, the random lockup machines are running 6.6.

[kinook]

Originally posted by NathanGaida
I've been experiencing the same thing. OS is Server2003. After upgrading to VBP6.6 our builds have been locking up randomly.

Which version did you upgrade from? At what point in the build does it typically lock up? For instance, does it always hang on an action that runs an external program? Can you ZIP and send (to support@kinook.com) the .bld file and a build log file?

Generally we run them w/ GUI, and what we see when we RDP to the machine where the build is locked is that VBP is up and running, but the program window is white and does not respond.

Does building via the console app instead of the GUI make a difference?

Ironically we have 5 of these machines, and only 2 of them are experiencing the lockup issue (randomly), the other 3 have not.

Please ZIP and send the VBP Help | About | Install Info details for each of the machines (indicating which versions go with the good and problem machines).

We have seen a similar issue in the past, and that was resolved by a "stealth" update Kinook did. Ie the version was still 6.4, but they updated one of the DLLs to 6.4.0.1.

I don't recall us having fixed an issue similar to this (we did fix a hang issue with the Replace in File action that you reported in 2003, but not in an interim build [there was a workaround]), but we do sometimes patch the download (without a main version number change) to fix a problem reported by one user that isn't likely to affect other users. We haven't made any changes to resolve the problem you're reporting now, but you can try downloading and installing the latest build if you don't have it (see http://www.kinook.com/Forum/showthre...?threadid=3056 for details).

Do you have any additional software installed on the build box (anti-virus, anti-spyware, etc.)? What sort of hardware are you running on? Is there anything from VBP in the Event Viewer? If you can narrow the problem down to a reproducible case that we can build here, please ZIP and send it. Thanks

[NathanGaida]

I would have been upgrading from 6.4.

Where does it lock up:
I've looked at the past 3 log files where it has frozen, and each one had it lock up right as it was compiling a VC2003 DLL.

We don't run the console app, so I don't know. And as it happens randomly (actually, it seems to happen at the worst possible times, freezing the nightly build when QA is really needing something in the morning) I can't comment on the difference.

Re: the stealth update. The one I referred to in my post wasn't the bug fix on replace files. I ran into a similar freeze issue with a recent version of Visbuild6 and searched the forums. Someone else mentioned that there was a new update of visbuild6 with a dll version# increment that fixed their problem, and it fixed ours as well (at the time).

Nothing else running on our machines that would interfere like that.

I'll have an email sent to support shortly.

[NathanGaida]

As a followup, the previous 6.6 I had installed was downloaded on the 17th of Dec, before your update on the 31st of Dec.

[kinook]

Originally posted by NathanGaida
We don't run the console app, so I don't know. And as it happens randomly (actually, it seems to happen at the worst possible times, freezing the nightly build when QA is really needing something in the morning) I can't comment on the difference.

Are the builds being started from a Scheduled Task, or how are they started?

Re: the stealth update. The one I referred to in my post wasn't the bug fix on replace files. I ran into a similar freeze issue with a recent version of Visbuild6 and searched the forums. Someone else mentioned that there was a new update of visbuild6 with a dll version# increment that fixed their problem, and it fixed ours as well (at the time).

Which forum thread?

[NathanGaida]

Scheduled task, which just starts a batch file.

As for the other forum thread, I don't know which it was. It was probably 6months ago, and I don't remember what I searched for to find it.

[score]

I'm seeing this problem too. I installed VBP 6.6 in the beginning of the year and am having it hang. It seems to be random. I am also running on windows 2003 server. I use it through remote desktop. The build normally takes about 4 hours, when I come back to check on it after an hour all I see is a white screen. Most of the time the build finishes without error. However it has happened twice today. This is getting frustrating. Management types are getting on my back!

[NathanGaida]

score,

I'd recommend downgrading to 6.4 (dunno about 6.5? but going back to 6.4 fixed my problem for now.)

Luckily I'm not worried about a long term fix, as they are out-sourcing us and in 2 weeks I no longer have to care about it working or not.

[score]

how do we get an older version?

[NathanGaida]

That I don't know. I happened to have forgotten to upgrade a machine from 6.4, so it was convenient for me. (That was how i discovered that 6.6 was the problem. The 6.4 machine always worked, and I just assumed I had updated them all)