VBP (64-bit) doesn't terminate at end of build

[jbenson]

I've been trying to set up a new build VM (Windows 8.1 64-bit on VMware). Unfortunately, Visual Build Pro 8.7 (64-bit) has been very "flaky" and unreliable during testing. The VBP is launched as a Windows scheduled task, runs under an administrator logon and the "Run with highest privileges" option is checked. No user is logged into the VM when the VBP task is launched.

I've done a lot of tinkering with the build project and made improvements but it's still not good enough. Some days - infrequently - it runs the entire build without trouble. Most days, however, it will get "stuck" on some build step and won't proceed beyond that point. I am basically out of ideas how to get past this.

Lately, I've found a combination of project steps that reliably runs to completion (i.e., the important build steps are all successful). The only problem is VisBuildCmd.exe doesn't terminate at the end but instead remains running. The Windows scheduled task is defined to terminate after a certain number of hours but it still leaves the VBP executable in memory. This tells me that VBP still suffers from some problem and can't be trusted.

Things I've tried:

* VisBuildPro.exe vs. VisBuildCmd.exe (the latter is currently in use)

* /MTA command-line option (seemed to have no effect or even make matters worse)

* Replace "Copy Files" VBP steps with alternatives (build no longer sticks on file copy operations, so this is an improvement)

* Implemented or confirmed all the other suggestions in the "stuck builds" FAQ

* Disabled the "Send Email" build step at the end of the project since VBP almost always sticks on this step (nothing written in the build log). By contrast, a "Send Email" step I added at the start of the build always works.

This morning I found three instances of VisBuildCmd.exe running in Task Manager - one instance from today's scheduled task, the others left from two previous days' builds. I ran the Sysinternals utility, Process Monitor, in an attempt to find information about these processes. According to ProcMon, the most recent VisBuildCmd.exe process seems to still be "active". Every minute or so, VBP engages in a flurry of registry activity, some of it involving HKLM\Software\Licenses, along with a small amount of file activity (C:\ProgramData\Temp, C:\ProgramData\Licenses folders). The two older processes appear to be entirely idle, according to ProcMon.

Can anyone suggest reasons for the above behavior? Or offer any ideas for how to get VBP to behave better in our environment?

Thanks

[kinook]

From that information, I would guess that the licensing / software protection component utilized by Visual Build (purchased from a now-defunct third party) is preventing the normal ending of the build process. How long has the build been running when this occurs? We've tried to reproduce this behavior with long-running builds launched from a scheduled task but have not succeeded.

One thing you could try is adding a Kill Process action at the end of the project to terminate the build process when it's done ([Builder.ProcessID]).

[jbenson]

The build has lately been taking 5 hours on average (give or take a few minutes).

Considering your answer, do you think I somehow messed up the VBP installation? Also, I may (probably?) have logged into the VM as a different admin user account when I installed VBP than the account the build runs under. Could that be part of the issue?

I had seen in the forums other advice to kill the VBP process but I was holding off in trying that approach until exhausting all other avenues (since it seems like an ugly hack).

Thanks for the help.

[kinook]

I wouldn't expect that to matter, but you could try uninstalling Visual Build, then reinstalling and registering with the user account that will be used for building.

[jbenson]

I previously tried setting the scheduled task to execute under my own logon account and the results were the same.

Is it possible the VBP licensing component is active because I neglected to register VBP? It's been so long since I installed VBP I don't recall (setting up this new VM has been an ongoing project for a long time, mostly relegated to the back burner).

[kinook]

I doubt that's the problem -- it probably wouldn't even build the project when unregistered, and would instead report a limit on the size of a project in eval mode.

[jbenson]

This is strange. Last Friday I added a new final step to my build project, a "Kill Process" step with [Builder.ProcessID] as the value. Tested the step in the VBP designer - VBP terminated. Let the build run over the weekend and again this morning for good measure, just to be sure what I was seeing. Despite the attempt to kill the process, VisBuildCmd.exe always remains running in Task Manager (and Task Scheduler says the latest task is still running). Monday morning there was three instances of VisBuildCmd.exe running (Saturday, Sunday and Monday builds). I rebooted the VM Monday and now (Tuesday) one VisBuildCmd.exe is running under this morning's scheduled task.

I am officially out of ideas. What could be going on?

[kinook]

What does the build log show? Did the Kill Process step execute? You might instead try configuring the scheduled task to stop the task if it runs longer than, say, 8 hours.

[jbenson]

The log says the Kill Process step executed. That's the last line in the log.

I already have an option set in the scheduled task to stop the task after a certain number of hours. The task shuts down as intended but the VisBuildCmd.exe process remains in memory. You see why this is all so maddening?

[kinook]

I guess whatever is preventing the process from exiting normally is also preventing it from being terminated. You might try a Run Program step with a command of

taskkill /f /pid [Builder.ProcessID]

Can you ZIP and send your .bld file and build log file to support@kinook.com? It might give us some ideas for reproducing the problem you're experiencing. Thanks.