Waiting X minutes for host to come online

[mkrautler]

I am trying to automate our Windows Update patching as much as possible & have setup a multi-row sequence running an advanced job queue which needs to update Server A & reboot it, then check for updates & install (if more updates are needed), before Server B begins the same process, then Server C, Server D, etc… Currently I have the job queue setup to wait 5 minutes after it reboots before it does a wait for host to be online, just in case it takes a while to finish the updates. Unfortunately this whole process takes a very long time. My question is, what does BatchPatch do to determine if the host is back online, does it check a specific Windows service & if so, which one? I was hoping to avoid the 5 minute wait after reboot before checking if a server is back online & a ping is not good enough for my situation. But if a server is still at the Updating Windows screen before booting all the way back to the desktop, I need to give it enough time.

[doug]

BatchPatch first does a ping, if the ping is successful, it does a WMI query. If the WMI query is successful, BP considers the target host to be online.

The method for how you determine that a machine is online is probably less important than how you determine that the machine has first actually gone offline and then come back online. If you want to speed up your current process so that you are not always waiting the full 5 minutes in between reboots, if you’re dealing with physical hosts then I would suggest that you consider the job queue option for “Wait for host to go offline and come back online.” If you’re dealing with VMs that can reboot extremely quickly, then this item can be problematic. The issue is that if a reboot of a VM occurs within just several seconds, which isn’t uncommon, then the machine won’t be successfully detected by BP as having ever gone offline. For physical machines, which generally take many seconds (or even minutes) to perform their reboot process, we have not ever observed this to be an issue.

Another way to do this that you could consider is using a custom script that you write that checks the last bootup time property on the target computer before it executes a reboot, and then checks the last bootup time property again on the machine after that. The script can then compare them to determine if the machine has definitely been rebooted and is now back online. We might add something like this in a future version of BP, but at the moment you’d have to script it out yourself.

[Kipp]

I am having an issue where my job queue eventually terminates on the step of wait for computer to go offline then come back online – I’ve been thinking it’s a timeout when the server is working forever on the configuring updates screen, but this mention of vm’s sometimes going on and off so quickly, sounds spot on. Maybe it’s a combo of both I’m experiencing. Have you sorted out any better solution for the quick vm reboot stalling this out?

Thanks,

Kipp

[doug]

Under ‘Tools > Settings > Grid Preferences’ look at the “Hosts are considered offline after X ping timeouts.” The default value is 3. However, if you’re having a problem where your VMs are rebooting within just several seconds (the timespan of 3 ping timeouts), then try changing it to 2. That will usually resolve it. I don’t recommend changing it to 1 because if any machine has just a single ping timeout then BP will think that machine went offline, so you’ll get consistent false alerts if the value is set to 1. However, 2 is usually a good middle ground option that should work well in most situations.

==========================

The other option is don’t even use ‘Wait for host to go offline and come back online’. Instead do something like this:

-Step where reboot occurs
-Wait 10 minutes
-Wait for host to be detected online

This will generally give you a good middle ground option. In many situations a 5 min wait will work fine instead of 10, but 10 minutes should ensure that you don’t run into any issues where a machine is taking a very long time to complete the initial shutdown sequence. If you set the wait time too low, then BP will get to ‘Wait for host to be detected online’ and will find that the machine is online but not because it rebooted already but rather because it hadn’t yet actually shutdown and was still online and working to complete the initial shutdown sequence.

[Kipp]

Thank so much, I’ll play around with these suggestions!

[Author]

Posts