This bit is the tricky bit. I had to upgrade multiple sites from vCloud 1.5.x to 5.1.3 and had a lot of problems. Thankfully I tested the upgrades in my lab and didn’t have those issues in the middle of the night when I upgraded the production sites.
So why is it tricky ? When vShield 5.1.3 was released it seems VMware just used the same upgrade procedures and replaced the version number from 5.1.2 to 5.1.3. That won’t work though. Or actually, it does sometimes.
Let me explain. In older versions of vShield the filesystem itself was relatively small. Specifically /common
Here I got at least 2.6GB available – this is great and only 2.5GB are needed in order to upgrade.
However, in a lot of environments, this drive is filled up with logs and other rubbish.
VMware did release a maintenance patch because of this. Basically all it does is cleaning up /common to make room for the upgrade file.
More info can be found here.
Now in my tests however, this worked fine for 5.1.2 – but failed 50% of the times for 5.1.3. Sometimes it worked when running the patch and reboot the VSM – cleaning up all sorts of files making JUST about enough room for the 5.1.3 file. But a lot of times – it failed.
By all means, try it, but I am posting here the way I upgrade our production systems, which worked 100% of the time. An indication of a failed upgrade is that after an initial reboot of the upgraded vShield appliance, it does not have any of your settings, nor do you see your datacenter. Bad, right. You also cannot just restore a settings backup as once upgraded, they will not be compatible anymore.
Unfortunately this is somewhat of an annoying process, easy enough, but annoying. You will see why. During the time I was writing this, 5.1.3 was replaced with 5.1.4 – so may as well go to that.
High Level Process is
1. Backup Settings
2. Install maintenance patch 5.0-939118 on vShield Manager 5.0.2
3. Upgrade vShield Manager from 5.0.2 to 5.1.2
4. Backup Settings
5. Deploy fresh vShield Manager 5.1.2 Appliance
6. Restore Settings
7. Install maintenance patch 5.1.2-997359
8. Upgrade vShield Manager from 5.1.2 to 5.1.4
9. Backup Settings
10. Deploy fresh vShield Manager 5.1.4 Appliance
11. Restore Settings
Easy, but annoying, right 🙂
Right, lets get started (VM Snapshots are your best friend here obviously).
1. Backup – most important thing (you should do that on regular basis anyway)
2. Apply maintenance bundle
VMware-vShield-Manager-upgrade-bundle-maintenance-5.0-939118.tar.gz
Go to Settings > Reports > Updates > Upload Upgade Bundle
Hit Install > Confirm Install
And let it do its thing.
And DO NOT panic when you see this
Just means it stopped the services and is rebooting. Just hit F5 after a few minutes and you should be back in business.
Login to the console of the VSM and confirm you have at least 2.5GB available.
Here you can see I got 2.6GB – just about enough to upgrade 5.1.2. I would bet 5.1.4 would fail 🙂
Anyway, now go ahead and upgrade to 5.1.2
3. Upgrade VSM to 5.1.2
Use the same procedure as above with the maintenance bundle. The VSM will automatically reboot. You can watch this on the console.
Once rebooted you may as well get some tea / coffee. The VSM will resync the inventory and it can take up to ten minutes. If you want to watch the process in the background, login to the console of the VSM.
There type the command
show manager log follow
You will see all sorts of “stuff” running through the screen. Once calmed down the VSM should be ready and you should be able to login.
Once the VSM is back up – confirm it has still all the bits, such as cluster(s) etc.
Edges (if any) are unlikely to show as the version of vShield Manager now does not match the version of the vShield Edges.
Don’t be too quick though. The web interface will be up before the actual inventory sync is completed.
If you are too quick, you might get funky errors, such as unknown error, unable to connect etc. – just log out and get another tea.
When seeing this error
Again, don’t panic, log off and get another beverage 🙂
Once the cake and tea is gone, have another look
You’ll notice the warning at the top regarding the virtual machine version. You can ignore that for now as we will be deploying a new VSM anyway.
4. Backup again !
Now backup your settings once again as seen in step #1
Annoyed yet ? No ? Good, lets continue
5. Deploy a new VSM
Now shutdown the VSM and deploy a new one. Here I am using the
VMware-vShield-Manager-5.1.2-943471.ova
You can obviously keep it as a backup for now but as you need the same IP, it needs to be shut down.
Once shut down, rename it as you likely want to use the same name too. But that obviously depends on your environment.
You can ignore these errors for obvious reasons:
Once you deployed the VSM and given it an IP, restore the settings again.
Here you can see why you deploy a new VSM – disk space. You finally got enough to play with
At the time of writing you need at least 2.5GB, here the new install has 41GB available – clearly VMware does not want to have these sort of issues again 😉
6. Restore Settings
After the initial boot and IP config (again, it can still take 5-10 minutes until you can login) restore the previous backup.
The VSM will once again automatically reboot, giving you enough time to get a drink.
Once the VSM is up – make sure you aren’t missing anything. As you can see – you will not see the Virtual Machine Version warning any more.
7. Install Maintenance Patch
Now you got vShield 5.1.2 running (or from this point on it is called vCloud Networking and Security).
Using the previous steps, install the maintenance patch for 5.1.2
VMware-vShield-Manager-upgrade-bundle-maintenance-5.1.2-997359.tar.gz
Did I mention tea ? VSM will reboot again so get a cuppa.
Once the VSM is back up, quickly confirm the new build (997359 instead of 943471)
Almost there …………
8. Upgrade vShield from 5.1.2 to 5.1.4
Should be pain-free. Same applies (follow previous steps), upgrade will take a while so have something .. tea ? Anyone ?
Not going to repeat what I have posted in the previous steps as they are identical.
Basically at this stage just
- Upgrade from 5.1.2 to 5.1.4
- Backup
- Power off upgraded 5.1.4 VSM
- Rename powered off upgraded 5.1.4 VSM
- Deploy fresh 5.1.4 VSM
- Restore Settings
- Confirm all is well
vCloud Director Upgrade Content
1. Upgrading vCloud Director Binaries / Database
2. Upgrading vShield Manager
3. Final Upgrade Touches