TESTED: PowerOn&Run and Live Migrate from 3PAR Snapshots or SmartCache

With Data Protector 9.04 the new features PowerOn & Run and Live Migrate from 3PAR snapshots or SmartCache were introduced. This feature allows to power on and test virtual machines (VMware), backed up using the VE integration. Recently I had the opportunity to implement this feature in a proof of concept and to present the solution to the customer. I have also been able to talk with some partners and customers meanwhile and bottom line is: a successful implementation! Some discussions around these features lead to the question whether Veeam is still required as supplementary backup solution, as Data Protector provides one more time an another attractive recovery option for virtual machines in Enterprise environments.

In the mentioned PoC I had performed the backup of multiple virtual machines into the SmartCache. In addition I backed up some machines using Zero Downtime Backup and implemented it into the customers backup and recovery strategy. PowerOn and Migrate worked perfectly when I used 3PAR snapshots and the SmartCache and I encourage all customers to test these new Data Protector features. If no 3PAR in place, no problem, you can perform the test with the SmartCache as I did in addition.

Speaking perfectly, two problems raised up. In one case, an error message when turning on the virtual machine was displayed: “A question needs to be answered before power on can be completed” and as a result “Error powering on”. In the second case, an error message came: “Share presentation failed” and as a result “Power On Virtual Machine failed”. In both cases, however, Data Protector has shown only a consequence of failure of another cause. Before anyone comes across the same error, follow the explanation of the cause.

“A question needs to be answered before power on can be completed”

Using Virtual Center, the unanswered question was displayed with the comment “Diese virtuelle Maschine wurde möglicherweise verschoben oder kopiert…” (German message). In debug files following was seen:

[ 99] 2015-08-19 14:23:46.463 ("/integ/vep/vepa/Plugins/Vmware/VmwareHelpers/ConfigUtil.cpp $Rev: 49146 $ $Date:: ":293)
[ 99] ===>> (6) ConfigUtil::answerQuestion {
[ 99] 
[ 20] [ConfigUtil::answerQuestion] waiting for different question
_vmx1
Diese virtuelle Maschine wurde möglicherweise verschoben oder kopiert. Um bestimmte Verwaltungs- und Netzwerkfunktionen konfigurieren zu können, muss VMware ESX wissen, ob diese virtuelle Maschine verschoben oder kopiert wurde. Wenn Sie es nicht wissen, antworten Sie mit "Ich habe sie kopiert". 
[ 20] [ConfigUtil::answerQuestion] Throwing exception: class std::runtime_error

Hold On! German error message in Data Protector? The Virtual Center Server definitely was installed with English Locale, however, the service and user was not set to English Locale. So wondering why I translated this topic for you? The error might appear to you too, especially when not located in native English speaking countries. With the link (Thanks to Sebastian): http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=2121646 the cause was fixed very soon and PowerOn and Migrate worked as expected.

“Share presentation failed”

[Normal] From: RSM@hpcs.customer.local ""  Time: 8/10/2015 9:54:59 AM
	Restore session 2015/08/10-10 started.

[Normal] From: RMA@hpcs.customer.local "SmartCache_gw1 [GW 18252:0:17740599057275740933]"  Time: 8/10/2015 9:55:38 AM
	STARTING Media Agent "SmartCache_gw1 [GW 18252:0:17740599057275740933]"

[Normal] From: RMA@hpcs.customer.local "SmartCache_gw1 [GW 18252:0:17740599057275740933]"  Time: 8/10/2015 9:55:39 AM
	Loading medium from slot \\hpcs.customer.local\D:\Cache\358d640a_55d57510_0908_13a1 to device SmartCache_gw1 [GW 18252:0:17740599057275740933]

[Major] From: RMA@hpcs.customer.local "SmartCache_gw1 [GW 18252:0:17740599057275740933]"  Time: 8/10/2015 9:55:45 AM
	Share presentation failed.

[Major] From: OB2BAR_VEAgent@hpcs.customer.local "/DATACENTER"  Time: 8/10/2015 9:55:45 AM
	Received ABORT request from RSM (ERR: Error starting backup/restore (BMA cannot be started or similar).)

[Major] From: VEPALIB_VMWARE@hpcs.customer.local "/DATACENTER"  Time: 8/10/2015 9:55:45 AM
	Preparation of replica vm 'virtualmachine' for restore failed ...

[Critical] From: VEPALIB_VMWARE@hpcs.customer.local "/DATACENTER"  Time: 8/10/2015 9:55:45 AM
	Power On Virtual Machine failed.

After some debugging I found that port 111 was in use ( nestat -ano). A look at the process list for the given PID indicated the responsible process – ONC / RPC Portmapper. The process (from QLogic’s OneCommand) occupied the port 111 and thus prevented a start of the NFS service. Solution: Disable the service ONC / RPC portmapper, start NFS service, and immediately the presentation to the ESX server is working.

By the way: to purge the powered on machines the command omnidbutilwas enhanced. With command omnidbutil -purge_expired_poweron_vms -daily you can cleanup the environment.