In the past I often informed about the free EADR feature in Data Protector to recover clients and cell server. The last time I wrote an article in 2014 including two small scripts to get things prepared and automated – see https://www.data-protector.org/wordpress/2014/04/eadr-cell-server-dp-8-1x-windows-2012-r2/. However, the batch files got broken due to changes for omniiso and so it was time to fix this problem, providing new batches and delivering a complete picture of disaster recovery for Data Protector Cell Manager. Get inspired for your choice of disaster recovery; the new batches can be found at the end of this article. All content applies to Cell manager running on Windows with current Data Protector version 9.06
Preparation:
- Plan the required steps
- Consider which Media Agents and devices to be used in case you need to execute EADR
- Select the desired DR method
- Document the media where IDB (backup Cell Manager) was written to.
- Verify you have access to the location of the required files
- Create a step-by-step checklist
- Create and execute EADR to verify it works in your environment and get trained for EADR on regular basis
- Avoid disaster by implementing Cell Server in MS Cluster, MC/ServiceGuard, VMware, …
- Get prepared
- Run regular and consistent backups (daily IDB, daily filesystem)
- Document important settings (TCP/IP properties, network cards and names, …)
- Include CONFIGURATION object into filesystem backup and make sure “Automatic Disaster Recovery Component” is installed on Cell Server
- Store important files in a safe place (SRD, obrindex.dat, …)
DR methods:
- AMDR – Assisted Manual Disaster Recovery
- For DR the installation of a disaster recovery operating system is required
- The original operating system is recovered by omnir command
- Requirements
- Disks of same or bigger size
- Type of filesystem and attributes of the new volumes must match
- Hardware and hardware configuration of new system must be the same as original
- OBDR – One Button Disaster Recovery
- All required information to recover are collected during backup
- OBDR device (emulating CD-ROM) used to boot directly from tape
- System is recovered with DP as it was at the time of backup
- Not available to recover Windows Cell Manager
- Requirements
- Tape device must support boot option
- Hardware and hardware configuration of new system must be the same as original
- Disks of same or bigger size
- EADR – Enhanced Automated Disaster Recovery
- Preferred method to recover Data Protector Cell Managers and clients
- Preferred method to recover Data Protector Cell Managers and clients that are part of Microsoft Cluster Server
- Requirements
- Prepare recovery CD, USB stick or network bootable image in advance and before disaster occurs
- Prepare removeable media containing encryption keys
- Disks of same or bigger size
- IDB backup must be newer than filesystem backup of Cell Manager
For any additional information you may refer to the “Disaster Recovery Guide”. The EADR – Enhanced Automated Disaster Recovery is the preferred method (and my favorite) to recover a cell server.
Support:
- EADR support
- Windows 2008 (x86/x64) – requires WAIK (Windows Automated Installation Kit) – Windows Vista/Windows Server 2008
- Windows 2008 R2 (x64) – requires WAIK (Windows Automated Installation Kit) – Windows 7/Windows Server 2008 R2
- Windows 2012 (x64) – requires ADK (Windows Assessment and Deployment Kit) – Windows 8/Windows Server 2012. Supports dynamic disks
- Windows 2012 R2 (x64) – requires ADK (Windows Assessment and Deployment Kit) – Windows 8.1/Windows Server 2012 R2. Supports dynamic disks
- Additional information
- OBDR on Windows supported for clients only
- OBDR / EADR on Linux supported for clients and Cell Managers
- On Windows and for EADR EGI/UEFI and GPT partitions are supported
- DR ISO’s for Linux systems must be created on Linux systems
- Comments for Cluster configurations
- Cell Server on Windows – manual or EADR
- Cell Server on Linux Cluster – MC/ServiceGuard < 11.20.20, RHEL Cluster Suite and SLES HA Cluster only as client
- For a complete list and additional restrictions refer to “HPE Data Protector 9.0x Disaster Recovery Support Matrix”
- Information in this document based on DP 9.06
You need to have the corresponding version of WAIK/ADK installed on your cell server. If you need to prepare for other Windows versions (normal Data Protector clients) as well, you need to install the specific WAIK/ADK version on separate Windows server (small virtual machine sufficient). For the installation of the ADK, installation of Deployment Tools and Windows PE are required only.
Preparing EADR on Cell Manager:
- Prerequisites
- Have the Disaster Recovery Component installed on the Cell Manager
- Have the correct ADK (Windows Assessment and Deployment Kit) installed – Windows 8.1/Windows Server 2012 R2 (Deployment Tools + Windows PE required)
- Have sufficient disk space available
- Automount
- Enabled – required for System Reserve partition
- Disabled – mount System Reserve partition (drive letter)
- The operating system must be activated (otherwise DR may fail)
- Do not select backup object versions which belong to checkpoint restart backup sessions (resume sessions)
- The required scripts are stored in
OMNIBIN
folder, in most casesC:\Program Files\OmniBack\bin\
- Create a filesystem backup specification including boot partition, DP installation volume, CONFIGURATION object. Adjust different settings like backup targets, schedule, etc.
- Define the Post-exec script “EADR_CS_FSBackup_Post.cmd”
- The disaster recovery image is saved to the Cell Manager during backup (P1S files and ClientName.img). Setting this option is much faster from the disk than from a backup medium, and is required here to automate stuff.
- Configure WinFS Options and when all settings configured safe the specification.
- Create the IDB backup specification
- Define the Post-exec script “EADR_CS_IDBBackup_Post.cmd”
The backup and the scripts:
- Filesystem backup specification for the cell server is executed, at the end post-exec stores information about session ID to file and executes the IDB backup.
- Script – EADR_CS_FSBackup_Post.cmd:
- Backup specification for IDB is executed, at the end post-exec creates ISO image.
- In some cases the session might get marked completed and some session messages will not appear.
- Script EADR_CS_IDBBackup_Post.cmd:
SET OMNIHOME=C:\Program Files\Omniback SET OMNIDATA=C:\Programdata\Omniback SET IDBBACKUP=CellServerIDB SET FS_SESSION_FILE=EADR_CS_FSBackup_SESSIONID.txt echo %SESSIONID% > "%OMNIDATA%\tmp\%FS_SESSION_FILE%" "%OMNIHOME%\bin\omnib.exe" -idb_list "%IDBBACKUP%" -barmode full -no_monitor
SET OMNIHOME=C:\Program Files\Omniback SET OMNIDATA=C:\Programdata\Omniback SET OMNIISOCOMMAND="%OMNIHOME%\bin\omniiso.exe" SET CELLSERVER=w2012r2dpcs.localdomain SET ISOPATH=%OMNIDATA%\tmp\ SET ADKPATH=C:\Program Files (x86)\Windows Kits\8.1\ SET MEDIAFILE_CS=usedmedia_IDB_CS.txt SET MEDIAFILE_FS=usedmedia_IDB_FS.txt SET SRDFILE=%OMNIDATA%\Config\Server\dr\srd\%CELLSERVER% SET P1SFILE=%OMNIDATA%\Config\Server\dr\p1s\%CELLSERVER% SET P1SIMAGE=%OMNIDATA%\Config\Server\dr\p1s\%CELLSERVER%.img SET /p "FS_SESSIONID="< "%OMNIDATA%\tmp\EADR_CS_FSBackup_SESSIONID.txt" SET IDB_SESSIONID=%SESSIONID% SET NETWORKSHARE=\\192.168.253.141\dr if exist "%ISOPATH%%CELLSERVER%.iso" (del /Q "%ISOPATH%%CELLSERVER%.iso") "%OMNIHOME%\bin\omnisrdupdate" -session %FS_SESSIONID% %IDB_SESSIONID% -location "%NETWORKSHARE%" -anyobj %OMNIISOCOMMAND% -session %FS_SESSIONID% %IDB_SESSIONID% -cd -anyobj -out "%ISOPATH%%CELLSERVER%.iso" -srd "%NETWORKSHARE%\recovery.srd" -rset "%P1SFILE%" "%P1SIMAGE%" -autoinject "%OMNIHOME%\bin\omnidb.exe" -session %SESSIONID% -media > "%ISOPATH%%MEDIAFILE_CS%" "%OMNIHOME%\bin\omnidb.exe" -session %SESSIONID% -media -detail >> "%ISOPATH%%MEDIAFILE_CS%" "%OMNIHOME%\bin\omnidb.exe" -session %FS_SESSIONID% -media > "%ISOPATH%%MEDIAFILE_FS%" "%OMNIHOME%\bin\omnidb.exe" -session %FS_SESSIONID% -media -detail >> "%ISOPATH%%MEDIAFILE_FS%" if exist "%ISOPATH%%CELLSERVER%.iso" ( copy "%ISOPATH%%CELLSERVER%.iso" "%NETWORKSHARE%" /Y copy "%ISOPATH%%MEDIAFILE_CS%" "%NETWORKSHARE%" /Y copy "%ISOPATH%%MEDIAFILE_FS%" "%NETWORKSHARE%" /Y ) exit
EADR at work:
- For the example VMware workstation was used to recover the cell server. The ISO was mounted to the VM and in EFI Boot Manager use CDROM was selected.
- The virtual machine starts from CDROM – DR ISO image
- The system is prepared for the restore, network is initialized and information regarding hardware and DR read from minios
- Select recovery scope and press finish
- The process continues
- The DR wizard is started
- Data Protector client is installed
- Restore process has been started
- The Offline Restore begins
- The IDB is restored at the end and after all filesystem objects, DR process is finished
- Some post tasks are executed and the system is ready to reboot
- After the recovered server has been started, verfiy IDB is consistent
Devices:
The EADR process demonstrated in this article was tested and verified for Windows 2012 R2 Cell Server using Filelibrary running on different host. Supported device and media agent combinations
- Filelibray on different server (media agent)
- StoreOnce Software Store on different server (media agent)
- StoreOnce
- Local attached tape devices
- Local attached tape libraries (you may load the media manually using uma command)
- Tape devices on different server (media agent)
- Tape libraries on different server (media agent)
Donwload:
EADR for Cell Server – DP 9.06 on Windows 2012 R2
Great. Thanks for sharing 🙂
Bye the way, can you share the steps to recover IDB in DP 9.06 in case IDB was backed up to Storeonce Software, which CM is also SOS server? Thanks alot
Hi Tai,
when the backup device and CM are on the same server, EADR will not work at all, because the Software Store would not be accessible.
So for EADR you need to have at least the device for IDB backup on a different server or it has to be a physical device.
Try considering StoreOnce VSA or appliance to avoid such a situation.
At least I will post an article very soon describing how to recover IDB only (and assuming server still working).
Best regards
Daniel
Pingback: OMNIOFFLR - Offline Restore Data Protector Internal Database
This is great but i cant seem to get the omniiso command working. I enabled the debug lines in the script and ran the command in isolation (after running the whole backup to get valid session id’s):
Creating the Disaster Recovery ISO image file. This may take a few minutes…
Cannot create the Disaster Recovery ISO image file.
Can’t see anything in logs or any kind of error message, Cellmanager is windows 2012
Hi Barry,
you may try it from command-line first and try how far you can get it. Unfortunately omniiso command does not provide any specific error I can check for when I had the problem with older versions of DP. I tested on W2012R2 using DP 9.06.
Best regards
Daniel
Thanks for the reply.
I have got a bit further and now have something specific to go on. In the auto DR log, it complains of a missing file from the ADK install:
20160705T154347 INFO redirector Output: An error occurred trying to open – C:\Program Files (x86)\Windows Kits\8.1\Assessment and Deployment Kit\Windows Preinstallation Environment\amd64\WinPE_OCs\WinPE-NetFX4.cab Error: 0x80070003
20160705T154347 INFO redirector Returning: 3 (3).
20160705T154347 ERROR image_creator We catch some unknown exception.
20160705T154347 INFO command Command: coolie (78, Dismounting/Committing MiniOS image.)
20160705T154347 INFO redirector Running command: C:\Program Files (x86)\Windows Kits\8.1\Assessment and Deployment Kit\Deployment Tools\amd64\DISM\dism.exe
20160705T154347 INFO redirector Options: /unmount-Wim /MountDir:”E:\ProgramData\OmniBack\drim\tmp\740DA6C0-89D0-4939-80AB-8626BD383F75\mount” /Commit /Quiet
20160705T154347 INFO redirector Timeout: 30 (min)
20160705T154405 INFO redirector Returning: 0 (0).
20160705T154405 INFO image_creator The unmount of Windows PE image succeeded.
20160705T154405 FATAL image_creator EXCEPTION at imgcreator.cpp(986):
20160705T154405 FATAL image_creator class drm::internal_error(109): The installation of Windows PE package (WinPE-NetFX4-Package) failed with an error condition (3).
Whilst I have WinPE-NetFX.cab in ADK, I do not have WinPE-NetFX4.cab. Doesn’t seem to be downloadable from microsoft.
Hi Barry,
are you sure you installed the right version of ADK? And that it installed all files required?
Best regards
Daniel
Ah, for some reason I had put 8.1 instead of 8.0 on.
Changing that has resolved the issue. Thank you for your assistance!
Hi Daniel,
You deserve an Olympic medal for this document! It is just what I had been looking for (HPE’s DR Guide is rather confusing in parts), and the method works like a dream, quickly resulting in an ISO on our offsite storage, available for use in case our main data centre is destroyed. The screen-shots and scripts were fantastically helpful, too.
The next stage is to burn & test the ISO . . . but first we need to find identical hardware.
Or maybe we just take all the disks out of our Cell Manager (and label them carefully), load it up with spare disks, and insert the ISO . . .
Thank you very very much!!
Ian Button
Halifax, UK
Hi Daniel,
Kindly advise on the difference between Online Restore, Remote Offline Restore, Local Offline Restore mode during EADR. Would it be possible to run Cell Manager EADR itself which has Media Agent installed without using other Media Server.
Regards,
Honng
Hi Teck,
Below the meaning of the terms:
Online Restore – the Cell Manager is still available and you would restore the IDB to a new location
Remote Offline Rerstore – in this case you execute omniofflr from any supported client to restore the Cell Manager
Local Offline Restore – the Cell Manager is not available. Of course you can use the local media agent. However, in this case only tape devices oder StoreOnce are working
Best regards
Daniel
Hi Daniel,
Thanks a lot for the explanation.
It seems like Cell Manager alone cannot perform Local Offline Restore while itself is Media Agent with StoreOnce. It works with another Media Agent Server only.
Kind regards,
Honng
Hi Teck,
it’s depending on DP version… with 9.08 it will be fixed.
Best regards
Daniel