Migrate DP 8.XX to DP 8.XX using new hardware or different MS Windows operating system

The migration of the internal database to new hardware or to a new Microsoft Windows operating system was no big deal in the past and before the release of Data Protector 8.xx. With the introduction of PostgreSQL as the new internal database this has been changed and several steps are necessary in order to perform a successful migration. The following is a guide that enables you doing a migration to new hardware, or, for example, the move from Windows 2008 R2 to Windows 2012 R2. For a successful migration, some requirements must be met; furthermore the migration is dependent on various conditions. For the listed instructions no guarantee can be given and it is recommended to perform the migration with an HP partner. For the elaborate article I would be glad to receive some donations.

Prerequisites and conditions:

• The command omnidbcheck -extended shows no errors. For Data Protector version 8.10 it might be necessary to apply hotfix QCCR2A50694 first. Successful omnidbcheck:
  

  C:\>omnidbcheck -extended
  Check Level             Mode                    Status
  ===========================================================
  Database connection     -connection             OK
  Schema consistency      -schema_consistency     OK
  Datafiles consistency   -verify_db_files        OK
  Database consistency    -database_consistency   OK
  Media consistency       -media_consistency      OK
  SIBF(readability)       -sibf                   OK
  DCBF(presence and size) -bf                     OK
  OMNIDC(consistency)     -dc                     OK
  DONE!

• The old “Detail Catalog Binary Files” (pre DP 8.XX) were fully migrated. The command (example) "C:\Program Files\Omniback\bin\perl.exe" omnimigrate.pl -report_old_catalog shows following output:
  

  Old Detail Catalog Binary Files size:
  
  DCBF (   0 files):                    0 MB
  Old filename data files:              0 MB
  Total:                                0 MB
  

  If there are still old “Binary Files” in place, you have to migrate the files first. For this purpose, the command (example) "C:\Program Files\OmniBack\bin\perl.exe" omnimigrate.pl -start_catalog_migration can be used. If for the “DCBF” 0 files are displayed, but “Old filename data files” are still displayed, this may mean that there were modifications to the old dcbf directories during or after the migration. In this special case you can copy the DCBF files from db40 to the db80 folder and run omnidbutil -remap_dcdir and omnidbutil -fixmpos.

• The old catalog and the old “Binary Files” must be deleted after the migration from version 6, 7 to version 8 (Attention: see hint DB40). If this step has not yet been performed or if you are uncertain about the status, you can use the command (example) "C:\Program Files\Omniback\bin\perl.exe" omnimigrate.pl -remove_old_catalog to delete the old files. With a successful completion of the command, the file global will be adjusted and the support for the old catalog will be removed (SupportOldDCBF=1).
• There must not exist any DCBF directory from a previous (pre 8.xx DP) Data Protector version. The check can be done using the GUI – Internal Database. Alternatively, export the IDB with the command omnidbutil -writedb <Pathname>. After the export, and the note on to copy directories, no DB40 path must displayed. By the way: the export of IDB to a connected network share is possible since v8.xx. Check export for db40 directories:
  

  Please make a copy of the following Internal Database directories and then press Enter to bring the Internal Database back to a fully operational state:
  
          "C:/Program Files/OmniBack/server/db80/msg"
          "C:/Program Files/OmniBack/server/db80/meta"
          "C:/Program Files/OmniBack/server/db80/dcbf/dcbf3"
          "C:/Program Files/OmniBack/server/db80/dcbf/dcbf0"
          "C:/Program Files/OmniBack/server/db80/dcbf/dcbf2"
          "C:/Program Files/OmniBack/server/db80/dcbf/dcbf1"
          "C:/Program Files/OmniBack/server/db80/dcbf/dcbf4"
  

  Note: in the example the separation of “Program Files” and “Program Data” has not yet been performed, see also following items.

• Note db40: If there are any DCBF files in folder db40 left after migration from pre DP 8.XX to DP 8.XX, it might have different causes. The migration of DCBF files is absolutely necessary before removing the old catalog. To check for old “Binary Files”, save select medium_name, dcbf_directory, dcbf_version from dp_dcbf_info i, dp_dcbf_directory d where i.dcbf_directory_seq_id = d.dp_numkey order by dcbf_version; into file dcbf_version.sql and execute the command omnidbutil –run_script dcbf_version.sql –detail. The output displays the media ID, folder and DCBF version. A media with -1 has not been migrated so far and can be migrated with the command omnimm -upgrade_dcbf <mediumID>. If for the DCBF folders (db40 path – see export IDB) a back-slash, instead of a forward-slash is displayed, you are required to export the IDB (omnidbutil –writedb <Pathname>). Modify the file dpidb.dat and change for the DCBF folders the back-slash into a forward-slash. Save the file and import the modified IDB using the command omnidbutil –readdb <Pathname>. With the command omnidbutil -remove_dcdir <PathName> you can remove the old catalog and DCBF directories. The most important part in file dpidb.dat should look as follow (example and without separation of “Program Files” and “Programdata”):
  

  20cf4d3f-d425-4f01-8823-9b9e9dcfa72c	1004	C:/Program Files/OmniBack/server/db80/dcbf/dcbf3	214748364800	3	2147483648	100000	44	6964137984
  20cf4d3f-d425-4f01-8823-9b9e9dcfa72c	1001	C:/Program Files/OmniBack/server/db80/dcbf/dcbf0	214748364800	0	2147483648	100000	48	9023758336
  20cf4d3f-d425-4f01-8823-9b9e9dcfa72c	1003	C:/Program Files/OmniBack/server/db80/dcbf/dcbf2	214748364800	2	2147483648	100000	39	6941310976
  20cf4d3f-d425-4f01-8823-9b9e9dcfa72c	1002	C:/Program Files/OmniBack/server/db80/dcbf/dcbf1	214748364800	1	2147483648	100000	40	6340190208
  20cf4d3f-d425-4f01-8823-9b9e9dcfa72c	1005	C:/Program Files/OmniBack/server/db80/dcbf/dcbf4	214748364800	4	2147483648	100000	63	7780794368
  

• The separation of “Program Files” and “Programdata” has been done in the past. If this is not the case and you plan to do the separation during the migration on the target hardware or platform, some additional steps are required. In this guide the separation was not done so far, so all steps are documented.
• The used user names and passwords (Windows) to start the Data Protector services remain the same on the target hardware or platform. If you plan to change this and if you want to use new user and password, please refer to the article „Data Protector 8.0 Windows – change user account for DP services“ (link: https://www.data-protector.org/wordpress/2013/11/data-protector-8-0-windows-change-user-account-dp-services/).
• The article does not apply for migrations from pre DP 8.XX versions.
• On the source system the most current Data Protector version should be installed before the migration is done, as this will ease the migration procedure. This does apply for source systems only when the separation of “Program Files” and “Programdata” has already been done. The reason is a bug in the installation sequence when upgrading from DP 8.0X to 8.10. In this guide the same version of Data Protector is installed on the target and the upgrade will be done on the target system after the migration has been completed.
• The name of the server and the IP addresses remain the same on the target cell server. If you are required to change that, additional steps might be necessary (export of clients beforehand, import of clients after migration, change cell server name, etc. – this will not be part of this guide).
• The server name has to start with a leading letter and must not start with a leading number. The reason is a limitation in the included application server (JBOSS/Tomcat).
• The internal database has not been restored in the past and there must not exist several DB80 directories (original and restored versions).
• All names in this guide are examples.
• After the migration, make sure you immediately start a backup for the server and the internal database. A restore of the IDB with data before the migration might not work, especially when the DCBF directories were changed during the migration. This will be possible only, when additional modification in the exported database is performed.
• The guide does not apply to MoM environments.

Step by Step:

• Make a backup of the IDB in case any error occurs. Using a Jukebox with some slots makes it easy to do a fast recovery. The media will be formatted and the IDB backup started. In this example the backup was done to e:\IDBBackup\01..05 and copied off the server after the IDB backup. In case you do not have sufficient local disk space, use a filelibrary configured to store the data on network share. The definition of the Jukebox and the used drive will be exported into text files. Definition Jukebox:
  

  E:\Program Files\OmniBack\bin>omnidownload -library IDBBackup
  NAME "IDBBackup"
  DESCRIPTION ""
  HOST servername.domain
  POLICY Jukebox
  TYPE File
  REPOSITORY
          "E:\IDBBackup\01"
          "E:\IDBBackup\02"
          "E:\IDBBackup\03"
          "E:\IDBBackup\04"
          "E:\IDBBackup\05"
  MGMTCONSOLEURL ""
  

  Definition drive:

  E:\Program Files\OmniBack\bin>omnidownload -device IDBBackup_Drive01
  NAME "IDBBackup_Drive01"
  DESCRIPTION ""
  HOST servername.domain
  POLICY Jukebox
  TYPE File
  POOL "IDBBackup"
  LIBRARY "IDBBackup"
  CONCURRENCY 10
  ENCRCAPABLE
  BUFFERS 16
  BLKSIZE 64
  RESTOREDEVICEPOOL NO
  COPYDEVICEPOOL NO
  

  In case of any errors you can use these text files to re-create the Jukebox (see help for command omniupload).

• Stop the Data Protector scheduler using the command omnitrig -stop
• Export the internal database with the command omnidbutil –writedb <pathname> (in this guide Z:\ExportIDB is used). On request copy the directories DCBF, MSG and META off the server (i.e. Z:\Migration).
• Stop the Data Protector services using the command omnisv -stop
• Copy all Data Protector directories off the server (i.e. to Z:\Migration). Skip the copy for the directories DCBF, MSG and META, as this has been done in previous step. Comment: when copying the directories, all symbolic links in IDB structure will be copied as whole, so please plan for sufficient disk space on target. Once you copied the directories all relevant Data Protector parts have been backed up, however, this does not apply for any additional services installed on the source system.
• Install the new box or operating system. Use the same server name, IP addresses, domain, groups, users and passwords. In case of changes, you need to check the configuration files step by step (see below).
• Install HP Data Protector 8.XX to “D:\Program Files\Omniback” and “D:\Programdata\Omniback” (example). Use the same version, the same patches or hotfixes as installed on source system.
• Configure impersonation:
  

  C:\Users\administrator>omniinetpasswd -inst_srv_user domain\administrator
  User 'domain\administrator' is configured to be used by Installation Server.
  

• Stop the Data Protector services using the command omnisv -stop
• On demand change the local security policy. This is required, if you plan to start the “Data Protector INET” service using a domain user, instead of the NT Authority\SYSTEM account. Add the user to the policies “Impersonate a client after authentication” and “Replace a process level token”.
• Copy back the DCBF, MSG and META directories to the new path. In this guide the path D:\Programdata\OmniBack\server\db80 is used.
• Change the DCBF directories in the file dpidb.dat (see export of IDB). This step is required only for target systems where the Data Protector directories are changed during the migration. Change of DCBF directories from:
  

  COPY dp_catalog_dcbf_directory (db_uuid, seq_id, dcbf_directory, maxsize, fill_sequence, min_freespace, max_dcbf_files, cur_num_of_files, cur_occupied_space) FROM stdin;
  20cf4d3f-d425-4f01-8823-9b9e9dcfa72c	1002	E:/Program Files/OmniBack/server/db80/dcbf/dcbf1	214748364800	1	2147483648	100000	47	6341406720
  20cf4d3f-d425-4f01-8823-9b9e9dcfa72c	1004	E:/Program Files/OmniBack/server/db80/dcbf/dcbf3	214748364800	3	2147483648	100000	44	6964137984
  20cf4d3f-d425-4f01-8823-9b9e9dcfa72c	1001	E:/Program Files/OmniBack/server/db80/dcbf/dcbf0	214748364800	0	2147483648	100000	48	9023758336
  20cf4d3f-d425-4f01-8823-9b9e9dcfa72c	1003	E:/Program Files/OmniBack/server/db80/dcbf/dcbf2	214748364800	2	2147483648	100000	39	6941310976
  20cf4d3f-d425-4f01-8823-9b9e9dcfa72c	1005	E:/Program Files/OmniBack/server/db80/dcbf/dcbf4	214748364800	4	2147483648	100000	63	7780794368
  

  Change of DCBF directories to:

  COPY dp_catalog_dcbf_directory (db_uuid, seq_id, dcbf_directory, maxsize, fill_sequence, min_freespace, max_dcbf_files, cur_num_of_files, cur_occupied_space) FROM stdin;
  20cf4d3f-d425-4f01-8823-9b9e9dcfa72c	1002	D:/ProgramData/OmniBack/server/db80/dcbf/dcbf1	214748364800	1	2147483648	100000	47	6341406720
  20cf4d3f-d425-4f01-8823-9b9e9dcfa72c	1004	D:/ProgramData/OmniBack/server/db80/dcbf/dcbf3	214748364800	3	2147483648	100000	44	6964137984
  20cf4d3f-d425-4f01-8823-9b9e9dcfa72c	1001	D:/ProgramData/OmniBack/server/db80/dcbf/dcbf0	214748364800	0	2147483648	100000	48	9023758336
  20cf4d3f-d425-4f01-8823-9b9e9dcfa72c	1003	D:/ProgramData/OmniBack/server/db80/dcbf/dcbf2	214748364800	2	2147483648	100000	39	6941310976
  20cf4d3f-d425-4f01-8823-9b9e9dcfa72c	1005	D:/ProgramData/OmniBack/server/db80/dcbf/dcbf4	214748364800	4	2147483648	100000	63	7780794368
  

• Restore the during the migration copied files in D:\Programdata\OmniBack\log\server. The most important files are media.log, Ob2Event.* and the folder auditing.
• Modify or restore the configuration files global (example: D:\ProgramData\Omniback\Config\Server\Options) and omnirc (example: D:\ProgramData\Omniback).
• Modify or restore the files cell_info, lic.dat and installation_servers. Comment: After the upgrade to DP 8.1X all installed licenses become invalid and must be regenerated using the portal http://webware.hp.com. This requires a current support contract with HP.
• Restore the directories: BarLists, Barschedules, consolidationlists, copylists, Datalists, dlgroups, dr, Integ, rptgroups, rptschedules, Schedules, users. For the files in folder users crosscheck the entries before overwriting.
• Modify the files hpdp-idb-cp.cfg and idb.config in directory (example) D:\ProgramData\Omniback\Config\Server\idb. In both files modify the pathes.
  Example hpdp-idb-cp.cfg:

  [databases]
  hpdpidb = host=127.0.0.1 port=7112 dbname=hpdpidb
  
  [pgbouncer]
  service_name=hpdp-idb-cp
  listen_addr = *
  listen_port = 7113
  unix_socket_dir = /tmp
  auth_type = md5
  admin_users = hpdp
  stats_users = hpdp
  pool_mode = transaction
  server_reset_query =
  ignore_startup_parameters = application_name,extra_float_digits
  server_check_query = select 1
  server_check_delay = 10
  max_client_conn = 1200
  default_pool_size = 50
  query_wait_timeout = 10
  log_connections = 0
  log_disconnections = 0
  log_pooler_errors = 1
  logfile = D:\ProgramData\OmniBack\log\hpdp-idb-cp.log
  pidfile = D:\ProgramData\OmniBack\log\hpdp-idb-cp.pid
  auth_file = D:\ProgramData\OmniBack\config\server\idb\ulist
  

  Example idb.config:

  PGHOSTNAME='localhost';
  PGIDBNAME='hpdpidb';
  PGUSER='hpdpidb_app';
  PGPASSWORD='bWhhbnTvZ3Xxbao2Zw==';
  PGSUPERUSER='hpdp';
  PGSUPERPASSWORD='anpabZl3bnh6dGY3eQ==';
  PGMAXOPEN_CONN_NUMBER='5';
  PGDATA_PG='D:\ProgramData\OmniBack\server\db80\pg';
  PGDATA_IDB='D:\ProgramData\OmniBack\server\db80\idb';
  PGDATA_JCE='D:\ProgramData\OmniBack\server\db80\jce';
  PGOSUSER='domain\administrator';
  PGPORT='7112';
  PGCPPORT='7113';
  PGWALPATH='D:\ProgramData\OmniBack\server\db80\pg\pg_xlog_archive';
  IDB_SEC_FLAG='11';
  

• On demand and depending on configuration it might be necessary to restore (copy back) additional files and directories. For example, when Exchange is backed up in the Environment, the folder (example) D:\ProgramData\Omniback\server\db80\vssdb\metadata60 is required.
• Start the Data Protector services using the command omnisv -start.
• Stop the Data Protector scheduler using the command omnitrig -stop.
• Import the IDB using the command omnidbutil -readdb <pathname> (in the guide Z:\ExportIDB is used).
• Stop the Data Protector services using the command omnisv -stop.
• Start the Data Protector services using the command omnisv -start.
• Check the consistency of the internal database using the command omnidbcheck -extended. No errors must be reported.
• For further checks start some backups and restore some data for backups done before the migration and after the migration.
• After successful migration the used Jukebox (IDBBackup) and for the Jukebox used directories can be removed. All other directories used during the migration should be kept for at least 7 days.
• On demand and if not already done on the source system, the upgrade to the most current Data Protector version can be started.