Data Protector - Passion for Data Protection

data-protector.org

OMNISPEED 2.0 for Data Protector 8.XX on HP-UX, Linux and Windows

Posted on by 2 Comments

The new version of OMNISPEED is available. Version 2.0 was especially developed for Data Protector 8.XX environments and supports HP-UX (Itanium) 11.31, Linux (RedHat and SUSE Linux Enterprise Server) and Microsoft Windows (2008 R2, 2012 and 2012 R2) for the Data Protector versions 8.00, 8.01 and 8.1X. The most important change is a direct connection to the internal database, therefore the tool must run on the cell server directly. Other changes include the increased runtime of 48 hours, to run the script for X times and the calculation for B2D devices (StoreOnce Gateways – Software/Hardware). The tool supports the international versions of Data Protector; this includes English, French, Japanese and tradtional Chinese, no matter which reginal settings are used. However, the output is in English. Because of the direct connection to the internal database, the runtime is faster. This allows an decreased value for the measurements; it is set to three, instead of five seconds.

For more information, please refer to the help. I look forward to further suggestions and hope you will enjoy the tool.

[wpdm_file id=22]

*******************************************************************************
*                                                                             *
* OMNISPEED v 2.00 - Average Speed of DP Sessions                             *
*                                                                             *
* CoPyRiGhT (c) 2014  Data-Protector.org - Daniel Braun                       *
*                                                                             *
*******************************************************************************
Starting script on: rx2600 - 27.03.2014, 17:17:18

Usage:
omnispeed.exe [options]

OPTIONS:
  --separated           output will be separated by ';'
  --continous, -c       script runs in loop mode until CTRL-C is
                        pressed. When no session is running, the script
                        stops automatically. The script will wait for three
                        seconds between each loop and stops after 57600
                        loops = 48 h.
  --sessionid, -s       get speed from completed session or more
                        details from current session.
  --number,    -n       number of cycles before exit. The script will wait
                        for three seconds between each loop and stops after
                        57600 loops = 48 h. Script can be stopped using CTRL-C.
  --debug,     -d       debug mode, output is written to omnispeeddebug.log.
  --exitcodes, -e       prints out the exit codes used in script.
  --version,   -v       displays the script version.
  --help,      -h       prints this help.

Omnispeed must run on cell server, remote execution not allowed.
Cause: Some commands are available on cell server only. In addition
the SQL commands must be executed locally.
Hint: On Windows cell server psexec can be used for remote execution.
Example psexec: psexec.exe \\servername -u domain\user -h cmd.exe
        After the password has been entered, omnispeed can be started.

Mail bug reports and suggestions to daniel-braun(at)data-protector.org.

Migrate DP 8.XX to DP 8.XX using new hardware or different MS Windows operating system

Posted on by 13 Comments

The migration of the internal database to new hardware or to a new Microsoft Windows operating system was no big deal in the past and before the release of Data Protector 8.xx. With the introduction of PostgreSQL as the new internal database this has been changed and several steps are necessary in order to perform a successful migration. The following is a guide that enables you doing a migration to new hardware, or, for example, the move from Windows 2008 R2 to Windows 2012 R2. For a successful migration, some requirements must be met; furthermore the migration is dependent on various conditions. For the listed instructions no guarantee can be given and it is recommended to perform the migration with an HP partner. For the elaborate article I would be glad to receive some donations.

Prerequisites and conditions:

  • The command omnidbcheck -extended shows no errors. For Data Protector version 8.10 it might be necessary to apply hotfix QCCR2A50694 first. Successful omnidbcheck:
    C:\>omnidbcheck -extended
Check Level             Mode                    Status
===========================================================
Database connection     -connection             OK
Schema consistency      -schema_consistency     OK
Datafiles consistency   -verify_db_files        OK
Database consistency    -database_consistency   OK
Media consistency       -media_consistency      OK
SIBF(readability)       -sibf                   OK
DCBF(presence and size) -bf                     OK
OMNIDC(consistency)     -dc                     OK
DONE!
  • The old “Detail Catalog Binary Files” (pre DP 8.XX) were fully migrated. The command (example) "C:\Program Files\Omniback\bin\perl.exe" omnimigrate.pl -report_old_catalog shows following output:
    Old Detail Catalog Binary Files size:

DCBF (   0 files):                    0 MB
Old filename data files:              0 MB
Total:                                0 MB

    If there are still old “Binary Files” in place, you have to migrate the files first. For this purpose, the command (example) "C:\Program Files\OmniBack\bin\perl.exe" omnimigrate.pl -start_catalog_migration can be used. If for the “DCBF” 0 files are displayed, but “Old filename data files” are still displayed, this may mean that there were modifications to the old dcbf directories during or after the migration. In this special case you can copy the DCBF files from db40 to the db80 folder and run omnidbutil -remap_dcdir and omnidbutil -fixmpos.

  • The old catalog and the old “Binary Files” must be deleted after the migration from version 6, 7 to version 8 (Attention: see hint DB40). If this step has not yet been performed or if you are uncertain about the status, you can use the command (example) "C:\Program Files\Omniback\bin\perl.exe" omnimigrate.pl -remove_old_catalog to delete the old files. With a successful completion of the command, the file global will be adjusted and the support for the old catalog will be removed (SupportOldDCBF=1).
  • There must not exist any DCBF directory from a previous (pre 8.xx DP) Data Protector version. The check can be done using the GUI – Internal Database. Alternatively, export the IDB with the command omnidbutil -writedb <Pathname>. After the export, and the note on to copy directories, no DB40 path must displayed. By the way: the export of IDB to a connected network share is possible since v8.xx. Check export for db40 directories:
    Please make a copy of the following Internal Database directories and then press Enter to bring the Internal Database back to a fully operational state:

        "C:/Program Files/OmniBack/server/db80/msg"
        "C:/Program Files/OmniBack/server/db80/meta"
        "C:/Program Files/OmniBack/server/db80/dcbf/dcbf3"
        "C:/Program Files/OmniBack/server/db80/dcbf/dcbf0"
        "C:/Program Files/OmniBack/server/db80/dcbf/dcbf2"
        "C:/Program Files/OmniBack/server/db80/dcbf/dcbf1"
        "C:/Program Files/OmniBack/server/db80/dcbf/dcbf4"

    Note: in the example the separation of “Program Files” and “Program Data” has not yet been performed, see also following items.

  • Note db40: If there are any DCBF files in folder db40 left after migration from pre DP 8.XX to DP 8.XX, it might have different causes. The migration of DCBF files is absolutely necessary before removing the old catalog. To check for old “Binary Files”, save select medium_name, dcbf_directory, dcbf_version from dp_dcbf_info i, dp_dcbf_directory d where i.dcbf_directory_seq_id = d.dp_numkey order by dcbf_version; into file dcbf_version.sql and execute the command omnidbutil –run_script dcbf_version.sql –detail. The output displays the media ID, folder and DCBF version. A media with -1 has not been migrated so far and can be migrated with the command omnimm -upgrade_dcbf <mediumID>. If for the DCBF folders (db40 path – see export IDB) a back-slash, instead of a forward-slash is displayed, you are required to export the IDB (omnidbutil –writedb <Pathname>). Modify the file dpidb.dat and change for the DCBF folders the back-slash into a forward-slash. Save the file and import the modified IDB using the command omnidbutil –readdb <Pathname>. With the command omnidbutil -remove_dcdir <PathName> you can remove the old catalog and DCBF directories. The most important part in file dpidb.dat should look as follow (example and without separation of “Program Files” and “Programdata”):
    20cf4d3f-d425-4f01-8823-9b9e9dcfa72c	1004	C:/Program Files/OmniBack/server/db80/dcbf/dcbf3	214748364800	3	2147483648	100000	44	6964137984
20cf4d3f-d425-4f01-8823-9b9e9dcfa72c	1001	C:/Program Files/OmniBack/server/db80/dcbf/dcbf0	214748364800	0	2147483648	100000	48	9023758336
20cf4d3f-d425-4f01-8823-9b9e9dcfa72c	1003	C:/Program Files/OmniBack/server/db80/dcbf/dcbf2	214748364800	2	2147483648	100000	39	6941310976
20cf4d3f-d425-4f01-8823-9b9e9dcfa72c	1002	C:/Program Files/OmniBack/server/db80/dcbf/dcbf1	214748364800	1	2147483648	100000	40	6340190208
20cf4d3f-d425-4f01-8823-9b9e9dcfa72c	1005	C:/Program Files/OmniBack/server/db80/dcbf/dcbf4	214748364800	4	2147483648	100000	63	7780794368
  • The separation of “Program Files” and “Programdata” has been done in the past. If this is not the case and you plan to do the separation during the migration on the target hardware or platform, some additional steps are required. In this guide the separation was not done so far, so all steps are documented.
  • The used user names and passwords (Windows) to start the Data Protector services remain the same on the target hardware or platform. If you plan to change this and if you want to use new user and password, please refer to the article „Data Protector 8.0 Windows – change user account for DP services“ (link: https://www.data-protector.org/wordpress/2013/11/data-protector-8-0-windows-change-user-account-dp-services/).
  • The article does not apply for migrations from pre DP 8.XX versions.
  • On the source system the most current Data Protector version should be installed before the migration is done, as this will ease the migration procedure. This does apply for source systems only when the separation of “Program Files” and “Programdata” has already been done. The reason is a bug in the installation sequence when upgrading from DP 8.0X to 8.10. In this guide the same version of Data Protector is installed on the target and the upgrade will be done on the target system after the migration has been completed.
  • The name of the server and the IP addresses remain the same on the target cell server. If you are required to change that, additional steps might be necessary (export of clients beforehand, import of clients after migration, change cell server name, etc. – this will not be part of this guide).
  • The server name has to start with a leading letter and must not start with a leading number. The reason is a limitation in the included application server (JBOSS/Tomcat).
  • The internal database has not been restored in the past and there must not exist several DB80 directories (original and restored versions).
  • All names in this guide are examples.
  • After the migration, make sure you immediately start a backup for the server and the internal database. A restore of the IDB with data before the migration might not work, especially when the DCBF directories were changed during the migration. This will be possible only, when additional modification in the exported database is performed.
  • The guide does not apply to MoM environments.

Step by Step:

  • Make a backup of the IDB in case any error occurs. Using a Jukebox with some slots makes it easy to do a fast recovery. The media will be formatted and the IDB backup started. In this example the backup was done to e:\IDBBackup\01..05 and copied off the server after the IDB backup. In case you do not have sufficient local disk space, use a filelibrary configured to store the data on network share. The definition of the Jukebox and the used drive will be exported into text files. Definition Jukebox:
    E:\Program Files\OmniBack\bin>omnidownload -library IDBBackup
NAME "IDBBackup"
DESCRIPTION ""
HOST servername.domain
POLICY Jukebox
TYPE File
REPOSITORY
        "E:\IDBBackup\01"
        "E:\IDBBackup\02"
        "E:\IDBBackup\03"
        "E:\IDBBackup\04"
        "E:\IDBBackup\05"
MGMTCONSOLEURL ""

    Definition drive:

    E:\Program Files\OmniBack\bin>omnidownload -device IDBBackup_Drive01
NAME "IDBBackup_Drive01"
DESCRIPTION ""
HOST servername.domain
POLICY Jukebox
TYPE File
POOL "IDBBackup"
LIBRARY "IDBBackup"
CONCURRENCY 10
ENCRCAPABLE
BUFFERS 16
BLKSIZE 64
RESTOREDEVICEPOOL NO
COPYDEVICEPOOL NO

    In case of any errors you can use these text files to re-create the Jukebox (see help for command omniupload).

  • Stop the Data Protector scheduler using the command omnitrig -stop
  • Export the internal database with the command omnidbutil –writedb <pathname> (in this guide Z:\ExportIDB is used). On request copy the directories DCBF, MSG and META off the server (i.e. Z:\Migration).
  • Stop the Data Protector services using the command omnisv -stop
  • Copy all Data Protector directories off the server (i.e. to Z:\Migration). Skip the copy for the directories DCBF, MSG and META, as this has been done in previous step. Comment: when copying the directories, all symbolic links in IDB structure will be copied as whole, so please plan for sufficient disk space on target. Once you copied the directories all relevant Data Protector parts have been backed up, however, this does not apply for any additional services installed on the source system.
  • Install the new box or operating system. Use the same server name, IP addresses, domain, groups, users and passwords. In case of changes, you need to check the configuration files step by step (see below).
  • Install HP Data Protector 8.XX to “D:\Program Files\Omniback” and “D:\Programdata\Omniback” (example). Use the same version, the same patches or hotfixes as installed on source system.
  • Configure impersonation:
    C:\Users\administrator>omniinetpasswd -inst_srv_user domain\administrator
User 'domain\administrator' is configured to be used by Installation Server.
  • Stop the Data Protector services using the command omnisv -stop
  • On demand change the local security policy. This is required, if you plan to start the “Data Protector INET” service using a domain user, instead of the NT Authority\SYSTEM account. Add the user to the policies “Impersonate a client after authentication” and “Replace a process level token”.
  • Copy back the DCBF, MSG and META directories to the new path. In this guide the path D:\Programdata\OmniBack\server\db80 is used.
  • Change the DCBF directories in the file dpidb.dat (see export of IDB). This step is required only for target systems where the Data Protector directories are changed during the migration. Change of DCBF directories from:
    COPY dp_catalog_dcbf_directory (db_uuid, seq_id, dcbf_directory, maxsize, fill_sequence, min_freespace, max_dcbf_files, cur_num_of_files, cur_occupied_space) FROM stdin;
20cf4d3f-d425-4f01-8823-9b9e9dcfa72c	1002	E:/Program Files/OmniBack/server/db80/dcbf/dcbf1	214748364800	1	2147483648	100000	47	6341406720
20cf4d3f-d425-4f01-8823-9b9e9dcfa72c	1004	E:/Program Files/OmniBack/server/db80/dcbf/dcbf3	214748364800	3	2147483648	100000	44	6964137984
20cf4d3f-d425-4f01-8823-9b9e9dcfa72c	1001	E:/Program Files/OmniBack/server/db80/dcbf/dcbf0	214748364800	0	2147483648	100000	48	9023758336
20cf4d3f-d425-4f01-8823-9b9e9dcfa72c	1003	E:/Program Files/OmniBack/server/db80/dcbf/dcbf2	214748364800	2	2147483648	100000	39	6941310976
20cf4d3f-d425-4f01-8823-9b9e9dcfa72c	1005	E:/Program Files/OmniBack/server/db80/dcbf/dcbf4	214748364800	4	2147483648	100000	63	7780794368

    Change of DCBF directories to:

    COPY dp_catalog_dcbf_directory (db_uuid, seq_id, dcbf_directory, maxsize, fill_sequence, min_freespace, max_dcbf_files, cur_num_of_files, cur_occupied_space) FROM stdin;
20cf4d3f-d425-4f01-8823-9b9e9dcfa72c	1002	D:/ProgramData/OmniBack/server/db80/dcbf/dcbf1	214748364800	1	2147483648	100000	47	6341406720
20cf4d3f-d425-4f01-8823-9b9e9dcfa72c	1004	D:/ProgramData/OmniBack/server/db80/dcbf/dcbf3	214748364800	3	2147483648	100000	44	6964137984
20cf4d3f-d425-4f01-8823-9b9e9dcfa72c	1001	D:/ProgramData/OmniBack/server/db80/dcbf/dcbf0	214748364800	0	2147483648	100000	48	9023758336
20cf4d3f-d425-4f01-8823-9b9e9dcfa72c	1003	D:/ProgramData/OmniBack/server/db80/dcbf/dcbf2	214748364800	2	2147483648	100000	39	6941310976
20cf4d3f-d425-4f01-8823-9b9e9dcfa72c	1005	D:/ProgramData/OmniBack/server/db80/dcbf/dcbf4	214748364800	4	2147483648	100000	63	7780794368
  • Restore the during the migration copied files in D:\Programdata\OmniBack\log\server. The most important files are media.log, Ob2Event.* and the folder auditing.
  • Modify or restore the configuration files global (example: D:\ProgramData\Omniback\Config\Server\Options) and omnirc (example: D:\ProgramData\Omniback).
  • Modify or restore the files cell_info, lic.dat and installation_servers. Comment: After the upgrade to DP 8.1X all installed licenses become invalid and must be regenerated using the portal http://webware.hp.com. This requires a current support contract with HP.
  • Restore the directories: BarLists, Barschedules, consolidationlists, copylists, Datalists, dlgroups, dr, Integ, rptgroups, rptschedules, Schedules, users. For the files in folder users crosscheck the entries before overwriting.
  • Modify the files hpdp-idb-cp.cfg and idb.config in directory (example) D:\ProgramData\Omniback\Config\Server\idb. In both files modify the pathes.
    Example hpdp-idb-cp.cfg:

    [databases]
hpdpidb = host=127.0.0.1 port=7112 dbname=hpdpidb

[pgbouncer]
service_name=hpdp-idb-cp
listen_addr = *
listen_port = 7113
unix_socket_dir = /tmp
auth_type = md5
admin_users = hpdp
stats_users = hpdp
pool_mode = transaction
server_reset_query =
ignore_startup_parameters = application_name,extra_float_digits
server_check_query = select 1
server_check_delay = 10
max_client_conn = 1200
default_pool_size = 50
query_wait_timeout = 10
log_connections = 0
log_disconnections = 0
log_pooler_errors = 1
logfile = D:\ProgramData\OmniBack\log\hpdp-idb-cp.log
pidfile = D:\ProgramData\OmniBack\log\hpdp-idb-cp.pid
auth_file = D:\ProgramData\OmniBack\config\server\idb\ulist

    Example idb.config:

    PGHOSTNAME='localhost';
PGIDBNAME='hpdpidb';
PGUSER='hpdpidb_app';
PGPASSWORD='bWhhbnTvZ3Xxbao2Zw==';
PGSUPERUSER='hpdp';
PGSUPERPASSWORD='anpabZl3bnh6dGY3eQ==';
PGMAXOPEN_CONN_NUMBER='5';
PGDATA_PG='D:\ProgramData\OmniBack\server\db80\pg';
PGDATA_IDB='D:\ProgramData\OmniBack\server\db80\idb';
PGDATA_JCE='D:\ProgramData\OmniBack\server\db80\jce';
PGOSUSER='domain\administrator';
PGPORT='7112';
PGCPPORT='7113';
PGWALPATH='D:\ProgramData\OmniBack\server\db80\pg\pg_xlog_archive';
IDB_SEC_FLAG='11';
  • On demand and depending on configuration it might be necessary to restore (copy back) additional files and directories. For example, when Exchange is backed up in the Environment, the folder (example) D:\ProgramData\Omniback\server\db80\vssdb\metadata60 is required.
  • Start the Data Protector services using the command omnisv -start.
  • Stop the Data Protector scheduler using the command omnitrig -stop.
  • Import the IDB using the command omnidbutil -readdb <pathname> (in the guide Z:\ExportIDB is used).
  • Stop the Data Protector services using the command omnisv -stop.
  • Start the Data Protector services using the command omnisv -start.
  • Check the consistency of the internal database using the command omnidbcheck -extended. No errors must be reported.
  • For further checks start some backups and restore some data for backups done before the migration and after the migration.
  • After successful migration the used Jukebox (IDBBackup) and for the Jukebox used directories can be removed. All other directories used during the migration should be kept for at least 7 days.
  • On demand and if not already done on the source system, the upgrade to the most current Data Protector version can be started.

Workaround: VMware GRE – DP 7.03/8.1X – vCenter 5.1 U2

Posted on by 8 Comments

After updating to vCenter 5.1 update 2 and with Data Protector 7.03 or 8.1X a blank page might be displayed when using VMware GRE. The reason might be a change in Tomcat of vCenter. When looking into the log files (i.e. C:\ProgramData\VMware\VMware VirtualCenter\Logs\localhost.YYYY-MM-DD.log) the error VMwareGRE: runCGI (stderr):Can't open perl script "C:\Program": No such file or directory might be displayed. As a workaround the config file web.xml of the VMware GRE plugin can be modified. When using a standard installation the file is located in C:\Program Files\VMware\Infrastructure\tomcat\webapps\VMwareGRE\WEB-INF\. The workaround will change how the parameters are handled.

Change the block:

<init -param>
<param -name/>executable
<param -value/>”C:\Program Files\OmniBack\bin\perl.exe” -I “C:\Program Files\OmniBack\lib\perl”
</init>

Changed block:

<init -param>
<param -name/>executable
<param -value/>C:\Program Files\OmniBack\bin\perl.exe
</init>
<init -param>
<param -name/>executable-arg-1
<param -value/>-I
</init>
<init -param>
<param -name/>executable-arg-2
<param -value/>C:\Program Files\OmniBack\lib\perl
</init>

Updated support matrices for Data Protector 8.10, 8.00 and 7.03

Posted on by 2 Comments

Update: On 2014/03/25 the support matrices for Data Protector 8.10, 8.01 and 7.03 were updated; please review.

Update: On 2014/02/25 the support matrices for Data Protector 8.10, 8.00 and 7.03 were updated and includes fixes from the January matrices

The support matrices for Data Protector 8.10, 8.00 and 7.03 were updated and can be downloaded from here: http://support.openview.hp.com/selfsolve/manuals (Passport account required). The most important change is the official support for a DP 8.X Cell Server on Windows 2012 R2 (non-Cluster environments).