The need, to backup the data from your servers to media and thus to maintain data security does not stop when backed up to tape, there is often a request to protect the tapes from further appending or overwriting or to move the media off the tape library and to store it on a safe place (safe, backup computer center, …). Each Data Protector admin knows this specific situation, finding out which media was used in the last full backup for vaulting. The experienced admin will configure Data Protector to send a report or use many different pools to keep the daily backup, weekly backup and monthly backup apart. Problem: The media sets often should then be moved to a own vaulting pool before they are taken off the library.
Solution: Just for this problem I wrote a small Perl script. As always, I write my scripts in Perl, as this language is very flexible for such simple tasks. The script does not need a full Perl installation, the stripped down version, which is included within the Data Protector installation on the Cell Server is sufficient. Functional way, in short: It searches for sessions in a specific period, analyzes the media and then the media will be moved to a vaulting pool.
How it works: The script searches for sessions of the time frames specified in the script. For the sessions the used media are detected. The list of media is now cleared of any existing duplicates (one media may have to be shared by multiple backup jobs) and for each media the pool is detected. If the the pool where the media belongs to should be excluded (configured inside the script), then the respective media is skipped, otherwise, the media is moved into the configured destination pool. If you need a bit more complex version it is possible more than one copy of the script with different values for the variables set and run. This also applies to different types of media (LTO and what else is there on the market…). At the end of the script an email with the information which tapes were moved to the destination pool is sent. Who does not like this feature can comment out the call to the binary… If you like it you need to get the tool “Blat” and to configure it – Link: http://www.blat.net. For simplicity, I have everything (the script, the batch file to call the script and the Blat binaries) in the bin directory of Data Protector, but even that can be changed. It could also have an extension where the media are moved into the mail slots (5 more lines of code). This of course makes sense only if the library has configured mail slots and the number of media to be moved offsite is less than or equal to the amount of mail slots in place.
Use case: You have configured the pools FilesystemWeekend, FilesystemDaily (for the file backup), DatabasesWeekend, DatabasesDaily (for database backup), DPDatabase, Vaulting und FreePool. On each monday morning the media which were used from friday evening until monday mornning must be moved into pool “Vaulting”. The daily pools and the DPDatabase pool (The backup of the internal database I always do to a separate pool!) should not be included for vaulting in this case. The required settings for the script can be find below.
Settíngs: What do you need to change for our use case?? Please note the with a backslash quoted @ for the email addresses and that yo have configure “blat” beforehand (only 1 command to execute).
$exclude_pools="FilesystemDaily,DatabasesDaily,DPDatabase,Vaulting";
$vault_pool="Vaulting";
$timeframe_start="60";
$timeframe_duration="60";
$mailrecipient="servicedesk\@data-protector.org";
$mailserver="127.0.0.1";
$mailsender="dataprotector\@data-protector.org";
$mailsubject="Media vaulting";
[wpdm_file id=4]
UPDATE Download: There was a minor bug when counting the media – new download below.
[wpdm_file id=5]
Use it for…: The script is best suited for small to mid-size companies. For larger environments I recommend to evaluate HP Media Operations. As an alternative I sell a script for large environments, it can be enhanced on your needs and on your request. To get more information and a offer, please send a mail with the subject “Off-Site Tape Vaulting” to daniel-braun@data-protector.org
Donate: If you like this article and if the script helps you in your daily Data Protector business, I would be glad to get a donation from you (5 € are sufficient) via PayPal. The link to donate you’ll find in the upper right corner of this site…
Reference: If you are behind a proxy and unable to download the file… below you will find the complete script. If you like it smaller, please feel free to modify, the script does not need to have the “prints” and can be cutted to the half size…
#!perl -I../lib/perl -X -CA
### needed modules
use Omniback;
# Get DP folders
my $OMNIHOME = Omniback::CmnQuotePath($Omniback::PAN{'BASE'});
my $OMNIHOME_STR = $OMNIHOME;
$OMNIHOME_STR =~ s/\"//g;
my $OMNIBIN = "\"${OMNIHOME_STR}/bin\"";
# define the exclude pools, pool for vaulting, ...
$exclude_pools="Pool1,Pool2,PoolX";
# the pool mus exist in DP
$vault_pool="Vaulting";
# the time frame to retriev the session information
# assuming the script should run monday 8am, the timeframe of
# 60 month covers backups starting friday evening
$timeframe_start="60";
$timeframe_duration="60";
# the stuff needed to send a mail at the end of the script
# you need to download blat (mail program) search for blat at google
$mailrecipient="firstname.lastname\@domain.com"; # please remind the quoted (at)
$mailserver="192.168.2.10"; # the ip address of the smtp server
$mailsender="dataprotector\@domain.com"; # please remind the quoted (at)
$mailsubject="Media vaulting"; # Subject for the mail
print "Generating Report\n";
print "- skipping pools: $exclude_pools\n";
print "- target pool is: $vault_pool\n";
print "- start time: $timeframe_start\n";
print "- duration: $timeframe_duration\n";
print "\n\nNote: Ignore messages regarding \"No medium found\"\n\n";
@exec=`${OMNIBIN}/omnirpt -report list_sessions -timeframe $timeframe_start $timeframe_duration -tab`;
foreach (@exec)
{
chomp $_;
next if /^#/;
(@sessionid)=split(/\t/,$_,23);
push(@sessions,$sessionid[22]);
}
foreach (@sessions)
{
@exec=`${OMNIBIN}/omnidb.exe -session \"$_\" -media`;
$media = $exec[2];
chomp $media;
($media_short,$dummy)=split(/ /,$media,2);
if ($media_short ne "")
{
push(@media,$media_short);
}
}
@newmedia=&del_double(@media);
print "Summary\n";
@movedmedia=();
foreach (sort(@newmedia))
{
($media1,$dummy)=split(/ /,$_,2);
$media1 =~ s/\[//ig;
$media1 =~ s/\]//ig;
@exec=`${OMNIBIN}/omnimm -media_info \"$media1\" -detail`;
$pool = $exec[2];
chomp $pool;
chop $pool;
$pool =~ s/Pool name : //ig;
if ($exclude_pools =~ /$pool/)
{
print "skipping media: $media1 \n";
}
else
{
print "moving media: $media1 \n";
@exec=`${OMNIBIN}/omnimm -move_medium \"$media1\" $vault_pool`;
push(@moved_media,$media1);
}
}
$anzahl=@moved_media;
if ($anzahl gt 0)
{
$message="";
$message="Vaulting media:\n============================\n\n";
$message=$message."Following media were moved to pool: $vault_pool for vaulting.\n";
$message=$message."Media:\n";
foreach (@moved_media)
{
$message=$message.$_."\n";
}
$message=$message."\n\nMail generated by robot\n\n";
`${OMNIBIN}/blat -to $mailrecipient -f $mailsender -subject \"$mailsubject\" -body \"$message\"`;
}
exit;
sub del_double{
my %all;
grep {$all{$_}=0} @_;
return (keys %all);
}
Hallo,
genau nach so einem Script habe ich schon lange gesucht.
Vielen Dank!
Das einzige was uns noch fehlen würde wäre die Möglichkeit auch gleich die Protection der Bänder (inkl. Cataloge Protection) zu erhöhen um die Datenbank bei einem Restore nicht neu einlesen zu müssen.
Grüße
RNI
Hallo,
the Catalog and Data Protection is not set on tape / media, it is set on session. So you would need to enhance the script to prolonge the protection for the wanted session using the command
omnidb -session < SessionID > -change_protection < Protection > -change_catprotection < Protection >.Best regards
Daniel