Veeam Backup & Replication is a great tool for backing up your virtual environment. U can use it to backup certain virtual machines or your whole environment by simply creating backup jobs in which u add virtual machines. Veeam has the option which sends an e-mail if a backup was successful or failed.

Sadly there is one thing I couldn’t find and it’s an option to check which virtual machines are being backed up and which are not in a virtual environment.

After some thinking I decided to combine Veeam and PowerCLI as a solution to my problem.

When a job is created there is an advanced option to add a note to a virtual machine which can be used for scripts.

When creating a job click “Advanced” when the following screen appears.

Once the advanced settings open, go to the notifications tab and select “Set successful backup details to this VM attribute”. Change the value from “Notes” to “Backup”.

Every time the job finishes successful Veeam will add an annotation called “Backup” to the virtual machine as seen on the picture below.

Below is a script which will collect all virtual machines in the defined vCenter and run a check on the backup annotation. This will perform a check if the backup was successful in the past 2 days. Finally it will send an overview in plain text via mail.

Please change the required settings on top of the script!

# Set the vCenter Server
$vcenter = "localhost"
# Set the SMTP Server address
$SMTPSRV = "<SMTP SERVER>"
# Set the Email address to receive from
$EmailFrom = "veeam@foonet.be"
# Set the Email address to send the email to
$EmailTo = "<RECEIVER>"
# Set the Email subject
$EmailSubject = "Veeam Backup Information"

# SMTPmail function
function Send-SMTPmail($to, $from, $subject, $smtpserver, $body) {
	$mailer = new-object Net.Mail.SMTPclient($smtpserver)
	$msg = new-object Net.Mail.MailMessage($from,$to,$subject,$body)
	$msg.IsBodyHTML = $false
	$mailer.send($msg)
}

# Get Backedup function
function Get-Backedup ($vm) {
	$val = $vm.CustomFields |where {$_.key -eq "Backup" } | select -Property Value
	$today = Get-Date -Format "d/MM/yyyy"
	$yesterday = get-date -format "d/MM/yyyy" -displayhint date ((get-date).adddays(-1))
	$backupDate = $val.Value
	if ( $backupDate -like "*$today*" -or $backupDate -like "*$yesterday*"  ) {
		$res = 1
	} else {
		$res = 0
	} 
	return ( $res )
}

Connect-VIServer $vcenter | Out-Null

$vms = Get-VM

$resultok=@()
$resultnotok=@()

foreach ($vm in $vms) {
	if (Get-Backedup($vm)) {
		$resultok += "$vm `r`n"
	} else {
		$resultnotok += "$vm `r`n"
	}
}

$body = "==================================================`r`n"
$body += "Backup report $vcenter:`r`n"
$body += "==================================================`r`n`r`n"
$body += "The following VMs have NOT been backed up:`r`n"
$body += "---------------------------------------------------------------`r`n"
$body += $resultnotok | Sort-Object
$body += "`r`n`r`nThe following VMs have been backed up:`r`n"
$body += "--------------------------------------------------------`r`n"
$body += $resultok | Sort-Object

send-SMTPmail -to $EmailTo -from $EmailFrom -subject $EmailSubject -smtpserver $SMTPSRV -body $body

That’s it. Feel free to improve the script or give hints and tips!

This is a very useful command if you need to find all symlinks in a certain directory:

ls -lR | grep "^l\|:$"

I’m a big fan of syslog-ng. All my devices which allow the usage of syslog send information to my syslog-ng server.

The server configuration below will send all information to specific day dirs for each device.

options { 
        use_fqdn(yes); 
        keep_hostname(yes); 
        use_dns(yes); 
        long_hostnames(off); 
        sync(0); 
        log_fifo_size(300); 
        create_dirs (yes);
        perm (0640);
        dir_perm (0750);
};

# Configure on which IP and port(s) we will listen
source s_net { 
        tcp(ip("<SERVERIP>") port(51400)); 
        udp(ip("<SERVERIP>") port(514));
        tcp(max-connections(500));
};

# This is the default behavior of sysklogd package
# Logs come from the unix stream.
source src { 
        unix-stream("/dev/log"); 
        internal(); 
};
 
# Log destinations.
 
# First some standard logfile
destination lpr { 
        file("/var/log/lpr.log");
};

destination mail { 
        file("/var/log/mail.log"); 
};
 
# Some `catch-all' logfiles.
destination messages { 
        file("/var/log/messages"); 
};
 
# Filter options. With this rules, we can set which message go where. 
filter f_lpr { 
        facility(lpr); 
};

filter f_mail {
        facility(mail); 
};

filter f_messages { 
        level(info..emerg) and not facility(mail,lpr); 
};
 
# The logging part
log { 
        source(src); 
        filter(f_lpr); 
        destination(lpr); 
};

log { 
        source(src); 
        filter(f_mail); 
        destination(mail); 
};

log { 
        source(src); 
        filter(f_messages); 
        destination(messages); 
};

# Automatic host sorting
destination hosts { 
        file("/var/log/syslog-ng/$HOST/$YEAR.$MONTH.$DAY/$FACILITY" owner(root) group(root) perm(0600) dir_perm(0700) create_dirs(yes)); 
};

# All information send to the server
log { 
        source(s_net); 
        destination(hosts); 
};

The client configuration will send all information to the syslog-ng server but we will also keep the messages and authentication log stored on the local machine.

options { 
        use_fqdn(no); 
        keep_hostname(yes); 
        use_dns(no); 
        long_hostnames(off); 
        sync(0); 
        log_fifo_size(300); 
};

# Logs come from the unix stream.
source src { 
        unix-stream("/dev/log"); 
        internal(); 
        file("/proc/kmsg" log_prefix("kernel: "));
};
 
# Some `catch-all' logfiles.
destination messages { 
        file("/var/log/messages"); 
};

destination authlog { 
        file("/var/log/auth.log"); 
};
 
# Where to send the information
destination logserver {
        tcp("<SERVERIP>" port(514));
};

# Filters
filter f_authpriv { 
        facility(auth, authpriv); 
};

# auth.log file
log { 
        source(src); 
        filter(f_authpriv); 
        destination(authlog); 
};

# Send everything to syslog-ng server
log { 
        source(src); 
        destination(logserver); 
};

After a few months (maybe weeks – depending on the number of devices u have) this will take up quite some space and therefor the following script can be useful. It will create a tar.gz on everything older than 1 week and remove the directory.

#!/bin/bash
for i in `ls /var/log/syslog-ng`; do
        cd /var/log/syslog-ng/$i;
        for j in `ls | egrep -v ".tar.gz"`; do
                dir01=`find . -name $j -mtime +7`
                if [ ! $dir01 = "" ]
                then
                        echo "$j is more than a week old - creating archive."
                        tar cvfz /var/log/syslog-ng/$i/$j.tar.gz /var/log/syslog-ng/$i/$j
                        rm -rf /var/log/syslog-ng/$i/$j
                else
                        echo "$j is not more than a week old - skipping."
                fi
        done;
done;

Just add this to a crontab every night and you’re good to go 🙂 !

Good luck and enjoy!

find / -type d -perm 0777 -exec ls -ld {} \;

Be gone evil directories and files! Be gone! 🙂

VMware vSphere is a very nice piece of virtualization software.

Sadly there are a few things that can go wrong and I had the “luck” to hit one of those. Waking up on a morning to notice that 4 VM’s are offline because of the ESXi host going down (hardware failure) isn’t something you want to wake up with. The worst news was that it was a new host which hadn’t been added to a cluster yet so no HA :-(.Thank god I run the machines from a storage!

After adding the storage to another ESXi host I was looking for a way to boot the machines from the new ESXi host. The problem was that vCenter saw the hosts as disconnected and VMotion wasn’t possible. Joy?

Thank god (again?) for SSH access on the machine. So I logged into the machine and managed to boot the machines on the new host by using the following command:

vim-cmd -H hostname_of_new_hypervisor /vmfs/volumes/datastorename/vm-dir/vm.vmx start

Weird thing about this: vCenter (which wasn’t able to do anything with the machines) did see the hosts being moved and changed them to the new ESXi host.