Any disaster recovery solution for uefi + secure boot?

Is anyone aware of a bare-metal disaster recovery backup solution that they have seen work with sles11 sp3 x86_64 with the following constraints:

  • UEFI
  • Secure boot enabled
  • backup solution - full + incremental (not an imaging product)
  • single server solution (not enterprise product)
  • backup/restore to/from external usb disk

I had success with Clonezilla creating/restoring images to/from usb3 external disk - but unfortunately, this is an imaging product and does not offer what customers expect from a backup solution.

I am testing on HP ML350 and DL380 G9 server hardware and have tested the following without success:
Farstone
Relax and Recover
Mondo Rescue
Active@ Disk Image
Acronis Trueimage Linux Server 11.5
Yosemite Server Backup
Storix
Microlite Backupedge
HP Data Protector
SEP - enterprise product not really suitable for single server - didn’t test
Bacula
Paragon

Hi
What filesystem(s) are you running on the systems?

Have you looked at autoyast to create an image?

How are you receiving the updates, just downloading, via SMT, SUSE Manager?

What medium are you using to backup to, and how much data?

I use a script to backup the UEFI data;

#!/bin/bash

# This program is free software; you can redistribute it and/or modify it
# under the terms of the GNU Lesser General Public License as published by
# the Free Software Foundation; either version 2.1 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful, but
# WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
# Lesser General Public License for more details.
#
# Copyright (C) 2013-2015 Malcolm Lewis <malcolmlewis@opensuse.org>

EFI_SGDISK_BIN=`which sgdisk`
EFI_BOOT_MGR_BIN=`which efibootmgr`

if (( $# != 3 )); then
	echo "Usage: $0 <EFI_DEVICE> <EFI_PARTITION> <BACKUP_LOCATION>" >&2
	exit 1
fi

EFI_DEVICE="$1"
EFI_PARTITION="$2"
BACKUP_LOCATION="$3"
BACKUP_DATE=`date +\\%m%d%y%H%M%S`

sgdisk -b /$BACKUP_LOCATION/`hostname`_$EFI_DEVICE\\_backup_$BACKUP_DATE.gpt /dev/$EFI_DEVICE
mount -t vfat /dev/$EFI_DEVICE$EFI_PARTITION  /mnt
tar -C /mnt -cvzf /$BACKUP_LOCATION/`hostname`_esp_$BACKUP_DATE.tgz EFI
umount /mnt
$EFI_BOOT_MGR_BIN -v > /$BACKUP_LOCATION/`hostname`_efibootmgr_$BACKUP_DATE.txt

tar cjvf /$BACKUP_LOCATION/`hostname`_backup_$BACKUP_DATE.tar.bz2 \\
	/$BACKUP_LOCATION/`hostname`_$EFI_DEVICE\\_backup_$BACKUP_DATE.gpt \\
	/$BACKUP_LOCATION/`hostname`_esp_$BACKUP_DATE.tgz \\
	/$BACKUP_LOCATION/`hostname`_efibootmgr_$BACKUP_DATE.txt

rm /$BACKUP_LOCATION/`hostname`_$EFI_DEVICE\\_backup_$BACKUP_DATE.gpt \\
	/$BACKUP_LOCATION/`hostname`_esp_$BACKUP_DATE.tgz \\
	/$BACKUP_LOCATION/`hostname`_efibootmgr_$BACKUP_DATE.txt

Thanks for the reply. Filesystems are:
vfat /boot/efi
reiserfs /
xfs /u remaining space

We use autoyast to build the servers (3 different autoyast files differing only by number of logical drives, 1,2, or 3). The autoyast files are included on the build DVD.
For 3 logical drives, and new xfs partition is created.

Oce in Germany manages our patch delivery - we install from DVD

Our preferred backup medium is 2TB and 3TB external USB drives (for SLES10 and SLES11 we have been using Lacie)
For SLES 10 and SLES11 we used Microlite Backupedge to create bootable usb backups containing all data on the Lacie drive. Disaster recovery was as simple as booting from the Lacie drive and selecting restore all.

At this point I am still looking for a commercially available product to include in all servers we sell for delivery to our customers. We have been able to offer this type of solution with SCO Openserver (tape backup), Caldera Openlinux (tape backup), SLES9 (dvd and tape backup), SLES10 (dvd, tape, and usb backup), and SLES11 (dvd and usb backup) since 1997.

Perhaps with uefi and secure boot there won’t be a commercial solution for SLES11 SP3 and uefi? In the future, when we move to SLES12 perhaps I will have more success?

Hi
So it would seem it’s only the UEFI part that is an issue? A simple dd to an equivalent partition on a USB device set to gpt would enable booting.

With a restore to new hardware and secure boot enabled, the keys will need to be re-enrolled into the system BIOS, this AFAIK can be scripted with MokManager, plus if keys are updated (with a kernel upgrade) it may require user interaction to accept/enter root password etc…

SLE 12 uses btrfs by default, plus with snapper enabled (important to configure for your needs) you do have access to bootable snapshots.

Hi,
'So it would seem it’s only the UEFI part that is an issue? A simple dd to an equivalent partition on a USB device set to gpt would enable booting."

a dd of /dev/sda1 to a vfat partition on the usb disk (/dev/sdb1) set to gpt won’t boot on my HP ML350 G9. It just returns to the one-time boot menu.

Hi
Is the partition type set to EF00?


Cheers Malcolm °¿° LFCS, SUSE Knowledge Partner (Linux Counter #276890)
SUSE Linux Enterprise Desktop 12 GNOME 3.10.1 Kernel 3.12.38-44-default
If you find this post helpful and are logged into the web interface,
please show your appreciation and click on the star below… Thanks!

Hi
If that doesn’t work (I just tried here on SLED 12, HP ProBook), then
mount the device and copy;

mount /dev/sdNx /mnt
cp -ar /boot/efi/* /mnt/

The script I provided backups the gpt data etc, but have a look at the
man page for gdisk (which I guess your using?) for other options.


Cheers Malcolm °¿° LFCS, SUSE Knowledge Partner (Linux Counter #276890)
SUSE Linux Enterprise Desktop 12 GNOME 3.10.1 Kernel 3.12.38-44-default
If you find this post helpful and are logged into the web interface,
please show your appreciation and click on the star below… Thanks!

Partition type is complicated. Yast2 partitioner will not let me select EF00. Best it can do is FAT. My booted hard disk has a 400MB /boot/efi partition that is type 0x103 EFI Boot as displayed by yast2 partitioner. Command line parted doesn’t seem to let me choose a type. fdisk lets me choose, among others, EF (Linux native) or EE - but when I chose ee, yast2 partitioner cannot see the partition to show me the type. And…I can’t mount it to copy anything to it. Seems like what I need is a tool to create an EFI Boot partition on the usb drive - perhaps some kind of uefi application?

Hi
Grab an openSUSE 13.1 liveCD which will have gdisk which is needed to
create the right partition type and a gpt device.


Cheers Malcolm °¿° LFCS, SUSE Knowledge Partner (Linux Counter #276890)
SUSE Linux Enterprise Desktop 12 GNOME 3.10.1 Kernel 3.12.38-44-default
If you find this post helpful and are logged into the web interface,
please show your appreciation and click on the star below… Thanks!

I found a way to get the yast2 partitioner to show EFI Boot for my usb disk partition - created 500MB FAT partition, then used cmd line parted to ‘toggle 1 boot’. Copied /boot/efi/ to this partition and tried to boot from it. But the server won’t boot from it. However, from the One-Time Boot Menu, I can select Run UEFI Application, then select this USB disk, then from File Explorer choose efi, then SuSE, then shim.efi and up comes the grub2 boot loader and if I select Linux it will boot from the hard drive. I don’t think any of this helps though.

Hi
So you can’t use your original method to recover the other partitions, then rebuild the UEFI partition (It’s recommended to be 100-300MB as well) or boot from the USB device and then rebuild the efi part?

So just to clarify, your preferred method for / and /u is ok, as in this can carry on. It’s just the efi backup/restore that is of concern?

I don’t understand how to build the efi part. With legacy bios it was grub, root(hdx,x), setup(hdx), and the mbr was installed and the boot loader would start. I have this /boot/efi partition on my hard disk but that isn’t a bootloader as it isn’t signed and the hardware won’t boot from it (I know this because when I copy it to my usb disk it is not recognized by the hardware and there is no attempt to boot from it). So if I was booted from a Live CD I wouldn’t know how to make the hard disk bootable - making a /boot/efi partition and copying files to it is not enough.

Hi
To enroll the keys you need to boot the MokManager efi (so the keys
need to be backed up) file then use the shim. Else the system will boot
in non-secure mode until secure boot is configured.

So if your backup/restore processes covers /boot/efi, / and /u,
recreates the gpt disk (see my script and using sgdisk restore option)
restores the data and partition types, all should be good to boot from
using the F9 key and selecting.

If you look at the script, there is the efibootmgr output, this is
needed to re-create the nvram efi boot entry via the efibootngr
command, which could be run as a post restore once only command.

Note, with any kernel update in secure boot, physical interaction is
needed to enroll keys with root password, just something to be aware
of…


Cheers Malcolm °¿° LFCS, SUSE Knowledge Partner (Linux Counter #276890)
SUSE Linux Enterprise Desktop 12 GNOME 3.10.1 Kernel 3.12.39-47-default
If you find this post helpful and are logged into the web interface,
please show your appreciation and click on the star below… Thanks!

Booting the “rescue mode” from the SUSE Linux Enterprise DVD product media and use xfsdump/xfsrestore.

XFS is the better solution for “/”. See Release Notes SLES 12 chapter 3.4 and 3.2.2:
https://www.novell.com/linux/releasenotes/x86_64/SUSE-SLES/12/

Thank you for the ideas, but really looking for a commercial solution for SLES11 sp3 and sp4 (uefi + secure boot) that we can offer to our customers so they can manage disaster recovery. Since our software requires SLES11 sp3 at this time, that is the platform I am looking for. In testing I am finding that all the vendors are having trouble with shim.efi and/or elilo.efi.