Today I came across a problem regarding dom0 partitioning and domUs.
We run a SLES11 based XEN Server on which we host 3 domUs. Today one of
my colleagues tried to add a new domU for some testing and nearly
deleted our productive Web Server in the process.
This happened because the the Partition of said Webserver dom0 is
marked as “free” if you look in Yast/Partition Manager under “unused
devices”. Normally this isn’t much of a problem, as long as you keep a
documentation somewhere about the Server’s configuration, keeping it up
to date, everybody else knows about that and reads it before trying to
modify something. However, it’s quite messy, and it gets worse with
every new domU you add.
This all would be less of a problem if there would be a SAN or maybe
I-SCSI behind the data store. Unfortunately all I have is a neat smart
array controller and two arrays with each 500GB in size.
Anyway, I am looking for a more bullet proof approach on that.
Something that shows up in Yast’s partition manager saying clearly to
keep your hands off. My first Idea was “Hey, great! LVM…”. But then
the Kernel isn’t able to boot from LVM boot partitions.
Next Idea was to mix a file device for my boot partition and using a
prepared LVM for the rest of the needed partitions. The best approach,
so far, but still quite ugly and
Does somebody have a decent solution that keeps the domU’s
disks/Partitions a little more “encapsulated” and manageable regarding
the effort to move them to an new Server, some day?
… My first Idea was “Hey, great! LVM…”. But then the Kernel isn’t
able to boot from LVM boot partitions. …[/color]
Hi Tom,
That first idea of yours… I echo it. I don’t understand the last
bit about no being able to boot?
As a general rule, if you want to run domU’s from LVM volumes, that’s
fine - but do make sure the dom0 is running on a traditional, or at
least non LVM filesystem. Otherwise you might have troubles.
We setup our Xen hosts with /boot swap and / on the first disk… and
maybe add an extra mount on that disk to put templates and ISO’s.
Then on shared storage or a second, third disk, etc have LVM management
run to assign volumes to domU as phy disk devices.
I’m a bit pressed in time (need to get to an appointment), but that’s
the rough sketch.
So far we only have the domU system OS disk on LVM and have domU data
disks hooked up as direct phy raw device on a iSCSI or fiber connected
SAN.
Cheers,
Willem
–
Novell Knowledge Partner (voluntary sysop)
It ain’t anything like Harry Potter… but you gotta love the magic IT
can bring to this world
Today I came across a problem regarding dom0 partitioning and domUs.
We run a SLES11 based XEN Server on which we host 3 domUs. Today one of
my colleagues tried to add a new domU for some testing and nearly
deleted our productive Web Server in the process.
This happened because the the Partition of said Webserver dom0 is
marked as “free” if you look in Yast/Partition Manager under “unused
devices”. Normally this isn’t much of a problem, as long as you keep a
documentation somewhere about the Server’s configuration, keeping it up
to date, everybody else knows about that and reads it before trying to
modify something. However, it’s quite messy, and it gets worse with
every new domU you add.
This all would be less of a problem if there would be a SAN or maybe
I-SCSI behind the data store. Unfortunately all I have is a neat smart
array controller and two arrays with each 500GB in size.
Anyway, I am looking for a more bullet proof approach on that.
Something that shows up in Yast’s partition manager saying clearly to
keep your hands off. My first Idea was “Hey, great! LVM…”. But then
the Kernel isn’t able to boot from LVM boot partitions.
Next Idea was to mix a file device for my boot partition and using a
prepared LVM for the rest of the needed partitions. The best approach,
so far, but still quite ugly and
Does somebody have a decent solution that keeps the domU’s
disks/Partitions a little more “encapsulated” and manageable regarding
the effort to move them to an new Server, some day?
Thanks in advance,
Tom.[/color]
Hi Tom,
I agree with Willem.
My servers only use local storage. Typically, I have a pair of 500GB
drives in a RAID 1 configuration. Some are hardware RAID, others use
software RAID. I create a /boot and swap partition for my Dom0. The rest
of the device is managed by LVM.
On my Dom0 I use a Logical Volume for / and another to hold my iso’s.
LV’s are created for my DomU’s as needed and left -unmounted- and
-unformatted-. When I need to, I add another pair of drives and use LVM
to manage them.
Before I started using LVM I would use an extended partition for
everything. Free space in the extended partition was -really- free
space. If a logical partition existed, it was used for -something-. One
of the nice things about LVM is that you can give your LV’s meaningful
names. As always, proper documentation can be a life saver!
–
Kevin Boyle
If you find this post helpful, please click on the star below!
\/
Hi Tom,
That first idea of yours… I echo it. I don’t understand the last
bit about no being able to boot?
[/color]
Since neither grub nor the kernel know how to access lvm, we’re not
able to put the boot partition on a logical volume. If that’d be
possible, my setup would look like this:
On dom0 I would create a volume group and call it SERVER1 with all the
needed logical volumes, like boot, root, srv, swap…
Since all the LVM information will be recognized by my domU (SERVER1),
this would be a nice, clean setup. No assumed free partitions would be
visible in Partition Manager.
My second Idea was to combine physical and file based disks in the
domU’s setup, where I would use the file-based disk for /boot and the
above setup for the rest.
It’s still not cleanly encapsulated, but at least it’s a more clean
approach and a little more coherent to the peoble who are new to this
stuff.
My servers only use local storage. Typically, I have a pair of 500GB
drives in a RAID 1 configuration. Some are hardware RAID, others use
software RAID. I create a /boot and swap partition for my Dom0. The rest
of the device is managed by LVM.
On my Dom0 I use a Logical Volume for / and another to hold my iso’s.
LV’s are created for my DomU’s as needed and left -unmounted- and
-unformatted-. When I need to, I add another pair of drives and use LVM
to manage them.
[/color]
By ISOs do you mean Setup CDs/DVDs or file-based disks for your boot
partitions? The latter would cover my second idea.
KBOYLE;2137386 Wrote:[color=blue]
Before I started using LVM I would use an extended partition for
everything. Free space in the extended partition was -really- free
space. If a logical partition existed, it was used for -something-. One
of the nice things about LVM is that you can give your LV’s meaningful
names. As always, proper documentation can be a life saver!
[/color]
That was my first setup, too. But I spent way too much time to describe
everything to new colleagues.
On dom0 I would create a volume group and call it SERVER1 with all the
needed logical volumes, like boot, root, srv, swap…
Since all the LVM information will be recognized by my domU (SERVER1),
this would be a nice, clean setup. No assumed free partitions would be
visible in Partition Manager.[/color]
And that’s where the general SLES XEN rule would abide against all in
one LVM setup.
If it’s one logical disk on the Xen host (and you can not split the
storage into multiple logical disks without loosing on effective storage
and layout), I’d propose to do something like:
part1 - 30 GB - ext3 - / (root and boot in one)
part2 - 3 Gb - swap
part3 - 80 GB - ext3 - /xentmpl (install ISO’s VM’s, template files
VM’s etc)
part 4 - rest of space - LVM type partition
VG-XEN
domU1-disk0
domU1-disk1
domU2-disk0
ect…
This would be if only having one logical disk to ‘play’ with on the Xen
host.
The only way to create a new disk would be to place it in the LVM VG as
there is no other free space on the partition. You could maybe also
instruct those creating new VM’s to use lvcreate commands… which would
then skip having to go into the disk partitioner.
As I’m not sure if you see it at an issue or not… having a domU use a
LVM volume as system disk is no issue. The domU will just see the handed
LVM volume as a normal disk and place it’s /boot on it without
complaining.
Cheers,
Willem
–
Novell Knowledge Partner (voluntary sysop)
It ain’t anything like Harry Potter… but you gotta love the magic IT
can bring to this world
By ISOs do you mean Setup CDs/DVDs or file-based disks for your boot
partitions? The latter would cover my second idea.Hi Tom,[/color]
Sorry for not getting back to you sooner. When I originally read your
post, I missed your question.
ISO is the standard format for creating CD’s and DVD’s. An ISO image is
a single file copy of the CD/DVD which, when appropriately mounted, can
be read as if it were physical media inserted in a CD/DVD drive.
On my Dom0, I keep ISO images of the media I use to create my DomU’s.
When I create a new DomU, I specify a CD/DVD storage device and point it
to the appropriate ISO image e.g. SLES-10-SP4-DVD-i386-GM-DVD1.iso. I
specify a second storage device to be used for the installation and
point that to one of my unmounted and unformatted logical volumes which
will usually be seen by the DomU as “sda”.
Does that help?
–
Kevin Boyle
If you find this post helpful, please click on the star below!