Getting Gentoo to work on a Asus P5Q Pro mobo with fake RAID

Wim Cossement wrote:
> J.O. Aho wrote:
>> Wim Cossement wrote:

>>> I built in everything I could, including ramdisk/initrd stuff (but I
>>> don't need to use that I guess or not?),
>>
>> Only if you built what you need to do the inital kernel boot as
>> modules, otherwise no.
>
> Modules? So built in stuff is not okay then?

initrd is only needed if you build things as modules, say you would make your
SATA and ext2 as modules, then you need to make an initrd image, which you
need to load during boot up.

>>> device
>>> mapper and RAID support, some SATA drivers but I can't seem to get my
>>> root file system mounted.
>>>
>>> Configuring GRUB was a bit akward but it seems to work.
>>>
>>> This is what I did:
>>>
>>> grub --device-map=/dev/null
>>
>> This isn't so good, as you can't load it from /dev/null, store it in
>> the same directory as everything else regarding grub.
>
> I thought this was needed so Grub is not confused when you use a funky
> setup.
> You start without any configuration and then specify your 1st device,
> hd0 by hand as I did.
> What do I need to do then? Please elaborate...

You need to specify that the device.map file will be located on disk, not
/dev/null, as grub will try to read from it.




--

//Aho

On Monday 27 July 2009 06:39, someone identifying as *J.O. Aho* wrote
in /alt.os.linux.gentoo:/

> Wim Cossement wrote:
>> J.O. Aho wrote:
>>> Wim Cossement wrote:
>
>>>> I built in everything I could, including ramdisk/initrd stuff (but
>>>> I don't need to use that I guess or not?),
>>>
>>> Only if you built what you need to do the inital kernel boot as
>>> modules, otherwise no.
>>
>> Modules? So built in stuff is not okay then?
>
> initrd is only needed if you build things as modules, say you would
> make your SATA and ext2 as modules, then you need to make an initrd
> image, which you need to load during boot up.

There is another scenario which requires an /initrd,/ and this is
applicable to Wim's situation, but he did something wrong.

See, another reason why you would need to have an /initrd/ is if you
have your root filesystem sitting on a software RAID, and the PATA or
SATA RAID functionality built into consumergrade motherboards is only a
hardware-assisted software RAID solution, not a true hardware RAID
solution. (I am elaborating on this for Wim's benefit, because I
presume that you already know this. )

There are basically three kinds of RAID...:

(1) Hardware RAID is when the adapter itself has a dedicated processor
and memory, and all the operating system needs to know about it is
what driver to use for the device. All RAID functionality is
handled by the adapter itself, and to the kernel, all disks in the
RAID array - regardless of whether it is a RAID 0, 1, 5, 6 or JBOD
- are seen as being a single physical disk.

(2) Software RAID is when a set of independent disks connected to one
or multiple disk controllers - which can be of mixed type - are
combined into a RAID array by the operating system itself. In
this scenario, the operating system does of course know that all
disks are separate devices, but (in casu) Linux /dmraid/ sets them
up to be used as a RAID array, which is then presented to userland
as a single physical disk. The RAID functionalities - i.e.
striping, mirroring, parity calculation, error handling et al - is
all taken care of inside the kernel itself.

(3) Hardware-assisted software RAID is like software RAID, but with
just enough hardware assistence present to allow for hardware error
checking and signaling. The kernel does all the RAID stuff, but the
hardware assists.

The latter is what Wim has in his machine and which he intends to use,
but as it is in essence still a software RAID, there are limitations as
to what you can do. For starters, Wim wants a RAID 0 set-up, which
means that there is striping without redundancy. This striping must be
taking care of by the kernel, and at boot time, the logic to handle the
striping is not yet available until the kernel is loaded.

In other words, the proper course of action for such a set-up would be
to totally discard the hardware-assisted RAID part - in other words,
switch it off - and simply use /dmraid/ to create individually striped
partitions. If you want to have the root filesystem living on a
stripe, then this requires that you have a separate */boot* partition
which itself is _*not*_ striped, because this is where the second stage
of the bootloader lives, and where the kernel image lives. And if you
want to load any drivers as modules, then you also need an /initrd/
because the modules are located under the */lib* directory, which lives
on the striped root filesystem.

So to sum it all up, Wim should...:

o disable the on-board RAID functionality; and

o either use a non-RAID'ed */boot* filesystem and an /initrd/
if he wants to have the root filesystem on a RAID 0 volume,
or use a non-RAID'ed */boot* _and_ root filesystem if he
wants to do without an /initrd./

Hope this was informative, and above all, helpful.

--
*Aragorn*
(registered GNU/Linux user #223157)

Easiest way to set up an initrd for raid is genkernel --dmraid

http://en.gentoo-wiki.com/wiki/RAID/Onboard


The grub configuration will look like:

title Gentoo Linux 2.6.30-r3
kernel (hd0,0)/kernel-genkernel-x86_64-2.6.30-gentoo-r3 root=/dev/ram0 real_root
=/dev/mapper/nvidia_edbefefd3 init=/linuxrc ramdisk=8192 dodmraid udev
#(change your_raid_set* to your root partition)
#init=/linuxrc may be needed if you get errors after the ramdrive loading
#udev is needed on some newer boards, add this if during boot it fails to find r
oot
initrd (hd0,0)/initramfs-genkernel-x86_64-2.6.30-gentoo-r3

Aragorn wrote:
> On Monday 27 July 2009 06:39, someone identifying as *J.O. Aho* wrote
> in /alt.os.linux.gentoo:/
>
>> Wim Cossement wrote:
>>> J.O. Aho wrote:
>>>> Wim Cossement wrote:
>>>>> I built in everything I could, including ramdisk/initrd stuff (but
>>>>> I don't need to use that I guess or not?),
>>>> Only if you built what you need to do the inital kernel boot as
>>>> modules, otherwise no.
>>> Modules? So built in stuff is not okay then?
>> initrd is only needed if you build things as modules, say you would
>> make your SATA and ext2 as modules, then you need to make an initrd
>> image, which you need to load during boot up.
>
> There is another scenario which requires an /initrd,/ and this is
> applicable to Wim's situation, but he did something wrong.

Well I'll have to brush up on that because I'm not that familiar with
initrd images but it's a nice thing to know... But I've done this before
so the knowledge is a bit rusty, to say the least

> See, another reason why you would need to have an /initrd/ is if you
> have your root filesystem sitting on a software RAID, and the PATA or
> SATA RAID functionality built into consumergrade motherboards is only a
> hardware-assisted software RAID solution, not a true hardware RAID
> solution. (I am elaborating on this for Wim's benefit, because I
> presume that you already know this. )
>
> There are basically three kinds of RAID...:
>
> (1) Hardware RAID is when the adapter itself has a dedicated processor
> and memory, and all the operating system needs to know about it is
> what driver to use for the device. All RAID functionality is
> handled by the adapter itself, and to the kernel, all disks in the
> RAID array - regardless of whether it is a RAID 0, 1, 5, 6 or JBOD
> - are seen as being a single physical disk.
>
> (2) Software RAID is when a set of independent disks connected to one
> or multiple disk controllers - which can be of mixed type - are
> combined into a RAID array by the operating system itself. In
> this scenario, the operating system does of course know that all
> disks are separate devices, but (in casu) Linux /dmraid/ sets them
> up to be used as a RAID array, which is then presented to userland
> as a single physical disk. The RAID functionalities - i.e.
> striping, mirroring, parity calculation, error handling et al - is
> all taken care of inside the kernel itself.
>
> (3) Hardware-assisted software RAID is like software RAID, but with
> just enough hardware assistence present to allow for hardware error
> checking and signaling. The kernel does all the RAID stuff, but the
> hardware assists.

I guess this is what I've been calling BIOS RAID.

> The latter is what Wim has in his machine and which he intends to use,
> but as it is in essence still a software RAID, there are limitations as
> to what you can do. For starters, Wim wants a RAID 0 set-up, which
> means that there is striping without redundancy. This striping must be
> taking care of by the kernel, and at boot time, the logic to handle the
> striping is not yet available until the kernel is loaded.

I'm only interested in the speed increase, so indeed I'll have to go for
number 2.

> In other words, the proper course of action for such a set-up would be
> to totally discard the hardware-assisted RAID part - in other words,
> switch it off - and simply use /dmraid/ to create individually striped
> partitions. If you want to have the root filesystem living on a
> stripe, then this requires that you have a separate */boot* partition
> which itself is _*not*_ striped, because this is where the second stage
> of the bootloader lives, and where the kernel image lives. And if you
> want to load any drivers as modules, then you also need an /initrd/
> because the modules are located under the */lib* directory, which lives
> on the striped root filesystem.

I will try this as soon I have some more spare time. I'm also aware you
need /boot mirrored in order to have a 'normal' copy of you kernel , but
another option might be to split up the file system even more in perhaps
a (small) mirrored root partition and putting /usr, /opt, /var and so on
on a stripe, no?
But this would make things too complicated...

> So to sum it all up, Wim should...:
>
> o disable the on-board RAID functionality; and

But I'm still confused since it's on that BIOS RAID now and was able to
install Gentoo and I'm able to load the kernel and boot from that
striped /boot (AFAIK).
How come the SysRescueCD did not have any problems on using that? Or are
booting for CD and HD not that comparable?

> o either use a non-RAID'ed */boot* filesystem and an /initrd/
> if he wants to have the root filesystem on a RAID 0 volume,
> or use a non-RAID'ed */boot* _and_ root filesystem if he
> wants to do without an /initrd./

As I said above, that's an option but the real PITA in the end will be
getting Windows to work (I did not by a $ 200 GPU for nothin' since
I attempted to install Windoze XP on a logical partition, this is also
quite a hassle.

> Hope this was informative, and above all, helpful.

Thanks for the tips so far, I guess it will take a few more days to get
it working.

I might even try installing Windows 7 in half a year... }

Wimmy

On Mon, 27 Jul 2009 20:31:42 +0200, Aragorn <aragorn RemoveThis @chatfactory.invalid> wrote:

>If fault tolerance is your goal, then I would definitely not recommend
>using RAID 0. RAID 5 is a good compromise on performance and fault
>tolerance, but it requires at least three disks and there is a small
>penalty to pay, i.e. when writing to a three-disk RAID 5, two blocks of
>data and one block of parity information must be written, and the
>parity must first be calculated. RAID 6 is the same thing but uses one
>more disk and two blocks of parity. So for writing, there is a
>performance hit at using RAID 5 or RAID 6 - if it is a software RAID,
>of course, because if it is a hardware RAID, the CPU won't even know
>about any of that.

If you're using something faster than a pentium I, the time needed to
handle software raid is neglible.

AZ Nomad wrote:
> Easiest way to set up an initrd for raid is genkernel --dmraid
>
> http://en.gentoo-wiki.com/wiki/RAID/Onboard

Yay! Tis back on the wire!

I'll keep you posted 'cause I'm sure I'll run into more quirks...

Wimmy
>
>
> The grub configuration will look like:
>
> title Gentoo Linux 2.6.30-r3
> kernel (hd0,0)/kernel-genkernel-x86_64-2.6.30-gentoo-r3 root=/dev/ram0 real_root
> =/dev/mapper/nvidia_edbefefd3 init=/linuxrc ramdisk=8192 dodmraid udev
> #(change your_raid_set* to your root partition)
> #init=/linuxrc may be needed if you get errors after the ramdrive loading
> #udev is needed on some newer boards, add this if during boot it fails to find r
> oot
> initrd (hd0,0)/initramfs-genkernel-x86_64-2.6.30-gentoo-r3

On Monday 27 July 2009 20:02, someone identifying as *Wim Cossement*
wrote in /alt.os.linux.gentoo:/

> Aragorn wrote:
>
>> There are basically three kinds of RAID...:
>>
>> (1) Hardware RAID is when the adapter itself has a dedicated
>> processor and memory, and all the operating system needs to know
>> about it is what driver to use for the device. All RAID
>> functionality is handled by the adapter itself, and to the
>> kernel, all disks in the RAID array - regardless of whether it is
>> a RAID 0, 1, 5, 6 or JBOD - are seen as being a single physical
>> disk.
>>
>> (2) Software RAID is when a set of independent disks connected to one
>> or multiple disk controllers - which can be of mixed type - are
>> combined into a RAID array by the operating system itself. In
>> this scenario, the operating system does of course know that all
>> disks are separate devices, but (in casu) Linux /dmraid/ sets
>> them up to be used as a RAID array, which is then presented to
>> userland as a single physical disk. The RAID functionalities -
>> i.e. striping, mirroring, parity calculation, error handling et
>> al - is all taken care of inside the kernel itself.
>>
>> (3) Hardware-assisted software RAID is like software RAID, but with
>> just enough hardware assistence present to allow for hardware
>> error checking and signaling. The kernel does all the RAID
>> stuff, but the hardware assists.
>
> I guess this is what I've been calling BIOS RAID.

Well, yes, it's the on-board RAID system of the motherboard, embedded
into the BIOS set-up program.

>> The latter is what Wim has in his machine and which he intends to
>> use, but as it is in essence still a software RAID, there are
>> limitations as to what you can do. For starters, Wim wants a RAID 0
>> set-up, which means that there is striping without redundancy. This
>> striping must be taking care of by the kernel, and at boot time, the
>> logic to handle the striping is not yet available until the kernel is
>> loaded.
>
> I'm only interested in the speed increase, so indeed I'll have to go
> for number 2.

It's the easiest software RAID solution, in my humble opinion. Linux
RAID is very good and quite performant.

>> In other words, the proper course of action for such a set-up would
>> be to totally discard the hardware-assisted RAID part - in other
>> words, switch it off - and simply use /dmraid/ to create individually
>> striped partitions. If you want to have the root filesystem living
>> on a stripe, then this requires that you have a separate */boot*
>> partition which itself is _*not*_ striped, because this is where the
>> second stage of the bootloader lives, and where the kernel image
>> lives. And if you want to load any drivers as modules, then you also
>> need an /initrd/ because the modules are located under the */lib*
>> directory, which lives on the striped root filesystem.
>
> I will try this as soon I have some more spare time. I'm also aware
> you need /boot mirrored in order to have a 'normal' copy of you kernel
> [...

I'm not sure what you're getting at here, because your intended set-up
is to use striping, not mirroring, and so you do not need any mirrored
kernel image.

If fault tolerance is your goal, then I would definitely not recommend
using RAID 0. RAID 5 is a good compromise on performance and fault
tolerance, but it requires at least three disks and there is a small
penalty to pay, i.e. when writing to a three-disk RAID 5, two blocks of
data and one block of parity information must be written, and the
parity must first be calculated. RAID 6 is the same thing but uses one
more disk and two blocks of parity. So for writing, there is a
performance hit at using RAID 5 or RAID 6 - if it is a software RAID,
of course, because if it is a hardware RAID, the CPU won't even know
about any of that.

For reading however, RAID 5 and RAID 6 are very fast because only the
striped data is read, not the parity blocks. Reading speed is
comparable to RAID 0, but you have the benefit of redundancy.

> ...] but another option might be to split up the file system even more
> in perhaps a (small) mirrored root partition and
> putting /usr, /opt, /var and so on on a stripe, no?
> But this would make things too complicated...

I myself always split off all that I can from the root filesystem, but I
don't use software RAID. My server has an SAS/SATA hardware RAID
adapter from Adaptec in it, and I use two RAID 5 set-ups of four disks
each, i.e. four SAS disks and four hotswappable SATA-II disks.

>> So to sum it all up, Wim should...:
>>
>> o disable the on-board RAID functionality; and
>
> But I'm still confused since it's on that BIOS RAID now and was able
> to install Gentoo and I'm able to load the kernel and boot from that
> striped /boot (AFAIK).
> How come the SysRescueCD did not have any problems on using that? Or
> are booting for CD and HD not that comparable?

I have no clue why it worked in your case, really. Normally, those
on-board RAID adapters are just hardware-assisted software RAID things.
It is however possible that the MBR gets mirrored or striped by the
on-board RAID or something weird like that. To be quite honest, I
don't know what exactly it is that they do. :-/

>> o either use a non-RAID'ed */boot* filesystem and an
>> /initrd/ if he wants to have the root filesystem on a RAID
>> 0 volume, or use a non-RAID'ed */boot* _and_ root
>> filesystem if he wants to do without an /initrd./
>
> As I said above, that's an option but the real PITA in the end will be
> getting Windows to work (I did not by a $ 200 GPU for nothin'
> since I attempted to install Windoze XP on a logical partition, this
> is also quite a hassle.

Ewww, Wintendo... :p

Well, okay, here's my advice on that... Windows *needs* an active
primary partition, eventhough the main Windows system need not be
installed in that partition. The active primary partition can be only
a few MB in size. It just needs that because of the DOS-style way it
boots, i.e. the partition must be marked active and must be primary, so
that the Windows bootloader can be installed in the bootsector of
the /sea/ /drive/, and from there on, the bootloader can load
NTKRNL.EXE (or whatever it's called) from an eventual /duh/ drive,
which can be a logical partition in the extended partition container.

So if you're going to dual-boot between UNIX and Wintendo, then you must
install Wintendo *first* and only after that is done can you start on
installing GNU/Linux. Linux does not care about whether a partition is
active or not, or primary or logical. So you can install GNU/Linux in
a series or logical partitions in an extended partition container.

The only thing you should keep in mind is that unless your hard disk is
of the IDE/PATA type and still being used as such - which is not the
case in modern kernels - your amount of usable partitions in Linux will
be limited to 15, the extended container itself also included. So
wisest might be - if you need more than that - to use logical volume
management instead.

>> Hope this was informative, and above all, helpful.
>
> Thanks for the tips so far, I guess it will take a few more days to
> get it working.
>
> I might even try installing Windows 7 in half a year... }

Well, each to his own, or something like that, but personally I'm not
into S&M. :p

--
*Aragorn*
(registered GNU/Linux user #223157)

On Mon, 27 Jul 2009 22:58:24 +0200, Marcel Bruinsma <we-love-all-spam.RemoveThis@gmail.com> wrote:
>Wim Cossement wrote:

>> I will try this as soon I have some more spare time. I'm also aware
>> you need /boot mirrored in order to have a 'normal' copy of you kernel
>> , but another option might be to split up the file system even more in
>> perhaps a (small) mirrored root partition and putting /usr, /opt, /var
>> and so on on a stripe, no?

>If the root fs is on a raid0 (or raid10, or raid5) md array, you need
>a separate boot fs. If you care about fault tolerance (raid0 seems to

dmraid starts the software raid system before any partitions are
mounted. It boots a partition off a raid volume just fine.

Wim Cossement wrote:

> I will try this as soon I have some more spare time. I'm also aware
> you need /boot mirrored in order to have a 'normal' copy of you kernel
> , but another option might be to split up the file system even more in
> perhaps a (small) mirrored root partition and putting /usr, /opt, /var
> and so on on a stripe, no?

If the root fs is on a raid0 (or raid10, or raid5) md array, you need
a separate boot fs. If you care about fault tolerance (raid0 seems to
imply you don't) you can have the boot fs on a raid1 array, provided
you use metadata format 1.0 (1.1 and 1.2 won't do). The boot loader
will read the kernel and initrd from one of the members of that array
without knowing that it's part of an md.

Because /usr and /opt are mainly read-only sub-trees, they are good
candidates for a raid5 array, /if/ you care about fault tolerance. Else,
you might as well put everything (except /home) on one raid0 array.

--
printf -v email $(echo \ 155 141 162 143 145 154 142 162 165 151 \
156 163 155 141 100 171 141 150 157 157 056 143 157 155|tr \ \\\\)
# Live every life as if it were your last! #

AZ Nomad <aznomad.3.DeleteThis@PremoveOBthisOX.COM> wrote:
>
> If you're using something faster than a pentium I, the time needed to
> handle software raid is neglible.

I call that a hard-to-kill myth.
Someone who tested with a 2.2 GHz X2 disagrees:
http://linux.com/news/hardware/servers/8222-benchmarking-hardware-raid...-linux-

"Wrap up
For RAID-10 using a smaller block size such as 256KB you get a noticeable
difference between hardware and software in rewrite performance. Hardware
beats out software substantially in output for parity RAID, with hardware
being 66% faster for block output and 20% faster for rewrite performance in
a 256KB chunk RAID-6."

Another factor is that the _rest_ of the system isn't slowed down when using
hardware RAID. You can DMA off big blocks, and let the RAID controller do
the rest, instead of having to manually split the data into multiple
destinations, invalidating your CPU cache and causing IO waits.

Regards,

--

*Art

AZ Nomad wrote:

> On Mon, 27 Jul 2009 22:58:24 +0200, Marcel Bruinsma
> <we-love-all-spam.DeleteThis@gmail.com> wrote:
>
>> If the root fs is on a raid0 (or raid10, or raid5) md array, you need
>> a separate boot fs. If you care about fault tolerance (raid0 seems to
>
> dmraid starts the software raid system before any partitions are
> mounted. It boots a partition off a raid volume just fine.

Which boot loader are you using?

--
printf -v email $(echo \ 155 141 162 143 145 154 142 162 165 151 \
156 163 155 141 100 171 141 150 157 157 056 143 157 155|tr \ \\\\)
# Live every life as if it were your last! #

On Tue, 28 Jul 2009 01:57:33 +0200, Marcel Bruinsma <we-love-all-spam DeleteThis @gmail.com> wrote:
>AZ Nomad wrote:

>> On Mon, 27 Jul 2009 22:58:24 +0200, Marcel Bruinsma
>> <we-love-all-spam DeleteThis @gmail.com> wrote:
>>
>>> If the root fs is on a raid0 (or raid10, or raid5) md array, you need
>>> a separate boot fs. If you care about fault tolerance (raid0 seems to
>>
>> dmraid starts the software raid system before any partitions are
>> mounted. It boots a partition off a raid volume just fine.

>Which boot loader are you using?

grub.

On Tue, 28 Jul 2009 06:06:24 +0200, Marcel Bruinsma <we-love-all-spam.DeleteThis@gmail.com> wrote:
>AZ Nomad wrote:

>> On Tue, 28 Jul 2009 01:57:33 +0200, Marcel Bruinsma
>> <we-love-all-spam.DeleteThis@gmail.com> wrote:
>>>AZ Nomad wrote:
>>
>>>> On Mon, 27 Jul 2009 22:58:24 +0200, Marcel Bruinsma
>>>> <we-love-all-spam.DeleteThis@gmail.com> wrote:
>>>>
>>>>> If the root fs is on a raid0 (or raid10, or raid5) md array, you
>>>>> need a separate boot fs. If you care about fault tolerance (raid0
>>>>
>>>> dmraid starts the software raid system before any partitions are
>>>> mounted. It boots a partition off a raid volume just fine.
>>
>>>Which boot loader are you using?
>>
>> grub.

>Looks like I'll have to update mine. My grub can't load the
>kernel from a striped array (or a raid5). I wasn't aware the
>more recent grub has improved that much.

I first used the gentoo dmraid recipe back in 2005. I doubt you have
an older version.

The recipe is something like this:

grub --device-map=/dev/null

"Now that you're inside of the grub environment, set up your raid
devices and install grub!"

grub> device (hd0,0) /dev/mapper/nvidia_abiccada1
grub> device (hd0) /dev/mapper/nvidia_abiccada
grub> root (hd0,0)
grub> setup (hd0)

Google for "gentoo dmraid"
http://en.gentoo-wiki.com/wiki/RAID/NVRAID_with_dmraid