Creating A RAID1 (Software RAID) Setup in Ubuntu Server 9.10

For many, creating a RAID setup may seem like an elusive task. Frankly, its not as I will show you.

Redundant Array of Independent Disks is a method whereby many hard drives are configured to act as one. There are two main types of setup in RAID:-

RAID0 is in favour of performance/speed while RAID1 is in favour of redundancy/reliability. Disks that are to be configured in RAID1 or RAID0 (which are supposed to be 2 in number) are supposed to be identical. There are other RAID setups that involve more hard disks but I won’t get into them here. In Brief, these are the ones i theoretically know of:-

* RAID0+1 – uses RAID0 and RAID1 at the same time with four identical hard disk drives. If one of the hard disk drive fails, the system becomes a RAID0 system.

* RAID10 – uses RAID0 and RAID1 at the same time also with four identical hard disk drives. If one of the hard disk drive fails, the system becomes a RAID1 system.

* RAID5 – this one is actually a RAID0 system that needs at least three identical hard disk drives. It stores parity information to improve the reliability of the disks. In a case whereby three drives are used, the total capacity will be double the size of each hard disk drive. e.g If I use three 750GB hard drives, my total hard disk space will be 1500GB since the rest of the space will be used to store this parity data.

Aaaaaaanayway….

In RAID0, when data is written to the hard disks, it is divided into several chunks whereby each chunk will be saved on a different drive. I won’t get into the intricacies of how RAID divides and distributes the data today. However, the main point is that if a 500KB file is stored and 250KB stored on each hard drive, it takes half the time to store each file since the hard disks are working in parallel therefore meaning MORE SPEED.

RAID1 on the other hand shows the two hard disks as one. By this i mean, like in my case, two 750GB hard drives set up in RAID1 will show as 750GB in your system. This is because, data that is sent to the 750GB RAID Volume is copied to both physical hard drives. This is why RAID1 is also known as mirroring. If one hard drive fails, all the data exists in the other hard drive..alas! REDUNDANCY and RELIABILITY in the flesh!I opted for data redundancy since i was setting up a server.

The next option that comes along the way is which method of RAID setup to use. RAID can either be setup as Hardware RAID or Software RAID. Hardware RAID can only work on machines whose motherboards have so called RAID Controllers. In this case, RAID is set up in the BIOS. The controller makes the OS think there’s only one drive and maintains the drives ‘invisibly’. I opted for Software RAID whereby no controller is needed. In this case, the OS knows about both drives and actively maintains both of them. The truth is, Linux software RAID is ideal for mirroring, and due to kernel disk caching and buffering it can actually be faster than RAID1 on lower end RAID hardware. However, for larger requirements like RAID 5, the CPU can still get bogged down with software RAID.

So… lets set up our two hard disks with Ubuntu Server 9.10 (Karmic Koala).

Follow the installation steps until you get to the Partition disks step, then:

  1. Select Manual as the partition method.
  2. Select the first hard drive, and agree to “Create a new empty partition table on this device?”.

    Repeat this step for each drive you wish to be part of the RAID array.

  3. Select the “FREE SPACE” on the first drive then select “Create a new partition”.
  4. Next, select the Size of the partition. This partition will be the swap partition, and a general rule for swap size is twice that of RAM. Enter the partition size, then choose Primary, then Beginning.
  5. Select the “Use as:” line at the top. By default this is “Ext3 journaling file system”, change that to “physical volume for RAID” then “Done setting up partition”.
  6. For the / partition once again select “Free Space” on the first drive then “Create a new partition”.
  7. Use the rest of the free space on the drive and choose Continue, then Primary.
  8. As with the swap partition, select the “Use as:” line at the top, changing it to “physical volume for RAID” then choose “Done setting up partition”.
  9. Repeat steps three through eight for the other disk and partitions.

There should now be a list of hard drives and RAID devices. The next step is to format and set the mount point for the RAID devices. Treat the RAID device as a local hard drive, format and mount accordingly.

  1. Select the RAID1 device #0 partition.
  2. Choose “Use as:”. Then select “swap area”, then “Done setting up partition”.
  3. Next, select the RAID1 device #1 partition.
  4. Choose “Use as:”. Then select “Ext3 journaling file system”.
  5. Then select the “Mount point” and choose “/ – the root file system”. Change any of the other options as appropriate, then select “Done setting up partition”.
  6. Finally, select “Finish partitioning and write changes to disk”.

If you choose to place the root partition on a RAID array, the installer will then ask if you would like to boot in a degraded state. At some point in the life of the computer a disk failure event may occur. When this happens, using Software RAID, the operating system will place the array into what is known as a degraded state. If the array has become degraded, due to the chance of data corruption, by default Ubuntu Server Edition will boot to initramfs after thirty seconds. Once the initramfs has booted there is a fifteen second prompt giving you the option to go ahead and boot the system, or attempt manual recover. Booting to the initramfs prompt may or may not be the desired behavior, especially if the machine is in a remote location. Booting to a degraded array can be configured several ways:

  • The dpkg-reconfigure utility can be used to configure the default behavior, and during the p rocess you will be queried about additional settings related to the array. Such as monitoring, email alerts, etc. To reconfigure mdadm enter the following:
    sudo dpkg-reconfigure mdadm
  • The dpkg-reconfigure mdadm process will change the /etc/initramfs-tools/conf.d/mdadm configuration file. The file has the advantage of being able to pre-configure the system’s behavior, and can also be manually edited by setting the ‘boot degraded’ option to true (BOOT_DEGRADED=true)

Finish the rest of the install and you’re good to go. For a more detailed and visual example, check out this guy’s detailed setup in Ubuntu Server 8.04. Not much has changed in the setup. CHEERS!

http://kuparinen.org/martti/comp/ubuntu/en/raid.html

Getting rid of virbr0/virbr1 interface in Ubuntu Server 9.10

After a fresh install of Ubuntu 9.04,  you might have a network interface virbr0 if you’ve selected to install a virtual machine emulator.  During the CD install, selecting vm, it will install kvm and xen.  It will create a network interface virbr0 or virbr1 to perform host networking.

Later, if you decide to install VMWare, VirtualBox or decide you really didn’t want a virtual machine server, you can remove the applications easily.  Just do a (sudo):

apt-get purge kvm

apt-get purge libxen3

Installation and configuration of a headless Ubuntu Server

Bear in mind that at the time of this exercise I was installing a Ubuntu Server 9.10 on VirtualBox 3.0.10.

1. First install all necessary packages. Apt-get virtualbox from the ubuntu repositories:

sudo apt-get install virtualbox-ose

You can also download the deb and install it manually:-

e.g

sudo dpkg -i virtualbox-3.0_3.0.10-54097_Ubuntu_karmic_amd64.deb

To get the proprietary version:-

(/etc/apt/sources.list)

deb http://download.virtualbox.org/virtualbox/debian karmic non-free

Then download Sun’s public key:- 

And finally:- 

sudo aptitude install linux-headers-$(uname -r) build-essential virtualbox-3.0 dkms

dkms is optional. Its a package that ensures that the VirtualBox host kernel modules are properly updated if the Linux kernel version changes

The group vboxusers will be created during installation. Note that a user who is going to run VirtualBox must be member of that group:-

sudo usermod -a -G vboxusers username


2. Since we want to setup our network in such a way that we have network bridges, we need to install bridge-utils.

sudo apt-get install bridge-utils

                                                                                                                 

3. The next step is to edit /etc/network/interfaces on the host machine and add the br0 interface with the same address as you primary interface, in this case eth0. Before you begin, backup your interfaces file since it is vital to your networking functionality.

sudo cp /etc/network/interfaces /etc/network/interfaces.`date +%F~%T`

This process varies depending on whether you’re host uses a static or dynamic IP address.

Dynamic

           $ sudo nano /etc/network/interfaces

           auto eth0

           iface eth0 inet manual

           auto br0

           iface br0 inet dhcp

               bridge_ports eth0 vbox0

           # The loopback network interface

           auto lo

           iface lo inet loopback

eth0 is the name of your interface, it can be different depending on your machine.br0 is an arbitrary name for the bridge.vbox0 is an arbitrary name for the device VBox will use. If you want more devices , you can add them:- bridge_ports eth0 vbox0 vbox1 vbox2 vbox3 vbox4  blah..blah..blahh

Static

       $ sudo nano /etc/network/interfaces

        auto eth0

        iface eth0 inet manual

        auto br0

        iface br0 inet static

             address xxx.xxx.xxx.xxx

             netmask xxx.xxx.xxx.xxx

             gateway xxx.xxx.xxx.xxx

             bridge_ports eth0 vbox0 vbox1

        # The loopback network interface

             auto lo

             iface lo inet loopback

4. Now restart your networking:- 

sudo /etc/init.d/networking restart

Ignore the messages complaining about the vbox# interface

Afterwards, ifconfig and you should see your br0 interface

5. Now create a file/etc/vbox/interfaces and add the following:-

vbox root br0

This file is used to declare the virtual interfaces needed by VirtualBox. Each line should be of the format [] . Declare as many interfaces as you want here for all your virtual machines. Just make sure they also exist in /etc/network/interfaces as shown earlier.

And now restart virtualbox to register the changes:-

/etc/init.d/vboxdrv restart

 

 N/B

Most sources on the internet indicate that you should use /etc/init.d/virtualbox-ose restart for VirtualBox OSE and /etc/init.d/vboxnet restart and for the proprietary version. On my version, only vboxdrv existed in init.d so if any of the two are the ones that exist in yours, don’t hesitate.

6. The last thing in the host’s network configuration would be to set the permissions on /dev/net/tun

                         $ sudo chown root:vboxusers /dev/net/tun

                         $ sudo chmod g+rw /dev/net/tun

This file is created with the default permissions every time the system restarts, so the best option would be to run the vms everytime as root. At this point its safe to say that installation and host configuration is complete. It is advised to restart your system.

Creating A New VM

1. First Register the Virtualbox machine with the machine name, in this case ubuntu_server1 (think of it as a shell we’ll customize later) :-

VBoxManage createvm –name ubuntu_server_1 –register

2. Then now give the VirtualBox machine all its properties:- 

VBoxManage modifyvm “ubuntu_server_1” –memory “256MB” –acpi on –boot1 dvd –nic1 bridged –bridgeadapter1 eth1

If you need to change any of the properties in future, do it the same way its done above e.g If I want to turn vrdp off at a later point, all I have to do is VBoxManage modifyvm ubuntu_server_1 –vrdp off   

3. Tell VirtualBox to mount the install ISO you want to install as the OS:- 

VBoxManage modifyvm ubuntu_server_1 –dvd /home/avallainafrica/images/ubuntu-9.10-server-i386.iso

4. Create a virtual hd for the virtual machine. In this case, the drive has been allocated 10GB :- 

VBoxManage createhd –filename “ubuntuserver1.vdi” –size 10000 –remember

5. Tell the virtual machine to use the hard drive you just created:-

VBoxManage modifyvm “ubuntu_server_1” –hda “ubuntuserver1.vdi”

Your virtual machine is now good to go. VirtualBox comes with a front-end called

VBoxHeadless, which produces no visible output on the host at all, but instead only delivers VRDP data. To start the virtual machine in your host, do this :-

 VBoxHeadless –startvm ubuntu_server_1

You should see such output when the machine is started:-

        VirtualBox Headless Interface 3.0.10

       (C) 2008-2009 Sun Microsystems, Inc.

       All rights reserved.

       Listening on port 3389

If you want to start the machine headless and detached in it’s own screen session so it’ll keep running even after you close your terminal session or log out of your machine:-

screen -d -m -S ubuntu_server_1 VBoxHeadless –startvm ubuntu_server_1

OR 

nohup VBoxHeadless –startvm &

To access the machine remotely, all you need to do is use the remote desktop client of your choice. Rdesktop worked swell for me on Ubuntu (sudo apt-get install rdesktop) . In this case:-

rdesktop 192.168.0.157

Remember that the virtual machine needs to be running in the host/server.

You will see the OS image you mounted booting and the rest is just normal installation of the OS in this case, Ubuntu Server. After installation, one thing you’ll notice when you start the machine again and access it via VRDP is that it will still boot from the ISO image. This is because the –boot1 option is still on dvd which is the mounted drive. Here’s how to fix this:-

VBoxManage modifyvm ubuntu_server_1 –-dvd none

There are quite a number of things you can do to tweak the configuration of your existing Vboxserver using VBoxManage. The best source for the wide array of commands is the VirtualBox user manual(http://www.virtualbox.org/manual/UserManual.html#vboxmanage) but I’ve compiled some

useful ones here:-


How to list VM information.

How to show the VirtualBox VM info

VBoxManage showvminfo

How to show the VM Harddrive info

VBoxManage showhdinfo

How to list running VM

VBoxManage list runningvms

How to list available VM

VBoxManage list vms

How to list available VM Harddrives

VBoxManage list hdds

How to list available ISO’s

VBoxManage list dvds

                                                                                                               
How to control the vm

How to pause VM

VBoxManage controlvm pause

How to resume VM

VBoxManage controlvm resume

How to reset VM

VBoxManage controlvm reset

How to poweroff VM (hard poweroff eg. pull the plug)

VBoxManage controlvm poweroff

How to send poweroff single to VM (tells VM OS to shutdown)

VBoxManage controlvm acpipowerbutton

How to attach a DVD / CD to a running vm

VBoxManage controlvm dvdattach

How to de-attach a DVD / CD from a running vm

VBoxManage controlvm dvdattach none

Restoring Ubuntu’s GRUB Loader after installing Windows

THIS EXCERPT WAS COPIED WORD FOR WORD FROM http://www.howtogeek.com/howto/ubuntu/reinstall-ubuntu-grub-bootloader-after-windows-wipes-it-out/ . I JUST PASTED IT HERE JUST IN CASE THE LINK BREAKS SOME DAY. 🙂

If you run a dual-boot system with Linux and Windows, this has happened to you. You had to do your monthly reinstall of Windows, and now you don’t see the linux bootloader anymore, so you can’t boot into Ubuntu or whatever flavor of linux you prefer.

Here’s the quick and easy way to re-enable Grub.

1) Boot off the LiveCD

2) Open a Terminal and type in the following commands, noting that the first command will put you into the grub “prompt”, and the next 3 commands will be executed there. Also note that hd0,0 implies the first hard drive and the first partition on that drive, which is where you probably installed grub to during installation. If not, then adjust accordingly.

sudo grub

> root (hd0,0)

> setup (hd0)

> exit

Reboot (removing the livecd), and your boot menu should be back.

Only read below if Windows is now missing from the boot menu

If you installed Ubuntu before you installed Windows, then Ubuntu will not have anything in the grub configuration for Windows. This is where you’ll have to do a bit of manual editing to the grub boot menu file.

If you open the file /boot/grub/menu.lst with the following command:

sudo gedit /boot/grub/menu.lst

You’ll see a sample section for Windows, which you’ll want to uncomment and add to the boot menu list in whatever position you want it in. (uncomment by removing the #’s)

# title   Windows 95/98/NT/2000

# root   (hd0,0)

# makeactive

# chainloader   +1

Note that you should also verify that hd0,0 is the correct location for Windows. If you had installed Windows on the 4th partition on the drive, then you should change it to (hd0,3)