2013-02-06

Mageia2 on EC2: Cruising Altitude

This is my fourth post on getting Mageia2 running on Amazon Web Services' Elastic Compute Cloud. See my first post in the series for an overview.

In the last post, I addressed the problem of having only a test kernel by tweaking the Mageia kernel SRPM and creating a gzipped kernel that can be used with the version of PV-GRUB supplied by Amazon. Now I'll walk through the steps of building an EBS backend instance instead of an instance-store backed instance.

You need a working Mageia setup on an instance-store backed instance before you can create the EBS backed one. Just launch the AMI created in a previous step and then attach a 32GB EBS volume to it. Using the EC2 API tools, you attach the volume like this:

SIZE=32
TARGETAZ=us-east-1a
INSTID=i-09abcdef

CMD=($(ec2-create-volume --size $SIZE --availability-zone $TARGETAZ --type standard))
VOLID=${CMD[1]}
ec2-attach-volume $VOLID --instance $INSTID --device /dev/sdg


You will also need some other components:
  1. "kernel-server" RPM created in a last post.
  2. A copy of ec2-get-ssh.sh for the mageia user
The second component is so you don't have to embed passwords in your AMI, but instead uses ssh public keys that are imported to (or generated by) AWS.

Another difference is that we add the kernel to the skip.list for upgrades, as we don't want to get a non-gzipped kernel. So, here's the steps for setting it up:

mkdir $HOME/ec2

# everything forward needs to be done as root 
sudo bash -o vi
cd $HOME/ec2
export PATH=$PATH:/sbin:/usr/sbin

# setup the filesystem
/sbin/mkfs -t ext4 /dev/xvdg

# mount the image for chroot
export CHRDIR=$HOME/ec2/loop
mount /dev/xvdg $CHRDIR

# create the minimum devices
mkdir $CHRDIR/dev
/sbin/makedev $CHRDIR/dev console
/sbin/makedev $CHRDIR/dev null
/sbin/makedev $CHRDIR/dev zero

# setup the minimum filesystems
mkdir $CHRDIR/etc
cat > $CHRDIR/etc/fstab << EOF
/dev/xvda1 /         ext3    defaults        1 1
none       /dev/pts  devpts  gid=5,mode=620  0 0
none       /dev/shm  tmpfs   defaults        0 0
none       /proc     proc    defaults        0 0
none       /sys      sysfs   defaults        0 0
EOF

# add required /proc filesystem
mkdir $CHRDIR/proc
mount -t proc none $CHRDIR/proc

# choose the best/fastest mirror
GET http://mirrors.mageia.org/api/mageia.2.x86_64.list | grep country=US
# setup the urpmi media locations in the chroot
urpmi.addmedia --distrib --urpmi-root $CHRDIR http://mirrors.kernel.org/mageia/distrib/2/x86_64
# install the minimum packages
urpmi --auto --urpmi-root $CHRDIR basesystem urpmi locales-en sshd sudo dhcp-client

# MASSIVE HACK TIME
rpm --root=$CHRDIR -Uhv custom-kernel/kernel-server-3.3.8-2.mga2-1-1.mga2.x86_64.rpm

# cleanup desktop kernel
chroot $CHRDIR
urpme kernel-desktop-3.3.8-2.mga2-1-1.mga2
rm -f initrd-desktop.img  vmlinuz-desktop 
# confirm there's a good initrd
cd /boot
stat initrd-3.3.8-server-2.mga2.img
mkinitrd initrd-3.3.8-server-2.mga2.img 3.3.8-server-2.mga2
exit

# set the kernel to load on boot
cat > $CHRDIR/boot/grub/menu.lst << EOF
default=0
timeout=0
title linux
  root (hd0)
  kernel /boot/vmlinuz-server ro root=/dev/xvda1 console=hvc0 BOOT_IMAGE=linux-nonfb
  initrd /boot/initrd-server.img
EOF

# do not upgrade the kernel, until upstream fixes the xz/gz issue
test -f $CHRDIR/etc/urpmi/skip.list || cp -p $CHRDIR/etc/urpmi/skip.list $CHRDIR/etc/urpmi/skip.list.orig
cat > $CHRDIR/etc/urpmi/skip.list << EOF
# Here you can specify the packages that won't be upgraded automatically
# for example, to exclude all apache packages :
# /^apache/
/^kernel/
EOF

# configure the chroot network for ec2
cat > $CHRDIR/etc/sysconfig/network-scripts/ifcfg-eth0 << EOF
DEVICE=eth0
BOOTPROTO=dhcp
ONBOOT=yes
TYPE=Ethernet
USERCTL=yes
PEERDNS=yes
IPV6INIT=no
EOF
cat > $CHRDIR/etc/sysconfig/network << EOF
NETWORKING=yes
CRDA_DOMAIN=US
EOF

# configure ssh
test -f $CHRDIR/etc/ssh/sshd_config.orig || cp -p $CHRDIR/etc/ssh/sshd_config $CHRDIR/etc/ssh/sshd_config.orig
cat $CHRDIR/etc/ssh/sshd_config.orig |
    sed -e 's/^#UseDNS yes/UseDNS no/g' |
    sed -e 's/^PermitRootLogin no/PermitRootLogin without-password/g' > $CHRDIR/etc/ssh/sshd_config
# create login account
chroot $CHRDIR /usr/sbin/useradd --create-home --home /home/mageia --shell /bin/bash mageia
(umask 0227; echo "mageia ALL=(ALL) NOPASSWD:ALL" > $CHRDIR/etc/sudoers.d/mageia)

# setup ssh public key
cp ec2-get-ssh $CHRDIR/etc/rc.d/init.d/ec2-get-ssh
chmod 0750 $CHRDIR/etc/rc.d/init.d/ec2-get-ssh
chown root:root $CHRDIR/etc/rc.d/init.d/ec2-get-ssh
chroot $CHRDIR /sbin/chkconfig ec2-get-ssh on

# dismount the chroot
umount $CHRDIR/proc
umount -d $CHRDIR
Now that the EBS volume is all set, it needs to be snapshotted and registered as an AMI. Here's what you do:

ec2-detach-volume $VOLID --instance $INSTIT--device /dev/sdg

# create a snapshot
CMD=($(ec2-create-snapshot --description "Mageia 2" $EBSVOL))
SNAPID=${CMD[1]}

# create AMI
AKIID="aki-88aa75e1"
ec2-register --name "Mageia 2" --description "Mageia 2" \

    --architecture x86_64 --root-device-name /dev/sda1 \
    --block-device-mapping /dev/sda1=$SNAPID --kernel $AKIID


Now you're ready to launch your EBS back Mageia2 Linux instance! Enjoy!

2012-12-21

Vi IMproved

Ubuntu tweak #2 - diediedie nano die!

$ sudo update-alternatives --config editor
There are 4 choices for the alternative editor (providing /usr/bin/editor).

  Selection    Path                Priority   Status
------------------------------------------------------------
* 0            /bin/nano            40        auto mode
  1            /bin/ed             -100       manual mode
  2            /bin/nano            40        manual mode
  3            /usr/bin/vim.basic   30        manual mode
  4            /usr/bin/vim.tiny    10        manual mode

Press enter to keep the current choice[*], or type selection number: 3
update-alternatives: using /usr/bin/vim.basic to provide /usr/bin/editor (editor) in manual mode

Ahh, much better!

2012-12-20

Disable dnsmasq in NetworkManager

I have recently converted my work desktop to Ubuntu 12.10. Most things were better, but I was seeing horrible DNS lag from dnsmasq. To disable it, I've done the following:

sudo vi /etc/NetworkManager/NetworkManager.conf
# comment out dns=dnsmasq
sudo restart network-manager

This will regenerate your resolv.conf and you'll see your DNS servers directly and not localhost.

2012-12-11

Mageia2 on EC2: Stormy Weather


This is my third post on getting Mageia2 running on Amazon Web Services' Elastic Compute Cloud. See my first post in the series for an overview.

In my last post I described how to create and upload an AMI that allows you to run Mageia2 on EC2. There were two issues with that method:
  1. The EC2 instances are using a one-off unverified kernel, obtained for testing purposes only.
  2. The instances launched can only be instance-store backed, ephemeral disk.
Both of these problems are solvable. We'll address the kernel first. The solution? Compile your own!

As with any good open-source project, you can easily obtain the source code. The same holds for Mageia. For recompiling the kernel, I plucked the kernel's SRPM file off mirrors.kernel.org. Reviewing the source, there were considerable tweaks made by the Mageia development team - so much so that they were bundled together inside the SRPM. Once I was able to dig into that tarball, I found where to enable CONFIG_KERNEL_GZIP and disable CONFIG_KERNEL_XZ in the configuration.

Now it was a matter of getting a system to build the kernel on. Initially, I tried to do it on my local seed Mageia VM, but the 10GB disk was too small to hold all the compiled kernel sources. So, I launched an instance using the EC2 console of the freshly-uploaded Mageia2 AMI. This is where I ran into the limitation of the first revision of the AMI I created - 2GB was insufficient to install all the compiler dependencies needed for creating a kernel package - but 8GB was ok.

Finally, I launched an m2.xlarge instance via the EC2 console with an 8GB root disk on instance-store to do the compilation. I wanted an instance with at least 2 cores to speed up the compile and sufficient additional space on ephemeral disk (/dev/xvdb) that could hold the compiled kernel sources. It still took a considerable amount of time - approximately 2 hours to compile the RPM. When I have to do this again in the future, I might consider a high I/O instance to reduce the time spent waiting for the compile. Either way, the cost is negligible if you remember to shut it down after you're done - the m2.xlarge was around $1.35 for 3 hours and a hi1.4xlarge would be $3.10 for one hour. For those who follow this blog, you should recognize the build script.

EDIT: on 1/18/13, I recompiled the kernel using an hi1.4xlarge and the final timing from "time ./do-build.sh" was:

real    51m5.356s
user    173m1.470s
sys     32m44.040s

Of course, it took a little longer than that to install all the packages needed for building, but it can be done in less than 2 hrs.


Here are the steps for compiling the Mageia2 kernel:

#
# prep stuff done as root
#
sudo bash -o vi
# mount the ephemeral storage
mkfs -t ext4 /dev/xvdb
mkdir /media/extra
mount /dev/xvdb /media/extra
# create some swap
dd if=/dev/zero of=/media/extra/swapfile00 bs=1024 count=4194304
mkswap /media/extra/swapfile00
swapon /media/extra/swapfile00
# setup space for kernel building
mkdir /media/extra/kernel
chown $USER:$USER /media/extra/kernel
exit

#
# build stuff done as normal user
#
# prep for kernel building
cd $HOME
ln -s /media/extra/kernel
cd kernel/
# bring down the source
curl -O http://mirrors.kernel.org/mageia/distrib/2/SRPMS/core/updates/kernel-3.3.8-2.mga2.src.rpm
mkdir SOURCES
cd SOURCES
# extract the source
rpm2cpio ../kernel-3.3.8-2.mga2.src.rpm | cpio -i
# make a working copy of the .spec file
cp -p kernel.spec ..
# extract the mageia customizations
tar Jxf linux-3.3.8-mga2.tar.xz
cd 3.3.8-mga2/configs/
# modify the kernel config for gzip compression
cp -p x86_64.config x86_64.config.orig
vi x86_64.config
# diff of what it looks like when it's done
$ diff -u x86_64.config.orig x86_64.config
--- x86_64.config.orig  2012-07-12 08:53:47.000000000 +0000
+++ x86_64.config       2012-11-15 04:48:37.000000000 +0000
@@ -67,10 +67,10 @@
 CONFIG_HAVE_KERNEL_LZMA=y
 CONFIG_HAVE_KERNEL_XZ=y
 CONFIG_HAVE_KERNEL_LZO=y
-# CONFIG_KERNEL_GZIP is not set
+CONFIG_KERNEL_GZIP=y
 # CONFIG_KERNEL_BZIP2 is not set
 # CONFIG_KERNEL_LZMA is not set
-CONFIG_KERNEL_XZ=y
+# CONFIG_KERNEL_XZ is not set
 # CONFIG_KERNEL_LZO is not set
 CONFIG_DEFAULT_HOSTNAME="(none)"
 CONFIG_SWAP=y

# rebuild the mageia customizations
cd ../..
mv linux-3.3.8-mga2.tar.xz linux-3.3.8-mga2.tar.xz.orig
tar Jcf linux-3.3.8-mga2.tar.xz 3.3.8-mga2

# install builder dependencies
sudo urpmi easyrpmbuilder
sudo urpmi elfutils-devel zlib-devel binutils-devel newt-devel python-devel pciutils-devel asciidoc xmlto docbook-style-xsl

# setup the build script
cat do-build.sh
#!/bin/sh -x
rm -rf BUILD BUILDROOT RPMS SRPMS tmp || true
mkdir -p BUILD BUILDROOT RPMS SRPMS tmp

OPTS=""
OPTS="$OPTS --with=server"
OPTS="$OPTS --without=desktop"
OPTS="$OPTS --without=desktop586"
OPTS="$OPTS --without=netbook"
rpmbuild $OPTS -bb --define="_topdir $PWD" --define="_tmppath $PWD/tmp" kernel.spec 2>&1 | tee kernel-build.txt

# do the build
time ./do-build.sh

# save the rpm
scp -p RPMS/x86_64/kernel-server-3.3.8-2.mga2-1-1.mga2.x86_64.rpm $REMOTE_SERVER:

Ratings and Recommendations by outbrain