The first step to creating a Mageia2 install on EC2 is to have a local Mageia2 system as your seed setup. Why? Because you must use urpmi, the Mageia package installer. It is equivalent to apt-get in Ubuntu or yum in CentOS/RHEL/Amazon Linux. Yes, you can use rpm to install individual packages (and we will) but urpmi is what talks to the media sets (e.g. repositories) and makes sure you have all your dependencies installed. Besides the man page and the urpmi page on the Mageia wiki, I found a good quick reference guide that helped.
I'll not cover the setup of your seed Mageia2 system here - the installation is was breeze. I used VirtualBox under Windows 7 to install from the Dual-arch ISO CD, but you could likely use any old PC you have laying around and install however you like - USB key, Live CD, whatever.
Also, I'm not doing to document getting the EC2 command line tools working on your seed system. Installing java was simple ("urpmi java" I believe) and getting the API tools and AMI tools installed and configured is well documented by Amazon and plenty of others.
Once you have a functioning Mageia2 system and working EC2 AMI and API tools, then we're ready to begin.
The initial steps we'll be following are a mix between the Mageia chroot install and the official documentation on how to create an EC2 instance-store backed AMI. Another major factor was choosing a kernel. Now any good distribution ships with it's own kernel, and Mageia is no different. And of course you can use your own kernel in EC2. The most efficient way to do this is to use the PV-GRUB AKI provided by AWS to load the kernel that is present on your instance's disk, which is what we'll do.
For the most part, all of this went well after some trial-and-error. However I did run across a few issues:
- Make sure you create a big enough loopback device. I started with 2GB and while it was enough for the base install, it wasn't enough once I started adding other packages later. My docs below use 8GB. The maximum is 10GB.
- Make sure you choose the right PV-GRUB AKI (more on kernels in a moment)
- Use a gzip compressed kernel, not a xz compressed kernel
- Choose the right mirror for urpmi.addmedia (distro.ibiblio.org was extra slow for me - mirrors.kernel.org was much faster)
- Make sure you install the critical packages. Without dhcp-client, you won't get your IP address and without an IP address, you're sunk. Same goes for sshd and sudo.
- At this stage, I didn't pay attention to the ssh key pairs built into the EC2 provisioning system. I baked a new public key of the "mageia" user into the install.
You must choose an AKI with "hd0" in the name if you want a raw or unpartitioned disk image (most images). Choose an AKI with "hd00" in the name if you want an image that has a partition table.Since I am doing a direct mke2fs of the loopback image, it doesn't have a partition table. However, I was using the wrong PV-GRUB AKI, the one for loopbacks with partition tables, resulting in nothing working with error messages that were confusing. I'm sure you could fdisk your loopback image and create partitions if you want, but I didn't see it as necessary as I often use the other ephemeral disks for swap, etc. So, I had to use a "hd0" PV-GRUB to get it to work.
After I got PV-GRUB set the system still wouldn't load. The error message from the system console was:
ERROR Invalid kernel: xc_dom_probe_bzimage_kernel: unknown compression format
In chatting with the very helpful "tmb" from the #mageia IRC channel on freenode, I was able to overcome this issue. The problem is that all Mageia kernels are xz compressed by default which is supported just fine by regular grub. However, the PV-GRUB AKI I was using didn't support xz compressed kernels. tmb provided me with a gzip compressed kernel to bootstrap by first EC2 instance running Mageia2. Thanks again tmb - without your help, I wouldn't have gotten this working!
After the kernel loaded, it was just straightforward trial and error troubleshooting till I was able to login. Here's the sanitized steps I used to get my first Mageia2 instance-store backed EC2 AMI uploaded:
# preparation export PATH=$PATH:/sbin:/usr/sbin # create a working directory mkdir $HOME/ec2 # create ssh public key ssh-keygen -t rsa -f $HOME/ec2/mageia -C "mageia@ec2" -P "" # setup the image dd if=/dev/zero of=mageia2-instance-store-v1.img bs=1M count=8192 # format it mke2fs -F -j $HOME/ec2/mageia2-instance-store-v1.img # everything forward needs to be done as root sudo bash -o vi # mount the image for chroot export MAGEIA_PUB_KEY=$HOME/ec2/mageia.pub export CHRDIR=$HOME/ec2/loop # mount the chroot location mount -o loop $HOME/ec2/mageia2-instance-store-v1.img $CHRDIR # create the minimum devices mkdir $CHRDIR/dev /sbin/makedev $CHRDIR/dev console /sbin/makedev $CHRDIR/dev null /sbin/makedev $CHRDIR/dev zero # setup the minimum filesystems mkdir $CHRDIR/etc cat > $CHRDIR/etc/fstab << EOF /dev/xvda1 / ext3 defaults 1 1 none /dev/pts devpts gid=5,mode=620 0 0 none /dev/shm tmpfs defaults 0 0 none /proc proc defaults 0 0 none /sys sysfs defaults 0 0 EOF # add required /proc filesystem mkdir $CHRDIR/proc mount -t proc none $CHRDIR/proc # choose the best/fastest mirror GET http://mirrors.mageia.org/api/mageia.2.x86_64.list | grep country=US # setup the urpmi media locations in the chroot urpmi.addmedia --distrib --urpmi-root $CHRDIR http://mirrors.kernel.org/mageia/distrib/2/x86_64 # install the minimum packages urpmi --auto --urpmi-root $CHRDIR kernel-server basesystem urpmi locales-en sshd sudo dhcp-client # MASSIVE HACK TIME # # kernel from tmb: # http://tmb.mine.nu/Mageia/2/ec2/ # mkdir tmb pushd tmb curl -O http://tmb.mine.nu.nyud.net/Mageia/2/ec2/kernel-server-3.4.18-1.mga2-1-1.mga2.x86_64.rpm curl -O http://tmb.mine.nu.nyud.net/Mageia/2/ec2/kmod-7-7.mga2.x86_64.rpm popd # install custom kernel rpm --root=$CHRDIR -Uhv tmb/*.rpm # insure the new ramdisk is created properly chroot $CHRDIR cd /boot mkinitrd initrd-3.4.18-server-1.mga2.img.a 3.4.18-server-1.mga2 exit # set the kernel to load on boot cat > $CHRDIR/boot/grub/menu.lst << EOF default=0 timeout=0 title linux root (hd0) kernel /boot/vmlinuz-server ro root=/dev/xvda1 console=hvc0 BOOT_IMAGE=linux-nonfb initrd /boot/initrd-server.img EOF # configure the chroot network for ec2 cat > $CHRDIR/etc/sysconfig/network-scripts/ifcfg-eth0 << EOF DEVICE=eth0 BOOTPROTO=dhcp ONBOOT=yes TYPE=Ethernet USERCTL=yes PEERDNS=yes IPV6INIT=no EOF cat > $CHRDIR/etc/sysconfig/network << EOF NETWORKING=yes CRDA_DOMAIN=US EOF # configure ssh test -f $CHRDIR/etc/ssh/sshd_config.orig || cp -p $CHRDIR/etc/ssh/sshd_config $CHRDIR/etc/ssh/sshd_config.orig cat $CHRDIR/etc/ssh/sshd_config.orig | sed -e 's/^#UseDNS yes/UseDNS no/g' | sed -e 's/^PermitRootLogin no/PermitRootLogin without-password/g' > $CHRDIR/etc/ssh/sshd_config # setup mageia account chroot $CHRDIR /usr/sbin/useradd --create-home --home /home/mageia --shell /bin/bash mageia mkdir --mode=0700 $CHRDIR/home/mageia/.ssh (umask 0077; touch $CHRDIR/home/mageia/.ssh/authorized_keys) cat $MAGEIA_PUB_KEY >> $CHRDIR/home/mageia/.ssh/authorized_keys echo "set -o vi" >> $CHRDIR/home/mageia/.bashrc chown -Rh 500:500 $CHRDIR/home/mageia/.ssh (umask 0227; echo "mageia ALL=(ALL) NOPASSWD:ALL" > $CHRDIR/etc/sudoers.d/mageia) # dismount the chroot umount $CHRDIR/proc umount -d $CHRDIR # setup for EC2 export EC2_ID=[aws account id #] export EC2_PRIVATE_KEY=[location of private key] export EC2_CERT=[location of signing cert] export EC2_ACCESS=[iam access key] export EC2_SECRET=[iam secret key] BUCKETNAME="$EC2_ID-mageia2-instance-store-v1" # create S3 bucket # you can use the AWS Console instead of s3cmd, if you like s3cmd mb s3://$BUCKETNAME # where to put the bundle parts pushd $HOME/ec2 mkdir mageia2-instance-store-v1 # create the AMI bundle # AKI for pv-grub-hd0_1.03-x86_64.gz partitionless PV-GRUB in US-East-1 AKIID="aki-88aa75e1" ec2-bundle-image -i mageia2-instance-store-v1.img -d mageia2-instance-store-v1 -r x86_64 --kernel $AKIID -k $EC2_PRIVATE_KEY -c $EC2_CERT -u $EC2_ID # put it on S3 ec2-upload-bundle -b $BUCKETNAME -m mageia2-instance-store-v1/mageia2-instance-store-v1.img.manifest.xml -a $EC2_ACCESS -s $EC2_SECRET # register it ec2-register $BUCKETNAME/mageia2-instance-store-v1.img.manifest.xml -n mageia2-instance-store-v1 popd