Dumping memcached

I needed to see if memcached was getting the values I thought it was getting. Everyone knows about "stats" to see if it is getting activity, but I looked around and found that it is possible to get some of that data out without knowing how your app stores the data. So I wrote a script to do it:

if [ "$1" != "" ]; then HOST=$1; fi
if [ "$2" != "" ]; then COUNT=$2; fi
for slab in `echo "stats items" | nc $HOST 11211 | grep :number | cut -d: -f2 -`
for item in `echo "stats cachedump $slab $COUNT" | nc $HOST 11211 | grep "^ITEM" | cut -d" " -f2 -`
echo "get $item" | nc $HOST 11211


On Cloud n+1

I spent the last few days setting up an autoscaling pool of servers on the Amazon Elastic Compute Cloud. They really have done an excellent job of putting together a great toolset and documentation. I've made some notes on how to do a basic setup, including using the EC2 Elastic Load Balancer. Another cool tool I was able to use for this project was Ubuntu's pre-built EC2 images and the cloud-init package, making auto-deployment of the servers very easy to do.

# Notes on setting up Amazon AWS Auto Scaling
# ===========================================
# ATonns Tue Oct 26 17:37:12 EDT 2010

export AVAILZONE="us-east-1a"
# create a launch config
export LCNAME="test-lc"
as-create-launch-config $LCNAME \
--image-id ami-f5e0049c \
--instance-type m1.small
# other key args:
# /* security group */
# --group {groupname}
# /* meta-data file */
# --user-data-file {filename}

# create a load balancer
export LBNAME="test-lb"
elb-create-lb $LBNAME --headers \
--availability-zones $AVAILZONE \
--listener "protocol=http,lb-port=80,instance-port=80"
# add some thresholds that will kick instances out
elb-configure-healthcheck $LBNAME --headers \
--target "HTTP:80$LBTESTURI" \
--interval 5 \
--timeout 2 \
--unhealthy-threshold 2 \
--healthy-threshold 5

# create auto-scale group
export ASGROUP="test-asg"
as-create-auto-scaling-group $ASGROUP \
--availability-zones $AVAILZONE \
--launch-configuration $LCNAME \
--min-size 1 \
--max-size 5 \
--load-balancers $LBNAME

# create a trigger
export ASTRIGGER="test-trig"
as-create-or-update-trigger $ASTRIGGER \
--auto-scaling-group $ASGROUP \
--period 60 \
--unit Seconds \
--dimensions "LoadBalancerName=$LBNAME" \
--namespace "AWS/ELB" \
--measure Latency \
--statistic Average \
--lower-threshold 0.25 \
--upper-threshold 0.75 \
--breach-duration 300 \
--lower-breach-increment=-1 \
--upper-breach-increment 1

# more metrics


# remove everything
as-delete-trigger $ASTRIGGER --auto-scaling-group $ASGROUP --force
as-update-auto-scaling-group $ASGROUP --min-size 0 --max-size 0
while [ $count -ne 0 ]
for i in `as-describe-auto-scaling-groups $ASGROUP --show-long`
type=`echo $i | cut -d, -f1 -`
if [ $type = INSTANCE ]
count=`expr $count + 1`
echo $count instances left
while [ $procs -ne 0 ]
for i in `as-describe-scaling-activities $ASGROUP --show-long | cut -d, -f4 -`
if [ "$i" != "Successful" ]
procs=`expr $procs + 1`
echo $procs processes still running
as-delete-auto-scaling-group $ASGROUP --force
as-delete-launch-config $LCNAME --force
elb-delete-lb $LBNAME --force


Enabling color "ls" on Mac OS X

In ~/.bashrc, add:
export CLICOLOR=1

And if you don't have it already, add to ~/.bash_profile:
if [ -f ~/.bashrc ]; then
source ~/.bashrc

Sometimes, it's the little things that you're used to seeing that make you feel at $HOME.

Setting the Mac OS X hostname

I know you can set the hostname in Preferences > Sharing > Computer Name, but it gets overwritten if you're on a foreign DHCP network that has hostnames assign. To prevent that from happening:

sudo scutil --set HostName {custom_hostname}

Found on this superuser.com thread.


Hugepages and KVM

I've seen the benefits of hugepages before when setting up Oracle and MySQL, but while doing some research I found an article on how to use them with KVM. However, patching /sbin/start_udev and creating an init script to make sure it stays patched just seems like a horrendous idea just to get hugetlbfs mounted on /dev/hugepages. My non-intrusive method is as follows:

1) teach MAKEDEV how to "create the directory" /dev/hugepages on boot. Actually it is creating an additional /dev/null device at /dev/hugepages/null, but it should be harmless to have multiple "null" (major 1, minor 3) devices and also harmless to mount on top of it.

echo 'c $ALLWRITE 1 3 1 1 hugepages/null' > /etc/makedev.d/01hugepages

2) tell udev to create it on boot if needed:

echo 'hugepages/null' > /etc/udev/makedev.d/52-hugepages.nodes

3) tell udev what the right permissions are for it:

echo 'KERNEL=="hugepages*", OWNER="root", GROUP="root", MODE="0775"' > /etc/udev/rules.d/52-hugepages.rules

4) Under CentOS/RHEL run "huge_page_setup_helper.py" to get your hugepages setup

5) Set the hugetlbfs to be mounted on boot:

echo 'hugetlbfs /dev/hugepages hugetlbfs defaults 0 0' >> /etc/fstab

That's it! After a reboot, you can check that hugepages are setup with "sysctl vm.nr_hugepages" and "grep -i huge /proc/meminfo" and check that hugetlbfs is mounted with "mount | grep huge".

Check /proc/meminfo once your KVM guests start to make sure the number of free pages decreases. If not confirm your guest's XML file has "<memoryBacking><hugepages/></memoryBacking>" below the "<currentmemory>" section and that they have "-mem-prealloc -mem-path /dev/hugepages/libvirt/qemu" in the qemu-kvm command line (it should be auto-set by libvirt).


Getting timing out of curl

curl -w " \
time_total %{time_total} \
time_connect %{time_connect} \
time_namelookup %{time_namelookup} \
time_pretransfer %{time_pretransfer} \
time_starttransfer %{time_starttransfer} \
time_redirect %{time_redirect}\n" http://www.example.com


Notes on burning a DVD using Linux

I can't believe I haven't done this before. I guess my desktop has always had a burner and a gui program to do this. Anyway after searching and finding some links, I'm making some notes for next time.

mkisofs -v -A $TITLE -V $TITLE -J -r -o dvd.iso $SOURCE_DIR
eject -t dvd
cdrecord -scanbus dev=ATA # find your DVD burner in the list
cdrecord -v dev=ATA:1,1,0 driveropts=burnfree -dao dvd.iso
eject dvd

I'm sure there's better ways to do this, but it worked pretty well for the first attempt.


rpm queryformat

Another reminder to myself - rpm query to show installed RPMs formatted with name of the file as they were installed (as per the default CentOS/RHEL naming scheme on the install media):

rpm -qa --qf '%{name}-%{version}-%{release}.%{arch}.rpm\n'


Restricting directory in Apache per logged in user

There's some great apache voodoo going on in this blog post. I'm going to basically be doing the same thing for a site I'm working on - which means no more editing custom basic auth blocks within the apache config - just create the user in LDAP and let the front-end developer create the matching directory with FTP.

My code snippet looks like this:

RewriteEngine On
# are they asking for a URL with thier username
RewriteCond %{REMOTE_USER} ^(.*)
RewriteCond %1:$1 !^([^:]+):\1$
# if not, send them there
RewriteRule /ldap-test/([^/]+) /ldap-test/%{REMOTE_USER}/ [R,L]

# is the directory there
RewriteCond /var/www/html/ldap-test/%{REMOTE_USER} !-d
# if not, send them off
RewriteRule /.* http://sorry.example.com/ [R,L]


Apache LDAP Configuration

Examples on how to authenticate a url served by Apache 2.2.x the OpenLDAP way and the Active Directory way

LoadModule authz_ldap_module modules/mod_authz_ldap.so

<IfModule mod_authz_ldap.c>

LDAPSharedCacheSize 500000
LDAPCacheEntries 1024
LDAPCacheTTL 600
LDAPOpCacheEntries 1024
LDAPOpCacheTTL 600
LDAPTrustedMode SSL
LDAPVerifyServerCert Off

<Location /ldap-status>
SetHandler ldap-status

<Location /openldap-protected>
Order deny,allow
Allow from all
AuthType Basic
AuthName "authentication required"
AuthBasicProvider ldap
AuthzLDAPAuthoritative on
# OpenLDAP way
AuthLDAPURL "ldaps://openldapserver.example.com/ou=people,dc=example,dc=com?uid?sub?(objectClass=*)"
AuthLDAPBindDN "cn=Manager,dc=example,dc=com"
AuthLDAPBindPassword "s3cr3tp@55w0rd"
Require valid-user

<Location /activedirectory-protected>
Order deny,allow
Allow from all
AuthType Basic
AuthName "authentication required"
AuthBasicProvider ldap
AuthzLDAPAuthoritative on
# Active Diretory way
AuthLDAPURL "ldaps://activedirectoryserver.example.com:3269/DC=example,DC=com?sAMAccountName?sub?(objectClass=*)"
AuthLDAPBindDN "ldap@example.com"
AuthLDAPBindPassword "s3cr3tp@55w0rd"
Require valid-user



Renaming a cisco access-list

I'm tired of forgetting this, so here's a reminder to myself on how to rename an access-list:

access-list old_acl_id rename new_acl_id

I know this work on ASA 8.2.* and better but doesn't work on IOS 12.2(*)SX.

Ratings and Recommendations by outbrain