Dumping memcached

I needed to see if memcached was getting the values I thought it was getting. Everyone knows about "stats" to see if it is getting activity, but I looked around and found that it is possible to get some of that data out without knowing how your app stores the data. So I wrote a script to do it:

#!/bin/sh
HOST="localhost"
if [ "$1" != "" ]; then HOST=$1; fi
COUNT=100
if [ "$2" != "" ]; then COUNT=$2; fi
for slab in `echo "stats items" | nc $HOST 11211 | grep :number | cut -d: -f2 -`
do
  for item in `echo "stats cachedump $slab $COUNT" | nc $HOST 11211 | grep "^ITEM" | cut -d" " -f2 -`
  do
    echo "get $item" | nc $HOST 11211
  done
done

On Cloud n+1

I spent the last few days setting up an autoscaling pool of servers on the Amazon Elastic Compute Cloud. They really have done an excellent job of putting together a great toolset and documentation. I've made some notes on how to do a basic setup, including using the EC2 Elastic Load Balancer. Another cool tool I was able to use for this project was Ubuntu's pre-built EC2 images and the cloud-init package, making auto-deployment of the servers very easy to do.

# Notes on setting up Amazon AWS Auto Scaling
# ===========================================
# ATonns Tue Oct 26 17:37:12 EDT 2010
#

export AVAILZONE="us-east-1a"
#
# create a launch config
#
export LCNAME="test-lc"
as-create-launch-config $LCNAME \
 --image-id ami-f5e0049c \
 --instance-type m1.small
#
# other key args:
#
# /* security group */
# --group {groupname}
# /* meta-data file */
# --user-data-file {filename}
#

#
# create a load balancer
#
export LBNAME="test-lb"
elb-create-lb $LBNAME --headers \
 --availability-zones $AVAILZONE \
 --listener "protocol=http,lb-port=80,instance-port=80" 
#
# add some thresholds that will kick instances out
#
export LBTESTURI="/DONOTREMOVE.php"
elb-configure-healthcheck $LBNAME --headers \
 --target "HTTP:80$LBTESTURI" \
 --interval 5 \
 --timeout 2 \
 --unhealthy-threshold 2 \
 --healthy-threshold 5

#
# create auto-scale group
#
export ASGROUP="test-asg"
as-create-auto-scaling-group $ASGROUP \
 --availability-zones $AVAILZONE \
 --launch-configuration $LCNAME \
 --min-size 1 \
 --max-size 5 \
 --load-balancers $LBNAME

#
# create a trigger
#
export ASTRIGGER="test-trig"
as-create-or-update-trigger $ASTRIGGER \
 --auto-scaling-group $ASGROUP \
 --period 60 \
 --unit Seconds \
 --dimensions "LoadBalancerName=$LBNAME" \
 --namespace "AWS/ELB" \
 --measure Latency \
 --statistic Average \
 --lower-threshold 0.25 \
 --upper-threshold 0.75 \
 --breach-duration 300 \
 --lower-breach-increment=-1 \
 --upper-breach-increment 1 

#
# more metrics
#
http://goo.gl/A4pAd

------------

#
# remove everything
#
as-delete-trigger $ASTRIGGER --auto-scaling-group $ASGROUP --force
as-update-auto-scaling-group $ASGROUP --min-size 0 --max-size 0
count="-1"
while [ $count -ne 0 ]
do
 count=0
 for i in `as-describe-auto-scaling-groups $ASGROUP --show-long`
 do
  type=`echo $i | cut -d, -f1 -`
  if [ $type = INSTANCE ]
  then
   count=`expr $count + 1`
  fi
 done
 echo $count instances left
done
procs="-1"
while [ $procs -ne 0 ]
do
 procs=0
 for i in `as-describe-scaling-activities $ASGROUP --show-long | cut -d, -f4 -`
 do
  if [ "$i" != "Successful" ]
  then
   procs=`expr $procs + 1`
  fi
 done
 echo $procs processes still running
done
as-delete-auto-scaling-group $ASGROUP --force
as-delete-launch-config $LCNAME --force
elb-delete-lb $LBNAME --force

Enabling color "ls" on Mac OS X

In ~/.bashrc, add:

export CLICOLOR=1

And if you don't have it already, add to ~/.bash_profile:

if [ -f ~/.bashrc ]; then
        source ~/.bashrc
fi

Sometimes, it's the little things that you're used to seeing that make you feel at $HOME.

Setting the Mac OS X hostname

I know you can set the hostname in Preferences > Sharing > Computer Name, but it gets overwritten if you're on a foreign DHCP network that has hostnames assign. To prevent that from happening:

sudo scutil --set HostName {custom_hostname}

Found on this superuser.com thread.

Hugepages and KVM

I've seen the benefits of hugepages before when setting up Oracle and MySQL, but while doing some research I found an article on how to use them with KVM. However, patching /sbin/start_udev and creating an init script to make sure it stays patched just seems like a horrendous idea just to get hugetlbfs mounted on /dev/hugepages. My non-intrusive method is as follows:

1) teach MAKEDEV how to "create the directory" /dev/hugepages on boot. Actually it is creating an additional /dev/null device at /dev/hugepages/null, but it should be harmless to have multiple "null" (major 1, minor 3) devices and also harmless to mount on top of it.

echo 'c $ALLWRITE 1 3 1 1 hugepages/null' > /etc/makedev.d/01hugepages

2) tell udev to create it on boot if needed:

echo 'hugepages/null' > /etc/udev/makedev.d/52-hugepages.nodes

3) tell udev what the right permissions are for it:

echo 'KERNEL=="hugepages*", OWNER="root", GROUP="root", MODE="0775"' > /etc/udev/rules.d/52-hugepages.rules

4) Under CentOS/RHEL run "huge_page_setup_helper.py" to get your hugepages setup

5) Set the hugetlbfs to be mounted on boot:

echo 'hugetlbfs /dev/hugepages hugetlbfs defaults 0 0' >> /etc/fstab

That's it! After a reboot, you can check that hugepages are setup with "sysctl vm.nr_hugepages" and "grep -i huge /proc/meminfo" and check that hugetlbfs is mounted with "mount | grep huge".

Check /proc/meminfo once your KVM guests start to make sure the number of free pages decreases. If not confirm your guest's XML file has "<memoryBacking><hugepages/></memoryBacking>" below the "<currentmemory>" section and that they have "-mem-prealloc -mem-path /dev/hugepages/libvirt/qemu" in the qemu-kvm command line (it should be auto-set by libvirt).

Getting timing out of curl


curl -w " \
time_total %{time_total} \
time_connect %{time_connect} \
time_namelookup %{time_namelookup} \
time_pretransfer %{time_pretransfer} \
time_starttransfer %{time_starttransfer} \
time_redirect %{time_redirect}\n" http://www.example.com


Notes on burning a DVD using Linux

I can't believe I haven't done this before. I guess my desktop has always had a burner and a gui program to do this. Anyway after searching and finding some links, I'm making some notes for next time.

TITLE="012345678901234"
SOURCE_DIR="/path/to/files"
mkisofs -v -A $TITLE -V $TITLE -J -r -o dvd.iso $SOURCE_DIR
eject -t dvd
cdrecord -scanbus dev=ATA # find your DVD burner in the list
cdrecord -v dev=ATA:1,1,0 driveropts=burnfree -dao dvd.iso
eject dvd

I'm sure there's better ways to do this, but it worked pretty well for the first attempt.

rpm queryformat

Another reminder to myself - rpm query to show installed RPMs formatted with name of the file as they were installed (as per the default CentOS/RHEL naming scheme on the install media):

rpm -qa --qf '%{name}-%{version}-%{release}.%{arch}.rpm\n'

Restricting directory in Apache per logged in user

There's some great apache voodoo going on in this blog post. I'm going to basically be doing the same thing for a site I'm working on - which means no more editing custom basic auth blocks within the apache config - just create the user in LDAP and let the front-end developer create the matching directory with FTP.

My code snippet looks like this:

RewriteEngine On
# are they asking for a URL with thier username
RewriteCond %{REMOTE_USER} ^(.*)
RewriteCond %1:$1 !^([^:]+):\1$
# if not, send them there
RewriteRule /ldap-test/([^/]+) /ldap-test/%{REMOTE_USER}/ [R,L]

# is the directory there
RewriteCond /var/www/html/ldap-test/%{REMOTE_USER} !-d
# if not, send them off
RewriteRule /.* http://sorry.example.com/ [R,L]

Apache LDAP Configuration

Examples on how to authenticate a url served by Apache 2.2.x the OpenLDAP way and the Active Directory way

LoadModule authz_ldap_module modules/mod_authz_ldap.so

<IfModule mod_authz_ldap.c>

LDAPSharedCacheSize 500000

LDAPCacheEntries 1024

LDAPCacheTTL 600

LDAPOpCacheEntries 1024

LDAPOpCacheTTL 600

LDAPTrustedMode SSL

LDAPVerifyServerCert Off

<Location /ldap-status>

SetHandler ldap-status

</Location>

<Location /openldap-protected>

Order deny,allow

Allow from all

AuthType Basic

AuthName "authentication required"

AuthBasicProvider ldap

AuthzLDAPAuthoritative on

#

# OpenLDAP way

#

AuthLDAPURL "ldaps://openldapserver.example.com/ou=people,dc=example,dc=com?uid?sub?(objectClass=*)"

AuthLDAPBindDN "cn=Manager,dc=example,dc=com"

AuthLDAPBindPassword "s3cr3tp@55w0rd"

Require valid-user

</Location>

<Location /activedirectory-protected>

Order deny,allow

Allow from all

AuthType Basic

AuthName "authentication required"

AuthBasicProvider ldap

AuthzLDAPAuthoritative on

#

# Active Diretory way

#

AuthLDAPURL "ldaps://activedirectoryserver.example.com:3269/DC=example,DC=com?sAMAccountName?sub?(objectClass=*)"

AuthLDAPBindDN "ldap@example.com"

AuthLDAPBindPassword "s3cr3tp@55w0rd"

Require valid-user

</Location>

</IfModule>

Renaming a cisco access-list

I'm tired of forgetting this, so here's a reminder to myself on how to rename an access-list:

access-list old_acl_id rename new_acl_id

I know this work on ASA 8.2.* and better but doesn't work on IOS 12.2(*)SX.