Allan Feid - Ubuntu

Using Amazon's CloudFormation, cloud-init, chef and fog to automate infrastructure

Allan Feid — Thu, 03 Mar 2011 02:08:59 +0000

Amazon recently announced their new CloudFormation API. This allows you to create what they call "stacks," which will bring up and provision various AWS resources. You are able to configure security groups, EC2 instances, RDS instances, and elastic load balancers just to name a few. With this level of support and ease of declaration, it makes you wonder how much longer things like RightScale are going to last.

In this post, I'm going to cover a couple different topics. Since there's a lot going on here, I may be brief to explain certain aspects. Hopefully I do a good job of getting the basic idea down, but just in case, here's a quick overview of what I'm attempting to accomplish:

create EC2 user data that can be used with cloud-init
have the user data kick off chef-solo to configure an instance
setup a simple CloudFormation template for easy launching
use fog to launch a CloudFormation template

EC2 user data and cloud-init

User data can be associated with EC2 instances on boot. The data can really be any arbitrary value. It is up to you to handle what is done with this data, however, the Ubuntu developers came up with a pretty cool concept: cloud-init. With the cloud-init package, you can format your user data to their specs, and have the instance perform various actions. This becomes extremely powerful. One of the most widely used features, is to setup a User-Data Script, which in cloud-init, means starting your userdata with "#!", the shebang. A user-data script will run around the same time as rc.local in the boot process making it a great place to perform service configuration.

What if you want more control, rather than relying on scripts? Some of the really cool parts of cloud-init include being able to install packages, specifying that you want apt-get to update, and even upgrade the system on boot. With the multi-part support, you use mime-types to combine multiple files together. This allows you to have both a user-script and a cloud-config portion of your EC2 user data, amongst other things.

With knowledge of cloud-init's user-scripts in mind, I started off with this basic script:

#!/bin/bash
log='/tmp/init.log'
apt-get update &>> $log
apt-get install -y ruby ruby1.8-dev build-essential wget libruby-extras libruby1.8-extras git-core &>> $log
cd /tmp
wget http://production.cf.rubygems.org/rubygems/rubygems-1.3.7.tgz &>> $log
tar zxf rubygems-1.3.7.tgz &>> $log
cd rubygems-1.3.7
ruby setup.rb --no-format-executable &>> $log
gem install ohai chef --no-rdoc --no-ri --verbose &>> $log
mkdir -p /var/chef/cache
git clone -b deploy git://github.com/crazed/cookbooks.git /var/chef/cookbooks &>> $log
mkdir /etc/chef
cat << EOF > /etc/chef/solo.rb
file_cache_path "/var/chef/cache"
cookbook_path "/var/chef/cookbooks"
json_attribs "/etc/chef/node.json"
log_location "/var/chef/solo.log"
verbose_logging true
EOF
cat << EOF > /etc/chef/node.json
{
  "www": {
    "document_root": "/srv/http",
    "server_name": "localhost"
  },
  "mysql": {
    "user": "user",
    "password": "user",
    "hostname": "localhost",
    "database": "drupal",
    "rootpw": "root"
  },
  "drupal": {
    "version": "7.0",
    "modules": [ "pathauto", "token" ]
  },
  "run_list": [ 
    "recipe[php-fpm]",
    "recipe[mysql::new_database]",
    "recipe[drupal::install]",
    "recipe[drupal::nginx-config]"
  ]
}
EOF
chef-solo &>> $log

This was cool and all, but I wanted to really separate my node attributes and solo configuration from the user-script. So I stumbled upon the part-handler portion of cloud-init. This allows you to write a bit of python to handle additional mime-types inside the EC2 user data. Here's a very basic chunk that adds support for text/chef-attributes and text/chef-solo mime types:

#part-handler
# vi: syntax=python ts=4

def list_types():
    # return a list of mime-types that are handled by this module
    return(["text/chef-attributes", "text/chef-solo"])

def handle_part(data,ctype,filename,payload):
    # data: the cloudinit object
    # ctype: '__begin__', '__end__', or the specific mime-type of the part
    # filename: the filename for the part, or dynamically generated part if
    #           no filename is given attribute is present
    # payload: the content of the part (empty for begin or end)
    if ctype == "__begin__":
       print "my handler is beginning"
       return
    if ctype == "__end__":
       print "my handler is ending"
       return

    if ctype == "text/chef-attributes":
        print "handling %s " % ctype
        import os
        attribs = '/etc/chef/node.json'
        d = os.path.dirname(attribs)
        if not os.path.exists(d):
            os.makedirs(d)
        f = open(attribs, 'w')
        f.write(payload)
        f.close()
        return

    if ctype == "text/chef-solo":
        print "handling %s" % ctype
        import os
        solo = '/etc/chef/solo.rb'
        d = os.path.dirname(solo)
        if not os.path.exists(d):
            os.makedirs(d)
        f = open(solo, 'w')
        f.write(payload)
        f.close()
        return

There's probably more effecient ways of doing the above, but this is all still a work in progress for me. Either way, continuing with the original script, I can also pull out the package installation and add this to a cloud-config portion of my user data using the following:

#cloud-config
packages:
- ruby
- ruby1.8-dev
- build-essential
- wget
- libruby-extras
- libruby1.8-extras
- git-core

Note the #cloud-config line at the begining. This tells cloud-init to handle data in the file as part of the cloud-config specification. Now that I have a cloud-config portion installing some required packages, and a part-handler to handle additional mime-types, I can put my chef node attributes and chef solo configuration into their own files.

# cat node.json
{
  "www": {
    "document_root": "/srv/http",
    "server_name": "localhost"
  },
  "mysql": {
    "user": "user",
    "password": "user",
    "hostname": "localhost",
    "database": "drupal",
    "rootpw": "root"
  },
  "drupal": {
    "version": "7.0",
    "modules": [ "pathauto", "token" ]
  },
  "run_list": [ 
    "recipe[php-fpm]",
    "recipe[mysql::new_database]",
    "recipe[drupal::install]",
    "recipe[drupal::nginx-config]"
  ]
}

# cat solo.rb
file_cache_path "/var/chef/cache"
cookbook_path "/var/chef/cookbooks"
json_attribs "/etc/chef/node.json"
log_location "/var/chef/solo.log"
verbose_logging true

I'm not going to go over the node.json part all that much, but basically my chef cookbooks will read this data in order to determine how to configure my instance. I want to set my http document root to /srv/http, setup a new mysql database, and install drupal 7.0 with the pathauto, and token modules. The solo configuration points out where I will be storing my cookbooks, which my stripped down user-script will be checking out from my github repository:

#!/bin/bash
log='/var/log/cloud-init.log'
cd /tmp
wget http://production.cf.rubygems.org/rubygems/rubygems-1.3.7.tgz &>> $log
tar zxf rubygems-1.3.7.tgz &>> $log
cd rubygems-1.3.7
ruby setup.rb --no-format-executable &>> $log
mkdir -p /var/chef/cache
git clone -b deploy git://github.com/crazed/cookbooks.git /var/chef/cookbooks &>> $log
gem install ohai chef --no-rdoc --no-ri --verbose &>> $log
chef-solo &>> $log

Now that I have all these files, I need to combine them using mime-multiparts. There's a script availabe in the cloud-utils package, but you won't really need to install this on your workstation/laptop. So just use bzr to checkout the latest version:

# bzr branch lp:ubuntu/cloud-utils
# cd cloud-utils
# ./write-mime-multipart --output=combined-userdata.txt \
part-handler.py:text/part-handler \
node.json:text/chef-attributes \
solo.rb:text/chef-solo \
cloud-config.txt \
user-script.sh

Note that I don't specify the mime-types for user-script.sh or cloud-config.txt, as they begin with the appropriate lines which cause cloud-init to handle them properly.

CloudFormation templates

So now that you have this slick user-data file, you're going to want to put more than one of these together to create a full infrastructure. This is where the CloudFormation "stacks" come into play. They are JSON based templates. This example will create you an EC2 Security Group and an EC2 instance.

{
  "AWSTemplateFormatVersion" : "2010-09-09",

  "Description": "Create a fully functioning Drupal installation." 
  "Parameters": {
    "InstanceType": {
      "Description": "Type of EC2 instance to launch",
      "Type": "String",
      "Default": "m1.small"
    },
    "WebServerPort": {
      "Description": "TCP/IP port fo the webserver",
      "Type": "String",
      "Default": "80"
    },
    "KeyName": {
      "Description": "Name of an existing EC2 KeyPair to enable SSH access to the instances",
      "Type": "String"
    }
  },

  "Mappings": {
    "AWSInstanceType2Arch": {
      "t1.micro"    : { "Arch": "64" },
      "m1.small"    : { "Arch" : "32" },
      "m1.large"    : { "Arch" : "64" },
      "m1.xlarge"   : { "Arch" : "64" },
      "m2.xlarge"   : { "Arch" : "64" },
      "m2.2xlarge"  : { "Arch" : "64" },
      "m2.4xlarge"  : { "Arch" : "64" },
      "c1.medium"   : { "Arch" : "32" },
      "c1.xlarge"   : { "Arch" : "64" },
      "cc1.4xlarge" : { "Arch" : "64" } 
    }, 
    "AWSRegionArch2AMI": {
      "us-east-1": { "32": "ami-3e02f257", "64": "ami-3202f25b" }
    }
  },

  "Resources": {
    "web1": {
      "Type": "AWS::EC2::Instance",
      "Properties": {
        "InstanceType": { "Ref": "InstanceType" },
        "SecurityGroups": [ { "Ref": "web" }, "default"  ],
        "KeyName": { "Ref": "KeyName" },
        "ImageId": { "Fn::FindInMap" : [ "AWSRegionArch2AMI", { "Ref": "AWS::Region" },
                            { "Fn::FindInMap": [ "AWSInstanceType2Arch", { "Ref": "InstanceType" }, "Arch" ] } ] },
        "UserData": { "Fn::Base64": " SEE BELOW " }
      }
    },

    "web": {
      "Type": "AWS::EC2::SecurityGroup",
      "Properties": {
        "GroupDescription": "Web security group, allows port 80 in",
        "SecurityGroupIngress": [ {
          "IpProtocol": "tcp",
          "FromPort": "22",
          "ToPort": "22",
          "CidrIp": "0.0.0.0/0"
        },
        {
          "IpProtocol": "tcp",
          "FromPort": { "Ref": "WebServerPort" },
          "ToPort": { "Ref": "WebServerPort" },
          "CidrIp": "0.0.0.0/0"
        } ]
      }
    }
  }

}

Taking a look at the template, you'll see it gets a bit messy in the user data portion of my instance. Essentially you need to make sure your userdata is compliant with the JSON format. To do that, I used this simple hackish ruby script:

# cat json_encode.rb
#!/usr/bin/ruby
require 'rubygems'
require 'json'

if ARGV.size != 1
  puts "Usage: #{$0} <file>"
  exit 1
end

def escape(string)
  parse = JSON.parse({ 'json' => string }.to_json)
  parse['json']
end

data = ''
File.open(ARGV[0]) {|f| data << f.read}
p escape(data)

# ./json_encode.rb combined-userdata.txt
...

Manipulating stacks with fog

Now you have a stack template that's fairly powerful. You get to configure a few things, such as what EC2 keypair to use and what instance type you want for the stack. You can even configure what web port will be opened on security group. How do you actually use this data? While Amazon provides command line api tools, I decided to add CloudFormation support to the fog gem and use this to create a basic tool for myself. Realistically, you could take fog and build a nice interface to launch or build your own templates using your own chef cookbooks/node attributes. As of this writing, CloudFormation is not included in the released fog gem, so you're going to have to checkout the latest code from github:

# mkdir projects && cd projects
# git clone git://github.com/geemus/fog.git

With everything in place, here's a simple script that will allow you to launch a stack:

#!/usr/bin/env ruby
require 'optparse'
require "#{ENV['HOME']}/projects/fog/lib/fog"

options = {}
parser = OptionParser.new do |opts|
  opts.banner = "Usage: #{$0} -n STACK_NAME -t TEMPLATE_FILE [options]"

  options[:name] = nil
  opts.on('-n', '--name NAME', 'Required name attribute') do |name|
    options[:name] = name
  end

  options[:parameters] = {}
  opts.on('-p', '--parameters KEY=VALUE', 'Set the parameters for a stack template', 
    '(can be comma separated key value pairs)') do |params|
    params.split(',').each do |param|
      k,v = param.split('=')
      raise "Invalid parameter definition" unless v
      options[:parameters][k] = v
    end
  end

  options[:template] = nil
  opts.on('-t', '--template FILE', 'Set the template file for this stack') do |file|
    options[:template] = file
  end
end

parser.parse!
[:name, :template].each do |p|
  raise "Missing required parameter: --#{p}" unless options[p]
end

config = YAML::load(File.open(ENV['HOME']+'/.fog'))[:default]
template_body = ''
File.open(options[:template]) {|f| template_body << f.read}

cf = Fog::AWS::CloudFormation.new(
  :aws_access_key_id => config[:aws_access_key_id],
  :aws_secret_access_key => config[:aws_secret_access_key]
)

def aws_params(hash)
  r = Hash.new
  c = 1
  hash.each do |k,v|
    r["Parameters.member.#{c}.ParameterKey"] = k
    r["Parameters.member.#{c}.ParameterValue"] = v
    c += 1
  end
  r
end
cf.create_stack(options[:name], {'TemplateBody' => template_body}.merge!(aws_params(options[:parameters])))

You can use it as follows:

# ./create-stack.rb --name drupal --parameters KeyName=allan,InstanceType=m1.large --template drupal_dev.template

This will upload your template definition and have Amazon read and launch the various resources that you have defined. As these resources are launched, you can get status messages using the DescribeStackEvents API call. Again, here's a simple script that's using fog to pull down these messages:

#!/usr/bin/env ruby
require 'rubygems'
require 'yaml'
require "#{ENV['HOME']}/projects/fog/lib/fog"

if ARGV.size != 1
  puts "Usage: #{$0} <stack name>"
  exit 1
end

config = YAML::load(File.open(ENV['HOME']+'/.fog'))[:default]
cf = Fog::AWS::CloudFormation.new(
  :aws_access_key_id => config[:aws_access_key_id],
  :aws_secret_access_key => config[:aws_secret_access_key]
)

events = cf.describe_stack_events('StackName' => ARGV[0]).body['Events']
events.each do |event|
  puts "Timestamp: #{event['Timestamp']}"
  puts "LogicalResourceId: #{event['LogicalResourceId']}"
  puts "ResourceType: #{event['ResourceType']}"
  puts "ResourceStatus: #{event['ResourceStatus']}"
  puts "ResourceStatusReason: #{event['ResourceStatusReason']}" if event['ResourceStatusReason']
  puts "--"
end

And what the output will look like:

# ./describe_stack_events.rb drupal
Timestamp: 2011-03-02T17:13:08Z
LogicalResourceId: drupal
ResourceType: AWS::CloudFormation::Stack
ResourceStatus: CREATE_COMPLETE
--
Timestamp: 2011-03-02T17:13:05Z
LogicalResourceId: web1
ResourceType: AWS::EC2::Instance
ResourceStatus: CREATE_COMPLETE
--
Timestamp: 2011-03-02T17:12:27Z
LogicalResourceId: web1
ResourceType: AWS::EC2::Instance
ResourceStatus: CREATE_IN_PROGRESS
--
Timestamp: 2011-03-02T17:12:27Z
LogicalResourceId: web
ResourceType: AWS::EC2::SecurityGroup
ResourceStatus: CREATE_COMPLETE
--
Timestamp: 2011-03-02T17:12:23Z
LogicalResourceId: web
ResourceType: AWS::EC2::SecurityGroup
ResourceStatus: CREATE_IN_PROGRESS
--
Timestamp: 2011-03-02T17:12:21Z
LogicalResourceId: drupal
ResourceType: AWS::CloudFormation::Stack
ResourceStatus: CREATE_IN_PROGRESS
ResourceStatusReason: User Initiated
--

Again you could simply use the provided Amazon tools, as they are probaly more feature-complete, but sometimes you want to be able to query for specific bits of data. Learning how to use fog will save you tons of time compared to parsing data that comes out of the Amazon tools. Fog really lets you do some powerful things with the Amazon APIs and is always gaining more features.

Current shortcommings and summary

This setup has a few issues that I currently don't know how to solve. Let's say you were to launch a stack that included two EC2 instances, a load balancer, and an RDS instance. During the creation phase, you want to import a basic DB schema into your RDS instance. Since you can't get shell on the RDS instance, you will have to use one of your EC2 instances to import the schema. You have a race condition here, how do you make sure your RDS instance is available before your EC2 instance attemps to import data? Right now the only thing I can think of is to have your chef recipe gracefully handle retries. It seems we need a way for resources of a stack to depend on other resources before being launched, or a notification system that could potentially kick off the chef run (SNS?).

Either way, I think CloudFormation, chef and fog can be used in conjuction with cloud-init to create quick deployments that are OS independent. This allows you to maintain your automation portions separately from your images, which as a consequence, promotes the use of vanilla images. With these basic ideas, you can easily swap out the node.json file for your own, and replace the user-script in such a way that it checks out your own cookbooks. The possibilities are really endless. Currently cloud-init supports Puppet out of the box, and I'd like to see some chef support here as well.

Recovering GRUB after accidentally writing to MBR

Allan Feid — Wed, 25 Feb 2009 03:58:33 +0000

The other day I decided to partition out a tiny bit of my hard drive (10 GB) to install Windows XP. I figured it was good to have around to play games in my spare time. No big deal, the install went fine. Then I realized Windows likes to take over the MBR. Grub was no longer my default boot loader. So in order to boot back into Ubuntu I needed a live CD. Honestly, any live CD should work fine for this (try Ubuntu's if you don't have a preference).

Once in you're in your live environment, you need to go into grub's console with the following command:

$ grub

You should then have a prompt as such:

grub>

If you're unsure of which partition you have your boot information you can use a quick find command like this:

grub> find /boot/grub/stage1

This should return something similar to (hd0,2). Use that for the following two commands.

grub> root (hd0,2)
grub> setup (hd0)

You should be good to go at this point. If you want to boot back into Windows you will stil need to add that to your /boot/grub/menu.lst and add the following all the way at the end.

title Windows
root (hd0,1)
makeactive
chainloader +1

This time replace (hd0,1) with the location of your Windows partition.

For further information you can take a look at the Ubuntu Wiki for Recovering Ubuntu After Installing Windows.

Avahi daemon ruins .local domains

Allan Feid — Fri, 13 Feb 2009 20:47:14 +0000

This has been known for quite some time. Avahi is used service discovery like finding printers. It's similar to Mac OS X's Bonjour service. Unfortunately, when you have a .local domain (which I do) it seems to mess with things. For example, I cannot refer to anything in the .local TLD by name. So to fix it I've been just stopping the service.

sudo /etc/init.d/avahi-daemon stop

This works but isn't a permanent solution. To fully remove Avahi from startup use the update-rc.d command.

$ sudo update-rc.d -f avahi-daemon remove
 Removing any system startup links for /etc/init.d/avahi-daemon ...
   /etc/rc0.d/K86avahi-daemon
   /etc/rc1.d/K86avahi-daemon
   /etc/rc2.d/S14avahi-daemon
   /etc/rc3.d/S14avahi-daemon
   /etc/rc4.d/S14avahi-daemon
   /etc/rc5.d/S14avahi-daemon
   /etc/rc6.d/K86avahi-daemon

This removes the script from the various run time levels and makes my life easier. For more information about this issue take a look at the Avahi and Unicast .local wiki page.

Pandora Desktop on Adobe AIR

Allan Feid — Thu, 18 Dec 2008 19:54:41 +0000

So today Adobe AIR came out of beta for Linux. That is great news because I also came across the Pandora Desktop (currently in beta) in the same day. These two complement each other very well. It is great for removing that one Firefox window or tab that I always have open to Pandora.

Take a look at Pandora Desktop in action:

And here you can see the slick menu accessible from a nice icon in your tray:

Being that this is a beta it is still very useful and I highly recommend you give it a try. The only problem I had was getting the full view Pandora open every now and then. That's not really needed though since you can do pretty much everything through the menu on the icon.

CIFS VFS no response error? Simple fix.

Allan Feid — Tue, 09 Dec 2008 18:26:10 +0000

So pretty much every time I reinstall I create auto-mounts in fstab for my Window's shares using smbfs. One problem that always happens is on shutdown or reboot those shares (using cifs) do not get umounted automatically. I always wind up Google searching and find the same post.

To fix the issue it's quite simple just put a simple script in a few directories and it will detect all your cifs mounts and automatically umount them. The script is written by Max Durden and the original post has these instructions but I'd like to post them here for ease of access for myself.

First create the file called mountcifs

#!/bin/bash
#
# mountcifs - Unmounts samba-cifs filesystems
# -> convenience script to be called in the shutdown/reboot sequence of Ubuntu Dapper 
#    as K02umountcifs
# Written by Max Durden
# max.durden@gmail.com
#

start() {
echo "mountcifs: nothing to do!"
}

stop() {
echo "Unmounting samba-cifs filesystems..."
MOUNTED_CIFS_DIRS=`mount -t cifs| cut -d\  -f3`
MOUNTED_DIRS_ARRAY=($MOUNTED_CIFS_DIRS)
for i in ${!MOUNTED_DIRS_ARRAY[@]}
do
  umount -l ${MOUNTED_DIRS_ARRAY[i]}
done
}

status() {
echo "Mounted samba-cifs filesystems..."
mount -t cifs| cut -d\  -f3
}

restart() {
echo "restart ..."
start
}

reload() {
echo "start ..."
start
}

force_reload() {
echo "force-reload ..."
start
}

case $1 in
start)
start
;;
stop)
stop
;;
status)
status
;;
restart)
restart
;;
reload)
reload
;;
force-reload)
force_reload
;;
*)
echo "Usage: mountcifs {start|stop|restart|reload|force-reload|status}"
;;
esac

exit 0

Save that to /etc/init.d/ (note: you need to be root) then make it executable.

chmod +x mountcifs

Then create a couple symbolic links so it launches on shutdown or reboot.

sudo ln -s /etc/init.d/mountcifs /etc/rc0.d/K02mountcifs
sudo ln -s /etc/init.d/mountcifs /etc/rc6.d/K02mountcifs

That's it. That stupid error that hangs your PC from shutting down will no longer exist.

Eee Tips: Reduce disk writes by utilizing tmpfs

Allan Feid — Tue, 02 Dec 2008 06:07:09 +0000

I've got an ASUS Eee PC 901/Linux with the solid state drive (SSD). I've been having issues with the /tmp dir and /var/log dir utilizing too much of my 4GB drive. So to fix this I came across this eeeuser.com thread which talks about moving those directories to RAM. The process is pretty straight forward and will increase the life of your SSD. Find out what I did below.

Overview
This will move /tmp, /var/tmp, and /var/logs to RAM using tmpfs which means everything will be deleted on a reboot. That is fine since both /tmp and /var/tmp are deleted on reboot anyway. For the logs, however, we can create a couple scripts to backup only the logs from the last session as it's not really necessary to keep tons of logs on a netbook. In case you're wondering, I'm running Ubuntu Eee on my Eee PC.

Edit /etc/fstab
As a quick warning, MAKE BACKUPS!! This file is crucial to the inner workings of Ubuntu. Once you've got a backup all you need to do is add a few lines to move the directories to tmpfs.

tmp     /tmp            tmpfs   noexec,nosuid,rw,size=8192k   0     0
vartmp  /var/tmp        tmpfs   noexec,nosuid,rw,size=1024k   0     0
varlog  /var/log        tmpfs   noexec,nosuid,rw,size=8192k   0     0

If you're wondering why I gave /tmp 8MB in memory it is because Flash videos need a good amount of space and will hang if you only give it 1MB. So if you don't plan on streaming anything through Flash you can lower size=8192k to 1024k.

Create a couple scripts for our log problem
So in order to only keep logs from the last session we'll make a simple script and have it launch on shutdown but before that create the directory /var/log_bkup.

sudo mkdir /var/log_bkup

That's where the tar archive of the logs will be stored. Here's the script to remove the old and bring in the new:

file: /etc/init.d/logbkup.sh

#!/bin/sh
if [ -f /var/log_bkup/log_bkup.tar.gz ]; then
        /bin/rm /var/log_bkup/log_bkup.tar.gz
fi

echo "Backing up logs to tar archive..."
/bin/tar czf /var/log_bkup/log_bkup.tar.gz /var/log

This next script will create a skeleton in /var/log so all the directories already exist to avoid any problems.

file: /etc/init.d/mklogdir.sh

echo "Creating directories in /var/log"
/bin/mkdir /var/log/apparmor
/bin/mkdir /var/log/apt
/bin/mkdir /var/log/cups
/bin/mkdir /var/log/dist-upgrade
/bin/mkdir /var/log/fsck
/bin/mkdir /var/log/gdm
/bin/mkdir /var/log/news
/bin/mkdir /var/log/samba
/bin/mkdir /var/log/unattended-upgrades
/bin/mkdir /var/log/installer

Make sure the scripts are executable

sudo chmod +x /etc/init.d/mklogdir.sh 
sudo chmod +x /etc/init.d/logbkup.sh

Link them to various rc directories so they run at the proper times (startup/shutdown)

sudo ln /etc/init.d/mklogdir.sh /etc/rc2.d/505mklogdir.sh
sudo ln /etc/init.d/logbkup.sh /etc/rc0.d/515logbkup.sh
sudo ln /etc/init.d/logbkup.sh /etc/rc6.d/515logbkup.sh

Finish up
Pretty much done at this point. Just want to remove everything in /tmp and /var/tmp to avoid any issues when remounting on next boot with everything running in tmpfs.

sudo rm -rf /tmp/* /var/tmp/*

Go ahead and reboot. You should now be running a system with a lot less disk writes. All your log writes will now only be written once per session instead of little writes here and there every time something happens.

The upgrade to Ubuntu 8.10 Intrepid

Allan Feid — Fri, 14 Nov 2008 22:11:28 +0000

Well I recently decided to go ahead and upgrade to the latest version even though it is not a Long Term Support (LTS) version. The upgrade itself took me about 3 hours since my internet connection isn't that great and I ran in to a few problems once it booted into the new version.

The first major problem was a flickering screen. My laptops screen seemed to flicker really fast between the actual screen and a black screen. It also moved the mouse to the upper left every time it did this. That made doing anything via UI almost impossible. The first thing I tried was booting into recovery mode and selecting the "xfix" option which will give you a default xorg.conf file. This seemed to fix the problem but then a few minutes later I realized it had just slowed down the flicker. Things were more managable but still extremely annoying. I thought this could possibly be an issue with the driver but switching between open source ATI driver and the proprietary one did not seem to improve/worsen the issue. Then it occured to be it could be the refresh rate acting up.To fxi this you can create a ModeLine to specify your monitors abilities. Mine supports 1680x1050 @ 60Hz so using this XFree86 Modeline generator I came up with the following and put it in my monitor section:

ModeLine   "1680x1050" 149.0 1680 1760 1944 2280 1050 1050 1052 1089

I then needed to tell xorg to use that line by adding the following to my screen section:

SubSection "Display"
    Depth     24
    Modes    "1680x1050"
EndSubSect

After a quick ctrl+alt+backspace to restart X everything seemed to work much better. Now the screen flickering is gone, however, my themes and Compizfusion were not.

In order to solve this problem, I simply right click on the desktop then selected Choose Desktop Background. In the Visual Effects tab I just needed to select the Extra option. Unfortunately, none of my settings were saved so I did have to manually reconfigure Compiz to do what I wanted. I also needed to change the window decorator to Emerald in order to get all of my eye candy working again.

Avant Window Manager or AWN had disappeared and needed to be reinstalled. I needed to reinstall it with the following:

sudo apt-get install avant-window-navigator

After that I had realized none of the applets were installed. This was sort of a pain in 8.04 since they were not included in the repositories. Luckily in 8.10 they were added but you need to install a bunch of packages to get them all working. To get the default applets working you need to type the following:

sudo apt-get install awn-applets-c-core awn-applets-c-extras awn-applets-python-core awn-applets-python-extras libawn-extras-dev libawn-extras0 python-awn-extras python-awnlib

Note that you will need the Universe repository enabled in order to install the applets.

Other Issues

Besides the more important issues I found on first upgrade, VMware Workstation 6.05 does not work anymore. I will have to try upgrading to 6.5 and see if that will work. Also my colection database for Amarok ahd to be rebuilt. This was pretty annoying as I have a large database but not exactly a life threatening issue. I'll be updating this with any other issues I find from the upgrade.

UPDATE: VMware Workstation 6.5 works nicely in 8.10 but does cause some weird keyboard mappings. The arrow keys do not work and there's all kinds of weird stuff going on with the numberpad. This isn't that big of a deal in most cases though.

Get Flash Player running in FF 3.1 Beta 1

Allan Feid — Tue, 04 Nov 2008 00:56:00 +0000

So I've been using FF3.1b1 and have loved it. The speed improvements and cool new alt+tab style tab switcher have really been incredible. GMail has never loaded so quickly for me. The only thing I couldn't get working right away was Flash. So I dug around a bit and realized it was a matter of copying a file into the plugins folder of my beta installation.

Head on over to Adobe's download site and choose the tar file to download. Extract the files anywhere you'd like and simply copy libflashplayer.so to /path/to/firefox/plugins (for me it was ~/firefox_beta/plugins but default installs are to ~/usr/lib/firefox/plugins in Ubuntu and ~/firefox/plugins for most distros). Go ahead and restart Firefox and Flash will now be working properly.

Fix dark theme's and Firefox

Allan Feid — Mon, 29 Sep 2008 03:52:59 +0000

Ah, this is something I've been meaning to address just haven't got around to it. Today I decided I'd look into fixing problems with dark themes and Firefox. Basically, if you're using a dark theme with Ubuntu 8.04 and Firefox 3 a lot of websites will show up with black or dark input boxes and black font inside those. Even some buttons will show up black with black font. This is a huge hassle when it comes to productivity.

To fix the issue, I was able to find a very useful article on Ubuntu's forums. It explains how to edit Firefox's forms.css and the need to install stylish with a few themes. It's pretty straight forward and fixed every dark theme problem that I've had so far.

Read the tutorial

My Desktop

Allan Feid — Wed, 24 Sep 2008 22:40:10 +0000

I'd like to share with you what Ubuntu Linux can look like with some tweaking. This shows that Linux doesn't have to look so boring as many people assume.

Pretty sweet way of switching between desktops is with CompizFusion

A screenshot of what my desktop typically looks like

Here's what the window switcher is like in CompizFuscion

As you can see my desktop looks much better than most default configurations. Many will argue over Gnome vs KDE but I personally prefer Gnome over KDE. It just seems to run more smoothly for me and feels a lot more intuitive. For those interested, here's what a default install of Ubuntu typically looks like:

Allan Feid - Ubuntu

Using Amazon's CloudFormation, cloud-init, chef and fog to automate infrastructure

EC2 user data and cloud-init

CloudFormation templates

Manipulating stacks with fog

Current shortcommings and summary

Tags:

Recovering GRUB after accidentally writing to MBR

Tags:

Avahi daemon ruins .local domains

Tags:

Pandora Desktop on Adobe AIR

Tags:

CIFS VFS no response error? Simple fix.

Tags:

Eee Tips: Reduce disk writes by utilizing tmpfs

Tags:

The upgrade to Ubuntu 8.10 Intrepid

Tags:

Get Flash Player running in FF 3.1 Beta 1

Tags:

Fix dark theme's and Firefox

Tags:

My Desktop

Tags: