Chef: Spin-up and bootstrap a node on AWS

A few quick instructions for spinning up a node on AWS using Chef and the Knife EC2 plugin.
Some of these steps I culled together from various sites and thought it might be helpful to have them all in one spot.

Base box: This is running from an instance I stood up on AWS (redhat)

Prep work:

DO: yum groupinstall Development tools
DO: Install Chef Client: curl -L | bash
DO: Download your starter kit from your Chef Hosted account and unzip in /opt/chef/ (assuming you have a hosted account already setup there)
DO: copy your AWS key file (.pem) to your /home/ec2-user/.ssh/ folder so your workstation can connect back to AWS to spin up new instances. (There may be a different or better way of doing this but it worked for me)


Install the Knife EC2 plugin…
sudo /opt/chef/embedded/bin/gem install knife-ec2

Configure the knife.rb for AWS…(knife.rb is located at something like… /opt/chef/chef-repo/.chef/knife.rb)

Knife File

current_dir = File.dirname(__FILE__)
log_level :info
log_location STDOUT
node_name "YOUR_USERNAME"
client_key "#{current_dir}/YOUR_USERNAME.pem"
validation_client_name "YOUR_ORGNAME-validator"
validation_key "#{current_dir}/YOUR_ORGNAME-validator.pem"
chef_server_url ""
cache_type 'BasicFile'
cache_options( :path => "#{ENV['HOME']}/.chef/checksums" )
cookbook_path ["#{current_dir}/../cookbooks"]

#A note about where to find this info…
#You will need to log into your AWS IAM ( Go to users, click your #username and you should see an area that says “Access Credentials”. You will likely need to create new #ones if you didnt already know what your keys were at this point. Do so and then make sure you download #them and keep in a safe spot. Fill out the next two lines…the rest is optional at this point but #useful to know that you can have this here.

knife[:aws_access_key_id] = “AWS_ACCESS_KEY_ID”
knife[:aws_secret_access_key] = “AWS_SECRET_ACCESS_KEY”

# Default flavor of server (m1.small, c1.medium, etc).
knife[:flavor] = “m1.small”

# Default AMI identifier, e.g. ami-12345678
#knife[:image] = “ami-b06a98d8”

# AWS Region
#knife[:region] = “us-east-1”

# AWS Availability Zone. Must be in the same Region.
#knife[:availability_zone] = “us-east-1b”

# A file with EC2 User Data to provision the instance.
#knife[:aws_user_data] = “”

# AWS SSH Keypair.
##knife[:aws_ssh_key_id] = “Name of the keypair you want to use in AWS…Instance> Keypair”


Final Command

Finally run this command to connect to AWS and start up your instance and install the recipe:
sudo knife ec2 server create -x ec2-user -S *KEY_NAME* -G default,www -r 'recipe[apache2]' -I ami-b06a98d8

-x = user that we’ll connect with SSH
-S = AWS SSH keypair
-G = Security groups
-r = recipes to run once the instance is up
-I = AMI Identifier e.g. ami-12345678

(This is assuming you already know how to store your cookbooks locally and upload them to your Chef server. )

Once this completes you should have a new instance on AWS with a working apache2 instance and should be able to navigate to the webpage you created for Apache. A separate post will be coming soon to cover these steps.


Possible Error

You could possibly run into this:
[fog][WARNING] Unable to load the ‘unf’ gem. Your AWS strings may not be properly encoded.
ERROR: Excon::Errors::SocketError: getaddrinfo: Name or service not known (SocketError)

If so install UNF via gem like so:
sudo /opt/chef/embedded/bin/gem install unf
Try your create command again…

Once you are finished playing around you can use Knife EC2 to actually delete the AWS instance and also remove the node from your hosted Chef management Console:
sudo knife ec2 server delete -y –purge INSTANCE_IDENTIFIER
sudo knife ec2 server delete -y --purge i-e639ca0d