Chef: Spin-up and bootstrap a node on AWS

A few quick instructions for spinning up a node on AWS using Chef and the Knife EC2 plugin.
Some of these steps I culled together from various sites and thought it might be helpful to have them all in one spot.

Base box: This is running from an instance I stood up on AWS (redhat)

Prep work:

DO: yum groupinstall Development tools
DO: Install Chef Client: curl -L https://www.opscode.com/chef/install.sh | bash
DO: Download your starter kit from your Chef Hosted account and unzip in /opt/chef/ (assuming you have a hosted account already setup there)
DO: copy your AWS key file (.pem) to your /home/ec2-user/.ssh/ folder so your workstation can connect back to AWS to spin up new instances. (There may be a different or better way of doing this but it worked for me)

 

Install the Knife EC2 plugin…
sudo /opt/chef/embedded/bin/gem install knife-ec2

Configure the knife.rb for AWS…(knife.rb is located at something like… /opt/chef/chef-repo/.chef/knife.rb)

Knife File
____________________________________________________________________________________________

current_dir = File.dirname(__FILE__)
log_level :info
log_location STDOUT
node_name "YOUR_USERNAME"
client_key "#{current_dir}/YOUR_USERNAME.pem"
validation_client_name "YOUR_ORGNAME-validator"
validation_key "#{current_dir}/YOUR_ORGNAME-validator.pem"
chef_server_url "https://api.opscode.com/organizations/YOUR_ORGNAME"
cache_type 'BasicFile'
cache_options( :path => "#{ENV['HOME']}/.chef/checksums" )
cookbook_path ["#{current_dir}/../cookbooks"]

#A note about where to find this info…
#You will need to log into your AWS IAM (https://console.aws.amazon.com/iam) Go to users, click your #username and you should see an area that says “Access Credentials”. You will likely need to create new #ones if you didnt already know what your keys were at this point. Do so and then make sure you download #them and keep in a safe spot. Fill out the next two lines…the rest is optional at this point but #useful to know that you can have this here.

knife[:aws_access_key_id] = “AWS_ACCESS_KEY_ID”
knife[:aws_secret_access_key] = “AWS_SECRET_ACCESS_KEY”

# Default flavor of server (m1.small, c1.medium, etc).
knife[:flavor] = “m1.small”

# Default AMI identifier, e.g. ami-12345678
#knife[:image] = “ami-b06a98d8”

# AWS Region
#knife[:region] = “us-east-1”

# AWS Availability Zone. Must be in the same Region.
#knife[:availability_zone] = “us-east-1b”

# A file with EC2 User Data to provision the instance.
#knife[:aws_user_data] = “”

# AWS SSH Keypair.
##knife[:aws_ssh_key_id] = “Name of the keypair you want to use in AWS…Instance> Keypair”
___________________________________________________________________________________________

 

Final Command

Finally run this command to connect to AWS and start up your instance and install the recipe:
sudo knife ec2 server create -x ec2-user -S *KEY_NAME* -G default,www -r 'recipe[apache2]' -I ami-b06a98d8

-x = user that we’ll connect with SSH
-S = AWS SSH keypair
-G = Security groups
-r = recipes to run once the instance is up
-I = AMI Identifier e.g. ami-12345678

(This is assuming you already know how to store your cookbooks locally and upload them to your Chef server. )

Once this completes you should have a new instance on AWS with a working apache2 instance and should be able to navigate to the webpage you created for Apache. A separate post will be coming soon to cover these steps.

 

Possible Error

You could possibly run into this:
[fog][WARNING] Unable to load the ‘unf’ gem. Your AWS strings may not be properly encoded.
ERROR: Excon::Errors::SocketError: getaddrinfo: Name or service not known (SocketError)

If so install UNF via gem like so:
sudo /opt/chef/embedded/bin/gem install unf
Try your create command again…

Once you are finished playing around you can use Knife EC2 to actually delete the AWS instance and also remove the node from your hosted Chef management Console:
sudo knife ec2 server delete -y –purge INSTANCE_IDENTIFIER
sudo knife ec2 server delete -y --purge i-e639ca0d

 

Advertisements

Chef Error while bootstrapping

Ive run into this error a few times while getting my VMs setup and Chef installed on them:

192.168.1.153 Starting Chef Client, version 11.6.0
192.168.1.153 Creating a new client identity for target3 using the validator key.
192.168.1.153 ================================================================================
192.168.1.153 Chef encountered an error attempting to create the client "target3"
192.168.1.153 ================================================================================
192.168.1.153 Authorization Error:
192.168.1.153 Your validation client is not authorized to create the client for this node (HTTP 403).
192.168.1.153 Possible Causes:
192.168.1.153 * There may already be a client named "target3"
192.168.1.153 * Your validation client (sethlearningchef-validator) may have misconfigured authorization permissions.
192.168.1.153 [2013-11-18T14:07:52-05:00] FATAL: Stacktrace dumped to /var/chef/cache/chef-stacktrace.out
192.168.1.153 Chef Client failed. 0 resources updated
192.168.1.153 [2013-11-18T14:07:52-05:00] FATAL: Chef::Exceptions::ChildConvergeError: Chef run process exited unsuccessfully (exit code 1)

Im not sure what causes it but here is the fix that works for me:

On management station:

knife client delete NODENAME
knife node delete NODENAME

On affected node:

sudo rm /etc/chef/client.pem
sudo chef-client

Then on your managment server so your bootstrap command again:

sudo knife bootstrap 192.168.1.153 --sudo -p USERNAME -N "NODENAME"