AWS Cloudformation Diagramming Shortcut

A quick little shortcut if you want a visualization of your Cloudformation stack…

  1. Log into the AWS console and go to Cloudformation.
  2. Click DESIGN TEMPLATE
  3. At the bottom of the screen look for the TEMPLATE tab.
  4. Copy your CF template you are working on and paste it in here. Make sure you are overwriting the prefilled JSON thats already there.
  5. In the upper right hand corner look for the Refresh button. It looks like 2 arrows going in a circle.
  6. Now you should see your CF stack visualized. Now you can click the Export button that is near the refresh button you just used.

 

Knife EC2 Server Create Error: Authentication failed

Sometimes with all of the rush and trying to keep track of a 1000 moving parts you might get stumped by a fairly simple issue. Here are a few things to check if you get hung up with an “Authentication failed for user” error when running a “Knife ec2 server create” command.

Waiting for sshd access to become availabledone
Connecting to 52.5.159.42
Failed to authenticate ec2-user - trying password auth
Enter your password:
ERROR: Net::SSH::AuthenticationFailed: Authentication failed for user ec2-user@52.5.159.42@52.5.159.42

Do you have your .pem file downloaded and installed with the correct permissions on your workstation you are running the command from?

It should be in the EC2-USER’s .ssh dir -> /home/ec2-user/.ssh

Make sure its chmodded 400

Make sure you have the knife.rb file set correctly to reference the .pem file for you…otherwise you will have a lot of typing for your command.

knife[:identity_file] = "/home/ec2-user/.ssh/aws-seth.pem"

Make sure you are using the correct user. Unless you have specifically changed something in your configurations by default you will be connecting as the “ec2-user”. So make sure thats what is trying to connect in your error output.

Hopefully these tips will help you narrow down the issue. You have to think about whats really happening and from where with Chef some sometimes these simple issues can really drive you nuts.

AWS CI WORKFLOW PLAN – Part 1

This is a multi-part documentation of how to setup a CI/CD workflow in AWS. Some of the tools that will be used are EC2, Chef, CloudFormation, Jenkins, Rspec, and Apache. If you have questions feel free to leave a comment or track me down on Twitter at @devopshomelab

Part 1 – Workstation Setup – The machine we will work from as a command center

Tools involved:
AWS, Linux, VPC, EC2, ELIP, S3, Chef

Setup VPC and a master instance to work from.
There is a little assumption that you know how to setup your own keys in AWS and also how to SSH using a terminal screen or Putty if you are on Windows.
1. Go to the VPC dashboard and use the wizard to create a new VPC using the “VPC with a Single Public Subnet” option.

2. Check the routing table. Look to see that you have 2 tables and 1 with a subnet.

3. Spin up an instance:
a. I prefer ami-1ecae776 Its RHEL based, free, and is a good medium platform to start with. NEXT
b. Choose t2.Micro NEXT
c. Make sure your network setting is set for your new VPC you created.
d. The subnet should be auto set but make sure it is a subnet of your VPC
e. For Public IP use the option for “Use subnet setting” We will create an elastic IP that you will use to connect to your VPC to administer and run your deployments from for other instances.
f.Shutdown behavior: STOP  and Enable termination protection NEXT
g. Set your size to 30Gb NEXT
h. Name your instance something like Admin or Master NEXT
i. Use an existing security group and chose the one from your new VPC…review and then launch and use your key set.

4. Once your instance has started up go to the Elastic IPs under Network and Seurity on the EC2 dashboard
a. Allocate a new Address and chose EC2
b. Select your ELIP and then click Associate Address
c. Select your instance and then click Associate
d. The Elastic IP is the ip that you will use to SSH to your new instance using your keys and logging in as the ec2-user.

5. Once you have gotten logged in run

sudo yum install update
gem install rspec
yum install git


Setting up Chef

Assumes you already have an account setup with the Management Console at http://manage.chef.io If you are new to Chef check out this link to learn some stuff: https://learn.chef.io/

1. Become root on your new box

2. Run this to install Chef. More info here: https://www.chef.io/download-chef-client/:

curl -L https://www.chef.io/chef/install.sh | bash

3. go to /opt/chef

4. Download your “Starter Kit” from the management console
a. If you are on Windows you can create a S3 bucket in AWS
b. Upload your zip file to your bucket
c. Set your zip file to public and then copy the URL.
d. From your instance do:

wget PASTE YOUR LINK HERE

e. Once it has finished downloading delete the zip from your bucket.
f. Unzip the downloaded file.

5. Go to /opt/chef/chef-repo and do

mv cookbooks cookbooks.old

6. Log into your GitHub account and create a new repo called “cookbooks”. You want to have GitHub initialize for you. Copy the https URL for the new repo.

7. On your box in /opt/chef/chef-repo do

git clone COPIED_URL_FROM_GITHUB

Ex. git clone https://github.com/sethfloydjr/cookbooks.git

8. You should now have a new directory in /opt/chef/chef-repo called cookbooks. Inside it your should have a README.md file if successful.

As prep for the next step lets go ahead and install the AWS Knife-EC2 plugin…

Install the Knife EC2 plugin…

sudo yum group install "Development Tools"
sudo yum group install "Development Libraries"
sudo /opt/chef/embedded/bin/gem install knife-ec2

Configure the knife.rb for AWS…(knife.rb should be located at /opt/chef/chef-repo/.chef/knife.rb)

Knife File
____________________________________________________________________________________________

current_dir = File.dirname(__FILE__)
log_level :info
log_location STDOUT
node_name "YOUR_USERNAME"
client_key "#{current_dir}/YOUR_USERNAME.pem"
validation_client_name "YOUR_ORGNAME-validator"
validation_key "#{current_dir}/YOUR_ORGNAME-validator.pem"
chef_server_url "https://api.opscode.com/organizations/YOUR_ORGNAME"
cache_type 'BasicFile'
cache_options( :path => "#{ENV['HOME']}/.chef/checksums" )
cookbook_path ["#{current_dir}/../cookbooks"]

#A note about where to find this info…
#You will need to log into your AWS IAM (https://console.aws.amazon.com/iam) 
#Go to users, click your #username and you should see an area that says “Access Credentials”. 
#You will likely need to create new #ones if you didnt already know what your keys were at this point.
#Do so and then make sure you download #them and keep in a safe spot. 
#Fill out the next two lines…the rest is optional at this point but useful to know that you can have this here.

knife[:aws_access_key_id] = “AWS_ACCESS_KEY_ID”
knife[:aws_secret_access_key] = “AWS_SECRET_ACCESS_KEY”

# Default flavor of server (m1.small, c1.medium, etc).
knife[:flavor] = “t2.Micro”

# Default AMI identifier, e.g. ami-12345678
#knife[:image] = “ami-1ecae776″

# AWS Region
#knife[:region] = “us-east-1c″

# AWS Availability Zone. Must be in the same Region.
#knife[:availability_zone] = “us-east-1c”

# A file with EC2 User Data to provision the instance.
#knife[:aws_user_data] = “”

# AWS SSH Keypair.
##knife[:aws_ssh_key_id] = “Name of the keypair you want to use in AWS...Instance> Keypair”

 

There’s a lot of steps here and this is a high level instructional for setting this up. Im assuming that if you are reading this you already have some knowledge of AWS, Chef, Linux, and Git. If not and something doesn’t work or you have a deeper question feel free to leave a comment and ill help out if I can.

 

 

Setting up an Apache server using AWS CloudFormation

This a is a brief and simple guide to spinning up an instance, installing Apache, and writing output to a webpage.

Note that by following this guide you can incur charges. Im not responsible for them and will not be picking up your bill.

Setup

1. Have an AWS account. You can signup for one here: http://aws.amazon.com/
2. Have a computer you can work from. This guide assumes you are running Linux. What other OS is there anyways? For me I have an instance on AWS that I startup whenever I want to work on stuff. On this computer you need to install the AWS CLI tool. You can install and setup the CLI tool by going here: http://docs.aws.amazon.com/cli/latest/userguide/cli-chap-getting-set-up.html
3. Basic understanding of JSON…not a requirement but it helps to understand what you are reading in the template.

The Template

First here is a link that will guide you further than im going to here on the ins and outs of templates and requirements for CloudFormation: http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/template-guide.html

So lets break down the template…

{
 "AWSTemplateFormatVersion" : "2010-09-09",

This is an optional but lets AWS know what version of the template you are using. This is NOT today’s date or something.

"Description" : "Create an Apache webserver with a webpage.",

Gives a description of what your template is for.

"Parameters" : {
    "InstanceType" : {
      "Description" : "Type of EC2 instance to launch",
      "Type" : "String",
      "Default" : "t1.micro"
    },
    "WebServerPort" : {
      "Description" : "TCP/IP port of the web server",
      "Type" : "String",
      "Default" : "80"
    },
    "KeyName" : {
      "Description" : "Name of an existing EC2 KeyPair to enable SSH access to the instances",
      "Type" : "AWS::EC2::KeyPair::KeyName",
      "ConstraintDescription" : "must be the name of an existing EC2 KeyPair."
    },
    "SSHLocation": {
            "Description": " The IP address range that can be used to SSH to the EC2 instances",
            "Type": "String",
            "MinLength": "9",
            "MaxLength": "18",
            "Default": "0.0.0.0/0",
            "AllowedPattern": "(\\d{1,3})\\.(\\d{1,3})\\.(\\d{1,3})\\.(\\d{1,3})/(\\d{1,2})",
            "ConstraintDescription": "must be a valid IP CIDR range of the form x.x.x.x/x."
        }
  },

Breaking down the parameters directive…
*If you dont give your parameter a “Default” you will need to pass in the option via the command line. This will be explained further but keep that in mind in this section.
InstanceType – Tells what kind you want to launch. The default here is t1.micro.
WebServerPort – You are telling what port your webserver will accept connections on.
KeyName – This is the name of your AWS key that you use to authenticate with. Use a key that has permissions to setup new instances and connect to them.
SSHLocation – The IP address range that can be used to SSH to the EC2 instances

"Mappings" : {
    "AWSInstanceType2Arch" : {
      "t1.micro"    : { "Arch" : "64" }
    },
    "AWSRegionArch2AMI" : {
      "us-east-1"          : { "64" : "ami-246ed34c" }
    }
  },

Mappings…tells what region and what AMI you want to use. For brevity I already knew what I wanted to use and that I wanted it to fail if not available so my listing of types and regions is VERY short. You can see further info here on mappings: http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/mappings-section-structure.html

Now we will talk about the Resources section. Its where all the real magic happens. Ill break it down some and then provide the full section at the bottom of this page.

"Resources" : {     
      
    "WebServerInstance": {  
      "Type": "AWS::EC2::Instance",
      "Metadata" : {
        "AWS::CloudFormation::Init" : {
          "configSets" : {
            "InstallAndRun" : [ "Install" ]
          },

          "Install" : {
            "packages" : {
              "yum" : {
                "httpd"        : []              
              }
            },

Here, we are saying that we want to initialize and install something on our instance. Notice that we are using “yum”. The AMI I use is an Amazon AMI which is RHEL based. It comes with some tools already setup and installed specific to AWS so its easier to use this AMI base. Here all we are installing is Apache, or “httpd”.

"files" : {
              "/var/www/html/index.html" : {
              "source" : "https://s3.amazonaws.com/BUCKET_NAME/index.html",  
              "mode"  : "000600",
              "owner" : "apache",
              "group" : "apache"
              },

              "/etc/cfn/cfn-hup.conf" : {
                "content" : { "Fn::Join" : ["", [
                  "[main]\n",
                  "stack=", { "Ref" : "AWS::StackId" }, "\n",
                  "region=", { "Ref" : "AWS::Region" }, "\n"
                ]]},
                "mode"    : "000400",
                "owner"   : "root",
                "group"   : "root"
              },

Here we are pulling down our simple html file from our S3 bucket. I could have easily added code here that would have written to the file the HTML code. Its just cleaner looking code wise to simply pull down the file from your bucket. We are also setting the conf file for cfn-hup. That’s a helper daemon that detects changes and reacts when you are making updates to your stack.

"/etc/cfn/hooks.d/cfn-auto-reloader.conf" : {
                "content": { "Fn::Join" : ["", [
                  "[cfn-auto-reloader-hook]\n",
                  "triggers=post.update\n",
                  "path=Resources.WebServerInstance.Metadata.AWS::CloudFormation::Init\n",
                  "action=/opt/aws/bin/cfn-init -v ",
                  "         --stack ", { "Ref" : "AWS::StackName" },
                  "         --resource WebServerInstance ",
                  "         --configsets InstallAndRun ",
                  "         --region ", { "Ref" : "AWS::Region" }, "\n",
                  "runas=root\n"
                ]]}
              }
            },
            "services" : {
              "sysvinit" : {  
                "httpd"   : { "enabled" : "true", "ensureRunning" : "true" },
                "cfn-hup" : { "enabled" : "true", "ensureRunning" : "true",
                              "files" : ["/etc/cfn/cfn-hup.conf", "/etc/cfn/hooks.d/cfn-auto-reloader.conf"]}}}}}
},

More cfn helper to manage changes. Also see that we are enabling and making sure that httpd and cfn-hup are running on startup. This could also be done through scripting placed into the template.

"Properties": {
        "ImageId" : { "Fn::FindInMap" : [ "AWSRegionArch2AMI", { "Ref" : "AWS::Region" },
                          { "Fn::FindInMap" : [ "AWSInstanceType2Arch", { "Ref" : "InstanceType" }, "Arch" ] } ] },
        "InstanceType"   : { "Ref" : "InstanceType" },
        "SecurityGroups" : [ {"Ref" : "WebServerSecurityGroup"} ],
        "KeyName"        : { "Ref" : "KeyName" },
        "UserData"       : { "Fn::Base64" : { "Fn::Join" : ["", [
             "#!/bin/bash -xe\n",
             "yum update -y aws-cfn-bootstrap\n",

             "# Install the files and packages from the metadata\n",
             "/opt/aws/bin/cfn-init -v ",
             "         --stack ", { "Ref" : "AWS::StackName" },
             "         --resource WebServerInstance ",
             "         --configsets InstallAndRun ",
             "         --region ", { "Ref" : "AWS::Region" }, "\n",

             "# Signal the status from cfn-init\n",
             "/opt/aws/bin/cfn-signal -e $? ",
             "         --stack ", { "Ref" : "AWS::StackName" },
             "         --resource WebServerInstance ",
             "         --region ", { "Ref" : "AWS::Region" }, "\n"]]}}}
},

Properties for the instance…Calling on the Mappings info and the yet to be mentioned SecurityGroup. This is also where the packages are installed and configured as you can see the scripting here. Also note that this has all been part of “WebServerInstance”. We are just ending that section.

"WebServerSecurityGroup" : {
      "Type" : "AWS::EC2::SecurityGroup",
      "Properties" : {
        "GroupDescription" : "Enable HTTP access via port 80",
        "SecurityGroupIngress" : [
          {"IpProtocol" : "tcp", "FromPort" : "80", "ToPort" : "80", "CidrIp" : "0.0.0.0/0"},
          {"IpProtocol" : "tcp", "FromPort" : "22", "ToPort" : "22", "CidrIp" : { "Ref" : "SSHLocation"}}
        ]
      }
    }
  },

This sets up the security group and allows access to ports 22 and 80 for http and ssh traffic.

"Outputs" : {
    "WebsiteURL" : {
      "Description" : "URL for newly created Apache server",
      "Value" : { "Fn::Join" : ["", ["http://", { "Fn::GetAtt" : [ "WebServerInstance", "PublicDnsName" ]}]] }
    }
  }
}

This section will output the info you need to know the URL of your new instance. You will look in the Output section of your CloudFormation stack to see the info which will be a link you can click to see your webpage.

AWS Command

So now you have your AWS CLI tool installed and you have your template written…now its time to fire up your stack and see the fruits of your labor.

The command you will want to run is:

aws cloudformation create-stack --stack-name STACK_NAME --template-url https://s3.amazonaws.com/BUCKET_NAME/template.json --parameters  ParameterKey=KeyName,ParameterValue=YOUR_KEY_NAME

This declares that you are using cloudformation to create a new stack, you are giving the stack name, location of where you will download and read the template from, and finally filling in the parameter of your AWS key.
*Remember earlier when I said that if you do not have a “default” for a parameter that you will need to provide it? This is where I was talking about. For the KeyName param we did not provide a default keyname to use. You can pass a number of parameters on the command line as we have here up to a maximum of 60. Really at that point you should be looking at a scripted option to provide those right?

So once you have executed the above command you can look in your CloudFormation and see your stack being created. If it fails it will stop and rollback the creation. Once it completes go to your Output tab and you should see the URL of your webpage. Click it and you should see whatever you put in your index.html file that is in your S3 bucket.

That’s about it. The best way to learn this is to play around with it and read through the CloudFormation documentation. Let me know if you have comments or questions.

 

Full template for you to copy and paste:

Make sure you use a json validator to be sure that the copy and paste didn’t mangle anything.
{
  "AWSTemplateFormatVersion" : "2010-09-09",

  "Description" : "Create an Apache webserver with a webpage.",

  "Parameters" : {
    "InstanceType" : {
      "Description" : "Type of EC2 instance to launch",
      "Type" : "String",
      "Default" : "t1.micro"
    },
    "WebServerPort" : {
      "Description" : "TCP/IP port of the web server",
      "Type" : "String",
      "Default" : "80"
    },
    "KeyName" : {
      "Description" : "Name of an existing EC2 KeyPair to enable SSH access to the instances",
      "Type" : "AWS::EC2::KeyPair::KeyName",
      "ConstraintDescription" : "must be the name of an existing EC2 KeyPair."
    },
    "SSHLocation": {
            "Description": " The IP address range that can be used to SSH to the EC2 instances",
            "Type": "String",
            "MinLength": "9",
            "MaxLength": "18",
            "Default": "0.0.0.0/0",
            "AllowedPattern": "(\\d{1,3})\\.(\\d{1,3})\\.(\\d{1,3})\\.(\\d{1,3})/(\\d{1,2})",
            "ConstraintDescription": "must be a valid IP CIDR range of the form x.x.x.x/x."
        }
  },

  "Mappings" : {
    "AWSInstanceType2Arch" : {
      "t1.micro"    : { "Arch" : "64" }
    },
    "AWSRegionArch2AMI" : {
      "us-east-1"          : { "64" : "ami-246ed34c" }
    }
  },

"Resources" : {     
      
    "WebServerInstance": {  
      "Type": "AWS::EC2::Instance",
      "Metadata" : {
        "AWS::CloudFormation::Init" : {
          "configSets" : {
            "InstallAndRun" : [ "Install" ]
          },

          "Install" : {
            "packages" : {
              "yum" : {
                "httpd"        : []              
              }
            },

            "files" : {
              "/var/www/html/index.html" : {
              "source" : "https://s3.amazonaws.com/BUCKET_NAME/index.html",  
              "mode"  : "000600",
              "owner" : "apache",
              "group" : "apache"
              },

              "/etc/cfn/cfn-hup.conf" : {
                "content" : { "Fn::Join" : ["", [
                  "[main]\n",
                  "stack=", { "Ref" : "AWS::StackId" }, "\n",
                  "region=", { "Ref" : "AWS::Region" }, "\n"
                ]]},
                "mode"    : "000400",
                "owner"   : "root",
                "group"   : "root"
              },

              "/etc/cfn/hooks.d/cfn-auto-reloader.conf" : {
                "content": { "Fn::Join" : ["", [
                  "[cfn-auto-reloader-hook]\n",
                  "triggers=post.update\n",
                  "path=Resources.WebServerInstance.Metadata.AWS::CloudFormation::Init\n",
                  "action=/opt/aws/bin/cfn-init -v ",
                  "         --stack ", { "Ref" : "AWS::StackName" },
                  "         --resource WebServerInstance ",
                  "         --configsets InstallAndRun ",
                  "         --region ", { "Ref" : "AWS::Region" }, "\n",
                  "runas=root\n"
                ]]}
              }
            },

            "services" : {
              "sysvinit" : {  
                "httpd"   : { "enabled" : "true", "ensureRunning" : "true" },
                "cfn-hup" : { "enabled" : "true", "ensureRunning" : "true",
                              "files" : ["/etc/cfn/cfn-hup.conf", "/etc/cfn/hooks.d/cfn-auto-reloader.conf"]}
              }
            }
          }

        }
      },
      "Properties": {
        "ImageId" : { "Fn::FindInMap" : [ "AWSRegionArch2AMI", { "Ref" : "AWS::Region" },
                          { "Fn::FindInMap" : [ "AWSInstanceType2Arch", { "Ref" : "InstanceType" }, "Arch" ] } ] },
        "InstanceType"   : { "Ref" : "InstanceType" },
        "SecurityGroups" : [ {"Ref" : "WebServerSecurityGroup"} ],
        "KeyName"        : { "Ref" : "KeyName" },
        "UserData"       : { "Fn::Base64" : { "Fn::Join" : ["", [
             "#!/bin/bash -xe\n",
             "yum update -y aws-cfn-bootstrap\n",

             "# Install the files and packages from the metadata\n",
             "/opt/aws/bin/cfn-init -v ",
             "         --stack ", { "Ref" : "AWS::StackName" },
             "         --resource WebServerInstance ",
             "         --configsets InstallAndRun ",
             "         --region ", { "Ref" : "AWS::Region" }, "\n",

             "# Signal the status from cfn-init\n",
             "/opt/aws/bin/cfn-signal -e $? ",
             "         --stack ", { "Ref" : "AWS::StackName" },
             "         --resource WebServerInstance ",
             "         --region ", { "Ref" : "AWS::Region" }, "\n"
        ]]}}        
      }
    },
    
    "WebServerSecurityGroup" : {
      "Type" : "AWS::EC2::SecurityGroup",
      "Properties" : {
        "GroupDescription" : "Enable HTTP access via port 80",
        "SecurityGroupIngress" : [
          {"IpProtocol" : "tcp", "FromPort" : "80", "ToPort" : "80", "CidrIp" : "0.0.0.0/0"},
          {"IpProtocol" : "tcp", "FromPort" : "22", "ToPort" : "22", "CidrIp" : { "Ref" : "SSHLocation"}}
        ]
      }
    }
  },
  
  "Outputs" : {
    "WebsiteURL" : {
      "Description" : "URL for newly created Apache server",
      "Value" : { "Fn::Join" : ["", ["http://", { "Fn::GetAtt" : [ "WebServerInstance", "PublicDnsName" ]}]] }
    }
  }
}

Setting up a Chef workstation with ChefDK

*This is assuming you are running on CentOS or some other RHEL platform

Download the Chef-DK package…
Go to: http://downloads.getchef.com/chef-dk/
Install the package…

sudo rpm -Uvh ChefDK.....rpm

Once its installed check it and make sure the install was successful…
Do:

sudo chef verify

 


Set System Ruby

Do:

which ruby

You might see something like this: ~/.rvm/rubies/ruby-2.1.1/bin/ruby
If you want to use the version of ruby that came with ChefDK do the following…assuming you are using BASH…
Do:

echo 'eval "$(chef shell-init bash)"' >> ~/.bash_profile

Do:

source ~/.bash_profile

Do:

which ruby

Install Git if you dont already have it…

sudo yum git install

 


Setting up the chef-repo

You can do this two ways….download the starter kit from your Chef server OR manually. In this case we will do this manually because I already happen to have a hosted Chef account and am also using my keys on other instances and dont want to have to set them all up again. So…go to your home directory and do:

git clone git://github.com/opscode/chef-repo.git

Then go to ~/chef-repo/ and do:

mkdir .chef

Three files will need to be placed in this directory:
– knife.rb
– ORGANIZATION-validator.pem
– USER.pem

In order to not upload your .chef directory which will house your certs do this:

echo '.chef' >> ~/chef-repo/.gitignore

Now you need to get the 3 files that go into your .chef directory.
Log onto your Chef server. For me this is located at: https://manage.opscode.com

Once logged in click ADMINISTRATION at the top then the name of your organization.

Knife.rb – Click “Generate Knife Config” and download the file. Place it in your .chef directory
ORGANIZATION-validator.pem – can be downloaded by clicking “Reset Validation Key” in the Administration page.
USER.pem – can be downloaded by clicking Users on the left hand side and then choosing your username, and finally clicking “Reset Key

 



Add Ruby to your Path

DO:

echo 'export PATH="/opt/chefdk/embedded/bin:$PATH"' >> ~/.configuration_file && source ~/.configuration_file

Now lets verify that we are all set…
DO:

cd ~/chef-repo

DO:

knife client list

You should see a list of your clients which will only be the one you are on for right now.

That’s it. Let me know if you have questions or run into issues or see mistakes.

Using AWS CLI to connect to MySQL on RDS

I messing around with RDS on Amazon Web Service for a project im working on and realized that there is not a true way to connect to the instance that the database is running on when you stand up a MySQL instance.

After poking around a little bit I came up with this solution and it worked ok for me. Probably needs to be refined and there is probably an easier way also but this worked for me.

First im going to assume you have a MySQL instance running in RDS. Once you have this done you will need to download and unzip the AWS CLI tool to a directory you have access to. > http://aws.amazon.com/developertools/2928

Once you have done this you will need to unzip the package. You can check out the README if you want or you can just do this…
1. copy the credential-file-path.template file and call it something like “cred-file” then chmod it 600
2. vi cred-file fill in the AWSAccessKeyId and AWSSecretKey values. You can get these values from IAM control panel in AWS under your user. If you are lost on that there’s a handy tool called Google.🙂
3. You will need to set AWS_RDS_HOME in your path
4. Make sure JAVA_HOME is set in your path

Once you have followed these steps you can now try it out…
Do: $AWS_RDS_HOME/bin/rds-describe-db-instances --aws-credential-file cred-file --headers
Hopefully your formatting will be better than what you see here but you should see something similar to this:

DBINSTANCE DBInstanceId Created Class Engine Storage Master Username Status Endpoint Address Port AZ SecAZ Backup Retention Multi-AZ Version License Publicly Accessible Storage Type
DBINSTANCE blah 2014-10-15T17:36:01.553Z db.t1.micro mysql 5 blah available blah.czbrj4wkmqs3.us-east-1.rds.amazonaws.com 3306 us-east-1b us-east-1a 7 y 5.6.19 general-public-license y gp2
SECGROUP Name Status
SECGROUP default active
PARAMGRP Group Name Apply Status
PARAMGRP default.mysql5.6 in-sync
OPTIONGROUP Name Status
OPTIONGROUP default:mysql-5-6 in-sync

Now you have tested and know you can access your MySQL db in RDS.

AWS Loadbalancing

For previous setup info see my post on setting up instances using Chef and AWS: HERE

AWS Loadbalancing

If you have completed the steps in the previous post you can repeat them a few times to spin up 3 or more instances. For this post lets assume I have spun up 3 instances named web001, web002, and web003.

1. In your AWS control panel navigate to your EC2 instances and on the left hand column click “Load Balancers” under “Network and Security”.

2. Click the “Create Load Balancer” button at the top.

3. Define Load Balancer Enter a name for your LB and for this example use “EC2-Classic”. By default port 80 is being monitored and since we have 3 web servers running httpd this will be all we need. Click Continue.

4. Configure Health Check For this example the only thing that you need to change is PING PATH. Change it to just “/” dropping the index.html that is already there. Leave everything else the way it is. Continue

5. Add Instances to Load Balancer Check off your 3 instances. Leave the settings for Availability Zone Dist as they are. Continue

6. Keys Here you can create a key and value pair for your load balancer. For example, you could define a tag with key = Name and value = Webserver. Continue

7. Review This will give you a run down of your load balancer before you create it. Click CREATE and it will be started up.

Once your LB has been started it will take a couple of minutes before your instances are in service. If you click the name of your LB  you can see the description tab below. Wait a couple of minutes and then copy the DNS Name which should be the first line that ends with (A Record) Ex. LB-001-201566711211.us-east-1.elb.amazonaws.com Paste this into your browser and you should see your webpage that you created in your instances.

Tabs

You have 7 tabs that contain information about your loadbalancer.

1. Description DNS Name can be found here along with brief info on the status fo the instances connected to this LB

2. Instances Status of the instances that are connected to the LB. You can also drop the instance from the LB here

3. Health Check The main thing to watch here is “Unhealty Threshhold”  You can also edit the Health Check settings here.

4. Monitoring CloudWatch metrics for the selected resources. You can also create alarms for your instances on this page.

5. Security Displays your security groups that are connected to this LB

6. Listeners The ports and protocols you are listening for on your instances.

7. Tags Tags that you created when setting up the LB and you can also create more from this page as well

 

This is a VERY basic run down of how to create a load balancer in Amazon AWS. If you have any questions or input feel free to use the comment section below.