Running LAMP, Ubuntu and Symfony on Amazon's EC2 - It's Awesome! (Part 2)

Woo! You're reading part 2 of a trilogy:
   - Part 1 - Running LAMP, Ubuntu...
   - Part 3 - Running LAMP, Ubuntu... (in progress)

totally unrelated

On our way to a fully-functioning cloud server

In Part 1 of this series, we signed up for our Amazon AWS account, got our local system setup with our AWS private key and certificate, booted an instance, and explored Amazon's EC2 console. Now things get interesting. Today we'll tackle the following:

1. Assign an elastic ip address to your instance
2. Customize your server and save it as an image (AMI) for use later
3. Attach and mount an EBS (Elastic Block Store) and point your MySQL databases at it
   • Turn your EBS on and "plug it in"
   • Mount your EBS and put MySQL on it
4. Setup Apache
5. Sync up a symfony project and get it rocking
6. Final Thoughts - What's left?

Assign an elastic ip address to your instance

Each time you start up your instance, you're assigned a different internal Amazon host / ip address that you can use to ssh into your system. Things get dicey, however, when you think about your DNS. If your instance needed to be restarted, then you'd also have to update your DNS A record for your new ip address. Since this can take many hours to propogate, the whole world could be looking at the old server way longer than you intended.

Finding the ip address assigned to you. After booting up an instance, Amazon always assigns it an ip address. You can bind this to an, existing static ip address (read below), but you may need to login to your new instance before doing that. To find out your ip address / host:

# ec2-describe-instances

The response will show all your current instances. Look for the ec2-##-###-###-##.compute-1.amazonaws.com. You can either use that as a host or use the ip address directly, which is simply the above numbers represented by # signs.

Enter the idea of "elastic ip" addresses. In EC2, you can "allocate" ip addresses. This means that you request a static ip address from Amazon which then stays with your EC2 account forever (unless you don't need it anymore and "release" it). Once you have an ip address, you can "Associate" it with whatever running instance you have and that ip address will begin resolving to that instance instantly. You may also now safely use this ip address for your DNS A record without fear of it needing to be changed later.

1. Login to your EC2 console: http://console.aws.amazon.com/
2. Click Elastic IPS on the left navigation
3. Click Allocate New Address near the top
4. Click Associate near the top
5. On the window that pops up, select the correct instance id and click Associate

You should now be able to ssh into your instance directly via the elastic ip address.

Are these elastic IP addresses free? Well, yes and no. As long as your elastic ip address is associated with one of your running instances it's completely free. Amazon charges only for elastic ip addresses that have been allocated to your account but which aren't associated with an active instance.

Customize your server and save it as an image (AMI) for use later

As explained in part 1 of this series, the server you've instantiated acts as a static entity. Yes you can make all of the changes and customizations you want to it just like any other server. However, as soon as you shutdown that instance, all of those changes are lost forever. The solution is to save your server setup as an image (called an Amazon Machine Image or AMI), which can then be used later to instantiate new instances. This actualy forces you into the very responsible habit of always having an image of your exact server setup. From here you can very easily restart your instance or launch several identical servers and load balance between them.

While this section is really more appropriate for after we get our server setup with LAMP, I've placed it here because of its importance. Always be sure to save a new AMI after materially changing your server. If you don't, those changes will die whenever you inevitablly need to restart your server.

1. ssh into your server
2. Shutdown Apache and MySQL (we haven't added these yet, but for future reference, you'll need to stop them)

   # /etc/init.d/apache2 stop
   # /etc/init.d/mysql stop
         

3. Unmount your EBS (again, we haven't mouned our EBS, but use this for future reference)

   # sudo umount /vol
         

4. Bundle your image. This will create an image of your instance and store it locally in /mnt

   # sudo ec2-bundle-vol -d /mnt -k ~ubuntu/pk-XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.pem -c cert-XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.pem -u 3634-9027-7607 -s 1536
       

5. You'll now upload your image from your cloud up to your S3 account. Before you do this step, you'll need to create a "bucket" in s3 where your image will live. Once that's setup, you'll need your amazon access key id and secret access key, which you can getfrom the "Access Identifiers" section of your aws account area (http://aws.amazon.com/account/)

   # ec2-upload-bundle -b your-s3-bucket-name -m /mnt/image.manifest.xml -a YOUR_ACCESS_KEY -s YOUR_SECRET_ACCESS_KEY
   	

6. In your local terminal (meaning NOT in your cloud terminal, but on your local machine), run the following command, replacing your-s3-bucket-name with the bucket you created and used in the previous step

   # ec2-register your-s3-bucket-name/image.manifest.xml
   	

   The above will "register" your saved image with Amazon, so that you can find and use it later to boot instances. The return from this call will be the unique identifier for this image, which you'll use when booting instances from it. The response should look something like this:

   response: IMAGE	ami-a12345bc

7. Now that we've finished the imaging of your instance, we've just gotta put the pieces back together

   # sudo mount -a
   # sudo /etc/init.d/apache2 start
   # sudo /etc/init.d/mysql start
     

Attach and mount an EBS (Elastic Block Store) and point your MySQL databases at it

As explained above, nothing on your actual server gets saved unless you explicitly create an image from it. In this way, your server really functions like a read only device: only save things to it that you don't want to keep.

So, where do we keep our dynamic content, our databases, our uploaded assets? Amazon's Elastic Block Store (EBS) is basically an external hard drive that you mount to your static instance. You can have as many EBS volumes as you need, they can be nearly any size and, most important, EBS volumes live forever and ever. Let's hook up an EBS and get MySQL running on it. We'll start by bringing our EBS into existence.

• Login to your EC2 console: http://aws.amazon.com/console/
• Click "Instances" on the left, then select the instance you currently have running. In the information that displays below, look for the "Zone" of your instance (e.g. us-east-1b). You'll need to create your EBS in the same zone as your instance, so keep that info handy.
• Click on "Volumes" on the left and then click "Create Volume" on the resulting screen.
• Set the size (see note below related to changing your size later) and choose the same Availability Zone as your instance. Leave the Snapshot blank and click Create. The volume will take just a little while to process.
• Once the instance is up and running, select the volume and click "Attach Volume". Basically, you've turned your hard drive on, but you still need to plug it into your instance. On the resulting screen, make sure the correct instance is selected and choose /dev/sdh as your Device. Click attach. Nice :)

Changing the size of your EBS
While you can't technically increase the size of your EBS after creation, it's actually quite easy in practice. From the console, simply choose your existing volume and click "Create Snapshot". Once this backup is complete, create a new EBS setting the Size to your new desired size and Snapshot to the name of the snapshot your just created. Now that you have two identical EBS volumes (except one is bigger than the other), simply detach the old volume (you'll need to unmount it first) and attach the new volume. Remount and celebrate. You're a cloud king.

Now that our EBS is alive and "plugged into" your instance, we need to mount it and get mysql put in the right spot.

• Shell into your instance
• First, we'll make sure we're updated and then get xfs and mysql installed. We'll use xfs to format our EBS volume.

  # sudo apt-get update && sudo apt-get upgrade -y
  # sudo apt-get install -y xfsprogs mysql-server
  # sudo modprobe xfs
  # sudo mkfs.xfs /dev/sdh
  # echo "/dev/sdh /vol xfs noatime 0 0" | sudo tee -a /etc/fstab
    

• Now we can simply mount our EBS at /vol

  # sudo mkdir /vol
  # sudo mount /vol
      

• Next, move the existing mysql files to the EBS volume then point MySQL to the correct database files on the EBS volume by using mount bind.

# sudo mkdir /vol/etc /vol/lib /vol/log
# sudo mv /etc/mysql     /vol/etc/
# sudo mv /var/lib/mysql /vol/lib/
# sudo mv /var/log/mysql /vol/log/
# sudo mkdir /etc/mysql
# sudo mkdir /var/lib/mysql
# sudo mkdir /var/log/mysql
# echo "/vol/etc/mysql /etc/mysql     none bind" | sudo tee -a /etc/fstab
# sudo mount /etc/mysql
# echo "/vol/lib/mysql /var/lib/mysql none bind" | sudo tee -a /etc/fstab
# sudo mount /var/lib/mysql
# echo "/vol/log/mysql /var/log/mysql none bind" | sudo tee -a /etc/fstab
# sudo mount /var/log/mysql
# sudo /etc/init.d/mysql start
  

Ok, let's regroup after the shell-command barf. An EBS volume is nothing more than a glorified external hard drive. All of that setup equates to 4 basic steps

1. turn your hard drive on
2. plugin your hard drive in
3. mount your hard drive
4. get mysql running on your hard drive

Setup Apache

With the hard work behind us, let's setup apache. Setting up apache is nothing special on the cloud. The only thing you have to do is make sure that everything ends up on your EBS (/vol), which includes your actual site files, as well as any logs that you might want to reference later.

# sudo apt-get install php5-curl php5-cli php5-tidy php5-xsl apache2 php5-mysql php5-syck

I'll leave the rest up to you - there are lot's of resources on how to get apache up and rocking.

Sync up a symfony project and get it rocking

If you've gotten this far and gotten your vhost setup for symfony, you're basically done. At this point, your cloud offers no additional differences from any other VPS that you've worked on before.

If you use Symfony's sync task, however, you'll have one more obstacle to jump through. Symfony's project:deploy task really wasn't built to shell into servers using a public key. No worries, I've already suffered through this and created a replacement task for use in Symfony 1.2. Just be sure to add a keypair value to your properties.ini file:

properties.ini
  [cloud]
    host=clouddomain.com
    port=22
    user=ubuntu
    dir=/vol/www/clouddomain.com
    keypair=/full/path/to/ec2-keypair.pem

Final Thoughts - What's left?

Ok, so this wasn't really an article on symfony, except for the last little task I threw in on the end. In general though, I had a hell of a time getting the cloud up. In the end, I'm a better person for it and I've been very happy with how well the cloud has been running. Being hooked up with Amazon also means that I can easily back up my EBS volumes to S3 as well as start up new instances just to play around with (e.g., try out performance tweaks away from your main server). Above all else, clouds a hella-fun but expensive unless you really need to flex something.

What's left to consider is a) cost and b) benchmarks. I'm thinking it'll be a trilogy.