Running LAMP, Ubuntu and Symfony on Amazon's EC2 - It's Awesome! (Part 1)

Posted by ryan in ec2, symfony, ubuntu

Woo! You're reading part 1 of a trilogy:
- Part 2 - Running LAMP, Ubuntu...
- Part 3 - Running LAMP, Ubuntu... (in progress)

Quickie Introduction to EC2

Amazon EC2 and Symfony
Louie, I think this is the beginning of a beautiful friendship.

For those of you that get bored with paragraph-style introductions to blog entries, here's a bulleted list for you:

  • End result feels like a VPS, except you can change the specs of your VPS in seconds.
  • Ability to launch a bunch of identical servers in seconds:
    • Kickass deployment: bring up a 2nd, identical server, push your new site version there, test vigorously, point amazon to use your new instance, shutdown the old instance.
    • Play around: Start up an identical server except with faster hardware. Benchmark and playaround. When you're done playing, shutdown the instance - you're only charged by the hour
  • By definition, you full server image is always saved to S3, so you have an exact copy of your setup in a very safe place
  • Backup your dynamic data (databases, uploads dir) to S3 in seconds
  • Basically, you can do a ton of cool stuff - I'm a believer!

The Setup

OS Ubuntu
Web server Apache 2.2 with MySQL & PHP5
End Use Symfony 1.2.8 Web Application
Mail Server Nope - this is being outsourced to google, no need for the headache or overhead

Signing Up & Downloading your Private Key and Certificate files

To get started, you'll need both an EC2 account and an S3 account.

  • EC2 - The actual cloud computing account
  • S3   - Storage, used to save images of your server

After you signup, you'll need to download 2 files: a Private Key and a Certificate file. These are basically the keys to the mansion. Here's a great resource on this simple step:

Whenever you start an instance [turn your "server" on], you must select a fully-functional image that you want that server to use. Fortunately, the ec2 community has published thousands of images for all operating systems and setups that you will use to boot from initially. From there you can customize your server exactly as you need it. When you're done, you need to save your new, customized image somewhere so you can boot from it. This is done easily in EC2 and stores the image on your S3 account. Once there, you can boot any new instances from your full-customized server image.

EC2 Tools

Basically, there are 2 ways to screw around with EC2:

  • Command line, via the awesome ec2-tools
  • Gui web interface, via the awesome Amazon console

In reality, you'll need to use both tools. First, let's setup your local computer to use the EC2 tools

sudo apt-get install ec2-api-tools
sudo apt-get install openjdk-6-jdk

Remember that Private Key and that Certificate file you downloaded from Amazon earlier? You'll need those here. Basically, the ec2 tools look for those files in a certain shell variable. Also, you'll need another variable for JAVA, which ec2 uses. Paste the following into your ~/.bashrc file:

export EC2_PRIVATE_KEY=$HOME/<where your private key is>/pk-XXXXXXXXXXXXXXXXXXXXXXXXXXXX.pem
export EC2_CERT=$HOME/<where your certificate is>/cert-XXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.pem
export JAVA_HOME=/usr/lib/jvm/java-6-openjdk/

Starting up your First Instance

Starting an "instance" basically means that you push the "power" button on one of Amazon's virtual servers. The cool part is that in addition to turning a machine on, you also specify exactly what operating system and setup you want that instance to have. There are literally thousands that you can use as a starting point, which you'll customize afterwards.

Amazon is loaded with images (known as AMI's) ready to be used to start up your machine. To view them, use the ec2-describe-images command. As this will return MANY results, I grep the results here for only ubuntu AMIs.

ec2-describe-images -a | grep ubuntu

IMAGE	ami-5059be39	canonical-cloud-us/ubuntu-intrepid-20090422-i386.manifest.xml	099720109477	available	public		i386	machine	aki-714daa18	ari-6a5bbc03

I chose the above image as it's the latest (at this time) AMI directly created by Canonical, the creators of Ubuntu itself. The important figure here is the ami-######## string, which identifies the AMI. Let's get a machine booted up with that image. From the command line:

# create a keypair, this'll be used all over the place for access
ec2-add-keypair ec2-keypair > ec2-keypair.pem
chmod 600 ec2-keypair.pem
ec2-run-instances ami-5059be39 -k ec2-keypair

That's it! You're EC2 instance is now botting up. The i-XXXXXXXX portion is the id of your instance, which is used to shut it down, among other things. You can continue to run the ec2-describe-instances to see the status of your instance. It usually takes a few minutes for the machine to fully start up, so wait until it says that it is "running".

As soon as you boot up an instance, you're being billed. To shutdown an instance, use the i-XXXXXXXX id from the ec2-describe-instances command.

ec2-terminate-instances i-XXXXXXXX

EC2 Console - GUI goodness and a flex of the power of EC2

Amazon's EC2 Console

EC2 comes with a fantastic gui control panel that allows you to boot up instances, manage your AMI's (images), and do a lot of other things that we'll touch on later. http://console.aws.amazon.com/

Here's a quick run-down of the more important links along the left side:

  • INSTANCES -> Instances:
    Displays all the actual booted up instances that you have with EC2
  • IMAGES -> AMIs (Amazon Machine Images):
    Allows you to browse all the images that are available (including ones that we'll create later for our own use). You can start up an instance with any images found here at any time
    These are basically hard drives that you'll mount to your server and is used for any dynamic data (databases, logging, web sites).
    Essentially a static ip you'll bind to your instance later
  • NETWORKING & SECURITY -> Security Groups
    Each instance belongs to one or more security groups (your instance belongs to the "default" security group by default). The security groups define which ports are open to the public. By default, your instance won't be open on any ports, but we'll open them up one by one as necessary
Your server is not persistent. In other words, if I boot up my instance, create a file, then shutdown my instance, that file is gone forever. To save changes to your server, you create your own image (AMI) which you can then boot from later. So, other than software changes, apache configurations, etc, your server should stand as a static begin.

What about databases, upload directories and log files? This is where the EBS volume comes in. An EBS volume is essentially a hard drive that exists as a separate entity from your instance, which you then mount to your instance. An EBS is persistent, meaning that any file you copy into it will remain after shutting down your instance. You can even connect to an EBS from multiple instances. This is where everything dynamic - particularly your databases - will live.

SSH into your instance

SSH'ing into your instance is simple. First, however, we need to open up our server on port 22. We do this by telling the "default" security group (read above about Security Groups) to allow connections from port 22. You'll also need to open up other ports later - specifically port 80 when you get your webserver rocking.

ec2-authorize default -p 22

To ssh in, we need to know the internal host that amazon has assigned to us. The host is that one that starts with ec2- and ends with .amazonaws.com. By the way, this will change each time you shutdown and reboot your image. No fear, later we'll bind a static ip address to the instances to solve this problem.

INSTANCE	i-########	ami-a37190ca	ec2-##-##-###-###.compute-#.amazonaws.com	...

While most EC2 images are setup so that you login as root, the one we installed (made by Cononical) uses the user "ubuntu" (you can then sudo easily once you're in). We'll use the ec2-keypair.pem that we created above to authenticate into the server.

ssh -i ec2-keypair.pem ubuntu@ec2-##-##-###-###.compute-#.amazonaws.com

You should now be in your server. As the welcome message explains, you can get a bunch of things setup on your server fairly easily by running the command sudo tasksel --section server. I selected the following:

  • Basic Ubuntu server
  • LAMP server
  • OpenSSH server

Scp your private key and certificate to your cloud

In order to do things such as creating an image from your server setup, you're going to need that pk-XXXXXXXXXXXXXXXXXXXXXXXXXXXX.pem and cert-XXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.pem files from earlier on your cloud. To get them up there, we can simply use scp

scp -i /path/to/ec2-keypair.pem /path/to/pk-XXXXXXXXXXXXXXXXXXXXXXXXXXXX.pem /path/to/cert-XXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.pem ubuntu@ec2-##-##-###-###.compute-#.amazonaws.com:

We'll come back and use those in part 2 of this article, so just let them sit there for now.

Where we are and what we're missing still

What we did:

  • Signed up for EC2
  • Setup ec2 tools
  • Picked out an image (AMI) and booted up an instance with it)
  • Logged into your new instance

What's still to come:

  • Currently, if you shutdown your new instance, any server changes you made are lost. We'll need to customize our server and then save our own AMI. This will take a snapshot of the server (with our customizations) which we can boot from later on.
  • Create an EBS volume (a hard disk that mounts to your server, but whos data persists after you shutdown your instance) and move all of the dynamic data to it (Mysql, webserver, logs, etc)
  • Associate an elastic ip address with your instance so you can setup your DNS to point at that ip address
  • Get a Symfony project rocking on your server
  • Explore potential tripping points, like how to properly rsync your symfony project to your EC2 instance
Thanks for the shares!
  • StumbleUpon
  • Sphinn
  • del.icio.us
  • Facebook
  • TwitThis
  • Google
  • Reddit
  • Digg
  • MisterWong
Posted by david Clark on 2009-08-01
great post as always!

the thing that has deterred me from playing with ec2 is not being able to assess cost. Any info?
Posted by Ryan on 2009-08-01
Actually, really good question - I meant to address that. I can't say concretely (because i've only started using it), but I do have a few comments. If you run 1 small instance 24 hours a day for a month, that'll run you about $72. That's the largest cost and you control it entirely. There are other variable costs (could "run" up in theory) related to storage and bandwidth, but these seem to be very small compared to the server instance cost. One great part of "playing" with EC2 is that you're only charged by the hour with no monthly minimum. So, if you spend a Saturday afternoon playing around, you may only run up a bill of less than $1 or so.
Posted by Tenshu on 2009-08-01
Well thats still pretty expensive 72€ a month!
Thats a big deception to see that cloud computing dont mâle webhosting more accessible for individuals.

I'll stick on my shared - but with ssh access - at alwaysdata for .... 5€ per month.
Posted by Ryan on 2009-08-01
@Tenshu - yes, you still need to be doing something that demands a little bit more control to merit the cloud at this point. It is $72 US dollars though, not euro.

Also, if you reserve an instance up front for a year, the price falls from $72 to about $49 a month.
Posted by leonardo diaz on 2009-08-02
For smallsites I think cloud computing it's not the best option, but for big sites, were you need to guarante the availability of the site could computing its the best.
I would like to see a benchmark on how many resquest can be processed with a small instance so we can compare with other hosting methods

@Ryan. Thanks for this excelent post, I'm waiting for the next.
Posted by Lakshmipathi.G on 2009-08-28
72$ is approx 3000/Month RS for me.
For my current shared hosting (1GB bandwidth,20MB space) I paid 1000 RS / year

So for small websites , Cloud is the right place :)

3000Rs/month is too costly for root access :) I perfer to hack my shared server ,lol
Posted by Lakshmipathi.G on 2009-08-28
oops ..sorry read as "So for small websites , Cloud is NOT the right place :)"

Posted by Ken on 2009-09-04
@Tenshu $72 is not a lot for what you're getting. I'm just now looking into this, it may possibly replace my dedicated dual xeon box, which I pay $99 a month for now. For most people shared hosting for a few bucks a month is the way to go, but once you're running a few hundred websites traditional hosting no longer makes sense. The nice thing about this is I no longer would need to buy hardware, and it looks like I might save some money too!
Posted by Ryan on 2009-09-05
@Ken - 100% in agreement - we started pushing our shared hosting beyond its limits. The cloud has been great.
Posted by Matt Robinson on 2009-09-30
I'm confused - I ran the AWS calculator, plugged in 24*30.5 for an average-ish monthly fee, plus 1 gig EBS and 4 Gig S3 (to store the AMI) and it came to about $35/month. What am I missing?
Posted by Ryan on 2009-09-30
@Matt Robinson

My guess is that you were looking at the "reserved" instances. The pricing I quote is for the "On Demand" instances. If you pay a fee up front, you can significantly reduce your monthly fees, which is almost always a good idea, once you know you're going to stick with it.
Posted by MG on 2009-11-04
hmm . if amazon could halve the resources or make 1/3 of the resources for an instance, and charge $20 - 35 per month. it will appeal to many small site hosters.
Posted by chip on 2010-07-25
Ubuntu and Symfony for thanx.
Posted by Andreas on 2010-10-22

thanks alot for grinding through this stuff, it was not trivial but finally got it working. Would be nice with some links to valuable Apache/Symfony links for the absolute beginner but google works just as well :-)
Posted by Rowan on 2011-11-04
Fantastic post! Helped me a great deal. A helpful hint:

If you are getting a "tasksel: aptitude failed (100)" error when calling tasksel you probably just need to run "sudo apt-get update" first.
Posted by Mike on 2012-04-01
Hey thanks for this series of articles! I'm trying to get a webserver setup with EC2. I'm having some trouble installing the EC2 tools using these commands:

sudo apt-get install ec2-api-tools
sudo apt-get install openjdk-6-jdk

I keep getting a "command not found" error

Was wondering if there are any special requirements for the local computer to install EC2 tools. Thanks!
Posted by Mike on 2012-05-08
Thanks for the post. Try a single user interface to install and run LAMP. For example ScaleXtreme's website (www.scalextreme.com) allows you to download LAMP and implement it into all your Amazon EC2 Servers (also Rackspace, Bluelock, HP Cloud Services, etc...).