Woo! You're reading part 1 of a trilogy:
- Part 2 - Running LAMP, Ubuntu...
- Part 3 - Running LAMP, Ubuntu... (in progress)
Quickie Introduction to EC2
Louie, I think this is the beginning of a beautiful friendship.
For those of you that get bored with paragraph-style introductions
to blog entries, here's a bulleted list for you:
- End result feels like a VPS, except you can change the specs of your
VPS in seconds.
- Ability to launch a bunch of identical servers in seconds:
- Kickass deployment: bring up a 2nd, identical server,
push your new site version there, test vigorously, point amazon to
use your new instance, shutdown the old instance.
- Play around: Start up an identical server except with faster
hardware. Benchmark and playaround. When you're done playing, shutdown
the instance - you're only charged by the hour
- By definition, you full server image is always saved to S3, so you have an
exact copy of your setup in a very safe place
- Backup your dynamic data (databases, uploads dir) to S3 in seconds
Basically, you can do a ton of cool stuff - I'm a believer!
||Apache 2.2 with MySQL & PHP5
||Symfony 1.2.8 Web Application
||Nope - this is being outsourced to google, no need for the headache or overhead
Signing Up & Downloading your Private Key and Certificate files
To get started, you'll need both an EC2 account and an S3 account.
- EC2 - The actual cloud computing account
- S3 - Storage, used to save images of your server
After you signup, you'll need to download 2 files: a Private Key and a Certificate file.
These are basically the keys to the mansion. Here's a great resource on this simple step:
Whenever you start an instance [turn your "server" on], you must select a
fully-functional image that you want that server to use. Fortunately, the ec2 community
has published thousands of images for all operating systems and setups that you
will use to boot from initially. From there you can customize your server
exactly as you need it. When you're done, you need to save your new, customized image somewhere
so you can boot from it. This is done easily in EC2 and stores the image on your S3 account.
Once there, you can boot any new instances from your full-customized server image.
Basically, there are 2 ways to screw around with EC2:
- Command line, via the awesome ec2-tools
- Gui web interface, via the awesome Amazon console
In reality, you'll need to use both tools. First, let's setup your local computer to use the EC2 tools
sudo apt-get install ec2-api-tools
sudo apt-get install openjdk-6-jdk
Remember that Private Key and that Certificate file you downloaded from Amazon earlier? You'll need
those here. Basically, the ec2 tools look for those files in a certain shell variable. Also, you'll need
another variable for JAVA, which ec2 uses. Paste the following into your ~/.bashrc file:
export EC2_PRIVATE_KEY=$HOME/<where your private key is>/pk-XXXXXXXXXXXXXXXXXXXXXXXXXXXX.pem
export EC2_CERT=$HOME/<where your certificate is>/cert-XXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.pem
Starting up your First Instance
Starting an "instance" basically means that you push the "power" button on one of Amazon's virtual servers.
The cool part is that in addition to turning a machine on, you also specify exactly what operating system
and setup you want that instance to have. There are literally thousands that you can use as a starting point,
which you'll customize afterwards.
Amazon is loaded with images (known as AMI's) ready to be used to start up your machine. To view them, use the
ec2-describe-images command. As this will return MANY results, I grep the results here for only ubuntu AMIs.
ec2-describe-images -a | grep ubuntu
IMAGE ami-5059be39 canonical-cloud-us/ubuntu-intrepid-20090422-i386.manifest.xml 099720109477 available public i386 machine aki-714daa18 ari-6a5bbc03
I chose the above image as it's the latest (at this time) AMI directly created by Canonical, the creators of Ubuntu
itself. The important figure here is the ami-######## string, which identifies the AMI.
Let's get a machine booted up with that image. From the command line:
# create a keypair, this'll be used all over the place for access
ec2-add-keypair ec2-keypair > ec2-keypair.pem
chmod 600 ec2-keypair.pem
ec2-run-instances ami-5059be39 -k ec2-keypair
That's it! You're EC2 instance is now botting up. The i-XXXXXXXX portion is the id of your instance, which is
used to shut it down, among other things. You can continue to run the ec2-describe-instances
to see the status of your instance. It usually takes a few minutes for the machine to fully start up, so wait until
it says that it is "running".
As soon as you boot up an instance, you're being billed. To shutdown an instance, use the i-XXXXXXXX id from the
EC2 Console - GUI goodness and a flex of the power of EC2
EC2 comes with a fantastic gui control panel that allows you to boot up instances, manage your AMI's (images), and do a lot of
other things that we'll touch on later.
Here's a quick run-down of the more important links along the left side:
- INSTANCES -> Instances:
Displays all the actual booted up instances that you have with EC2
- IMAGES -> AMIs (Amazon Machine Images):
Allows you to browse all the images that are available
(including ones that we'll create later for our own use). You can start up an instance with any images
found here at any time
- ELASTIC BLOCK STORE -> Volumes
These are basically hard drives that you'll mount to your server
and is used for any dynamic data (databases, logging, web sites).
- NETWORKING & SECURITY -> Elastic IPs
Essentially a static ip you'll bind to your instance later
- NETWORKING & SECURITY -> Security Groups
Each instance belongs to one or more security groups
(your instance belongs to the "default" security group by default). The security groups define which ports
are open to the public. By default, your instance won't be open on any ports, but we'll open them up one
by one as necessary
Your server is not persistent. In other words, if I boot up my instance, create a file, then shutdown my instance,
that file is gone forever. To save changes to your server, you create your own image (AMI) which
you can then boot from later. So, other than software changes, apache configurations, etc, your server should
stand as a static begin.
What about databases, upload directories and log files? This is where the EBS volume comes in. An EBS volume
is essentially a hard drive that exists as a separate entity from your instance, which you then mount to your
instance. An EBS is persistent, meaning that any file you copy into it will remain after shutting down your
instance. You can even connect to an EBS from multiple instances. This is where everything dynamic - particularly
your databases - will live.
SSH into your instance
SSH'ing into your instance is simple. First, however, we need to open up our server on port 22. We do this by
telling the "default" security group (read above about Security Groups) to allow connections from port 22.
You'll also need to open up other ports later - specifically port 80 when you get your webserver rocking.
ec2-authorize default -p 22
To ssh in, we need to know the internal host that amazon has assigned to us. The host is that one that starts
with ec2- and ends with .amazonaws.com. By the way, this will change each time you shutdown and reboot your
image. No fear, later we'll bind a static ip address to the instances to solve this problem.
INSTANCE i-######## ami-a37190ca ec2-##-##-###-###.compute-#.amazonaws.com ...
While most EC2 images are setup so that you login as root, the one we installed (made by Cononical)
uses the user "ubuntu" (you can then sudo easily once you're in). We'll use the ec2-keypair.pem
that we created above to authenticate into the server.
ssh -i ec2-keypair.pem ubuntu@ec2-##-##-###-###.compute-#.amazonaws.com
You should now be in your server. As the welcome message explains, you can get a bunch of things setup on your
server fairly easily by running the command sudo tasksel --section server. I selected the following:
- Basic Ubuntu server
- LAMP server
- OpenSSH server
Scp your private key and certificate to your cloud
In order to do things such as creating an image from your server setup, you're going to need that
pk-XXXXXXXXXXXXXXXXXXXXXXXXXXXX.pem and cert-XXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.pem files from earlier
on your cloud. To get them up there, we can simply use scp
scp -i /path/to/ec2-keypair.pem /path/to/pk-XXXXXXXXXXXXXXXXXXXXXXXXXXXX.pem /path/to/cert-XXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.pem ubuntu@ec2-##-##-###-###.compute-#.amazonaws.com:
We'll come back and use those in part 2 of this article, so just let them sit there for now.
Where we are and what we're missing still
What we did:
- Signed up for EC2
- Setup ec2 tools
- Picked out an image (AMI) and booted up an instance with it)
- Logged into your new instance
What's still to come:
- Currently, if you shutdown your new instance, any server changes you made are lost. We'll need to customize
our server and then save our own AMI. This will take a snapshot of the server (with our customizations) which
we can boot from later on.
- Create an EBS volume (a hard disk that mounts to your server, but whos data persists after you shutdown
your instance) and move all of the dynamic data to it (Mysql, webserver, logs, etc)
- Associate an elastic ip address with your instance so you can setup your DNS to point at that ip address
- Get a Symfony project rocking on your server
- Explore potential tripping points, like how to properly rsync your symfony project to your EC2 instance