Woo! You're reading part 2 of a trilogy:
- Part 1 - Running LAMP, Ubuntu...
- Part 3 - Running LAMP, Ubuntu... (in progress)
totally unrelated
On our way to a fully-functioning cloud server
In Part 1
of this series, we signed up for our Amazon AWS account, got our local
system setup with our AWS private key and certificate, booted an instance,
and explored Amazon's EC2 console. Now things get interesting. Today we'll tackle the following:
- Assign an elastic ip address to your instance
- Customize your server and save it as an image (AMI) for use later
- Attach and mount an EBS (Elastic Block Store) and point your MySQL databases
at it
- Setup Apache
- Sync up a symfony project and get it rocking
- Final Thoughts - What's left?
Assign an elastic ip address to your instance
Each time you start up your instance, you're assigned a different internal
Amazon host / ip address that you can use to ssh into your system. Things get dicey, however,
when you think about your DNS. If your instance needed to be restarted, then
you'd also have to update your DNS A record for your new ip address. Since this
can take many hours to propogate, the whole world could be looking at the old
server way longer than you intended.
Finding the ip address assigned to you.
After booting up an instance, Amazon always assigns it an ip address. You can bind
this to an, existing static ip address (read below), but you may need to login to
your new instance before doing that. To find out your ip address / host:
# ec2-describe-instances
The response will show all your current instances. Look for the ec2-##-###-###-##.compute-1.amazonaws.com.
You can either use that as a host or use the ip address directly, which is simply
the above numbers represented by # signs.
Enter the idea of "elastic ip" addresses. In EC2, you can "allocate" ip addresses. This means
that you request a static ip address from Amazon which then stays with your EC2 account
forever (unless you don't need it anymore and "release" it). Once you have an ip address,
you can "Associate" it with whatever running instance you have and that ip address will begin
resolving to that instance instantly. You may also now safely use this ip address for your
DNS A record without fear of it needing to be changed later.
- Login to your EC2 console:
http://console.aws.amazon.com/
- Click Elastic IPS on the left navigation
- Click Allocate New Address near the top
- Click Associate near the top
- On the window that pops up, select the correct instance id
and click Associate
You should now be able to ssh into your instance directly via the elastic ip address.
Are these elastic IP addresses free? Well, yes and no. As long as your elastic ip
address is associated with one of your running instances it's completely free.
Amazon charges only for elastic ip addresses that have been allocated to
your account but which aren't associated with an active instance.
Customize your server and save it as an image (AMI) for use later
As explained in
part 1
of this series, the server you've instantiated acts
as a static entity. Yes you can make all of the changes and customizations
you want to it just like any other server. However, as soon as you shutdown
that instance, all of those changes are lost forever. The solution is to
save your server setup as an image (called an Amazon Machine Image or AMI),
which can then be used later to instantiate new instances. This actualy forces
you into the very responsible habit of always having an image of your exact
server setup. From here you can very easily restart your instance or launch
several identical servers and load balance between them.
While this section is really more appropriate for after we get our server
setup with LAMP, I've placed it here because of its importance. Always be sure
to save a new AMI after materially changing your server. If you don't, those
changes will die whenever you inevitablly need to restart your server.
- ssh into your server
- Shutdown Apache and MySQL (we haven't added these yet, but for future reference,
you'll need to stop them)
# /etc/init.d/apache2 stop
# /etc/init.d/mysql stop
- Unmount your EBS (again, we haven't mouned our EBS, but use this for future reference)
# sudo umount /vol
-
Bundle your image. This will create an image of your instance and store it locally in /mnt
# sudo ec2-bundle-vol -d /mnt -k ~ubuntu/pk-XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.pem -c cert-XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.pem -u 3634-9027-7607 -s 1536
-
You'll now upload your image from your cloud up to your S3 account. Before you do this step, you'll need to create a "bucket" in s3 where your image
will live. Once that's setup, you'll need your amazon access key id and secret access key, which you can getfrom the "Access Identifiers"
section of your aws account area (http://aws.amazon.com/account/)
# ec2-upload-bundle -b your-s3-bucket-name -m /mnt/image.manifest.xml -a YOUR_ACCESS_KEY -s YOUR_SECRET_ACCESS_KEY
-
In your local terminal (meaning NOT in your cloud terminal, but on your local machine), run the following command, replacing your-s3-bucket-name with
the bucket you created and used in the previous step
# ec2-register your-s3-bucket-name/image.manifest.xml
The above will "register" your saved image with Amazon, so that you can find and use it later to boot instances. The return from this call will be the
unique identifier for this image, which you'll use when booting instances from it. The response should look something like this:
response: IMAGE ami-a12345bc
- Now that we've finished the imaging of your instance, we've just gotta put the pieces back together
# sudo mount -a
# sudo /etc/init.d/apache2 start
# sudo /etc/init.d/mysql start
Attach and mount an EBS (Elastic Block Store) and point your MySQL databases at it
As explained above, nothing on your actual server gets saved unless you explicitly create an image from it. In this way, your server really functions
like a read only device: only save things to it that you don't want to keep.
So, where do we keep our dynamic content, our databases, our uploaded assets? Amazon's Elastic Block Store (EBS) is basically an external hard drive
that you mount to your static instance. You can have as many EBS volumes as you need, they can be nearly any size and, most important, EBS volumes
live forever and ever. Let's hook up an EBS and get MySQL running on it. We'll start by bringing our EBS into existence.
Turn your EBS on and "plug it in"
- Login to your EC2 console: http://aws.amazon.com/console/
- Click "Instances" on the left, then select the instance you currently have running. In the information that
displays below, look for the "Zone" of your instance (e.g. us-east-1b). You'll need to create your EBS
in the same zone as your instance, so keep that info handy.
- Click on "Volumes" on the left and then click "Create Volume" on the resulting screen.
-
Set the size (see note below related to changing your size later) and choose the same Availability Zone as your instance.
Leave the Snapshot blank and click Create. The volume will take just a little while to process.
-
Once the instance is up and running, select the volume and click "Attach Volume". Basically, you've turned your hard drive
on, but you still need to plug it into your instance. On the resulting screen, make sure the correct instance is selected
and choose /dev/sdh as your Device. Click attach. Nice :)
Changing the size of your EBS
While you can't technically increase the size of your EBS after creation, it's actually quite easy in practice. From the
console, simply choose your existing volume and click "Create Snapshot". Once this backup is complete, create a new
EBS setting the Size to your new desired size and Snapshot to the name of the snapshot your just created. Now that you have
two identical EBS volumes (except one is bigger than the other), simply detach the old volume (you'll need to unmount it first)
and attach the new volume. Remount and celebrate. You're a cloud king.
Now that our EBS is alive and "plugged into" your instance, we need to mount it and get mysql put in the right spot.
Mount your EBS and put MySQL on it
- Shell into your instance
- First, we'll make sure we're updated and then get xfs and mysql installed. We'll use xfs to format our EBS volume.
# sudo apt-get update && sudo apt-get upgrade -y
# sudo apt-get install -y xfsprogs mysql-server
# sudo modprobe xfs
# sudo mkfs.xfs /dev/sdh
# echo "/dev/sdh /vol xfs noatime 0 0" | sudo tee -a /etc/fstab
-
Now we can simply mount our EBS at /vol
# sudo mkdir /vol
# sudo mount /vol
-
Next, move the existing mysql files to the EBS volume then point MySQL to the correct database files on the EBS volume by using mount bind.
# sudo mkdir /vol/etc /vol/lib /vol/log
# sudo mv /etc/mysql /vol/etc/
# sudo mv /var/lib/mysql /vol/lib/
# sudo mv /var/log/mysql /vol/log/
# sudo mkdir /etc/mysql
# sudo mkdir /var/lib/mysql
# sudo mkdir /var/log/mysql
# echo "/vol/etc/mysql /etc/mysql none bind" | sudo tee -a /etc/fstab
# sudo mount /etc/mysql
# echo "/vol/lib/mysql /var/lib/mysql none bind" | sudo tee -a /etc/fstab
# sudo mount /var/lib/mysql
# echo "/vol/log/mysql /var/log/mysql none bind" | sudo tee -a /etc/fstab
# sudo mount /var/log/mysql
# sudo /etc/init.d/mysql start
Wait, what the hell just happened?
Ok, let's regroup after the shell-command barf. An EBS volume is nothing more than a glorified external hard drive. All of that setup
equates to 4 basic steps
- turn your hard drive on
- plugin your hard drive in
- mount your hard drive
- get mysql running on your hard drive
Setup Apache
With the hard work behind us, let's setup apache. Setting up apache is nothing special on the cloud. The only thing you have to do
is make sure that everything ends up on your EBS (/vol), which includes your actual site files, as well as any logs that you might
want to reference later.
# sudo apt-get install php5-curl php5-cli php5-tidy php5-xsl apache2 php5-mysql php5-syck
I'll leave the rest up to you - there are lot's of resources on how to get apache up and rocking.
Sync up a symfony project and get it rocking
If you've gotten this far and gotten your vhost setup for symfony, you're basically done. At this point, your cloud
offers no additional differences from any other VPS that you've worked on before.
If you use Symfony's sync task, however, you'll have one more obstacle to jump through. Symfony's project:deploy task
really wasn't built to shell into servers using a public key. No worries, I've already suffered through this and
created a replacement task for use in Symfony 1.2. Just be sure to add a keypair value to your properties.ini file:
properties.ini
[cloud]
host=clouddomain.com
port=22
user=ubuntu
dir=/vol/www/clouddomain.com
keypair=/full/path/to/ec2-keypair.pem
Final Thoughts - What's left?
Ok, so this wasn't really an article on symfony, except for the last little task I threw in on the end. In general though, I had a hell
of a time getting the cloud up. In the end, I'm a better person for it and I've been very happy with how well the cloud has been running.
Being hooked up with Amazon also means that I can easily back up my EBS volumes to S3 as well as start up new instances just to play
around with (e.g., try out performance tweaks away from your main server). Above all else, clouds a hella-fun but expensive unless
you really need to flex something.
What's left to consider is a) cost and b) benchmarks. I'm thinking it'll be a trilogy.