skip to Main Content
+971 55 881 1725 hmimthiaz @ imthi.com

Imthiaz’s Blog powered by Amazon CloudFront

I have been experimenting with Amazon Web Services for sometime. I was successful in offloading media traffic with the help of Amazon S3 and Amazon CloudFront plugin for WordPress. But it was never enough. Please don’t think my blog traffic is very high, but this is the only website I have with which I could experiment and explore the new frontiers of cloud.

Two weeks ago I decided that I should start serious blogging again and as a part of commitment I would move the blog to a better hosting environment. Moving to Media Temple about a year ago was one of serious mistake. In terms for the scaling etc, media temple is good. In terms of server management I missed WHM and Cpanel very much. Plus media temple has many uninformed outages for my blog which was seriously disturbing for me.

Once again I was out searching for cloud based hosting. I experimented with Amazon EC2 micro instance which turned out awesomely bad experience. The instance never had dedicated CPU so my blog could get stuck for few seconds at times and it was bad. I started checking out some my previous hosting companies. Westhost had moved to cloud based hosting solution and their reseller hosting package was under my budget and was loaded with WHM. Moved all my sites to the new hosting environment.

After moving the site I decided to serve my blog with Amazon CloudFront. CloudFront has a feature to serve files from custom origin. With this feature I can create a distribution and point the source of the distribution to my web server. For more information on this please refer Amazon documentation for CloudFront.

Comments – When you use CloudFront the communication is one way, so having WordPress comments is out of the question. I have to look for hosted comments system. LiveFyre came to rescue, and it has 100% compatibility with WordPress.

Mobile Theme – CloudFront supports single cache for a page, So having a separate theme for mobile was not applicable. My friend Shuja Shabandri suggested to use responsive theme. After searching for a while I came with simple and responsive design Bugis by Elmastudio.

Content Expiry – CloudFront by default does caching for 24 hours. I had to reduce the caching to 30 minutes for HTML and others to 24 hours. For this I used Expires Headers from Apache.

Will be updating more on this latter 😉

Update: Did a load test using Load Impact Free Load Test and the load time were impressive…

AWS – Adding extra EBS volume to your EC2 instance

When you boot an Amazon Linux EC2 instance it boots with a 8GB EBS volume. If you need more space you need to add additional drives. For this you need to use EBS volumes.

Before you start the process please have look at the current partition blocks loaded in your server. You can do so using the contents of partition file.

Now you goto EBS volume manager in AWS console and create a new volume, make sure the zone is the same in which your EC2 instance is running.

Once the volume is created you need to attach this to an instance. You can right click on the created volume and say attach. Select the instance then device will populate automatically, you can either leave it or change if you need specific device name.

Now check the partition file again. You can see a new device being added.

The volume attached is not ready for use. It is like a new hard disk. You need to partition and format the same. In our case I am going to use the full disk as one partition. So I am going to skip the fdisk setup and jumping right into formatting the volume.

The format process can take few seconds just be patient. The drive is ready to use, and to do the same we need mount it.

If you wish this device to mount automatically when you reboot the server make sure you add this to your fstab file.

When I started which Amazon Cloud, I found these things in a hard way, hope this can help someone 🙂

Apache Log Format for Amazon EC2 with Elastic Load Balancer

The journey through the Amazon Cloud (AWS) is quite amazing. Recently I had to use two EC2 instance with Elastic Load balancer. Once the website was served with load balancer only the load balancer IP was logged as remote IP.

After Googling for few hour and with trial and error method I got the correct Apache log format so that the real remote IP is logged.

I created a custom log format with a name “combined_new” and used it in all the virtual host configured in Apache. But you can use the format however you want.

Before you make any changes to httpd.conf please go through Apache log module.

I hope this small tip helps 😉

Note: Always backup your original httpd.conf before you make any changes.

Download & Process Amazon Cloudfront Logs with Awstats

These days we use Amazon Cloudfront for content delivery. Amazon has made it very easy to deliver files in a Amazon Simple Storage Service (S3) bucket using Amazon Cloudfront distribution. If you are using Cloudfront as Content Delivery Network (CDN) your next task will be monitoring the usage. For this Amazon Cloudfront has a provision to store access logs to a S3 bucket. My hurdle was to process the log files stored by Cloudfront. For sites hosted with apache I use Awstats for reading the logs. So my vote was for awstats. Please follow the steps one by one 😉

1. Need to download the log files stored in the S3 bucket. For this I had to use the a python script done by wpstorm.net but I had to make some modification so that it worked for me. Please follow the blog post if you need any help setting up the required libraries.

get-aws-logs.py

Note: The above script will download the s3 logs to specified folder. Please make sure you put your Amazon access keys.

2. Now we have bash script which will uses the above python script to download the log files and combine all of them into a single log file and then it will be analyzed by awstats.

Warning: Please read through the script files and make necessary changes needed.
Note: You should have awstats installed on your system. The bellow script uses awstats.
Note: You can download the script files at the end of this blog post where awstats configuration with custom setup for cloudfront log format is also provided.

get-aws-logs.sh

I would suggest you to test run the above scripts on a staging / testing environment before moving to a production. Again please change the scripts with your domain details and Amazon access keys.

Download the scripts to download and process Amazon Cloudfront Logs with Awstats.

Have a nice journey exploring the cloud 😉

Working with Amazon Web Services (AWS)

Today I started my journey to explore and learn more about Amazon Web Services (AWS). My mission is to setup a wordpress blog completely in the cloud. This blog post is just a mention about the journey and will be posting things in detail in the days to come.

What I did so far as follows..

  • Created a new EC2 instance with Basic 32-bit Amazon Linux
  • Logged into the instance and installed Apache, PHP and mysql client tools
  • Created a new RDS instance with mysql version 5.1.50
  • Added EC2 instance IP in RDS for allowing access.
  • Installed wordpress on the EC2 instance and configured the database
  • Created a new elastic volume storage for 1GB and attached the same to EC2 instance
  • Formatted the elastic volume storage and mounted the same for wordpress upload directory.
  • Created a new Elastic IP and attached the EC2 instance to the IP.
  • Created a cname cloud for imthi.com and pointed the A level record to the Elastic IP
  • The blog cloud.imthi.com is ready that is hosted in Amazon cloud 🙂

Phew, trust me it was not easy and I had to create and delete many EC2 instances to understand how things work. I am not sure whether the setup I had done is the best way.. I did some load test on the setup and it was not looking good

My next mission is to fire up multiple EC2 instances and to use a load balancer to handle the load so that the graph will look better in the days to come 😉

Back To Top