Posts Tagged ‘bill’

Amazon CloudFront and S3 maximum cost

Thursday, May 19th, 2011

When using utility computing platforms such as Amazon CloudFront, S3, EC2, or similar platforms from other companies, you are charged according to how much usage you or other people make of the files and services you publicly expose.

If you configure your files or services to be private only to you then you can easily control the cost. But when you make them accessible to the public (e.g. by hosting your website images on CloudFront) your costs are determined by people you have never met.

The costs tend to work out pretty cheap under normal usage, and these utility computing platforms are a great idea in theory. But what if the usage isn’t normal? What if someone decides to launch a DDOS attack on you by downloading huge amounts of data from your CloudFront or S3 account for example? I’ve no doubt that Amazon could handle the load, but could your bank account, or business cope?

cloud maximum cost worst case scenario

I’m not singling out Amazon here; the same applies for any ‘pay for computing power / bandwidth’ service. It is just that Amazon’s services set the benchmark for other similar companies so their platform provides a good example to base my calculations on.

The chances of such an attack happening to you are probably small, but the impact could be large, so it is worth doing the calculations before signing up.

Calculating the Amazon CloudFront worst case scenario

In order to calculate the worst case scenario we first need to understand how the pricing works. Amazon provides a page of pricing information on their website. You are charged for both bandwidth used, and the number of HTTP requests. The bandwidth cost varies depending on where in the world the data is downloaded from.

There are also some minor costs for file storage if you use S3 to store your data, and for loading the data if you use a custom origin server – but these are likely to be very minor costs so I will only use the bandwidth and number of requests for my calculations.

Next we need to know the maximum amount of bandwidth and requests that can be be used with your default account. Although it isn’t very easy to spot, there is a default limit listed on that pricing page of 1000 requests per second and 1000 megabits (megabits not megabytes – 1000 megabits = 125 megabytes) per second. That is a lot of data for any small hobby or business site but Amazon can cope with it. They even provide a request form if you need a higher limit.

Using these values you can see that in an hour the maximum number of requests is 3,600,000 and the maximum bandwidth usage is about 450 gigabytes.

The most expensive region to download data from is Japan so I will base my calculation on the data being downloaded at the maximum number of requests and bandwidth from the Japan prices (starting at $0.201/GB and $0.0095 per 10,000 requests as of May 2011). Of course in reality it would be impossible for someone to hit you with the maximum load, but that doesn’t matter – I’m calculating the theoretical worst case scenario here.

As the bandwidth pricing is tiered (as you move up the usage tiers the cost goes down) it is a bit hard to calculate the cost using a calculator. Amazon does provide its own cost calculator but I didn’t fancy using it for each data point so I’m using Excel instead. I’m using their cost calculator only to cross check my values to make sure that they are correct to an acceptable margin of error.

If you ever want to calculate tiered pricing then the SUMPRODUCT function in Excel is what will make it easy (once you figure out its cryptic syntax that is).

CloudFront worst case hour by hour cost for the first day

The first graph shows the theoretical worst case scenario cost for the first 24 hours of this imagionary DDOS attack on your account.

amazon cloudfront maximum cost per hour

After the first hour 0.45TB and 3,600,000 requests could have been charged to your account at a cost of nearly $94. By the end of the day the cost is just over $2200.

If an attack like this started how long would it take you to notice? Unless you sign up to a 3rd party monitoring service you wouldn’t find out about the problem until you next signed into the Amazon Web Services (AWS) console.

Even if you try to check everyday you can be hit with a big cost while you are asleep, on a plane, at dinner, or with friends. For a big business $2200 might be nothing, but how many hobbies website builders who are using AWS could say the same?

CloudFront worst case day by day cost for the first month

The next graph shows the maximum cost over 30 days. The subtle curve is the effect of the tiered pricing kicking in reducing the bandwidth cost each time you pass the next tier.

amazon cloudfront maximum cost per day

After a week your cost is over $11,000, at two weeks it is nearly $22,000, and after 30 days it could be over $42,000. About $40,000 of that is the bandwidth cost and only $2,000 is the requests cost. At the end of 30 days a maximum of 324TB of data could have been downloaded (theoretically).

Would Amazon spot and stop the bandwidth usage?

If this happened to you would Amazon (or similar companies) warn you?

I can’t find anything in their information that suggests they would. I’m not even sure if they’d notice. Although the bandwidth usage and number of hits might be huge for you, it’s peanuts for Amazon. Remember I’m basing these figures on their default throttled account. You can request much higher usage limits if you want.

Would you get a refund if this happened? Again I don’t know as their information mentions nothing about this. It does say that use are you are soely responsible for the use of your content, which may mean that if it is misued (e.g. DDOS attack) it is your responsibility to pay. You can understand that from Amazon’s point of view they will still incur costs for their own bandwidth whether the bandwidth usage is beneficial to you or not.

It would be good to see Amazon (and similar companies) clarifying their policies about whether they will offer any assistance in the event of an attack.

What do people pay for normal CloudFront / S3 usage?

There are many people posting about their AWS bills. Most of the stories I’ve read are about people saying how small their bill is. Here is a list of accounts I’ve found, some are using CloudFront, and some S3 (prices are mostly per month).

$0.12 – for a couple of days usage.
$0.86 – to host static files for a month.
$1.00 – a month to host static files.
$1.00 – a month at most for website data.
$1.22 – in a month for website images.
$1.30 – to make site go faster .
$1.50 – to host nutritional leaflet each month.
$1.69 – for a months image hosting to make site faster.
$1.81 – to serve blog images for a month.
$2.00 – for image hosting a month.
$5.00 – to make large site faster with CloudFront.
$5.00 – a month to make site ‘Reddit/Digg proof’.
$70.00 – a month on S3/CloudFront worked out cheaper than their dedicated server.
$440.40 – for storing images produced by a MySpace application.
$500,000 – a year saved by SmugMug by using S3.

Has this problem been mentioned before?

This problem has been mentioned many times on Amazon’s Web Services forums, so Amazon are aware of the issue, but as far as I’m aware no one has ever (until now!) produced worst case data for excessive usage.

Taking this thread as an example.

  • 2006 5th May – Someone requests feature to limit cost / bandwidth
  • 2006 5th May – Amazon reply to say that this ‘is in the works’. The plan is to enable you to cap the monthly charges.
  • 2006 – Other’s join in the conversation saying they would like the bill capping feature.
  • 2006 14th September – Amazon say the feature is ‘forthcoming’ but that they have no update on timing.
  • 2007/2008 – More people request the feature or updates from Amazon.
  • 2008 3rd June – Amazon reply saying it is still on their list of things to add but that they don’t expect to release it this year.
  • 2008 – A few more customers request the feature.
  • 2008 25th October – Amazon reply again saying it is still on the list, but they have no date to announce.
  • 2008/2009 – A few more customers request bill capping on AWS.
  • 2009 23rd March – Amazon reply and say the feature has been calendered to appear late this year or early next year.
  • 2009 3rd June – Amazon reconfirm the late this year / early next year timeframe in reply to another poster.
  • 2009/2010 – Many more replies from customer requesting some kind of cost limiting feature.

What can Amazon do?

Firstly they could make it possible to reduce the allowed bandwidth and number of hits. As the connection is already throttled, and as it is possible to increase the values, I’m sure it wouldn’t be that hard for them to allow you to reduce the limits.

And secondly they could allow you to configure a cost limit per day/week/month. Perhaps with a warning when 70% of your bandwidth is used up for example. This is what many metered web hosts do if you are near their limit.

A combination of both of these would probably be a good solution for most people who are worried about their potential bills.

Also as I previously mentioned it would be good for them to have clear public policies of what would happen if your account were hit by a DDOS attack leading to an unusually large bill.

What can I do?

If you are concerened you can add your voice to the discussions happening on the AWS forums, or try to contact Amazon directly to request these cost control features.

You can sign up for a 3rd party AWS monitoring service, which could notify you if the costs go abover a certain amount.

And finally

I’ll stress again that this problem is by no means unique to Amazon. In fact I’m not aware of any utility comuting provider that allows you an easy way to limit your costs, or throttle your usage. Therefore it won’t help you if you to change to another similar company.

I’m not saying that you shouldn’t use these services; in fact they are very useful and usually work well with good reliability. Just be aware of the risks.

Most importantly check your AWS account very regularly to monitor your costs.

Update: 11th May 2012

Amazon have now introduced automatic billing alerts. These won’t stop your charges from going up, but at least you will be notified if your pre-set limit is reached. See http://aws.amazon.com/about-aws/whats-new/2012/05/10/announcing-aws-billing-alerts/.

Update: 8th June 2012

They have now introduced a billing data API http://aws.typepad.com/aws/2012/06/new-programmatic-access-to-aws-billing-data.html.

Still no way of forcibly capping the charges to a pre-determined amount, but they are at least providing more ways to monitor the charges.