Hey SG, a new update to our servers went live today. In the past couple of weeks our web server has been overloaded during peak hours. It hasn't been too severe, but you might have noticed slight delays at certain times of the day. If you view the attached graph, you can see CPU usage on our web server throughout a few days last week. It dips below 50% usage during off-peak hours, and starts to hit 100% usage during peak hours.

This represents two problems. First off, we need less resources during off-peak hours, and secondly, more resources during peak hours. This brings us to the world of load balancing. As of today, we're now running a load balancer and multiple smaller web servers. They're 1/4 the size of our previous web server, but we'll be running between two and twelve of them at any given time. With our current traffic, we'll likely see three web servers running during off-peak hours and six during peak hours. This helps to reduce costs when the site is less busy, and allows resources to automatically scale as traffic increases throughout the day.

Also, our scheduled tasks have been moved to a separate smaller server. These tasks include such things as generating winners, calculating contributor levels, distributing points, pulling data from the Steam API, and more.

If you view the second attached image, you can see a simple concept of how this works. When you visit the site, you'll connect to the load balancer, which sends you to one of the available web servers, which then connects to our database server.

With our current setup, we should be able to scale to three times our existing traffic, and then our database server will need to be upgraded. I'd like to invest some time further growing the site this year, and the new setup will help to give us the flexibility to do so.

View attached image.
View attached image.
8 years ago*

Comment has been collapsed.

Thanks for all your work! :D

8 years ago
Permalink

Comment has been collapsed.

Problem with browser like Avan after Server update

  • Can not finde the Server url on Mozila emulated
  • Ip (bann) blocked with Edge emulated. (Your IP adress is teporaryly blocked)

I can only use steamgifts.com with normaly borwser wich including:

  • iD sharing
  • no add block
  • no maleware block

so think now you can only use steamgifts with sharing youre id and take adds -.-

Click here for a Poll

8 years ago*
Permalink

Comment has been collapsed.

Thanks!

8 years ago
Permalink

Comment has been collapsed.

Thanks!

8 years ago
Permalink

Comment has been collapsed.

Stop repeating everything I say.

8 years ago
Permalink

Comment has been collapsed.

I thought it would have been blocked automatically?

8 years ago
Permalink

Comment has been collapsed.

Right? I don't know why it does this sometimes.

8 years ago
Permalink

Comment has been collapsed.

Right? I don't know why it does this sometimes.

8 years ago
Permalink

Comment has been collapsed.

Right? I don't know why it does this sometimes.

8 years ago
Permalink

Comment has been collapsed.

Right? I don't know why it does this sometimes.

8 years ago
Permalink

Comment has been collapsed.

Right? I don't know why it does this sometimes. :3

8 years ago
Permalink

Comment has been collapsed.

Ctrl v

8 years ago
Permalink

Comment has been collapsed.

Ctrl c

8 years ago
Permalink

Comment has been collapsed.

We thank you for your servers :)

8 years ago
Permalink

Comment has been collapsed.

The bigger the better!

8 years ago
Permalink

Comment has been collapsed.

Well, site sure has grown big if you now need load balancing. :)

8 years ago
Permalink

Comment has been collapsed.

Couldn't login with Firefox, but I suppose that's on purpose, cause I use an adblocker plugin with that browser...

edit: not login, but actually connect to steamgifts.

8 years ago
Permalink

Comment has been collapsed.

Thank you for all your hard work :)

8 years ago
Permalink

Comment has been collapsed.

Good to know, I remember the dreaded day when I couldn't enter any GAs because it would just get stuck loading. The horror...

Tnx for putting the time and work into avoiding any further potential issues.

8 years ago
Permalink

Comment has been collapsed.

Nice, thx!

8 years ago
Permalink

Comment has been collapsed.

nice. :)

8 years ago
Permalink

Comment has been collapsed.

Nice to know about the upgrade :)

8 years ago
Permalink

Comment has been collapsed.

Thank you, cg. I was wondering, just in terms of a cost analysis, will you be billed on a per usage basis with the new topology? How much of a savings per month, with the current userbase, do you expect to have? I'm assuming you've been footing the cost of operations yourself all this time. So again, thank you!

8 years ago
Permalink

Comment has been collapsed.

The site is running on AWS, so the individual servers are billed per hour, or they can be purchased for one or three year terms at reduced prices. I'll likely purchase 3 or 4 web servers upfront for the year, since they'll be online the majority of the time. Any additional servers will then be billed per hour, which will be more cost efficient since they'll be online less than half of the day.

I don't expect to see any savings with the current traffic. The little we save during off-peak hours will be offset by extra costs during peak hours. The main advantage is being able to scale. If a developer gives away 100,000 copies of a AAA game and it goes viral, our web server isn't going to be overloaded, but we'll have more servers automatically start up to meet our needs. Then when that giveaway is over, the unnecessary web servers will terminate themselves.

Keep in mind the web servers are a small cost, and they're less than 20% of the total expensive. The majority is the database server (55%), and bandwidth (25%).

8 years ago
Permalink

Comment has been collapsed.

Thank you. Interesting, the load balancer evaluates when more servers are needed and it distributes the work between them, is right?

8 years ago
Permalink

Comment has been collapsed.

Yes, and that part is actually critical, because it can mean costs ramping up quickly if it behaves poorly.
Also, if it's not fully custom, Amazon will try to scam you all they want, including network usage.

8 years ago
Permalink

Comment has been collapsed.

Thanks.

8 years ago
Permalink

Comment has been collapsed.

Really interesting to see all that stuff. Thanks for the upgrade :)

8 years ago
Permalink

Comment has been collapsed.

Thank you cg! :D

8 years ago
Permalink

Comment has been collapsed.

awesome, thanks for fantastic work you are doing :)

8 years ago
Permalink

Comment has been collapsed.

nice :)

about the 1st graphic though, what's the reference time-zone?

8 years ago
Permalink

Comment has been collapsed.

UTC.

8 years ago
Permalink

Comment has been collapsed.

Thanks cgod!

8 years ago
Permalink

Comment has been collapsed.

I hope it helps and does not go into the undesired direction (more costs).

8 years ago
Permalink

Comment has been collapsed.

awesome! Thanks cg! :3

8 years ago
Permalink

Comment has been collapsed.

Great! THX

8 years ago
Permalink

Comment has been collapsed.

Thanks!!!

8 years ago
Permalink

Comment has been collapsed.

Great!

8 years ago
Permalink

Comment has been collapsed.

Thank you Steamgifts so much for your generosity and hard work. <3

8 years ago
Permalink

Comment has been collapsed.

Sign in through Steam to add a comment.