Digg effect vs Yahoo front page listing

Many website owners are dreaming about hundreds of thousands of web visitors and thousands in ad profits, but will your web site handle the load and keep up the file serving? Probably not.

Digg frontpage listing usually sends 120.000  to 170.000 web visitors in two to three hours. It depends for how long you are listed on Digg’s frontpage and what is the topic.  In over 95% of cases this will probably be too much for simple dedicated server – if you are hosting WordPress blog with some plugins activated (that require some extra load) and without tuning the server box.

You can activate some caching plugins (wp-cache, wp-super-cache, 1blogcacher), configure more efficient content distribution (for example, use lightweight web daemon to serve static files), use less includes and optimize plugin code for performance. This will probably handle the Digg frontpage load.

What if you land Yahoo! frontpage listing? This will be a real challenge for your systems and system administrator. Yahoo! is very popular and if your site gets listed in frontpage they can easily “send you” a few million of web visitors. Once again, depends for how long the listing will stay on the front page. Yahoo! traffic burst will definitely make your web server unreachable and I am saying this with 99.9999% confidence. You need multiple servers to handle  the load and do load balancing, and of course have upstream bandwidth available to serve the content. You need to employ perfect caching architecture, preferable keeping the page 100% static and load it in the server memory for super fast serving to web visitors. If you read it from the hard drive – your servers will experience very high IO and load, thus slowing down web responses.

Last time when one of our client’s got listed on Yahoo! frontpage the site (actually a blog) got around ~900.000 unique visitor landings, over 200.000 visitors browsed site further reading other articles and comments.  The traffic burst from two web front-end servers serving the content was close to 195Mbps (2 x 100Mbps links were fully maxed out) and I am talking about serving HTML/PHP content, not the images from these servers.

This is why you need to plan architecture to be scalable (horizontally) and easily affordable. One server box will not make it even if you get top-notch mega box for 20k USD, period.


Comments

  1. Bob says:

    Remember that your server 100Mbps up-link will be quite saturated depending on your page and static content size – images, css and javascript. Keep it simple. Keep it cached.

  2. John says:

    Offload images and CSS serving to a CDN and you will increase your site performance

Leave a Reply