Monetize
Affiliate Marketing, Search Engine Obfuscation, and Internet Profiteering

Can Your Site Survive a Digg?
Saturday June 24th 2006, 6:22 am
Filed under: Random Things

At its peak, a front page story on Digg will send dozens of visitors per second to your website. Is it ready for the traffic?

Having a link on the front page of Digg.com presents websites with one of the fastest influxes of traffic possible. Similar waves of traffic might come from Slashdot.org or a national media website like HowardStern.com. When the story first hits, you will receive hundreds of uniques per minute, all grabbing the same page. How can you prepare your website to stand up to the crushing wall of visitors? Follow below for a few easy steps that you can take either before or after being ‘Dugg’ to make sure that your site stays online and responds quickly to each page request.

 The Digg Effect
Traffic can be substantial and last for days as other sites pick up the link.

In no way does the author condone the manipulation of Digg or other social bookmarking sites by attempting to promote stories that are off-topic or profit-driven in nature.

1. Change your dynamic page to flat HTML. If the page visitors are landing on is written in PHP, ASP, Perl, CFM, or anything besides HTML, change it. Simply go to the page and select (in Firefox) File > Save Page As. Put the HTML page into a new directory on your site and make sure that is the page people land on. You will have to edit the HTML to reflect the new location by making sure that the images and CSS still load and the links still work. If the landing page is a blog post or another interactive page, people will still be able to post comments if the form is targeted, but you will have to occasionally move the comments from the live version to the flat HTML version manually by viewing the source and copy-pasting.

By removing the scripting language and database from the page load, you’ve probably done enough to buy yourself breathing room. If you can only do one thing to prepare, do this.

2. Move your images and CSS to a different hosting account. If you have more than one hosting account, you can spread around the bandwidth usage by relocating the CSS and image files. Not only will you save on bandwidth by possibly avoiding overage charges, you will also speed up the load time by distribution. Remember, you only have to do this for the main page on which people are landing; you don’t have to make the changes site wide as most visitors will only view the first page.

3. Reduce the number of images on the page. Regardless of how many bytes the image is, even if it is 1 x 1 pixel, it still takes an additional HTTP request for each image to load. Many people don’t realize it isn’t the size of the images that will bring you down in a Digg, it is the number of images. If you have a page that is 1 HTML file, 1 CSS file, and 4 images for the entire page, that is a total of 6 HTTP requests per page load. If you have a glossy photoshop-cut-chrome-stylin’ layout, you likely have 30 or more small images per page. Dozens of HTTP requests per page load will bring you down… avoid it. If you have to change the layout of the landing page to remove the excess images, do it. It makes that big of a difference!

4. Notify your host. If you see your story with 30+ Diggs about to be pushed to the home page, now is the time to notify your hosting company, not after. This blog was Dugg twice last week; it only went down once and that was because the host took it down suspecting a DDOS attack! Warning your host to expect a huge influx of traffic is the right thing to do. While you’re on the phone, ask about your bandwidth situation. If you need to upgrade your hosting account to avoid overages, ask if you can barter the upgrade for a ‘hosted by’ link on your soon-to-be-famous page. I got a free account upgrade from Primehost last week simply by asking and putting a text link in the blogroll.

What to do if you’re already being Dugg? If you are blindsided by the traffic because someone else submitted your page, don’t panic. You can bring the site back up after a crash. First, find out what page is being Dugg and start from step 1 above. Second, you’ll need to redirect the Dugg page to your new flat HTML version. If your page is normally in PHP, it is as simple as replacing the Dugg PHP file with a header(“location:”) redirect to the HTML. You may have to call your host and ask for a reboot. By performing the steps above, you will be able to keep any page online during a Digg, even using the cheapest shared hosting.


 Digg it  Add to del.icio.us

34 Comments so far
Leave a comment

This great advice. I haven’t had anything of mine make it to the home page of Digg, but you never know.

I’ve actually seen a WordPress plug-in that does some sort of caching so that pages are not built dynamically for every request, but the set up seemed like too much trouble. Of course, like those in the US who fail to get health insurance, I may regret not looking in to the plug-in further.

Comment by Marios Alexandrou 06.24.06 @ 7:42 am

Great advice!
And this deservs to be bookmarked:).

Comment by vibhash 06.24.06 @ 9:23 am

Yep great advice…. I doubt I’ll ever have that problem but I’m gonna keep this info handy just in case…

Thanks for sharing…

Comment by Keith L. Dick 06.24.06 @ 11:33 am

I always enjoy my time here. Thanks.

Comment by Jamie 06.24.06 @ 11:40 am

I’ve had several sites dugg before, and I’ve never had to do any of the things you’re suggesting. All you need is a good host and smart code. Pair and TextDrive both host sites of ours in a shared hosting environment that have been dugg, and all of the sites were running PHP and had dynamic data from a MySQL database. If the code is horribly inefficient and the hosting provider is sub-par, then yes, you better do what you suggested, otherwise you’re going down.

Comment by Jon Henshaw 06.24.06 @ 1:02 pm

There’s a dialogue from a play that goes: “Can you dig it? I knew you could.” I have to try to find the play.

Anyway, I dig your good advice!

Come back to ABW — your posts always add sizzle :)

Comment by Rhia 06.24.06 @ 3:45 pm

WP-Cache will make a cache of recently requested pages and you could tweak it so that it will maintain the cache for some time (handy if you are being dugg at that moment), and set-up is really easy. Really easy. It’s useful even if you have low traffic because it speeds things up just flicking between pages; as long as you don’t NEED gzip all the time, I’d say go with it. It only stops the loop from running all the time, of course–there is still http requesting and rendering to be done.

Comment by Sam Jackson 06.26.06 @ 2:23 pm

I’ve submitted this story to digg. Hopefully your site will hold up, aha.

Comment by Rick 06.26.06 @ 2:25 pm

Great advice. I hadn’t thought about notifying my ISP. I will be writing their number down in my wallet in case I can’t get on to the admin page…
Thanks
Jambell

Comment by Jambell 06.26.06 @ 2:31 pm

I have had a couple popular ones. No where near the 1000 diggs, But even a hundred or so had me paying more for my limited bandwidth. The site got nuked a couple of times. Though, I made no revenue I did manage to get on Del.icio.us.

I still get great traffic from one of my articles that made it to the front page of digg.

Comment by elamb 06.26.06 @ 3:07 pm

I think Wordpress already has built-in caching, but maybe we need a plugin that can generate a static version of the site. Remember MT?

Comment by Rami Kayyali 06.26.06 @ 3:16 pm

If the page that is already linked to on Digg is dynamic – the best hope may be to place a link to redirect to a static URL… ASAP…

Because Digg users will report a flaw in the blink of an eye, and Digg’s software automatically Kills the Link submission after a certain amount of reports.

And some members have several accounts so one report can bloom into several depending on who is reporting it

But this is a bitter-sweat tactic – because – Google spiders Digg’s homepage voraciously, so in essence you have lost the value of a great Back link that would have helped your page on the SERPs – long after it has ceased being Digged

Comment by Search Engines WEB 06.26.06 @ 3:18 pm

Excellent write up.

Comment by GaryWilson 06.26.06 @ 3:19 pm

Great article – wish I had this problem ;-)

Comment by Tim 06.26.06 @ 3:25 pm

WoW and I was just going to submit my site to digg. I need to work on lowering the size of my homepage then ^^.

Comment by resten 06.26.06 @ 3:35 pm

Great advice indeed. And if you’re more tech-savvy or have a geek friend, ask (probably) him to implement a very basic caching (pre-parsing to HTML if nothing is changed, or in heavy traffic make it half a minute or so..) there are a lots of tutorials on this and for blog engines you can find suitable plugins.

And of course, write/make/present/do something in order to be digged :)

Comment by PAStheLoD 06.26.06 @ 3:36 pm

Great article! You are #8 on the front page right now! I hope you follow your own advice. :-)

Comment by WiredDeals 06.26.06 @ 4:00 pm

If you have the cash, simply invest on a better server like a Pentium D 940 2GB RAM. Such server config can suvive any types of digg.

Comment by Ian 06.26.06 @ 5:06 pm

I don’t worry too much about my site to go down, even if it is shared between 400 users. I guess one should always been cautious, but I’m pretty confident when it comes to my site being dugg. I’ve been dugg once, which gave about 5000 uniques in 24 hours, showing no trace of server stress within that time. Then again, I use quick loading layouts, and custom dynamic pages for quick loading. (How many sites load in 2 seconds on 56kb dailup?)

I’ve always wondered if it is really worth it to put my css in a separate stylesheet. It’s an extra request, but it gets catched by the visitor.

I never thought of contacting my host when something like this could, or is happening. It’s smart. I’m glad I read this article. Thanks.

Comment by Slythfox 06.26.06 @ 5:08 pm

Your tutorial is now dugged LOL :)

Comment by Wattie 06.26.06 @ 5:20 pm

This is why I have my blog hosted by blogger.com, I worry about blogging, and getting my blog linked, and blogger.com worries about the traffic, they do what they do best, and I do what I do best. And best of all, blogger.com is FREE!!!

Comment by James 06.26.06 @ 6:55 pm

>>This is why I have my blog hosted by blogger.com

Good idea; as long as you use your own URL so you don’t ever worry about losing the site and incoming links.

Comment by Alex 06.26.06 @ 7:00 pm

The Free/Open Source (GNU GPL) program dijjer can also help distribute the bandwidth:

http://dijjer.org/

You’ll have to setup your site to use dijjer, and people would also need to install it. It’s like automatic bittorent for http/html.

Comment by Dijjer 06.26.06 @ 7:00 pm

Good hints, but what if your website requires dynamicly generated pages? My website is sort of social network / community site. How do I know if I have enough juice to handle the dynamiclly generated traffic?

Comment by Event Zip 06.26.06 @ 7:13 pm

>>what if your website requires dynamicly generated pages?

Usually all that will matter is the landing page. Make sure the traffic goes to a static page; having a home page that does not require database access would be generally beneficial for almost any site.

One way to do this is to write out datafiles to the web server every few minutes or when certain things update. So when someone makes a new post or creates a new profile, the DB writes out a datafile into the file system (ie, by serializing an array of items). The web server uses that datafile to pullinfo to the home page so that the database server is not needed. This sort of data delay is common and lets your home page get hit incessantly without causing undue load.

Comment by Alex 06.26.06 @ 7:30 pm

 what if your website requires dynamicly generated pages?

Then you use this tutorial instead.  I’ve survived three FP diggs and kept the site dynamic the entire time with this method.

Comment by Adam Knight 06.26.06 @ 8:18 pm

Great advices thanks!
looks like digg users love your site :D

Comment by Mike 06.26.06 @ 11:47 pm

It would be nice to have pages written in HTML, however if you have thousands and millions of pages, it would be difficult to keep track of HTML pages, and especially if you require the template every now and then. Dynamic pages are, of course, the only way to go. Furthermore, I do not see Digg having much problem with dynamic pages.

Comment by Keith 06.27.06 @ 1:31 am

These are nice tips. Found more related information on how to survive slashdotting, which might interest others.

Comment by jari 06.27.06 @ 5:29 am

Developing with RubyonRails allows you to set static comment as a feature of the framework. It’s as easy as specifying which parts of the page you want as static or dynamic.

Really interesting food for thought here.

Comment by Heath Weaver 06.27.06 @ 6:39 am

This is great advice. I wished I had read it before I was dugg! I couldn’t survive being dugg at Tetsou when my article ‘ 7 deadly sins of email’ was posted. And this was the first of my articles to have been posted. Within hours I had floods of traffic hitting my page which is dynamically served by PHP.

As the server slowed, my provider notified me I was under a DOS attack! Ha ha. I explained about the Digg article and we tried to shore the server – but to no avail as my articel quickly hit the frontpage.

Be careful what you digg for.

Tetsou

Comment by Tetsou 06.27.06 @ 9:51 am

I think Wordpress already has built-in caching, but maybe we need a plugin that can generate a static version of the site. Remember MT?

The build-in caching is disabled by default and isn’t a proper output cache… it’s just used for caching of frequently used PHP objects, like user, page, and category information. If you want to survive the Digg effect with WordPress, you need the WP-Cache2 plugin, which caches the HTML output of pages. I’ve pushed sites to well over 1200 page views per second with this, and other tweaks, using WordPress. Digg and Slashdot together at the same time aren’t even going to come close to hitting that.

Comment by Mark Jaquith 06.27.06 @ 8:39 pm

Or, if you run Apache, you can prepare your site to automatically redirect visitors coming from Digg, Slashdot etc. to go through Coral Cache:
http://www.bigbold.com/snippets/posts/show/1957

Comment by Anonymous 06.28.06 @ 9:47 am

[...] don’t try this Digg experiment at home before you read Monetize’s “Can Your Site Survive a Digg?” The piece predicts that the tidal wave of hits [...]

Pingback by FYI News: Dulles Technology Corridor Residents are Digg Diggin’ Slate’s Redesign « EverythingAshburn.COM loves Brambleton 06.20.08 @ 10:14 pm



Leave a comment
Line and paragraph breaks automatic, e-mail address never displayed, HTML allowed: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>