Wigitize.com, Post-Digg

on January 09, 2008

Yesterday I hit the Digg front page with my Building a .com in 24 hours article. Apart from that I was also number one and number nine in the delicious rankings. Very exciting!

What is also interesting is that my rapidly build website – wigitize.com – got a lot of hits. My blog got about 25000 uniques, Wigitize about 10000. All these hits caused an incredible load on my small 256MB memory server so I quickly bought a second one and scaled up. Yesterday, I could finally stabilize the backgrounding on the website and everything is running fine ever since. Someone offered me a free slicehost slice too last night!

It’s also funny that people commented on how I must have too much time on my hands. I even got offered participation in startups and companies haha. In fact, the opposite is true, but I do love hacking away in my free time. Besides, it was xmas, that means coding, drinking and eating right?!

While combating the performance issues I had yesterday, I learned these technical lessons:

Make sure you’re running the right version of tools. Apparently I was running a very old backgroundrb version. So I now upgraded to Ezra’s 0.2.1 version

Debug things in production-mode on your local machine! Some code – especially funky code like backgroundrb – will behave differently on production as opposed to dev. Often people like me are strictly developing in ‘development’ mode and than publishing to staging/production environments. I found a mayor issue that was screwing up everything in backgroundrb by debugging things in production mode on my MacBook.

Regular expressions can be dangerous too! Sometimes Feed detection workers started eating up 100% CPU. After debugging things for a long time, I found out that my regular expression was going berserk on sites like msn.com. So I changed expressions in the feed detecting algorithm from: 
1

/<link.*href=['"]*([^\s'"]+)['"]*.*application\/rss\+xml.*>/.match(html)
to: 
1

/<link.*href=['"]+([^'"]+)['"]+.*application\/rss\+xml.*>/.match(html)

Wigitize flaky due to Scaling Issues

on January 08, 2008

Update: Last 18 hours I have been combating performance issues because I got digged. Things seem to work more stable now, but still the background processing fails on me sometimes. Good lessons are being learned here so I will wrap that in a nice article soon!

Update: I just added another 1 GB slice, that should get us going for a while. Now just have to wait for the damn DNS entries to update.

I am preparing a new machine as we speak. Wow, did not to run into these luxury problems already! Thanks for the support!

If anyone thinks of buying a slicehost.com slice, please use this referral URL to donate:

https://manage.slicehost.com/customers/new?referrer=129028580

Machine 1 (256mb memory, how I started out with 2 days ago):

Machine 2 (1024mb memory) new