Posts

Showing posts from November, 2011

It's all about timing baby!

Image
Image: Suat Eman / FreeDigitalPhotos.net Yeah, it's about timing. There was a problem in one of our projects at moove-it related to slow processing. There is a daemon spawning new threads to process certain new entries to the database. The entries come from a different system, that's the reason for this program which processes each new entry. Sometimes at certain hours of a day there are peaks in the entries to the database and the process will fall behind by about 20.000 entries or more. So we started analyzing the code to understand what was happening and why it took so long. We noted that each new thread the daemon spawned took about 5 seconds to complete its task. As we narrowed the measurement we came up with some code that took 5 seconds to execute but it only involved access to the database. So thanks to Aaron Patterson's ( @tenderlove ) talk at RubyConf Uruguay about "Who makes the best asado" where he talked about rails and how it manages threads