SameShirtEveryDay.com

Personal blog of the one called Alex Gorbatchev, from Toronto, Canada.

Distributed RDoc and a DRb tip

Posted on September 16th, 2007 by Alex Gorbatchev. In Ruby. No comments yet...

First distributed Ruby attempts

Noobkit is powered at its core by RDoc. Obviously it’s not real time, far from it. In fact, full refresh of Noobkit’s database which includes 26 Gems together with Rails and Ruby Core takes about 25 minutes on a single Core 2 Duo 2.4Ghz with 4GB ram box.

About 20 minutes of that time is taken by RDoc parsing the source code. This is what I’m currently working on making better. I have used this article as a starting point and wrote a client/server script on top of customized RDoc.

The end result is having 4 threads running across 2 boxes each parsing a separate source file fed to it by the server.

This being the first time I’ve worked with DRb or Ruby threads in general, one thing that kept annoying me was inability to break with Ctrl+C when using DRb.thread.join. I’ve devised a simple waiting loop instead:

def wait_for_it(&block)
  return if block.nil?
  loop do
    sleep(0.1)
    break unless yield
  end
end

# ...do DRb magic...

# This will stall the current thread until DRb thread is finished.
wait_for_it {  DRb.thread.alive? }

The same function can also be used to insure that all workers have finished their job before doing something else:

# When a worked is done, it's added back to the queue
wait_for_it { @available_workers.size != $workers.size }
You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.

No comments yet, be the first one!

Leave a Reply

Allowed tags: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong> , rel="nofollow" in use - no link dropping, no keywords or domains as names; do not spam, and do not advertise!

home
Subscribe to this blog Follow me on Twitter My bookmarks on Delicious My photography on Flickr