Bridging Rails to Amazon SimpleDB using ActiveResource

January 20th, 2008

With SimpleDB, Amazon added the long-awaited database-like service to it’s web services portfolio. A database was the one thing missing for building a complete web hosting stack with Amazon’s services.

Using SimpleDB together with Ruby on Rails was the thing I immediately wanted to try. Some problems had to be dealt with first:

  1. Amazon does not provide any Ruby code for SimpleDB access
  2. SimpleDB has quite some limitations as detailed in a previous post

As Lars Schenk outlines on his Blog at least three different projects on RubyForge are addressing problem number one - only one of them has actual code, though. And that is aws-sdb by Tim Dysinger from Hawaii.

OK, using Tim’s aws-sdb gem, one can get access to SimpleDB using Ruby. But using SimpleDB as a drop-in-replacement for a relational database and connecting Rails’ ActiveRecord to it would require a fairly complex adapter - I am not sure, if this can be done at all, actually.

Looking at Rails, another interface comes to mind: ActiveResource. ActiveResource was written to connect model objects to RESTful web services as their datastore. That sounds like a fit. The APIs are different, but the functionality needed by ActiveResource can be provided by SimpleDB - it’s all just CRUD after all.

All that’s needed would be a adapter, a proxy, a proxy server? Yes, that’s right. As it turned out, it’s not too hard to write one, so here it is :-)

Announcing AWS SDB Proxy

My AWS SDB Proxy is a HTTP proxy server bridging ActiveResource calls from Rails to Amazon’s SimpleDB Web Service allowing it to be used as a storage backend for Rails applications.

The proxy will listen for web service calls initiated by ActiveResource models and forward the requests to SimpleDB using the aws-sdb gem.

Install the AWS SDB Proxy Plugin from RubyForge as usual:

script/plugin install http://rug-b.rubyforge.org/svn/aws_sdb_proxy

Then follow the instructions provided in the README.

Features and Limitations

SimpleDB (and thus AWS SDB Proxy) do not use any pre-defined schema. Every record can potentially have different attributes. SimpleDB also has no data types associated with it’s attributes, all data will be stored as strings.

AWS SDB Proxy adds a special _resource attribute, allowing storage of multiple models within the same SimpleDB domain. Record ids are generated using a SHA512 hash function to make key collisions extremely unlikely.

Recommend Martin Rehfeld on Working With RailsIf you like this plugin, please consider recommending me on Working with Rails. Thank you!

Related Posts

Entry Filed under: Tools, Announcements, Ruby on Rails, Amazon Web Services, Howto

12 Comments Add your own

  • 1. Ben  |  January 21st, 2008 at 12:16 pm

    Actually, there is another project with simpledb code. It was released almost immediately by a developer from the NYTimes:

    http://rubyforge.org/projects/nytimes/

    I’m excited about your plugin, i’ll have to try it out.

  • 2. martin.rehfeld  |  January 21st, 2008 at 12:48 pm

    @Ben: The nytimes SimpleDB wrapper looks really promising. I’ll sure take a look at it as I’m currently not 100% satisfied with the aws-sdb gem, either (had to monkeypatch it a little).

  • 3. martin.rehfeld  |  January 21st, 2008 at 3:28 pm

    Quick update: I just gave the amazon_sdb gem by Jacob Harris (NY Times) a test drive and sadly must assess that the released gem 0.6.5 is a real PITA - missing source files and missing methods all over (svn version looks better). The API is much richer than aws-sdb’s by Tim Dysinger, though. Let’s wait and see how they both evolve…

  • 4. Tim Dysinger  |  January 21st, 2008 at 8:15 pm

    I purposely kept the API simple (heh no pun intended) so as to not presume how people would want to use SDB.

    Creating Item and Domain objects had crossed my mind but I wanted the initial release to just be access to SDB - not trying to do anything fancy on top yet.

    More later…. :)

    Heh martin - if you have patches then let’s have them! I still have to do the rdoc and more polish.

  • 5. Tim Dysinger  |  January 21st, 2008 at 8:46 pm

    I opened the aws-sdb project up a bit today and added a request for developers.

  • 6. martin.rehfeld  |  January 21st, 2008 at 8:51 pm

    @Tim: Just sent you an email with my one-liner ;-)

  • 7. Josh Adams  |  January 22nd, 2008 at 12:37 am

    Tim,

    You know Jamis has decried Needle as unnecessary? http://weblog.jamisbuck.org/2007/7/29/net-ssh-revisited

  • 8. martin.rehfeld  |  January 23rd, 2008 at 8:41 pm

    Tim has just released version 0.1.2 of the aws-sdb gem. As Amazon will discontinue the API version used in version 0.1.1 shortly, a update is highly recommend. 0.1.2 is working great with AWS SDB Proxy.

  • 9. dash  |  September 18th, 2008 at 6:06 pm

    Hi,

    When I try to execute “rake –trace aws_sdb:create_domain DOMAIN=my_test_domain”, I get the following error:

    ** Invoke aws_sdb:create_domain (first_time)
    ** Invoke environment (first_time)
    ** Execute environment
    ** Execute aws_sdb:create_domain
    rake aborted!
    wrong number of arguments (3 for 1)
    /Users/dharmesh/NetBeansProjects/FooPetsTranslations/vendor/plugins/aws_sdb_proxy/lib/aws_sdb_proxy/server.rb:27:in `initialize’
    /Users/dharmesh/NetBeansProjects/FooPetsTranslations/vendor/plugins/aws_sdb_proxy/lib/aws_sdb_proxy/server.rb:27:in `new’
    […]

    Please advise. Thanks in advance!

  • 10. martin.rehfeld  |  September 18th, 2008 at 7:34 pm

    @dash/#9: This post is fairly old, so it might not be in sync with later developments. Please try to use aws-sdb 0.1.2 instead of the current 0.3.1 and see if it works ok with that.

  • 11. Kelly  |  February 15th, 2009 at 10:48 pm

    Do you have any plans to update this plugin to keep it in sync w/ aws-sdb gem? I get the same error that @dash got. I’d really rather not roll back to the old gem version. It looks like there have been a lot of bug fixes since then.

  • 12. Martin Rehfeld  |  February 17th, 2009 at 10:18 am

    @Kelly/#11: Not really, I’m afraid. Think of AWS SDB Proxy more as a proof of concept. aws-sdb 0.1.2 works fine for that purpose.

Leave a Comment

Required

Required, hidden

Some HTML allowed:
<a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <code> <em> <i> <strike> <strong>

Trackback this post  |  Subscribe to the comments via RSS Feed

Most Recent Posts