Pagination with ActiveResource

Ok, so we have a RESTful web service written in Ruby on Rails which has thousands of records to display. Viewing them all at once times out the client badly. What do we do? Pagination comes into the rescue, of course.

How do we accomplish that?

First try:
We added will_paginate to both the REST server and the Rails client. We render using the erb to include the total_entries and per_page attributes required by the will_paginate in the client side.

Problems arise with this. Hash conversions extension of Rails only expects that an array type node has only one kind of child, and it's the list of what it represents. However, we put the two attributes into the returned array too. What happened? Since XML is parsed out as a hash in the first place, and hash is unordered. The conversions extensions will pick any value from the hash whose key is not 'type'. So, sometimes it gets the correct things, other times, it doesn't. The only error message it sends out is "can't typecast #{entries}" where entries is mistakenly the value of either our total_entries or per_page.

It worked in ALL of our entities we paginated, except the last one. With the same code, doing only one different thing in one different cases, I pulled 75 hairs out of my ugly beard. What could possibly go wrong? Is it the REST server? Is it the find? NO! They behave in other entities, except this one. ... ok... stack trace.. ok, meta programming, we can't go any further... oh my god... it's the conversions!!!

To plow ahead and go forward with this approach, we can change the conversions.rb to detect anything NOT 'type', 'per_page', or 'total_entries'. BUT my pair over here is very strongly against changing rails. So came his idea ...

Take 2:
Apparently it's quite hard to have an array with a custom instance method. So, we don't do it. On the REST side, we don't include will_paginate at all. On the client side, we call the REST like so:

Model.find(:all, :params => {:limit => blah, :offset => blah})

blah's are calculated according to which page it is and how many per page we want.

On the REST side, we have to process the params a little bit. The params are sent in with strings as keys, all we need to do is change them all to symbols.

Model.find(:all, {:limit => params[:limit], :offset => params[:offset]})

As you can see here, you can access the params using the symbol as the key, but it is actually stored as strings, and the error you see is:

can't recognize key limit, offset

Not very obvious, and I lost 20 hair of my beard figuring this out.
Also, you need an extra action on the REST side to tell how many record it has. Call that from the client and keep it as a param for the WillPaginate::Collection creation


This way, we get only what we need from the REST, the big half purpose of pagination. (The lesser half is the usability.) Now, we construct the WillPaginate::Collection like so:

@entities = WillPaginate::Collection.create(page, per_page, count, returned_array)

Finally, in the view, you paginate the @entities as you would a paginate collection you got from the ActiveRecord counterpart.

Thanks Arnold for coming up with this idea for me to blog and brag about solving this. LOL