Bash My Nonsense

Fun with ssh

2010-10-07T10:33:00.000-07:00

I was trying all morning to ssh into a server with the public key authentication. There's a page about troubleshooting the public key authentication. I suspected the server is so old it uses ssh 1.5, but no. I can't think of something else beside googling on the error message:

Agent admitted failure to sign using the key.

And there it is: a bug on ubuntu.

Indentation, indentation...

2010-04-08T17:19:00.001-07:00

So, lately I see many developers prefer what I call ASCII Art Indentation style such as:


Class Foo
  def bar(params = {})
    Barf.find(params.merge :key  => "value",
                           :key2 => "value2", ... and so on )
  end
end

with the claim that it reads better, and you know exactly where to look for things.

I say it's completely opposite. It is no where near intuitive.

Say you have to change params to parameter_name_that_makes_more_sense ? you have to adjust your whole code base to reflect your tiny change, and apparently no one does.

For something like:


  Sundae.new(Icecream.with_fruit.no_sugar.all(:include => [:nutrition_facts, :production_plan]), :whipped_cream => WhippedCream.no_fat.first(:conditions => ["make is not :make and calorie < :calorie", {:make => "Booga", :calorie => 120}]))

How do you systematically define the ASCII Art Indentation scheme that every one will follow and give exactly the same indentation? Where do you look for the make of the whipped cream? Not easy.

And for some reason, people loath my style of indentation: 1 more soft tab when you open a paren, brace, or bracket followed by a new line. and new line and one fewer soft tab before the close of the matching paren, brace, or bracket. It is said it's hard to see where things are, and it increases the number of lines. But behold:


  Sundae.new(
    Icecream.with_fruit.no_sugar.all(
      :include => [:nutrition_facts, :production_plan] 
    ), 
    :whipped_cream => WhippedCream.no_fat.first(
      :conditions => [
        "make is not :make and calorie < :calorie", 
        {:make => "Booga", :calorie => 120}
      ]
    )
  )

You can choose not to indent if your list is small enough, less than, say, 40 characters, half the 80x24 line.

You can write your code easily to indent stuff for you, consistently.

You can instruct people easily to indent the exact same way and give the exact same result.

You know exactly where you miss your close paren.

You know exactly where you can find your make of the whipped cream.

Your code rarely falls off the right edge while leaving an airstrip on the left side.

Prove me wrong, someone.

.not is EVIL!

2010-04-08T16:52:00.000-07:00

What's a .not?

.not is a decorator method that decorates the object, so that the boolean method call that immediately follows the not call is negated. For example, if self.is_italic? returns true, self.not.is_italic? returns false.

Why would people use .not?

The only reason I have heard so far is because it sounds more like English...Pffft, sounds more like English!

Person.find_by_name("Paul Frank").not.is_popular?

How is that anyway similar to English: Is Paul Frank not popular?

What's wrong with logical negation '!' ? The only reason I have heard so far is that it's at the head of the statement, and with long condition, it falls off the screen... Pffft, falls off the screen!

First, I don't believe in reinventing the wheel that is worse than the original one.
Second, instead of using a CPU to flip a bit, we malloc the heap, create an object, extend some other object with this object, which alias the method to use a CPU to flip the flipping bit!
Oh, did I mention it's stupid?

In addition to that, .not is broken. In any given object, you cannot call .not on the boolean variable.


Class Person
  @humorous = false;

  def foo
    @humorous.not #=> is not going to do anything!
  end
end

More over, what prevents someone from doing foo = bar.not, and a thousand lines later, some other call foo.is_real? without knowing that it is being 'notted'?

So you cannot get rid of ! anyway. The best you can do is use the not operator to avoid !, then you have precedence problem to think about.

People then say oh who cares, how many CPU time do you use over that? I'd rather have more style than looking like C.

Well, you see, you don't usually run around a building complex just to go from your desk to the break room 25 feet away, even though it doesn't affect one bit the history of mankind. You don't do that because it's stupid. Oh, and did I mention .not is stupid in exactly the same way?

! has been around since the dawn of computer programming. Whoever thinks ! is not readable should quit programming and do some research on herbal hair gel. If your condition is way too long for you to see ! on a 24" 1080p widescreen, it's a code smell that you should rethink your logic, or model design in general. Whoever thinks ! is ugly and remind them of C should appreciate their own breadth of knowledge, and that without C there is no .not, and the fact that languages are means, not the goal.

Remind yourselves to always do the simplest thing that makes sense. .not is not simple, and not making much of a sense either.
\No new line at the end of file.

Uploading a file.... from browser to a webapp, to another webapp ???

2009-01-22T12:01:00.001-08:00

It's been six and a half months since I last do anything interesting, but here it is.

Problem: We have a RESTful web service, and a web application that talks to each other. We want to allow the user to upload a file, particularly binary, to the web app, and put that in the database in the REST server.
- We can't make a multipart request from the app to the REST server, it will come out as a string literal "".
- We can't call read on the UploadedStringIO and put that into the request from the app to the REST server. It will come out as gibberish string.
- We can't put the specific header that we want to do a multi-part request. I didn't even dare track that down.
- I was going to try make a raw HTTP request, saving the file in a temporary file on the app somewhere, but before I did that, I realized something...

IT'S ALL XML!

When we make requests to REST server using ActiveResource, the request body is marshaled into xml format. That means all your binary is going to mess up the xml document, and/or recognized as the string literal above.

What do I do?

A multi-part stream has a few important attributes: original_filename, content_type, and the file data we can get by calling read on the IO object. So I created a hash with these attribute names as keys (call the content whatever you like, make sure your REST server knows what it is and fetch it accordingly), and the value from the user's request as the value. I encode the binary content with the Base64.encode64. Now this hash can be marshaled easily. It's all ASCII text. So, go ahead and put that in your request parameters and fire away.

On the server side, handle the request as you would normally, except when you write the file to your storage, remind yourself that it is encoded by Base64.

Now comes the other way around, showing what we just uploaded. Again, it's all xml up until it reaches the app. So, fetch what you stored, make sure it goes out the REST server with the binary content encoded with Base64, and decode it at the app when you're ready to present it on screen.

This works pretty well with ActiveResource. A nice feature request to ActiveResource people would be to automate all this. :)

Asymmetrical database structures between production and non-production environments

2008-06-03T12:23:00.000-07:00

So long time ago, before I was born as a consultant, there was some specific needs for the database tables. What we need was a table to mock external databases that we don't have access to in the development environment but the production does. What we did was creating a table right in the test cases and deal with it there.

Problems followed.

For one, MySQL and Rails doesn't have support for nested transaction.
For two, All tests written are in transaction that is rolled back every time the method ends.
For three, we need to ensure that the implementation employs the transaction mechanism somehow.

Whenever we create a new table, the transaction stops working. Whatever we did to the database, it was showing in the later test methods. Therefore there is no way we can ensure that when the transaction was supposed to be rolled back, it was indeed rolled back.

Solution:

The best we could do at that time was clearing out the tables ourselves. Andy Kotlinski wrote an assertion called assert_in_transaction to work around the problem. Talk to him if you would like more information on it.

Several months passed. The project was shelved for whatever reason.
Yet other months passed. The client decided to dig the zombie out of the grave.

All hell broke lose.

Somehow, Rails 2.0 doesn't allow us to work around it the way we did any more. The reason was something I couldn't remember any more because I forgot to blog about it at the time of digging that grave. What do we do?

Solution:

Instead of creating the tables in the test methods, I thought of a different way.

I want to have the transaction working.
I hate putting Data Description Language into the tests.
I like the "Elegance: Simple and Powerful", quoting Dr. David Matuszek from the University of Pennsylvania.

It turns out that we completely overlooked the power of Ruby. Database migrations are Ruby files. It means we can do pretty much anything a Ruby file can do. So we employ a simple condition to the migration


if RAILS_ENV == 'test' 
  create_your_fancy_test_tables
else
  do_some_funny_stuff
end

Not bad, but still, not good. Why? Because we also want it in the development database too. And what happens when you have 8 different environments and you have to hand pick them?

Take two:


unless ['production'].include? RAILS_ENV do
  create_your_fancy_test_tables
else
  do_some_funny_stuff
end

Better. But still, what happens when you have 20 different migrations ahead?

Take three:

Define a module like so:


module SkipEnv
  def skip_env
    unless ['production'].include? RAILS_ENV do
      yield
    end
  end
end

Put the module in the lib directory. Then, in the migration:


class << self do; include SkipEnv; end

def self.up
  skip_env do
    create_your_fancy_test_tables
  end
end

def self.down
  skip_env do
    drop_your_fancy_test_tables
  end
end

Hmm.. something's missing: the do_your_fancy_stuff. It turns out, our do_your_fancy_stuff does absolutely nothing, so we just cut it out.

It has to be a class method because the migration up and down methods are class methods.

Alright, looks good. We can create test tables anywhere except production, or any other environments you put into the array, and we can choose to skip environments in anything having access to the lib folder of the rails app, not only migrations. Awesome.

Further refinement would be to put it as a plugin, and install it in any Rails app you happen to have the need for such mechanism.

We didn't do that, because the zombie was buried back in the ground... again. This time, the reason is that to fix it and make it works for Rails 2 and have ActiveResource support, we would need to take a month and work on it full time.

Pagination with ActiveResource

2008-05-08T12:43:00.000-07:00

Ok, so we have a RESTful web service written in Ruby on Rails which has thousands of records to display. Viewing them all at once times out the client badly. What do we do? Pagination comes into the rescue, of course.

How do we accomplish that?

First try:
We added will_paginate to both the REST server and the Rails client. We render using the erb to include the total_entries and per_page attributes required by the will_paginate in the client side.

Problems arise with this. Hash conversions extension of Rails only expects that an array type node has only one kind of child, and it's the list of what it represents. However, we put the two attributes into the returned array too. What happened? Since XML is parsed out as a hash in the first place, and hash is unordered. The conversions extensions will pick any value from the hash whose key is not 'type'. So, sometimes it gets the correct things, other times, it doesn't. The only error message it sends out is "can't typecast #{entries}" where entries is mistakenly the value of either our total_entries or per_page.

It worked in ALL of our entities we paginated, except the last one. With the same code, doing only one different thing in one different cases, I pulled 75 hairs out of my ugly beard. What could possibly go wrong? Is it the REST server? Is it the find? NO! They behave in other entities, except this one. ... ok... stack trace.. ok, meta programming, we can't go any further... oh my god... it's the conversions!!!

To plow ahead and go forward with this approach, we can change the conversions.rb to detect anything NOT 'type', 'per_page', or 'total_entries'. BUT my pair over here is very strongly against changing rails. So came his idea ...

Take 2:
Apparently it's quite hard to have an array with a custom instance method. So, we don't do it. On the REST side, we don't include will_paginate at all. On the client side, we call the REST like so:

Model.find(:all, :params => {:limit => blah, :offset => blah})

blah's are calculated according to which page it is and how many per page we want.

On the REST side, we have to process the params a little bit. The params are sent in with strings as keys, all we need to do is change them all to symbols.

Model.find(:all, {:limit => params[:limit], :offset => params[:offset]})

As you can see here, you can access the params using the symbol as the key, but it is actually stored as strings, and the error you see is:

can't recognize key limit, offset

Not very obvious, and I lost 20 hair of my beard figuring this out.
Also, you need an extra action on the REST side to tell how many record it has. Call that from the client and keep it as a param for the WillPaginate::Collection creation

This way, we get only what we need from the REST, the big half purpose of pagination. (The lesser half is the usability.) Now, we construct the WillPaginate::Collection like so:

@entities = WillPaginate::Collection.create(page, per_page, count, returned_array)

Finally, in the view, you paginate the @entities as you would a paginate collection you got from the ActiveRecord counterpart.

Thanks Arnold for coming up with this idea for me to blog and brag about solving this. LOL