smallroomsoftware.com

Testing SSL pages (login pages etc) in development mode doesn't seem to be supported out of the box with Rails. Turns out that webrick includes an SSL server and it isn't too hard to set. You need this script (mentioned here). I changed the :ip setting and then saved it in my project as script/ssl_server. The trick is to run both the script/server and the script/ssl_server scripts at the same time. The problem is that you'll be using non-standard ports so the ssl_requirement plugin will fail to provide the correct redirects (it assumes ports 80 and 443). There's a ticket and patch about this but it doesn't look like it's going to be applied anytime soon. Let's just patch the code ourselves, replacing ssl_requirement's ensure_proper_protocol method.

def ensure_proper_protocol
  return true if ssl_allowed?

  if ssl_required? && !request.ssl?
    redirect_to "https://" + request.host + (RAILS_ENV == 'development' ? ':3001' : '') + request.request_uri
    return false
  elsif request.ssl? && !ssl_required?
    redirect_to "http://" + request.host + (RAILS_ENV == 'development' ? ':3000' : '') + request.request_uri
    return false
  end
end

So I ran into another problem with my SSL pages, this time in production. If you use the assets host feature, you'll find your SSL protected pages reference assets with normal unsecured http URLs. This will cause a lot of security warning dialogs for IE users so you have to deal with it. You need all referenced media to be server over SSL. Luckily, you can supply a Proc to generate asset URLs and this Proc receives the current 'request' object which we can interrogate. Turns out that this is such a common situation, the example in the rails docs gives the solution. Let's get our four hostname pipelining back too:

ActionController::Base.asset_host = Proc.new { |source, request|
  if request.ssl?
    "#{request.protocol}#{request.host_with_port}"
  else
    "#{request.protocol}asset#{(source.hash % 4)}.rentability.com"
  end
}

Finally, I have Most Blue Skies running reasonably well on one of our servers. Here's the "live" world view:

... with some including current "bluest" sky here.... At the moment, the server struggles to run enough sky calculations to keep the simulation up-to-date. I think it runs about 10 per second. That's on a dual 2Ghz G5 Xserve. We currently have 500,000 places in the database to test which will take about 14 hours to run through. Lets say half of those are in the dark and half or the remainder are under cloud. That's still several hours worth of calculations. So I need to either learn Fortran and improve the SMARTS software for batch processing (not that likely), buy more servers or code some kind of Most Blue Skies @ Home system where the public or sponsors can donate some processing power to the project. Maybe I can find a Fortran to Java translator. That might be a way forward...

For a long time we've wanted to get better looking property adverts onto Rentability. We thought about offering several layout templates that the owner could choose from. Then we thought about totally "floating" layouts where the user creates several "blocks" of content (text or photos) that are designed to flow down the page and be happy with different page widths and font sizes. The user doesn't have precise control over the placement of blocks. Instead, they have a bunch of blocks, of different dimensions that slot together down the page (from right to left and top to bottom) filling all available space. With fixed element widths (say a third or two thirds of the page) you can get a nice magazine/newspaper style look. Of course, a simple CSS float:left won't work all that well - you get a lot of holes where elements sort of clear:right as they wrap around down the page. So I tried to create this 'flowy' layout with javascript. The javascript measures each block, figures out how it slots into the layout, then positions it absolutely.

In the editing UI I've used scriptaculous to animate the reflowing of reordered blocks. This isn't always entirely intuitive - try the demo below - but then the whole approach is a compromise of sorts.

It's not beyond the realms of possibility that a layout will flow slightly differently on two different browsers. Actually, Sim's advert for the Pigeonnier lays out differently on IE than it does on Safari. Both looks great though! It's interesting how robust this technique can be.

Unless I've shown you a preview, you're probably wondering what the hell I'm going on about. Here's a demo page that you can play with (the changes you make are only visible by you).

Anyone pointing trac at a subversion repository built with svnsync might run into the same problem as me - trac shows commits but they show with author set to the username passed to the svnsync command and not the real author from the source repository. My best guess is that this happens because trac detects the checkin before synsync has copied over the revision properties (that includes changing the author). So to fix it, just add a post-revprop-change hook script (to the target repository) that tells trac to resync the revision concerned:

#!/bin/sh
REV="$2"
TRAC_ENV="/path/to/trac/environment"

/usr/bin/trac-admin "$TRAC_ENV" resync "$REV"

/usr/bin/python /path/to/trac-post-commit-hook -p "$TRAC_ENV" -r "$REV"

Update: 19/02/08:

After receiving a lot of duplicate emails from trac I realised that the above approach isn't quite good enough. svnsync updates revision properties more than once per commit (once for each property I guess) so you end up resyncing redundantly and, in my case, calling trac-post-commit-hook several times (resulting in duplicate ticket updates). So I decided not to use the subversion hook scripts and to do what I needed to do to trac directly after running svnsync. Here's the script I run with cron:

#!/usr/bin/env ruby

# This is script is run every 5 minutes. It syncs the svn repository onto the
# trac server. It also manually tells trac to resync each revision and calls
# the trac-post-commit-hook for each revision.

SVN_USER = "rails"
FROM = "svn://www.example.org/project"
TO = "file:///path/to/repos-mirror/"
TRAC_ENV = "/path/to/trac/environment"
POST_COMMIT_HOOK_SCRIPT = "/path/to/trac-post-commit-hook"

sync_output = `svnsync sync --non-interactive --username "#{SVN_USER}" "#{TO}" "#{FROM}"`

unless sync_output.empty?
  revisions = sync_output.scan(/Committed revision (\d+)/).flatten
  puts sync_output

  for rev in revisions
    puts "Telling trac about revision #{rev}..."
    puts `/usr/bin/trac-admin "#{TRAC_ENV}" resync "#{rev}"`
    puts `/usr/bin/python "#{POST_COMMIT_HOOK_SCRIPT}" -p "#{TRAC_ENV}" -r "#{rev}"`
  end
end

How come every time I install trac I do it a different way? Mostly, I'm installing on Centos or Fedora these days. The RPMs available rpmforge only go up to trac 0.10. It seems sensible to go with the 0.11 beta for a new installation - the trac admin web UI is included by default, there are fewer dependencies and the default ticket workflow has changed a bit. After some fiddling about with the manual download, realising that trac.cgi is missing (maybe it's deprecated) I realised that installation is fairly straightforward if you get setuptools (some strange python "egg" installer) and use modpython rather than cgi/fcgi (it's more efficient and easier anyway).

So here are my steps so I don't forget them:

# As root
wget http://peak.telecommunity.com/dist/ez_setup.py
python ez_setup.py

easy_install Pygments
easy_install Genshi
easy_install Trac

trac-admin /srv/trac/myproject initenv
chown -R apache:apache /srv/trac/myproject

nano /etc/httpd.d/trac.conf

# And add something like:

<Location "/trac">
  SetHandler mod_python
  PythonInterpreter main_interpreter
  PythonHandler trac.web.modpython_frontend 
  PythonOption TracEnvParentDir /srv/trac
  PythonOption TracUriRoot /trac

  # The following gives basic HTTP authentication
  AuthType Basic
  AuthName "Trac"
  AuthUserFile /srv/users.htpasswd
  Require valid-user

  # ... which should be used with SSL:
  SSLRequireSSL
</Location>

service httpd restart

I had a session preparing the Rentability front-end for localization the other day. Luckily I found the Gibberish rails plugin which makes localizing the static text fairly painless - you can store the translated strings in yml files, and the syntax is very simple. I made a slight patch to gibberish so it'll print the key next to each string on the live app so that the human translator can see where each key is used on the running site:

module Gibberish
  module Localize
    alias_method :old_translate, :translate
    def translate(string, key, *args)
      str = old_translate(string, key, *args)
      "#{str} <span class=\"translation-key\">#{key}</span>"
    end
  end
end

You can put that in environment.rb (maybe if RAILS__ENV == 'development'). Here's some CSS:

span.translation-key {
  font-size: 11px;
  font-weight: normal;
  color: black;
  background-color: yellow;
  font-family: geneva;
  border: 1px solid #444;
  padding: 0 1px;
  text-transform: none;
  font-style: normal;
}

Yesterday turned out to be a landmark for Rentability. We processed our first real, genuine enquiry from a real, genuine prospective holiday-maker! To quote Dr. Emmett Brown, "It works! It works! I finally created something that works!".

This actually happened quicker than we'd predicted and it was helped along by the fact that we have a much larger number of properties in Whistler, BC than everywhere else in the world put together.

I'm back in France and we're getting rentability.com ready for a few days of face-to-face alpha testing and then we'll start a period of beta testing. Multi-unit properties (e.g. B&B) are in for beta now which is great.

Quite lovely to be back! It's a busy house but we're making progress. We must be working hard - I've been here over a week and we've played no Collin Mcrae (although that might have something to do with Josh's new Wii).

This was easier than I though. Beast is a great piece of forum software written with rails. There is a thread on the forum about integrating beast into an existing app but no official advice on how to go about it. In fact, the official advice seems to be not to and to maintain a separate forum application. This is obviously ideal, but I want to reuse our user accounts on the forum. I couldn't think of a good way to achieve this. No-one seems to have attempted to build a shared login/single sign-on system in rails (at least I couldn't find one). I imagine most rails coders would not want that kind of extra complexity and not would I really. So I summoned up all my courage and decided to try to merge beast into rentability with as few hacky changes as possible.

In general it went pretty smoothly. I added beast's model, controller and helper classes, views and three plugins. I excluded the user controllel/model and views because I have my own, also based on acts_as_authenticated. I copied a few lines from beast's environment.rb into my own. Then I copied beast's routes into my own routes.rb. I had to change the routes quite a lot to accommodate our path prefix that we use to pass the current locale around - our URLs look something like www.rentability.com/en/controller/action/blah where the first element is the current locale. I wanted to keep this scheme on forum pages so I had to update all the routes. This is turn breaks all of beast's views because they use named routes like forum_path(forum) and I had to change them all to use a hash parameter: forum_path(:id => forum). Annoying. See the routes documentation in the rails docs for more info about that.

I had to update our user authentication code because it varies a lot from what acts_as_authenticated produces. For a start it uses some different column names (and has some additional ones) in the users table and uses :user_id as the session variable name rather than just :user (you'll need to change that to get beast's tests to pass).

Took about a day and half to iron out the problems and the result is great!

I'm definitely glad we have the forums incorporated like this. Running them as a separate application would have been quite limiting and would prevent us from integrating them more tightly with other parts of the site (or would at least have made it a lot more work to do so).

One this that I'm really glad I bumped into, and that I'd recommend to anyone else battling with CSS layouts, is Yahoo's baseline stylesheets. These are three, quite small stylesheets that try to create a level playing field between all modern browsers and they do a pretty good job. The most sophisticated and clever of the three is a set of column layout classes that provide you with indestructible column layouts both fixed width and fluid! No more 'popping under' on IE! No javascript involved either like other solutions I've seen.

So we finally acquired our domain name. We are now officially Rentability.com! Very exciting indeed.

I was pleased to discover the other day that PayPal are now offering their Website Payments Pro service outside of the US. This is the top-end of their merchant solutions and allows you to collect credit-card details and process them via a web-service without the customer needing to register with PayPal or be transferred onto a PayPal managed website. This is exactly what we need and PayPal should provide a more reliable service than the other smaller processing companies (like Protx who I've used and have not been too impressed with).

It's hard to see why anyone would choose WorldPay over PayPal WPP. WorldPay (really just Royal Bank of Scotland) are far more expensive, don't offer a web-service and had an awful admin interface last time I looked.

I've been using ActiveMerchant to access the PayPal API from Ruby. Unfortunately I'm going to have to upgrade it a bit to deal with Switch/Maestro cards. I'm always upgrading/fixing things to deal with Switch cards!

Update! 15/Jan/07:

A chap from PayPal emailed me to tell me that WPP in the UK uses a different API (a simpler REST-like API rather than SOAP). So the following instructions are really only relevant to the US service. PayPal have integration instructions for UK clients here.

There's not a lot of documentation for ActiveMerchant (actually, there is none) but it seems quite easy to get the hang of. To get started using it with PayPal WPP you should grab a PayPal developer account and check out [Website Payments Pro documentation] on the 'integration center'. I followed the instructions on a few people's blogs for getting hold of API certificates and they all turned out to be out of the date. PayPal's step-by-step guide is much better. One this that might catch you out is that, when creating your sandbox merchant account, you should specify that you are in the US rather than the UK or anywhere else, otherwise you won't be able to enable the account for the payment pro feature. (see update above)

Once you've downloaded the test API certificate, you add it to your project. I saved mine as config/paypal_test.pem. Then you add something like:

ActiveMerchant::Billing::Base.gateway_mode = :test
ActiveMerchant::Billing::PaypalGateway.pem_file =
    File.read( File.dirname(__FILE__) + '/../paypal_test.pem' )

... to your config/environments/development.rb file.

Using the direct payment API is fairly easy. It's something like this:

creditcard = ActiveMerchant::Billing::CreditCard.new({
    :number => '4242424242424242',
    :month => 8,
    :year => 2007,
    :first_name => 'Joe',
    :last_name => 'Smith',
    :type => 'visa',
    :verification_value => '000'
  })

address = {
  :address1 => 12 My Street',
  :address2 => 'Somewhere',
  :city => 'London',
  :state => 'London',
  :zip => '123ABC',
  :country => 'GB',
  :phone => '01234 567890'
}

gateway = ActiveMerchant::Billing::Base.gateway(:paypal).new(
    :login => 'my_api_login',
    :password => 'MY_API_PASSWORD'
  )

# preauthorize $10
response = gateway.authorize(1000, creditcard,
    :ip => 'xx.xx.xx.xx', :address => @address)

But to adhere to PayPals terms you have to offer the 'PayPal Express Checkout' option on your site which means, for your customer, a trip to PayPal's site and back. This is how you redirect a user to PayPal for express checkout:

response = paypal_gateway.setup_express_purchase(
  1000,
  :order_id => @cart.id,
  :return_url => url_for(:action => 'paypal_success'),
  :cancel_return_url => url_for(:action => 'paypal_cancel'),
  :description => "description of what customer is buying"
)
if response.success?
  url = ActiveMerchant::Billing::PaypalGateway.redirect_url_for(response.params['token'])
  redirect_to url
end

And when the customer has chosen their paypal payment method they return to your site:

def paypal_success
  response = paypal_gateway.get_express_details(params[:token])
  if response.success?
    @cart = Cart.find(response.params['invoice_id'])
    # Now show some kind of confirmation screen
    # After user clicks 'Confirm payment' execute something like
    # this to secure the funds:
    response = paypal_gateway.purchase(1000, nil, :express => true,
      :token => params[:token], :payer_id => params[:payer_id])
  end
end

So I'm in France for a while, working on this travel website with Josh and Sim. It is freakishly warm here. Actually it seems to be getting warmer. Feels like summer, only greener.

Here are some screenshots of what we've been working on:

I've been using the globalize-rails plugin for translation and localisation of my webapp. I'm fairly happy with with the plugin - it makes a good attempt to globalise most aspects of your application including strings in views and strings used anywhere else (uses the same mechanism, a .t method on instances of the String class) and also data in the database. This is the complicated bit - efficiently loading translations when records are loaded using a non-default locale. Seems to work though pretty well though but with two limitations:

• Doesn't work when you use an :include option on associations or when calling ActiveRecord::Base::find. Instead, an :includetranslated option can be used but that only works with :belongsto associations which seems like the least useful behaviour.
• Data isn't translated if you used find_by_sql. I guess this isn't really surprising but as my main 'search' function has to use findby_sql I suspect it might cause me some problems. Maybe there is a way to join with the globalizetranslations table and have it load translated attributes when it unmarshals to ActiveRecord objects.

I've created a reasonable UI for translating data:

In my view (say _form.rhtml) I have something like:

<%= text_field 'space_decl', 'name' %>
<%= translation_links 'space_decl', 'name' %>

Here's the helper module. The css just sets the alpha for untranslated languages:

span.flags img {
  margin-right: 4px;
}

span.flags img.todo {
  filter: alpha(opacity=50);
  -moz-opacity: .50;
  opacity: .50;
}

With literally minutes to spare we installed most blue skies. Josh heroically persevered with our light and it now seems to work well. The Koreans really know how to celebrate and art show! Opening events for the show were quite astonishing, involving ambassadors, riot police, prime ministers, secret agents, helicopters, blimps, rock a big concerts, several pop stars.

I'm now in Japan. I caught the 'beetle' high speed ferry from Busan and the shinkansen up to Kyoto. Today I go to Tokyo!