smallroomsoftware.com

Just got back from a pretty intense three week session redeveloping and installing a second showing of Most Blue Skies at the Nikolaj in Copenhagen. Our piece was part of the Rethink show. It was nice to be back in Copenhagen, such a calm place compared to my new home in London. The Nikolaj had rented us a flat in Sydhavn for most of our stay. We also had access to the Spaceframe and a room in Christiania so plenty of options as crunchtime approaches...

Our main aim with this install was to produce, one way or another, a much more impressive sky blue colour than we achieve in Korea. In Korea, we used a CYMK stage lamp, the "Sea Changer" which Josh had a hellish time trying to target for accurate colour representation. So, this time we decided to use an LED based light, plus diffuser, to create our slab of blue. The question was weather to use an existing commercial panel (an EvenLED, lent to us by Martin) or to build our own. Even though it's a very nice product, the EvenLED only has 16 RGB LED triplets and we were worried that it just wouldn't be particularly bright in the gallery.

In the end we decided to attempt to build our own panel but with the same dimensions as the EvenLED so if we failed to finish in time or something catastrophic happened, we could fall back on the Martin panel. The choice of LEDs was the next step. We had about €2000 in the budget that we could allocate to LEDs. These LEDs are surface mounted but you can buy them pre-mounted onto a "star" - a sort of star-shaped aluminium heat-sink. We decided to lay the LEDs out in a 9 by 9 grid wiring each row serially - we wouldn't be able to control each LED individually but we would be able to control each row independently and produce a gradient of colour - something we were keen to experiment with. We chose the Luxeon Rebel RGB "Endor" triplets, next to which we would add a Cree royal blue LED to boost the range of blue's we could produce.

So we would need 81 of each plus a few spares. One lessons, that we never seem to learn, is that unless you only need a handful of a particular component (like 10) then you may it incredible difficult to source what you need at short notice. Suppliers of most components don't seem to keep anywhere near what a newcomer might consider reasonable stocks on hand. We ordering 50 or 60 from a German company but ended up just overnighting the whole lot from a US supplier and returning the German shipment.

The next step was figuring out how to control the LEDs. We have some experience with the DMX lighting protocol which we used for the Sea Changer in Korea. It's a pretty unsatisfying system and DMX hardware is just insanely overpriced for what you get but with heavy time constraints we decided to stick with what we know. We now know that all the cool kids build their DMX controllers on Arduino boards (we know for next time) but we ended up having to spend around 400 sterling for a horrible plasticy ARTNET Node which is just a simple ethernet to DMX bridge. We still needed DMX based LED controllers. We found a really nice product - the eldoLED POWERdrive.

These LEDs produce a lot of heat! Josh wired one of the Luxeons up to an eldoLED with no extra heatsinking and all the wires desoldered themselves within a minute or so. So we figured we needed some serious heat-sinking. No problem - we'll just build the entire panel as a giant heat-sink :-)

Then came the soldering. A lot of soldering... You really don't want an underpowered soldering iron when soldering onto a massive heat sink. And electronics shops get annoyed if you keep pestering them for "something a bit more powerful" so arm yourself well.

We decided to bolt the power supplies and controllers onto the back of the panel. The position of the bolts was the one thing we didn't really design or plan before getting the drills out. This resulted in quite a few unplanned "ventilation holes". The lesson we learned - plan absolutely everything with your 3D CAD program as the things you think you can improvise will end up taking up a unexpectedly long time.

The first time we powered up the light in the gallery, we tripped the fuse for the entire ground floor. We got our own dedicated circuit but it kept on tripping. In the end we divided the nine power supplies into three sets of three so we could spread the load over several fuses. We didn't really anticipate the large draw of power as the power supplies switched on. We were surprised to have problems as the LEDs only come to about 800 watts. The guys at the gallery did a great job building the box to mount the panel in:

I made a lot of changes to the software to make it easy for us to throw together a lot of machines dedicated to doing sky colour calculations. The film school donated a lot of old machines to us, and a very dedicated man from the school helped us a lot by wiping the hard drives (rather than destroying them) saving us a lot of hassle finding new drives. I had a very boring couple of hours installing Centos linux on all of these machines:

I experimented with Capistrano to automate some tasks over all the machines. I had some success but sort of lost track with it as we ran short of time.

Eventually we did finish on time. With the usual after-opening few days of tweaking and tying up loose ends. We found that LogMeIn is useful for getting back into machines left in strange places (for monitoring and diagnostics). Next time, we'll try to get hold of an Axis webcam and leave it in the gallery. I have some more photos from the trip here.

Here's a short screen capture of the giant map warming up - it shows current calculations across the world and updates live with data streaming in from the old machines (you may have to full screen to see anything). You can just about tell it's North and South America:

Finally, I have Most Blue Skies running reasonably well on one of our servers. Here's the "live" world view:

... with some including current "bluest" sky here.... At the moment, the server struggles to run enough sky calculations to keep the simulation up-to-date. I think it runs about 10 per second. That's on a dual 2Ghz G5 Xserve. We currently have 500,000 places in the database to test which will take about 14 hours to run through. Lets say half of those are in the dark and half or the remainder are under cloud. That's still several hours worth of calculations. So I need to either learn Fortran and improve the SMARTS software for batch processing (not that likely), buy more servers or code some kind of Most Blue Skies @ Home system where the public or sponsors can donate some processing power to the project. Maybe I can find a Fortran to Java translator. That might be a way forward...

With literally minutes to spare we installed most blue skies. Josh heroically persevered with our light and it now seems to work well. The Koreans really know how to celebrate and art show! Opening events for the show were quite astonishing, involving ambassadors, riot police, prime ministers, secret agents, helicopters, blimps, rock a big concerts, several pop stars.

I'm now in Japan. I caught the 'beetle' high speed ferry from Busan and the shinkansen up to Kyoto. Today I go to Tokyo!

Most Blue Skies is progressing nicely! After an incredibly frustrating week or so battling with odd data formats we had a few breakthroughs and we finally have our 'core' data downloading and being used in the sky colour calculations. Josh has also checked in some nice code to convert the spectrum that SMARTS gives us to a RGB value. I hooked it all together today and it looks great!

For those of you who are interested in using MODIS Level 2 data with GIS applications (and can hopefully find this page via Google), I can probably offer some advice. As you'll know most GIS software doesn't support the HDF format that the MODIS data comes in so you need to find some tools to convert it into something you can work with. Finding the right tool, however, is a real pain. Also MODIS use a slightly extended version of HDF (just to make things even more difficult) so you need to find tools that are specially written to support MODIS data. The Level 1 and 2 products store data in 'swath' format. This basically means raw instrument readings each with a longitude and latitude coordinate attached. You need to therefore reproject and interpolate this data into some kind of grid in some projection that your GIS software/libraries can handle. Don't bother trying to use HEG to produce GeoTIFFs unless you are using certain data sets from certain products that don't have fill values and meet a bunch of idea criteria which I can't remember. The problem with HEG is that it doesn't use nearest neighbour interpolation when reprojecting the MODIS 'swath' to a geographically projected grid (even though you might tell it to). So this messes up scientific data sets where the values shouldn't be interpolated, e.g. bitfields (e.g. cloud mask data). On top of that, HEG doesn't honour the fill values and interpolates between these fill values and the actual data so you get even more bad data coming out.

In the end we HDFLook to convert each swath to a gridded linear projection in a new HDF file. It seems to do a very good job (although I'm having some problems with swaths that cross the international date line, not sure why). HDFLook can also produce a GeoTIFF but it seems to do a worse job than HEG at choosing the right data type for the TIFF data (you can store floats in a TIFF apparently). It always produces an RGB tiff which isn't very useful if your SDS uses floats. So I just produce a gridded HDF file with HDFLook. Then I've been using the Java HDF API to read the data directly from the new HDF file and build a coverage with GeoTools. I wanted to avoid calling any JNI code from the main app so I wrote a separate simple program to read the HDF files and spit out the values. Ask me if you want the Java code I used to do this (it's not particularly pretty).

Okay, so this what I think the sky looks like in Afghanistan today! (See previous post for what the hell I'm going on about...)

So it looks good. Some things are working.

Hugo sent me this link yesterday. Nice dancing! Made me want to pack up and just go somewhere. Hmm, well, maybe soon.

So in the last week or two I switched from working on "Departure Lounge" over to working on "Most Blue Skies". This is an art project that will be exhibited at the Busan Biennale 2006 in South Korea. This is mostly Lise's project but Josh is providing a lot of help, saving me from prolonged frustration on numerous occasions.

Although smaller in scale, the project shares a lot of similarities with Black Shoals (which was just a lot of fun and is one of my fondest memories). Both projects involve the processing of large amounts of data and turning it into some visual form. In this case we're processing atmospheric data in order to show the visitor where in the world they would have to go if they wanted to see the bluest sky at that very moment. We will also show what shade of blue they would see! The colour will be projected in some way onto a big square in the middle of the room and if you go behind the square you will see all the amazing technology that goes into producing that single lovely colour. As you might expect, calculating the colour of the sky in real-time over the whole of the world is turning out to be pretty tricky. We had to find algorithms for simulating radiation from the sun given some atmospheric parameters (took a while) and then try to find data sources for those parameters and try to get data that is as up to date as possible. Of course you have to grapple with mismatched units, satellites (or shuttles!) not flying where you want them to fly, millions of different co-ordinate system, file formats and access methods and also people insisting of writing their code in Fortran and still thinking in terms of punch cards (seriously!).

The other day we went to talk to the Atmospheric Physics department at Imperial about Blue Skies (but mostly we talked about Black Shoals because that actually exists ;-). The place seemed deserted when we arrived and we figured no-one would come but then everyone appeared (I think they thought it would be cooler that in their offices (it was a hot day)). Unfortunately it wasn't, but by that time they were trapped and could not escape from our gripping presentation! Anyway, a few people seemed to get quite into it and we got some good suggestions. Also happened to meet the cousin of someone Markus knows who works in bio-informatics. Small world.

We now have just four weeks to finish (I'm being optimistic and not counting a week of mad overnight in-gallery hacking in Korea). Pretty scary really! So I'd better get back to my GeoTiffs and aerosol optical depths...

Today Josh, Lise and I went the Met Office in Exeter to see if we could find some scientists willing to help up with Most Blue Skies project. It was nice to visit the west-country. I really think people get friendlier as you go West.

We also saw a sign which told us that the BIKINI state of the Met Office is...

There is an increased likelihood of an attack, but no defined target. It can also mean "potential terrorist threat".

So there you go. Today I was just two BIKINI steps away from all out war!