Software Development as a Profession?

A few years ago I used to be quite active in Rails community and then I stopped. I haven't been to a Singapore ruby/rails community meet up for 2 years and many of my friends ask me why - if I completely moved from Ruby to Python or simply what happened. Even though, I spend more and more time working with Python, I am still spend 13 - 14 hours a day doing Ruby programming. The actual reason has much more to do with Rails positioning. Back in April 2007, when the Singapore rails community was just starting to form and we had our first bigger meeting I wrote about it here. Most of it is now irrelevant but I was quite surprised when I read my 2nd point - purpose of the meetings - especially the part about the community divide to free hackers and enterprise developers. Back then I believed both were good and necessary for the community. I still believe it today only the situation has changed. While back then it was pretty much balanced, most of the community today consists of free hackers. After Dave Thomas moved from programming to music (okay - from rails to ruby 1.9) there hasn't been anybody to represent the enterprise interest and the rails community and rails image evolved to a cool web 2.0 free hacker paradise with little interest in enterprise.

This became very clear thanks to Bob Martin's key note on RailsConf '09 and very nice rebuttal by DHH.  Actually I've listened to that key note for several times now - Robert Martin is so expressive and (using the Smalltalk parallel) so diplomatic delivering something so critical to the community. For the past year and a half, I've been fighting with so many of the things he mentions - so many things I couldn't understand but make so much more sense now. 

It makes so much sense what he says about arrogance - I usually call it the 'cool' factor. Not doing the dirty job and only doing the clean things. Belief that our tools are somehow better, that our language is so good that we don't have to follow the rules, we don't have to do the regular (enterprise) things. It may be one of the reasons why Ruby still doesn't have a proper PDF or Excel handling libraries. The problem here is that this kind of arrogance is usually very subtle - but hits really hard and on places you would never expect. 

Rails is too easy to make a mess. This single statement pretty much sums up everything I said last week about software development being hard. Yes Rails is easy - in fact so easy that it makes you believe there is nothing you couldn't fix overnight. It's so easy that it makes you believe you don't need to know anything else. The problem is that all that ignorance will come back 3 months down the road (hopefully you'll be able to quit and let somebody else handle the mess).

Clean code. You look at the method it's pretty much what you'd expect. 

TDD. As a programmer I pretty much grew up on books and blogs of people like Martin Fowler, Robert Martin, Dave Thomas, Kent Beck and others from the group. TDD was the only way I was able to work. TDD is a No 1 book on my list of books for beginners. Yet I've been having the hardest time selling TDD - even to my own employees. They all know they HAVE to do tests - but only very few of them do it voluntarily or really understand why. TDD is simply not a culture around here (SG) - in fact, many find TDD culturally insensitive - at best a nice theory but totally useless in practice - just go through the Singapore ruby mailing list. Sensitive or not - it made it to our contracts for software developers.

Lastly, the discussion goes to software development as a profession and to what constitutes the professional portion of our job. Surprisingly enough, Robert Martin seems to think it's the discipline, rules and principles, it's the tough things we have to do and it's keeping them up when the pressure of a deadline mounts.  While there are people that are against being professional (see the DHH's rebuttal), for many people around me it makes sense. It seems so obvious that no body is really doing it.

Software Development is Hard

This is something that's been bothering me for the longest time. And NOT because I want to feel good about myself telling others how hard my job/work is. The reason is quite opposite. Our company has gone through expansion and shrinking cycle for several times. Each time we change a few things, achieve better results but frankly nowhere near the ideal situation. The last time round we came up with some important structural changes, but it doesn't really stop there. Given an appropriate and workable structure, the next question is: what kind of people are we looking for? What are the hard skills and what are the soft skills they should have? What should their motivation be? 

Long time ago I wrote about 10 books to start learning software development. It was an attempt to address the fundamental areas of software development - to take a side in plethora of different possible approaches to them and say this is how we do it here. I really didn't mean to say read those 10 books and you're a top class software developer. I used to start my Ruby on Rails trainings with the following picture:

I found the picture somewhere on the net - but as an ex-java developer I owned and read every single book from the stack (even though I ditched Struts for Spring in 2003 so my stack would have Spring books instead of struts). The ruby stack is pretty much all that was available back then. It would be very nice to think that after reading 2 books you'll know everything you need to know :-). Just for the record, my ruby and rails collection today has exactly 25 books plus a Ruby on Rails Essential Training from Linda.com. Regardless of which stack you're looking at, however, the fundamental problem with the picture is that those books will teach you only very small portion of what you need to know - they only cover 1 area of software development (to be fair the java stack has books on some of the other aspects). They only teach you the language but they are not making you a programmer - just like knowing all 12 notes doesn't make you a musician and being able to write doesn't make you a writer. You still have to add a stack on patterns and principles, databases, testing, basic design, HTML+CSS, object oriented analysis and design, agile techniques and practices, meta-programming, deployment and other aspects. It's extremely important to say that those books are not novels you don't just read them - you really have to work with them, do the examples, play with it try it in different scenarios - basically deep practice with them. Reading in a few hours is totally useless and waste of time. 

This is what you have to spend your first 10,000 hours on.

The problem is that after those 10,000 hours you only know HOW ... you still don't know WHAT - a.k.a. the domain. Depending on your job this can very narrow or very broad or your every project may be in a different domain. That's not to say that you have to become a domain expert in every domain - but you can hardly write an accounting software if you refuse to learn about invoices, receivables, GL postings, financial statements and so on. While in traditional model there's system architect (or designer) that is expected to define everything to the last step with screen layouts and function of the last button and programmer simply translates the UML diagrams to code agile models require a very opposite approach. Software developer is no longer just shoveling coal from one pile to another. Instead, he/she is an active part of the development process - sitting with customers, communicating and shaping the final product. 

For example, when I was working on my first inventory control system I spent 7 months in the warehouse doing every role from unloading the deliveries from suppliers, preparing deliveries to customers, picking the stock for walk-in customers, through weekly stock checks representing the logistics department on management meetings. I was even a delivery boy for several days to understand what the drivers go through. Doing accounting systems I was a part of accounting department from data entry clerk, going through the end of the month salary preparation stress, working on financial statements for management and going through audits - working with auditors to understand what they need and what's the best way to present it to them. You may say it's totally crazy - and to be perfectly frank many of my (ex)employees do - you did not finish university to be some store clerk and if you wanted to be an auditor you would have studied finance - not programing. As one of my ex-employee (he used to liken himself to Douglas Crockford) put it - I am a luminary programmer - I cannot care about invoices and whatever other domain nonsense. 

The problem is that our customers don't give a damn about how luminary programmers we are. They don't know the difference between Java or Ruby, Rails or Spring, Apache or Tomcat. In fact, a few years back when I said I was programming in Java one of my customer asked if it's not too expensive to fly to Java every week and asked if it's because of the environment or cheap massages. They don't need a drill - they need holes. I have long 3 - 5 hours meetings with customers every day but we don't talk about programming - we talk about their issues and their problems. It's about understanding those problems and finding a solution - yet I could hardly find a solution if I had no idea what they're talking about. Problem is that many programmers I know don't even have any interest to understand.

To say it differently - not so long ago I spent 9 months in a restaurant - doing everything from serving, cashier, bar-tending to helping in the kitchen. Kitchen is a single THE most stressful place in the restaurant. Surprisingly the difference between great chefs, mediocre chefs and fired-in-3-days chefs wasn't how well they could fry the fish or roast the beef or how they're goulash tasted - in one word how well they could COOK. They all had to be able and were able to cook. The difference was in preparation. Great chefs would come in as early as 7am and started by cleaning the kitchen, they personally received all the deliveries checking the quality of everything - pretty much smelling every tomato, fish or pork chop. They would check they tools, go through the dishes, cutlery, prepare the condiments and just before opening for lunch sweep the restaurant floor one more time and check every single table one more time just to make sure that it's all perfect and ready. In the afternoon break they would talk to suppliers prepare samples and work on improvements. Mediocre chefs would turn up around 11:00 just in time  to defrost the fish heat up the soup and have a cigarette. In the afternoon break they'd be sleeping in the office - because legally, it was they're break.

The software development is very hard - but not because of programming. Being able to program doesn't make you a great programmer just like being able to cook doesn't make you a great chef. It's a given that programmer should be able to write a clean code with tests. What sets them apart is how they approach the other aspects - especially the low glam aspects. I realize that all that I just said goes right against the core values of Rails framework and general perception of Rails. After four years in rails I am very sure I am not a COOL developer working on the next, much COOLER Google or Facebook or Web 3.75 webapp. We do clinically uncool systems - dare I say the word enterprise systems and only a part of the work is actual programming.

I started by asking about motivation - now looking back at what I said - one would have to be crazy just to do all that stuff once - what kind of an insane person would suffer through all this learning and then suffer again on every project (not even mentioning finish stress and starting stress and stress during every iteration - considering a good project is never finished it's like a living organism). Now imagine you find this kind of person, after all there all sorts of crazy people around (like climbers, actors, musicians, etc.) that go through a similar ordeal - how would you pay them? If a fresh grad without any experience asks for $6,000 because he has an option of stress free MNC job (at least here they do) - what can you offer? Martin Fowler once said that they start charging clients on 1 million - when it runs out they charge again. We're almost there - but not quite yet :-) 

Last week I read Talent is Overrated by Geoff Colvin. There is a section dedicated to the the motivation of top performers which is very interesting - so I'll come back with more on this topic (hopefully) soon.

The Talent Code

I read an interesting book couple of weeks ago. The Talent Code by Daniel Coyle. While the whole book is really good I particularly liked the section explaining the "sudden" appearance of extraordinary talents - like Brazilian football players, Bronte's sisters, Mozart, venetian sculptors and painters and so on. How it's believed that all those people were simply born with great talent and their great work just happened. And how the reality was almost a total opposite. The main idea behind the book is that no matter what you do, in order to achieve a world class skill you need to spend 10,000 hours in practice. The skill is basically "neural connections" in the brain. Those connections are created during practice but what determines the level of the skill is the "quality" of those connections. What increases the quality of the connection is myelin by insulating nerve fibers. The 10,000 hours is necessary to create sufficient myelin. What's interesting is that it's not just any practice - the best results are achieved in "deep practice".

The author goes on to explain how any of the great talents already had their 10,000 hours clocked in by the time they were "discovered". How bronte's sisters had written tens of training books before Jane Eyre or Wuthering Heights, how Mozart had his 10,000 hours of practice in very early age and how venetian painters and sculptors got their hours of training in apprenticeship. They were not born great but started from scratch and achieved their greatness by increasingly improving their skills. Interesting thing is that the apprenticeship didn't mean that they would simply paint the whole day and after 10 years became masters. No - they had to do all sorts of things - especially all sorts of "low level things" - like setting the canvas, preparing chisels and so on for their masters. Only after they learnt the basics could they move on to more difficult things. And this is what basically constitutes the "deep training" - choose a goal just outside your comfort zone and keep on failing until you achieve it. Then repeat the process. 

To sidetrack a bit to IT - this is where I think today's system breaks down. Software development is hard. And it only starts with the technical skills - you still need to understand the domain (when working on accounting system you have to have a very solid understanding of accounting), you have to be able to work with people and translate their ideas and feelings about something they've never seen to reality. And there is no magic shortcut - you have to have your 10,000 hours clocked in before you can really do something.  Programming is like writing - just like bronte's sisters - you have to write a lot a lot a lot and about everything you can find - only then your writing will start making sense. Unfortunately, I wouldn't really count any hours spent at school. Maybe it's an unfortunate situation around here - but the IT schools here are set to producing CIO's making decisions on a golf course rather than doing programing or any other actual IT work. To their defense that's pretty much what market here wants - most of the fresh grads from IT will find a good shake-leg work in a bank or MNC paying at least $10,000 per month. I meet quite a few of those people in my trainings or recently, as the banks scale down certain areas, in interviews. 

Anyway, my point here was that only very very few people in IT are willing (or forced) to go through the whole journey - from setting up networks, installing computers, moving to servers, maintaining different operating systems, doing backup / recovery, working with databases, optimizing databases, writing SQL, programming in several languages for several years, understanding patterns going through several releases and so on. And none of that means "I used MS Access in 3rd semester".

Another interesting thing was the difference between masters and beginners. The difference was explained on chess players. The difference between chess masters and beginners is that masters can remember the whole board set up on one look. There is a twist, though. They could only remember the set up from an actual game. If the figures were in random order the chess masters photographic memory vanished. The reason why chess masters could remember the board setup from actual games is because they were not seeing the individual pieces - they were seeing the patterns. 

I picked up the book because I wanted to know more about how people learn - I wanted to understand why after a year of training and working some of my employees are not able to do even a simple task, why is it that they spent a week installing a server without any success while somebody else can do the same work in 1 hour. Why some spend months and months programming a piece of functionality creating a total mess and somehow not "seeing" the simple solution that can be finished in a few hours or days. I used to be especially puzzled about the not seeing part. Even after showing them the simple solution they just couldn't understand it and had to go a big round of all possible wrong ways to arrive at the same solution. I guess some of them see patterns while others see thousands of lines of commands. As a result, in the light of the book's 10,000 hours I start to look very differently at this kind of things. The good news is: it's learnable. The bad news is: it may take 10 years for someone to learn it.

Rails 2.3 with Ruby 1.9

After the longest time we decided to upgrade the rails version in our applications. The last upgrade was to 1.2.6 year and a half ago. As we have quite a number of applications I am quite reluctant to upgrade unless it's really really worth it. There was quite a number of positive changes in Rails 2.2 (like the database connection pooling, thread safety and built in internationalization) but it wasn't compatible with Ruby 1.9 and I really didn't feel like going through all the apps just to go through them again to upgrade it to Ruby 1.9.

After release of Rails 2.3 compatible with Ruby 1.9 I could finally go ahead with the upgrade. And it turned out to be quite easy - but unfortunately some things haven't been ported to 1.9 yet. One of the main advantages of 1.9 is supposed to be increase in performance. It can be easily demonstrated by running recursive Fibonacci test. Unfortunately not even one 

of my applications needs faster Fibonacci :-). Having run most of our apps on 1.9 for around 2 weeks now I don't really see any tremendous performance increase or lower memory requirements. Several of our systems require extensive calculations (especially bigger accounting systems) but the performance there is dependent on many other factors like database speed, array sorting and grouping and so on and I am still working out the optimizations here.

What really really really helped me during the migration was David Black's book The Well-Grounded Rubyist. I heard him speaking several times but I was quite surprised by how well written the book was. It's really sharp

 and to the point without useless 'technicalities'. At the same time it's very readable and not dry like API reference.

Okay back to Rails. As I said most of the things went smooth BUT here's the list of some that didn't :-)

1) Mongrel doesn't work - not so much of a problem for me as I don't use it in production and in development I switched to Thin. 

2) File upload on nginx doesn't work - it seems to be an issue with Rack that is bundled with Rails 2.3. This is a bit of an issue as I was just starting to like nginx and I used it for some my lighter applications. 

3) Problems with encoding - this a single issue that I didn't manage to resolve or find a workaround. We've done some projects in Europe and they have to support slavic languages with all their funny characters. As 1.9 supports so many encodings I really didn't expect any problems there but I didn't manage to get pass the

incompatible character encodings: UTF-8 and ASCII-8BIT

when data from database contain special characters. There are several patches available in lighthouse here and here but non of them really worked for me - safe for

contact.name.force_enconding('utf-8')

4) send_data doesn't work on binary files with message

invalid byte sequence in UTF-8

. I used to use it for on the fly generated images but had to replace it with creating a temp file and then use

send_file

5) ruby zip hasn't been updated yet and still references ftools that have been replaced by fileutils. I had to change it manually along with File.move reference replaced with FileUtils.move

6) quite a number of plugins stopped working and as many are no longer supported I decided to ditch a few of them. I have never been a fan of the small rails plugins that only adding some syntactic sugar not just because they break on every update of Rails (even worse with Ruby upgrade) but mostly because of their obtrusiveness. Even though they make the code shorter it's usually much less readable as you have to learn and understand the plugin.

File Downloads in Rails on Apache

Over the past half a year we worked on several systems for logistics/courier/delivery companies. It was quite fun as a big logistics and inventory control system was my very first project when I arrived to Asia 9 years ago.  Last week we launched one of them. It was slightly different as most of our systems are used mainly internally but this one is also used by clients of the courier company to book deliveries. Each delivery requires printing of confirmation documents, internal routing documents, then signed and stamped documents are uploaded back to the system as a proof of delivery for clients. It has quite a number of other (technically) interesting features like integration with google maps to geo-tag the orders and notifying drivers in the area - I'll write about it soon. 

As this courier company handles several hundreds of deliveries every day it requires quite a number of uploads/downloads of documents and images. The issue with uploads and downloads is that they will tie up rails processes for much longer than usual requests. The problem is that rails processes are quite expensive in terms of memory  (around 70 - 100MB for simple operations). Do 20 concurrent uploads/downloads and all your rails processes and memory will get tied up - and quite wastefully as there is no 'dynamic' processing during upload or download. The process just sits there and waits for the file to get delivered. Luckily there's a simple way how to move this burden to Apache. This kind of upload/download processes will take only around 2MB on average and your rails processes are free to handle the dynamic requests.

To let apache handle file downloads use x_sendfile. All you need to do is install x_senfile module on apache and adjust you config a little. Here's the installation instructions for Ubuntu 8.04:

cd /usr/local/src
wget http://tn123.ath.cx/mod_xsendfile/mod_xsendfile.c
apxs2 -cia mod_xsendfile.c

Open apache config file and add the module line:

sudo vim /etc/apache2/apache2.conf

LoadModule xsendfile_module /usr/lib/apache2/modules/mod_xsendfile.so

Add the following 2 lines to your virtual host file

sudo vim /etc/apache2/sites-available/my_app

XSendfile on
XSendFileAllowAbove on

You will have to restart the apache and you should be done:

sudo /etc/init.d/apache2 restart

Now in your controller handling download add x_sendfile attribute and set it to true

  send_file("/home/test/big_file.zip",
            :filename => "big_file.zip",
            :type => "application/zip,
            :x_sendfile=> true)

The uploads are equally easy. You just need to apache modules libapreq2 and modporter.

wget http://mirror.nus.edu.sg/apache/httpd/libapreq/libapreq2-2.12.tar.gz
tar -xzf libapreq2-2.12.tar.gz
cd libareq2-2.12
./configure --with-apache2-apxs=/usr/bin/apxs2
make
sudo make install

Now download and install modporter. Get it from the github. Either use the download link and unzip or just clone the repository

git-clone git://github.com/actionrails/modporter.git
cd modporter

On Ubuntu I had to change reference to apxs in Rakefile as the in ubuntu it's apxs2.

vim Rakefile

Change the APXS line to the following:

APXS           = "apxs2"

No just run rake:

sudo rake

add the following to your apache2.conf

sudo vim /etc/apache2/apache2.conf

LoadModule apreq_module /usr/lib/apache2/modules/mod_apreq2.so
LoadModule porter_module /usr/lib/apache2/modules/mod_porter.so

To actually use it in your rails application you will need modporter plugin.

script/plugin install git://github.com/actionrails/modporter-plugin.git

Add the following to the application_controller.rb (application.rb before Rails 2.3)

class ApplicationController < mod_porter_secret = "secret">

and just enable porter in you vhost file:

sudo vim /etc/apache2/sites-available/my_app

Porter On
PorterSharedSecret secret

And that's all now if you check the uploaded files they will be a ModPorter::UploadedFile Credit for this goes to Koz from theRailsWay.com - you can find more information here.

Another Training

Another training's over. Very different but equally challenging.

Seems that Ruby's been making headway to corporate world as most of the participants this time were there on their employers' budget. When I compare it with our first RoR trainings around 2 years ago with Rails 1.0 just out and more than 30% of the things still on a wish list and the only way to learn RoR was Agile Web Development With Rails...this time it was much more business-like. There's like 20 books out, including a (bit obsolete) video from Linda.Com and pretty much everybody knows about rails.

Another aspect that has changed is Rails versions. With Rails 2.3 and Ruby 1.9 out there's a lot of changes and incompatibilities that complicate the learning. Most of the books and reference materials use Rails 2.0 or early versions of 2.2. This makes it little bit difficult to have training on 2.3 as most of the "further" reading is incompatible. While the incompatibilities may seem easy to solve for experienced developers they can be quite exasperating for a newbies. On the other hand, people are quite reluctant to start learning something that is already obsolete.

Similar is true for Ruby 1.9, even though after burning they're fingers several times until it got finally compatible with Rails most people would still prefer to use 1.8.6 or 1.8.7.

Dynamic Columns In Ext JS Grid

I just had another unproductive afternoon. Another thing I would expect to work out of the box that turned out to be not that easy. It was not the first time I needed dynamic number of columns in the grid but until now I was always able to find a workaround. Fortunately, today I couldn't go around and had to solve the problem once and for all.

What I needed was a column model dynamically adjusting depending on the data received from data store (i.e. JSON provided by server). Something like the following:



Where each site has a different set of dimensions (rows) and different set of auditors (columns).

It turned out that the problem was actually two-fold - first dynamically change the columns and then display the editable checkboxes instead of true/false values.

To change the columns I used metaData attribute in my JSON. Including metaData attribute causes Ext.data.JsonReader to be adjusted to the new set of fields and other config parameters and fire metachange event. Here's an example of the JSON with metaData:

{
 "results": 3,
 "auditors": [
  {"dimension_name": "Audits and Reviews", "auditor_3": false, "dimension_id"
: 10, "id": 10, "auditor_4": false, "auditor_18": false},
  {"dimension_name": "Emergency Management & Fire Safety", "auditor_3": false, "dimension_id": 8, "id": 8, "auditor_4": false, "auditor_18": false}
  .
  .
  .
 ],
 "metaData": {
  "fields": [
   {"name": "id"},
   {"type": "string", "mapping": "dimension_name", "name": "auditor[dimension]"},
   {"type": "int", "mapping": "dimension_id", "name": "auditor[dimension_id]"},
   {"type": "bool", "name": "auditor_3", "header": "Dasha Alexo"},
   {"type": "bool", "name": "auditor_18", "header": "Tester 3"},
   {"type": "bool", "name": "auditor_4", "header": "Zobak Zobakovic"}],
  "root": "auditors",
  "totalProperty": "results",
  "id": "id"
 }
}

In this case it's the number of auditors (and their names) that's changing. My data types are booleans but displaying any other data type would look pretty much the same.

Next I needed to update the column model based on the new meta data. I did that by adding metachange listener to my store

auditors_store.addListener("metachange", function(store, meta){
 var grid = Ext.getCmp('auditors-grid');

 var columns = [
  {header: 'Dimension', dataIndex: 'auditor[dimension_id]', hidden: true},
  {header: 'Dimension', dataIndex: 'auditor[dimension]', width: 200}
 ];
 
 for (var i = 3; i < meta.fields.length; i++ ) {
  var plugin = new Ext.grid.CheckColumn({
    header: meta.fields[i].header,
    dataIndex: meta.fields[i].name,
    grid: grid,
    width: 120
   });
  
  columns.push(plugin);
  plugin.init(grid);

  //use columns.push( { header: meta.fields[i].header, dataIndex: meta.fields[i].name, type: meta.fields[i].type }); for fields that don't require plugins
 }
 grid.reconfigure(auditors_store, new Ext.grid.ColumnModel(columns));
});

I had to create and initialize the plugin for each new column as I needed editable checkboxes. For normal fields just use the usual column config object { header: meta.fields[i].header, dataIndex: meta.fields[i].name, type: meta.fields[i].type }.

The actual magic happens is in the last line. Grid has an obscure method reconfigure( Ext.data.Store store, Ext.grid.ColumnModel colModel ) that allows to switch the store and column model (the columns displayed) on the fly.

My next problem was to make the checkboxes editable. To do that I had to put a small fix to the CheckColumn plugin (see the plugin) as the original plugin assumes that the grid is not rendered when initializing, however, when dynamically adding columns the grid is already rendered so I had to change the following:

 init : function(grid){
  this.grid = grid;
   this.grid.on('render', function(){
    var view = this.grid.getView();
    view.mainBody.on('mousedown', this.onMouseDown, this);
   }, this);   
 },

to the following:

 init : function(grid){
  this.grid = grid;
  if (this.grid.rendered) {
   var view = this.grid.getView();
   view.mainBody.on('mousedown', this.onMouseDown, this);
  } else {
   this.grid.on('render', function(){
    var view = this.grid.getView();
    view.mainBody.on('mousedown', this.onMouseDown, this);
   }, this);   
  }
 },

And that's all - nice and simple. If only there were more books on Ext :-(

Agile Employment Contracts (part 1)

A lot has been said said about agile development and agile principles like content delivery over contract negotiation, flexibility of contracts and no upfront specification. We've been loving and doing that for some years now. One thing I was not able to wrap my head around for the longest time, though, was how to manage (traditional) employees in this kind of a non-traditional arrangement. Most of the agile books spend only very limited time on this topic pretty much always retorting to anecdotal evidence of "self motivated teams rising to the occasion and living happily ever after"

When dealing with clients, both parties are bound by the contract (however agile and flexible) and there are clear and simple business rules to be followed. In our case it comes down to if client doesn't like what they're getting they don't pay. The payment happens after every iteration so both parties have a tangible motivation/interest to get things done. Both parties enter the contract without previous baggage and maintain pretty much equal status.

Not at all as simple with (traditional) employees. They usually start with limited knowledge and require extensive training. In our company it takes several months and even after they've mastered the technical skills they still need months and months of experience to grasp various aspects of different domains and business processes we deal with every day. We use commission from closed iterations and collected payments as a motivation. Main portion of the money, however, still comes in the form of monthly salary. Given high level of stress, steep learning curve of ever changing technology, demanding customers, even more demanding bosses it's only matter of time when the initial motivation wears out and everyday routine kicks in. In many of my ex-employees, this was accompanied with a feeling that they've learned more than they'd ever thought they would and their price on the market has soared indefinitely. Sarcasm aside, after reaching this level, people start to be more relaxed and try to find motivation/amusement in experimentation with latest gadgets, obscure technologies or start working on they're own version of Rails or other frameworks. What's worse, it slowly starts to affect the delivery of project(s).

Now let's switch to our shoes. We spent the money on training and non-productive period of the employee. We see that the results are diminishing and we see that the employee is even starting to abuse others requiring help and claiming that he is a slow programmer and that it's not his domain knowledge, etc. Problems in delivery cause delayed payments and smaller and less frequent commission for the employee. No matter how little employee produces (and this can quickly become negative value when the quality reaches throw away and rework levels) his salary is still guaranteed to him by employment act.  If we let him go we lost all our investment in training and we will incur extra costs replacing him mid-project. During all this time there is absolutely nothing we can do to protect our side. Yes, we can motivate, but if that doesn't work we've got nothing.

Agile development contracts seem to be calling for agile employment contracts.

Ubuntu 8.04 mod_rails

Around a half a year ago I started slowly switching my servers to Apache mod_rails. I wasn't very keen after my first test around September on one of my Slicehost servers. Without any application running apache, ruby and postgresql processes tookup around 500MB of memory - around 100MB per processes. When I experimented with my other servers I found out this happens on 64bit architecture - 32bit servers took up less than half of that memory. I am still running performance tests to compare the performance and behavior of mod_rails vs Lighttpd - I'll be writing about it soon. For now here's the installation for Hardy Heron (I assume you have the build-essential, ruby and so on already installed):

sudo apt-get install apache2 apache2.2-common apache2-mpm-prefork apache2-utils libexpat1 ssl-cert apache2-prefork-dev
sudo gem install passenger
sudo passenger-install-apache2-module

Now add the 3 generated lines (almost last on the screen) to config file:

sudo vim /etc/apache2/apache2.conf

Right now it looks something like the following - BUT THE VERSION NUMBERS WILL CHANGE SO COPY THE LINES FROM THE PREVIOUS PROCESS

LoadModule passenger_module /usr/local/lib/ruby/gems/1.8/gems/passenger-2.1.2/ext/apache2/mod_passenger.so
PassengerRoot /usr/local/lib/ruby/gems/1.8/gems/passenger-2.1.2
PassengerRuby /usr/local/bin/ruby

In the same config file add the server name and NameVirtualHost otherwise the server will complain at restart:

ServerName XYZ
NameVirtualHost *:80

save the config file and continue - we'll need to enable modrewrite and disable default site:

sudo a2enmod rewrite
sudo a2dissite default

now just configure your application by adding your_app_name to /etc/apache2/sites-available/ with the following content:

sudo vim /etc/apache2/sites-available/your_app_name

<VirtualHost *:80>
ServerName  abc.def.com
ServerAlias xyz.com

DocumentRoot /home/bohm/rails/access/current/public

</VirtualHost>

where abc.def.com and xyz.com are the URLs of your application and DocumentRoot points to the public folder of your application. To enable the application run

sudo a2ensite your_app_name
sudo /etc/init.d/apache2 reload

To disable it run

sudo a2dissite your_app_name
sudo /etc/init.d/apache2 reload

And you're done :-)

Multiline Tree nodes in ExtJS

While I really like Ext and the component model behind it, some simple things seems to be missing - an example would be multiline nodes in TreePanel. After some searching I managed to find a  MultilineTreeNodeUI plugin.

Here's how to use it:

new Ext.tree.TreePanel({
 //renderTo:'treecontainer',
 id: 'upcoming_events',

 loader: new Ext.tree.TreeLoader({
  dataUrl:'/datasource/upcoming_events_json',
  baseParams: {},
  baseAttrs: {uiProvider: Ext.ux.tree.MultilineTreeNodeUI}
 }),
 root: new Ext.tree.AsyncTreeNode({
  expanded: true
 }),
 rootVisible: false,
 collapsed: false
})

Just include the MultilineTreeNodeUI.js in the page and create a usual TreePanel. The only difference is to include baseAttrs: {uiProvider: Ext.ux.tree.MultilineTreeNodeUI} in your loader. Multilne tree requires the data in the following format:

{
 text: '09 April 2009',
 iconCls: 'calendar',
 details: ['Program for the day...'],
 uiProvider: Ext.ux.tree.MultilineTreeNodeUI,
 children:[
  {
   text: '9:30',
   iconCls: 'time',
   details: ['Win $1M', 'Sure win...'],
   uiProvider: Ext.ux.tree.MultilineTreeNodeUI,
   leaf: true
  },
  {
   text: '11:30',
   iconCls: 'time',
   details: ['Buy private jet', 'Or something'],
   uiProvider: Ext.ux.tree.MultilineTreeNodeUI,
   leaf: true
  }
 ]
}

Where the additional lines are provided by details attribute. It's an array - one item per line - maybe not an ideal solution but still workable - worst case you just split the longer line by number of characters. The uiProvider will generate the multiple lines. Your controller doesn't have to do anything about it as it will be added automatically by the baseAttrs: {uiProvider: Ext.ux.tree.MultilineTreeNodeUI} we added in the loader.

And here's the outcome: