Software Development is Hard

This is something that's been bothering me for the longest time. And NOT because I want to feel good about myself telling others how hard my job/work is. The reason is quite opposite. Our company has gone through expansion and shrinking cycle for several times. Each time we change a few things, achieve better results but frankly nowhere near the ideal situation. The last time round we came up with some important structural changes, but it doesn't really stop there. Given an appropriate and workable structure, the next question is: what kind of people are we looking for? What are the hard skills and what are the soft skills they should have? What should their motivation be? 

Long time ago I wrote about 10 books to start learning software development. It was an attempt to address the fundamental areas of software development - to take a side in plethora of different possible approaches to them and say this is how we do it here. I really didn't mean to say read those 10 books and you're a top class software developer. I used to start my Ruby on Rails trainings with the following picture:

I found the picture somewhere on the net - but as an ex-java developer I owned and read every single book from the stack (even though I ditched Struts for Spring in 2003 so my stack would have Spring books instead of struts). The ruby stack is pretty much all that was available back then. It would be very nice to think that after reading 2 books you'll know everything you need to know :-). Just for the record, my ruby and rails collection today has exactly 25 books plus a Ruby on Rails Essential Training from Linda.com. Regardless of which stack you're looking at, however, the fundamental problem with the picture is that those books will teach you only very small portion of what you need to know - they only cover 1 area of software development (to be fair the java stack has books on some of the other aspects). They only teach you the language but they are not making you a programmer - just like knowing all 12 notes doesn't make you a musician and being able to write doesn't make you a writer. You still have to add a stack on patterns and principles, databases, testing, basic design, HTML+CSS, object oriented analysis and design, agile techniques and practices, meta-programming, deployment and other aspects. It's extremely important to say that those books are not novels you don't just read them - you really have to work with them, do the examples, play with it try it in different scenarios - basically deep practice with them. Reading in a few hours is totally useless and waste of time. 

This is what you have to spend your first 10,000 hours on.

The problem is that after those 10,000 hours you only know HOW ... you still don't know WHAT - a.k.a. the domain. Depending on your job this can very narrow or very broad or your every project may be in a different domain. That's not to say that you have to become a domain expert in every domain - but you can hardly write an accounting software if you refuse to learn about invoices, receivables, GL postings, financial statements and so on. While in traditional model there's system architect (or designer) that is expected to define everything to the last step with screen layouts and function of the last button and programmer simply translates the UML diagrams to code agile models require a very opposite approach. Software developer is no longer just shoveling coal from one pile to another. Instead, he/she is an active part of the development process - sitting with customers, communicating and shaping the final product. 

For example, when I was working on my first inventory control system I spent 7 months in the warehouse doing every role from unloading the deliveries from suppliers, preparing deliveries to customers, picking the stock for walk-in customers, through weekly stock checks representing the logistics department on management meetings. I was even a delivery boy for several days to understand what the drivers go through. Doing accounting systems I was a part of accounting department from data entry clerk, going through the end of the month salary preparation stress, working on financial statements for management and going through audits - working with auditors to understand what they need and what's the best way to present it to them. You may say it's totally crazy - and to be perfectly frank many of my (ex)employees do - you did not finish university to be some store clerk and if you wanted to be an auditor you would have studied finance - not programing. As one of my ex-employee (he used to liken himself to Douglas Crockford) put it - I am a luminary programmer - I cannot care about invoices and whatever other domain nonsense. 

The problem is that our customers don't give a damn about how luminary programmers we are. They don't know the difference between Java or Ruby, Rails or Spring, Apache or Tomcat. In fact, a few years back when I said I was programming in Java one of my customer asked if it's not too expensive to fly to Java every week and asked if it's because of the environment or cheap massages. They don't need a drill - they need holes. I have long 3 - 5 hours meetings with customers every day but we don't talk about programming - we talk about their issues and their problems. It's about understanding those problems and finding a solution - yet I could hardly find a solution if I had no idea what they're talking about. Problem is that many programmers I know don't even have any interest to understand.

To say it differently - not so long ago I spent 9 months in a restaurant - doing everything from serving, cashier, bar-tending to helping in the kitchen. Kitchen is a single THE most stressful place in the restaurant. Surprisingly the difference between great chefs, mediocre chefs and fired-in-3-days chefs wasn't how well they could fry the fish or roast the beef or how they're goulash tasted - in one word how well they could COOK. They all had to be able and were able to cook. The difference was in preparation. Great chefs would come in as early as 7am and started by cleaning the kitchen, they personally received all the deliveries checking the quality of everything - pretty much smelling every tomato, fish or pork chop. They would check they tools, go through the dishes, cutlery, prepare the condiments and just before opening for lunch sweep the restaurant floor one more time and check every single table one more time just to make sure that it's all perfect and ready. In the afternoon break they would talk to suppliers prepare samples and work on improvements. Mediocre chefs would turn up around 11:00 just in time  to defrost the fish heat up the soup and have a cigarette. In the afternoon break they'd be sleeping in the office - because legally, it was they're break.

The software development is very hard - but not because of programming. Being able to program doesn't make you a great programmer just like being able to cook doesn't make you a great chef. It's a given that programmer should be able to write a clean code with tests. What sets them apart is how they approach the other aspects - especially the low glam aspects. I realize that all that I just said goes right against the core values of Rails framework and general perception of Rails. After four years in rails I am very sure I am not a COOL developer working on the next, much COOLER Google or Facebook or Web 3.75 webapp. We do clinically uncool systems - dare I say the word enterprise systems and only a part of the work is actual programming.

I started by asking about motivation - now looking back at what I said - one would have to be crazy just to do all that stuff once - what kind of an insane person would suffer through all this learning and then suffer again on every project (not even mentioning finish stress and starting stress and stress during every iteration - considering a good project is never finished it's like a living organism). Now imagine you find this kind of person, after all there all sorts of crazy people around (like climbers, actors, musicians, etc.) that go through a similar ordeal - how would you pay them? If a fresh grad without any experience asks for $6,000 because he has an option of stress free MNC job (at least here they do) - what can you offer? Martin Fowler once said that they start charging clients on 1 million - when it runs out they charge again. We're almost there - but not quite yet :-) 

Last week I read Talent is Overrated by Geoff Colvin. There is a section dedicated to the the motivation of top performers which is very interesting - so I'll come back with more on this topic (hopefully) soon.