Commodity Hardware

According to last week's Economist, the United States Air Force are buying 2,200 units of Sony PlayStation 3. The idea is to get a cheap hardware (subsidized by SONY as their actual revenue comes from selling the games), install Linux and have a super-powerful cluster at 10% of the normal cost. I've had a similar idea last year - coming out the economic boom left me with number of spare notebooks and number of customer needing a server but with advancing recession with less and less money for a new one. Out of necessity I set up temporary servers using my spare laptops. While I did expect some problems - like failing hard drives, performance issues and so on... surprisingly enough 1 year later not one of those laptops/servers failed. Not only that - I even found the performance on a par with my other regular servers.

That got me thinking...according to the Moore's law the computing power is doubling fast enough to make last month's average laptop faster than last year's high end server. Over the years I've worked with number of servers and number of brands - from Dell, through HP, Compaq to NEC. Strangely enough, no matter how expensive the server (or any computer for that matter) - the only sure thing is that it will break. E.g. I use MacBook Pro 17" that cost me over $5,000. Yet, everything (with the exception of keyboard) has already failed and I spent on repairs more than the initial amount. Just the display with the motherboard was more than $4,000. The worst thing about it was that every time something happened the notebook spent at least a month (up to 2 months) in the workshop. My old Dell notebook broke down just as much and it used to be very frustrating to wait until the next business day for technician to come to my office and fix it. Having to actually go to the workshop, spend 1 hour there + 1 hour on the way and have your bloody expensive notebook taken for a month... Anyway, the point is that even my high end servers fail. I once had 4 out of 5 hot-swap SCSI drives fail at the same time. Another time just 3 of them did. At the end I found it much more important to prepare for the hardware failure than trying to prevent it.

Coming back to the cheap notebooks - you can get a reasonably good notebook with 4GB RAM for around $800. With Dell's next business day service (or even better the 4 hour service) you pretty much don't have to worry about it for the next 2 or 3 years. Should anything happen you can simply swap it for a spare one. This actually addresses whole set of points and issues I face with conventional servers. The installation and troubleshooting usually requires a senior person going onsite wasting at least half a day. The notebooks, on the other hand, can be pre-installed in the office and simply delivered to the customer by a junior staff who just needs to plug it to the network and everything is done. I still remember moving 4U servers (54kg each) a few years back. This obviously requires a proper backup strategy. The software part of which (i.e. what to backup and when and how and offsite strategy) is the same as on a regular server. For faster recovery in notebook servers I clone the drive to an external USB drive and have the backup copied also on this external drive (pgpool-II is really great for this). The cloning can actually be automated on monthly or weekly basis to have a reasonably fresh image on the external drive. If the drive fails (by far the most common thing on my servers as well as notebooks) all that needs to be done is to replace the drive. This takes a couple of minutes on my Dell and Compaq notebooks and even a non-technical person can do it. Once the system is back on-line, with everything installed and the backup conveniently on the drive, it takes less than 5 minutes to restore the database and deploy the latest revision of the system. And of course, can be done from anywhere.

I am not saying that notebook servers are fit for every situation - I am only saying that they're fit for most of our customers - situations where there's limited number of users (typically around 30 - 50) and most of them in the same location. While the internet connection for businesses in SG is as slow as it is expensive, it still still usable for even several offices accessing the system. I am still finding it hard to understand why in such a high tech place like SG it costs around $100 a month to get 100MB downstream/10MB upstream for residential connection yet it costs over $200 to get 1.5MB down/374kB for the office. There's so many initiatives for bringing internet to consumers - from free Wireless@SG, through iCell's lighting up East Cost Park and food courts, internet on the buses to great quality and dirt cheap 3G internet (I pay $20 for 50GB a month) - yet there is very little for the content providers. There's been a big push coming from government and SingTel for cloud computing. I haven't really heard of any local cloud provider worth mentioning - there are several offering either ridiculously high prices or ridiculously low specs or both. We've been using Rackspace's cloud servers ever since they became available - besides the lowest price they offer really great support - yet I've been moving more and more customers away to the notebook servers - from the peak of 19 servers I now only have 4 left. The main reason is actually price/performance ratio. My usual application requires (at least) 1GB memory instance the price of which comes to around S$70 (around 50 USD) which comes to around S$800 a year. The problem is that 1MB instance is sufficient only for average performance - it is not enough during peaks - and even small performance hiccups during peak result in great frustrations for the users. As S$1000 can buy you a 4GB Dell with next day on-site service for 2 years - it's quite compelling - especially when budgets are tight.

Android

I don't know what is wrong with me but, somehow, I ended up on an underdog technology again. With the recent and still fragile recovery of the economy we've seen a lot of interest for mobile access to our applications. As our applications are web based the basic access is there but with more and more sophisticated smartphones just simplified CSS and layout that fits the small screen doesn't cut it anymore.

I've been using iPhone for quite some time now. Besides the usual stuff everybody would do I also spent some time testing out the API for the kind of boring business applications we do. While the basic search/form screens are pretty easy to do (I can hardly imagine a platform where that would be difficult) I hit quite a number of problems with more complicated apps. While most of those issues disappeared on a jail-broken phone, I can foresee some resistance asking my customers to jailbreak their 20 phones. Technology issues aside there's still a major problem with distribution. Pretty much the only way to effectively distribute the iPhone applications is through AppStore, which may not be the most ideal way for our kind of applications used by a few people. While there is an enterprise license that allows private AppStore it's available for companies with minimum of 500 employees...

Thanks to the CodeXtreme competition I got to play with Android. Of course, it doesn't feel as 'cool' as iPhone...which, luckily is not so much of an issue with enterprise apps. Despite the fact I was happily (almost) Java-free for the past 4 years I found the java based development to be a positive thing. Not only because Java has become THE language at universities around and even fresh grads don't feel totally lost and overwhelmed by a completely new language and environment but I actually found it quite efficient for writing our kind of apps. There's already quite a number of open source apps around that can help with inspiration or solutions to many problems. To mention freedom...Android allows me to do whatever I need or want without any artificial restrictions. I have to do whatever the customers require and it's quite enough to fight with actual constrains, like screen size, Internet connection, GPS timeouts without having to worry about someone's idea of what should or should not be allowed. On top of all that Android not only allows me to do whatever I want - it actually allows me to deploy my apps. The distribution channel is open (not to mention free). While I understand the arguments for AppStore as a guardian of purity and quality of distributed applications...it's not really workable or applicable in my business.

My point is that just like with many other technologies, smartphones are slowly moving from home user to corporate user. And corporate users have quite different needs and expectations. It's not so much about coolness and effects as it is about getting the job done. Whether it is to access CRM, process on-site sale, collect data, fleet/position monitoring, authenticate access... Currently, we're working on a few bigger projects with Android integration. Two of them are mostly for data collection. While data collection has been around since like forever - a simple integration of camera, barcode scanning support, GPS and immediate data update on the server gives it a completely different dimension. Instead of waiting months for the paper forms to be processed you have instant results with a lot of important metadata. You can immediately see the trends and adjust as you go. The rest of the applications we're working on utilize GPS (location based systems). Just recently, the government implemented a system for tracking social workers. They fixed an RFID tag on the doors of senior citizens that the social workers need to visit. The social workers are given portable RFID readers and on arrival to the place they scan the tag. On their return the scanned data is synchronized with the system to plot the social worker's visits. We've got similar requests from a number of companies - from courier companies, through transportation companies to pretty much any company that provides delivery or onsite services. In many such cases it's not an option to use RFID (aircond repair shop can hardly put RFID tag on all their customers doors). While there are solutions for fleet tracking, the cost of $200 per vehicle per month is not only too much for a small company but it's also tied to a vehicle. While the AGPS is far from perfect even with all the timeouts and wrong position readings it still can provide quite accurate and very timely position.

The technology usually doesn't change in the vacuum. During the previous economic boom we were replacing DOS and Access based applications with web based systems thanks to the rise of cheap Internet and cheap portable computers. Most of our clients (especially business owners) didn't initially really grasp the advantage of being able to access their applications from anywhere - what really mattered at the end was that they could access it from home. After all, they've been working from the office their whole lives - as that's where all the resources and all the data were. Then they slowly started using the systems from outside the office...you know check customer's phone number, check the order status because they got a call from customer...soon they were leaving office early able to do parts of the work in evenings from living room, coming in late - being able to check sales, invoicing, bills, attendance, work orders or any other operational details from home just as well as from the office. Today, the same customers have smartphones...not just because it's cool - but because they can access their data from ANYWHERE...which is kind of cool, after all. The paradigm is shifting again.

Professional Certification

Just a few days ago I was laughing at real-estate agents that they may soon be required to pass an entrance examination to test them in practical knowledge as well as moral soundness (out of curiosity will this be a multiple choice test?). While it was expected - they are pretty much the last profession without any education or knowledge required, yet they represent you on your single highest monthly expense (in SG). The fact that there is no law regulating the real estate market does not make the situation easier - it doesn't mean no law - nothing to know - everybody plays by they own rules. Two days later I stopped laughing...seems that we're next...

As I've said so many times on so many places, I really believe in professional certification (and I completely respect those who don't), however, it really matters how this certification is done. For that reason I (mostly) don't really care about academic (university) results as they mostly represent ability to know a little about many things but nothing about one thing. They are very important in other aspects, though. As explained in books like The Talent Code and Talent is Overrated I believe that to really know something you need to clock-in your 10,000 hours of practice. I teach ruby/rails programming in 2.5 days course and I am able to cover almost everything to get you started writing even ERP solution, but does that mean that after 2.5 days of intensive training you're proficient in ruby or rails? Yes - if you spend the next few years working with it.

As such I was first quite enthusiastic to read that the government is stepping in to regulate our profession as well. Thanks to the thousands of failed projects and millions of government money wasted they decided that the current practice of audit after the project completion is as effective in weeding out failing vendors as a postmortem is in weeding out smoking. While many times the problem is on both sides - the vendor's as well as the client's - the experience of the vendor can make up for a lot of issues on clients side. Most of our clients went through a software development experience once or twice in their lives - while we go through it every day. I am talking about SME market here - where the projects go anything but as planned. Running a small business is like yachting - constantly monitoring the wind, the sea and reacting to every small change - either capitalize on it or run from devastating effects. While it's essential to standardize the processes - otherwise there's no business to run - you have to be ever ready to unfreeze the process adjust and freeze again. And your IT infrastructure either moves with you or against you. That's why the SME landscape looks so different to government regulated, without exception SAP dominated MNC market.

Back to certification, imagine my surprise to find out that the certification has been outsourced to an NGO that specializes in Microsoft certification. In fact, they do exclusively Microsoft certification and now this. First I thought, this may not be so bad - I can surely brush up on some VB craft if that's what it takes to be professional in 2010. But it gets worse. There is no preparation and there is no exam. Your professional level is black box decided solely based on 4,000 words self assessment essay of your past 6 years. No interview, no actual review of your work - 4,000 words of narration. On top of this you of course have to sign up for membership in this great organization because the certificate is only valid with a paid membership. Talk about business model - the costs - read a short essay - the revenue $800 for reading and $100 every year after. And since it's becoming a government requirement - not only the new members HAVE to come to you - even the old ones HAVE to stay - a truly new level of marketing and retention. I think universities have a lot to learn here - imagine your degree is valid so long that you pay a yearly fee. But I understand - even NGOs have to make money - and if you liken it to bar certificates lawyers have to pay every year it's not even so unusual. What still worries me a little is that Microsoft is to decide my professional level. I have a great respect for them - the company as well as the products. The problem is that I simply slipped during university and I somehow didn't get to back the right path ever since. I can only imagine a Microsoft grown professional reading my essay - my master thesis was on open source (this was back in 2000 when open source was fairly new to the business as well as academia). Besides having converted number of companies data centers from windows to linux with the exception of 1 software project all the others were done using non-microsoft technologies like Ruby, Java, Python, PostgreSql. If it was not enough we've been pioneering agile methodologies since maybe 2002 - I have not worked on waterfall model project since 2001.

Some of my friends actually suggested to write the essay as I normally would and then simply search and replace...Linux for windows, posgresql for MS Sql, java for J++, ruby for VB.net or C#, agile for waterfall.

Don't get me wrong - I am all for professional certification. All the other professions have it - so why not us. Lawyers have to pass the bar, accountants have ACCA and CPA, doctors have their attestations, even my mother as a teacher has to get recertified every 2 years. Even not so knowledge intensive professions have certifications - e.g. taxi drivers here are tested on the knowledge of the shortest path between any two points in Singapore. Of course, all those tests are considered only a prerequisite for practice - nobody really things that passing a bar will make you Allan Shore or Denny Crane. What it does is that it gives you guarantee that your lawyer or your doctor or your accountant achieved certain industry recognized standard and passed independent assesment. Why is independence of this assesment so important? After all, market numbers (or any numbers for that matter) clearly show that MS knows best how to do things. How would you like if every doctor would have to be certified by Pfizer or Merck or whoever had the best market numbers last year? Every doctor has to use their drugs anyway...

I definitelly think the governmet deserves a lot of credit for trying to improve current situation. Maybe it won't be as simple as appointing one company to award black box professional certifications...but it certainly is a step forward. And of course I know it won't be so hard...we'll all have to pass it after all.

Rehab

And it's here. I am in rehab. Okay...not really...I managed to sneak in my iPhone. But otherwise rehab in it's nicest form.

We left in the morning to have some nice breakfast...changed the plans midway to maybe make it an 'overseas' trip and get lunch in Indonesia (just a boat trip away) and when we got here we decided to stay a night in one of the resorts...

Man, how I miss these short trips...they yank you out of reality and maybe because of the sudden nature have even stronger effect than long planned holiday. This time it's even amplified by the fact that I left my notebook at home (completely unsuspecting). Over the years I got used to working pretty much anywhere...on planes, airports, coaches, starbucks and even public transport. Anyway, I've been planning on running away for some time now. Usually when we go somewhere we set up the main tent around bar area with wifi and easy access to drinks. Not this time...as all I have is iPhone...and luckily free 3G Internet from SG (only on the beach). BTW the photo is real - it's the view from the hotel restaurant.

Anyway...computer is not the only addiction I got the break from lately. Thanks to the economic meltdown I got at least a short breather from the break neck speed of events of the past few years and finally got time to sit down and think. Think what out of the 1000 things I did which was the one that actually made some sense. Should I look at it as Thomas Edison...I had no failures I just found 1000 ways of how not to do things. If I look at it from any sane point of view the statistics are much more bleak. But...the new day is coming...and hopefully yesterday won't be forgotten. At least by me I hope. One thing I learned when helping out in changi prison and later working with the guys after release is that no matter how long the rehab, no matter how much time you have to contemplate everything wrong and every possible better way, your old ways only become more and more traecherous. You already know what and how can get wrong and that makes you believe that, as Eddison, you are getting better when, in fact, you're falling into the same trap...but because of your experience you'll be able to get deeper and deeper. I met number of different people inside...mostly drug addicts (careful - the word has a different meaning in SG than in many other countries) but also gamblers, fraudsters and fighters (even a hit man). While the drug addicts are pretty much doomed (all of them desparately plan to leave for another country but as they can get passport only after 2 years most most most most most of them can't last clean longer than a year...that was a 'most' for every hope). The fighters have it hard but they stand a bit better chance. Whom I found especially interesting were gamblers. Entrepreneurship and gambling have surprisingly a lot in common...way more than someone on a regular salary would imagine. You could almost say that gambling is enterpreneurship stripped of the hassle of idea, product, employees and pretty much everything besides the money. You start with no or very little idea where you'll end up. Sunshine doesn't last all morning...clobber doesn't last all day. What's the difference? When you run a business all this neusances become your chips. They can turn against you or they can turn to be your winning asset. Your only asset. I've probably lost more in wrong investments than any of my gambling friends...just like I've (l)earned way more than most of them. Not because the stakes are higher...because every failure or win is not just about winning or loosing...because it's not about what happens after I've won or lost....it's the journey (battle) that counts. As Andrey Bolkonsky in war and peace said....wars are not won by strategies or numbers....they're won by soldiers that want to win the war. And in every single case it comes down to that...I should say it again...but no matter how many times you've heard it you cannot possibly feel the pain of those words until you've lived through it. It's like listening to a marathon runner saying it was realy hard on 24th kilometer but than I decided to go on. If you've ever been there you know that he's saying that his feet were hurting like crazy, his knees checked out, his heart throbbing, his breath was nowhere to be found and the last pieces of sanity was saying you've proved your point...of course you can do it..,but who really cares??? Foolish...stupid even...ask any enlighten egoless swami....yet all this vanity...is it really vain? I don't know...I feel like I don't know much anymore.

I do hope that this coming upturn I will have more faith in me and in what I am doing than any opportunity hunter will have in me parting with my money. After all it has, without exception, paid for all the other 'cannot lose' ideas. The other ideas were not much more stupid than whatever I am doing...in fact the strategy was way better...in most cases the numbers were incomparably better...yet it somehow reached a point were it stopped being worth fighting for. But...why? Recently I was asked if I blieve in feng shui. After all, I did spend quite an amount of money on feng shui consultants. It never had any effect...when the consultant said you have a wealth door and I was sitting in wealth section we were in fact loosing money. When they said we should move out immediately we were in fact making money. When I pointed that out to one of the feng shui master he seemed completely unsurprised...he said I am trying to do things first and hope to believe in them later (ie when the facts can substantiate it) - unfortunatelly faith works the other way round. It picks up where reason leaves of...but if it's not there before reason took over it want be there after neither.

It's easy to believe when you see the proof sitting in front of you...but by than it's not believing any more...it's simply accepting the facts.

The Passionate Programmer in Business

Last week I wrote about The Passionate Programmer, about my and Chad's very similar ideas on many things. I closed by saying that it's hardly feasible or sustainable for any SME to base the business on passionate programmers. Imagine my surprise opening the PragPub Magazine yesterday where Chad Fowler talks about replacing your people with the worst people available. The idea is quite old and comes from Michael Gerber's The E-Myth Revisited. I read this book back in 2005 and I've been in internal fight with this book ever since. Even in my bookshelf it sits right next to it's antithesis Good to Great by Jim Collins. The idea Chad is getting at is that you should replace great people with great systems and get mediocre people to be run by those systems. Much like a clock - made of hundreds of small cogs and pieces, each easily replaceable and while totally insignificant alone, having a specific, clearly defined and essential role in the clock.

Let's backtrack a bit. I really loved The E-Myth Revisited book for the questions it asks. In fact, a bigger part of the book is dedicated to depicting a typical small business. Painstakingly describing day after day with such a detail and accuracy - I don't know a business owner who wouldn't resonate with every page. Unfortunately, it leaves out most of the answers. Actually, the answers are there, but while you can really feel the pain of the questions, the answers are put almost bluntly - like an external consultant just took over and reads his 6 steps to happiness. It took 10 years for the real answers to arrive. They're in the E-Myth Mastery. While this book never neared the popularity of the first one, I found it infinitely more useful. The main difference is that while the questions remained, the second book provides actual answers. While the first one is based on observation and consultancy, the second is based on first hand experience and the author lives through every sentence. I bought the book in 2006 but I didn't get round to reading it until late 2008. And it was a deja-vu. I read maybe 50 books over those 2 years - yet I didn't read the one that could/would have actually helped me. It was like somebody wrote a book about the what was going to happen, but I couldn't read it because I was too busy heading for the catastrophes I could have avoided if I just read the damn book.

Now back to the good vs. bad people question. Our business is streamlining and automating of business processes. Over the years we worked with companies from more than 30 different industries. Anything from travel agents, through media, manufacturing, logistics to financial consultants and portfolio management. People usually get our systems to either decrease their staff numbers while increasing performance or keep the numbers and radically improve performance. The way to do it is based on what Michael Gerber says - take a process and orchestrate it. In our case, it's followed by computerization, where the computer system does the heavy lifting. In Michael Gerber's case, once the system is orchestrated, the cogs (aka the least qualified people you can find) can be put in places and the show is ready to go on. Your role as a mastermind, is to grease the cogs, innovate and move from technician to manager (and later to businessman). This has worked remarkably well - just look around - this is the principle behind every McDonald's, Burger King, Starbucks, Subway, and so on. As a freshman in university I applied for job in McDonald's but at the end chose to work in research institute. I regret that decision to this very day. While I made considerably more money working in the research institute I lost opportunity to learn about how one of the very successful businesses is run. On the other hand, I've seen a number of places where just systemizing didn't work all that well. Take accounting as an example. The process is well defined (even though with considerable amount of leeway in asian countries), there are formal checks in place like audit, tax checks and so on. Yet I've seen it in quite a mess in a number of places. While there are better accountants just like there are not so great accountants most of them are able to do accounting. The cause was usually somewhere else. Most of the business owners are masters of their trade, they are the masterminds behind the systems and they are the puppet masters running the operation. When it comes to accounting, however, they're lost. So they hire an accountant. And then another. And data entry clerk. And temp staff because of the back log. And still the accounts are in mess. Buying a better (more expensive) accounting system doesn't help much neither. What's missing is usually leadership. It's not an accountant the technician but accountant the puppet master. Human systems don't tend to be as iron clad as the clockwork or jet engine. That's where the 1 obsession of the extraordinary executive (build and maintain a cohesive leadership team) comes from. You cannot be everything to everybody and people are not cogs - you can't just put them in places and expect them to function flawlessly.

I've always believed that you get only as good people as you deserve. And I've paid the price to learn this again and again and again. Most of the work I've been around is not linear. You can hardly say that if one CEO produces 1 mil of profits five CEOs would turn the same company in 5 mil business. Just like if a system takes 1 developer 3 months to develop it doesn't mean it would take 3 developers (offshore or not) 1 month. Just like if one sales person can close deals worth 100k per month it doesn't mean 3 of them will close 300k.

The Chad's article goes on to explain business automation likening it to automation of software development. While interesting to read - this is how software developers must feel when being compared to construction workers :-). I guess (literally guess) that Chad has passed the euphoric stage in his business and tries to see where is all that going. I wouldn't be surprised if it looked awful lot like working in a big company he left, but with way more responsibilities and way less money. That's where I went through the same Freedom musings. Hopefully, I am wrong and the wine and roses will last forever. They didn't for me, though. Time came when I realized that my business was just a business - without the magic IT formula from The 50 Great E-Businesses and instead of counting millions or billions on my account I had (and still have to) change from passionate programmer passionately pursuing his own sweet career to someone running the business. It's not just about me anymore...

Anyway, today I found a book that does a great job explaining just how different running a business is from software development. The book is The 100 Best Business Books of All Time. I am not saying that reading this book or even all 100 books listed will make you the best business person of the century or that it will solve all your problems. I merely believe that reading some of them will help you see what's on the other side and consider it before it's too late. There's a saying that smart people learn from mistakes of others...Never happened to me, but I hear it's great :-)

Passionate Programmer

It's really been some time since I wrote something. I've been really busy - more than usual busy with all the projects but additionally busy trying to workout a solution to some of my perennial problems.

I finally managed to read The Passionate Programmer. It almost felt like reading my own notes. Over the past few years I've been talking and writing about quite a number of issues from this book. It's almost unbelievable that somebody has so similar views. Starting with the saxophone experience - I played saxophone for 10 years - I can relate really well to Chad's stories and parallels. Of course, that's not to say that because some things work in music they will work in programming as well. As Martin Fowler once said software development is software development and we shouldn't be comparing ourselves to other industries and drawing misleading or wrong parallels. Anyway, regardless of whether the conclusions are right or wrong the flashbacks to saxophone playing experience added extra level of depth (at least for me).

Out of the many ideas that struck a cord with me...here are a few I noted down

Invest in Learning
The nerve!!! Judging by the reactions I receive whenever I talk about importance of proper education and practice for programmers - it's ranking among the worst insults you can ever make. Every programmer worth his salt knows that learning is for dummies. We were born better than them (basically everybody else). Sarcasm aside, I believe that a lot of programmers would benefit from experience of playing a musical instrument. Daily (i.e. every day) practice and performances. When you're performing on stage without practice - okay let me rephrase that - when you're on stage without hundreds of hours of practice - you're very likely to screw up - and if you do you're like naked. There's nowhere to hide. The whole audience looking at you and everybody knows it was you who screwed up. Believe me, the next time you'll think more than twice before coming on stage without proper practice. When you program without practice (or learn on the job) you're just as likely to screw up...the only difference is that you can blame it on the customer, on the specs, on the technology, or whatever or whoever else around you. Many times you can just comfortably hide behind your boss as he/she is the one facing the customer.

Work expands so as to fill the time available for its completion
This is an interesting concept. In the industry where everybody complains about way too tight deadlines - Chad goes on to say that just by making the deadlines even tighter you can easily double or even triple your productivity. While I am still to meet an employee that would confirm this - it's definitely true for me and all my ex-staff that went independent. It's really fun to see that things that took them several months to finish while being employed can suddenly be done in a month when they're on their own.

Writing Skills
"If you can’t organize your thoughts in your mother tongue so that others can clearly understand them, how can we expect that you can do it in a programming language? The ability to shape an idea and lead a reader through a thought process to a logical conclusion is not much different from the ability to create a clear design and system implementation that future maintainers will be able to understand." While this politically incorrect statement would get you to prison in a number of countries it's surprisingly true. And quite opposite to the geek stereotype. Interestingly enough, ability to speak and organize thoughts clearly has become a very important factor for us. Try to notice the similarities between way people speak and code - if you have a programmer that is very verbose - talking, talking, talking and still not making the point clear - chances are his code will be very similar - long winded, using unnecessary constructs and never quite clear on what it's supposed to do. On the other hand, if you have an introverted programmer that says 10 words a day, chances are his code will be very terse, taking one liners to the whole new level.

Businesses and those who run them are interested in business results
A thing you learn really fast when you're on your own but it's your biggest enemy when you're an employee. I remember a fight I had with one of my employees - his conclusion was that he just wanted to have a life. And he was right. Why should he worry about the deadlines or delivering what the customers needed? His goal was to have a life and the work was one of the main constrains - there was no direct link between his performance and quality of life. Not seeing and understanding this was my biggest failure running the business earlier on. While we had the basic performance based structure in place it was so detached from the company's culture and pretty much left for everybody to make their own interpretation. The challenge is to reorganize from a black box to a concert stage where everybody is crystal clear on how his performance is linked to the achievements of the whole team and how that translates to the distribution of rewards. Chad puts it very bluntly: "Sure, you got it done, but what was it? Why did it matter? How was this so-called accomplishment not just a waste of company time?" He couldn't be more right. I cannot even count how many times people told me that they had been making sacrifices, sitting in the office 8 or 9 hours every day and I still complain about their performance. The issue here was that while they counted hours I was counting closed iterations and customer satisfaction.

It's easy to get hung up on technology choices
Especially when our technology of choice is the underdog... I believe this idea comes from the Pragmatic Programmer - something along the lines of when all you have is a hammer everything starts to look like a nail. I've always tried not to be attached to technology, however, having a 50 projects investment in a technology makes you very close to being dependent on whatever choices you made. Yet, an investment in wrong technology - no matter how much you like it is usually much higher than investment in a different technology that is actually right for the job.

Go independent
This is quite a dangerous point. While definitely true - there are uncountable advantages to running your own show - just go for any motivational seminar by T. Harv Eker, Robert Kiyosaki and the likes - but being the best technician is only a start. Even though, based on the book 50 Great eBusinesses there's a number of teenagers without any experience that are managing billion dollar businesses (or one man shows) out of they're parents living rooms - vast majority of businesses and people are not so lucky. While Chad describes this as natural next step and spends only a few pages explaining the benefits and some of the obvious challenges it's a whole new ball game - one that has very little to do with programming.

Finally, the book got me really thinking. I've been running the software business for 7 years now and while we've been fighting with lots of problems, the single most difficult thing has been to get the right developers. I wrote about different aspects of it in a number of posts analyzing it from many different angles. This books is/says exactly what I've been trying to say and find for all this time. So many people told me that my ideal developer is a mix of too many and too different skills, and that such animals are more rare than a unicorn. On one hand it was very encouraging to see that somebody else has the same ideas and expectations, on the other it's quite a gamble to base your business on something so rare. Yes, it's possible to find one or two and get started. It may even be feasible to hack a new Web 4.5 killer app and sell it to Google for millions of dollars, but it may be much harder create a sustainable business out of it. It may work for huge multinationals that Chad is talking about as they can afford $300,000 packages but hardly any SME can do that. In fact, I couldn't sleep for almost 2 weeks trying to find a solution questioning my every belief and experience with this business. I tried to unhung my self from the technology, the product, the people and try to weigh the price and the risk of each option today and 10 years from now. It's been quite an exercise - I am now iteratively implementing the changes - if at least 25% of it works out this could be a book that totally changed my business and life. We'll see in half a year :-)

Ext JS Image Field

Yesterday I spent way too much time on another simple thing in EXT that I would have expected to work out of the box. What I needed was to upload a company logo to a profile and, of course, display that logo in the form. While it's very simple to display an image in a panel or box, it's a totally different thing to display the in image in the form and update it together with other form fields. What I needed is something like this:

and after form submit it should automatically load the new image:

For image upload I used the image upload plugin from EXT examples. To display the image I needed to extend the field component like this:

Ext.form.ImageField = Ext.extend(Ext.form.Field, {
    autoCreate: {tag: 'img'}
    ,setValue: function(new_value){
        if (new_value == undefined || new_value == null) {
            this.el.dom.src = '/images/no_image.png';
        } else {
            this.el.dom.src = '/images/thumbnail/' + new_value;
        }
    }
    ,initValue : function(){
          this.setValue(this.value);
    }

    ,initComponent: function() {
        Ext.apply(this, {

        });

        Ext.form.ImageField.superclass.initComponent.apply(this);
    }
});

Ext.reg('image_field', Ext.form.ImageField);

This is a bit simplified version as I removed the rails specific code, but anyway the main points are here:
line 2 creates image tag (rather than the default text field tag)
line 5 defaults the image src property to a placeholder (in case there's no image uploaded)
line 7 - in case there is a valid image it will change the src path to that image - in my case it's routed to Images controller and action thumbnail which returns a downscaled version of the image.

Other than this don't forget to set the FormPanel's fileUpload: true and refresh the form on submit success. I usually send the updated object JSON in the response to update action.:

  def update
    contact = Contact.find(params[:id])
    on_bind(contact)
    success = contact.save
    contact = Contact.find(contact.id) if success
    
    logger.info("saved")
    
    respond_to do |format|
      format.ext_json do
        json = contact.to_ext_json(:success => success,
                                   :methods => [:image_id],
                                   :includes => [:address],
                                   :messages => {:notice => l(:contact_updated), :warning => l(:contact_update_error)})
        render :json => json #:json => {:success => success}
        headers["Content-Type" ] = "text/html"
      end
    end
  end

!!!Notice line 16 - you have to manually set the content type to "text/html" because rails will default to "text/javascript" which will not work with file uploads!!! It works with fileUpload: false but it doesn't with fileUpload: true. I don't really know if Rails or EXT is right here but did take me some time to figure this out.

this will produce the following JSON (I removed the unnecessary fields):

{"success": true,
    "messages": {"notice": "Contact was successfully updated.", "warning": "contact_update_error"},
    "data": {"contact[image_id]": 15, "contact[active]": false, "contact[client_account_id]": 2, "contact[client_account_no]": "100",
        "contact[created_at]": "2009-07-07T10:31:54+08:00", "contact[credit_term_id]": 1}}

Now in the forms success method I can use the result.data to update the form:

success: function(form, action) {
                      form.setValues(action.result.data);
                  },

Losing Faith

Lately, I've been going through a lot of frustrations with Ruby (by ruby I don't meant just the language but the whole ecosystem and infrastructure based on Ruby). I am in a finishing phase on several projects. It's a time when the finer details get addressed. When there's no more time to do it later, differently or drop it all together. It's when everything that is supposed to work should actually work. The problem with Ruby is that it's been in this kind of finishing phase pretty much ever since I started using it (2004). If you look at it from far enough it seems like there is everything in it but when you come closer and actually try using it you'll find out that a lot of it is in less then pre-alpha stage (of course labeled as production). No language is an island unto itself. No matter how great the language you probably don't want to create all the basic libraries from scratch.

When I made a switch in 2004 Ruby was fairly new, Rails was fairly new so I expected this kind of problem, but you could feel the change in the air. Pretty much all the industry heavy weights were talking about it - Rails conf had a better key note line up than any other conference around, number of books was written and the whole Ruby ecosystem was created over almost overnight. Sad truth is that many of the heavy weights are still talking about it today but they're yet to do their first project in Ruby / Rails. Despite the explosion like surge in popularity little has changed in terms of actually usable Ruby libraries (actually I have the same feeling about 1.9 as well). This is a stark contrast to some of the uncelebrated languages out there. For example I find myself writing increasingly more and more code in Python. I don't think Python is the next Ruby, but I was quite surprised how easily it addressed things I struggled with for 5 years in Ruby.

An example could be excel libraries. I still use the same excel libraries I did 5 years ago (for export I switched recently to xlwt, for import because of the changes in roo I've been using JXL for years now switching to xlrd). The export is great only it doesn't support much formatting and breaks on spreadsheets over 10MB (actually somewhere over 6MB) and the files sometimes don't work on later versions of MS Office. There's been a rewrite that is able to go over 10MB but still without formatting I would need and with a license that doesn't allow commercial use. Python's xlwt - despite a very crude and unpolished appearance is able to produce a complete excel spreadsheet with all the formatting (numbers, dates) (including page setup) and all that in much shorter time and with less resources (Ruby 1.8 4m 28s, Ruby 1.9 2m 36s and Python 33s). I have quite a few more examples but I really don't mean this as a Ruby - Python comparison. This experience, however, got me thinking if Ruby is right for the job I've been trying to do (ie uncool enterprise systems). There are pretty much 2 things for me in Ruby (okay - 2.5 the 0.5 being RubyMine - I'll write about it in some other post). One of them is expressiveness of the language and the other is Rails. Ruby as a medium is incredibly expressive, reads almost like English and allows to do pretty much anything (iterating/mapping an array in 1 line, etc). The down side - as Robert C. Martin put it - is that it's really easy to make a mess. After several months of going through somebody's else code I couldn't agree more. A nice entanglement of block and yields and procs can challenge most of the PHP spagetti code any day. While this expressiveness is incredibly powerful, many times I opt for more verbose solution - Java programmer's code on Monday morning as Gregg Pollack would say - that I will be able to understand 2 years down the road - or that my junior developer will understand.

The other thing in Ruby for me is Rails. The only thing with Rails is that after the years it's becoming a bit too entangled, while it made a lot of sense initially to bundle everything together (ORM, routing, templating, etc.) initially now it starting to be a reason why Rails slowly falls behind in certain areas (either that or Rails team has simply decided to focus on Web 2.0 start up market and let others handle enterprise problems). As surprising as it may seems the world around has not been snoozing after Rails was created and many people took the best ideas (just like Rails did from J2EE world) replaced what didn't work and the whole industry moved forward. While few years back Ruby and Rails were pretty much necessities now there's a lot of alternatives and in many different language. To channel out my current frustrations I started experimenting with quite a number of frameworks and platforms. After so many years I really know what I want and how it fits with my existing projects and infrastructure. Very briefly -

• I am looking for a modular framework where I can choose different ORM, different templating engine and so on where each component is developed independently by people who understand that specific area. As each of my project is very different sometimes I need the flexibility to change components. It was a great thing back in my Java times (Spring + Hibernate + Webwork). 
• I really need an enterprise friendly dynamic language. Life's just too short to be wasting time trying to force a language to do things it's inventors don't consider important. 
• I need a basic ecosystem of libraries (from XML, JSON, through excel, PDF, OO to image manipulation). 
• I need a framework that supports TDD out of the box. 
• I need a platform with (at least some) formal education and certification available. I need it because I employ people and certification really helped me back in Java times. To pass even the first Java certificate required people to study and to really understand the language and they simply took it as an independent measure of their skills. 
• I need a language with a proper IDE. I use Vim every day - BUT not for programming. I've been using RubyMine for more than a month now and I feel like a blind man that got his eye sight back.
• I need reasonable deployment option - both scalable and resource efficient. 
• Increasingly more important - I don't want the language/platform community to offend others

It's almost uncanny how many items point back to Java :-). Actually, it's kind of sad that things I took for granted 5 years ago in Java are still not available in the most celebrated and most wildly endorsed language of today. If you look at it - nothing really substantial changed in the way we do the backend (MC) part of the systems. A lot has changed on the View side, but the models and controllers use the same principles as years back. I've used Python/Pylons quite a lot lately and it scores quite high on a number of areas (only so-so on the IDE, though). It seems much more mature compared to Ruby with more matured libraries supporting the enterprisey stuff (so ostracized in Rails community). On the other hand it's marketing is nowhere near Ruby's - you don't really hear people talking about wishing to work in Python. I am a bit worried about the performance - but that's something I just have to try on a real project. A big issue still remains the education and certification. I went through this with Ruby already - no matter how easy and great and superior the language you still need to spend some time learning and then even more time working in it before you really understand it. While there's quite a number of books on Python not much of training courses (at least not in Singapore) and there doesn't seem to be any certification around.

Extracting Data From MS Sql Server on MacOS

I've had really a rough time lately getting data from legacy systems based on MS Sql Server. In each case I got only the .bak backup file and had to do the rest. The whole thing is a bit easier if you run everything on windows, but as I did the development on Mac I'll cover how to connect to MS Sql from MacOS as well.

Restore database from backup

You will need a windows machine with MS Sql Server running (of course :-) Luckily, you can download it for free from here. Next connect to the database server using the command line utility osql (you will need to have it in your PATH)

cd C:\Program Files\Microsoft SQL Server\MSSQL\Binn
osql -E

You may need to create a new user:

use master
go
EXEC sp_addlogin 'peter', 'pass45'
go

Now let's have a look at the backup file (put it in some easy location so you don't have to type out the path):

Restore FILELISTONLY FROM DISK='c:\AMS.bak'

This query allows us to find out the logical name of the database and log file which is needed to appropriately restore a database to a new path. What you get is basically an original location of the data and the log file. In my case I got:

AMS_Data      D:\Program Files\Microsoft SQL Server 2000\MSSQL\data\AMS_Data.MDF
AMS_Log      D:\Program Files\Microsoft SQL Server 2000\MSSQL\data\AMS_Log.LDF

Which means that the original installation was on drive D:\. As my temp server is on C: I will have to recover with changing the locations of the files

RESTORE DATABASE ams
FROM DISK='c:\AMS.bak'
WITH
MOVE 'AMS_Data' to 'C:\Program Files\Microsoft SQL Server\MSSQL\Data\ams.mdf',
MOVE 'AMS_Log' to 'C:\Program Files\Microsoft SQL Server\MSSQL\Data\ams_log.ldf'
go

The only important thing here is that 'AMS_Data' and 'AMS_Log' names match the ones from the previous query. Now you should hopefully see that your database has been restored. Now, will just get access to the database:

use ams
go
EXEC sp_grantdbaccess 'peter'
go
sp_addrolemember db_owner,peter
go

There's more info on how to use osql here. Now we should be set to connect to the database from development machine. This was quite a number of steps just to restore the database - compared to createdb tmp; psql tmp <>Connecting to MS Sql from MacOS We will need to install and set up a couple of things:

sudo port install rb-dbi +dbd_odbc
sudo port install freetds

set up connection to database

/opt/local/etc/freetds/freetds.conf

Add your server to the end of the file:

[Ams]
 host = 192.168.11.106
 port = 1433
 tds version = 8.0

where the host is IP of your windows machine. After this step you should be already able to connect to MS Sql Server:

tsql -S Ams -U peter -P pass45

You should get the server database prompt (just like the osql on windows machine). You can try some commands like

use ams
go
select * from contacts
go

Now set up ODBC connection. Go to Applications -> Utils -> ODBC Administrator 1) add driver with following descriptions:

Description: TDS
Driver File: /opt/local/lib/libtdsobdc.so
Setup FIle: /opt/local/lib/libtdsobdc.so
Define as:   System

2) add User DNS DSN: Ams Desc: old AMS database server add 2 keyword/value placeholders. To update them click on it and press TAB. Set it to following values:

ServerName: Ams   (or whatever you set in freetds.conf)
Database: ams   (or whatever your database name)

now you should be able to test iODBC. Note that I am using sudo for this as it doesn't seem to work without sudo complaining Data source name not found and no default driver specified. Driver could not be loaded (0) SQLSTATE=IM002.

sudo iodbctest "dsn=Ams;uid=USERNAME;pwd=PASSWORD"

You should be now in interactive terminal again. Once all this works connecting from Ruby is really easy:

require 'rubygems'
require 'dbi'

DBI.connect('DBI:ODBC:ams', 'USERNAME', 'PASSWORD')

Rescuing the data

This is just a short ruby script I use to extract all the data from MS Sql and import it to Postgresql. It's not any functional database conversion - it's just to get the data out to something that's easier to work with and doesn't require windows machine running. It may have an issue with binary fields - it worked on most of them but it did choke on a couple of fields. DBI actually provides more metadata like NOT NULL attribute, primary key, etc. so the script could generate a more precise copy of the original database but this was enough for what I needed. You may run into unknown data type - especially if you try to import it to a different database engine - all you need to do is just update the method update_type to return correct mappings of the data types.

require 'rubygems'
require 'dbi'
require_library_or_gem 'pg'

def escape(text)
 return "NULL" if text.nil?

 text = PGconn.escape("#{text}")

 return "'#{text}'"
end

  def self.update_type(col)
    type = col["type_name"]
    type ||= 'timestamptz'
    type = type.downcase
    
    case type
    when 'clob'
      return 'varchar'
    when 'varbinary'
      return "oid"
    when 'long varbinary'
      return "oid"
    when 'double'
      return 'double precision'
    when 'tinyint'
      return 'smallint'
    when 'char'
      return "char(#{col['precision']})"
    else
      return type
    end
  end



 dbh = DBI.connect("DBI:ODBC:ams", "peter", "pass45")

 sth = dbh.prepare('select name from sysobjects where type = \'U\' ORDER BY NAME;')
 sth.execute
 tables = sth.collect{|row| row[0]}

 tables.each do |table|
   sth = dbh.prepare("select * from #{table} ")
   sth.execute

      create = "CREATE TABLE #{table}(\n"
      create << sth.column_info.collect{|col| "\"#{col['name'].downcase}\" #{update_type(col)}"}.join(",\n")
      create << ");\n\n"

      puts create

      sth.each do |row|
        create << "INSERT INTO #{table} (#{sth.column_info.collect{|column| "\"#{column['name'].downcase}\""}.join(', ')}) VALUES (#{sth.column_info.collect{|col| escape(row[col['name']])}.join(', ')});\n"
      end
      
      create << "\n\n"
      
      output = File.new("data_ams.sql", "ab+")
      output.puts create
      output.close
      
      #puts create
    end

To import the data to Postgresql is as simple as:

createdb ams_backup
psql ams_backup <> noise.txt

Software Development as a Profession?

A few years ago I used to be quite active in Rails community and then I stopped. I haven't been to a Singapore ruby/rails community meet up for 2 years and many of my friends ask me why - if I completely moved from Ruby to Python or simply what happened. Even though, I spend more and more time working with Python, I am still spend 13 - 14 hours a day doing Ruby programming. The actual reason has much more to do with Rails positioning. Back in April 2007, when the Singapore rails community was just starting to form and we had our first bigger meeting I wrote about it here. Most of it is now irrelevant but I was quite surprised when I read my 2nd point - purpose of the meetings - especially the part about the community divide to free hackers and enterprise developers. Back then I believed both were good and necessary for the community. I still believe it today only the situation has changed. While back then it was pretty much balanced, most of the community today consists of free hackers. After Dave Thomas moved from programming to music (okay - from rails to ruby 1.9) there hasn't been anybody to represent the enterprise interest and the rails community and rails image evolved to a cool web 2.0 free hacker paradise with little interest in enterprise.

This became very clear thanks to Bob Martin's key note on RailsConf '09 and very nice rebuttal by DHH.  Actually I've listened to that key note for several times now - Robert Martin is so expressive and (using the Smalltalk parallel) so diplomatic delivering something so critical to the community. For the past year and a half, I've been fighting with so many of the things he mentions - so many things I couldn't understand but make so much more sense now. 

It makes so much sense what he says about arrogance - I usually call it the 'cool' factor. Not doing the dirty job and only doing the clean things. Belief that our tools are somehow better, that our language is so good that we don't have to follow the rules, we don't have to do the regular (enterprise) things. It may be one of the reasons why Ruby still doesn't have a proper PDF or Excel handling libraries. The problem here is that this kind of arrogance is usually very subtle - but hits really hard and on places you would never expect. 

Rails is too easy to make a mess. This single statement pretty much sums up everything I said last week about software development being hard. Yes Rails is easy - in fact so easy that it makes you believe there is nothing you couldn't fix overnight. It's so easy that it makes you believe you don't need to know anything else. The problem is that all that ignorance will come back 3 months down the road (hopefully you'll be able to quit and let somebody else handle the mess).

Clean code. You look at the method it's pretty much what you'd expect. 

TDD. As a programmer I pretty much grew up on books and blogs of people like Martin Fowler, Robert Martin, Dave Thomas, Kent Beck and others from the group. TDD was the only way I was able to work. TDD is a No 1 book on my list of books for beginners. Yet I've been having the hardest time selling TDD - even to my own employees. They all know they HAVE to do tests - but only very few of them do it voluntarily or really understand why. TDD is simply not a culture around here (SG) - in fact, many find TDD culturally insensitive - at best a nice theory but totally useless in practice - just go through the Singapore ruby mailing list. Sensitive or not - it made it to our contracts for software developers.

Lastly, the discussion goes to software development as a profession and to what constitutes the professional portion of our job. Surprisingly enough, Robert Martin seems to think it's the discipline, rules and principles, it's the tough things we have to do and it's keeping them up when the pressure of a deadline mounts.  While there are people that are against being professional (see the DHH's rebuttal), for many people around me it makes sense. It seems so obvious that no body is really doing it.

Software Development is Hard

This is something that's been bothering me for the longest time. And NOT because I want to feel good about myself telling others how hard my job/work is. The reason is quite opposite. Our company has gone through expansion and shrinking cycle for several times. Each time we change a few things, achieve better results but frankly nowhere near the ideal situation. The last time round we came up with some important structural changes, but it doesn't really stop there. Given an appropriate and workable structure, the next question is: what kind of people are we looking for? What are the hard skills and what are the soft skills they should have? What should their motivation be? 

Long time ago I wrote about 10 books to start learning software development. It was an attempt to address the fundamental areas of software development - to take a side in plethora of different possible approaches to them and say this is how we do it here. I really didn't mean to say read those 10 books and you're a top class software developer. I used to start my Ruby on Rails trainings with the following picture:

I found the picture somewhere on the net - but as an ex-java developer I owned and read every single book from the stack (even though I ditched Struts for Spring in 2003 so my stack would have Spring books instead of struts). The ruby stack is pretty much all that was available back then. It would be very nice to think that after reading 2 books you'll know everything you need to know :-). Just for the record, my ruby and rails collection today has exactly 25 books plus a Ruby on Rails Essential Training from Linda.com. Regardless of which stack you're looking at, however, the fundamental problem with the picture is that those books will teach you only very small portion of what you need to know - they only cover 1 area of software development (to be fair the java stack has books on some of the other aspects). They only teach you the language but they are not making you a programmer - just like knowing all 12 notes doesn't make you a musician and being able to write doesn't make you a writer. You still have to add a stack on patterns and principles, databases, testing, basic design, HTML+CSS, object oriented analysis and design, agile techniques and practices, meta-programming, deployment and other aspects. It's extremely important to say that those books are not novels you don't just read them - you really have to work with them, do the examples, play with it try it in different scenarios - basically deep practice with them. Reading in a few hours is totally useless and waste of time. 

This is what you have to spend your first 10,000 hours on.

The problem is that after those 10,000 hours you only know HOW ... you still don't know WHAT - a.k.a. the domain. Depending on your job this can very narrow or very broad or your every project may be in a different domain. That's not to say that you have to become a domain expert in every domain - but you can hardly write an accounting software if you refuse to learn about invoices, receivables, GL postings, financial statements and so on. While in traditional model there's system architect (or designer) that is expected to define everything to the last step with screen layouts and function of the last button and programmer simply translates the UML diagrams to code agile models require a very opposite approach. Software developer is no longer just shoveling coal from one pile to another. Instead, he/she is an active part of the development process - sitting with customers, communicating and shaping the final product. 

For example, when I was working on my first inventory control system I spent 7 months in the warehouse doing every role from unloading the deliveries from suppliers, preparing deliveries to customers, picking the stock for walk-in customers, through weekly stock checks representing the logistics department on management meetings. I was even a delivery boy for several days to understand what the drivers go through. Doing accounting systems I was a part of accounting department from data entry clerk, going through the end of the month salary preparation stress, working on financial statements for management and going through audits - working with auditors to understand what they need and what's the best way to present it to them. You may say it's totally crazy - and to be perfectly frank many of my (ex)employees do - you did not finish university to be some store clerk and if you wanted to be an auditor you would have studied finance - not programing. As one of my ex-employee (he used to liken himself to Douglas Crockford) put it - I am a luminary programmer - I cannot care about invoices and whatever other domain nonsense. 

The problem is that our customers don't give a damn about how luminary programmers we are. They don't know the difference between Java or Ruby, Rails or Spring, Apache or Tomcat. In fact, a few years back when I said I was programming in Java one of my customer asked if it's not too expensive to fly to Java every week and asked if it's because of the environment or cheap massages. They don't need a drill - they need holes. I have long 3 - 5 hours meetings with customers every day but we don't talk about programming - we talk about their issues and their problems. It's about understanding those problems and finding a solution - yet I could hardly find a solution if I had no idea what they're talking about. Problem is that many programmers I know don't even have any interest to understand.

To say it differently - not so long ago I spent 9 months in a restaurant - doing everything from serving, cashier, bar-tending to helping in the kitchen. Kitchen is a single THE most stressful place in the restaurant. Surprisingly the difference between great chefs, mediocre chefs and fired-in-3-days chefs wasn't how well they could fry the fish or roast the beef or how they're goulash tasted - in one word how well they could COOK. They all had to be able and were able to cook. The difference was in preparation. Great chefs would come in as early as 7am and started by cleaning the kitchen, they personally received all the deliveries checking the quality of everything - pretty much smelling every tomato, fish or pork chop. They would check they tools, go through the dishes, cutlery, prepare the condiments and just before opening for lunch sweep the restaurant floor one more time and check every single table one more time just to make sure that it's all perfect and ready. In the afternoon break they would talk to suppliers prepare samples and work on improvements. Mediocre chefs would turn up around 11:00 just in time  to defrost the fish heat up the soup and have a cigarette. In the afternoon break they'd be sleeping in the office - because legally, it was they're break.

The software development is very hard - but not because of programming. Being able to program doesn't make you a great programmer just like being able to cook doesn't make you a great chef. It's a given that programmer should be able to write a clean code with tests. What sets them apart is how they approach the other aspects - especially the low glam aspects. I realize that all that I just said goes right against the core values of Rails framework and general perception of Rails. After four years in rails I am very sure I am not a COOL developer working on the next, much COOLER Google or Facebook or Web 3.75 webapp. We do clinically uncool systems - dare I say the word enterprise systems and only a part of the work is actual programming.

I started by asking about motivation - now looking back at what I said - one would have to be crazy just to do all that stuff once - what kind of an insane person would suffer through all this learning and then suffer again on every project (not even mentioning finish stress and starting stress and stress during every iteration - considering a good project is never finished it's like a living organism). Now imagine you find this kind of person, after all there all sorts of crazy people around (like climbers, actors, musicians, etc.) that go through a similar ordeal - how would you pay them? If a fresh grad without any experience asks for $6,000 because he has an option of stress free MNC job (at least here they do) - what can you offer? Martin Fowler once said that they start charging clients on 1 million - when it runs out they charge again. We're almost there - but not quite yet :-) 

Last week I read Talent is Overrated by Geoff Colvin. There is a section dedicated to the the motivation of top performers which is very interesting - so I'll come back with more on this topic (hopefully) soon.

The Talent Code

I read an interesting book couple of weeks ago. The Talent Code by Daniel Coyle. While the whole book is really good I particularly liked the section explaining the "sudden" appearance of extraordinary talents - like Brazilian football players, Bronte's sisters, Mozart, venetian sculptors and painters and so on. How it's believed that all those people were simply born with great talent and their great work just happened. And how the reality was almost a total opposite. The main idea behind the book is that no matter what you do, in order to achieve a world class skill you need to spend 10,000 hours in practice. The skill is basically "neural connections" in the brain. Those connections are created during practice but what determines the level of the skill is the "quality" of those connections. What increases the quality of the connection is myelin by insulating nerve fibers. The 10,000 hours is necessary to create sufficient myelin. What's interesting is that it's not just any practice - the best results are achieved in "deep practice".

The author goes on to explain how any of the great talents already had their 10,000 hours clocked in by the time they were "discovered". How bronte's sisters had written tens of training books before Jane Eyre or Wuthering Heights, how Mozart had his 10,000 hours of practice in very early age and how venetian painters and sculptors got their hours of training in apprenticeship. They were not born great but started from scratch and achieved their greatness by increasingly improving their skills. Interesting thing is that the apprenticeship didn't mean that they would simply paint the whole day and after 10 years became masters. No - they had to do all sorts of things - especially all sorts of "low level things" - like setting the canvas, preparing chisels and so on for their masters. Only after they learnt the basics could they move on to more difficult things. And this is what basically constitutes the "deep training" - choose a goal just outside your comfort zone and keep on failing until you achieve it. Then repeat the process. 

To sidetrack a bit to IT - this is where I think today's system breaks down. Software development is hard. And it only starts with the technical skills - you still need to understand the domain (when working on accounting system you have to have a very solid understanding of accounting), you have to be able to work with people and translate their ideas and feelings about something they've never seen to reality. And there is no magic shortcut - you have to have your 10,000 hours clocked in before you can really do something.  Programming is like writing - just like bronte's sisters - you have to write a lot a lot a lot and about everything you can find - only then your writing will start making sense. Unfortunately, I wouldn't really count any hours spent at school. Maybe it's an unfortunate situation around here - but the IT schools here are set to producing CIO's making decisions on a golf course rather than doing programing or any other actual IT work. To their defense that's pretty much what market here wants - most of the fresh grads from IT will find a good shake-leg work in a bank or MNC paying at least $10,000 per month. I meet quite a few of those people in my trainings or recently, as the banks scale down certain areas, in interviews. 

Anyway, my point here was that only very very few people in IT are willing (or forced) to go through the whole journey - from setting up networks, installing computers, moving to servers, maintaining different operating systems, doing backup / recovery, working with databases, optimizing databases, writing SQL, programming in several languages for several years, understanding patterns going through several releases and so on. And none of that means "I used MS Access in 3rd semester".

Another interesting thing was the difference between masters and beginners. The difference was explained on chess players. The difference between chess masters and beginners is that masters can remember the whole board set up on one look. There is a twist, though. They could only remember the set up from an actual game. If the figures were in random order the chess masters photographic memory vanished. The reason why chess masters could remember the board setup from actual games is because they were not seeing the individual pieces - they were seeing the patterns. 

I picked up the book because I wanted to know more about how people learn - I wanted to understand why after a year of training and working some of my employees are not able to do even a simple task, why is it that they spent a week installing a server without any success while somebody else can do the same work in 1 hour. Why some spend months and months programming a piece of functionality creating a total mess and somehow not "seeing" the simple solution that can be finished in a few hours or days. I used to be especially puzzled about the not seeing part. Even after showing them the simple solution they just couldn't understand it and had to go a big round of all possible wrong ways to arrive at the same solution. I guess some of them see patterns while others see thousands of lines of commands. As a result, in the light of the book's 10,000 hours I start to look very differently at this kind of things. The good news is: it's learnable. The bad news is: it may take 10 years for someone to learn it.

Rails 2.3 with Ruby 1.9

After the longest time we decided to upgrade the rails version in our applications. The last upgrade was to 1.2.6 year and a half ago. As we have quite a number of applications I am quite reluctant to upgrade unless it's really really worth it. There was quite a number of positive changes in Rails 2.2 (like the database connection pooling, thread safety and built in internationalization) but it wasn't compatible with Ruby 1.9 and I really didn't feel like going through all the apps just to go through them again to upgrade it to Ruby 1.9.

After release of Rails 2.3 compatible with Ruby 1.9 I could finally go ahead with the upgrade. And it turned out to be quite easy - but unfortunately some things haven't been ported to 1.9 yet. One of the main advantages of 1.9 is supposed to be increase in performance. It can be easily demonstrated by running recursive Fibonacci test. Unfortunately not even one 

of my applications needs faster Fibonacci :-). Having run most of our apps on 1.9 for around 2 weeks now I don't really see any tremendous performance increase or lower memory requirements. Several of our systems require extensive calculations (especially bigger accounting systems) but the performance there is dependent on many other factors like database speed, array sorting and grouping and so on and I am still working out the optimizations here.

What really really really helped me during the migration was David Black's book The Well-Grounded Rubyist. I heard him speaking several times but I was quite surprised by how well written the book was. It's really sharp

 and to the point without useless 'technicalities'. At the same time it's very readable and not dry like API reference.

Okay back to Rails. As I said most of the things went smooth BUT here's the list of some that didn't :-)

1) Mongrel doesn't work - not so much of a problem for me as I don't use it in production and in development I switched to Thin. 

2) File upload on nginx doesn't work - it seems to be an issue with Rack that is bundled with Rails 2.3. This is a bit of an issue as I was just starting to like nginx and I used it for some my lighter applications. 

3) Problems with encoding - this a single issue that I didn't manage to resolve or find a workaround. We've done some projects in Europe and they have to support slavic languages with all their funny characters. As 1.9 supports so many encodings I really didn't expect any problems there but I didn't manage to get pass the

incompatible character encodings: UTF-8 and ASCII-8BIT

when data from database contain special characters. There are several patches available in lighthouse here and here but non of them really worked for me - safe for

contact.name.force_enconding('utf-8')

4) send_data doesn't work on binary files with message

invalid byte sequence in UTF-8

. I used to use it for on the fly generated images but had to replace it with creating a temp file and then use

send_file

5) ruby zip hasn't been updated yet and still references ftools that have been replaced by fileutils. I had to change it manually along with File.move reference replaced with FileUtils.move

6) quite a number of plugins stopped working and as many are no longer supported I decided to ditch a few of them. I have never been a fan of the small rails plugins that only adding some syntactic sugar not just because they break on every update of Rails (even worse with Ruby upgrade) but mostly because of their obtrusiveness. Even though they make the code shorter it's usually much less readable as you have to learn and understand the plugin.

File Downloads in Rails on Apache

Over the past half a year we worked on several systems for logistics/courier/delivery companies. It was quite fun as a big logistics and inventory control system was my very first project when I arrived to Asia 9 years ago. Last week we launched one of them. It was slightly different as most of our systems are used mainly internally but this one is also used by clients of the courier company to book deliveries. Each delivery requires printing of confirmation documents, internal routing documents, then signed and stamped documents are uploaded back to the system as a proof of delivery for clients. It has quite a number of other (technically) interesting features like integration with google maps to geo-tag the orders and notifying drivers in the area - I'll write about it soon.

As this courier company handles several hundreds of deliveries every day it requires quite a number of uploads/downloads of documents and images. The issue with uploads and downloads is that they will tie up rails processes for much longer than usual requests. The problem is that rails processes are quite expensive in terms of memory (around 70 - 100MB for simple operations). Do 20 concurrent uploads/downloads and all your rails processes and memory will get tied up - and quite wastefully as there is no 'dynamic' processing during upload or download. The process just sits there and waits for the file to get delivered. Luckily there's a simple way how to move this burden to Apache. This kind of upload/download processes will take only around 2MB on average and your rails processes are free to handle the dynamic requests.

To let apache handle file downloads use x_sendfile. All you need to do is install x_senfile module on apache and adjust you config a little. Here's the installation instructions for Ubuntu 8.04:

cd /usr/local/src
wget http://tn123.ath.cx/mod_xsendfile/mod_xsendfile.c
apxs2 -cia mod_xsendfile.c

Open apache config file and add the module line:

sudo vim /etc/apache2/apache2.conf

LoadModule xsendfile_module /usr/lib/apache2/modules/mod_xsendfile.so

Add the following 2 lines to your virtual host file

sudo vim /etc/apache2/sites-available/my_app

XSendfile on
XSendFileAllowAbove on

You will have to restart the apache and you should be done:

sudo /etc/init.d/apache2 restart

Now in your controller handling download add x_sendfile attribute and set it to true

 send_file("/home/test/big_file.zip",
           :filename => "big_file.zip",
           :type => "application/zip,
           :x_sendfile=> true)

The uploads are equally easy. You just need to apache modules libapreq2 and modporter.

wget http://mirror.nus.edu.sg/apache/httpd/libapreq/libapreq2-2.12.tar.gz
tar -xzf libapreq2-2.12.tar.gz
cd libapreq2-2.12
./configure --with-apache2-apxs=/usr/bin/apxs2
make
sudo make install

Now download and install modporter. Get it from the github. Either use the download link and unzip or just clone the repository

git clone git://github.com/actionrails/modporter.git
cd modporter

On Ubuntu I had to change reference to apxs in Rakefile as the in ubuntu it's apxs2.

vim Rakefile

Change the APXS line to the following:

APXS           = "apxs2"

No just run rake:

sudo rake

add the following to your apache2.conf

sudo vim /etc/apache2/apache2.conf

LoadModule apreq_module /usr/lib/apache2/modules/mod_apreq2.so
LoadModule porter_module /usr/lib/apache2/modules/mod_porter.so

To actually use it in your rails application you will need modporter plugin.

script/plugin install git://github.com/actionrails/modporter-plugin.git

Add the following to the application_controller.rb (application.rb before Rails 2.3)

class ApplicationController < ActionController::Base 
mod_porter_secret = "secret"
.
.

and just enable porter in you vhost file:

sudo vim /etc/apache2/sites-available/my_app

Porter On
PorterSharedSecret secret

And that's all now if you check the uploaded files they will be a ModPorter::UploadedFile Credit for this goes to Koz from theRailsWay.com - you can find more information here.

Another Training

Another training's over. Very different but equally challenging.

Seems that Ruby's been making headway to corporate world as most of the participants this time were there on their employers' budget. When I compare it with our first RoR trainings around 2 years ago with Rails 1.0 just out and more than 30% of the things still on a wish list and the only way to learn RoR was Agile Web Development With Rails...this time it was much more business-like. There's like 20 books out, including a (bit obsolete) video from Linda.Com and pretty much everybody knows about rails.

Another aspect that has changed is Rails versions. With Rails 2.3 and Ruby 1.9 out there's a lot of changes and incompatibilities that complicate the learning. Most of the books and reference materials use Rails 2.0 or early versions of 2.2. This makes it little bit difficult to have training on 2.3 as most of the "further" reading is incompatible. While the incompatibilities may seem easy to solve for experienced developers they can be quite exasperating for a newbies. On the other hand, people are quite reluctant to start learning something that is already obsolete.

Similar is true for Ruby 1.9, even though after burning they're fingers several times until it got finally compatible with Rails most people would still prefer to use 1.8.6 or 1.8.7.

Dynamic Columns In Ext JS Grid

I just had another unproductive afternoon. Another thing I would expect to work out of the box that turned out to be not that easy. It was not the first time I needed dynamic number of columns in the grid but until now I was always able to find a workaround. Fortunately, today I couldn't go around and had to solve the problem once and for all.

What I needed was a column model dynamically adjusting depending on the data received from data store (i.e. JSON provided by server). Something like the following:



Where each site has a different set of dimensions (rows) and different set of auditors (columns).

It turned out that the problem was actually two-fold - first dynamically change the columns and then display the editable checkboxes instead of true/false values.

To change the columns I used metaData attribute in my JSON. Including metaData attribute causes Ext.data.JsonReader to be adjusted to the new set of fields and other config parameters and fire metachange event. Here's an example of the JSON with metaData:

{
 "results": 3,
 "auditors": [
  {"dimension_name": "Audits and Reviews", "auditor_3": false, "dimension_id"
: 10, "id": 10, "auditor_4": false, "auditor_18": false},
  {"dimension_name": "Emergency Management & Fire Safety", "auditor_3": false, "dimension_id": 8, "id": 8, "auditor_4": false, "auditor_18": false}
  .
  .
  .
 ],
 "metaData": {
  "fields": [
   {"name": "id"},
   {"type": "string", "mapping": "dimension_name", "name": "auditor[dimension]"},
   {"type": "int", "mapping": "dimension_id", "name": "auditor[dimension_id]"},
   {"type": "bool", "name": "auditor_3", "header": "Dasha Alexo"},
   {"type": "bool", "name": "auditor_18", "header": "Tester 3"},
   {"type": "bool", "name": "auditor_4", "header": "Zobak Zobakovic"}],
  "root": "auditors",
  "totalProperty": "results",
  "id": "id"
 }
}

In this case it's the number of auditors (and their names) that's changing. My data types are booleans but displaying any other data type would look pretty much the same.

Next I needed to update the column model based on the new meta data. I did that by adding metachange listener to my store

auditors_store.addListener("metachange", function(store, meta){
 var grid = Ext.getCmp('auditors-grid');

 var columns = [
  {header: 'Dimension', dataIndex: 'auditor[dimension_id]', hidden: true},
  {header: 'Dimension', dataIndex: 'auditor[dimension]', width: 200}
 ];
 
 for (var i = 3; i < meta.fields.length; i++ ) {
  var plugin = new Ext.grid.CheckColumn({
    header: meta.fields[i].header,
    dataIndex: meta.fields[i].name,
    grid: grid,
    width: 120
   });
  
  columns.push(plugin);
  plugin.init(grid);

  //use columns.push( { header: meta.fields[i].header, dataIndex: meta.fields[i].name, type: meta.fields[i].type }); for fields that don't require plugins
 }
 grid.reconfigure(auditors_store, new Ext.grid.ColumnModel(columns));
});

I had to create and initialize the plugin for each new column as I needed editable checkboxes. For normal fields just use the usual column config object { header: meta.fields[i].header, dataIndex: meta.fields[i].name, type: meta.fields[i].type }.

The actual magic happens is in the last line. Grid has an obscure method reconfigure( Ext.data.Store store, Ext.grid.ColumnModel colModel ) that allows to switch the store and column model (the columns displayed) on the fly.

My next problem was to make the checkboxes editable. To do that I had to put a small fix to the CheckColumn plugin (see the plugin) as the original plugin assumes that the grid is not rendered when initializing, however, when dynamically adding columns the grid is already rendered so I had to change the following:

 init : function(grid){
  this.grid = grid;
   this.grid.on('render', function(){
    var view = this.grid.getView();
    view.mainBody.on('mousedown', this.onMouseDown, this);
   }, this);   
 },

to the following:

 init : function(grid){
  this.grid = grid;
  if (this.grid.rendered) {
   var view = this.grid.getView();
   view.mainBody.on('mousedown', this.onMouseDown, this);
  } else {
   this.grid.on('render', function(){
    var view = this.grid.getView();
    view.mainBody.on('mousedown', this.onMouseDown, this);
   }, this);   
  }
 },

And that's all - nice and simple. If only there were more books on Ext :-(