For many years, there has been a trend in all industries to look to outsourcing as a method to “reduce costs” and “improve efficiencies.” In the software industry, this has given rise to large numbers of firms in countries where the cost of labor is low; India, Eastern block countries, Malaysia, etc. This is a popular topic in management and business books. But does it really work? Does it let a company produce better software for less? I think not. And I think it fails on several fronts.

Team building - Producing quality software requires a strong team. What makes a strong team? The question that is usually asked, is whether a team has “jelled.” To me this means are they all thinking on the same wavelength. Does the team communicate outside of the traditional methods of email, specifications, designs, memos, meetings. Do they get together for a beer or dinner after work to socialize, but end up hashing out that tricky algorithm? The thing you may see here is that all of this requires physical proximity. It is very hard to create this jelled team when you are in 4 time zones spread across the globe. As an example, I had a conference cal yesterday with all of the software managers in Swisslog; in order to make it work, I had to wake up at 5am and the guys in Australia were up at 10pm.

“But wait!” you say, “what about all of those open source projects that have people all over the globe? They can do it! Why can’t you?” Simply put, they are different. If you look at most open source projects, you will find while a large number of people may have contributed, there are usually a very small group of core contributers who do most of the work. They receive patches and suggestions from the group but do the critical work themselves. The other huge difference is that open source projects don’t have to be cost effective, everyone is working for free.

Costs - Cost is the bean counter argument for outsourcing. Why pay $120/hour for a programmer in the US when you can get 4 programmers in India for $30/hour? On the surface it seems like you an get far more work done for less money. But in reality can you? I don’t think so. From my experiences, you end up spending far more money with offshoring. The cost of the off-shored programmers is cheaper, but you end up spending more money on on-shore activities. In order to have a successful offshore, you need rock solid requirements, specifications, and designs. You also need far stronger project management. This all takes time.

Communication - So now you have a bunch of code from your offshore team, now what? You need to test it, which will inevitably find bugs. If you are in the US and working with a team in India, you generally communicate via email. I have heard the argument that since they are nearly 12 hours off that they can be productive while you sleep. That sounds good, but in reality what it means is that every email “transaction” now takes 24 hours to complete.

Obviously I am not a fan of this method of development. Can it work? I’m sure it is possible for large projects where the entire project takes place in another country, but then you are not really offshoring, you are just onshoring it in another country. I have seen multiple companies I have worked at try to make this work, and I don’t believe it can in the small/medium sized tech companies.

What do you think, am I right?

If you want to progress and grow in your career, you have to confront new challenges. When I started as a developer, the challenges were straightforward; write code at the direction of my technical lead. The scope of my decision making power was small. I could choose how to implement an algorithm, but the larger decisions were out of my hands.

As I grew, I wanted to make more of a difference in the bigger picture, but still at the technical level. I moved up to the role of technical lead. The biggest challenge was no longer the technical, it started to be people. What did the customer want? Not what did they say, but what did they need. Learning that people are not computers was hard. I couldn’t assume that my audience would understand what I understood. I had to learn people. And people are messy, illogical, emotional, greedy, and unpredictable, the antithesis of computers. It took time, but I got good at it.

This brings me to today. As a manager of technical staff, I mostly deal with people. While understanding the technology is important, understanding the problem is more important, and most important is knowing who to apply to the problems to achieve the best results. This has its easy side when you are acting in the positive direction; applying the right person to the right task. The much harder part is when you make a mistake and have the wrong person in a role. You have to be willing to make a switch. I may know it is the right thing to do, but it is hard. Seeing what is best overall may not be percieved to be best by all the individuals involved.

These challenges have forced me to grow in my job and as a person. I do hope I am making the right choices. I’ve had great teachers so I feel confident. I think if I ever stop worrying it is time to quit.

So much for my musings at 38,000 feet.

I have a favorite question to ask people on interviews for software development: “Do you see programming as an art or a science?” There is no “right” answer to the question, I want to see how they answer it. For me the answer is both. While technical competency is critical, the mark of a truly talented software developer, or more accurately, designer is a creative spirit. While a nice lead in, my interview question only partly relates to what I thought I’d write about. The last few days of crappy weather have put me in a thoughtful mood, one where I found myself settling on the topic of creativity and what makes people (or more importantly, me) happy.

Looking at software again, the aspect of development that kept me up at night, brain turning, writing notes, forcing me to go into the office early was the challenge of solving the problem. Let someone else do the execution, that is the gear turning part, the true joy comes in figuring out how to make the machine. And I’m really good at designing the software machine. The challenge and sometimes frustration comes when I have the passion but not the skill (yet) to execute what I can see in my head.

My photography is definitely at that stage. I love photography but haven’t reached the point where I can always capture what I see in my mind’s eye. I am very proud of what I have produced, have filled my house with great art that I created, and enjoy looking and having it looked at. However, it doesn’t feel like I have “got it” yet. There is something that seems just out of reach, as if it were right there that if I could just get my hands on it, everything would click. When Tiffany and I saw an exhibit of Ansel Adams it is blindingly obvious that he had that thing. Whatever that thing is.

It makes me think of when I was a kid. My father was a very talented artist; painting, drawing, doodles all over the place. It was as if his mind could not function if he didn’t keep that part of his brain busy. It is an an aspect I did not think I had inherited but maybe I have, just in different mediums.

It is cool that the desire to create seems independent of the tools. For me it can be capturing a beautiful image, creating a wonderful meal, writing an elegant email on a meaningless topic, developing a cool solution to a computer problem, building a simple, clean website, or even just a nice turn of phrase. The urge to create is a great thing and it is amazing how satisfying it is.

The internet is big. And it has lots of cool stuff. Too much at times.

For the longest time, when I found a site I found interesting, that I might use again, that I wanted to read on a regular basis, I bookmarked it. Plain and simple. Functionality that has been around since NCSA Mosaic (yes, I used that on the old AIX birds systems at W&M). Bookmarks scale moderately, but not past a certain point. I certainly crossed that line when I had over 1000 of them.

The first change I made several years ago centered around sites that I wanted to read regularly. With the advent of RSS feeds and the centralized service provided by Newsgator, I moved all of the blog and news sites I read out of my browser and into my RSS readers (NetNewsWire for Mac and iPhone and FeedDemon for Windows). This system works wonderfully. I get notified when I have new articles to read, they are shared across all my computing devices, and the software is well written and intuitive.

But it wasn’t enough.

I still had over a 1000 bookmarks. In looking at them, I saw that they fell into two categories:

• Websites I need to access on a regular basis (i.e. banks, reference sites, etc)
• Websites I don’t want to forget or ones I think may be useful “later”

The first group of sites are perfect for bookmarks. Their number scales linearly with my life and seems to be relatively bound. There are only so many websites I will visit on a regular basis, just as there are a limited number of stores I will visit regularly in real life.

The second group is more of an extended history function. Like that restaurant that I went to 6 years ago in New Orleans that I want to remember if anyone says they are visiting the city or I go there again at some point. Hardly a regular piece of information, but I don’t want to forget. Of the 1000 plus websites I had bookmarked, probably 95% of them fell into this category. They completely obliterated the 5% I needed to use regularly no matter how well I organized the structure.

I needed a new solution.

I have settled on using Delicious to keep track of those “might be useful later” websites. It will allow me to be a digital packrat when it comes to websites, but keep the clutter out of my day to day operations. I hope the tagging and searching features will allow me to retrieve that stored knowledge in the future; only time will tell. I have imported all of those 1000 bookmarks into the site and will in the next day or two purge them from my browser. I can’t wait.

If you are curious, nosy, or just bored, you can browse through all of them on my Delicious page here. Enjoy.

So, it is kinda a big day. It seems right and proper that our first black president is being inaugurated the day after Martin Luther King Jr. day. As I was driving to the airport in Richmond this morning the VDOT highway signs south of Richmond were advising people driving to the inauguration to call 511 for traffic information. Note this is over 100 miles to DC. So yeh, I think it will be a big day.

I have high hopes for the Obama administration. Moreso than the obvious milestone for race relations in the US, I see his election to be a generational shift. Having a young president with a better understanding of where the country might go in the future. I think that is more important to me I hope he does well and can live up to some of the hype surrounding him. However the blind faith I heard on the radio on my drive gives me pause; people declaring categorically that he will be the best president ever. I certainly hope so, but none of us know. He may be a Kennedy or an FDR, but sadly he could be a Harding. Only time will tell.

I know however that I look forward to finding out.

I mentioned in my last post that I spend time thinking how I would run a software group if I could start from scratch. I won’t touch on how one retrofits this into an existing organization with institutional inertia (see previous entry on monkies and firehoses). I wanted today to talk about the tools I think are critical to a team.

[Note: I wrote this orginally at 30,000ft on my way to Jaxonville, FL so there are no links to the products below. I will try to come back and add them later. Updated 12/12/2008 with links]

Source Control

This is non-negotiable. Whether you use a commercial tool like Perforce or open source tools like Subversion, you must have this. It needs to be the right hand of all developers, ingrained into their daily habits. All code must be checked in at every opportunity. If working in a shared codebase, one must be careful to not “break the build” but developers should plan their work so that it can be done at least once a day if not more often. This takes discipline but if the habbits are developed from the start, it will pay off in the end.

Online bug/issue tracking

Tools like Bugzilla allow a team to keep track of problems and tasks pending while giving project managers and senior management a view into the project that can be tracked over time. But in order to be used, the system needs to be simple and straightward to use. The instant it becomes burdensome or even annoying to use, it will become neglected and not used. This applies to data entry, searching, and reporting.

Wiki for documentation

This probably the most controversial. It is my believe that using tools like Microsoft Word are horrible tools for collaborative document writing; exactly the thing you need for the creation of the most important documents in a software project; requirements documents, functional specifications, technical design documents, test plans, and more. Word docs get emailed around, changed by different people at different times with little tracking in place. More time is spent in document management than in actual content creation.

How does a Wiki solve this? You get a central document that anyone can access and edit without emailing them around and having to reconcile changes. It provides change control and tracking so you can see who makes what changes. It also has the built in capability to link between documents to show the relationship between a design section, it’s corresponding functional description, and the requirement it is supposed to fulfil. Many wikis also have the facility to host discussion forums tied to a document but seperate from the content itself. There are obvious challenges of freezing the document at key points in time I believe this is small in comparison to the benefits.

I have never been able to implement this particular idea but I am convinced it will make everyone more efficient.

(And, yes, a wiki can be used for so much more and it should, but that rarely takes as much convincing.)

While I don’t have a huge readership here, I know most of them are software people, so what do you think? Anyone have a great idea and the money to let me give it a go?

2 Posts in 2 days, wow. But it is largely just a big cut and paste, so I guess I don’t get too much credit.

There’s an old joke, so old that I don’t even know for certain where it originated, that’s often used to explain why big corporations do things the way they do. It involves some monkeys, a cage, a banana and a fire hose. You build a nice big room-sized cage, and in one end of it you put five monkeys. In the other end you put the banana. Then you stand by with the fire hose. Sooner or later one of the monkeys is going to go after the banana, and when it does you turn on the fire hose and spray the other monkeys with it. Replace the banana if needed, then repeat the process. Monkeys are pretty smart, so they’ll figure this out pretty quickly: “If anybody goes for the banana, the rest of us get the hose.” Soon they’ll attack any member of their group who tries to go to the banana. Once this happens, you take one monkey out of the cage and bring in a new one. The new monkey will come in, try to make friends, then probably go for the banana. And the other monkeys, knowing what this means, will attack him to stop you from using the hose on them. Eventually the new monkey will get the message, and will even start joining in on the attack if somebody else goes for the banana. Once this happens, take another of the original monkeys out of the cage and bring in another new monkey. After repeating this a few times, there will come a moment when none of the monkeys in the cage have ever been sprayed by the fire hose; in fact, they’ll never even have seen the hose. But they’ll attack any monkey who goes to get the banana. If the monkeys could speak English, and if you could ask them why they attack anyone who goes for the banana, their answer would almost certainly be: “Well, I don’t really know, but that’s how we’ve always done things around here.” This is a startlingly good analogy for the way lots of corporations do things; once a particular process is entrenched (and especially after a couple rounds of employee turnover), there’s nobody left who remembers why the company does things this way. There’s nobody who stops to think about whether this is still a good way to do things, or whether it was even a good idea way back at the beginning. The process continues through nothing more than inertia, and anyone who suggests a change is likely to end up viciously attacked by monkeys. But this is also a really good analogy for the way a lot of software works: a function or a class or a library was written, once upon a time, and maybe at the time it was a good idea. Maybe now it’s not such a good idea, and actually causes more problems than it solves, but hey, that’s the way we’ve always done things around here, and who are you to suggest a change? Should I go get the fire hose?

This is taken from an article Let’s talk about Python 3.0 by James Bennett. I hope he won’t mind I have reposted it.

Why did I post this as a whole? Because it is so true. It is a human behavior I have known and not understood throughout my life in software. For a long time, I was just frustrated without knowing why. The realization that I was different, that I did not have as strong a shackle on the past was a blessing when it came. Not that I was any less frustrated when I saw the practice in life, but I knew why. It allowed me to focus my efforts and help people understand that just because we “always did it that way” did not make it a good idea.

Being the person to play that role is hard. People do not like hearing it. And sadly, getting attacked by the monkeys is not fun at all. It wears on you when you do not see a way to convince people. You bring them numbers, show them the tole their policies are taking in terms of dollars, in terms of people, in terms of families, and they can’t understand that there could be a better way if they only let go of the past.

But it is in me to be that person, and I love doing it. I have plans and I’m stubborn and I can’t wait for the opportunities to do the right thing and see the results.

Most of you know that I read a lot. I have a hard time explaining sometimes why I read so much, nor why I get a certain pleasure in just amassing books. I have approximately 780 books in my house; I say approximately not because I guessed, but because I may have missed a few while counting. I won’t say I have read them all, but I have probably read at least 90% of them. Suffice to say, I like reading.

When I have headache, I read and the headache goes away. When I’m pissed at the world, I find a book and a dark cave and chill in a pleasant elsewhere. Forget about the knowledge and ideas passed along via the written word, reading a book brings a calm to my crazy NADD-driven world.

That is from a blog I read, Rands in Repose from a guy who is a software manager. But that is irrelevant to this discussion. He sums up how I feel about books quite well.

As vices or habits or neurosis go, collecting books is one I can be proud of. I hope I never stop. What am I reading now? The Player of Games by Ian M. Banks. Before that? The Last Theorem by Aurthur C. Clarke and Frederick Pohl. What’s next? I can’t wait to find out.

A month or so back I proposed that I would post entries on the topics of this election that concern me, comparing the main party candidates. As should be obvious, I have not done so. The reasons for this are immaterial but should not imply a lack of interest in these topics. A simple example as to why is that I am on a flight to Iowa right now that I booked less than 4 hours ago.

The election is now less than two weeks away. The competition has brought about many suprises; the first African American is running for president, Virginia seems likely to vote Democratic, it seems likely both of our Senate seats will be held by Democrats. The presidential contest seems to represent a generational shift in politics. On my way to the airport I heard a statistic saying that the 18-24 demographic favors Senator Obama by a 56% to 32% margin. Quite an indicator as to the direction of the country. Something that makes me happy.

As my readers should know, this matches my feelings. While I respect Senator McCain (I supported him in the last Republican primary against Bush), I find that to become “electable” in the eyes of the Republican base, he has compromised the things he stood for. He has fallen to playing the game of “politics of fear” as it has been called. The tone of his campaign has turned so ugly it is just disrespectful; so much so that at times he even appears embarrassed by it.

I believe that it is time for the next generation of politicians to take the lead. That is not to say that they are perfect, they are human like the rest of us. I have hope that Senator Obama will be a good president. He needs to be as we have many challenges ahead of us.

A nice view of Chicago. Hopefully it came out.