Your company has a knowledge debt problem

Wednesday

Jun162010

Wednesday, June 16, 2010

Organization,

Recruiting

When your company lacks experience in tools and techniques that can make it more productive, your company has knowledge debt.

Companies tend to operate in ways that exacerbate their knowledge debt problem. Consider this fairly typical job ad:

Initech is seeking an experienced Software Engineer to join
the engineering team.

Responsibilities

    * Design core, back-end software components
    * Analyze and improve efficiency, scalability, and stability
      of various system resources 

Requirements

    * M.S. Computer Science or related field preferred
    * 2+ years of Java experience
    * Expert in relational data modeling and query
      optimization using MySQL

I would posit a guess that this company uses Java for the majority of its work and uses MySQL on the back-end. Naturally, the company wants to recruit people who share that skill set and can "jump right in" and contribute.

This mindset is fundamentally flawed. A company should be hiring for problem solving skills, programming ability, and cultural fit, not for any specific skill set.

If anything, a company should prefer candidates who are experienced in a different set of technologies. Any worthwhile programmer will be able to pick up the technologies necessary to work with the existing code base. Hiring people with different skill sets gives the team instant experience in a new set of tools for solving problems.

Recognizing knowledge debt

Every time a new language, database, tool, or technique is invented, a company accumulates knowledge debt. Unless a company actively tries to combat knowledge debt, the company will continue to use the same old tools and techniques that feel "comfortable", fall behind, and eventually look something like the companies still using Enterprise Java today.

Similar to how technical debt has interest payments in the form of manual labor and maintenance costs, knowledge debt has interest payments in the form of being less productive than possible.

You have to recognize knowledge debt as a problem before you can combat it. Every company I've been involved with has suffered from it in some form or another, especially the larger organizations. People have difficulty stepping outside their comfort zone to solve problems in different ways with different tools.

Combat knowledge debt with experimentation

You have to make a constant investment in combating knowledge debt or you're guaranteed to fall behind. One way to combat knowledge debt is to sacrifice some productivity in the short-term by experimenting with different technologies in the hope of improving productivity for the future.

Experimentation can be difficult in the face of large legacy systems. Small, relatively independent projects are the best candidates for experimentation.

For example, I first used Clojure to implement a Thrift service to collect records to a distributed filesystem. The project was independent from the rest of our code base which made it a great candidate to try out Clojure. The project taught me a great deal about Clojure, and Clojure has been a major win for my company.

The tools you use for personal productivity are some of the best candidates for experimentation. Try a new code editor like Netbeans or Emacs. If you're used to Subversion for version control, try out Git for new projects or for open source work.

Ultimately your company needs to establish a culture of experimentation and encourage people to go outside their comfort zone.

Paying off knowledge debt through experimentation can increase your technical debt. By trying out different technologies, you're diversifying your infrastructure and making it more complicated. You may need to pay off this technical debt at some point by consolidating technologies.

Combat knowledge debt with recruiting

As I've alluded to already, another great way to combat knowledge debt is to hire people with different skill sets. This way you gain the knowledge of the ins and outs of different technologies without having to make the learning investment yourself.

Of course, just because you have individuals with useful knowledge doesn't mean your company will be able to make use of that knowledge. You need mechanisms to spread awareness of those people as resources for those tools and techniques. There's no single answer on how to spread awareness, but some ideas are having regular internal presentations to present tools and techniques to the rest of the team, or for a larger company, having an internal database mapping tools and technologies to people.

Conclusion

Besides the long term productivity gains, keeping knowledge debt low can have subtler benefits. Low knowledge debt can help with recruiting since great programmers want to work with cutting edge tools and technology. Combating knowledge debt also exposes you to different ways of looking at problems that can make you more effective with older technology. As an example of the latter point, Eric Raymond has a famous quote about Lisp that "[learning Lisp] will make you a better programmer for the rest of your days, even if you never actually use Lisp itself a lot."

So get out of your comfort zone and pay off your knowledge debt. You won't regret it.

You should follow me on Twitter here.

48 Comments |