BIG disclaimer: I'm not formally trained in computer science (aside from two classes at MIT in 2000) and I haven't worked closely with that many programmers or teams (maybe 10 or so).
I've gotten to be a lot more productive as a programmer since I started casually 15+ years ago and seriously about 10 years ago. By productive, I mean shipping something that actually works in as short a time as possible.
Lately I've been thinking about the origin of all this productivity. I think it all comes down to doing a few things well. Say you want some code that does X.
- Tools. Often the quickest way to do X (especially if X is complicated) is to grab something off-the-shelf. My goto repository is CPAN, or even higher level some essentially (to me) black box Apache project. Once I reach the limitations of that tool, then I might jettison it, but when the goal is to ship quickly, nothing beats using existing code.
- Simple Algorithms. If I have to do it myself, I initially try to accomplish X in the simplest way possible by breaking it down into trivial steps. I know there is a famous programming quote that pertains to this process, but I can't find it right now--someone in the comments help me out please :)
This breakdown process has certainly become easier over time for me as more tools/data structures/design patterns/etc. pop into my head immediately just from having used them a lot. But the real key is to push off all complexity until later. Get something working, then add to it incrementally.
Pushing off complexity is not as easy as it sounds. A lot of things that seem essential are really not. What I do is make a critical path walk through of what is absolutely needed to accomplish X. Anything that improves X but is not on that critical path, I write down on a list to revisit later.
When you do the initial breakdown, it is quite possible that some of those (still non-trivial) steps can be accomplished by Tools, i.e. you can cobble together some stuff of your own with other people's stuff to get X working quickly. Just because one tool doesn't do X outright, doesn't mean you should do everything yourself.
- Debugging. I've seen people waste the most time here, literally aimlessly changing lines of code hoping it will solve their problems. When approaching debugging, I first find out where the problem is, i.e. the exact line of code or config file or whatever that is causing the problem. That should not take long, and before you ask anyone else about it you should know exactly where the problem lies.
If it is my code and I've been working on it recently, I usually have a pretty good mental model for how it all works and can get to the relevant line almost immediately. If it is not my code, e.g. an open source tool or sysadmin thing, I usually make educated guesses and then essentially do a binary search printing variables out as I go until I see where it messed up.
- References. If you are stuck in either Algorithms or Debugging, it is likely people have talked about your problem on the Internet already, e.g. on HN, StackOverflow, a mailing list, tutorial, FAQ or in a book. Knowing how to use references is key to doing things quickly. I can't tell you how many people I've tried to help who needed to RTFM.
Before you ask anyone, you should look online. If you're really stuck, ask someone on IRC or post your own question to an online forum. Then work on something else while you give people time to respond. When you're really stuck, there is no reason to spin your wheels endlessly. I've also found sleeping on it often magically produces a solution in the morning.
Those are the main things that I think contribute to my programming productivity. Other than those, there are some largely idiosyncratic habits I've formed that certainly help me but I'm not sure how useful they would be to you:
- I run code constantly, literally every change I make I re-run stuff to see if it improved/is working. This process makes it so I usually only have one bug at a time to deal with. It's part of how I work through that whole simple algorithm thing above.
- I put # For debugging lines in everywhere when debugging and then I leave them there so when I come back I can easily uncomment them and get some decent debugging output for that particular component.
- I try not to be clever and comment anything that looks complicated with a short explanation right by that piece. I used to try to be a lot more clever, but it never worked out well.
- I'm not an early adopter of new tools/languages unless they seem to fit X exactly. I'm still using Perl with no framework :). But I have started using nginx, memcached, etc.
Update: more good comments on HN here, including this one from xentronium: "There were awful lot of tasks I tried to achieve via pure coding and rewriting / rewiring blocks here and there. All they actually needed was some analysis on paper." I have had the same experience.
Update2: this post is re-printed in the Software Developer's Journal.