The Sad State of SQL Refactoring

I love refactoring tools.

The ability to select a variable in code, right-click on it, choose the Rename command from a context menu, and then safely rename that variable in every location in which it appears is a boon to my programming productivity. Similarly, the ability to take a large block of complex logic and extract it to its own method is really handy. Or, the ability to reorder parameters, and know that every piece of code that calls that method will be updated appropriately saves me tons of time and improves my confidence in the quality of the code.

When you refactor code of any kind, you enhance its readability without changing the way it works. But refactoring by hand is frequently difficult, tedious, and error-prone. That's why refactoring tools exist and why the essential service they provide is important. They allow us to improve the maintainability of code, hopefully reducing maintenance costs, quickly and efficiently.

Enter SQL languages.

In theory, there is a standard for SQL languages: the ANSI SQL-92 standard. One would like to think that it would be a simple matter to create refactoring tools for any SQL based on the ANSI standard for SQL. One would be wrong. You can't use the standard as the sole basis of a refactoring tool.

Any given database vender wants to strive to make their product unique, to stand out from the crowd. And so, they don't entirely conform to the standard. They have additional features that separate them from each other. For example: Oracle organizes functions and procedures into packages. Microsoft does not. Microsoft allows a bit data type on columns. Oracle does not. And, although we all loathe to think of Access as a real database, Access provides a boolean data type that you can use in columns, while Oracle does not.

Now, let's also talk about legacy support. Oracle has been around for a very long time. Version 2 came out in 1979, from what I gather. Their legacy join syntax looks nothing like the ANSI standard (and that's not a criticism, just a simple statement of fact):

SELECT Customers.CustomerId, Company, OrderID
FROM Customers C, Orders O
WHERE C.CustomerID (+) = O.CustomerID
UNION
SELECT Customers.CustomerId, Company, OrderID
FROM Customers C, Orders O
WHERE C.CustomerID = O.CustomerID (+)

In ANSI SQL, this is :

SELECT Customers.CustomerId, Company, OrderId
FROM Customers C FULL OUTER JOIN Orders O ON
(C.CustomerId = O.CustomerId)

Now, we could add in all the different SQL variations that we know that are out there:

• ANSI standard
• Interbase/Firebird
• IBM  SQL
• Microsoft Transact SQL
• MySQL
• Oracle PL/SQL
• PostgreSQL
• Access
• FoxPro

...and on and on and on, but you get the point. There are a lot of SQL variants out there. And their goal is to accomplish the same thing:

• Create a data store.
• Occasionally, modify the structure of the data store.
• Get data out of a data store.
• Put data into a data store.
• If supported, execute code within the confines of the data store and (optionally) return a result.

Behold, SQL in a nutshell.

Over the lifetime of these various products, they have added features that they have to maintain for legacy support. Somewhere out there there's a business that absolutely must have that feature in place or their whole process will come crashing down. Don't you dare remove it. It doesn't matter that there are better features (likely based on a standard); there are applications out there for which they don't have the source code anymore, or which no one understands, and they're too afraid to touch.

Now, all these reasons have led us to a scenario where we have vastly different implementations of SQL. Sure, they share a lot in common, but they also have radically different feature sets and syntax. And that situation, in and of itself, has led us to one frightening and sad conclusion:

We will likely never have a tool that is able to connect to any database and be able to correctly refactor its SQL.

And that's just a damned shame. Because there's a lot of SQL out there. Stored procedures, functions, views, triggers, even inline SQL in applications and all that other jazz. But the amount of work it would take to get us to a point where a refactoring tool could recognize any variant of SQL and correctly parse it, refactor it, and not hose the code is enormous.

It's a pity, really. I could see tons of use for a tool like this. In my own office, we work with SQL Server, Oracle, and Atomix databases. What I wouldn't give for a tool that could refactor SQL to enhance its readability without changing the way it worked.

And who knows? Down the road, we may be working with something else.

But this is the world we live in. If we want to refactor SQL, it looks like we'll have to settle for separate refactoring tools for each language. And then each will come with its own quirks. That might be good or bad, but it's likely the best we can hope for for now.

 

Technorati Tags: SQL, Best Practices, Productivity, Code Quality, Refactoring, Software Development

Defects and the Scientific Method

Let's leap back a few years to that frightening time in Junior High School. Remember those years? September, possibly October. It was still hot, and the air conditioning didn't work. Your science teacher was standing in the back of the room, with an overhead projector and had a slide up for you to copy down. On it, he had this information:

THE SCIENTIFIC METHOD

Ask and define the question.

Gather information and resources through observation.

Form a hypothesis.

Perform one or more experiments and collect and sort data.

Analyze the data .

Interpret the data and make conclusions that point to a hypothesis.

Formulate a "final" or "finished" hypothesis.

Ah, remember those days? Remember how boring it was? Remember thinking to yourself, "I'll never use this?"

Well, as a software developer who is tasked with maintaining software that is virtually guaranteed to contain defects, you can be certain that you need to be intimately familiar with The Scientific Method.

The Scientific Method provides a clear roadmap for defect isolation. In fact, anyone who has any real experience isolating defects without disturbing the rest of the system has (whether he's aware of it or not) used the Scientific Method to do so. Here's how it breaks down:

1. Ask and define the question. The software should behave in this manner, but it does not. What is the cause of this problem, and how do we fix it?
2. Gather information and resources through observation. In a controlled environment that mimics production as closely as possible, reproduce the defect. If possible, step through the code and observe its behavior.
3. Form a hypothesis. The defect is caused by this behavior in the system (or by the behavior of this external system).
4. Perform one or more experiments and collect and sort data. Implement a code fix; attempt to reproduce the defect using the fixed code. Observe the results.
5. Analyze the data. Did the code fix have the desired effect? If so, how?
6. Interpret the data and make conclusions that point to a hypothesis. Was the code that was modified the cause of the defect, or was it merely a symptom of an underlying problem requiring further resolution?
7. Formulate a "final" or "finished" hypothesis. If the defect is fully repaired, check all code into the repository. Otherwise, continue the analysis until you have rooted out the underlying cause of the defect.

Simply put, there's no guesswork in defect resolution. It is a rational, thinking process, much like a game of Sodoku. If you approach any defect and just yank an answer out of thin air, You're Doing It Wrong.

Instant answers to defects are a dangerous game. Your first, instinctive answer to any problem is likely to be wrong; the chances of this being true will only rise as your code base grows in size. As your product gains features, you'll want to take greater care to make sure that you have taken the time to disturb absolutely nothing outside of the defect you're trying to correct. In that case, take some advice: Keep your grubby fingers to yourself. Touch only the code in the defect domain. The best way to do that is to have a plan for defect resolution, and I strongly encourage you to apply The Scientific Method.

Developing software is a task for those who can think. It is not a task for the simple-minded, the lazy, or the inattentive. You have to be willing to pay attention to the details, and to invest the time it takes to hunt down a defect in painstaking detail to get to the root of a problem.

A good software developer knows the difference between a symptom and a disease, and how that correlates to software defects. Sure, you have a NullReferenceException, and your code is missing a handler for that. But is the problem the missing exception handler, or is the problem that a null somehow got into a table that should never have had it, or that a stored procedure in the database returned nulls when they were never expected? Which one is the symptom? Which is the disease? Make sure you're fixing the right defect. Don't just prescribe aspirin when the software needs invasive surgery. To find that out, you need to think critically. You need to apply the Scientific Method.

The Absurdity of "Don't Reinvent the Wheel"

As developers, we've had this adage drilled into us from the beginning: Don't reinvent the wheel. In short, don't rewrite what's already been written. The idea is sound, in theory. You can save yourself time and money if you'll simply reuse existing code and/or components rather than writing them yourself from scratch. This time and money is saved up front when you write it (or would have written it), and down the road, when you have to maintain your system.

However, I'd like to point out another, equally applicable adage: There's nothing new under the sun. Anyone who's ever tried to write a novel, a short story, a play, a movie, a song, or a piece of software, will know this one simple truth: somewhere, at some point in time, it's already been written.

Every algorithm, every piece of code that we will ever attempt to write has already been written somewhere, at some point in time, by someone. Only the names have been changed. You're not inventing anything that is completely new, that's never been seen before. You should wisely disabuse yourself of that notion as quickly as possible.

In the grand scheme of things, at the application level, you may very well have an idea for a system that is unlike anything that has been done to date. But the algorithms that drive it have already been written. Bubble sorts, hashes, exception handlers, encryption, data access, socket management, shopping carts, entire application frameworks, date management, document management, serialization, port I/O, and all that other stuff has already been done. Further, it's already been done several times over in many different languages to varying degrees of success.

Tragically, if you're using a large application framework, like Java's EE or Microsoft's .NET, the chances are good that the functionality you're looking for is built right into the framework itself. The problem is that the framework is so vast that you'll spend more time looking for it, and determining whether or not it works the way you need it to work than you would just rewriting it yourself.

Application frameworks are stunningly afflicted with feature creep. They must do everything under the sun, must meet every possible need. The problem, then, is that their scope becomes so broad, so vast, that no one in their right mind could possibly grasp the totality of all that they can do. It is inevitable that anyone using them will reinvent some of their functionality. The scale of that functionality might be small (reformatting dates) or it might be substantial (pooled database connections).

In the end, it's absurd to think that we can possibly avoid reinventing the wheel. Of course we're going to reinvent it. Every application we write is a reinvention of someone else's wheel. It just so happens that our wheel is a custom wheel. All this paranoia about reinventing the wheel is blown out of proportion. A proper buy vs. build decision should never be neglected; but don't ever think for one minute that what you're creating hasn't been created before.

Consider the scenario where you're under the gun to get a product out the door. And I mean it's a really tight schedule. And don't act like it's a perfect world, and you have leverage over the schedule. This is reality here. In the real world, the customer controls the schedule, because it's tied to when the product is released, and that's tied to this big, huge monstrosity in another state or another country. The product's delivery schedule is a train barreling down the track at 120mph and no one short of God can stop it. Now, you have a very finite amount of time to work in. You need an algorithm. You know you could write it. Or you could look to see if someone else has written it.

If you do the whole Web search thing, you have to ask yourself a few questions: Is it from a source you trust? Is it in the language you're using, or do you have to convert it? Does it work? Does it need to be tweaked? If any of these fail, you're back to the drawing board. Time's wasting here, and that train's getting closer to its destination. If all the answers pass, you have to make sure you don't run into any copyright or licensing issues with that code. (You are paying attention to that, aren't you?)

If you decide to peruse your application's framework, you'd better hope it's well documented, and very easy to search. Good luck using the search features in .NET. It's not like the ASK.COM interface, where you can ask, "How do I convert a date in DOD format to Gregorian format?" Yeah. Good luck with that. On the other hand, you could ask your coworkers. They might know. Then again, they might not. If they don't, you're off to Google to get the information. Here's hoping you get a timely and accurate response.

Sure, this is an extreme example. But it makes my point: At some point, the work has to get done. You can't afford to spend days or weeks scratching your head about whether or not that wheel's already been invented. Believe me, it has. The problem is, there are a countless number of wheels, and none of them are labeled, and you don't know where to find the wheel you're looking for.

Stop wasting time, and invent your own damned wheel.

After all, whatever code you might reuse, is just someone else's reinvention of the same wheel.

SOX Compliance and the Waterfall Method

So here I am, working away at my job, coding in a vacuum as always, a development team one. Some things never change. But other things, inevitably, do.

Our company was recently purchased. The new company has grand plans to eventually go public. With its eyes set on that prize, they have hired a consulting firm to help them achieve SOX compliance. This firm (who shall remain nameless) is busily churning out reams of process drafts to help us in that endeavor and submitting them to us for approval. It was only a matter of time before the SDLC for software development arrived on my desk for review.

Now, for those not familiar with how things run at our company, I'll simply refer you to this post, which rather succinctly sums it up. While some things have changed, most things, by and large, remain status quo. I have managed to convince them of the value of hiring temp testers prior to releasing builds, so we've shown moderate improvement there. But I'm still wearing tons of hats, and completely driving the entire development process single-handedly. And the company adamantly refuses to hire any other developers to help out. It's also worth noting that since the acquisition, the number of new software projects piling up on my to-do list is rapidly approaching the double-digits. So any process that these guys throw at me is going to affect me and it's going to affect me pretty damned profoundly.

You can imagine my utter shock and amazement when the plan that was presented to me for review and acceptance clearly stated that we were to implement, in excruciating detail, the Waterfall Method.

This presented several problems to me right off the bat:

1. Whoever presented this plan is clearly unaware of the fact that the waterfall method clearly doesn't work. The very man who initially described it (Winston Royce), pointed out quite clearly that it doesn't work, and suggested an iterative model as a clearly superior alternative. See the actual article for proof.
2. There is no way that we'd be able to implement that process with a development staff of one person. The process outlined requires that the roles are separately defined and filled by distinct individuals. We don't have individuals to fill those separate roles, and the company refuses to hire them.
3. Even if we did implement the process, the timeline to implement software solutions for our customers would become so bloated that the customers would drop us like a rock. Our biggest customer demands a release every three months. If we adopted the Waterfall Model as it's spelled out in the SOX compliant process they submitted, it would take three months just to spec out the iteration. Not that the model would permit the iteration.

Consider this quote, dated in 2004, for crying out loud:

Asked for the chief reasons project success rates have improved, Standish Chairman Jim Johnson says, “The primary reason is the projects have gotten a lot smaller. Doing projects with iterative processing as opposed to the waterfall method, which called for all project requirements to be defined up front, is a major step forward.”

In his blog entry, Waterfall Method: A Colossal Blunder, Jeff Sutherland points out the following interesting tidbits in his comments:

The Waterfall process is a "colossal" blunder because it has cost 100s of billions of dollars of failed projects in the U.S. alone. Capers Jones noted 63% failure rates in projects over 1M lines of code in 1993. By the late 1990's, military analysts were documenting a 75% failure rate on billions of dollars worth of projects. In the U.K. the failure rate was 87%.

...

Let me reiterate, for projects over $3M-$5M, the Waterfall has an 85% failure rate. For those projects that are successful, an average of 65% of the software is never used. The Waterfall is a collosal blunder. The most successful Waterfall company I have worked with had a 100% Waterfall project success rate with on time, on features, and on budget. This led to a 100% failure rate in customer acceptance because the customer's business had changed or because the customer did not understand the requirements.

In his article, Improve Your Odds of Project Success hosted to SAP NetWeaver Magazine, David Bromlow provides the following chart that shows how the Waterfall Method makes it difficult to start effectively managing project risk until much later in the project compared to more agile methodologies:

In their article, From Waterfall to Evolutionary Development (EVO), Trand Johnson and Tom Gilb had this to say:

After a few years with the Waterfall model, we experienced aspects of the model that we didn’t like:

• Risk mitigation was postponed until late stages;
• Document-based verification was postponed until late stages;
• Attempts to stipulate unstable requirements too early: change of requirements is perceived as a bad thing in waterfall;
• Operational problems discovered too late in the process (acceptance testing);
• Lengthy modification cycles, and much rework;
• Most importantly, the requirements were nearly entirely focused on functionality, not on quality attributes.

Others have reported similar experiences:

• In a study of failure factors in 1027 IT projects in the UK, scope management related to Waterfall practices was cited to be the largest problems in 82% of the projects. Only approximately 13% of the projects surveyed didn’t fail (Taylor 2000);
• A large project study, Chaos 2000 by The Standish Group showed that 45% of requirements in early specifications were never used (Johnson 2002).

Finally, I'll offer this, from the article Proof Positive by Scott Ambler in Dr. Dobb's Journal:

Agility’s been around long enough now that a significant amount of proof is emerging. Craig Larman, in his new book Agile and Iterative Development: A Manager’s Guide (Addison-Wesley, 2003), summarizes a vast array of writings pertaining to both iterative and incremental (I&I) development, two of agility’s most crucial tenets, noting the positive I&I experiences of software thought leaders (including Harlan Mills, Barry Boehm, Tom Gilb, Tom DeMarco, Ed Yourdon, Fred Brooks and James Martin). More importantly, he discusses extensive studies that examine the success factors of software development. For example, he quotes a 2003 study conducted by Allen MacCormack and colleagues, to be published in IEEE Software, which looked at a collection of project teams of a median size of nine developers and 14 months’ duration. Seventy-five percent of the project teams took an iterative and incremental approach, and 25 percent used the waterfall method. The study found that releasing an iteration’s result earlier in the lifecycle seems to contribute to a lower defect rate and higher productivity, and also revealed a weak relationship between the completeness of a detailed design specification and a lower defect rate. Larman also cites a 2003 Australian study of agile methods, in which 88 percent of organizations found improved productivity, 84 percent experienced improved quality, 46 percent had no change to the cost of development, and 49 percent lowered costs. He also cites evidence that serial approaches to development, larger projects and longer release cycles lead to a greater incidence of project failure. A 2001 British study of 1,027 projects, for example, revealed that scope management related to waterfall practices, including detailed design up-front, was the single largest factor contributing to failure, cited by 82 percent of project teams.

So, with all this overwhelming information at our disposal (which is just the little bit I could scrape up with Google in about an hour), and years of historical evidence that proves empirically that Waterfall doesn't work, why on earth would you wield impose it as the one and only process to be used for all projects, regardless of size or complexity across your entire organization?

It's like voluntarily picking up a cursed +4 Vorpal Sword of Mighty Cleaving: it chops your own head off the moment you touch it.

It's sheer, absolute lunacy. Particularly in our case, where we lack the time, the resources, or the desire to acquire the resources to properly implement it as it's written. We'll be bogged down in a bureaucratic quagmire of Dagoban proportions.

You'll have to forgive me if this seems like a rant. But that's exactly what it is.

It might be time to brush off that resume. Sometimes, enough lunacy just piles up that you start to realize that there's no one behind the wheel who has any firing synapses in the brain.

Technorati Tags: SOX, Waterfall, Waterfall Method, Best Practices, Process, Software Development

Who's Testing Your Software?

There's a common mistake in software development: trusting the developers to test the software. Historically speaking, developers are the worst kind of testers, because we tend to use the software only as we designed it to be used. It takes a special kind of developer to be able to think outside the box and think like a user with little or no computer savvy.

In the comment thread to the article, Microsoft Admits Vista Update Glitch, one poster made this point:

Beta testing is not getting the bugs out of software because they got the wrong people doing it. Don't use computer savy [sic] people to beta test, use people like my wife who don't have a clue what makes the computer work. She can discover any glitch in software code, guaranteed. Her gift also applies to use of TV remote controls, etc.

To which came this reply (edited for brevity):

This is the best answer I have read for several years. Beta testers are people who do not do things that cause problems, rather, they look for features and bugs that are sometimes not there...The best Beta testers are people who are not knowledgeable and those who don't know the difference of double or single click.

These folks are referring specifically to Microsoft's beta tests for its operating systems (more specifically, for Windows Vista). But the general sentiment is true and universal: users who have never been exposed to your software in the first place, and have had little exposure to technology are frequently the best ones to determine whether or not it actually works. They have a disturbingly accurate ability to ferret out bugs that borders on the psychic.

As developers, we like to believe that our software is rock solid, easy to use, painfully obvious, and bulletproof. A user who can't tell the difference between clicking and double clicking, or why it's a bad idea to keep lots of applications open at once on a machine with limited resources, is the prime candidate for testing your software. If it's a Web application, find someone who's rarely used the Web or who only uses it for the basics: IM and email. One thing that they'll be able to tell you right away is whether or not the user interface is actually usable. And if you think for one minute that you shouldn't be designing clean, minimalist interfaces for the lowest common denominator of user, you've probably never met the average computer user. There are far more of them than there are of us.

We have some pretty interesting users for our Web applications. Some of them are fond of ignoring on-screen instructions. Tooltips, online help, field prompts, clearly written button text, user training...not much of that seems to make a difference. When all of that fails, what does the application do? How robust is it? How gracefully does it handle bad user behavior? For that matter, how gracefully does it recover from bad application, network, or hardware behavior? And does it alert the user to that kind of thing in a clear, friendly, and meaningful way?

You can't determine that sort of behavior by trusting your developers or your unit tests to find them. Inexperienced users will find far more than your tech savvy users will. That's not to say that your testing team shouldn't include tech savvy users; it absolutely should. But make sure that you include novice computer and Web users in your testing team.

Refactor Yourself

Take a moment to stock of where you are now. What skills do you have? How can you improve them? Every day of your career, you should be learning something, improving something, refining something. Your skill set should be undergoing constant refactoring. This can only make you more efficient. If the stuff you're learning isn't making you more efficient, discard it.

At some point, you have to have the guts to go against the grain. Just because a "best practice" works for someone else at some other company doesn't necessarily make it a "best practice" for you and your company. A "proven methodology" isn't necessarily going to be a "proven methodology" for you. Have the guts to challenge the status quo. If it's not making you more efficient, it's likely hindering you. Refactor it out.

If your team doesn't have the funds to learn some new technique, seek that knowledge personally. There is no reason that your company's inability to fund team education should hold you back. Buy books and read. Search the Internet. Read blogs and programming newsgroups. Experiment with code. Ask your peers. Never stop seeking knowledge. Never stop learning.

Take some of your old code, copy it, and then refactor the hell out of it. You'll be surprised what you can learn by simply refactoring code: more efficient ways to implement things that you did before (and will likely do again), better algorithms that work faster, use less resources, and are easier to maintain. Refactoring improves your skill set. Refactoring your own code, on your own time, is a personal competition against yourself to improve your own skill set.

You don't need to compete against anyone else. Coding cowboys, platform fanboys, methodology purists, conspiracy theorists...you shouldn't be worrying about them. You should worry about yourself. Make yourself as good as you can possibly be. Every day, ask yourself this essential question: "How can I improve myself today?" Find that way, and then do it. Set aside a little time every day to refactor your skill set.

Each day is an opportunity to make yourself a little bit better, a little more efficient than you were the day before. With each passing day, you have the opportunity to become smarter, faster, wiser, more valuable. But that means taking care to constantly revise your skill set. Have the wherewithal to discard habits and ideas that simply don't work. If you suspect you're doing something one way simply because you've always done it that way, or because that's the way everyone else does it, question it. If you can't see a tangible benefit to it, refactor it out.

Look, I'm not Gandhi or anything. But I can tell you this: I firmly believe that the key to success in this field is a personal commitment to growth. Don't trust anyone to just hand you knowledge, or to stumble across the skills you'll need. You have to actively reach out and take the skills and knowledge you need to be successful. It's an active task. It's not going to be something you just acquire through osmosis.

We all have to get to a point where we realize that we're not as efficient, not as smart, not as skilled, and nowhere near as good as we could be. There's always someone out there who's better than we are.

Our goal isn't to compete with them. Our goal is to constantly aspire to be better than we are right now, at this very moment.

A Parameter Validation Framework

I need some help here. I'm hoping you'll review the following "framework" and give me your feedback. I need to know (1) if it's a good idea, (2) if the architecture is sound, and (3) how it can be improved. If it's a generally useful idea, I'll open it up on an open source project forum where it can be expanded for general use. Otherwise, I'll go back to the drawing board. (I'm not sure at this point if it's a worthwhile idea.)

The Background

In reviewing my projects, I noticed that I tend to be lazy about validating parameters to methods. I reviewed a number of projects that I was working on and noticed that it tended to be true no matter what the project was, or what the parameter types were. When I sat back and thought about it, and paid attention to myself doing it, it came down to two primary issues that led to me avoiding it:

• I was usually in a rush to get a product out the door. This tends to always be the case, since the product's schedule is out of my control, and there isn't much that I can do about it. I can only negotiate for a minor change in the project's time. Then, I'm left to work with the amount of time I'm given. So I try to do the best work I can in the time I'm allotted, and hope to high heaven that it's good work. That often means that I code like hell (classic software mistake), and hope that nothing breaks.
• Parameter validation code tended to be verbose. When I really wanted to say, "Parameter y should be a positive number," the resulting code looked like this:
  If myParameter <= 0 Then
     Throw New ArgumentOutOfRangeException("myParameter", "Must be a positive number.")
  End If

These two issues alone might not seem like a big deal in and of themselves. But when it comes to encouraging code quality, I take them very seriously. I abhor sloppy code, especially when it's mine. I want to be sure that parameters are valid, and that when a method receives parameters, they are checked rigorously to ensure that I'm getting valid values throughout any system I'm developing.

The Goal

I set out to come up with a way to make it easier to validate parameters. I wanted a way to make it so easy to validate them that I would find it enjoyable to do so. I wanted it to make my code more legible, with a minimal amount of impact on its performance. Knowing that I basically lack discipline, I knew I needed a software solution that would exploit Visual Studio's capabilities to remind me to check for tests that I might not have normally thought of. (We have a tendency to inadequately document the requirements—again, due to time constraints—and I need to be able to think about logical tests that make sense for the parameter as I'm coding them.)

So the goals for the framework are:

• Ease of use. This was the number one design goal. If the thing wasn't easy to use, it was pointless to build it in the first place, since it was being designed to encourage validation of parameters. If the framework made it harder to do that, it was defeating the purpose.
• Understandability. The code should enhance the readability of the code.
• Performance. The methods were going to be called with a high degree of frequency. It was critical that the methods have a very low overhead on the call stack. If at all possible, the number of objects being created should be kept to a minimum. However, because we're adding code, some overhead is unavoidable.
• Reliability. It has to work. And it has to work reliably and predictably every time.
• Extensibility. It has to support other data types.
• Maintainability. The framework's code has to be easy to read, understand, and maintain. We do not want to have to bloat the development time of the projects that rely upon it with weeks of maintenance time just to fix and expand the parameter validation library.

The Solution

I set out to come up with a way to make it easier to validate parameters. I wanted a way to make it so easy to validate them that I would find it enjoyable to do so. I wanted it to make my code more legible, with a minimal amount of impact on its performance. I wrote a set of classes that, I believe, accomplished that. The resulting "framework" (if you can call it that) is based on the Assert.That model popularized by NUnit and JUnit. When you want to prove that a parameter is positive, you write the following:

Sub Foo(ByVal integerParameter As Integer)
   Validate.That(integerParameter, "integerParameter").IsPositive()
   ' Do something interesting with integerParameter
End Sub

It's short, sweet, and to the point. If integerParameter contains any value that is less than 1, an ArgumentOutOfRangeException is thrown, with an appropriately formatted message bearing the parameter's name.

The solution involves the following classes:

Class	Description
Validate	A class factory that instantiates strongly-typed parameter validator objects. These are derived from ParameterValidatorBase. Provides the heavily overloaded That method.
BooleanValidator	Inherits from ParameterValidatorBase. Provides methods for validating Boolean parameters, such as IsTrue and IsFalse.
ConnectionValidator	Inherits from ParameterValidatorBase. Provides methods for validating IDbConnection parmaeters, such as IsNotNull, IsOpen, and IsClosed.
DateValidator	Inherits from ParameterValidatorBase. Provides methods for validating date parameters, such as Equals, IsBetween, IsGreaterThan, IsGreaterThanOrEqualTo, IsLessThan and IsLessThanOrEqualTo.
EnumValidator	Inherits from ParameterValidatorBase. Provides methods for validating enum parameters. This is the only parameter validator that relies on reflection. It's sole method is IsValid.
IntegerArrayValidator	Inherits from ParameterValidatorBase. Provides methods for validating integer array parameters, such as IsNotNull and IsNotEmpty.
IntegerValidator	Inherits from ParameterValidatorBase. Provides methods for validating integer parameters, such as Equals, IsGreaterThan, IsGreaterThanOrEqualTo, IsInRange, IsLessThan, IsLessThanOrEqualTo, IsNegative, IsNotNegative, IsOneOf, IsPositive, and NotEqualTo.
ListValidator	Inherits from ParameterValidatorBase. Provides methods for validating IList parameters, including IsEmpty, IsNotEmpty, IsNotNull, and IsNotNullOrEmpty.
ObjectValidator	Inherits from ParameterValidatorBase. Provides methods for validating any Object parameter, including IsNotNull and Equals.
ParameterValidatorBase	The base class for all validators. Provides the name of the parameter and a protected property to store the parameter's value.
StringArrayValidator	Inherits from ParameterValidatorBase. Provides methods for validating string arrays, including IsNotNull, IsNotEmpty, and IsEmpty.
StringValidator	Inherits from ParameterValidatorBase. Provides methods for validating strings, including Contains (overloaded), Endswith (overloaded), IsNotEmpty, IsNotNull, IsNotNullOrEmpty, and StartsWith (overloaded).
TransactionValidator	Inherits from ParameterValidatorBase. Provides methods for validating IDbTransaction parameters, including HasValidConnection, HasOpenConnection and IsNotNull.

Other parameter validator types are being added as the need for them arises.

When a call is made to Validate.That(), the framework instantiates an appropriately typed parameter validator object. The caller then invokes one or more of the methods on that object to prove that the parameter is valid. The parameter validator merely tests the condition specified by the method name; if the test evaluates to False, an appropriately typed exception (derived from ApplicationException) is thrown. The parameter validator is responsible for properly formatting the message and passing the name of the parameter; this is spiffy because some of the exception objects are inconsistent about the order in which the name and message parameters are passed to them. The framework standardizes the order through the use of overloads: parameter, name, message.

For example, the source code for the ConnectionValidator class looks like this:

Option Explicit On 
Option Strict On

Imports System.Data

Namespace Validation

   Public Class ConnectionValidator
      Inherits ParameterValidatorBase

      Friend Sub New(ByVal connection As IDbConnection, ByVal name As String)
         MyBase.New(connection, name)
      End Sub

      Public Sub IsClosed()
         If Not Connection Is Nothing Then
            If (Connection.State And ConnectionState.Closed) = 0 Then
               Throw New ArgumentException("Operation requires a closed connection.", Name)
            End If
         End If
      End Sub

      Public Sub IsNotNull()
         If Connection Is Nothing Then
            Throw New ArgumentNullException(Name)
         End If
      End Sub

      Public Sub IsOpen()
         If Not Connection Is Nothing Then
            If (Connection.State And ConnectionState.Open) = 0 Then
               Throw New ArgumentException("Operation requires an open connection.", Name)
            End If
         End If
      End Sub

      Public Sub IsValid()
         IsNotNull()
         IsOpen()
      End Sub

      Private ReadOnly Property Connection() As IDbConnection
         Get
            Return DirectCast(InnerValue, IDbConnection)
         End Get
      End Property
   End Class

End Namespace

The Pros & Cons

I've noticed that I am, indeed, far more likely to validate parameters now with this framework. I'm catching a lot more defects with it as well. The idea behind it seems to be working. However, it does have a few problems:

• It's failing to meet its extensibility requirement. In order to add new types, I have to hand-code a new overload to the That method into the Validate class. I need to find a new way to do that.
  
• It's not properly localized. (You can see the hard-coded English strings in the code sample above.)
  
• It adds overhead to the stack trace, which can be confusing to users who don't know what it is.
  
• It lacks a way to conveniently perform multiple tests on the same parameter validator object (aside from Visual Basic's With...End With block, which just looks unnatural). The current implementation tends to ask you to create multiple objects to work with the same parameter; although the objects maintain very little state (two variables, both object references) it's still more than I'm comfortable with.

Despite its drawbacks, most of which are addressable, its benefits appear to be worthwhile. My code quality is rapidly improving. Defect rates are dropping noticeably (and I feel comfortable attributing a good portion of it to better parameter validation).

The Request for Comments

So there you have it. At this point, I would really like to hear back from the community, and find out what you think of this thing. Should I be doing this? Should I be doing it better? How would you improve on it? What would your concerns be if you were using it or designing it yourself?

Remember, I'm the quintessential vacuum coder here, so your input is valuable to me.

Feature Creep Accelerates Entropy

Many folks cruising around Digg recently have seen the numerous articles pointing to the claims by Steorn that their Orbo device can produce "free" energy, in direct violation of the Second Law of Thermodynamics. Basically, the device claims to produce more energy than it uses.

Scientists everywhere are choking on their degrees, and with good reason. Such a device would be either (a) a ridiculous grab for attention based completely on fantasy, or (b) a radical shift in our understanding of thermodynamics.

But this post isn't about the Orbo device. It's about a key principle of the Second Law of Thermodynamics: Entropy. Specifically, we're interested in how entropy affects software. To see where I'm coming from, however, let's review:

A measure of the amount of energy in a physical system not available to do work. As a physical system becomes more disordered, and its energy becomes more evenly distributed, that energy becomes less able to do work. For example, a car rolling along a road has kinetic energy that could do work (by carrying or colliding with something, for example); as friction slows it down and its energy is distributed to its surroundings as heat, it loses this ability. The amount of entropy is often thought of as the amount of disorder in a system.

entropy. Dictionary.com. The American Heritage® Science Dictionary. Houghton Mifflin Company. http://dictionary.reference.com/browse/entropy (accessed: July 06, 2007).

 

The car, rolling down the road, only has so much energy to work with. Some of that energy is kinetic (forward motion), and some is being dissipated as friction on the tires and by the air blowing past the car. But there's a finite amount of energy involved. If the driver taps the brakes, more energy is transferred to the brakes, but it's the same amount of energy. The car simply slows down, which has the net effect of reducing the severity of an impact with a tree.

So what the heck does this have to do with software?

All software suffers from entropy. You can only cram so many features into it before it has become so large that it has more unusable features than it has usable features. The code itself, at that point, becomes a chaotic mish-mash of bits and pieces of work cobbled together by people who came and went over time, riddled with cracks where their styles, philosophies, and standards didn't quite mesh well. The quality of the source code itself begins to degrade, becoming less and less maintainable, sprinkled with routines, variables, classes, and modules that are never used. The older the product gets, the more pronounced the entropy effect will be, until someone either cans the project, or demands a full rewrite.

Users tend to have intense, often emotional bonds with their favorite products. Those feelings tend to border on the fanatical when those products are easy and simple to use. They get really upset when you unnecessarily complicate them.

Not surprisingly, the products that seem the most resistant to entropy are the ones with the fewest features. They just work. Light switches are a wonderful example of a device that resists the effects of entropy. You just flip the switch, and the light comes on. But then, down the road, someone fancified them with dimmers. Now the knob pops off. Sometimes, the dimmer breaks. You need special bulbs. People cry out for plain switches. So the dimmer rarely gets used. Entropy claimed it.

Toasters, similarly, are a simple device that resist the effects of entropy. Insert bread; push a lever; electrical current is applied to the heating element; the rack pops up, and you have toast. It's simple. But when we complicated it with toaster ovens, microwaves, and bagel toasters, things got tricky. More features, like programmable times, and self-cleaning, and defrosting, and popcorn timers became complex and we couldn't figure out how to use them. So they became unusable. Entropy claimed them.

In software, similar things happen all the time. We toil for hours to get our features just right. We labor over their design, making sure that this feature we've envisioned is perfect, because we know that users will love it, they need it, they absolutely have to have it. But the truth is that users love simplicity. And the more features you add to a system, the sooner entropy will claim it.

Microsoft Word, for instance, has more features than anyone could possibly use in their lifetime. Yet an entire army of people labored for weeks and months to get them just right. Microsoft Office 2007 is a shining example of unusable energy outweighing the usable energy. It's the poster child for the entropy effect among software products.

When you're designing products, as I do for a living, think carefully about feature creep. The fewer features your product has, the better the chances are that those features will be used. Increasing the number of features increases the chances that most of them will be unused. It's entropy at its best.

Vacuum Coding Syndrome: The Need for Peer Review

I stumbled across this interesting quote today as I was looking for information about the effectiveness of peer code reviews:

There are some thing's you can't unit test. You can't unit test design or architecture. And sometimes you can get unit tests wrong—you might assert the wrong things. There is always a place for human eye-balling of code.

Matt Quail, at JavaOne 2007

This statement resonates profoundly with me because of the situation that I'm in. As I've mentioned previously, I code alone. I have no peers with whom I work and can share designs and submit them for review. So when I make a bad design decision and then implement it, I'm stuck with it. Lately, I've been going over my code, and when I come across some of my older stuff, I'm frequently left wondering just what in the hell I was thinking.

There are designs and architecture decisions that I made three years ago that seemed perfectly reasonable at the time, and now I look at them with abject horror.

In one case, I knew that our database architecture was going to be massive, and that it was going to be changing frequently. I also knew that the data model classes were going to have to be constantly updated to stay in synch with the database, and that there was no way I was going to be able to do all that manual work without missing something. I needed to be able to update the database schema, push a button, have the classes regenerated, and then move along to focus on the business logic.

So I wrote a program that did that. It was, to be sure, a very fast, very efficient program. But it didn't generate very pretty code. In fact, it generated a lot of classes that we never use. For every table, it generated an insert, update, delete, exists, delete all, and select all stored procedure. For each stored procedure it had generated, it created a wrapper class that correctly created the SqlCommand, populated its parameters, and invoked the procedure. It also generated a data model class and a collection class. It also generated a strongly typed primary key class, and the DAC class that invoked the stored procedure classes.

Now that's a lot of code. At the time, it seemed like an act of sheer brilliance, and I marveled at the fact that if the schema changed I could simply regenerate the classes and know that they matched the schema. But the number of stored procedures quickly grew out of proportion to what we actually needed for the system we were building, and the vast majority of them were never used. And if those procedures were never being used, you can bet that the generated classes were never being used, and neither were the methods on the DAC classes.

That program, thankfully, was decommissioned early on, but it left a massive amount of code in its wake. Newer code no longer follows the model that it established, but is, in fact, leaner, smaller, and uses far fewer classes and stored procedures. It's a joy to work with the new code. But I groan and cringe every time I have to wade through that older code. 

It was a case of very efficiently created code bloat. It would very likely have been stopped early on had someone been there to either (a) review the design of the code generator, (b) point me to an existing tool that was better at it, or (c) convince me that we actually had time to just do it the right way in the first place.

In any case, I'd never do it that way again. I have been duly chastised by my own foolishness.

I'm absolutely certain that I'm not the only one suffering from Vacuum Coding Syndrome. I'm equally certain that I need to have someone reviewing my designs and my code. So as I sat chain-smoking today, looking at a piece of paper with an abominably badly written piece of code printed on it that I knew I had to fix, I realized that I really needed to start submitting my designs for peer review to someone. Anyone, at this point, really. Because I really need to stop making silly mistakes. I'm the one that has to live with them.

So I got to thinking about it. In the past, I've submitted small snippets of code onto Usenet for review, and the response has been mixed. I've submitted some of my good code onto The Code Project and the response has actually been pretty good (but that code works). Now my brain is churning over a different kind of forum.

What I need is a forum where developers can submit code and/or designs for peer review. That would be the whole point of the forum. The online community is overflowing with professionals who are far more experienced, far more intelligent, and far more capable than I am. It would be a great boon to lone developers if there was a site designed specifically to allow us to post our code samples (sans IP) and designs (in some ubiquitous format) for peer review. 

I've seen products like CodeCollaborator, which foster collaborative code reviews and allow users to work together on code reviews using software. You can chat, compare notes, diff the files, and so on. Why hasn't anyone thought to do this on the Web? I'm not envisioning everything that CodeCollaborator does on the Web; what I am envisioning is a simple forum, like The Code Project or Construx's Software Best Practices.

So here I am, scouring the web, looking for such a resource. If any of you happen to know of one, I'd be much obliged if you could point it out to me. I'd roll my own, but given my history of making poor design decisions, and my lack of access to resources capable of reviewing my designs...

Well, you know. <shrug/>

On Self-Control and Software Development

This essay was written months ago, and never posted. I resurrected it today, in light of certain recent entries.

Recently, as I was working to deliver a major release on a product I'm working on, I found myself sidetracked by a little project of my own.

You see, there's this little problem with one of the data fields in the database. It's not major, just an annoyance, like a four year old poking you in the ribs for an hour, asking repeatedly, "Does this bug you?"

Well, it's been bugging me for ages. And I found myself today doing database queries and pasting data into Excel to have Excel build update queries for me using formulas (nice little time saver that is) so that I could include those statements in the SQL script to accompany the next major release.

And then it hit me: no one asked for this. It's not included in the test plan for this release. It's gold plating. I'm doing this because I want to, not because the customer asked me to.

Whoa, there, cowboy. Get a grip on yourself. Set that stuff aside, and focus on what you need to do, and not what you want to do. There are far more important deliverables to worry about, and you don't have time to waste on unauthorized features or fixes. Especially when those fixes are for issues that don't negatively impact the application. (It was a display issue--first name before last name.) It's just fluff.

In reflection, I find myself experiencing these kinds of monumental self-control issues all the time. I get really excited about the things I could do for the customer, and I really want to do them for them. But the truth is that just because I can do something for them, it doesn't mean that I should do it.

Any change that I make to the product has the potential to introduce new defects into the system. That's why every change that I make to it must be tested.  It's why there's so much testing involved in software. (And if there isn't, something's seriously wrong.) And the testing doesn't just occur here, at my desk. It happens at the client. The product undergoes rigorous user acceptance testing. And testing isn't cheap--it consumes precious man hours, which equates to someone's hourly wages. And if I haven't gotten it right, it has to be fixed and retested. It can amount to massive amounts of money in man hours of testing.

Lets not forget the impact that the change has on updating the test plan, the release notes, requirements documentation, and user guides. Plus any associated costs with reprinting and redistributing them.

And what happens if the customer decides that my unauthorized change needs to be taken out? What if its impact on the system is so drastically negative that it must be removed? Can it be easily rolled back? And if it must be removed, what are the costs associated with doing so, and republishing all the updated documentation and builds?

Are you getting my point yet? The cost of a simple change isn't just what it takes me to code and test it at my desk. That's just the tip of a massive iceberg.

It takes a lot of self-control to prevent myself from adding features to a system when those features aren't (1) requested by the customer, (2) included in the project plan, and (3) absolutely critical to the current release.

The problem, I think, is that a lot of developers out there, myself included, don't get sufficient mentoring in the discipline of self-control when it comes to software development.

For example, we're all hailing the virtues of refactoring code to improve its maintainability, and I agree that that's a good and useful thing. But how many developers know that just because you can refactor a piece of code doesn't mean that you should? How many developers are out there bogging down project schedules because they're busy refactoring code when they should be developing software that meets the requirements for the project deadline?

(And here, I will sheepishly raise my hand.)

It occurs to me that before I ever modify a piece of code, before I ever touch that keyboard and write any new class or method, or create any new window or Web page, I should be asking myself, "Is this in the project plan? Is it critical to the current release?" If it doesn't satisfy one of those questions, I shouldn't be doing it.

The key to getting that product out the door on time is staying focused, and not getting sidetracked by fluff. Take it from someone with experience: it's easy to get sidetracked by fluff. Adding cool features is easy to do, because you're excited about it, and motivated to do it. Working on the required deliverables is hard work; it requires discipline and self-control. You have to stay focused and keep your eyes on the target. (You thought I was going to say "ball," didn't you?)

But we, as human beings, don't want to do what we need to do, we want to do what interests us, and what excites us. It takes an act of sheer will to resist that urge, to restrain ourselves, and get the real work done. I would imagine that one of the things that separates a mature developer from a novice developer is quite likely his or her ability to resist that urge to introduce fluff into software.

In the end, I think it might be a good idea if programming courses included curricula on self-control as a discipline for developers. And I mean that quite seriously. We need to have it drilled into our heads that we shouldn't be adding anything to the product that only serves our own sense of what's cool or useful. That's not to say that sometimes developers can't predict useful features before the users do; but they cannot and should not be introduced haphazardly into a product: they should be included as a planned feature as part of a scheduled release, so that they can be adequately tested and documented, and not just suddenly sprung upon someone as an easter egg.

There's a time and a place for everything. Gung-ho initiative has its proper place; software isn't one of them.

Traffic School for Software Projects

Ever wonder if there's a correlation between the way some folks develop software and the way they drive their cars?

The Tailgaters

These are the ones who are constantly on your ass, going 90 MPH, and flipping you the bird because you are holding them up. You might even be going faster than the guy to your right, but that's apparently not fast enough for the maniac behind you. If you get out of this guy's way, he will inevitably zoom past you, race right up on the next guy, and do the same thing to him. These guys are an accident waiting to happen. They aren't thinking about the safety of others, or about the possible parking lot ahead of them on the road. All they care about is getting where they're going as fast as possible. If some poor slob gets angry enough, he's going to slam on his brakes, and there's going to be a loud screeching sound followed by a crash, car parts flying everywhere, sirens, flashing lights, and lawsuits.

In the software world, these are the folks who are constantly hounding you, demanding, "Is it done yet? Is it done yet? Come on! Hurry up!" The quality of the project takes a back seat to the speed of its delivery. They will likely sacrifice people, resources, process, features, and quality and not worry about the risks that await them down the road. It's all about meeting the deadline, and God help those who get in their way.

Advice to you Tailgaters: SLOW THE HELL DOWN! It's better to get there alive than in a body bag. You aren't saving that much time by going so fast you make Mario Andretti look like an old man with a walker. You're risking lives, and you will eventually crash and burn. I'll wager that alot of you think you've never been in and accident. Well, that might be true. But I bet you've caused a lot of them.

Same goes for you software tailgaters. Slow it down. You want the project to get there in one piece, with as much of its promised feature set and team in-tact as possible. Your constant pressure isn't helping anyone. A healthy dose of encouragement is one thing. Unhealthy pressure (tailgating) is quite another; it kills morale, and results in high turnover, which leads to increased costs and loss of employee buy-in. So knock it off; let your foot off the accelerator!

The Weavers

Weavers like to move in and out of traffic, zipping from one lane to another, often unexpectedly, and rarely using their turn signals. Their lane changes are unpredictable, haphazard, unannounced, and a source of frustration and terror to those around them. They are the reason that defensive drivers exist. You can't predict where they're going, when they're going to change directions, or what direction they're going to move in. The best you can do is hope you have power steering, anti-lock brakes, and really good peripheral vision.

Weavers drive like they're the only ones on the road; changing lanes doesn't require a turn signal because, well, they don't care if anyone's in harm's way. All that's important is that they get into the lane they want to be in. They just yank that wheel to the left or right and fwoop! there they are where they want to be, while your brakes squeal, your car lurches, and heart hammers in your chest. We won't talk about the string of expletives that explodes from your mouth.

In the software world, these are the guys on the project who like to suddenly shift direction midstream. It might be a change in the process, in the coding standard, in the project's vision. It could be a change in how the teams are set up, in the reporting structures, in the UI's design, in the tool selection, in anything. Anything. But they thrive on change, and they don't really care about the impact that those changes have on the people around them. They're oblivious to the others around them. Change is exciting! It's good! It's cutting...no, it's bleeeeeeeeding edge!

Weavers work like they're the only ones on the project, with complete freedom to do whatever they like. No warning is necessary; asking for someone's advice or opinion about it's suitability is pointless because it doesn't matter to them. They want to do it, so they're going to do it. Who care's if anyone else likes it?

Advice to the Weavers: Learn to use your mirrors and turn signals! That means you actually have to become aware of the other drivers on the road. Yes, I know. That means you sadly have to realize that you are not the center of the universe, and that there are others who share the road with you.

And for those of you who think you can just change anything about a project midstream, learn to think differently. Change introduces risk into a project. Unannounced risk is unmanaged risk. Unmanaged risk increases the chances of project failure. (Crash and burn, Mav. Crash and burn.)

The Gawkers

Gawkers are a special breed of road hazard. These folks will be traveling down the road and spot something they find of interest, and slow down to a crawl to scrutinize it. In the process, the people behind them must slow down as well. This creates a nifty domino effect in traffic. Interestingly enough, what these folks find of interest is usually nothing of interest at all. But the ensuing traffic jam becomes a subject of much heated debate.

Software projects are commonly plagued by Gawkers: individuals who have no real business being involved in the project, but who nonetheless feel compelled to attend every meeting and inject their two cents, suggest things, and add "filler material" to waste everyone's time. They want to be CC'd on every email, take part in every hallway conversation, and stop by the developers desks to "check-in" and provide "moral support." They pepper us with questions and phone calls, spot-checks and emails, instant messages, and reminders, additions to meeting agendas, and interjections about meaningless and completely unrelated minutia in the middle of our meetings when we'd rather be back at our desks doing real work that matters.

Advice to the Gawkers: For God's sake people, stop gawking and drive! Pay attention to the road ahead of you, ignore that pigeon crapping on the rail, or that car parked on the side of the road, and just drive! You're holding up traffic for miles and leaving a string of accidents behind you.

And all you busybodies who seem to think you have to be involved in projects that have nothing to do with you, "Move along people, nothing to see here." We have work to do, and you're holding us up. Just leave us alone so we can get it done.

The Gadgeteers

We've all seen them. Some of us have been them. Driving down the road, accelerator hammered to the floor, while trying to operate some electronic gadget, be it a cell phone, Blackberry, iPod, radio, or even something as innocuous as the car radio. These geeks are so distracted by what they're doing that they'll careen recklessly from side to side within their lane, as if they're using the Braille Method to stay within the lines. They'll speed up, slow down, and be so engrossed by the spectacular fabulousness of their toys that they have become a danger to everyone else around them.

With the Gadgeteer, it's all about the toys. These devices aren't required to operate their vehicle: they're distractions that make it unsafe to operate the vehicle, especially at high speed, because they make it impossible for the driver of the vehicle to focus on what he or she should be focusing on: the road, and the other vehicles on it.

Software professionals are often distracted by the shiny, flashy appeal of the latest tools. But the truth of the matter is, you can write software in Notepad or any simple text editor and compile it with a command-line compiler. You don't need the flashy, glitzy tools to get the job done. (For the difference between need and want see here.) They might make your job easier, more comfortable, and less of a nightmare, but you don't truly need them. So don't get all obsessed about them and put other people at risk over them. Certainly don't become obsessed about them.

And whatever you do, don't let them constrain you or the project.

Advice to the Gadgeteer: First off, shut up and drive. If you're so compulsive about yakking on a cellphone 24/7, get an implant with a neural interface. Oh wait, we don't have that technology yet. Probably because NO ONE NEEDS TO BE THAT CONNECTED! So disconnect, relax, and enjoy the drive. Stop fiddling with all those gadgets, put them down, and pay attention.

If you're a software Gadgeteer, learn to do without for a while. Learn what it is to use primitive tools. You might actually learn to appreciate the tools that you do have. All those big, flashy programs you use are likely overkill for what you do, anyway. How many of the features do you actually use? Does the system really make you more productive, or does it spend a lot of time distracting you with countless configuration options, window placement options, font and color options, and so on? Wouldn't it be nice if you could actually focus on the code for a change?

The Lost Navigators

There's not a lot of hope for a Lost Navigator. Drifting slowly through the highways, byways, and side-streets of the country, looking for their exit or side street, hopelessly lost, and without a clue, these poor folks present the rest of us with a horrible choice: to be patient, or to pass. Our frustration knows no bounds. They drive slowly, obviously looking for something; sometimes, you can see them, scanning a mapbook, or consulting a crumpled set of MapQuest directions. In the worst case, they have no directions whatsoever to go by. Inevitably, they never seem to stop and ask for directions. They are lost, they are slow, and they are frequently oblivious to the line of cars piling up behind them as they look desperately for the place that they should be going.

Sometimes, in a moment of relief, they see their exit, and swerve to enter it. But frighteningly, this is all too often done from the left lane of the highway, and they must cross two or three lanes of traffic at the last moment to reach their offramp. And so they do. Cars swerve to get out of their way, slamming on brakes, honking horns, and nearly going off the road themselves.

Their problem, usually, is that they simply lack a decent navigational system, a decent set of instructions that tell them how to get where they want to go. It's not always their fault. Sometimes, however, you get the pig-headed kind who decide they can find their way through anything and can't be bothered to stop and ask for directions.

Often, in software development, there are folks who are placed in roles for which they are not well equipped. They are assigned a task, given little to go on, and told to do it. They have the little bit of information they're given, and they set out. So they move slowly, trying to find their way, and they might stop occasionally to ask for help. But their progress is slow, and as a result, their piece of the project moves slow. That's not so bad, actually; it's the guys who refuse to ask for help that cause the problem. By refusing to seek help, they're making a bad situation worse.

Advice to the Lost Navigators: Get better instructions. MapQuest lies. Frequently. Improve your situation! And for you blockheads who won't ask for directions, get off your high horse. Although I'll guess that your problem's a genetic defect of some sort, so I won't push that issue any further.

If you're in a software project, and you're struggling due to a lack of information, get that information! You can't be expected to do your job without it. If you need information about process and such, seek out excellent sources of information from places like the Software Best Practices forums maintained by Construx software. Interact with other developers in the Google newsgroups (drop me a line and I'll point you to them). Check out The Code Project and the programming topics at reddit. Read MSDN. Ask other developers where they go for information online. There's no reason for you to suffer from a lack of information when there's a wealth of free information from experienced developers providing it online.

And there are Others...

We know there are more. Let's not forget the Makeup Artist, the Conversationalist, the Belligerent Road Hog, and so on. But I'm sure you all can figure those out for yourselves.

And I'm sure you all have ideas of your own.  

Methodology Fundamentalism

If you aren't taking part in Steve McConnel's Software Best Practices forums, you're missing out. It's not very active yet, but it's got tons of potential. By all means, check it out.

In a really interesting thread, a poster suggested that he'd like to ban the use of the term, "Best Practices," given that it's become something of a convenient excuse that IT professionals use to excuse every insane practice under the sun, regardless of its logical suitability to the business or environment. In his particular case, he cites this lovely bit of whimsy:

The IT manager at my last place of employment had set up the network to change passwords every 30 days. His response every single time the multitudes cried out in pain was to hold up a sheet of "best practices". Yahhh!

Now, admittedly, lots of companies have this policy in place. But if the only reason you can explain the 30-day password change policy is because it's on a best practices list somewhere, you've got a problem.

The Chief Software Engineer at one of my previous employers had a staunch policy that Is-A inheritance had to be maintained at all times, and could not be violated at any time whatsoever, even if you had a perfectly good reason for doing so. He would rewrite your code behind your back to make sure it happened. "Best practices, dontcha know." End result: confused developers who had code changed on them, extra code written to support additional classes, angry developers, adversarial relationships, hostile work environment, and, eventually, employee turnover.

Blind adherence to Best Practices is a Very Bad Thing. You need to have a compelling reason to do so. If you're doing it just for the sake of doing it, you're shooting yourself in the foot, and likely pissing people off in the process.

The same is true if you're just blindly grabbing hold of The Next Big Methodology. We're inundated with methodologies in this business. Test Driven Development, Refactoring, Visual Modeling, Big Upfront Design, the Waterfall Method, Use Cases, Agile, Extreme Programming, and on and on and on. Every one of them involves risk. The risk is largely due to the fact that there are people involved, and people tend to clash, especially when they're under pressure. So you can't just foist any old methodology on them; them you have to pick the one that's going to work best for them, given the number of people that you have, the resources at your disposal, the time you have, and the project you're doing.

And let's be very clear, people: There is no Silver Bullet Methodology.

A few months ago, I responded to a thread on the Braidy Tester (a great site, love Michael Hunter's stuff) that blind adherence to "best practices" and adoption of "The Next Big Thing" without adequate analysis of the risks involved was unwise. I contended that sometimes you had to deviate from what everyone else was doing and develop your own customized methodology that might be a hodgepodge in order to get the job done, because what worked for ABC corporation didn't necessarily work for Your Real World Company, Inc.

A Microsoft Developer, who shall remain unnamed, said, in short, that I had an amateur opinion.

But the truth is that I've watched a lot of folks (and entire companies) become Best Practice Zealots, and turn this whole thing into a Fundamentalist Religion. Best practices and methodologies are supposed to help you get your job done better, faster, on time, within budget, and according to specification and customer expectations. They aren't supposed to be an iron rod with which you can bludgeon everyone around you. Too many companies are willing to just snatch up the next Silver Bullet Methodology and apply it without determining whether or not it fits their business model, their project, or their customer.

If you're going to embrace "best practices" remember this: they're guideposts, sitting on the side of the road, pointing you towards success. If you're using methodologies, they're supposed to be the route that gets you where you're going without hitting any tolls, construction sites, congestion points, and so on. Best practices and methodologies are supposed to be used together, to get you where you want to go: to a successful product delivery, with the whole team, the client, and the product intact. If anything is going awry, be willing to reevaluate your selection of practices and methodologies.

Caveat: Don't switch methodologies in the middle of the project. Jeesh. I'm not that stupid. (Though I can see where some people might think so.)

Dew is to Water As Want is to Need

On a recent Coding Horror entry, Jeff Atwood brought up a really interesting point about the power of observing users versus asking them. Paraphrased, what users actually want is typically not what they think they want or tell you they want.

It's funny and sad because it's true.

It took me a long time to understand the difference between want and need. I may want a nice, tall fizzy bottle of Mountain Dew, but my body needs water. It doesn't need Mountain Dew. Sure, Mountain Dew tastes better, and I like the fizz, and I look way more cool when I'm holding it, but I don't need it. I need water to hydrate my body and keep me numbered among the living. There's a big difference.

Similarly, when users tell you they want a piece of software that does X, Y, and Z, what they usually need is something that does A and B. (Usually, A and B are something on the order of "It works well" and "It doesn't corrupt my data.")

Nonetheless, trying to get users to tell you what they need is akin to extracting molars from a chicken. It's nearly impossible. They'll give you something like this:

• It has to look really good. You know, like SILF. (Software I'd Like to @#$%)
• It has to be fast. Really fast. Like, it has to be so fast that I get whiplash when it starts up.
• It can't hoard memory. Cuz I'm using Windows 98. In fact, can you make it use no memory at all?
• It has to be secure. Really secure. Like, Fort Knox secure. Oh, but I want to be able to pass it around on the Internet and share it with all my friends. Or on a USB drive. Or whatever. Ooh! BitTorrent!
• It has to be a Web app to. With Flash. In fact, do it all in Flash. But I have to be able to use it on my cell phone. And on my XBox. And FireFox. FireFox totally pwnz Micro$oft.
• Everything has to be done in my company's colors: Black and brown. I want all the text in this really cool dark brown color, and the background all has to be black! It's bitchin'! And flames everywhere! And I have this cool soundtrack I want to play throughout the whole thing! And every button should be a different color, and they should make machine gun sounds when you click them! And then explode with a giant fireball!
• I've only got a budget of $500.
• I need it by Friday.

Oh, God. The horror. The humanity!

While this list is obviously heavily laden with hyperbole, it's not too far from the truth. A completely user-driven set of application specifications, expressing their wants would likely provide nothing that they needed. At some point, you have to realize that users are, in fact, dreaming about toys.

In the end, software is about enabling users to get their work done faster and easier with a minimal amount of hassle. But users don't know that. Largely, they think it's about fun. Not all software is a first-person shooter or a massively multiplayer online role-playing game. (Sadly.) As a member of a software team, it's our job to identify their needs, and create software that meets those needs without getting sidetracked.

If you're starting a new system from scratch, don't waste your time asking the users what they want; ask them what they need. Explain the difference to them. (Begin by making sure you understand it yourself.) Explain that everything that isn't needed that is added to a product incurs additional cost and delays the delivery of the product, and that someone has to pay for it.

Build systems into your products to monitor which features are being used the most. Don't trust users to tell you. They aren't thinking about it. This kind of information is very useful for determining where to spend your time improving the product. It's also useful for identifying features that you think are critical (in business applications, especially) that should be used, but aren't being used. You can figure out why they aren't being used and target those areas for resolution.

If you write a solid subsystem like that for your software, you won't have to rely on user feedback, which isn't always reliable. A solid monitoring system will not lie to you. And that information will help you make much more intelligent decisions in the future. It's a question of silently observing the users as they actively use the product, instead of asking them about it after they've done so. All of us have a pretty short attention span when it comes to software use. I have no idea what the most used commands are in any given software package based on my own usage scenarios. For that reason, you couldn't ask me to tell you what commands or features I use and get a meaningful answer from me.

So think about whether or not you need it. If you do, invest the time to do it. But don't do it because you want it. Do it because you need it.

Dew? Or water?

Everything I Need to Know about Debugging I Learned from CSI

I started to write this blog entry a few months ago, but it quickly got out of hand. With a little encouragement from a tester at Microsoft I was encouraged to publish it to a magazine, but things at work got out of hand, and I just never found the time. Since I am the quintessential procrastinator, I've decided to just publish it here, so that it will at least get published in some form. So, without further adieu, I present it for your general amusement.

—Mike

Okay, I admit it. I'm a CSI dweeb. No, I don't like CSI: Miami (it blows). I like the original CSI. And although all of those shows are equally implausible and unrealistic (let's face it, no CSI team is that thorough, that precise, or that good), the very premise of crime scene investigation and its parallels to defect resolution hit me like a ton of bricks recently.

At first thought, it's just a corny idea. But then, the more I thought about it, the more I realized that the idea isn't as silly as it sounds. I thought about writing this article as a parody, but when I set out to do so, it didn't turn out that way.

Some folks might look at this and laugh their butts off. But read it, think about it, chew on it, and then, after you're done, if you still think the parallels aren't striking, go ahead and laugh.

The Basics of Crime Scene Investigation and Defect Resolution

We’ve all done the dirty work in software development: defect resolution. In many companies, it’s the first place where new developers are unceremoniously dumped when they are brought on board. The thinking is that it will familiarize them with the product. “You’ll learn the code!” they say. “Then you can move into the development.” These poor saps are armed with reams of source code, an IDE, and a compiler, and sent marching into the battlefield with a stack of defect reports and an order to make progress repairing a system with which they typically have no experience whatsoever.

This thinking is fundamentally flawed. You don’t want someone who doesn’t know a lick about a complex software system trying to resolve its defects. But that’s a subject for another article.

When confronted with a defect report, there are a certain number of predictable responses that tend to flash through every developer’s mind when he hears it:

1. “It’s an ID10T error.” This one’s my favorite. It can’t possibly be a defect in any code that I wrote. The user must have done something wrong. Everyone knows users are st00pid. I mean, just look at them. They’re like, lame. And stuff.

   In the Dark Ages, this might have flown, but this is the 21^st Century. There’s this thing called a presumption of innocence.

2. “We already fixed that.” Another keen insight. If it’s already fixed, why is it still happening? If it’s already been fixed, you’ll have to provide proof that it’s been fixed in a build in your test environment. If you can’t prove that, then what you’ve likely done is fixed the wrong thing and claimed victory. As we’ll see later, this will fall under the novel concept of “Convicting the wrong suspect.”

3. “He did what? You’re not supposed to do that.” Okay, let’s get something straight: just because a developer might not do something doesn’t mean that a user won’t. Users do unpredictable things all the time. And your lack of coding for it doesn’t mean that it’s not a defect. Holding the users at fault for being unpredictable is not an acceptable excuse. Inadequate code coverage in the test plan is a defect. Get used to it.

4. “It’s a known issue and we can’t do anything about it.” Okay, that’s marginally acceptable. Sometimes. But have you made an effort in the software to barricade the users from the effects? Trained them? Documented it? Why are users still running into it?

5. “I know exactly what causes that. Let me fix that right now.” The most fatal of all the answers. This knee-jerk reaction is what leads to reaction #2. This is always a bad response, and only in the rarest of cases is it ever right. I would estimate the chances of it being right as roughly equivalent to those of a stray cosmic ray setting off a nuclear disaster that ended the world within three seconds of your reading this sentence. Okay, that’s extreme. But you get my point.

   Defects resolved this way are rarely documented properly. Test plans are rarely updated to ensure that the fix is correctly tested. They’re just quietly slipped in, like an Easter egg, and no one is any wiser. The only thing that gives them away is that there’s a new version of the file in the source code repository. (You do have a source code repository, right?) And that’s assuming it’s the only change in that version of the file.

6. “Oh God. What now?!” Don’t even tell me this has never crossed your mind. We’re all swamped. Products slip, schedules get crazy, we work overtime, and work piles up. We try to prioritize, but things get missed. Defects get buried in a stack, and some of them just don’t get fixed. We don’t see defects as challenges, we see them as annoyances, burdens, more junk sitting on our plate when we’re already seriously overtaxed.

We all know that there’s more clever and witty responses out there. Some of them I just can’t put in print. But for the purposes of this article, I think we’ve painted a pretty accurate picture of defect resolution as it stands today: it’s viewed as a dull job, one that’s resented, a pain in the neck, and one that no one looks forward to.

Let’s face it. You have to essentially tell the developers that their code is broken. Or, you have to tell the users that they don’t know what they’re doing, or that there’s nothing that can be done, and that they just have to wait. Either way, it’s a no-win scenario for you. You always come out the bad guy. No one wants to cooperate with you, because they know that you’re only going to give someone bad news. If you’re new to the company, you probably don’t even know anything about the product to begin with, so you’re flying by the seat of your pants as well. And if your company is like most, you don’t have the best equipment or software to make finding those defects as easy as it could or should be.

Who the heck would want that job?

Now, turn your attention to another group of individuals who are stuck in the very same situation. Their job is no different. They have to do the same basic thing. They have to wade into a situation that they know nothing about, typically understaffed and underequipped, and determine whether or not a problem occurred. Then they have to accuse someone of being in the wrong, or telling both sides that no wrongdoing took place at all (potentially angering both sides). Through it all, their job is to figure out the who, the what, the where, the why, and the how of it all. Crime scene investigators do this every day. They wade into a new crime scene, knowing only that a crime may have been committed, that one or more suspects are at large, and they have a crime scene to work with. They’re given the evidence, and told to run with it. Sound familiar? It should.

Crime scene investigation is essentially the act of solving a complex problem: finding the truth in a vaguely described problem when you’ve got few hands, little money, a lack of resources, a finite amount of time, and every witness can be a suspect. At the end of each case, they have to render their findings, and simply state the facts, regardless of whether or not the victim or the justice system likes it. Sometimes they’re praised, sometimes they’re despised. But they’re frequently overworked and underpaid, and the amount of care they have to take to get their jobs done is mind-boggling. If they make a mistake that tampers with the evidence, an entire case can get thrown out of court.

The Process

The job of the crime scene investigator is to determine the following:

1. Whether or not a crime was committed.
2. If a crime was commited, what the crime was.
3. If a crime was commited, who committed it.
4. if a crime was commited, how it was commited.

You’ll note that the investigator is not responsible for prosecuting the crime. His job is simply to collect the evidence, analyze it, and form a theory that fits the facts and leads to the perpetrators of the crime (if any).

The crime scene investigator uses the scientific method to arrive at his or her conclusions. The American Heritage Dictionary defines the Scientific Method as:

n. The principles and empirical processes of discovery and demonstration considered characteristic of or necessary for scientific investigation, generally involving the observation of phenomena, the formulation of a hypothesis concerning the phenomena, experimentation to demonstrate the truth or falseness of the hypothesis, and a conclusion that validates or modifies the hypothesis.

In other words, “Prove it, buster.”

Here’s the gist of it: You need to gather the facts, form a hypothesis based on the facts, and then prove your hypothesis. In crime scene analysis, proving the hypothesis leads you to one or more suspects who is or are more than likely guilty of committing the crime. You don’t rely on the “hunch.” Hunches put innocent people behind bars, wasting taxpayer dollars, and getting cases thrown out of court or convictions overturned on appeal.

In defect resolution, the same practice applies. You gather the facts, determine whether or not an actual defect exists, and then review the facts to create a theory. Then you prove the theory. If you can’t prove the theory, you don’t have a case. You’ll likely fix the wrong code, incorrectly mark it as “not reproducible,” “by design,” or “user error,” or fix part of the problem while the other parts that contributed to the problem remain uncorrected.

As a CSI conducts his investigation, certain guiding principles govern the way that the investigation is conducted. These are:

• Humans lie and make mistakes; evidence doesn’t. When you can’t rely on the witnesses, keep going back to the evidence to find the truth.
• You always want to convict the guilty party. You never want to convict the wrong party of the crime. When you do get a conviction, you want it to stick; you never want it to be overturned on an appeal.
• Your first suspect is usually not the right suspect. Knee-jerk reactions tend to be wrong, and based on faulty assumptions. Careful evaluation of the evidence leads you to the right suspect(s).
• You want to convict all of the guilty parties, not just one or some of them.
• Don’t be swayed by your emotions or personal involvement. Always remain detached and objective.
• Expensive tools aren’t always required to analyze the evidence. Sometimes, it’s simple tools that can be found at our fingertips every day that will do the trick.
• Patience and persistence rule the day.
• There ain't nothin' glamorous about this job. It's full of blood, gore, hate, anger, greed, fecal matter, tire tread, and a lot of pavement. No one ever cooperates willingly, but they all want answers now. And no one is ever guilty. Get used to it.

So without further adieu, let's see how the CSI process parallels defect resolution. Hold onto your butts people, it's going to be a bumpy ride.

The Process

Identify the Crime

Any time an alleged defect occurs in your product, treat it as a crime. After all, some part of your code has theoretically failed to meet its contractual obligation to the end user (or, so we’re assuming for the purposes of this article). You’ll first want to know what this alleged crime was. Was data corrupted? Did the software simply vanish off the screen? Did an error message appear? Did the screen lock up? Was sensitive data compromised?

Once you identify the crime, you'll need to categorize it. Its severity helps you to determine how quickly it needs to be resolved.

It's important to note, however, that at this point, you don't know that a defect has actually occurred. All you really know is that something happened. You still have to prove that it's a defect. So you start taking copious notes. This is why you need a defect tracking system. You need a place where you can record as much information about the event as you possibly can--preferably in one place.

Identify the Victim and Witnesses

The victim and witnesses provide valuable insight to what happened when the crime occurred. But it’s important to realize that witness accounts tend to be fuzzy at best.

Crimes and defects tend to catch people by surprise—they’re usually not paying close attention when these things happen, and the panic factor is pretty high, so relevant and often important details tend to escape their notice. You’ll still find their input valuable for recreating the series of events that led up to the event, and certain general information about it; but you will do well to remember that witnesses typically are not an authoritative source of information.

Identify the Crime Scene

When an alleged defect occurs in your product, treat the event as a crime scene.

You’ll want to know where and when the event occurred, what version of your product was being used, what OS it was being used on, what browser was being used, any plug-ins or service packs applied, what the user load was at the time, and so on. Any of these might have a bearing on the crime that was committed. You’ll need to know this information so that you know exactly which version of your software to use when you recreate the "crime scene" later.

Preserve the Crime Scene

It is absolutely imperative that you preserve the state of your software while you are attempting to identify the cause of the defect that occurred. If the environment is changing, someone is tampering with the evidence, and the evidence can no longer be relied upon to point you to the right suspect.

This is why a solid revision control process is critical to defect resolution. Every build must be labeled in your source code repository so that you can recreate it, and test it for defects. You must be able to recreate the environment later, and that means being able to use the same version of the software that the defect occurred in. You’ll hopefully have the means to do it on the same OS, with the same browser and plug-ins that the victim was using, but that’s not always feasible due to cost constraints. But having access to the source code that was used to create the software is absolutely essential. Your suspect may be hiding in there somewhere.

Collect the Evidence

The evidence is what you will base your findings on. Everything else will be ignored, because only the evidence can be relied upon to tell you the truth. Evidence includes the source code for the build in question, a fresh copy of the database, any exceptions that occurred, event log entries, data files, screen shots, and other output from the software generated at the time that the defect occurred.

Do not include email communications as evidence unless they were system-generated; interpersonal communications are testimony, not evidence.

Collect Testimony

Testimony includes emails, voice mails, and oral accounts from users that describe what happened when the defect occurred. It is vitally important to note that testimony is not evidence. Rather, testimony helps you to evaluate the evidence. Testimony is subject to witness credibility and the fallibility of human recollection.

That probably sounds pretty harsh, but it’s a simple statement of fact. As we’ve mentioned before, folks tend to be caught off guard when something goes wrong. They aren’t expecting someone to snatch their purse, jack their car, or corrupt their data. It takes them by surprise. Consequently, they don’t tend to be looking for the vital details that you need from them when you are trying to figure out what happened. They’ll tend to remember vague details, but not the specifics.

There’s also the uncomfortable truth that we simply don’t like to admit that we might have done something wrong. So we’re reluctant to divulge information. And we’re emotional when our data is corrupted or software that we’re required to use doesn’t behave as it’s supposed to and we’ve got tons of work to do. We get angry, even hostile. It’s human nature. We all do it. But the net effect is that our testimony in those situations isn’t always as objective as it might be. It’s subjective, defensive, and guarded.

Finally, witnesses can only tell you what they saw, not what happened internally. If it were a medical condition, we would say that they saw the symptoms, and not the underlying disease. As a professional, you don’t want to treat the symptom; you want to root out the disease. But a witness can’t tell you anything about the disease because she simply can’t see it.

So the witnesses’ testimony helps you to evaluate the evidence, but it isn’t evidence in and of itself.

Be careful, however, that you do not treat witnesses with hostility. Just because witnesses may not be accurate sources of information does not mean that they are dishonest sources of information. Always treat them with respect and understanding. Remember the golden rule when interviewing the witness: You’ll get more with honey than you will with vinegar.

Analyze the Evidence

Once you have all of the evidence, you must analyze it to determine what happened. Sometimes, the crime that was reported turns out not to be the crime that occurred. It turns out to be the wrong crime altogether. Or there was no crime at all. Careful analysis of the evidence determines whether or not a crime occurred at all; if one did, analysis of the evidence determines when it occurred, where, and who the likely suspects are. You can use the testimony to evaluate the evidence, such as to reconstruct the order of events that led up to the crime in question. But, again, do not treat the testimony as evidence.

You should use every available tool at your disposal to evaluate the evidence, including your application’s debugger, tracing tools, the event viewer, query tools, hex viewers, file parsers and viewers, system diagnostic utilities, network utilities, and so on. Expensive tools aren’t always required. A simple text editor is often sufficient for viewing data files, and a baseline graphic editor will suffice for viewing graphics files in most cases. Wherever possible, use the simplest tool that will accurately evaluate the evidence before you. There’s no need to inflate the costs of your investigation.

The outcome of your analysis should be a theory of the crime. You should have one or more suspects: the portion(s) of your code or the external components that caused the defect.

The next step is not to rush off and fix the code. Rather, you need to prove your theory. After all, you want to convict the suspect, and you want to convict the right suspect. And you don’t want to put this suspect in jail, only to find out that the same exact crime is being committed by another suspect that you hadn’t considered. This is especially true if your suspect is the victim.

Recreate the Crime

So now you have a theory. You just have to prove it. And you have to prove it beyond a reasonable doubt. So, you have to recreate the crime scene, and then walk through the crime itself. That means putting the victim and the witnesses back where they were at the scene of the crime, and taking all the steps that lead up to the moment when the crime occurred.

At the end of the recreation, if your theory called for one suspect, there can only be one suspect who contributed to the crime. If there’s more than one suspect at the end of your recreation, you’ve got a problem. If, in the course of recreating the crime you find that some other unexpected entity was involved, you need to go back to the evidence collection step, and start over. You’ve got another suspect out there somewhere that you didn’t know about.

Once you’ve identified all your suspects and you are reasonably sure you can prove they are the cause of the problem, you need to verify that your theory cannot be disproved. Is there any possible way that the crime could have been committed by another suspect? After all, that’s what a defense attorney would claim; the defense is going to do everything in their power to shoot your case full of holes. You want to be absolutely sure that your case is ironclad. As a developer attempting to identify the cause of a defect, you want to make sure that you’ve eliminated all the possible causes of a defect. Is there any other way that this defect might be caused that you haven’t considered? If there are, you need to account for them.

One last caveat: Never make the assumption that the victim is the suspect; conversely, never assume that the product is the suspect. Prove it. Be sure. And make sure that the evidence proves your case. Don’t rely on hunches or speculation. Neither the end-users nor the developers are going to appreciate being accused of being in error. If the suspect is an external entity, the developers are even less likely to be happy, because you’ve just identified extra work for them; be absolutely certain you have the facts to back up your case.

If you cannot disprove your theory, you’re ready to move on and prepare your case.

Prepare Your Case

Document everything. Preserve the evidence. If this case ever presents itself again, you’re going to want to know what you did to research it. A decent defect tracking tool is invaluable in this regard. If you lack one, there’s no reason you can’t keep it in your source code repository (unless there’s a storage limitation on it).

Even if you can’t identify the suspect, and this case is unsolved, you can keep this one in your Cold Case Files. If it rears its ugly head again, you can reopen the case and you’ll have all that evidence from the previous investigation at your disposal.

Obtain a Warrant

Now, with evidence in hand, and a solid provable case, you’re ready to obtain a warrant for the suspect. Up until now, you haven’t had enough to do that. But with the evidence, which doesn’t lie, and a thoroughly documented case, you can make your case to the development team. You’ll have the information to convince them that a defect exists, or it doesn’t.

Carefully lay out the facts, tell them what happened, how you determined that it occurred, and how you eliminated all the other possible suspects. Rely on facts, not conjecture. This is where your personal detachment is critical. You’re not supposed to be on their side or the users’ side. You’re on the truth’s side.

If no defect exists, say so. If one does exist, say so. Don’t make an issue out of it or point fingers. Simply state the facts. Be sure to point out how severe the issue is in terms of data corruption, application downtime, usability and so forth—so long as those pieces of information are based on facts and not opinions. These pieces of information will help the project team decide how quickly the defect should be resolved.

Arrest the Suspect(s)

Once the development team is convinced that the defect is real, the development team will take the information you’ve collected and use it to prioritize and correct the defect. This particular defect should no longer victimize anyone. Once the defect is resolved, the case is closed.

In Closing...

Most developers are constantly burdened with having to research and resolve defects. I know I am. But the problem is that we tend to treat them as annoyances. We don't see them with the weight that they deserve. To us, they're just "something that went wrong" and need to be addressed. So we quickly glance at the code, make a best guess and accuse the first suspect that walks by. All too often, we accuse the wrong suspect. Just as frequently, we take the defect report that’s given to us, add it to the growing list of things to do, and hope to get around to it at a later date when “more pressing concerns” don’t occupy our attention.

Perhaps the problem stems from how we view defects. Perhaps we see them as just another blip on the radar—another defect. I suspect that as developers we tend to think of defects solely in the light of the code base, and rarely in the light of the victim: the end user affected by the defect itself.

But what would happen if we changed our thinking by creating teams that viewed defects as offenses against victims that set out to prosecute or acquit the suspect by collecting the evidence and evaluating the testimony from witnesses? By elevating the perceived seriousness of the defect, perhaps we can increase the desire to get them corrected, and get them corrected correctly the first time. Too farfetched? Corny? Maybe. Maybe not.

Don’t make the mistake of thinking that I’m advocating the creation of a real CSI unit in your software shop or IT department (that would be an absolute disaster and insanity in and of itself). I don’t think you need to treat end-users as hostile witnesses. What I am advocating is the application of the scientific method to the resolution of defects: Know the difference between evidence and testimony and the value of proving your case. It’s likely to be far superior than scratching the first itch that irritates you. Think your way through a defect; get serious about resolving it correctly, deterministically, cost-effectively. There’s nothing in that statement that detracts from code quality. In fact, it enhances it.

I would hope that we all want to write better, more stable software, and that when defects are found, we'd address them quickly and with a minimal amount of cost. Part of minimizing that cost is identifying the right cause of the defect the first time.

What started out as a parody for me led me to rethink my process for defect resolution. There are ways that I can tighten it up, and improve my process. I might get laughed out of the door, but when it comes down to it, the only real thing that matters is whether or not I nailed the real perpetrator, and did so more efficiently than I did before. And isn't that the whole point of this exercise?