Wednesday, July 18, 2007

Why Does Software Cost So Much?

So my sister-in-law recently discovered that the computer a "friend" built for her has an illegal copy of Windows XP installed on it. (Big surprise, that.) When she asked me what she should do about that, I told her, quite simply, that she'd need to get a legal copy of Windows. When I told her what it cost, she was flabbergasted. She looked at me with disgust on her face and demanded, "I don't get it! Why is software so expensive?"

I looked her dead in the eye and said, "Do you really want me to explain it to you?"

Now, she knows that I write software for a living, and she still, in her naiveté, had the temerity to ask that question of me. She's well aware of how constantly stressed out and tired I am from the work I do. So she backed up a second, swallowed, and said, "No, I guess not."

In hindsight, however, I think she posed a fair question. I hear it alot. It's probably a fair statement that most folks have no real grasp of why software costs as much as it does. So I will try to point out some of the reasons why I believe that it is. These are my opinions, mind you; your mileage may vary.

Software is expensive for one very important reason: the people who make it have to live, pay their bills, and feed their families. Beyond all other reasons, that's probably why it's so darned expensive. It's a job, just like any other. And, like any other field, it's not just a simple task: it's a very complex task that requires lots of people, lots of resources, and lots of time.

(There seems to be this preconception out there that writing software is easy because using it is easy. It's just a game, right? How hard can it be?)

I can't speak for the open-source community who do everything over the Web, and develop software on a volunteer basis. But then again, this article doesn't necessarily apply to them. But certain things must be in place in order to develop software in a corporate environment:

  • You have to have somewhere to do it. If you're a freelancer, working out of your house, you have rent or a mortgage to pay. On the other hand, if you're a business, it means you need a lease.
  • For businesses with a building, you need insurance, so that you're covered in case your employees are injured on your premises. You need flood and fire insurance as well.
  • You have to pay utility bills (heating, cooling, electricity, water, etc.). 
  • You have to pay your local, state and federal taxes.
  • You have to have office equipment. Depending on how many people are involved, you may need LOTS of it: desks, chairs, bookshelves, computers, printers, fax machines, extra toner, phones, paper, notebooks, file folders, filing cabinets, trash cans, pens, and all that other jazz. You'll probably need whiteboards for collaboration. If you don't think that stuff is expensive, start browsing your local office supply store web sites. You'll be in for a big surprise.
  • You have to employees. Employees get paid. Depending on how complex your product is, it can take anywhere from a handful to hundreds of employees to develop your product.
  • You have to provide your employees with benefits (medical, dental, 401k, workmen's compensation, etc.), unless they're contractors. If they're contractors, they have to cover their own benefits which means they get paid more than W-2 employees.
  • You have to have lawyers to ensure that your intellectual property is protected from copyright infringement.
  • You have to spend money and invest time to develop a plan for what the product will do. This is often a long, arduous process that involves a lot of arguing, shouting, frayed nerves, and confusion.
  • You have to spend money and time creating mock-ups of what it will look like. This mockup goes through several revisions as people heatedly debate why they hate it, what they want changed, and what a crappy job you did in the first place. They'll want to know why you didn't understand that by "circle" meant an ellipse and that by "square" they meant a beveled rectangle with a shadow in the lower right-hand corner. They'll send you back to the drawing board. Repeatedly. What one user demands another will despise. No one will ever be completely satisfied.
  • You have to spend money to finally start developing the actual software. You'll do this with a severe shortage of time, money, and hands, and the developers will want to kill the management. They'll be unhappy because the specs will likely be hastily put together, the time will be insufficient, and they'll be grumbling about the likelihood of "crunch time scheduling" that requires overtime that they won't be getting paid for.
  • You have to spend LOTS of money testing the solution once it's put together. This is the hardest part of the whole thing. You will have made the mistake of thinking that it works. It doesn't. It never does. Just when you think you know what you're doing, a user is right there to prove you wrong. The system will have to be ripped apart, rewritten, retested, over and over and over again. And all of that takes time. An adversarial relationship will develop between the testers and the developers. Finger-pointing will rule the day. Some people will quit, costing you money as you have to replace them unexpectedly and retrain them. The schedule will slip, costing you further. And as long as you're doing that, you're paying for rent, utilities, taxes, payroll, benefits, office supplies, insurance, and everything else that goes with it.
  • You have to spend money to develop a marketing campaign for the product.
  • You have to spend money to package the product (put it on disk, print the materials that go inside the box, put it in the box, shrink wrap it).
  • You have to spend money to ship it to the retailers who will carry it, or negotiate deals with online resellers who will carry it for you.
  • You have to employ a customer support staff after the product ships. (Benefits, payroll, etc.)
  • You have to pay to support the customers when they call to complain about the product after it ships, or when they can't figure out how to install it or find the icon to start it up or find the CD ROM drive on their computer. You have to deal with irate customers who think it's your fault that the software has too many features when they're the ones that demanded them. You have to deal with customers who are angry and seem to think that it's okay to scream at you because they made a mistake and spilled soda on their keyboard, or the dog yanked the laptop onto the floor, or they didn't pay their cable bill and they want to know why your Web-based software doesn't work anymore. And you have to smile through it all while the bills keep piling up.

These are just a few of the reasons that software is so expensive.

And let's bear in mind one very important fact: writing software is not like writing an essay, or building a house, or fixing your car. Writing software is damned hard to do. Hell, I seriously suspect that disarming bombs is easier.

From where I sit, the computer is just a stupid box armed with the vocabulary of a three year old. It will do exactly what you tell it to do, very quickly, and very efficiently. If you tell it to do the right thing, it will do it very quickly, and very efficiently. Tell it to do the wrong thing (like, erase all files in the My Documents folder) and it will do it very quickly, and very efficiently. This is why I get irritated when someone says, "Oh, the computer hiccupped!" or "The computer screwed up." No, the computer did not. The computer did exactly what some programmer told it to do.

You have to teach a computer everything. Consider a simple task that we take for granted, such as "Transfer $100 from my savings account into my checking account." In its simplest state, a computer is completely incapable of doing that. It doesn't know what a transfer is, what an account is, what savings is, or what checking is. It doesn't know what a dollar is, or a bank is. It has to be taught everything. Software is about teaching that dumbass box with a 3-year-old's vocabulary how to do extremely complex tasks without making any mistakes.

In short, a computer programmer is an individual who sits down at a computer every day, and figures out how to handhold a mechanical idiot through tasks simple and complex, mastering the task of tutoring that idiot through those tasks until the idiot becomes a savant. It's grueling, frustrating, and infuriating. It takes time, and it's expensive. Depending on how big the task is, it might take lots of programmers to teach the idiot how to do the task.

So there you have it. That's why I think software is expensive.

Coincidentally, I also think it's why I and a lot of my peers in this field are psychotic (or very close to it).

4 comments:

Anonymous said...

I too write software for a living and I disagree with you on most of the things.

Most of the points that you have put down apply to most of the jobs. Investment, resources, planning, time, effort, complexity, arguing with others - all of them. So, these are not justifications to why software is so expensive.

"And let's bear in mind one very important fact: writing software is not like writing an essay, or building a house, or fixing your car. Writing software is damned hard to do. Hell, I seriously suspect that disarming bombs is easier."

Oh... hold it a bit. That's so not true. Let's not go as far as disarming bombs, but let's look at our neighbor - hardware. Have you seen how boards are designed and made? Let's not look at how complex that process is - they have to get it right at the first shot! Can you even imagine that? With all the code changes you make and the compiling, unit testing and system testing... you can validate your piece of code all along. For a minute, think how you will design a board - you can't 'compile', you can't 'test', you _have_ to get it right the first time. I am sure your heart is beating faster. Oh BTW, have you ever built a house? Don't lose your credibility by making vague remarks like this.

Have you seen those programs on Discovery on building a bridge over the highway without disturbing the highway (you think it's possible? It is, they did it), making a tunnel for the supersonic train etc.? If you haven't, you should. It will change your opinion about software complexity.

Mike Hofer said...

@anaamica,

You'll have to forgive me for the use of the use of the literary element of hyperbole. I sometimes do that intentionally to get people to see my point, but others think I'm actually serious. That's the problem with print (or cyber-print). You can't see my face when I'm writing it.

However, no matter how you slice it, writing software is damned hard. Can you imagine your parents doing it? Your siblings? Your cousins? I know mine couldn't. Programming is inherently difficult because it requires the developer to pay exacting attention to a monumental number of details at the same time, and the software has to be right.

It's also made hard by the simple expedient of the political nature of getting it done, and the social interactions that must take place in order to get software completed. There's an astronomical amount of pushing and pulling that must take place, and the developer is typically caught square in the middle. Its an unfavorable position to be caught in. With enough pushing an pulling, a project becomes a crushing force that destroys good talent, and drives it out of the industry. I've seen it happen. In 20 years, I've seen some great talent leave the industry due to badly managed projects that became nothing but a black hole.

Hardware, on the other hand, is typically built on an assembly line.

wyldwolf said...

I agree with the first comment that you can ignore pretty much anything having to do with "running an office" on your list - every single business has these. You can also toss in most "compilers (or whatever) are expensive" - every single skilled job has expensive tools. If you think I'm kidding go price out a plasma cutter or pipe bender.

The real reason why software is expensive is because it has not become a commodity yet. This is very common in skilled jobs. Look at car repair (since I picked it for the tools example...), they can easily charge 80-100 $ a hour. Which for the record is more than many developers make.

Just explain to your sister-in-law that she not buying a pair of knock-off jeans from some sweat shop in china and leave it at that.

Or ask here why she pays her accountant, or repairman (car, A/C, whatever) so much.

None of which means that software development is not hard, but that is not WHY it is expensive.

At least we aren't being sued for malpractice (yet).

Kevin
http://technogeek.org/

Anonymous said...

"However, no matter how you slice it, writing software is damned hard. Can you imagine your parents doing it? Your siblings? Your cousins? I know mine couldn't."

That doesn't mean writing software is difficult. It just means that they are not trained for it. If they go through the same grind of four years of computer programming, I am sure they will make decent programmers. And that's what I am - an average programmer - I don't consider myself very good at programming.

Can you imagine making an idol, making a wooden chair, repairing your television, writing a sonnet? Does that mean all these are complex? You are not trained for it, that's it.

"It's also made hard by the simple expedient of the political nature of getting it done, and the social interactions that must take place in order to get software completed. There's an astronomical amount of pushing and pulling that must take place, and the developer is typically caught square in the middle. Its an unfavorable position to be caught in. With enough pushing an pulling, a project becomes a crushing force that destroys good talent, and drives it out of the industry. I've seen it happen. In 20 years, I've seen some great talent leave the industry due to badly managed projects that became nothing but a black hole."

See, this is true for any job. Peek into a newspaper office. You will see more politics there than in any other industry. Teaching... have you looked at that? Why do you think there are not many good teachers/professors/lecturers in the field these days? Same reason... bad management.

"Hardware, on the other hand, is typically built on an assembly line."

I don't think you got what I was pointing at. Hardware that is built on assembly line is what goes out for production. I am talking about those prototype hardware, specifically boards, which are newly designed. If the boards have market value, they are then made on an assembly line. When the hardware team is working on the board design and the software team is working on the board design. Software guys have an open play ground, where they can experiment and keep making changes to their code. Hardware people are not that fortunate - once they decide on their design and order for a board, they keep their fingers crossed. Either the board works, or it doesn't. They don't get to compile, they don't get to press F5, they don't get to press CTRL+Z.