Detailed software specifications

I've been working by a middle sized software company on a product for the last two years. I've had the luck to work with some really good fellows and learned a lot how to write good code. On the other had, I also learned how to fail.

I don't want to bother you with the whole story, I just want to pick one aspect which I'm concerning as the major cause of failing with a product or project:

BIG REQUIREMENT UP FRONT

It is a fairly big product, with about 30 developers working on (two to four scrum teams). The company spent a lot of efforts on gathering requirement (some man years) resulting in a handful of requirement documents (so about 3-4 kg if you print). Its really impressive and a honorable job writing that much, however, after the requirement phase was closed we have started by creating technical specifications.

It was partially allowed us to write some proof of concept but it was about 6-7 month (with about 3-6 chaps) writing on specifications, attending workshops, negotiating with field experts (not with customers, but with experienced developers with exhaustive knowledge of the business domain).

Since we were intended to be agile (sorry for telling this) we started in the meantime to implement the features/requirements, that was already specified in detail. The implementation was written partially by the same people who wrote the specification, or by someone else.

I had the favor in both writing and implementing specifications, and I must say that I recognized how senseless those specifications are. We have hundreds of thousands of sites of documentation divided in numerous word documents hanging around on the internal network. Each of them was reviewed and released and corrected and reworked, and still after a year there barely exists a chapter which describes what the software really does!

I mean there are three situations I saw by implementing a part of such a detailed specification:

The developer say: "What the hell is this? This will never work!...

1. ... "But who cares, its written by XY analyst and reviewed by this and that committee, so who cares..."

2. ... A new meeting initiated with the product owner and the field experts to clarify the specification. This takes of course about a week to organize, and actually ends up with a whole new specification because the "old" one has a lot of unanswered questions that was forgotten by everybody, and so on...

3. ... Some of the developers discussing on the topic during the development and made some correction to the specification.

I don't want to discuss these options one by one, but the quint essence of each of them is: the detailed specification has never hit the target, and I dare to say, that the reason is, because in such a complex domain it is not possible to specify selected requirements up front! It will always fail for some degree!

Ok I know it seems just like a big complaint, but I wrote this to show you the problems we really faced by this kind of process.

So actually my answer for the question: 


How could we make it better?

is

AGILE MODEL DRIVEN DEVELOPMENT

It is not a kind of magic, just the agile approach to requirement elicitation and is part of the agile release management. The lifecycle for software projects using AMDD is as follows:

Read the original article at www.agilemodeling.com

So without explaining a lot (there are many articles about this, written much better than I could write it), the clear advantage is that there is no such a huge wasted time. The specification and the requirement are just as accurate as you need to start the development.

Of course this does not explicitly mean that no exhaustive specifications are written, but if you really do timeboxing your iterations, you even do not have the time for detailed specifications.

It's really odd for many developers for the first time, but it works. It works even more if you consider creating other specifications like executable test cases, wiki pages, whiteboards, etc... (You can read more about media richness theory and its advantages at modernanalyst.com or even more in detail at www.agilemodeling.com).

Do not forget, the real aim of planning is to get a shared understanding of a specific problem. If your planning process does not increase the shared understanding of the problem, than it fails!

According to my experience the BRUF approach with detailed specifications doesn't really increases the shared understanding of problems.

I do cut this post here, and I would really like to get some feedback from you!

What are your experiences with your specifications?

What kind of specifications do you use?

etc,etc,etc...