SCRUMHockey

For not so long ago I started playing icehockey. I came in a team as a complete rookie. For a mounth ago, I came in a new firm for a project as an external developer in a SCRUM team. In my experience there are many simmilarities how good softeware and icehockey teams work.

The first time I was on ice this year, it was a disaster for me. The guys there were all well trained playing hockey since ever, making a good team; and then I came there with really basic skills and no training at all. It was pretty devastating to be such a nut, however I made some perceptions how the team reacts and what measures they toke to improve the teams performance (and of course mines too).

The team had started the training as usual and I was just dropped into the deep water. They let me suffer for a while but then one of the best player in the team, who also had some coaching experienses, just stood beside me and started to gave me some advices on how to do the excercises better.

The first training match I played, was ad-hoc at all. I was asked which position I liked to play and just tried to keep up with the others (with rather less than more success). After the match the team made some discussions and I was briefed again and got some technical and strategical hints, how to make the most out of my lousy abilities. Of course during the game there was also hints mad by the others, but they were more concentrated on playing the game, and making the most out of it.

So that is how actually we train in the team, and after some trainings I was just wondering how similar it is to the way a good SCRUM team should work.

Actually when I went to a new firm for about a month ago, I have had a very similar experience. I was new to the project, to the team and the whole framework they used was also pretty new for me. So after two days of trainings (some very basic overview of the architecture) and basic project information I was started a sprint with the team. In the first sprint I was the one who got the small issues, mainly bugfixes, to warm up and have some overview over the modules integrated into the project. For the first two issues I was in pair programming with a senior team member, and for the rest it was mainly a question/answer game I played.

In the second sprint I was already working on more complex issues, and now I'm just about implementing additional features for some user stories. By the sprint reviews and retrospectives we discussed about the tooling and the way we should process our issues. We coverd some coding practices and some testing questions, and I was asking some open questions to get a better view of the project.

My conclusion from this story is, that a team is successfull if there is a strong coherence. All the success is about teamplay! In the sport is it most of the time self-evident, but why is it all so many times overseen by software teams? Why do teams still rely on estimates of some 'superhero', and let one senior write specifications and stories all alone? Why do managers beleave, that a team review or a sprint planning meeting with all the team members is waste of time and/or resources and try undermine it, or just pull the majority of the team out of those meetings?

In my eyes the only way to succeed with software projects is to enforce teamwork. And the more risky the project is the more this rule applies!

Detailed software specifications

I've been working by a middle sized software company on a product for the last two years. I've had the luck to work with some really good fellows and learned a lot how to write good code. On the other had, I also learned how to fail.

I don't want to bother you with the whole story, I just want to pick one aspect which I'm concerning as the major cause of failing with a product or project:

BIG REQUIREMENT UP FRONT

It is a fairly big product, with about 30 developers working on (two to four scrum teams). The company spent a lot of efforts on gathering requirement (some man years) resulting in a handful of requirement documents (so about 3-4 kg if you print). Its really impressive and a honorable job writing that much, however, after the requirement phase was closed we have started by creating technical specifications.

It was partially allowed us to write some proof of concept but it was about 6-7 month (with about 3-6 chaps) writing on specifications, attending workshops, negotiating with field experts (not with customers, but with experienced developers with exhaustive knowledge of the business domain).

Since we were intended to be agile (sorry for telling this) we started in the meantime to implement the features/requirements, that was already specified in detail. The implementation was written partially by the same people who wrote the specification, or by someone else.

I had the favor in both writing and implementing specifications, and I must say that I recognized how senseless those specifications are. We have hundreds of thousands of sites of documentation divided in numerous word documents hanging around on the internal network. Each of them was reviewed and released and corrected and reworked, and still after a year there barely exists a chapter which describes what the software really does!

I mean there are three situations I saw by implementing a part of such a detailed specification:

The developer say: "What the hell is this? This will never work!...

1. ... "But who cares, its written by XY analyst and reviewed by this and that committee, so who cares..."

2. ... A new meeting initiated with the product owner and the field experts to clarify the specification. This takes of course about a week to organize, and actually ends up with a whole new specification because the "old" one has a lot of unanswered questions that was forgotten by everybody, and so on...

3. ... Some of the developers discussing on the topic during the development and made some correction to the specification.

I don't want to discuss these options one by one, but the quint essence of each of them is: the detailed specification has never hit the target, and I dare to say, that the reason is, because in such a complex domain it is not possible to specify selected requirements up front! It will always fail for some degree!

Ok I know it seems just like a big complaint, but I wrote this to show you the problems we really faced by this kind of process.

So actually my answer for the question: 


How could we make it better?

is

AGILE MODEL DRIVEN DEVELOPMENT

It is not a kind of magic, just the agile approach to requirement elicitation and is part of the agile release management. The lifecycle for software projects using AMDD is as follows:

Read the original article at www.agilemodeling.com

So without explaining a lot (there are many articles about this, written much better than I could write it), the clear advantage is that there is no such a huge wasted time. The specification and the requirement are just as accurate as you need to start the development.

Of course this does not explicitly mean that no exhaustive specifications are written, but if you really do timeboxing your iterations, you even do not have the time for detailed specifications.

It's really odd for many developers for the first time, but it works. It works even more if you consider creating other specifications like executable test cases, wiki pages, whiteboards, etc... (You can read more about media richness theory and its advantages at modernanalyst.com or even more in detail at www.agilemodeling.com).

Do not forget, the real aim of planning is to get a shared understanding of a specific problem. If your planning process does not increase the shared understanding of the problem, than it fails!

According to my experience the BRUF approach with detailed specifications doesn't really increases the shared understanding of problems.

I do cut this post here, and I would really like to get some feedback from you!

What are your experiences with your specifications?

What kind of specifications do you use?

etc,etc,etc...

Building the right team(s)

The system being produced will tend to have a structure that mirrors the structure of the group that is producing it, whether or not this was intended. One should take advantage of this fact and then deliberately design the group structure so as to achieve the desired system structure. (Conway's Law)

If Conway's law does really apply, one of the most important things you should do is to build the right teams.

The first crucial question is about the size of the team. SCRUM  suggest to build small teams with 5 +/-2 peoples. Amazon.com introduced the "Two pizzas rule" to build their software teams.

One may ask: Why should we build a small team, when a big team has some clear advantages: 

• they can include members with more diverse skills, experiences and approaches
• they are not as much at risk to the loss of a key person
• they can provide more opportunities for individuals to specialize in a technology, or in a subset of the application.

These are really impressive properties of large teams, but what about the small ones? Here are some of the advantages of small teams:

• less social loafing
• more chance of constructive interaction
• nobody is going to fade in the background
• harmful over-specialization is less likely to occur
• less time is spent coordinating effort

Without discussing these properties in detail, just move along to the next problem:

The product/project you need to accomplish has a pretty small time budget, and its business domain is too complex to finish it with a small team. 

The question is how to build small teams that cooperate on building a complex system?

By building the teams keep "Conway's Law" in mind. Either you build feature teams or component teams.

Feature teams are responsible for an end-to-end delivery of working (tested) features, whereas component teams works on some part of the system, such as on a persistency framework, on business layer or may build some GUI framework.

Organizing a multiteam project into feature teams have many advantages. Since the feature team delivers end-to-end functionality, its members working through all the layers of the architecture, which maximizes the learning about the architecture and design of the product. A feature team includes all skills needed to go from an idea to a running, tested feature and so it ensures that these individuals are communicate at a daily basis.

Component teams, are used to deliver software to another team on the project rather than directly to users. Take care that a component team only builds components as a feature team ask for them. Since in this case the feature team is a kind of product owner, they must prioritize and also review the work of the component team. Guessing about future requirements is dangerous, and might lead to develop unnecessary,  or even worth, unusable components.

If you recognize that the team structures impeding the ability to use scrum, that issue should be raised during an end-of-sprint retrospective. You should prefer stable teams over a project, but do not stick on a structure if it does not work! Take care on personal conflicts too. Social issues might reduce the productivity of the team on the short and can lead to quitting of individuals on the long.  

The power of immutability

The final keyword in java has a wide scope of usage. It can be used with:

• classes - public final class Foo {...}
• methods - public final void Bar() {...}
• member variables - private final int a;
• local variables - final int xMin = 2;

In generally the final keyword makes the corresponding class/method/variable immutable.  

For classes and methods immutability means that it can not be specialized respectively overridden, whereas for variables it means that after the initialization, their value can not be changed.

Java does not make any analysis of your classes, so member variables declares as final must be initialized in the constructor of the corresponding class. After the initialization, their value is freeze.

Consider the following snippet:

1:  public class Foo {  
2:    protected final int xMin;  
3:    protected final int xMax = 10;  
4:    
5:    public Foo() {  
6:      xMin = 5;  
7:      xMax = 10; // This is of course a compiler error  
8:    }  
9:  }  
10:    
11:  public class Bar extends Foo {  
12:    public Bar() {  
13:      super():  
14:      xMax = 15; // Compiler error  
15:    }  
16:  }  

I used to use final really intensive in my code, for two reason:

• it reduces the possible states of a class
• it makes multithreading a lot easier

The possible states of a class are reduced, since the amount of its members that might change (put the class in a different state) is reduced. It makes much more easier to understand, but also to test your classes. Further more, it reduces the possibility of failures. I saw an extreme example, how bad it can come. 

If you use final, it is good to know how it really works. Consider the following snippet:

1:  public class Foo {  
2:    private final List<String> names = new ArrayList<String>();  
3:    public void addName(String name) {  
4:      names.add(name); // this gonna work 
5:    }  
6:    public void releaseList() {  
7:      names = null; // compiler error  
8:    }  
9:  }  

For some time ago I received an issue that I had to fix in some module, that I was not familiar with. I tracked down the bug to a particular class, and I recognized that the class has more than 20 member variables and about 15-20 methods on them. In this particular case it is not just an issue of immutability, but also of a poor or even missing design. After some basic refactoring (only aiming to reduce the possible states of the class) about 16 of the member variables was transformed in a final variable. This step made the class much more easier to understand and to manage. 

If you go further it is even meaningful to use immutable variables in a method. These provides some hints on your intentions, and might help in the future if you need change the method or fix bugs.

Now, let's consider the impacts of immutability on multithreading:

One of the most difficult things about writing concurrent programs is deciding how to protect mutable shared state. Java provides a locking primitive, synchronized, but locking primitives are difficult to use. You have to worry about

• data corruption (if you lock too little)
• deadlock (if you lock too much)
• performance degradation (even if you get it right)

Using immutable objects, you don't need to take care about synchronization. Functional languages, such as SCALA do enforce immutability, and are really powerful in multithreaded applications.

My thoughts on assertions in Java

An assertion is a statement in the Java^TM programming language that enables you to test your assumptions about your program. For example, if you write a method that calculates the speed of a particle, you might assert that the calculated speed is less than the speed of light. 


Oracle gives clear statements where assertions should be used, and in which situation you should avoid using them, however, I used to use assertions in a bit different way.

Assertion is not just a good stuff to check invariants, pre- and postconditions in your code, but it helps for your colleagues by working with your code. You can understand assertions as a kind of security cordon.

They tell a lot for someone who need to fix some bugs or implement some additional feature in or to your code. In many cases they are even more useful than comments (ok, I'm clearly not a fan of comments... just se my corresponding post).

Oracle says: Do not use assertions for argument checking in public methods.
You should follow this for your public API. You really should. But I decided to use assertions in my library internal classes for public methods too. In my team we use assertions in a pretty intensive manner, even in public methods, and for arguments checking, and the helped us a lot. I prefer them over exceptions. You can read in an old post of Joel Spolsky about exceptions:

"... In fact they are significantly worse than goto's:
They are invisible in the source code.
They create too many possible exit points for a function.
..."


Consider the following code snippet:

public class CommandExecutor<T extends Target> {
    private final T targetOfExecution;
    
    public CommandExecutor(T targetOfExecution) {
        assert targetOfExecution != null;

        this.targetOfExecution = targetOfExecution;
        ...
    }

   ...

    public void execute(Command<T> c) {
        // prepare target for command execution
        // do not need to check the targetOfExecution

       // I prefer to get some assertion during development and tests
       // instead of waiting for a runtime np-ex
       assert c != null;

      c.execute( targetOfExecution );

     // do some post operations
    }

}

Assuming that this class is internal to your library, and not part of the private API, the assertion in the constructor will do a favor for you.

Instead of exceptions the assertions can be turned off, so you can combine them with return values to improve the quality of your source.

Further more combining assertion with immutability, the execute method will be more readable, since you can save some null checks.

I would be happy to read your thought on this topic.

Why will your project fail without a PO

How many times have you already heard form your PO about a feature: oh, that is cool, but it's not the way I thought it.
If you hear this statement frequently you should be keen on some improvements to your processes, but if you hear it once in a sprint (by the review) you are definitely in big trouble.

In this case the following questions should come up during the retrospective:

- Do our stories have clearly defined DoD?
If the stories don't have DoD or those are not well defined it is likely to fail the real target of the story. If you have questions on some of the DoD you must request the PO for negotiation. Don't be afraid of discussing on remove or add some criteria to the list, since the DoD should be negotiable anyway. Keep in mind, that detailed discussion during the sprint planing meeting might cover up some details and even led to split up stories (see some patterns on splitting stories).

- Why did our PO not complain about it during the sprint?
It's important to negotiate the stories with your PO. If you/the team has questions about the story or feels uncertain about some DoD the PO must be available for you to clarify the problematic issues. If you have the feeling that the PO is not available for the team, or the team must often wait for a meeting with the PO, it is clearly a big impediment, and your scrum master must take care about it.

After all it is likely to have recent questions about the stories, and the product owner must available for the team.
A good, committed team will depend more on the PO than on the scrum master, whereas new teams are likely need more the scrum master and less the PO.

Simple product backlog

There are a lot of complex tools to manage product backlog, and do a lot more. You probably know Greenhopper for JIRA, or IcedScrum just to mention two of them. 

These tools are good, but a bit heavy weighted if you are looking for a simple possibility to manage your product backlog for a small or middle sized project.

For my purposes, I've made a simple excel sheet (or better say a Libre calc sheet), that I'd like to share with you.

It has some additional features like:

• Progress estimation (sprint estimation) based on recorder or assumed velocity
• Velocity chart
• Story burn-down chart

I'm really interested on any feedback or on suggestions for improvement. I do upload an excel version too, but I can't test it, so if you gonna be sure, just use the ods file.

ProductBacklog.xls
ProductBacklog.ods

We can't finish our stories! Help

You have a great team, that works hard and uses the best IDE of the market and the chaps do talk to each other; and still you they can not manage to finish the stories they have in the sprint backlog.

Seems to be a well known problem? If so, just try to answer the following questions:

- Do you have a detailed sprint planning?
You should have! And for a 4 week sprint, it should be at least 4 hours. Just an hour is not enough for that.

- Does your product owner prepares user stories for the team, with some estimation beforehand?
If she does, please take care that the team not sees the estimations of the PO. They would just influence the own estimation and might lead to untruthful commitments.


- Is the product owner really available for the team, through the whole sprint?
Take care about it. The team might have questions during the development. If they need to interrupt frequently while the product owner is not available, the story will possibly fail the sprint. Or even worth, if they develop something that is not accepted by the product owner (afterwards notabene).


- What is the average size of the user stories that are selected for the sprint?
If you feel that stories are two expansive, than consider splitting them. There are some good techniques that might help you. By splitting of stories, consider fulfilling the INVEST criterions. Another good article on this topic can be found here.


- Do you have an exhaustive DoD for your stories?
If the answer is no, than you have an insufficient sprint planning! It is of immense importance to have a well defined exhaustive DoD for each story. It helps to negotiate it with the PO. Even more important is the discussion about the DoD. Many questions and potential problems can be covered up by discussing the list.


- Is your team really committed? What is the commitment of the sprint?
It is helpful to have an overall goal for each sprint. A commitment such as: "We are going to finish all the stories selected for the sprint" is pretty much senseless. You should have something like: "With this sprint the user should be able to create and manage his profile" or it can be some nonfunctional requirement that has some impact on the user experience: "With this sprint we are going to improve the responsiveness of the GUI application." Print it onto the Story board to keep it in mind all the way long. Time to time, it can be helpful to ask the team, how they feel about the commitment.

Comments on code comments

Some easy topic for saturday evening!

Please do not comment your code!
Or do it less than you do now!

I know you have learned that comments are good, and yes, I was very happy to see lots of comments in an assembly code, but please, do not comment each line of your java code.

Consider the following snippet:

1  /**
2   * This method calculates some value.
3   * @param stockId stock identifier
4   * @return integer value
5   */
6   private int calculateSomeStockValue(int stockId) {
7       // the initial value of the algorithm
8       final int initialValue = 10;
9
10      EntityManager em = null;
11      try {
12          // create an entity manager to access the database
13          em = this.managerFactory.createEntityManager();
14          assert em != null;
15
16          // read the stock from the database
17          final Stock stock = em.readStockValue(stockId);
18
19          // if no stock found return the initial value
20          if ( stock == null )
21                return initialValue;
22
23         // retrieve the average price of the stock
24        final double stockPrice = stock.getCurrentPrice();
25
26         return stockPrice;
27      } finally {
28             // close the entity manager if not null
29             if ( em != null )
30                 em.close();
31      }
32 }


Just be honest, what additional information gives you those comments? I mean in spite of the fact that they make the code unreadable, and they are probably even false (see line 23,24 - is it a bug in the source, or just an old comment)!


Do not comment your code, or if you comment it consider writing useful information! Rather then use speaking variable and method names. Don't be afraid of long names. Your code will more readable through them.


Less comment is more!

More on hierarchical scrum teams

Software development can make so much fun! One of the big idea behind scrum, is to enforce a face to face communication in the team. To get the power of the community, through the whole development lifecycle. For me it was one of the main arguments to start working with scrum. A really good team works much more efficient, produces much better code and of course software quality too. And by the way, it does make much more fun to work this way.

However, i find that the team should not be hierarchically organized. Every developer in the team should be equal (not the way our old Orwell meant it: "ALL ANIMALS ARE EQUAL, BUT SOME ANIMALS ARE MORE EQUAL THAN OTHERS." ;) we all know how the story ended up!). Of course there are seniors and more advanced members, but they will be forming the team, and other team members will respect them without any hierarchy.

The power of scrum is that it enforces the individual, and at the same time reinforces teams. It gives one the feeling of freedom. The developer is not a resource any more that can be assigned to one or another team, for 10% of a week and just do some job for the manager that got him from his boss. His performance can not even measured. The smallest unit is the TEAM! At least concerning controlling!

For the old fashioned manager, who manages his resources and got his weekly reports about the progress on this and that project, this seems to be a pretty inconvenient situation, if suddenly, that particular excel rubric can not be rewritten, or moved to another project, or even can not be evaluated/measured on its own.

Who cares, which developer had made it. It was the team! And may be one or another developer is not so good, but we all have our strengths and weaknesses. And you can be sure, if a team keeps somebody, than he/she's worth to be kept.

However, scrum does not mean the loss of control. If you look at it, it gives much more control for the management, since it delivers real value! It's not about working hours spent on this or that, it's just about functionality, you've got integrated into the project. It's about transparency, you got by the product backlog and by the story board. It's about risk mitigation you can get by the short iterations. The main problem with scrum is that it is different! It is not the old school, and chaps of the old school need some time to change (could ask Galileo he knows it well! ). I see pretty the same situation with scrum. You can prove it by doing. You should just start dong it! But the old school won't start with it, since we all know: the Earth is the center of the Universe.

Another big advantage of scrum comes with effort estimation. Planning is a hard stuff. As the old good Churchill said once: "It is difficult to make prediction, especially about the future".

So, what to do (the ultimate  recipe of planning):

1) Get an experienced developer, and let him/her be a leader or manager. You can be sure, that with the time she got away from the code.

2) Let him the one who is authorized to make effort estimation (give him a star ;) )

3) Discuss the requirements and wait for the magic number: 42

4) Ok. if you have a good project manager she would still multiply it by two! Just to be sure ;)

It sound so cool, since we know, the chap who made the prediction was a really good software developer... but forget it. It fails in most of the cases completely. The chap with the star will communicate less and less with the poor developers! He is better now. He is authorized! The effort estimation is his responsibility, so he doesn't need to consult with the developers! He might do it, but it won't be enough, and it gonna be even less and less. I'm pretty sure about it (unfortunately).

But wait, just call your team together for a game. Just meet a room for a day and play poker with them! The development team has the most adequate information to make a good effort estimation. If they are not influenced by some lead or boss or manager or whatsoever, and are supported by a qualified scrum master and product owner, the result of the planning will be impressive!

In my team we have made many mistakes, and did not apply the process the way scrum suggests, but still we achieved much better results on effort estimation by introducing user stories, and start using story points.

But again. You can call it ideal day, or story point, or potato, it doesn't matter what the unit of estimation is. The real matter is, how to get to the result! 

"The plan is nothing but planning is everything" 

development leads in scrum !?

Just assume that we decided to became agile and managing our development process in accordance to scrum. So we gonna have some product owner (PO), scrum master. Beside the development team we also have software architects involved in the product development.

The scrum master must help the team to improve, to be more affective. He tries to identify and to remove impediments, arguing with the PO, keeps the management off the team (if its necessary to do so), and so forth.

Anyway, I wondering whether we need an extra role have introduced to scrum: development lead.

Or may be the scrum master is a kind of development lead?

In the referred page its well defined that "The development lead is the mid-point in the path between being a developer and being the solutions architect." 

and further 

"The development lead is responsible for fleshing out any of the details in the architecture that the Software Architect didn't spell out in detail and for the creation of program specifications from which the developers work."

I thought, as naive as I am, that in a scrum team the team is mixed up, with some senior developers some newbies some field experts, and discuss intensive on User Stories during the planning and the development, under the instruction of the scrum master (who should/could(?) also help the team to understand the architectural aspects resp. constraints). On particular questions the architect(s) should be available. 

What is a development lead good for in such a process/ organization? May be we could just kick the scrum master and establish a development lead instead. Are we then still agile?

Further in the article: "The development lead spends a great deal of time managing the development process." But if we claim using scrum, is this not the role for a scrum master?

The last question for now: is it actually a matter how we label a particular role in the development team? Makes it our development process/SCRUM better or worse if we don't have scrum master(s), but development lead(s)?

Who cares!?

Kick off

Welcome stranger,

at this irrelevant, tiny point of the Internet's universe. At this place I'm going to start to jabber over my daily desperation in the maze of software development. I'm currently pottering with product development and have some years of experiment in this field.

At my company we are suffering under an over managed over controlled, vast and incalculable development process, with some sick project/organizational hierarchy, that merely blocks the chaps pretty much. At this point I might exaggerate a bit, however you can judge it for yourself by following my blog.

I'm really interested onSCRUM and agile development methods, and unfortunately I can tell a lot about how NOT to do it. :(

I hope that with stories of mine I can keep you away of doing the same mistake, and may be some of you can give some clue on how to make it right/better. ;)

On the other hand I'm gonna post some design considerations, and technical questions that are worth of thinking on. And of corse some anti-paterns, to see how we should write code if we would like to became a manager. :)

As you might already recognized, I'm not a native english speaker, however I think my literarily underdeveloped, or I could simply say BAD english will still suffice for this task.

So I'll cut this boring, useless piece of byte-flow now

cu