Why Should I INVEST in User Stories?

INVEST is a another great acronym from the Computer Sciences industry, Department of Agile (yes, DOA, and I’m well aware – thank you very much); It stands for Independent, Negotiable, Valuable, Estimable, Sized (or Small), and Testable. Those are indeed valuable attributes of User Stories, of any kind of requirements at all, and the acronym itself implies some effort on behalf of the creator of user stories (i.e. the Product Owner), with a potential return on investment.

But that isn’t good enough. A catchy name may be enough for most developers to try something new (seriously, when does a developer ever need a reason to try something new?), but not so for management. To many managers, however, this can sound like Impossible, Not-gonna-deliver, Vague, Everything-is, So-what, and Test-the-whole-goddamn-project!

In this post, inspired by a twitter-conversation I had a while ago, I hope to explain what this INVESTing in user stories really means, and why it is worth doing.

I is for INDEPENDENT

In my opinion, writing independent user stories will be your single greatest return on investment (no pun intended – this time). In one of my recent posts, I described how work items (stories, tasks, features, etc.) are far more likely to complete later rather than earlier if they more than one dependency, while having just a single dependency (i.e. starting the new item is simply dependent on completing the previous one) will  increase the likelihood that it will complete on time.

Of course, most of the work in writing good user stories is in this part as well. You are required to reimagine how you think of your work. You need to do away with layered one-shot thinking that is probably necessary in every industry except software. Since the actual construction of the software is for all intents and purposes near-instantaneous, you can (and should) think about vertical increments of value, when you come up with your requirements. Each such story should be worded so as to be independent of any previous (or forthcoming) stories.

In some cases, you may find it impossible to break a story’s dependencies, no matter how you try. In those cases, try to look at the story with its dependencies as a single story. Find the commonality of them and reimagine it.

To summarize – Independence in your stories will translate directly into increased probability of delivering on time!

N is for NEGOTIABLE

Let’s get one thing clear: This attribute is not a copout for developers! A story’s negotiability does not give developers the right to decide not to do it. The developers do not decide what the story will be, what its scope is, when it will be delivered, and what is required to make it complete. Oh, the developers can and should discuss the story and its impact with the PO, and the PO may decide to change the story based on developer input, but it is ultimately the PO’s right (and responsibility) to make any changes to the story.

The full meaning of negotiability, is that the contents of the story (i.e. scope, priority, requirements) can be changed at any time, from the moment they are introduced, and up until the the developers commit to it (i.e. the beginning of its sprint).

This is perhaps both the easiest attribute to add to your user stories, and the hardest. It is easy, because it requires no extra work from the PO or team. It is hard, because it may require you to let go of any preconceptions you may have of the story (i.e. those you had when you first wrote it). It requires you to allow people to move your cheese.

The return on this investment is quite valuable, however; Your stories will be rejuvenated, and will be more valuable to the customers and user, because they have been recently revalued, evaluated and updated. This attribute is particularly synergetic with the next one:

V is for VALUABLE

You’d think that given the high premium on the time of developers, and the fact that most projects are running late, as it is, it would go without saying that everything we develop is valuable. Unfortunately that is not the case. According to the CHAOS Report, by the Standish Group, fully 64% of developed functionality is rarely or never used!

All of these features have something in common: They were added without any consideration of their value or necessity to the users or the customers!

The valuable attribute of user stories helps reduce these numbers, by making sure that anything and everything the team develops has value to the users. When the PO comes up with a story, and cannot find a way to describe its value, it is possible that there is no value. Our tendency to add these valueless stories to the product comes from the traditional idea that you have to define everything upfront, or it won’t be in the project. Thanks to negotiability, it is possible to focus only on what is valuable.

This attribute also protects the system from developers’ desires to build a framework (we all love developing frameworks – it’s a conditioning or something), or do any work that doesn’t translate into value to the customer. If some technical work is required for some story, add the work as a task to that story, not as a technical story; this will more clearly radiate to management what is required to complete a story, and will give the PO the ability to consider this extra work and its impact when evaluating the story.

E is for ESTIMABLE

There is a lot of good literature out there on how to estimate properly. While there are several, equally viable techniques out there for estimating, everyone pretty much agrees that relative estimates are easier than absolute ones, and smaller items are easier than larger ones. For more info on how to estimate properly, you can read my previous post on the subject, here.

There is more to it, though, than just the techniques. The story should be clearly scoped. There should be a beginning and an end.

Imagine a story such as “As a user, I want a text editor, so that I can take notes”. How much will it take to develop that? I have no idea. If the user only needs a multi-line textbox with persistence capabilities, then I can probably have it done in a few hours (yes – a few hours, because nothing ever takes just 5 minutes!). If the user is expecting RTF capabilities with copy and paste, it’ll take longer, and if he’s looking to give MS Word some serious competition, it’ll probably take a few man-years (just guessing here; It’s too big for me to give a good estimate).

I see estimability as the culmination and grounding of the next two attributes, Size and Testability (see below).

S is for Size

“Sized Appropriately”, or “Small”, as some call it means that the story is small enough to estimate. In Scrum, and other time-boxed methodologies it is expected that the story will be scoped in a way that makes it possible for the team to complete a few of them in one sprint – at the very least, they should be able to complete one story.

This attribute requires the PO to define in detail what it is that he wants the team to deliver. In many cases, the actual user stories that the team works on are the elaboration of the broad scopes defined in the “Epics” or “over-stories” or “super-stories”.

What the PO gains is the ability to release early and often. The team get the ability to create less complex solutions, because the problems are smaller.

Remember, the story should be defined in terms of their business value, not the implementation details! The details are to be left to the developer tasks, and are of no interest to the users or customers.

T is for Testable

Testability basically means that it is possible to test the product to see if it fulfills the requirements of the user story. A testable user story has a finite and measurable set of acceptance criteria, by which it is possible to prove the validity of the product.

This last, but most certainly not least of attributes serves everybody who has a stake in the product:

• By defining the story’s acceptance criteria, the PO has the opportunity to define more specific demands of the story, than what appear simply in the “title” (“As a… I want… so that…).
• Both the Developers and the QA gain an understanding of what the scope of the story is and what it is not, thus reducing a great source of contention about whether something is a bug, or a new feature (if I had a penny…)
• Stakeholders gain an understanding of what they are going to receive by the release.

The acceptance criteria, like the rest of the story, should be worded in the product’s domain language, which means that they should be defined in the users’ terms, rather than the developer / QA terms.

The return on your INVESTment here should be obvious: The PO gains a tighter control of what will be produced, without unnecessarily elaborating on the technical solution, while the developers and QA have a better understanding of what they are required to do.

In conclusion

I think that every part of the INVEST model has value. Stories that you have invested in will be:

• More likely to complete on time
• Current and updated
• Valuable to the customer and / or the users, with less unnecessary technical fluff
• Clearer, with less dissonance between POs, developers and testers
• Less complex to develop, easier to test, and quicker to get feedback as well as quicker to deploy
• Easier to prove and validate and quicker to gain acceptance by the customer / user

That’s all. I think it’s a worthy goal, well worth the cost. What say you?

Technorati Tags: Agile,Scrum,User Story,INVEST,test,estimation

Proving Pair Programming: How and Why it Works

In the first two posts in the Demystifying Agile, I wrote about how to plan release with Scrum and how to use agile estimation techniques in a Scrum project. Delving deeper into the agile software development process, there is one practice that is by far the least popular with traditional managers, and even with many developers. It is the practice of pairing on development tasks. In this post I will attempt to clear the air a bit, explaining why it works and how to make it work for you.

First, a story…

Roughly five years ago I had to rewrite a project from scratch. The details that led to that point are of lesser import, but what is important is that it was a codebase that took three developers eight months to develop and it had four months worth of ugly patches that sorely needed to be refactored. And a few breaking changes in the requirements. And I had 30 days. And two developers (myself included).

Rather than splitting the work, we decided to pair. We paired for 6-7 hours out of eight working hours every day.

The result was more robust, extensible and better performing than before.

And we did it in 20 days.

Why did it work?

From a strictly mathematical standpoint, it shouldn’t have been done. I’ve rewritten projects before, and while it often doesn’t take as much time as it originally did, it doesn’t take a lot less time. We took the work of 24 man-months, and completed it in 40 man-days.

Here are what I believe to be the reasons we achieved the results:

1. With two developers, equally participating on the project, we had immediate code-review done as we develop. It is a (should be) well known fact that the shorter the feedback cycle, the easier it is to act on the feedback. With near-instantaneous feedback, the improved quality is simply automatically baked in.
2. Two developers have two different sets of experiences, bringing in a different point of view. Any iffy idea is immediately examined. You don’t get to to sip your own cool-aid when you pair.
3. With another developer near you all day (or most of the day), you are more focused on your work. You don’t go off on a tangent, reading up on a new technique or gizmo, and you don’t get lost in the quagmire of email: You explicitly decide which distractions to acknowledge and which to postpone. And if you don’t, your pair will help you.
4. It’s much more difficult to cut corners with someone else (your pair) looking over your shoulder; your guilt will stop you, or his will. Any corners cut (or conversely, taking too long to complete a task) will only be the result of an explicit decision by both of you. Explicitness tends to cut down on occurrences.
5. Two-legged-distractions (a.k.a corporate-drive-by incidents, a.k.a. managers and coworkers) tend to bother you less when you’re sitting with someone else. They often go look for someone easier to single out if possible. There’s strength in numbers. If that fails, the fact that you’re already sitting with someone will give you a good valid excuse to ignore the distraction.

As you can see, these observations, my experiences, are not easily measurable. They are, however, no less real.

How to make it work for you

First, forgive me for the following copout: YMMV (Your Mileage May Vary); nothing I can write is guaranteed to work for everybody. But I do expect the following ideas to work for most developers who are willing to give it a shot:

1. Give it a shot. Seriously, it won’t bite. And you can always quit (though winners don’t quit, and quitters don’t win).
2. Don’t do it the whole day, and don’t work a long day. Pairing is very draining because it demands a focus far greater than you are likely to achieve on your own. Work for no more than 8-9 hours a day. Pair for no more than 6-7 of them. Leave an hour or so for email, meetings, easy no-brainer tasks, and other nuisances.
3. Pair with someone that could review your work. Offer to pair with that developer on his task after that.
4. Don’t hog the keyboard. Switch every now and then. Otherwise you have one developer and a bored observer who’s wasting his or her time.
5. Don’t declare it – just do it. Pairing is a hard sell. Results aren’t. Besides, it is usually easier to get forgiveness than it is to get permission.

I hope this post helps you add a trick to your agile bag. It worked wonders for me.

Assaf.

Technorati Tags: Agile,Pair Programming,Scrum

Do Agile Estimation Techniques Really Account for Scrum Projects’ Successes?

After several years of experience working with, training and coaching teams on Scrum, I have no doubt that agile (empirical) estimation techniques are better than classic, predictive estimation techniques. One question that kept nagging me over time is whether or not using story points, T-shirt sizes, or ideal hours truly explains the increased success of Scrum based techniques?

It took me a while, but I finally figured it out. And the answer is a definite ‘NO’!

All Estimation Techniques are Merely Guesses

At the end of the day (or at least, at the end of the planning session), any estimation is just a guess: sometimes its more, and sometimes its less. This means that the distribution of the actual time versus the estimation should be pretty much normal: In large numbers, about half of the actual results will be less than the estimation, and half will be more. On average, according to the Law of Large Numbers, it evens up.

The better the estimation technique is, the steeper the curve will be, i.e. the closer actual results will be to the estimation.

My experience is that agile techniques are in fact better at predicting the actual results, but the fact remains that if you make enough of these predictions, then regardless of the technique, the variance should cancel itself out and the sum should settle on the average!

This means that the quality of the technique used to estimate each task has little or no impact of the success of estimating the time to complete the entire project!

Then Why Do Estimations Fail?

If estimations behave “normally” whole project estimations should not fail. I don’t buy into all that “programmers are notoriously optimistic” sentiment; I’m a developer myself and there is not a single soul alive that will call me optimistic, and yet, non-agile projects I worked on overran their deadlines, like everybody else’s. Something must be out there, that either changes the normal distribution, or invalidates the law of large numbers or both. And it has to be something that is present in classic waterfall-ish projects but not in agile ones.

It Is All About the Independence!

The main difference, in my opinion between agile and non-agile projects is how they are planned and constructed. An agile project plan is basically a stack of user stories, with no* dependencies between them, whereas a waterfall project plan is a Gantt chart describing tasks and their interdependencies.

Why Is This Important?

Let’s assume a task that has two dependencies. Each were estimated, and each have a normal distribution of the actual amount of time to complete vs. the estimated time. The following figure describes such a “plan”:

Each task has a 50% chance to be completed before time and 50% to be completed after (There is of course no statistical chance of being completed “on time”). Now, let’s say we’re interested in calculating the probability of Task C to be completed before time, or not to finish late. For the sake of ease, we’ll assume all three tasks to have the same estimated duration. For Task C to begin early, it needs both of its dependencies to complete early. Since the chance of each to complete early is 50%, the chance for both is (0.5 * 0.5) = 0.25, or 25%. The chance for this not to happen, i.e. for at least one of them to be completed late is 1-0.25 or 75%. this means that there’s a 75% for Task C to complete late, and of course it has its own probability of being late, which means that the chance of Task C completing late (and making the whole project run late) is more than 75%!

It is easy to see that the more complex a project is, the more dependencies each task has the greater the chance of being late is; The distribution of the actual duration of tasks with dependencies is a negatively skewed distribution.

Scrum User Stories Have No Dependencies

With this in mind, Scrum user stories have no* interdependencies. If two stories do have a dependency, they are should be joined to be treated as one (and if they are too big, they can and should be split again, though in a way that doesn’t create dependencies). With no dependency (or, if you wish, a single dependency that is the completion of the prior story), we get a plan that looks like this:

The chance of B starting early, which is the same chance of A ending early, is 50%. The chance of B completing early is still 50%! Even better the variance is reduced, which means that there is an increased probability of the total project duration being close to the the estimate (i.e. a steep probability distribution).

What Does This Mean?

This means, that while a project with multi-dependent tasks has an increased skew towards lateness with each added task, a project with no dependency (beyond sequence) has an increased tendency towards the mean, or the estimate.

This means that by the very nature of what we are estimating, agile projects tend to succeed (i.e. be on budget and on time), than non-agile.

Moreover, the actual estimation technique doesn’t matter – it is the estimation of independent units of work that makes the difference.

Therefore, my conclusion, and advice is that the very first change a team needs to make is to start working on independent features – call them user stories, features, or MMFs – it doesn’t matter; just keep them independent of each other!

Hope this helps,

Assaf.

Technorati Tags: Agile,Estimation,Scrum,Sprint Planning,User Story

How to Write Automated Tests for Console Applications

I don’t know about the rest of you, but I still find myself writing a lot of console applications. Most of the time these are utilities or services or some kind of customization or process automation scripts. Until recently, if I wanted to write automated tests for these, I would have to separate the business logic from the program itself and test the functions. While this is obviously a good practice for any moderately complex project, it sometimes might be overly complicated for what is needed. Moreover, this doesn’t work for black-box user acceptance tests.

The System.Console Class

In the .NET Framework console apps revolve around, well, the Console class. You get user input with Console.Read() (and ReadLine(), and ReadKey(), and so on). Conversely, you send output to the user with Console.Write() and Console.WriteLine().

So let’s take a look at the code inside the .NET Framework. By the way, I used Telerik’s JustDecompile to browse the code. If you don’t already have a decompiler of choice, I suggest you take a look at it. It’s simple, elegant and free (and no, I don’t own stock).

So, here's the code behind Console.ReadLine():

   1: public static string ReadLine()

   2: {

   3:     return Console.In.ReadLine();

   4: }

And here’s Console.WriteLine():

   1: public static void WriteLine()

   2: {

   3:     Console.Out.WriteLine();

   4: }

As it turns out, Console’s read and write methods are merely pass-through methods that call the same methods on the In and Out TextReader and TextWriter properties (again - respectively).

The text reader and writer classes, as so eloquently put in the class descriptions on MSDN, “represent a reader [and writer] that can read [and write] a sequential series of characters”. Well, that still is more helpful than some descriptions I’ve seen... Anyway, they are the abstract parents of the string and stream reader (and writer) classes. Why is this important? Well, the In and Out properties are by default set to the keyboard input and screen output streams.

Now, if only there was a way to redirect those streams…

Enter Console.SetIn() and Console.SetOut() respectively. Hmm, that’s a lot more respect than I usually show in a blog post…

With SetIn, I can redirect the In text-reader to a StringReader, and with SetOut, I redirect the output to a StringWriter.

I can now, quite easily initialize the string reader to a value that contains whichever input, or set of inputs I want to use, and the console application will read characters or lines as needed. I can then read the output of the string writer, and compare it with expected results.

And the most beautiful part of it is that there is (almost) no need to modify the console application in any way for this to work.

Example

The following example was written in C#, using MS-Test (forgive me) to test the code. The application is a simple one that plays the game 7-Boom (a local variation of the game BizzBuzz). As you will see, except for making the Program class and Main() method public in lines 1 and 3, so that I can reach the code from another (test) assembly, I did nothing I wouldn’t do in any other console application:

   1: public class Program

   2: {

   3:     public static void Main()

   4:     {

   5:         Console.Write("Enter upper limit: ");

   6:         var range = int.Parse(Console.ReadLine());

   7:  

   8:         for (var i = 0; i < range; i++)

   9:         {

  10:             var num = ((i % 7 == 0) || i.ToString().Contains('7')) ? "BOOM" : i.ToString();

  11:             Console.Write("{0}, ", num);

  12:         }

  13:     }

  14: }

And here is a test:

   1: // Arrange

   2: using (var sw = new StringWriter())

   3: {

   4:     using (var sr = new StringReader("100"))

   5:     {

   6:         Console.SetOut(sw);

   7:         Console.SetIn(sr);

   8:  

   9:         // Act

  10:         SevenBoom.Program.Main();

  11:  

  12:         // Assert

  13:         var result = sw.ToString();

  14:         Assert.IsFalse(result.Contains('7'));

  15:     }

  16: }

That’s it! I set the input that the user would have entered in line #7, and redirect the I/O in lines 6-7. All that remains is to assert that the output in line #14 meets the expectations.

By the way, this works exactly the same in Java as it does in C#. The difference is that in Java you will use System.setIn() and System.setOut() to set the PrintStreams (the In and Out properties used in Console.Read & Console.Write).

So now you can write test-driven (and behavior driven) console applications with greater ease.

Hope this helps,

Assaf.

Technorati Tags: Agile,BDD,C#,Java,TDD,Test,Unit Tests

10 Signs that Your Team Isn’t Really Agile

[UPDATE – This post was recently translated by Choepidiui (I hope I got the name right) into Korean. For the Korean version of this post, please go to his site]

With all the hype in the software industry about Agile, Scrum and so on, it is no surprise that many companies hear or read about the virtues of these methods, perhaps at a conference or in a technical white paper, and think “we need this here”. Unfortunately, some organizations are not as ready to make the mental shift required to embrace the agile way of doing things…

Below is a (by no means conclusive) list of signs that your team waded in toe-deep, rather than going the whole nine yards:

You might not be truly agile if…

1. Your team is not responsible for the entire story

Perhaps the most important aspect of an agile team is that it is cross functional. This means that the team, as a whole, have all the skills required to deliver a valuable product increment. If a team has only client developers, then all server-side tasks are probably relegated to a different team. In this case, the team is not handling a story from end to end; when they complete their work, no value has been added to the product – not in the eyes of the customer! Teams that work like this resemble a factory where the product sits on the conveyor belt and different layers are added to it by different teams. At no moment till the last team does its thing is the feature (or product) done.

It is possible to have a successful development process without cross-functional (feature) teams, but it simply isn’t agile.

2. Testing is done by another team

Another equally important quality of an agile team is the ability to deliver a working product increment. If the developers call a feature that has merely been coded and not tested “done”, then there is a serious flaw with their definition of done. This is often another example of not being cross-functional: QA is simply another team. In order for a team’s work to be considered done, it should be tested and integrated and ready to use.

3. You are not INVESTing in your user stories

INVEST, is a mnemonic acronym coined by Bill Wake that describes the values of a good user story: Independent of other user stories, Negotiable scope up to the sprint in which they’re developed, Valuable to the end user or customer, Estimable by the development team, Sized appropriately so that the team can complete a few of them in one sprint, and Testable so that the team can prove that they have successfully completed the requirements.

Only if the product owner INVESTs in the stories, will he be able to order them by their value to their customer, and be able to promise that when each is complete, the customer will receive a valuable increment to the product. Likewise, the team will be able to deliver these increments rapidly, and with confidence in their quality.

4. Tasks are assigned to team members by a manager

In a truly agile team, each member can pick which tasks he or she will work on next, as long as it pertains to the next most valuable user story. Traditionally, project managers or team leaders would assign tasks to team members, often months in advance. Even though this is done for the sake of optimizing a project schedule, it is a wasteful process that can’t possibly take into account how the entire project might change over time, nor what the team members’ skills and desires might be at the time.

A team member may want to focus on improving his skills, or try to learn to do something outside his comfort zone. A responsible, self-managing team will find a way to make sure the tasks get done as soon as needed and in line with the reality of the project as it is at the present. The best the manager can truly hope for is to get out of the way. Besides, he has more important things to do.

The only time a project manager should expend the effort of allocating resources is when a task requires a resource that the team doesn’t have. Those should be few and far between. If this is a recurring issue, it is a sign that the team isn’t cross-functional enough (see the first point, above).

5. Your team is told how much work to commit to

If a PM gives the team a certain set of resources (members), a deadline (release or sprint), and then tells them how much work they should commit to (or vetoes their commitment), then the project manager is guilty of attempting to break the iron triangle. A team should be able to commit to as much work per iteration as they believe they can (and should base their estimates on past experience).

If this isn’t the case, then there is nothing agile about the plan; the plan was set in advance, etched in stone, and you aren’t willing to renegotiate it, even at the implicit expense of quality.

For a team to be truly agile, it must be accepted that the plan might be delivered with less than all of the stories originally planned (and it should be the least valuable ones that get booted).

6. You are reporting rather than discussing your progress

A subtle and insidious sign that you aren’t really agile is that you report your progress to the project manager scrum master, not to the team, at the daily stand up meeting. The meeting is there so that the (entire) team will have a sense of what each other is doing, what they are stuck on, and what they are planning to do.

Despite the seemingly managerial nature of the three questions asked (“what did you do since then last meeting?”, “what are you planning on doing till the next one?”, “what might block your progress?”) , the questions are not there for the manager to keep tabs on the team, but to elicit another team member’s thoughts on the matter (“can I help?”, “does this affect my tasks?”, etc.).

Each team member should be addressing the rest of the team. The PM should be there to listen, and perhaps answer questions; he should not become the focus of the meeting.

7. You are not focusing on completing the most important user story

Sometimes, a team may divide their work into stories and break them down into tasks, but instead of swarming on the topmost story and divide the tasks among members, the stories are divided. This has two un-agile consequences:

First, at a given time, all of the stories may be partially completed (regardless of value), instead of part of the stories (based on value), totally complete. This transforms the team and the iteration plan into a waterfall-ish plan, rather than an agile one, as its success is based on the premise (hope) that everything will be completed on time, otherwise, nothing is.

Second problem is that each member’s work has less impact and import to his team members. It is not their story he’s talking about, after all. The other members have less of an incentive to listen and help. The daily meeting becomes a mere progress report that can easily be replaced with a query to the chosen ALM tool.

For a team to be agile, they have to be geared towards working on the same problem, and put all their effort into cranking out complete features, as quickly as possible.

8. You changed to agile last year, and haven’t changed a thing since

Here’s a tricky one – you think you nailed it; you may have brought in an expert way back then, and we worked out a plan, and it seems to work, and now it’s etched in stone and  nobody can change it. Well guess what – you’re missing on an important part of being agile: Actually being agile! Reality changes, new ideas and new influences come up all the time. There’s always something that could be done better.

An agile team holds retrospectives on a regular basis, where they put their heads together to think of what they did right since the last meeting, what they could do better in the next iteration, and what visible actions need to be taken to improve.

These visible actions most often manifest as a change to the work process.

Note: If you are using an IT-based ALM tool to manage your work process (e.g. TFS, Greenhopper, etc., Version One) you should take care to account for the impact of the change on the tool.

Time to Publish

Sometimes there are unforeseeable problems that mean you reach the deadline, and you haven’t completed all of your planned work. If you storm your stories and make sure you deliver one complete story at a time, ordered by priority, you will likely have a good enough product (yes I know I made this point before, but it’s worth repeating…)

So it’s time to post. I know I promised 10 signs in the title, but it turns out that I can’t. At least I came through with the most important ones, fully ready on time, no?

Well, if you don’t feel that way, and you think it is more important to deliver everything, than it is to meet a deadline, then here’s sign #9.

9. You are not ready to release on time with an incomplete scope

If you can’t organize your work in such a way that you’re ready to deliver the most important features, those you’ve completed, at any given moment (especially at the deadline), then your plan isn’t agile. You are likely building your product in layers, like a house, rather than in vertical slices, of product value increment.

An agile product is built one user story (or value increment, or minimal marketable feature – doesn’t matter how you call it) at a time. For more on it, read my post on how to plan agile releases.

Here’s another one:

You can always release the remaining features at a later date, with a minor release increment…

10. You are not getting customer feedback every sprint

At the end of every sprint, an agile team demonstrates the latest increment to the product. The audience may include any and all stakeholders in the project, it definitely includes the product owner / project manager and should, when relevant, include the customer. Some teams use this as some formal milestone to show their progress, and continue with the (original) plan for the next sprint – regardless of the results! This is not agile.

A truly agile team will leverage the demo to elicit comments from the stakeholders about the recent development, to make sure that they are on track, and that if the requirements or environment changed during the last sprint, these changes will be reflected in the product backlog and potentially in the upcoming sprint backlog!

Remember, and agile team embraces change, rather than avoiding it.

Summary

There. That’s 10 signs. I hope you found them enjoyable as well as educating.

So do you have any more signs that should be added to this list? Please add them in the comments.

Assaf.

[EDIT]

11. You’re not agile if you can’t deal with a change in the original scope.

Now, I know I said there’d be (only) 10 signs, but based on early, prerelease feedback, I was asked to add another one. I showed them to a friend of mine before posting, and he liked what he saw. In fact, he liked the idea so much, that he suggested another one that I might add. After the release however, he suggested I change the wording. I hope you like this one better..

Technorati Tags: Agile,Definition of Done,Scrum,User Story

How to Use Agile Estimation in Scrum Projects

Like it or not, estimations are a big part of professional and commercial software development. Estimations are used in most projects regardless of methodology: Waterfall or Scrum, traditional or agile, every stakeholder needs to know the numbers. Projects succeed or fail as a result of these. Unfortunately, humans, as a rule, tend to suck at estimating.

This post will compare and contrast the different estimation techniques and applications used by both agile and non-agile methodologies, and explain how to use them (and not misuse them), particularly in a Scrum project.

What is Wrong with how We Estimate?

Traditional project plans involve using calendric estimates in order to synchronize the development efforts of the team(s) by placing the work on a time line (usually, using a Gantt chart). This means that for every part of the solution, the developers must predict the when it will be ready (i.e. the delivery date). This means they need to know when they will start, how much time it will take to deliver the work, and how much time will be “wasted” on impediments (both foreseeable and unforeseeable). While knowing this makes the project planners’ work easier, obtaining this knowledge is extremely difficult; Estimating the development effort is difficult enough, but predicting the total amount of time that will be spent not developing is near impossible.

What’s worse, is that contrary to popular belief, the Law of Large Numbers does not apply to project plans! Due to the nature of dependencies, a single late dependency is enough to push the delivery date of a dependent component. On the other hand, in order for a component to be delivered early, it is necessary that all dependencies will be completed early. In short, lateness aggregates, while earliness does not.

A side effect of this is that the more complex the Gantt based project is, the more lateness is likely to be accumulated, as the project advances, and the further we are in the project, the more we run the risk of being late.

Of course, a late project means that the software is delivered at a higher cost, with less features, and / or simply later than expected and agreed. This means that a late project often becomes a failed project.

Clearly, a different method of doing things is necessary; Agile methodologists came up with different estimation techniques to reduce the chances and ramifications of failure.

Agile Estimations Are Easier to Make

The three most common agile estimations are T-Shirt Sizes, Ideal Time, and Story Points:

• T-Shirt Size estimations are coarse grained separation of requirements into buckets of small, medium, large, or very-large efforts. This estimate is of course extremely easy to make intuitively, and of course insufficient as a basis for project planning due to its low resolution, but is often sufficient for the purpose of pricing single features (example scenario might be when required to give a customer a cost estimate for some custom feature in the system).
• Ideal Time estimates are similar to calendric estimates in that they involve high resolution predictions of the effort involved in developing a feature. The difference is that ideal estimates are made while ignoring all surprises and interferences! The question that ideal estimates answer is “how much time will it take to develop a solution, assuming you work on nothing else, and nothing else disturbs you”. Taking the unknowable impediment factor out of the equation improves the accuracy of the estimates, but you can’t simply substitute ideal time for calendric time because the unknown impediments still do exist, and the fact remains that humans aren’t good at making absolute estimates, with no point of reference. In Scrum, Ideal Hours are often used to estimate the effort of completing tasks, which should be short, simple and straightforward.
• Story Points are a radically different method of estimating development effort. With story points you do not go out and say how much work something is, in and of itself, but rather state how the effort to develop one component compares to the effort of developing another. These estimates are much easier to make, and surprisingly accurate because we are much better at comparing things than estimating them. Of course, comparisons and relationships have no unit of measure, and thus can’t be used to establish a timeline. At least not in the normal sense. In Scrum, Story Points are usually used to estimate the effort of completing user stories (hence the name).

Agile Estimation Techniques Make Better Plans

While agile estimation techniques tend to end up more accurate than calendric estimates, the estimations are still guesses, and guesses still mean risk. Agile teams try to reduce this risk by reducing the amount of guesswork involved in the project plan.

Classic project plans involve estimating the time it will take to deliver each component, and add up all the estimates to create one big guess-of-a-plan. While agile project plans involve estimating the work on each component as well, that is where the similarity ends.

The agile project plan is not based on the prediction of work effort, but rather on empirical evidence! Only the team’s first iteration (ever) is planned based on a prediction. The rest are based on the measured evidence of previous iterations (or sprints, as they are often called in Scrum-shops). The team estimates that it will successfully complete as much work in the next iteration as it did in the previous iteration(s).

For example, if a team successfully completed 120 ideal hours worth of work (that is, the amount of work they did added up to what they estimated as 120 ideal hours), then they will estimate that they will do the same in the next iteration. This technique works the same with story points: A team that can complete an average of 50 story points in an iteration is likely to repeat that success in the next one. A team’s velocity, or cadence can then be measured in ideal hours per iteration or story points per sprint.

What about interruptions? Well here’s the beauty of the technique: You can consistently account for the predictable ones (e.g. vacations, repeat meetings, scheduled events) and factor those out, and you can consistently ignore the unplanned ones, as they are likely to be more or less an equal part of every iteration of the team.

Where’s the Difference?

The difference between predictive plans (classic, waterfall-ish, Gantt based) and empirical plans (Agile, Scrum) are:

1. Predictive plans are based on more estimation (i.e. guesses, i.e. risk) than empirical ones
2. Empirical plans have an increasingly sound statistical ground as the project progresses
3. Predictive plans have an increasing chance of running late, as the project progresses
4. With predictive plans the unknown factors aggregate, with empirical plans they cancel each other out.

Finally, as stated in the previous post, agile plans, being based on stories that are designed to be discrete units of business-value, with as little interdependencies as possible, agile plans further mitigate the risk of estimation failure by causing the ramification to be failure to deliver on time only the least valuable features!

In Conclusion

To me, the agile choice is a no brainer. Agile plans are simply safer to execute, the chance of failure is lesser, and the result is less harmful. I would even go as far as to say that companies promising to deliver a non-trivial project without using an agile plan of some sorts are being less than honest – with themselves as well as the customer.

Moreover, I think it is high time that customers demand their contractors to use agile methods in order to win contracts.

Technorati Tags: Agile,Estimation,Scrum,Release Planning,Sprint Planning,Story Points,Ideal Hours

Demystifying Agile: How to Plan Releases with Scrum

The agile release process might sometimes seem like voodoo (or even dark voodoo) to some managers and developers, particularly those more accustomed to the traditional way of planning processes used in waterfall projects. This post will explain the inherent problems with waterfall projects, how agile methodologies such as Scrum overcome these problems, and of course, the reasons that the agile release planning method works.

You Can’t Control Everything

The PMBOK call it the project management triangle; the agile community call it the Iron Triangle. Regardless of the name, it is a real and hard constraint: Of the three aspects of the project – scope, resources (or cost), and time you may pick and control any two. The third cannot be controlled without harming the quality, and is calculated (roughly) by the following formula:

This means that if you have a project where the budget is fixed, and there is a hard deadline (i.e. release date), then you cannot control the scope that will be delivered by the team by the time of the release.

Moreover, Brook’s Law, states that adding people to a late (i.e. already started) project will only make it later. According to this, since the cost of a software project is mostly the cost of the team members, the resources must remain fixed for the project (or release).

Nevertheless, traditional project managers seem to try to define the entire scope of a project, define the releases and milestones (i.e. the time), with a fixed preset budget (i.e. cost).

It is simply doomed to fail. According to the Standish Group and the Chaos Report, most projects (68 percent) do.

Why do Projects Fail?

According to Martin Fowler, it is not the projects that fail, but rather the estimates that fail; it is our attempt to calculate how much time it will take to deliver a certain scope with a certain amount of resources.

I would argue, that the failure to estimate is the reason that projects fail. We really don’t know how to estimate. Projects fail because they rely too heavily on the estimates.

How do Agile Methods Reconcile with the Iron Triangle?

Given values such as courage and honesty, it is no surprise that agile methodologists simply acknowledge that estimation is not a good basis for a release plan, and search for alternatives.

Given the Iron Triangle, Brook’s Law, and the fact that most customers demand a hard release date, an agile PM accepts that he cannot control the scope that will be delivered; that is, while the PM may make guesses (i.e. estimates) as to the content that will be delivered on the deadline, he cannot be sure that every last task will be completed.

What he can promise, and make certain of, is that he will deliver the best product possible on the release date, with the rest of the desired features in consecutive releases.

How do Agile Methods Deliver the Best Product on Time?

By slicing the entire scope into vertical slices, small increments in valuable functionality (often called user stories), the agile PM now has a stack of user stories to be released, or a product backlog. By defining the stories in such a way that they have as few interdependencies as possible, the PM may order them as he desires. The PM will of course, choose to order them by business value to the customer.

The development team, in turn, will swarm on each story, by order of value (priority), and focus on completely delivering one story at a time (completely means coding, testing, integrating and anything else required).

In this way, at any given moment, including the release date, the team has a product that is shippable and is the most valuable product they could possibly develop.

Is the Best Product Possible Good Enough?

While the best product might not be everything that was envisioned, because again – that is simply not possible: not with waterfall, not with agile, not anyway – the “Best Product” will have the capabilities most valued by the customer.

The findings of the Standish Group show that only 20% of the features in any given product are frequently used, while 64% are rarely or never used. Therefore, statistically speaking, even if you only complete 36% of your scope, you may still have a good enough product!

Of course, assuming that you aren’t slacking, and the project’s deadline isn’t insanely short, it is more likely that you will complete a significantly larger portion of your scope. It is even possible that you will have a “good enough” product that you can ship even before the deadline! There is not a single customer that will say no to reducing time to market!

So I’d say, yes – the Best Product is good enough.

And that’s how it’s done. I hope

How about your products? Do you work this way? Are your experiences similar? Share!

Thanks,

Assaf.

Technorati Tags: Agile,Estimation,Scrum,User Story,Release Planning,Sprint Planning