Software and I

Do Agile Estimation Techniques Really Account for Scrum Projects’ Successes?

2012-02-19T13:13:00.001+02:00

After several years of experience working with, training and coaching teams on Scrum, I have no doubt that agile (empirical) estimation techniques are better than classic, predictive estimation techniques. One question that kept nagging me over time is whether or not using story points, T-shirt sizes, or ideal hours truly explains the increased success of Scrum based techniques?

It took me a while, but I finally figured it out. And the answer is a definite ‘NO’!

All Estimation Techniques are Merely Guesses

At the end of the day (or at least, at the end of the planning session), any estimation is just a guess: sometimes its more, and sometimes its less. This means that the distribution of the actual time versus the estimation should be pretty much normal: In large numbers, about half of the actual results will be less than the estimation, and half will be more. On average, according to the Law of Large Numbers, it evens up.

The better the estimation technique is, the steeper the curve will be, i.e. the closer actual results will be to the estimation.

My experience is that agile techniques are in fact better at predicting the actual results, but the fact remains that if you make enough of these predictions, then regardless of the technique, the variance should cancel itself out and the sum should settle on the average!

This means that the quality of the technique used to estimate each task has little or no impact of the success of estimating the time to complete the entire project!

Then Why Do Estimations Fail?

If estimations behave “normally” whole project estimations should not fail. I don’t buy into all that “programmers are notoriously optimistic” sentiment; I’m a developer myself and there is not a single soul alive that will call me optimistic, and yet, non-agile projects I worked on overran their deadlines, like everybody else’s. Something must be out there, that either changes the normal distribution, or invalidates the law of large numbers or both. And it has to be something that is present in classic waterfall-ish projects but not in agile ones.

It Is All About the Independence!

The main difference, in my opinion between agile and non-agile projects is how they are planned and constructed. An agile project plan is basically a stack of user stories, with no* dependencies between them, whereas a waterfall project plan is a Gantt chart describing tasks and their interdependencies.

Why Is This Important?

Let’s assume a task that has two dependencies. Each were estimated, and each have a normal distribution of the actual amount of time to complete vs. the estimated time. The following figure describes such a “plan”:

Each task has a 50% chance to be completed before time and 50% to be completed after (There is of course no statistical chance of being completed “on time”). Now, let’s say we’re interested in calculating the probability of Task C to be completed before time, or not to finish late. For the sake of ease, we’ll assume all three tasks to have the same estimated duration. For Task C to begin early, it needs both of its dependencies to complete early. Since the chance of each to complete early is 50%, the chance for both is (0.5 * 0.5) = 0.25, or 25%. The chance for this not to happen, i.e. for at least one of them to be completed late is 1-0.25 or 75%. this means that there’s a 75% for Task C to complete late, and of course it has its own probability of being late, which means that the chance of Task C completing late (and making the whole project run late) is more than 75%!

It is easy to see that the more complex a project is, the more dependencies each task has the greater the chance of being late is; The distribution of the actual duration of tasks with dependencies is a negatively skewed distribution.

Scrum User Stories Have No Dependencies

With this in mind, Scrum user stories have no* interdependencies. If two stories do have a dependency, they are should be joined to be treated as one (and if they are too big, they can and should be split again, though in a way that doesn’t create dependencies). With no dependency (or, if you wish, a single dependency that is the completion of the prior story), we get a plan that looks like this:

The chance of B starting early, which is the same chance of A ending early, is 50%. The chance of B completing early is still 50%! Even better the variance is reduced, which means that there is an increased probability of the total project duration being close to the the estimate (i.e. a steep probability distribution).

What Does This Mean?

This means, that while a project with multi-dependent tasks has an increased skew towards lateness with each added task, a project with no dependency (beyond sequence) has an increased tendency towards the mean, or the estimate.

This means that by the very nature of what we are estimating, agile projects tend to succeed (i.e. be on budget and on time), than non-agile.

Moreover, the actual estimation technique doesn’t matter – it is the estimation of independent units of work that makes the difference.

Therefore, my conclusion, and advice is that the very first change a team needs to make is to start working on independent features – call them user stories, features, or MMFs – it doesn’t matter; just keep them independent of each other!

Hope this helps,

Assaf.

Technorati Tags: Agile,Estimation,Scrum,Sprint Planning,User Story

How to Write Automated Tests for Console Applications

2012-02-06T09:53:00.000+02:00

I don’t know about the rest of you, but I still find myself writing a lot of console applications. Most of the time these are utilities or services or some kind of customization or process automation scripts. Until recently, if I wanted to write automated tests for these, I would have to separate the business logic from the program itself and test the functions. While this is obviously a good practice for any moderately complex project, it sometimes might be overly complicated for what is needed. Moreover, this doesn’t work for black-box user acceptance tests.

The System.Console Class

In the .NET Framework console apps revolve around, well, the Console class. You get user input with Console.Read() (and ReadLine(), and ReadKey(), and so on). Conversely, you send output to the user with Console.Write() and Console.WriteLine().

So let’s take a look at the code inside the .NET Framework. By the way, I used Telerik’s JustDecompile to browse the code. If you don’t already have a decompiler of choice, I suggest you take a look at it. It’s simple, elegant and free (and no, I don’t own stock).

So, here's the code behind Console.ReadLine():

   1: public static string ReadLine()

   2: {

   3:     return Console.In.ReadLine();

   4: }

And here’s Console.WriteLine():

   1: public static void WriteLine()

   2: {

   3:     Console.Out.WriteLine();

   4: }

As it turns out, Console’s read and write methods are merely pass-through methods that call the same methods on the In and Out TextReader and TextWriter properties (again - respectively).

The text reader and writer classes, as so eloquently put in the class descriptions on MSDN, “represent a reader [and writer] that can read [and write] a sequential series of characters”. Well, that still is more helpful than some descriptions I’ve seen... Anyway, they are the abstract parents of the string and stream reader (and writer) classes. Why is this important? Well, the In and Out properties are by default set to the keyboard input and screen output streams.

Now, if only there was a way to redirect those streams…

Enter Console.SetIn() and Console.SetOut() respectively. Hmm, that’s a lot more respect than I usually show in a blog post…

With SetIn, I can redirect the In text-reader to a StringReader, and with SetOut, I redirect the output to a StringWriter.

I can now, quite easily initialize the string reader to a value that contains whichever input, or set of inputs I want to use, and the console application will read characters or lines as needed. I can then read the output of the string writer, and compare it with expected results.

And the most beautiful part of it is that there is (almost) no need to modify the console application in any way for this to work.

Example

The following example was written in C#, using MS-Test (forgive me) to test the code. The application is a simple one that plays the game 7-Boom (a local variation of the game BizzBuzz). As you will see, except for making the Program class and Main() method public in lines 1 and 3, so that I can reach the code from another (test) assembly, I did nothing I wouldn’t do in any other console application:

   1: public class Program

   2: {

   3:     public static void Main()

   4:     {

   5:         Console.Write("Enter upper limit: ");

   6:         var range = int.Parse(Console.ReadLine());

7:

   8:         for (var i = 0; i < range; i++)

   9:         {

  10:             var num = ((i % 7 == 0) || i.ToString().Contains('7')) ? "BOOM" : i.ToString();

  11:             Console.Write("{0}, ", num);

  12:         }

  13:     }

  14: }

And here is a test:

   1: // Arrange

   2: using (var sw = new StringWriter())

   3: {

   4:     using (var sr = new StringReader("100"))

   5:     {

   6:         Console.SetOut(sw);

   7:         Console.SetIn(sr);

8:

   9:         // Act

  10:         SevenBoom.Program.Main();

11:

  12:         // Assert

  13:         var result = sw.ToString();

  14:         Assert.IsFalse(result.Contains('7'));

  15:     }

  16: }

That’s it! I set the input that the user would have entered in line #7, and redirect the I/O in lines 9-10. All that remains is to assert that the output in line #16 meets the expectations.

By the way, this works exactly the same in Java as it does in C#. The difference is that in Java you will use System.setIn() and System.setOut() to set the PrintStreams (the In and Out properties used in Console.Read & Console.Write).

So now you can write test-driven (and behavior driven) console applications with greater ease.

Hope this helps,

Assaf.

Technorati Tags: Agile,BDD,C#,Java,TDD,Test,Unit Tests

10 Signs that Your Team Isn’t Really Agile

2012-01-29T13:11:00.001+02:00

[UPDATE – This post was recently translated by Choepidiui (I hope I got the name right) into Korean. For the Korean version of this post, please go to his site]

With all the hype in the software industry about Agile, Scrum and so on, it is no surprise that many companies hear or read about the virtues of these methods, perhaps at a conference or in a technical white paper, and think “we need this here”. Unfortunately, some organizations are not as ready to make the mental shift required to embrace the agile way of doing things…

Below is a (by no means conclusive) list of signs that your team waded in toe-deep, rather than going the whole nine yards:

You might not be truly agile if…

1. Your team is not responsible for the entire story

Perhaps the most important aspect of an agile team is that it is cross functional. This means that the team, as a whole, have all the skills required to deliver a valuable product increment. If a team has only client developers, then all server-side tasks are probably relegated to a different team. In this case, the team is not handling a story from end to end; when they complete their work, no value has been added to the product – not in the eyes of the customer! Teams that work like this resemble a factory where the product sits on the conveyor belt and different layers are added to it by different teams. At no moment till the last team does its thing is the feature (or product) done.

It is possible to have a successful development process without cross-functional (feature) teams, but it simply isn’t agile.

2. Testing is done by another team

Another equally important quality of an agile team is the ability to deliver a working product increment. If the developers call a feature that has merely been coded and not tested “done”, then there is a serious flaw with their definition of done. This is often another example of not being cross-functional: QA is simply another team. In order for a team’s work to be considered done, it should be tested and integrated and ready to use.

3. You are not INVESTing in your user stories

INVEST, is a mnemonic acronym coined by Bill Wake that describes the values of a good user story: Independent of other user stories, Negotiable scope up to the sprint in which they’re developed, Valuable to the end user or customer, Estimable by the development team, Sized appropriately so that the team can complete a few of them in one sprint, and Testable so that the team can prove that they have successfully completed the requirements.

Only if the product owner INVESTs in the stories, will he be able to order them by their value to their customer, and be able to promise that when each is complete, the customer will receive a valuable increment to the product. Likewise, the team will be able to deliver these increments rapidly, and with confidence in their quality.

4. Tasks are assigned to team members by a manager

In a truly agile team, each member can pick which tasks he or she will work on next, as long as it pertains to the next most valuable user story. Traditionally, project managers or team leaders would assign tasks to team members, often months in advance. Even though this is done for the sake of optimizing a project schedule, it is a wasteful process that can’t possibly take into account how the entire project might change over time, nor what the team members’ skills and desires might be at the time.

A team member may want to focus on improving his skills, or try to learn to do something outside his comfort zone. A responsible, self-managing team will find a way to make sure the tasks get done as soon as needed and in line with the reality of the project as it is at the present. The best the manager can truly hope for is to get out of the way. Besides, he has more important things to do.

The only time a project manager should expend the effort of allocating resources is when a task requires a resource that the team doesn’t have. Those should be few and far between. If this is a recurring issue, it is a sign that the team isn’t cross-functional enough (see the first point, above).

5. Your team is told how much work to commit to

If a PM gives the team a certain set of resources (members), a deadline (release or sprint), and then tells them how much work they should commit to (or vetoes their commitment), then the project manager is guilty of attempting to break the iron triangle. A team should be able to commit to as much work per iteration as they believe they can (and should base their estimates on past experience).

If this isn’t the case, then there is nothing agile about the plan; the plan was set in advance, etched in stone, and you aren’t willing to renegotiate it, even at the implicit expense of quality.

For a team to be truly agile, it must be accepted that the plan might be delivered with less than all of the stories originally planned (and it should be the least valuable ones that get booted).

6. You are reporting rather than discussing your progress

A subtle and insidious sign that you aren’t really agile is that you report your progress to the ~~project manager~~ scrum master, not to the team, at the daily stand up meeting. The meeting is there so that the (entire) team will have a sense of what each other is doing, what they are stuck on, and what they are planning to do.

Despite the seemingly managerial nature of the three questions asked (“what did you do since then last meeting?”, “what are you planning on doing till the next one?”, “what might block your progress?”) , the questions are not there for the manager to keep tabs on the team, but to elicit another team member’s thoughts on the matter (“can I help?”, “does this affect my tasks?”, etc.).

Each team member should be addressing the rest of the team. The PM should be there to listen, and perhaps answer questions; he should not become the focus of the meeting.

7. You are not focusing on completing the most important user story

Sometimes, a team may divide their work into stories and break them down into tasks, but instead of swarming on the topmost story and divide the tasks among members, the stories are divided. This has two un-agile consequences:

First, at a given time, all of the stories may be partially completed (regardless of value), instead of part of the stories (based on value), totally complete. This transforms the team and the iteration plan into a waterfall-ish plan, rather than an agile one, as its success is based on the premise (hope) that everything will be completed on time, otherwise, nothing is.

Second problem is that each member’s work has less impact and import to his team members. It is not their story he’s talking about, after all. The other members have less of an incentive to listen and help. The daily meeting becomes a mere progress report that can easily be replaced with a query to the chosen ALM tool.

For a team to be agile, they have to be geared towards working on the same problem, and put all their effort into cranking out complete features, as quickly as possible.

8. You changed to agile last year, and haven’t changed a thing since

Here’s a tricky one – you think you nailed it; you may have brought in an expert way back then, and we worked out a plan, and it seems to work, and now it’s etched in stone and nobody can change it. Well guess what – you’re missing on an important part of being agile: Actually being agile! Reality changes, new ideas and new influences come up all the time. There’s always something that could be done better.

An agile team holds retrospectives on a regular basis, where they put their heads together to think of what they did right since the last meeting, what they could do better in the next iteration, and what visible actions need to be taken to improve.

These visible actions most often manifest as a change to the work process.

Note: If you are using an IT-based ALM tool to manage your work process (e.g. TFS, Greenhopper, etc., Version One) you should take care to account for the impact of the change on the tool.

Time to Publish

Sometimes there are unforeseeable problems that mean you reach the deadline, and you haven’t completed all of your planned work. If you storm your stories and make sure you deliver one complete story at a time, ordered by priority, you will likely have a good enough product (yes I know I made this point before, but it’s worth repeating…)

So it’s time to post. I know I promised 10 signs in the title, but it turns out that I can’t. At least I came through with the most important ones, fully ready on time, no?

Well, if you don’t feel that way, and you think it is more important to deliver everything, than it is to meet a deadline, then here’s sign #9.

9. You are not ready to release on time with an incomplete scope

If you can’t organize your work in such a way that you’re ready to deliver the most important features, those you’ve completed, at any given moment (especially at the deadline), then your plan isn’t agile. You are likely building your product in layers, like a house, rather than in vertical slices, of product value increment.

An agile product is built one user story (or value increment, or minimal marketable feature – doesn’t matter how you call it) at a time. For more on it, read my post on how to plan agile releases.

Here’s another one:

You can always release the remaining features at a later date, with a minor release increment…

10. You are not getting customer feedback every sprint

At the end of every sprint, an agile team demonstrates the latest increment to the product. The audience may include any and all stakeholders in the project, it definitely includes the product owner / project manager and should, when relevant, include the customer. Some teams use this as some formal milestone to show their progress, and continue with the (original) plan for the next sprint – regardless of the results! This is not agile.

A truly agile team will leverage the demo to elicit comments from the stakeholders about the recent development, to make sure that they are on track, and that if the requirements or environment changed during the last sprint, these changes will be reflected in the product backlog and potentially in the upcoming sprint backlog!

Remember, and agile team embraces change, rather than avoiding it.

Summary

There. That’s 10 signs. I hope you found them enjoyable as well as educating.

So do you have any more signs that should be added to this list? Please add them in the comments.

Assaf.

[EDIT]

11. You’re not agile if you can’t deal with a change in the original scope.

Now, I know I said there’d be (only) 10 signs, but based on early, prerelease feedback, I was asked to add another one. I showed them to a friend of mine before posting, and he liked what he saw. In fact, he liked the idea so much, that he suggested another one that I might add. After the release however, he suggested I change the wording. I hope you like this one better..

Technorati Tags: Agile,Definition of Done,Scrum,User Story

How to Use Agile Estimation in Scrum Projects

2012-01-22T16:38:00.001+02:00

Like it or not, estimations are a big part of professional and commercial software development. Estimations are used in most projects regardless of methodology: Waterfall or Scrum, traditional or agile, every stakeholder needs to know the numbers. Projects succeed or fail as a result of these. Unfortunately, humans, as a rule, tend to suck at estimating.

This post will compare and contrast the different estimation techniques and applications used by both agile and non-agile methodologies, and explain how to use them (and not misuse them), particularly in a Scrum project.

What is Wrong with how We Estimate?

Traditional project plans involve using calendric estimates in order to synchronize the development efforts of the team(s) by placing the work on a time line (usually, using a Gantt chart). This means that for every part of the solution, the developers must predict the when it will be ready (i.e. the delivery date). This means they need to know when they will start, how much time it will take to deliver the work, and how much time will be “wasted” on impediments (both foreseeable and unforeseeable). While knowing this makes the project planners’ work easier, obtaining this knowledge is extremely difficult; Estimating the development effort is difficult enough, but predicting the total amount of time that will be spent not developing is near impossible.

What’s worse, is that contrary to popular belief, the Law of Large Numbers does not apply to project plans! Due to the nature of dependencies, a single late dependency is enough to push the delivery date of a dependent component. On the other hand, in order for a component to be delivered early, it is necessary that all dependencies will be completed early. In short, lateness aggregates, while earliness does not.

A side effect of this is that the more complex the Gantt based project is, the more lateness is likely to be accumulated, as the project advances, and the further we are in the project, the more we run the risk of being late.

Of course, a late project means that the software is delivered at a higher cost, with less features, and / or simply later than expected and agreed. This means that a late project often becomes a failed project.

Clearly, a different method of doing things is necessary; Agile methodologists came up with different estimation techniques to reduce the chances and ramifications of failure.

Agile Estimations Are Easier to Make

The three most common agile estimations are T-Shirt Sizes, Ideal Time, and Story Points:

T-Shirt Size estimations are coarse grained separation of requirements into buckets of small, medium, large, or very-large efforts. This estimate is of course extremely easy to make intuitively, and of course insufficient as a basis for project planning due to its low resolution, but is often sufficient for the purpose of pricing single features (example scenario might be when required to give a customer a cost estimate for some custom feature in the system).
Ideal Time estimates are similar to calendric estimates in that they involve high resolution predictions of the effort involved in developing a feature. The difference is that ideal estimates are made while ignoring all surprises and interferences! The question that ideal estimates answer is “how much time will it take to develop a solution, assuming you work on nothing else, and nothing else disturbs you”. Taking the unknowable impediment factor out of the equation improves the accuracy of the estimates, but you can’t simply substitute ideal time for calendric time because the unknown impediments still do exist, and the fact remains that humans aren’t good at making absolute estimates, with no point of reference. In Scrum, Ideal Hours are often used to estimate the effort of completing tasks, which should be short, simple and straightforward.
Story Points are a radically different method of estimating development effort. With story points you do not go out and say how much work something is, in and of itself, but rather state how the effort to develop one component compares to the effort of developing another. These estimates are much easier to make, and surprisingly accurate because we are much better at comparing things than estimating them. Of course, comparisons and relationships have no unit of measure, and thus can’t be used to establish a timeline. At least not in the normal sense. In Scrum, Story Points are usually used to estimate the effort of completing user stories (hence the name).

Agile Estimation Techniques Make Better Plans

While agile estimation techniques tend to end up more accurate than calendric estimates, the estimations are still guesses, and guesses still mean risk. Agile teams try to reduce this risk by reducing the amount of guesswork involved in the project plan.

Classic project plans involve estimating the time it will take to deliver each component, and add up all the estimates to create one big guess-of-a-plan. While agile project plans involve estimating the work on each component as well, that is where the similarity ends.

The agile project plan is not based on the prediction of work effort, but rather on empirical evidence! Only the team’s first iteration (ever) is planned based on a prediction. The rest are based on the measured evidence of previous iterations (or sprints, as they are often called in Scrum-shops). The team estimates that it will successfully complete as much work in the next iteration as it did in the previous iteration(s).

For example, if a team successfully completed 120 ideal hours worth of work (that is, the amount of work they did added up to what they estimated as 120 ideal hours), then they will estimate that they will do the same in the next iteration. This technique works the same with story points: A team that can complete an average of 50 story points in an iteration is likely to repeat that success in the next one. A team’s velocity, or cadence can then be measured in ideal hours per iteration or story points per sprint.

What about interruptions? Well here’s the beauty of the technique: You can consistently account for the predictable ones (e.g. vacations, repeat meetings, scheduled events) and factor those out, and you can consistently ignore the unplanned ones, as they are likely to be more or less an equal part of every iteration of the team.

Where’s the Difference?

The difference between predictive plans (classic, waterfall-ish, Gantt based) and empirical plans (Agile, Scrum) are:

Predictive plans are based on more estimation (i.e. guesses, i.e. risk) than empirical ones
Empirical plans have an increasingly sound statistical ground as the project progresses
Predictive plans have an increasing chance of running late, as the project progresses
With predictive plans the unknown factors aggregate, with empirical plans they cancel each other out.

Finally, as stated in the previous post, agile plans, being based on stories that are designed to be discrete units of business-value, with as little interdependencies as possible, agile plans further mitigate the risk of estimation failure by causing the ramification to be failure to deliver on time only the least valuable features!

In Conclusion

To me, the agile choice is a no brainer. Agile plans are simply safer to execute, the chance of failure is lesser, and the result is less harmful. I would even go as far as to say that companies promising to deliver a non-trivial project without using an agile plan of some sorts are being less than honest – with themselves as well as the customer.

Moreover, I think it is high time that customers demand their contractors to use agile methods in order to win contracts.

Technorati Tags: Agile,Estimation,Scrum,Release Planning,Sprint Planning,Story Points,Ideal Hours

Demystifying Agile: How to Plan Releases with Scrum

2012-01-05T18:36:00.001+02:00

The agile release process might sometimes seem like voodoo (or even dark voodoo) to some managers and developers, particularly those more accustomed to the traditional way of planning processes used in waterfall projects. This post will explain the inherent problems with waterfall projects, how agile methodologies such as Scrum overcome these problems, and of course, the reasons that the agile release planning method works.

You Can’t Control Everything

The PMBOK call it the project management triangle; the agile community call it the Iron Triangle. Regardless of the name, it is a real and hard constraint: Of the three aspects of the project – scope, resources (or cost), and time you may pick and control any two. The third cannot be controlled without harming the quality, and is calculated (roughly) by the following formula:

This means that if you have a project where the budget is fixed, and there is a hard deadline (i.e. release date), then you cannot control the scope that will be delivered by the team by the time of the release.

Moreover, Brook’s Law, states that adding people to a late (i.e. already started) project will only make it later. According to this, since the cost of a software project is mostly the cost of the team members, the resources must remain fixed for the project (or release).

Nevertheless, traditional project managers seem to try to define the entire scope of a project, define the releases and milestones (i.e. the time), with a fixed preset budget (i.e. cost).

It is simply doomed to fail. According to the Standish Group and the Chaos Report, most projects (68 percent) do.

Why do Projects Fail?

According to Martin Fowler, it is not the projects that fail, but rather the estimates that fail; it is our attempt to calculate how much time it will take to deliver a certain scope with a certain amount of resources.

I would argue, that the failure to estimate is the reason that projects fail. We really don’t know how to estimate. Projects fail because they rely too heavily on the estimates.

How do Agile Methods Reconcile with the Iron Triangle?

Given values such as courage and honesty, it is no surprise that agile methodologists simply acknowledge that estimation is not a good basis for a release plan, and search for alternatives.

Given the Iron Triangle, Brook’s Law, and the fact that most customers demand a hard release date, an agile PM accepts that he cannot control the scope that will be delivered; that is, while the PM may make guesses (i.e. estimates) as to the content that will be delivered on the deadline, he cannot be sure that every last task will be completed.

What he can promise, and make certain of, is that he will deliver the best product possible on the release date, with the rest of the desired features in consecutive releases.

How do Agile Methods Deliver the Best Product on Time?

By slicing the entire scope into vertical slices, small increments in valuable functionality (often called user stories), the agile PM now has a stack of user stories to be released, or a product backlog. By defining the stories in such a way that they have as few interdependencies as possible, the PM may order them as he desires. The PM will of course, choose to order them by business value to the customer.

The development team, in turn, will swarm on each story, by order of value (priority), and focus on completely delivering one story at a time (completely means coding, testing, integrating and anything else required).

In this way, at any given moment, including the release date, the team has a product that is shippable and is the most valuable product they could possibly develop.

Is the Best Product Possible Good Enough?

While the best product might not be everything that was envisioned, because again – that is simply not possible: not with waterfall, not with agile, not anyway – the “Best Product” will have the capabilities most valued by the customer.

The findings of the Standish Group show that only 20% of the features in any given product are frequently used, while 64% are rarely or never used. Therefore, statistically speaking, even if you only complete 36% of your scope, you may still have a good enough product!

Of course, assuming that you aren’t slacking, and the project’s deadline isn’t insanely short, it is more likely that you will complete a significantly larger portion of your scope. It is even possible that you will have a “good enough” product that you can ship even before the deadline! There is not a single customer that will say no to reducing time to market!

So I’d say, yes – the Best Product is good enough.

And that’s how it’s done. I hope

How about your products? Do you work this way? Are your experiences similar? Share!

Thanks,

Assaf.

Technorati Tags: Agile,Estimation,Scrum,User Story,Release Planning,Sprint Planning

Demystifying Agile

2012-01-05T18:35:00.001+02:00

Understanding the mechanics of Scrum is easy; otherwise there’s no way they could be taught in 16 hours, in a Certified Scrum Master class. Understanding why the mechanics work is a different thing altogether. In this series of posts, I will attempt to explain how and why they work the way they do; I firmly believe that attempting to use the mechanics of scrum without understanding the Agile spirit is a recipe for failure. I hope you enjoy the series.

Table of Contents:

Demystifying Agile: How to Plan Releases with Scrum
How to Use Agile Estimation in Scrum Projects
Proving Pair Programming: How and Why it Works
How to Manage a Self Managing Team
Agile Contracts: Unveiling the Trick to Happy Customers
Agile Tips & Tricks: How to Solve Real Issues with Scrum

Bookmark this page! More posts to come…

Technorati Tags: Agile,Scrum

A Software Developer’s New Year’s Resolution

2011-12-25T11:50:00.001+02:00

Or: 5 Things You Can Do Today to Improve Your Software Product

New Year’s eve is around the corner. It is the end of this year, and there’s a whole new year full of potential to be better, just waiting to begin. So how can you make it better? If you, like me, are a professional developer, then you can improve your life (and your boss’s and your team’s) by reducing software delivery related stress. Here are 5 things you can do, effective immediately, to make your product better:

1. Make a Version Management Plan

If your version control system is a mess, the first thing you should do is straighten it out. If you are not using one, then you should begin doing so immediately. There really is no excuse not to do so. For starters, there are plenty of high quality tools out there that are open source and free (git, mercurial, and subversion are the most common ones), as well as other commercial tools (I happen to use and consult on the use of Microsoft’s TFS). Either download and start using a free one, or get one that offers a trial period.

If you are using a VCS, but it’s a mess, and you find yourself mired deeply in merge-hell every time you want to deliver your software (internally or externally; it doesn’t matter), then you should straighten it out. Unfortunately, if you have a considerably sized source repository, this may seem as an impossible endeavor. Here’s one way to alleviate at least some of the pains of software delivery, in a few easy steps:

Create a new root-level folder named sources. This folder will contain your new source code repository.
Create a branch from your current repository, and call it main (e.g. /sources/main). This will represent your latest and greatest version, and should be kept as stable as possible, at any moment. No development will be done on it; you will only merge completed changes into it.
Create a branch from main, named dev (e.g. /sources/dev). This branch will be the branch that developers work on to create new features, checking code in and out from there. If there are multiple independent teams, you may have multiple dev branches.
Create a releases folder, which will contain release branches from main. Every time you wish to release your software (internally or externally), create a release branch from main (e.g. /sources/releases/1.2).

This is just one possible branching plan. It works particularly well for agile development, where effort is made towards developing as many features as possible before a release, merging them into main when ever they are done, and branching when the iteration is over. Note the following division of work:

New features are developed only in dev. They are merged back into main when completed.
Fixes are developed on release branches and reverse-integrated into main, and then back to dev, so that they won’t get lost.

This structure allows you to develop new software and stabilize a release at the same time without one effort polluting another. It is also a step towards being able to release a new version anytime.

If your development and release mechanism is well established, but not suited to fit this branch plan, make one that does work. Just make sure to be able to support feature and fix separation as well as the ability release anytime you need.

2. Automate Your Build

Even if all you do is copy sources from the repository to some drop folder, you have some sort of a build process. Automate it. Even just a part of it. Even just the compilation phase. Every step you let a computer do rather than yourself, is one less step to worry about. If you aren’t already using an automated build tool, you should start now. There are many open source tools as well as commercial ones. Most are powerful enough to suit your needs. Remember – If you can run it from command line, you can automate it as well.

Every time you run the build, you will not only create a deployment package of your system, but you will get a good indication about the health of your product (if the product doesn’t build, you will want to know sooner rather than later). Additionally, you will have the start of a platform for testing the system.

For a synergetic benefit with your versioning plan, schedule the following automated builds:

Build the software every time you check in changes to your code (on dev or a release branch); most build and VCS tools support this – it is called continuous integration.
Build the software every time you wish to merge dev to main. It is even better if you require a successful build as a prerequisite for the merge. This is called a gated check-in in many tools.
Schedule a nightly build on main. This will let you know what the state of your system is. Fix any problems here first. Keeping a healthy build will allow you to deliver software at the drop of a hat. Amaze your PM!

3. Add a Health Report

Even a simple email message to the team in case a build fails will greatly improve your ability to fix problems with your software. Shorter feedback cycles means that you can fix problems when their causes are still fresh in your mind! Let your build system email you if it fails. Almost every system supports it. Conversely, you may wish to post successful build reports to some company wiki. Let everyone know of your successes.

4. Add an Automated Test

While the benefits of TDD as a design mechanism are far greater than just having a suite of tests, we should not belittle the benefit of an automated regression test. Even a simple record & replay automated smoke test, with a black-box test tool will greatly improve your system. Start with a single test that runs through as many parts of your system as you can easily manage. (Note – this might mean you need to add a deployment action to your automated build). Report if it fails. Run it every time you build. The more tests you add, the greater confidence you’ll have in your build system! Maintain your tests; fix any tests that fail due to legitimate changes and remove those that are no longer relevant. You don’t want your system to cry wolf!

5. No Broken Windows!

The most important thing you can do is to make a new year’s resolution that you will not suffer any problem with your software delivery lifecycle to remain unfixed! Once you fall off the wagon, your system will start to decay back to uncontrollable chaos. Think of maintaining your ALM like brushing your teeth. You may not like it but you do it every day (more than once), or your teeth will rot out until ultimately you won’t be able to eat!

In conclusion,

My new year’s resolution is to vigilantly keep my teams’ products “on the wagon”.What is yours?

Happy Holidays, and Happy New Year!

See you all next year,

Assaf.

Technorati Tags: Agile,Build,TDD,Test,Report,Version Control

How to Measure Technical Debt

2011-12-10T09:02:00.001+02:00

Or: An Executive’s Guide to the Cost of Software Architecture Shortcuts

In my experience, in any software project of sufficient complexity, it is necessary to cut corners in the software architecture. These shortcuts are decision that you make now so that you will be able to release your next version of the software that much earlier; these shortcuts are the result of not having enough time to do things “right”.

This is called your project’s Technical Debt.

Like financial debt, which is the metaphor used to explain it, technical debt happens when you “buy” more software features than you have resources (developers and time) for. Like financial debt, it accrues interest over time. And like financial debt, if you don’t pay it back, you may eventually get foreclosed and go bankrupt. That last part is not a metaphor. Your software project simply ends up costing more to maintain than the value you derive from it, and either the project or the company gets shut down, or both.

There are essentially two ways to deal with debt (technical, like financial):

You pay it all at once – This means that you do not “withdraw” any more features, until your debt has been fully paid. It has the benefit of being able to make a fresh start. It has the disadvantage of risking “starvation” while you try to pay the “bank”.
You pay it in installments – This means that you pay off your debt, a small part at a time, still “withdrawing” features, but less than you normally would, cutting down to essentials, for as long as you are still paying off the debt. It takes longer to finish, but you can still “eat” while paying.

Of course there’s the third option of continuing to withdraw as before, until you go broke, or luckily win the ‘lottery” (investment, buy-out, etc.)

There is no one clear way to deal with debt (financial or technical). It will depend on the situation, and the players. And most of all it will depend on the amount of debt.

Quantifying the Debt

With financial debt, it is easy to determine how much you owe. There’s the amount of money you borrowed, and the interest rate. Both are usually well known, and it is just a matter of keeping track of it, to make sure you can withstand it.

With technical debt, you still have the amount you borrowed, and the interest rate, but you usually don’t know the numbers.

Let’s start with figuring out the derived value of taking the loan. Whenever you demand that a feature or set of features be completed faster than it would normally take if the team were to maintain an appropriate quality level, you are “borrowing” from the architecture. Figure out how much time you saved, and multiply it by the hourly cost of the developers. That is the amount of money you borrowed from the architecture. Figure out what the reduction in the Time to Market will earn you, and that is the derived value of the loan.

The interest rate is not so easily calculated a-priori, but it can be estimated (with about as much accuracy as the development cost estimates your team makes – which may be good or bad; depends on your experience):

Map out the areas impacted by the shortcut
Estimate how much it would cost (developers x time) to develop a feature in the impacted areas, before the shortcut(s)
Now, estimate the cost to develop the same feature after introducing the technical debt

The difference in the cost is the interest you are paying. Keep track of the interest as it will change with every change you make to the system. A good practice is to calculate the interest every iteration or sprint. You should consider doing this as part of the sprint planning. This will have the added benefit of giving the product owner the opportunity to pay off some of the debt.

Note that like financial debt, the interest grows over time. The rate varies, and it is difficult to guess what it will be, except by projecting based on past experience. As you begin to accumulate this data, you will be able to make educated decisions on when you can afford to go into technical debt, and when you should avoid it. You should reassess your technical debt’s interest rate every sprint planning.

Armed with this knowledge, developers should now be able to collaborate with management in order to decide how to develop and whether or not it is okay to cut a corner on that next feature.

Finally, remember: Your architecture is a cold blooded loan shark, who will just as soon take your installments, as he will bust your project’s kneecaps over a missed payment.

What do you think? Do you have a preferred method of calculating / estimating technical debt, or a way to handle it?

Please share!

Technorati Tags: Agile,Estimation,Scrum,Sprint Planning,Technical Debt

10 Reasons to Avoid Test Driven Development

2011-12-07T18:29:00.001+02:00

Test Driven Development (TDD), and all of its derivatives (BDD, ATDD) are, in my opinion great methods to drive a team’s development efforts, and raise the quality of the product. But TDD is not a silver bullet. It doesn’t fit every project. The following post lists the top ten reasons not to write automated tests for your code.

If one or more of these conditions apply to you, you should consider forgoing TDD, and in fact any agile techniques, as they will be a waste of effort:

10. There is no client

Sometimes you are developing a product that will not be used by anyone. In this case, any effort expended on raising the quality is a complete waste. Nobody will care.

9. The client is an avid tester

Some people love nothing more than to beta-test new software. The joy of finding new bugs and trying to figure out what went wrong is what they live for. Others are scientists at heart, and love trying to go over stack traces, in order to reverse-engineer the code. If your client happens to be one of those, then writing automated tests will take all the fun out of using your software. Don’t do it!

8. The project is short, simple and straight-forward

If your team can complete the project in a short period of time (no more than a few weeks), and will never, ever have to reopen it for maintenance, then the benefits of maintainability, reusability and extensibility will be lost on you. Spending time and effort on these values is wasteful.

7. Your architecture is perfect

If there is no way to improve your architecture, then there is no need for it to be extensible. TDD, by the nature of its incremental development flow, forces the architecture to be extensible, simply by making you extend it as you go, and this, like lipstick on a pig, is something you just don’t need.

6. Your documentation is perfect

You never miss an API, and any change you make to your software gets documented instantaneously. The tests you create with TDD serve as a form of documentation, an example of how to use the API. Properly named and written tests actually explain the context of the test, so it is easy to find the test that shows you what you need to understand. Your documentation is so complete, that writing tests are a clear violation of the DRY principle, so you should clearly avoid tests.

5. Your team never changes and all members’ memories are perfect

The collective memory never forgets a single line of code it wrote, nor what the context was when writing it. You therefore, do not need the tests to remind you what the code does, why it does it, or how to use it. This also means that your team members never leave, nor are any new members recruited, because if this were to happen, you’d lose memories, or have members who don’t remember the code (having not been there when it was written). If this is the case, don’t bother with tests; they will just interfere with your incredible velocity.

4. “Done” means the code is checked in

Many teams have a definition of done (DoD) that means that the feature is “done” when it is in a state that the end user can receive and run (coded, tested, deployed, documented, etc.). Many others however, your team included, prefer a simpler and more easily achieved definition that accepts “checked in” as “done”. For you it is sufficient that the developer declared that he or she completed his part, and anything else is someone else’s responsibility. If you don’t need the code to be tested for the product owner / manager / user to accept it, then you are better served moving on to the next feature as soon as you can, instead of dragging on your relationship with this feature.

3. You’re paid to code, not test

Ignoring the fact that unit tests are code (sophistry), testing is what we have testers for. Perhaps your team's testers are fast enough that they can run tests on your code and give you feedback within mere moments, pinpointing the areas where you broke the code, so you can fix it while the changes are fresh in your mind, as well as a complete regression suite on the product, in case you broke something in a different component every night (they don’t mind working nights; they love the peaceful quiet). Good for you, cherish those testers, and make sure they have enough work so they won’t get bored and move on to a more challenging company.

2. Debugging doesn’t count, and testing takes too long

Like with any competitive company, your team must deliver on time, which means they must make estimates on the time it will take to deliver. Since your DoD doesn’t include testing, and you probably can’t guess how long it will take to debug the feature, what with all the cycling back and forth from development to QA, you estimate how long it will take to code it. If you want to meet your commitment, you can’t be adding a 20% overhead to your delivery time or you’ll miss the deadline. Worse, if you add 20% to your estimates, your manager might call you out on padding the estimates, which is his job. If that happens, who knows what might happen? Better play it safe.

1. It’s just a theory

Like Evolution (and Gravity), it’s just a theory. Even if all of the above reasons weren’t valid, nobody has ever successfully proven that this product could be completed faster and with better quality using new-age development methodologies like TDD. It’s just a matter of opinion.

Test yourself

Now, to test whether or not you should use test driven development, go over the above list. Count how many reasons apply to you. If you scored ten points, don’t use TDD. In fact, if you scored more than one (reason #8 might actually be legitimate), don’t write any code at all. Perhaps you’d be better served choosing a career that has fewer unknowns and moving parts. Perhaps paving roads?

Disclaimer: This post was written… Aw, just figure it out yourself!

Technorati Tags: Agile,Scrum,TDD,Estimation,Unit Tests,Definition of Done

How to Break a User Story into Tasks

2011-11-25T15:32:00.001+02:00

One of the vaguest aspects of Scrum is breaking stories into tasks, which is done in (the second half of) the Sprint Planning. Most of the questions I get when training or coaching boil down to one question: “How do I break a user story into tasks”?

Defining The Problem

My favorite definition for a user story is that it is a thin, vertical slice of functionality, describing a small increment in value to the user (or customer).

A task, on the other hand is a step taken by the development team, required to fulfill the requirements of the user story. I further like to define, as a rule of thumb, that a task should take half a day to 2 days to complete.

The question therefore should be what are the steps the team needs to take for the story to be considered complete?

Look at Your Architecture

Explicitly or implicitly, your project has an architecture. It might be anything from a master piece drawn out in painstaking elaboration by a the architecture team, to a high-level conceptual architecture diagram. You should look at the components in your system, and ask your self which components do you need to modify (or add) for the story.

For example, Let’s say that we have a product that is a social networking web site of some sorts. given a story such as In order to broaden my network, as a user, I want to be able to import contacts from Gmail. Looking at a typical architecture for such a product, my tasks could be something like:

Add a new Import Contacts page
Add a service to get contacts from Gmail
Modify the contact class to accommodate Gmail specific data (let’s assume such exists)
Modify the contacts data schema

Assuming these tasks are right-sized (4-16 hours), that’s it. If one of these tasks is bigger, break it down. If a task is smaller, merge it with a related one (if such exists. If not, don’t sweat it. My 4-16 rule is just a guideline). We now have all the tasks for us to complete the coding of the story.

Look at the Definition of Done

Of course, just coding your story isn’t sufficient for the story to be truly done. In Scrum, “done” means ready to ship, or as I like to define it: the story is as it will be received by the customer / user. Agile teams have a list of requirements that should be true for any story they complete. You should look at your Definition of Done (DoD) to see what tasks remain.

For example, let’s say your DoD includes the following:

Code Complete
Automated UAT (User Acceptance Tests)
Documentation is updated
Added to shipping package (e.g. InstallShield)

In this case, you should add tasks that help you achieve the story’s DoD. For the above example you might do the following:

Write UATs for the story
Document the Contacts Import feature

Assuming you didn’t have to add any new assemblies (i.e. libraries, packages, DLLs, executables, etc.), to your installation, you probably wouldn’t have any need to modify your packaging tool. No need to add a task for that.

That’s It!

You now have 6 tasks for the story:

Add a new Import Contacts page
Add a service to get contacts from Gmail
Modify the contact class to accommodate Gmail specific data (let’s assume such exists)
Modify the contacts data schema

Write UATs for the story
Document the Contacts Import feature

You can now estimate how much time each one will take, and are ready to move on, and break down the next story.

Technorati Tags: Agile,Scrum,Tasks,Sprint Planning,User Story,Definition of Done

How to Refactor Your Code

2011-11-22T12:01:00.001+02:00

If you develop your products using TDD or any of its derivatives (ATDD, BDD, etc), you are following the Red → Green → Refactor loop. If you are just getting started with TDD, you might find that while the red and green (write a failing test, and then make the test pass) are simple enough to understand and to demonstrate, the refactor phase is much less clear, and I’ve found few sources that explain it properly. In this post I will attempt to thoroughly explain what I do, and what I consider to be a best practice.

Refactor to Remove Redundancy

I usually use the refactor phase to reconstruct my code by removing duplications in code. I do this mostly in my production code (the code that will ship to my customer). I often leave duplications in my test code (the code that tests the production code), because for my tests I value readability over adhering to the DRY principle.

You should always start with your failing test (or unsatisfied specification, if you're into BDD). Production classes and methods that you pull from your tests should be public.

Note: If you're developing in .NET, you may wish to consider using the internal keyword (and adding the InternalsVisibleTo assembly attribute to your production assembly, so that your test project can use the code) instead. Then you would make it public only if another production assembly depends on it.

When you reach the refactoring phase (after making at test pass, or before writing a failing one), you should look at the code and walk through the following steps:

Any substantial or complex code that you need in two (or more) methods should be extracted into a private method, and called from both (all) of the original locations.
- Note: Any method that you create as part of the refactoring phase of your TDD should be made private.
Make helper methods protected only when a derived production class needs them.
Any method created in the refactoring phase (now private or protected), that you need in another production class should be extracted to a helper class and made internal (or public in any language that doesn’t support the concept).
If the helper class is needed in another assembly, then:
- If the other assembly already depends on the helper class' assembly, make the helper class public (if it isn't already).
- If the other assembly does not depend on the helper class' assembly, extract the helper class to a helper assembly (the best fit, or a new one) and make it public. Be sure to make the original assembly reference the new helper assembly, if it doesn't already.

Refactor for Readability

There are no real steadfast rules for this. What I usually do is go over the code I just wrote (or modified) and look for any violations of team coding standards, or poorly named identifiers (classes, methods, variables, etc.). I usually don’t have very many of those, as I tend to name them properly (which is to say, to my own satisfaction, but then again I’m often considered the strictest coder on my teams).

One trick I do use, is to go over the methods and see if I it reads easily. By easily I mean is it fluent? I don’t go all-out trying to develop a fluent API, but just see if the classes, methods and variables lend themselves towards some semblance of the English language. The following is an example of a block of “fluent” code:

Code Snippet
 var user = new User(usernameInput.Text, passwordInput.Text);    
  
if (userManagementService.Authenticates(user))    
{    
    Navigate.To(requestedWebPage);    
} 

Fluency isn’t a must-have, but I find it better than commenting. Comments can lie. Code doesn’t.

Refactor for Complexity Reduction

Cyclomatic Complexity is a code metric that measures how many distinct paths exist in a program’s code. A high value is more difficult to read and maintain. I find that sometimes reducing complexity makes it easier to maintain the code. I therefore advise on doing this kind of refactoring especially when you need to improve your understanding of the codebase, rather than after making every test pass.

Here’s a list of techniques that can reduce the complexity of your code:

Invert conditional statements – If your code is nested in if statements, where on one branch you do something and on the other you do nothing, you might invert the condition (checking whether the condition does not occur), and then return (exit the method) if the if-statement evaluated to true. The following block of code will run if the statement evaluates to false (what would originally have run if it did evaluate positively).
Extract blocks of code into private methods. The complexity is moved to a different method

Refactor for Extensibility

This refactoring technique ties in with complexity reduction, and requires the use of some design patterns. I usually use the Strategy Pattern for this kind of refactoring. The idea is to remove multi-branch conditionals (such as chained if statements or switch-case statements) and extract the different cases into concrete strategy classes.

There are several benefit to this refactoring technique:

New cases can be added without modifying (or with minimal modification to) existing code. This reduces the chance that existing code may break
Cyclomatic complexity is reduced
The strategies can be tested separately if you feel the need to (remember that they are originally covered by the original tests).

Other Refactorings

There are many more reasons to refactor code, but the above cover most of the cases I come across. If you are interested in reading more on refactoring software, I suggest reading the following books:

Refactoring: Improving the Design of Existing Code by Martin Fowler, Kent Beck, John Brant and William Opdyke
Refactoring to Patterns by Joshua Kerievsky
Professional Refactoring in C# & ASP.NET (Wrox Programmer to Programmer) by Danijel Arsenovski

Hope this helps.

Assaf.

Technorati Tags: Agile,TDD,BDD,Refactor,Extreme Programming,XP

What Will Microsoft Give Away at the Build Conference?

2011-08-29T11:33:00.001+03:00

In just over two weeks, on September 13th, the Build Windows conference will begin in Anaheim, CA. Microsoft is expected to unveil Windows 8 and show software and hardware developers how to use it. The good ol’ guys in Redmond have kept a really tight lid on what exactly is going on there, and even now, at the time of the writing of this post, they haven’t revealed any information about the agenda and the sessions.

There have been several posts, some seem better informed, while others seem like baseless gossip, regarding what is going to happen with WPF, .NET and the rest of the existing development tools Microsoft have supported for the last decade or so, given some unfortunate remark regarding HTML5, CSS & Javascript being the new tools of choice for developing Windows 8 “immersive” applications.

One thing that they have kept extremely quiet about however, is what they will giveaway as gifts to those who came to the conference

Giveaways

The year 2011 is, without doubt, the year of the tablet. IPad2, several Android Honeycomb tablets and now the demos that Microsoft made at Computex and Channel9 (all of which show case Windows 8 on tablets) all point towards a tablet being the main focus of development this year.

Moreover, in Google I/O 2011, Google handed out 5000 Samsung 10.1 Galaxy Tabs, one to each registered guest.

Microsoft can’t afford to, with all the hype created around Windows 8 and tablets this year, do any less.

It is my best, educated guess, that Microsoft will be handing out tablets with the latest build of Windows 8!

So, What’ll it be, exactly?

In the aforementioned Computex demo, in the 19th minute (18:30, to be exact), Mike Angiulo, Microsoft’s Vice President of Windows Planning demoed the new ARM based developers’ tablet (designed especially for software and hardware developers, with extra ports, and all), and said, and I quote:

“Now we have these developer reference systems that are all up and running… these are systems that are built for hardware and software developers… what they do represent are integrated, working PCs”

Mike went into some detail about the developer tablet’s specs:

ARM based, Snapdragon, 1.2GHZ chip
13mm thick
11.6” Screen
Extensive USB support (seamless connection to a DOK was demonstrated, worked as expected)
720p rear camera
All of the common sensors (proximity, orientation, etc.)

It looks like it runs (“flies” even) beautifully, instantly loading and playing an H.264 full HD video. Real-world usage will probably not be as good as demoed, but still…

Based on this, I believe that we (yes, I’m registered, and going to be there) will receive the developer reference tablet running Windows 8.

Remember, if I was right, you read it here first!

See you at “Build”,

Assaf.

Alternatives to Planning Poker

2011-08-12T11:00:00.000+03:00

This blog post is inspired by a tweet I saw:

The best thing about planning poker, and in my opinion the one thing that it brings to the table that no other system does, is the escape it offers from herd mentality. Planning Poker allows each member of the team to make his own opinion uninfluenced by others, and then allows the team as a whole to use Wisdom of Crowds to reach a good estimate.

The downside is that it is silly.

At least that’s how it looks from the outside. I mean, come on! A bunch of grownups spending precious hours of the work day, at the office, taking up a conference room while playing cards, until someone pulls a Joker (or whichever card signals the need for a coffee break)? I know the value of games and having fun at work, but I’m pretty sure that most adults are capable of enjoying good solid productive work without having to look goofy.

Here’s what I suggest:

Discuss the PBI (Product Backlog Item) for a short while (2 minutes seems like a good average).
Everyone takes a quiet moment to make a gut estimate. It doesn’t matter how it is made – relative (i.e. story points), absolute (ideal days) or rough (T-Shirt sizes), and writes it down.
Alongside the estimate everybody lists the top reason (or few reasons, if you must – but that’s all!) he believes his estimate to be true. Examples could be:
1. “We’ve done this before. Just extract and modify the ACME algorithm”
2. “Nobody’s done it here before”
3. “The data access layer is a mess. It will take a week to refactor”
4. “Only George knows how to do it, and he’s off, snorkeling in Aruba…”.
5. “A simple state machine and we’re done! I’m the state machine guru!”.
6. “Haven’t got a clue. Just want to play it safe.”
When everyone is done, merge the reasons, and decide on the best estimate.
Rinse and repeat: If consensus is reached, repeat with the next item. Otherwise, repeat with the same.

Does it look familiar? It should. But shhhh! Don’t tell anyone. Let’s keep it our secret.

Oh, and you can call it team estimation. Now that’s a good, serious meaningful name, isn’t?

Now run along, have some fun (the good productive, serious kind, not the goofy kind).

Assaf.

Another Stab at Work Estimation

2011-08-09T16:06:00.001+03:00

The Story

A couple of days ago, I was at a client, helping with his company’s migration from one version control system to another. As part of the work, we had to get the latest version of the source (i.e. download them to a local folder on the PC), copy it to the target system’s workspace (i.e. perform a standard windows copy to another folder on the PC), and then check in the sources.

I want to discuss the copying process. Yes, the copy.

Not much to copying you say? Well, essentially, you’re right. You select the folder that you are going to copy from (I selected a folder that was just over 1GB in size, and just over 50,000 files altogether) . Then you copy it to the target folder. That’s it. Oh, and you wait.

That’s what I really want to discuss. The Wait.

This was my experience:

First I had to wait for a few seconds (I think it was about 10 seconds) while windows calculated
Next, it started copying, and said it would take roughly 3 minutes.
After a few more seconds, of copying it reevaluated the work at 4 minutes or so.
After that windows started to progressively update the estimate until it reached about 5:30 minutes.
At that point, the estimation seemed to be on track, as the remaining time steadily went down.
About 5 seconds or so from the end of the copy, it began taking longer for each second to complete (i.e. it took longer than 5 seconds to complete from that point, and it took longer than a second to tick off one second from the remaining time).
Finally, Windows reported that it was done. 0 remaining seconds. Means it’s done, doesn’t it? I thought so. Windows seems to have a different definition of done, apparently. It took 10 more seconds till it was really done.

Sounds familiar to anyone? It should. Anyone who ever used a computer (I’ll hazard a guess that other PCs and operating systems have similar experiences) probably doesn’t even notice it anymore, seeing as it is such a common occurrence.

That’s not what I’m talking about, though. I’m talking to all of you corporate developers, team leaders and managers.

The Moral

In my experience many (perhaps most?) teams go through a frighteningly similar cycle whenever they estimate the cost of development on a feature, or a project.

This is (all to often) my experience:

The team spends some effort “calculating” the effort required to develop each feature. Usually, it seems like the same 10 seconds are spent, the results are equally optimistic, and just as “accurate”.
The team makes their estimate (optimistic and inaccurate).
Next, they start working on the problem. Soon enough, the team encounters the first problem they didn’t think of. The estimate goes up.
This goes on several times, pushing the deadline into the slack assigned to development.
Eventually, the team reaches some rhythm, and it seems like the team is on track. At least they report that they are.
Towards the end, when the deadline is just around the corner. In what is sometimes called “Crunch-time”, or the “robustness” phase, the team keeps announcing “we’re almost done”, or “we’ll finish tomorrow”. This usually takes quite a bit more than a day, of course.
Then they say that they’re done. Only they’re not. They completed part of the work – coding. “Testing? What’s that – that’s what QA are for. We meant done coding, not ready to deploy. What’s the matter with you?!?” It will take some more time before it actually is fit to let out in public.

And everybody is surprised.

That is what I don’t get. A computer, performing a copy, one of the simplest and straightforward tasks, can’t get the estimates right. And I was copying locally! And that was to another folder on the same disk (even the same logical drive).

How do you expect a human being, performing an act of research & development, doing something nobody did before (or at the very least not done by this human being, in this environment), to out-perform a computer?

I know managers and customers depend on these numbers. I suggest a 12 step program to deal with that. There are alternatives.

Welcome to the world of Agile development.

See you in the next post..

Assaf.

P.S. The next folder I had to copy, I used FastCopy. Took me less than half the time. Didn’t waste any of it on estimations.

Our Talk at Sela Dev-Days–ALM Best Practices with TFS 2010

2011-06-28T15:12:00.001+03:00

Today Yuval Mazor and I gave the talk above in front of ~20 people, as part of the ALM day of Sela Group’s Dev Days 2011. People were quite receptive – and I believe we got them to understand not only how you do things in TFS 2010, but also why. Some of the things we talked about:

Best practices for managing software development projects with TFS 2010
What are work items and how to use them (including customizations and links)
How to properly build a branching plan
Where and how to apply automated builds and CI

Unfortunately, we didn’t have enough time to cover TFS 2010 reports in detail – which means another talk is in order!

The presentation is available online here.

Thanks to Shai Raiten for the photography:

A big thank-you to our audience – it was great having you!

Software & I

2011-06-28T15:09:00.001+03:00

Hello all,

It took me a considerable amount of time to come up with a name for this blog. It was particularly difficult, because I didn’t want to pigeon-hole myself and limit the blog to just one subject. My other blog, Nightmare on ALM Street, specializes in gems about my current primary job – implementing application lifecycle processes with Microsoft Team Foundation Server.

But there’s more to me – even professionally – than just that. I’m a software developer, and while I mostly develop on the Microsoft technology stack (e.g. .NET and C#), I also dabble in other languages such as Java for Android development, non TFS / Agile / Scrum methodology, and even things that are not software related.

So I needed a blog that will talk about all that – Software being a major part of it, and other things that interest me.

In the end I came up with a simple and straight forward title – Software and I.

I promise to talk about anything that interests me – and hopefully, it will be of interest to you as well.

Lastly, I would like to give a very special thank you, to my good friend, Maor Linn, for designing the blog’s title banner. If you like it, give a holler – I’ll be sure to let him know.

Thanks,

Assaf.