Agileista :

Testing user stories

Mon, 31 Mar 2008 12:55:00 -0700

Having had some time to myself recently, I've been reflecting on all the weird and wonderful implementations of Agile/Scrum that I've either; heard about, observed or been directly involved with. In doing this I've been reminded that it's all to easy to overcomplicate your agile process. In some cases even ignoring what could be considered the basics all together, without first trying them. One that I was reminded of recently is testing.

Why test?
Well it seems obvious doesn't it - we test stuff to see if it works. More than that we have tests and criteria so that we can see just how well it works and how healthy the work is. We test so that we can deliver value through quality software. So that we know what level of quality we are achieving; testing is one slice of the quality cake, a leading indicator of code and functional quality and sometimes lagging indicator of quality issues.

Creating acceptance tests also allows the Product Owner and the Team to explore the requirements and behavior together - its part of the conversation that they have when fleshing out the requirement before an iteration.

What is testing?
There are two main sides to testing and both are about delivering value . Above the waterline you have quality indicators that are visible to everyone. These deal with the experiential quality of the story. Below the waterline is code quality testing. This helps us to understand how expensive development and maintenance of the code is. A good agile or scrum implementation uses Acceptance Criteria or Q&A style functional black box testing, adding code testing such as unit testing and continuous integration. Omitting to do either side of this testing can have some pretty negative consequences.

Different types of testing
Acceptance criteria cover the visible or functional above the waterline tests. Validating functional correctness of the story. By defining how the story will interact with the user and visa versa the Product Owner (PO) can test for expected functionality. Acceptance criteria can also define any restrictions in functionality: for example, only accepting Visa for an ecommerce payment story or stating that no registrations can come from a web based email account.

From the team's perspective these tests also let the team know when they are ‘done’ with a story because if a story does not pass its Acceptance Criteria, it is not accepted as complete by the customer or Product Owner. If this happens the team are not awarded the estimated points for the story and the story is considered to have not been delivered, remaining on the backlog until it is.

Acceptance tests can also be used for release and regression testing. This moves us nicely on to code quality and build quality testing. The team need to know that what they've coded won’t break everything. Continuous Integration checks the entire code base each time code is committed to the repository and Test driven development can help you to build safe to re-factor code that also conforms to the acceptance criteria tests. The quality state of the code-base will affect the complexity estimates the team make for a given story as much as anything.

If you make sure to get those acceptance criteria exposed and agreed before each Sprint then at least that gives you and your team the ability to implement as much or as little testing as needed. If you don’t have acceptance criteria then your options for testing and knowing if you're doing the right thing are limited. What are everyone else’s experiences of testing in an Agile or Scrum environment?

Too many streams spoil the broth?

Tue, 23 Jan 2007 02:19:00 -0800

I was chatting with Eben about the current state of affairs at work. We got around to talking about the personnel and projects that my place has on the go.

The development staff consist of seven developers (one of which is the MD, one of which is me, with the obvious coding time impact for both of us) and roughly three testers.

The projects on the go, split roughly by a combination of technology and client number seven major and two minor.

If I count the two minor projects as one, and the MD's and my available development time as one developer, that's 1.125 projects, per developer!

This level of task diversity for our developers had never occurred to me before, in my defence this was largely the status quo before I even joined (bar one project or so). I wasn't exactly going to join and be able to convince the upper management to give up revenue streams.

All this is a far cry from Eben's situation of one team, aimed at one collective software goal (or at least, that's my current take on their situation).

It's obvious we need more staff, and there is an ongoing recruitment process. In the meantime the demographics lead to some interesting thoughts around the right way to structure the teams.

I may as well get around to admitting one of the Agile sins: We have a great deal of "infrastructure code". While I understand this can often be a bad thing, in this case it does serve to unify a lot of the work done in one language - indeed three of the biggest projects share exactly the same shared code base, with client-specific code on top.

Therefore I could potentially unify teams by language. This would help unify developer interaction. It would spread domain knowledge. It would create more of a team environment as the biggest team would now comprise of five 1/2 developers, plus two testers (say). I think this could be a marked improvement for the team dynamic; giving them the chance to really interact.

However, it presents a difficult question from the other side of the Agile coin: Specifically with reference to the Product Owner. How could a unified team like this serve the needs of the respective Product Owner for each client? I think the joint most important part of Agile is that it ends up (if done correctly) creating software the client wants. How then can the Scrum Master (say) use such a unified team such that each Product Owner's priorities can be folded into one Product Backlog?

It's this challenge that has prevented me from unifying the team, so I'm paying the price in terms of the greater complexity my current set-up creates.

An Agile Balanced Score Card

Fri, 19 Jan 2007 06:00:00 -0800

Recently I've been looking at how to implement a balanced score card (BSC) approach to performance management, on top of our existing Scrum practices. We have decided that BSC is the best overall approach for the whole business and that we would need to find a way to integrate it with our Agile processes and metrics as these have been very successful. My first thought was that as long as we don't measure performance at an individual level we'd be ok. Wikipedia defines the BSC approach in it's introduction as:

"The key new element is focusing not only on financial outcomes but also on the human issues that drive those outcomes, so that organizations focus on the future and act in their long-term best interest. The strategic management system forces managers to focus on the important performance metrics that drive success. It balances a financial perspective with customer, process, and employee perspectives"

I posted to the Yahoo! scrumdevelopment group asking for advice and examples of how to do it. I'm happy to report that I got some great advice from people who've been there and done it. Mike Cohn advised:

"I'd then bring the company goals back to the engineering department and we put together our own version of a BSC. For the engineering group we came up with four categories that were different from Kaplan and Norton's. We then came up with strategic objectives, core outcomes,performance drivers, and critical success factors for each along the lines of traditional BSC advice"

It's encouraging to know that the great and the good of the Agile world have been there and done it. I'm now working up our own score card categories and strategic objectives in line with Mike's advice.Tom Popendiek advised me to take a look at Creating a Lean Culture by David Mann from Productivity Press, and further advised not to measure at the individual level.

Software development is a team activity, there's no getting away from it. When developing software is done in teams it stands to reason that measuring performance at the individual level can disrupt and damage the team and its output.

Technorati Tags: BSC balancedscorecard agile scrum metrics management

Measured measuring

Fri, 03 Nov 2006 23:45:00 -0800

KPI's can destroy a teams morale and productivity if they are not carefully considered. Every organisation likes to measure the performance of their employees, some more formally than others. It's pretty easy to see what your metric is in a sales environment when deciding who is and who is not doing their best. Measure the value of their sales activity and you get a pretty good idea how well they are performing right?

In a development environment it can be tricky to know what to measure. There are plenty of potential performance metrics to choose from, everything is done on a computer so naturally you can track and measure it. The problem is one of choice. Whatever metrics you define as your criteria for success will become the focus for improvement. Hence the power of both measuring and rewarding salespeople on pure sales numbers.

For developers, measure lines of code (LOC) committed and they will duly write more LOC. LOC alone means nothing; it could be crappy code right? It's better to measure for the outcome you want. Find some way of measuring quality and that becomes the incentive. Measure it and the team will naturally seek to improve it.

In Scrum there is the team's sprint burndown, which shows how quickly they get through their story points and how likely they are to finish the sprint tasks. A longer trend you could measure is release planning, where you do a similar thing to a burndown but for the whole project/release.

This is all fine for estimating and project planning but in measuring quality it's not much help. Again, what you measure as 'quality' will depend on your situation but some possibilities include: unit test coverage, defect count, uptime, customer feedback, use statistics. Whatever it is you measure, make sure you can get the data reliably and at the right level of detail.

Transparent visibility

Wed, 20 Sep 2006 10:00:00 -0700

Every project has largely silent stakeholders who do not actively input into the project but do monitor its progress. These may be the stakeholders who finance the project or the may simply be senior management. By using Agile techniques you can get these people on board by providing them transparency and visibility.

Like it or not, visibility is one of the traits that Agile brings to the party. This visibility is a natural output of Agile process and forms an important part of the inspect and adapt feedback loop. You must be able to see what's going on so you can diagnose problems and make appropriate changes.

What you see of your project depends on what you measure; what you measure will be affected by the measuring. This is an important and related feedback cycle - setting metrics will shape project outcomes (I'll cover this in detail in my next post)

One of the metrics collected in Scrum is the estimated number of task hours remaining in the Sprint. We can use this information to create what is called a Burndown chart. A burndown chart is basically the team's remaining estimated time plotted against remaining time. The shape of this chart can help the Scrum master to see how well an iteration is going and to plan ahead for the coming iteration. To keep track of the software build as a whole a Release and Product Burndown can be used. This will track team output vs. time to release

The first step is in estimating tasks for users stories. User stories that have been chosen for a Sprint need to be unbundled into smaller actionable tasks. By doing this the team get a much clearer picture of the user story and tasks can then be estimated in 'man hours to complete'.

Estimates for each active task are updated by team members as they work their way through a Sprint. When a task is complete it is updated to show zero hours remaining. If a task is not completed by the end of that day it is re-estimated to reflect time remaining to complete it. This information is used to track the amount of work hours remaining in a given iteration; the Burndown chart.

This detailed level of time tracking can make some developers feel uncomfortable; they can feel that they are being monitored for efficiency like workers on a widget production line. When this happens team members may not update their hours on tasks and may slip into simply reporting a task is finished - some may not even do that!

Sprint Burndown charts become inaccurate and difficult to create when this happens. The teams ability to work together can be inhibited by this kind of scenario too. No one knows what's being worked on and the inevitable clashes begin. Luckily for us Boris Gloger and Tobias Mayer have come up with a simple and elegant solution.

During Sprint planning the team should not estimate the amount of time to complete a task, but simply ensure that each task is sufficiently small enough to be completed in a work day. By doing this a Burndown chart can still be produced and the team don't feel like they are being production line managed.

A by-product of this approach is that the team task board becomes much simpler and more effective. Every day during the 15 min Scrum meeting team members move their chosen tasks for the day into the In Progress column of the board and throughout the day those tasks are completed and moved to the appropriate column. This provides the transparency and team visibility that Agile promises and management demands.