Testing User Stories
Posted by: Eben Halford in Agile Practices on August 21st, 2008
Having had some time to myself recently, I’ve been reflecting on all the weird and wonderful implementations of Agile/Scrum that I’ve either; heard about, observed or been directly involved with. In doing this I’ve been reminded that it’s all to easy to overcomplicate your agile process. In some cases even ignoring what could be considered the basics all together, without first trying them. One that I was reminded of recently is testing.
Why test?
Well it seems obvious doesn’t it – we test stuff to see if it works. More than that we have tests and criteria so that we can see just how well it works and how healthy the work is. We test so that we can deliver value through quality software. So that we know what level of quality we are achieving; testing is one slice of the quality cake, a leading indicator of code and functional quality and sometimes lagging indicator of quality issues.
Creating acceptance tests also allows the Product Owner and the Team to explore the requirements and behavior together – its part of the conversation that they have when fleshing out the requirement before an iteration.
What is testing?
There are two main sides to testing and both are about delivering value . Above the waterline you have quality indicators that are visible to everyone. These deal with the experiential quality of the story. Below the waterline is code quality testing. This helps us to understand how expensive development and maintenance of the code is. A good agile or scrum implementation uses Acceptance Criteria or Q&A style functional black box testing, adding code testing such as unit testing and continuous integration. Omitting to do either side of this testing can have some pretty negative consequences.
Different types of testing
Acceptance criteria cover the visible or functional above the waterline tests. Validating functional correctness of the story. By defining how the story will interact with the user and visa versa the Product Owner (PO) can test for expected functionality. Acceptance criteria can also define any restrictions in functionality: for example, only accepting Visa for an ecommerce payment story or stating that no registrations can come from a web based email account.
From the team’s perspective these tests also let the team know when they are ‘done’ with a story because if a story does not pass its Acceptance Criteria, it is not accepted as complete by the customer or Product Owner. If this happens the team are not awarded the estimated points for the story and the story is considered to have not been delivered, remaining on the backlog until it is.
Acceptance tests can also be used for release and regression testing. This moves us nicely on to code quality and build quality testing. The team need to know that what they’ve coded won’t break everything. Continuous Integration checks the entire code base each time code is committed to the repository and Test driven development can help you to build safe to re-factor code that also conforms to the acceptance criteria tests. The quality state of the code-base will affect the complexity estimates the team make for a given story as much as anything.
If you make sure to get those acceptance criteria exposed and agreed before each Sprint then at least that gives you and your team the ability to implement as much or as little testing as needed. If you don’t have acceptance criteria then your options for testing and knowing if you’re doing the right thing are limited. What are everyone else’s experiences of testing in an Agile or Scrum environment?
An Agile Balanced Score Card
Posted by: Eben Halford in Agile Practices on August 21st, 2008
Recently I’ve been looking at how to implement a balanced score card (BSC) approach to performance management, on top of our existing Scrum practices. We have decided that BSC is the best overall approach for the whole business and that we would need to find a way to integrate it with our Agile processes and metrics as these have been very successful. My first thought was that as long as we don’t measure performance at an individual level we’d be ok. Wikipedia defines the BSC approach in it’s introduction as:
“The key new element is focusing not only on financial outcomes but also on the human issues that drive those outcomes, so that organizations focus on the future and act in their long-term best interest. The strategic management system forces managers to focus on the important performance metrics that drive success. It balances a financial perspective with customer, process, and employee perspectives”
I posted to the Yahoo! scrumdevelopment group asking for advice and examples of how to do it. I’m happy to report that I got some great advice from people who’ve been there and done it. Mike Cohn advised:
“I’d then bring the company goals back to the engineering department and we put together our own version of a BSC. For the engineering group we came up with four categories that were different from Kaplan and Norton’s. We then came up with strategic objectives, core outcomes,performance drivers, and critical success factors for each along the lines of traditional BSC advice”
It’s encouraging to know that the great and the good of the Agile world have been there and done it. I’m now working up our own score card categories and strategic objectives in line with Mike’s advice.Tom Popendiek advised me to take a look at Creating a Lean Culture by David Mann from Productivity Press, and further advised not to measure at the individual level.
Software development is a team activity, there’s no getting away from it. When developing software is done in teams it stands to reason that measuring performance at the individual level can disrupt and damage the team and its output.
Measured measuring
Posted by: Eben Halford in Agile Principles on August 21st, 2008
KPI’s can destroy a teams morale and productivity if they are not carefully considered. Every organisation likes to measure the performance of their employees, some more formally than others. It’s pretty easy to see what your metric is in a sales environment when deciding who is and who is not doing their best. Measure the value of their sales activity and you get a pretty good idea how well they are performing right?
In a development environment it can be tricky to know what to measure. There are plenty of potential performance metrics to choose from, everything is done on a computer so naturally you can track and measure it. The problem is one of choice. Whatever metrics you define as your criteria for success will become the focus for improvement. Hence the power of both measuring and rewarding salespeople on pure sales numbers.
For developers, measure lines of code (LOC) committed and they will duly write more LOC. LOC alone means nothing; it could be crappy code right? It’s better to measure for the outcome you want. Find some way of measuring quality and that becomes the incentive. Measure it and the team will naturally seek to improve it.
In Scrum there is the team’s sprint burndown, which shows how quickly they get through their story points and how likely they are to finish the sprint tasks. A longer trend you could measure is release planning, where you do a similar thing to a burndown but for the whole project/release.
This is all fine for estimating and project planning but in measuring quality it’s not much help. Again, what you measure as ‘quality’ will depend on your situation but some possibilities include: unit test coverage, defect count, uptime, customer feedback, use statistics. Whatever it is you measure, make sure you can get the data reliably and at the right level of detail.
Transparent Visability
Posted by: Eben Halford in Agile Practices, Agile Principles on August 21st, 2008
Every project has largely silent stakeholders who do not actively input into the project but do monitor its progress. These may be the stakeholders who finance the project or the may simply be senior management. By using Agile techniques you can get these people on board by providing them transparency and visibility.
Like it or not, visibility is one of the traits that Agile brings to the party. This visibility is a natural output of Agile process and forms an important part of the inspect and adapt feedback loop. You must be able to see what’s going on so you can diagnose problems and make appropriate changes.
What you see of your project depends on what you measure; what you measure will be affected by the measuring. This is an important and related feedback cycle – setting metrics will shape project outcomes (I’ll cover this in detail in my next post)
One of the metrics collected in Scrum is the estimated number of task hours remaining in the Sprint. We can use this information to create what is called a Burndown chart. A burndown chart is basically the team’s remaining estimated time plotted against remaining time. The shape of this chart can help the Scrum master to see how well an iteration is going and to plan ahead for the coming iteration. To keep track of the software build as a whole a Release and Product Burndown can be used. This will track team output vs. time to release
The first step is in estimating tasks for users stories. User stories that have been chosen for a Sprint need to be unbundled into smaller actionable tasks. By doing this the team get a much clearer picture of the user story and tasks can then be estimated in ‘man hours to complete’.
Estimates for each active task are updated by team members as they work their way through a Sprint. When a task is complete it is updated to show zero hours remaining. If a task is not completed by the end of that day it is re-estimated to reflect time remaining to complete it. This information is used to track the amount of work hours remaining in a given iteration; the Burndown chart.
This detailed level of time tracking can make some developers feel uncomfortable; they can feel that they are being monitored for efficiency like workers on a widget production line. When this happens team members may not update their hours on tasks and may slip into simply reporting a task is finished – some may not even do that!
Sprint Burndown charts become inaccurate and difficult to create when this happens. The teams ability to work together can be inhibited by this kind of scenario too. No one knows what’s being worked on and the inevitable clashes begin. Luckily for us Boris Gloger and Tobias Mayer have come up with a simple and elegant solution.
During Sprint planning the team should not estimate the amount of time to complete a task, but simply ensure that each task is sufficiently small enough to be completed in a work day. By doing this a Burndown chart can still be produced and the team don’t feel like they are being production line managed.
A by-product of this approach is that the team task board becomes much simpler and more effective. Every day during the 15 min Scrum meeting team members move their chosen tasks for the day into the In Progress column of the board and throughout the day those tasks are completed and moved to the appropriate column. This provides the transparency and team visibility that Agile promises and management demands.
Some good practices for Agile coding
Posted by: Eben Halford in Agile Practices on August 21st, 2008
While the process framework behind Scrum and Agile in general gets you thinking about how you organise the work and the facilitate the team, it’s only half the story.
For software development teams there are some engineering practices that can help the team code well within the people and organisational processes that Agile implements. The aim of these engineering practices is to allow the development team to create the best software product possible. These are enabling approaches that allow us to focus on the most important aspect of software development; not engineering practices but building the right software.
The idea is to define and agree a basic way of working that allows us to focus on writing great software that satisfies the business. By using some good engineering practices we remove the waste in a developers day and allow them to get on with coding.
What follows is a mixture of engineering practice and working practices that I’ve had some success with.
- Control your code – It’s just good practice to use revision control however you decide to run the project. We like subversion and git.
- TDD/BDD – Test Driven Development or Behaviour Driven Development. Write your tests first and then write your code to fulfil those tests. This way you don’t end up with no time to test and it means you’re doing the design up front in small chunks. People often wonder where the Architecture is in Agile, well if you do TDD the way it was meant to be done then when you write your tests you’re actually designing how the code should behave. Effectively the TDD cycle of test-code-refactor is an inspect and adapt design process, that a developer goes through in cycles lasting minutes.
- War room – Your team and their interactions are the single the most important factor in the success or failure of a project. If you can get all your team in one room and make it their war room it will help the team to be cohesive and develop a sense of shared values and team practices or behaviours.
- Continuous Integration and Automated Testing – Automate both unit testing and acceptance or functional testing. Ad hoc testing is a bad idea – by automating the building and testing in the code control system you ensure that the software builds at every check-in; this is continuous integration. Developers will feel more comfortable changing existing code and refactoring, no one forgets to run the tests, you spot errors early, development becomes more flexible. For a test framework in Rails apps there is Rake, Zentest and Test::Unit for Ruby, and for JUnit for Java. For acceptance testing there is Selenium and Fitnesse. Cruise control for Ant enforces continuous integration in Java.
- Quiet time – When you’re in the zone and cranking out quality code like it was gong out of fashion, the last thing you need is an interruption. Once you’ve been rudely awakened it can take ages to get back into the right frame of mind for problem solving. Agree a quiet time with no phones and email or devise a physical signalling system like a busy flag so people know not to interrupt.
- Automate those documents – Use software that extracts relevant information from the code and creates some of your documentation for you. That way your documents show you what is really happening with the software. It almost documents itself. So much more useful than some spec that says what should be happening.
You should also look at pair programming. Once I’ve had more experience of it I’ll post my thoughts on it. Check Ron Jeffries site for more.
Remember, not every practice works for every situation, try them out and keep what fits. Make sure to get the team to agree their working practices – enforced ways of working that validate and measure the wrong practices and metrics wont deliver you software, let alone agility.
Intro to Scrum
Posted by: Eben Halford in Agile Introduction on August 21st, 2008
I’m going to give a brief overview of an Agile development process called SCRUM. At the end of this article are some links to further information for those of you that want to read about SCRUM in more depth.
First some terminology:
- Scrum Master – The person who enables the Scrum process, a facilitator
- Product Owner – The person ultimately responsible for defining the product and its features
- Iteration – A set cycle of development time
- Sprint – Scrum name for the iteration cycle
- Development team – Everyone it takes to get from requirement to tested releasable code (coders, testers, Q&A, Analysts, DBA’s etc)
- Velocity – A measure of the completed features for a given iteration
The first thing you need to begin Scrum is a Product Backlog. This is essentially a list of features or in my case user stories. This initial list – just enough to get you started – should be prioritised by the Product Owner according to business priorities. Next you’ll need to decide on your iteration lengths. Typically a team starts with iteration lengths of 30 days but some teams prefer 2 week iterations.
Now you’re ready to begin Sprint planning. Sprint planning is usually split into 2 four hour section – during the initial phase of planning the Development Team sit with the Product Owner and the Scrum Master to estimate items on the Product Backlog. These should be sized according to their relative complexity – that is ‘how big is this feature?’ not ‘how long will it take?’ Points are awarded to each User Story. This meeting is usually time-boxed to around 4 hours with as many features as possible estimated in that time. Times for daily scrums and the end of Sprint Review and Sprint Retrospective are also agreed.
With the backlog estimated and prioritised the team work out how many of the most important items on the Product Backlog they think they can do in the allotted Iteration or Sprint period. This can be based on the teams current Velocity if the team have been sprinting together for a while or it can be based on the expert opinion of the team. Based on the items chosen from the Product Backlog a Sprint Goal is agreed with the Product Owner.
For the second half of Sprint Planning the team take the list of items to be worked on during the Sprint – the Sprint Backlog – and break down each item into more granular tasks covering everything from coding to meetings. These tasks are estimated in hours to complete. The team then commits to deliver the functionality in the Sprint Backlog along with the overall Sprint Goal.
Once the Sprint begins there are daily meetings of around 15 min where the team gets a chance to inspect and adapt again. Three key questions are asked: What have you done, what are you going to do and what has been an impediment to your work? It is then the Scrum Masters role to work with the business to remove those impediments to progress for the team.
Once the iteration comes to a close the larger inspect and adapt cycle kicks in. There is a Sprint Review with the product owner to go over all the code completed in that iteration. The Product Owner runs through the software to be sure it meets the original requirements and if it does the team are awarded the points for each user story deemed complete. These team points are their Velocity – a measure of the amount of functionality/complexity that they can deliver per iteration. This review period is also a chance for the team to share their work and get feedback from the rest of the company.
The best time for a Sprint Retrospective is right at the end of the iteration. During the retrospective the team looks at what went well and what didn’t and looks to adapt accordingly. Once the retrospective has been held its back to Sprint Planning and the iteration cycle kicks in again.
Some further reading.
Books:
Agile Project Management with Scrum
Agile software Developement with Scrum
Websites:
Agile History
Posted by: Eben Halford in Agile Introduction on August 21st, 2008
Before I get into my experiences of implementing Agile lets take a brief look at some background.
Agile refers to a conceptual framework of processes for managing high risk, high entropy projects such as software development. The Agile Alliance has this to say -
“In the late 1990’s several methodologies began to get increasing public attention. Each had a different combination of old ideas, new ideas, and transmuted old ideas. But they all emphasized close collaboration between the programmer team and business experts; face-to-face communication (as more efficient than written documentation); frequent delivery of new deployable business value; tight, self-organizing teams; and ways to craft the code and the team such that the inevitable requirements churn was not a crisis”
Lean management, an influential and successful movement in management theory shares some of the same values and principles as Agile. One of the pioneers of Lean was Toyota Corporation of Japan. Throughout the 1990’s Toyota was making striding gains in the U.S. car market and was fast becoming the market leader. Theorists and consultants from around the world lauded them as the model of efficient, high quality, modern car manufacturing.
Lean management principles and Just in Time (JIT) manufacturing techniques meant that Toyota – a conglomerate of over 150 companies – was able to sustain its production line output whilst holding only three days of stock at any one time. Envious competitors and detractors alike were waiting for Toyota’s new management techniques to blow up in their face.
In early 1997 Toyota woke up to their first real crisis since implementing Lean processes. By 9 a.m. on February 1st Aisin Kariya plant number one had burned to the ground. A key plant in the manufacture of P-valves, Aisin produced over 32,500 P-valves per day for the Toyota group. Its demise left Toyota facing an uncertain future with only three days of production capacity left before suffering potentially huge and irretrievable loss in sales and profits. This had dire consequences for not only Toyota but the Japanese economy as a whole.
Remarkably, the Toyota network of associated companies and suppliers managed to re-organise themselves into a new structure (emergence – self organising) and had managed to begin manufacture of P-valves again only two days later. By working together with no haggling over commercial contracts or technical rights, more than 200 helped out with over 62 taking direct responsibility for p-valve production. This emergent behavior can be paralleled with the operation and response of organic networks and systems such as ecosystems responding to a change in the environment, or the neural networks repairing themselves after brain damage.
What does all this have to do with Agile you say? Well one of the important outcomes that that we try to nurture in Agile is emergent behavior from our teams and organisations. We want to create an environment that is conducive to autonomous emergent behavior. When you embrace change and integrate your supply chain communications and methods within an Agile or Lean environment wonderful unexpected things can grow out of it.