8 reasons why you SHOULD track every bug
The times when you shouldn’t track bugs formally are so few that there shouldn’t even be a debate on the topic. As developers, the tracking of bugs is one of those things that you need to encourage whole heartedly. There are more uses for bug tracking than you would think. Keeping bug tracking as a glorified to-do list puts you at a loss. I have 8 good reasons why everyone should have some sort of tracking system.
Not tracking bugs might work in a solo environment if you are sharp, but in a team environment, the ball will get dropped quickly.
Here are 8 reasons why a bug list is more than just a to-do list:
1.) Find problem areas in an application
If you see an increase in bugs in a given area, you have the possibility to improve project performance. You can’t fix a problem you don’t know about.
2.) Determine the severity of your bugs
Do you have trivial bugs or critical bugs? If you are seeing more critical bugs, it could be an indicator that you need to institute some sort of code review or pairs programming.
3.) See who is causing the bugs
Who is generating all the bugs? Is it because they were assigned more (in quantity or complexity)?, is it because they just don’t check their code? Like #1, find the problem to fix the problem.
4.) See who is fixing the bugs
If you have one developer fixing all the bugs for a team (or maybe just another developer), It is a signal to be more aware of the situation. Unless they are hired to do so, nobody likes cleaning up someone else’s mess. Those situations can boil over quickly.
5.) Get an indication of testing quality
What bugs are slipping past testing rounds and what bugs are not? Do you need to change (or increase) the test cases in a given area? Most importantly: what slipped into a production release? Testing is a safety net, treat it well.
6.) Help resolve feature creep
This is a new trick to me, but it has been working quite well. Feature creep is sometimes a reaction to poor communication. Some customers get so worried that their good ideas will be lost that they demand that those ideas gets worked into the upcoming production release. You can work with the customer to document which production version each feature should be released in. You will demonstrate that you are listening to your customer, and you will get a head start on future business requirement documents.
I know feature creep treads on the agile/heavy argument, but as long as you are a developer, feature creep will be an issue. There will be a point where one more item will push you over a hard deadline. Whether you are agile or heavy, any help should be welcome in dealing with the issue.
7.) Debriefing: Measure development team progress across projects
Your team needs to grow as a team, or something needs to be done. There are a number of items that you can review across projects:
How many bugs per lines of code did you have (I know it is rough, but it can be a help)?
How many bugs in total got released into production?
What was the average severity of the bugs?
What percentage of total time was spent on testing vs. break-fix?
8.) Make the customer happy
This is the bottom line of any project. Your sponsors need to be pleased with your work. You make a customer happy by not only doing your job well, but communicating clearly while you do it. Nothing is going to upset a customer more than pointing out a bug/flaw and then having the developer forget to fix it. A Subject Matter Expert (SME) isn’t going to be comfortable until you can tell wrong from right. Clearly identifying bugs and flaws in interpretation of the SME demonstrates your aptitude. Listening to your customer’s enhancement ideas and documenting them (#6) rather than telling them to bring it up in future versions will make your customer very happy.
Bugs are going to happen, take the opportunity to learn from the ones that come with lessons. Being agile or heavy shouldn’t determine how bug tracking is to occur. There should still be significant testing even if you are releasing working code to the customer on a more frequent basis. Working code is more than just code that compiles/interprets without a fatal error. No one should be checking in code that hasn’t been reviewed, so it should be a non-issue. You can release rough code to demonstrate progress, but nothing should go into production without testing, and part of testing is tracking and documenting what has been found.
Tags: bugs, fundamentals
April 24th, 2007 at 9:13 am
You’ve listed 8 important goals, and every team should have some way(s) of addressing them. A good defect/task tracking tool can indeed help with this. However, there are other mechanism to reach the same ends, so the value of these goals does not imply the necessity of one particular way of reaching them (that is, with the help of a good bug tracking tool). I believe for some contexts, a good bug tracker is the best way to help with these goals, and in some contexts, there are more effective ways.
So, while I agree with the spirit of this article, I disagree with your statement that “there shouldn’t even be a debate on the topic”.
For example, I generally prefer to work on small teams in an agile manner. This makes certain options more realistic than when working on a big, process-heavy team.
Below I list how I prefer to meet these goals, *when possible*. Basically I encourage rigorous, test-centric and feedback-centric development practices to discover bugs early, and discipline and team commitment to fixing bugs immediately when they are discovered. If testers and developers are integrated on one team, the lifespan of a bug after being discovered is extremely short, and bugs rarely make it into production, then a bug tracker becomes less necessary.
1.) Find problem areas in an application: keep the bug count extremely low. When more than one or two bugs is discovered in an application area, it can be identified easily without bug tracking software.
2.) Determine the severity of your bugs: Again, by focusing on keeping bug count low (by identifying and squashing bugs early), this becomes easy without a bug tracker. Serious bugs are rare enough that they are memorable.
3.) See who is causing the bugs: With the close collaboration that is expected on an agile team, this becomes obvious quickly.
4.) See who is fixing the bugs: Again, on a small, agile team, this becomes obvious quickly.
5.) Get an indication of testing quality: Agile teams often have a strong focus on early and constant testing. If *any* bugs slip into production, it’s a testing issue. They are few enough that a bug tracker shouldn’t be necessary.
6.) Help resolve feature creep: I like your idea here very much, and I would encourage many teams to use it. I prefer to use a slight variant, in that we keep a feature backlog, which we occasionally insert bugs that need fixing into (rather than a bug list that we occasionally insert feature requests into).
7.) Debriefing: Measure development team progress across projects: I actually think metrics can be misleading here. The important thing isn’t how many bugs made it into production, but how much these bugs affected the business’s reputation and bottom line. If your bug count is low enough, you can discuss each specific instance individually rather than talking about the total bug count for each severity rating in your bug tracker.
8.) Make the customer happy: When a customer points out a bug/flaw, it should be fixed immediately, rather than going into a bug tracker. This is not always possible, so a bug tracker or some other task list is often necessary.
As I mentioned above, these methods are much easier to apply in certain environments than others. I am not proposing to do away with bug trackers, but I will contest that they are not always the *best* way to meet your goals.
April 24th, 2007 at 10:20 am
Ryan,
What you said makes sense. I can definitely agree with what you are saying, but I would like to make one point and ask one question:
The spirit of the article was to show that bug tracking is more than a to-do list. I think there can be benefit to having the data for review in an any environment even if you don’t NEED it (such as above). Documenting bugs is too quick to call it heavy
Plus, given the agile style of development, less bugs would qualify as appropriate to enter into the system.
My question is this:
The points you made rely on an experienced team of competent developers. What happens when they aren’t at that point? Do you use some sort of mentoring system, an experienced agile manager, or some form of both?
April 24th, 2007 at 9:32 pm
In an earlier day, I designed a system that would allow us to measure developer value by using a bug metric among MANY other items. That is, we measured how many bugs were later found in code that a developer wrote. We established a baseline as used that to self measure people as they matured as developers.
I bring this up, as it’s just another useful thing to get out of tracking your bugs.
The program was a great success, by the way. It allowed coders to measure their own progress as well as immediately demonstrate improvement in their own mastery of coding and the tools being used.
April 28th, 2007 at 3:42 pm
[…] 8 reasons why you SHOULD track every bug- Charles gives “8 reasons why a bug list is more than just a to-do list.” […]
May 22nd, 2008 at 12:14 pm
For anyone out there in doubt of the relevance of a dedicated bug tracking system in their case, why not justtry it out with a leightweight solution such as 16bugs.com or BugWiki.com? Both allows you to get started in seconds with no installation, configuration or learning to do.