|
As a test manager, one of the most embarrassing, and all to frequent, situations I find myself in is when I am reviewing bug reports with the development manager and he passes me a stack and says,
“We haven’t a clue what these mean.”
I look at the first one and it says,
Title: System Crashes,
Description: System crashed
or words to that effect.
The development manager looks at me and says,
“What the hell are we meant to do with that? How does it crash? What data are they using? What were they doing beforehand? How do your guys expect us to fix stuff if we don’t know what the problem really is?
I quickly point out the window, and why he is turned away, run from the room in shame.
Guys we have to do better!
Why Report It
Ok, once we get over the fact that we love finding bugs, that its what we are paid to do and it’s the thing that gets our juices flowing, we have to ask the awkward question, ‘Why do we report bugs?’
As far as I am concerned there is only one real answer, ‘So we can get things fixed.’ The reason for reporting a bug is to get it fixed, the whole reason for writing a bug report is to give the developers the right information to give them the best chance of fixing it.
So, the key is to use all the information we have at hand to give a full and comprehensive report.
What To Include
No Brainers
It doesn’t matter if you are using a popular issue tracking tool, an in-house Access database or Word and Excel paper sheets in a folder, ALL bug reports should include non negotiable bits of information in the header. These include.
Full details of the system under test, (Release Title, Version Number, Build Number)
Environment Details (What server build, configuration details)
Issue Number
Tester name and contact details.
Test Run date and Time
Title & Description
Give the bug a meaningful title, not ‘system crashes’ but, ‘when attempting to view X pc resets’
The description should take the main title and Fully explain it, for example.
“Having run a full data upload (see attached data set ref) I was attempting to follow the steps described in the test script (ref:xxx) I got to step 11 where I was expecting to be able to see the full data report displayed, however on selecting the ‘Display Report’ option from the dropdown list in the title bar (not the Display Report button) the system displayed an error message reading ‘blah blah blah’ (see attached screen shot). On closing the error message the PC reset, and resumed displaying the normal desktop with no application open.
See attachments for: System Log, Test Script, Data, screenshot.
Note: This appears similar (but not the same as) Issue Nos 123.”
Test Script Details
However one item I feel should always be included, but often is not, is the full test script details. Test Script Name, Reference and the specific test step that has failed.
This should be in such a format that the developer can quickly and easily find the test script (you do have formal written test scripts right?) and the associated test data pack (you do have reusable test data packs as well yea?).
This will prove invaluable to the developer when he is struggling to recreate your issue. He will be able to set up the exact scenario, walk through the exact steps, looking to see what is going on under the hood, and using the exact same data get the exact same result…. or not. If not, then it indicates an environmental issue, if yes then it indicates a coding error (or feature!).
Screen Shots.
If at all possible screen shots should also be provided, particularly if you are able to annotate them to point out your concerns, important points, easily missed details etc.
So if you don’t have the facility to capture screen shots, go get an application now! I tend to use Snagit for my screen captures as it allows me to do all I want both in terms of what I capture and how I annotate it, however there are a number of other solutions.
Logs
If you run with any logging enabled (application failure, network traffic etc) be sure you include these as well.
Severity Priority
Depending on local practice you may or may not add a severity and priority. Out of choice I prefer not to add these at the time the issue is raised, but later after reviewing the issue with the relevant stake holders. A low priority low severity bug from a technical viewpoint, might well be high from a business viewpoint.
Tips for Successful Bug reporting
Take Notes At The Time
Have incident recording sheets with you as you test. Where ever possible make copious notes at the time the error arises. Stop the test run, make the notes, go on. Don’t think you can simply record an a failed step, finish the test and go back and remember what to write up.
All The ‘W’s’
The age old classic questions apply when writing the incident report ask?
Who, What, When, Where, Why, and How.
Answer these and the developer will have a good understanding of the issue and circumstance
My No 1…. All Time Best….. Killer Issue Raising Tip
Get out of your seat and talk to the developers!
If there is an issue I am not sure about or don’t understand, or if I think it would help the developer understand the issue better if he saw it in action, or it is difficult to put in words, I get up and ask a developer to have a look see and tell me what he thinks. It works wonders, give it a try.
|
