I can confidently say that nobody likes software bugs. Users hate them because they make the application or tool that they need unusable. Developers hate them equally as much because they are a manifestation of flaws in their code, and they can quickly sap valuable time away from other tasks as they need to be addressed.
We have a shared culture of addressing bugs as they come in. No matter the size or importance. You get a bug, you fix it. No exceptions. You can see where this may cause issues with larger work items, as time spent fixing bugs is time that could be spent making cool things.
Regardless of the implications, I like to believe that we and our users are both on the same team here, however. No reasonable software developer delights in frustrating their end users, and no reasonable user would intend to frustrate a developer when they find issues when using their code. Finding and fixing bugs usually leads to better code, which should (in theory) lead to less bugs. (On second thought, some fringe test engineers may like bugs for this reason.)
This is by no means a unique topic. There are plenty of others who have described this more thoroughly, more eloquently, and more timely than I, and I invite anyone interested who wants more detail to partake in their wisdom:
The Art of Bug Reporting: How to Market and Get Your Bugs Fixed?
The first things that come to my mind as I start writing this article are the words of Cem Kaner - "The best tester…
What I hope to achieve through this post is to add the human element to this conventional wisdom. I’ll be taking you on a field trip to a GitHub issue that shows what the experience may be like for the person on the either side of the ticket, and add the relevant context to improve this report, and avoid losing time by needing to follow up with additional questions or information.
Hopefully, we can all walk away from this experience more fulfilled and equipped to empower each other to be able to address the issues we see in the software we love. 😊
In the most general sense, a good bug report should tell the reader, “How can I recreate this issue?” Usually you want to be able to answer as many of these questions as efficiently as possible:
1. On what version of the software does this occur?
2. Is there any other relevant software that I need to use in conjunction with this to make the issue happen?
3. What exactly do I need to do make this issue happen?
3.a. Does this happen only for a particular input or application state?
4. Are there any failure logs or files that I can provide that help pinpoint the cause of this issue?
Any user-reported reported insights are usually appreciated as well, one can speed up the bug-fixing process by helping answer as much as possible, the question, “How can I fix this issue” . Depending on the nature of the software and issue tracking system, these questions may or may not be answerable but help greatly:
- Is there a known component that may be causing this to happen?
- Is this already a known issue?
- Are there any lines of code that may be related to the issue?
First some context:
This report comes from the Babylon Exporter repository on GitHub. These tools integrate into 3dsMax and Maya, two great 3d modelling applications, to allow the user to export their created scenes to the glTF file format. Unfortunately, these will be very 3d-oriented, but hopefully the lessons can be interpreted and applied by anyone. 😊
An example of a well-intentioned but not very helpful report: “Help! This thing doesn’t work and I don’t really know why…but it worked before.”
As an end-user, one shouldn’t need to know the complete internals of the tools they use. We have an unfortunate user here that suffers an application crash when loading a particular file.
However, there is much more we can do to make this as informative as we have the ability to do.
As the developer, when I encounter such a report, my first thought is usually “How the heck am I going to figure this one out?” This particular issue does not specify the version, so it’s hard to tell what’s going on in the submitter’s computer to cause this issue. In addition, the file that causes this issue was not provided either. (In this specific field, that’s somewhat understandable given the pervasiveness of NDAs, but sometimes one can quickly create an approximation.)
In addition, in order to reproduce this issue given the current context I now need to:
1) Create a scene that matches the user’s description
2) Pray that this causes the reported crash
Oof. Not an ideal debugging process. As an impostor 3d artist, I know that creating reproduction scenes myself may take anywhere from a couple minutes to a couple hours, especially one as complex as the issue submitter suggests. This is not really an option if there’s other tasks on the table to accomplish. Unfortunately for this particular user, instead of the ideal cycle of:
we now have a turnaround cycle of:
Now how can we avoid this? Following our list of questions from earlier, we can make a quickly actionable report without too much extra technical know-how:
While this approach should work for simpler issues and simpler projects, larger and busier projects should take it a step further to have explicit bug reporting rules to encourage good reporting practices. See the Babylon.JS repo for a good example of these best practices expressed more formulaically:
Even if you’re not able to diagnose your issue, a good report with issue reproduction steps or materials makes it easy for others to diagnose it for you, and that makes it easy for everyone to be happy.