Category Archives: SBTM

SBTM, Session Based Testing

Session-Based Testing Using Instant Messaging

Here’s a scenario: a programmer friend is creating a new web-based product for his company. He wanted someone to augment the existing testing that was already underway, and hired me to add more firepower. It is a distributed environment, which means that everyone in the office telecommutes. We decided to start with me doing an initial session, via instant messaging. We blocked out a time for the session, determined a focus (“mission”) and I started testing.

As I tested, I typed what I was doing in the instant messaging (IM) client, along with questions, concerns, crashes and bug ideas. He followed along through the application as I tested, asked clarifying questions as I discovered problems.

After ten minutes, I had generated enough ideas for that mission. He logged 6 bugs in his fault-tracking system. I had discovered 5, he had discovered 1 himself while following me along and monitoring the server and database.

I’ve done session-based testing over IM quite a lot with different distributed projects, like open source projects such as Watir and Session Tester, and with some of my clients that use telecommuting, or are completely virtual. We’ve used and adapted some of the ideas from SBTM to help provide more structure around the actual test sessions.

Using session-based testing with IM seems to work best in pairs. Once a focus and rough timebox are determined, test setup is out of the way, both pairs initiate a session through an IM client. One drives by testing, and the other saves the conversation locally, and creates notes, logs bugs, investigates underlying behavior, etc.

This style is natural (we communicate a lot with IM clients anyway) and doesn’t interrupt the natural testing flow if one person acts more as the primary test idea generator/executor and the other acts as the primary scribe. If you want to add some variation to your testing approach, give it a try.

SBTM, Session Based Testing, Session Tester, tools

Announcing Session Tester

UPDATE: Unfortunately, this is now a dead project. While we had amazing initial community support, the project waned after an economic downturn. We were unable to staff up a team to carry it forward. If you would like to take up the torch and start the project up again, contact me for the source code.

We’re pleased to announce a new open source, lightweight session-based testing management tool called Session Tester. Aaron West and I have been working on this project off and on for quite a while, and finally have a public beta release. We expect the beta release will be a little rough around the edges, but we’re pleased to have created something that we hope will help make session-based testing more accessible. We are following the open source model of “release early, release often” and value your feedback. Your feedback will help shape the tool as it emerges from Beta. Bug reports and usability issues are very welcome.

It’s early days yet and the tool is quite simple. User feedback will influence what features are added to the finished product(s). That’s the beauty of the open source product development process, where the evolution of the product is transparent and guided by the user community.

Session Tester was written in Java, so it should run on any operating system that has a recent version of the Java Runtime Environment. We have tested with Windows XP, Vista, Mac OS X (Intel) and Linux.

We developed this tool as an aid for testers who are using session based testing management. (Our initial interpretation of SBTM outlined in the article is more lightweight than some might be used to.) It provides an easy way to time your sessions, remind you of your testing missions, and to record your notes. It also provides an idea primer in case you get stuck, and a testing cheatsheet to review while you are in the thick of testing and would like some ideas.

Even testers who aren’t in regulated environments and don’t take detailed notes find the tool helps them add structure to their thinking when exploratory testing.

Getting Started

Download Session Tester 0.1 here. This is a beta release, so we don’t have proper installers yet. Be sure to have a recent JRE installed (at least JRE 6, update 6), or else Session Tester might not run properly.

If you are unsure about your JRE, go to the Sun site for a new version. We last used update 11, but anything recent should work: Java SE Downloads. There are also instructions on installation and usage in the user guide. Once you have a recent JRE, just unzip into a folder, double click the executable jar file, and start testing.

This is a beta release, so we expect bug reports and usability issues, as well as feature requests. Feel free to share any of those ideas on our forum. Please contribute to the community there. We will also be looking for contributors to the project itself.

Current Features

  • a timer to keep track of your session length
  • note taking that is automatically saved and formatted into XML, with extra tags for more organization
  • optional reminders
  • mission reminder so you don’t lose focus
  • session end reminder so you can start winding your session down
  • an idea primer for those times when you get stuck when trying to generate test ideas (thanks to Michael Bolton for introducing us to oblique strategies and contributing to the current list)
  • an exploratory testing cheatsheet, supplied by Elisabeth Hendrickson (check under the Help menu)

Currently, we are building the tool for individual testers to use to take notes, manage their sessions, prime ideas, and review their session files in XML format. Eventually, we will extend this to team management.

Project Goals

Our goal with this tool is to help train testers on how to do session-based testing, and make the practice easier for everyone. We also thought hard about the idea generation part of testing, and try to use the interruptions for note taking as opportunities idea generating or focusing tools, instead of just annoyances.

Thanks

Thanks to Patrick Lightbody for generously providing the space and for supporting this project, as well as to James and Jon Bach for creating SBTM, Antony Marcano for the inspiration and useful feedback, and to Jared Quinert and Mike Kelly for their usability feedback. Thanks to Michael Bolton for the inspiration for the idea primer, and for contributing test phrases to it, and thanks to Elisabeth Hendrickson for granting us permission to provide her cheatsheet as a thinking reference for testers who are using the tool.

Questions?

Sign up and post questions, concerns, comments, queries, etc. on our clearspace discussion forum.