Part 1

In my last post, I posed a testing problem. This post begins to address how I view the problem, and my motivation behind posing it.

Models

In software testing, a “model” is a representation of something we are trying to understand. It might be a representation in our mind (an idea), a diagram (like an Entity-Relationship diagram), or it might be a physical representation. A model helps us think of a system or concept in a certain way, and as a representation is not going to be perfect or complete. We can have many models for the same thing – our only limitation is our imagination.

Here’s an example:

How many ways can you model a web page? I can think of at least three categories:

1. as an end-user, or consumer view where I am using a web page for some purpose. I see the webpage as having text, colors, pictures and hyperlinks. I visualize that web page, served up in a web browser when I think about a particular page I find useful, and want to go to it. I interface with it and relate to it as it is shown through a web browser.
2. as a web developer. I tend to develop web pages using a text editor and do all of the HTML, CSS, etc. by hand. I interface with the web page with a text editor, and I visualize it and relate to it by the HTML view which is made up of markup tags. I interface with the web page primarily through a text editor.
3. as a web designer. I either have a visual picture in my mind of the layout and design of a web site I want to create and I follow that, or sometimes I draw it out using a tool. Even a rough sketch of a general layout on a piece of paper or on a whiteboard will convey that idea. I visualize the web page according to that layout.

These are three simple examples of modeling a web page. I added a couple of dimensions to help describe the models – a user, and an interface. Each one is different, but I may switch between all three as I’m developing a web page, or when I’m testing.

Since I often test web applications, I think of these different views a lot. I think of the first two very often, especially since I did a lot of web development in the mid-late ’90s. When I started writing web applications, I also learned to model from a programming context. My HTML tags, page layout, and content would be generated according to program logic, so I had a fourth model category derived from my role as a web application programmer. Understanding all the components used in rendering a web page is important to me when testing a web application. Everything from the web browser to the application server, web server, database if one is used, any drivers that are used, as well as the program code design is important information that can help with testing. I actually visualize what is happening across the entire application path when data is input into a web page, sent to the web server, processed, and the page I am on is updated. Even a simple HTML page has an interesting path. From the web server we have a text document that contains HTML markup, with certain tags that denote it as such. When a web browser asks for that page, the web server takes the tags, processes them and sends them via HTTP to the machine that is asking for it. The web browser takes that HTTP response, interprets it, and if it comes through without an error, it processes the HTML and displays it in a web browser.

Why is this important?

Modeling applications like this helps me imagine areas of potential problems based on experience and technical knowledge of the various technologies used, and also helps me narrow down or get repeatable cases for bugs I find in the application. It also helps me generate different testing ideas instead of one “happy path” based off of requirements documents, I can think of other potential usage of the software that might have the potential for problems. When I work with software users in their own environment, I am always amazed at how they use the software in unintended ways which they are perfectly happy doing.

What other models can you think of when testing web applications? Who are the users? Not all of them will be human. What are the interfaces? Not all of them will involve a Graphical User Interface like most modern, familiar web browsers.

But I’ve Run Out of Ideas!

The problem I described was something I deliberately chose as a modeling problem. Notice that in the energy industry, when I got a failure on a search attempt that contained the word “well”, I immediately had a suspicion of what the problem might be. But if I was in another industry, “well” may not be in my common vocabulary as a noun, but primarily an adverb. (Note that language is important when testing.) In my description here, we have at least two modeling categories of the software. One as “enterprise search”, and one as “my energy company search”. The ideas that each model represent are going to be subtly different. That subtle shift between models can be the difference between tracking down the source of a bug or not. But if I didn’t have energy experience, how can I find that model that will help me track down the bug?

I call this so-called running out of ideas “testing model myopia”. If we were omniscient, we would know all the possible ways to model the software, and we’d know where all the bugs were. If we were semi-omniscient and merely knew all the possible ways to model our software, we’d have an easier time. Since we aren’t, we have to use our skill as testing thinkers to break out of mental blocks. This can be very difficult if the entire team has testing model myopia as I described in the problem. When under project stress, with constraints on a development team such as time and resources, a tester may have to build a case in order to be able to pursue a testing idea. The predominant model of the software as shared by the team can be very entrenched, and it can be threatening to have that model questioned or analyzed. Nevertheless, as a tester, we are what Jerry Weinberg has described: “someone who can imagine that there might be another way.”

One of the first places I would start to tackle this problem is to imagine other models of the software that we aren’t addressing when testing.

Stay tuned for Part 2.