During our workshop about grassroots content testing at the STC Summit in Sacramento, the participants listed difficulties and barriers to testing enterprise content—particularly when we haven’t been asked to do so! This series of posts is intended to let us generate some solutions to those difficulties.
I’ve decided to straight-up post the Post-its and let you all discuss.
Here were some challenges people experienced when they used their scenarios to test the sample content.
Facilitator giving too much info at the beginning
We discussed that it can be a delicate balance between providing setup information in the scenario and asking your tester to perform a task. You want to provide a realistic amount of information and avoid undue frustration, while still getting meaningful results.
So, what do you do if it becomes apparent during the testing session that you have failed at that balance? Is it appropriate to ad lib and try to make better use of valuable testing time?
Taking notes and moderating was too much
I believe it. So what are some ways to sufficiently document a testing session?
“Getting the subject to think out loud”
How do you encourage the tester to keep sharing if they tend to get quiet? In his book, Rocket Surgery Made Easy, Steve Krug recommends simple, repetitive phrases like, “What are you thinking now?”
“I like to ask too many probing questions”
Ha, I can relate to this. I write the scenario, but when I’m in the moment, I think of so many more questions that I want to ask. What’s an appropriate amount of extemporaneous investigation?
What if the content is not holding up well under scrutiny?
During the session, it may become apparent that the content you’re testing is not passing the tests. The tester is not finding the information, or is not being persuaded, or the content is not relevant to the task. You might have already suspected this was going to happen, based on the process of defining the success metrics and writing the scenario. Or you may have chosen content that didn’t quite match the scenario. Your co-workers are watching; perhaps even the author of the content, if that person isn’t you. Perhaps you were already uncomfortable about writing that scenario.
It’s risky to reveal inadequacy. How do you handle this?
I think this is another area where well-defined, agreed-upon success criteria will serve you. These should not be criteria that you defined all by yourself, or else your criteria can be called into question in addition to your testing methods. So, how do you involve the right people?
Ok, your turn.
What has tripped you up during testing sessions? Or, do you have tips about the problems we’ve listed? Let us know in the comments.
