Researchers and designers often aim to improve something unfamiliar to them. As the four of us soon found out, we also needed to remodel our research tools.
For background on the Rwanda we arrived in and the project we worked on, check out the companion to this post: The World Cup, Rwanda, and User Experience Research.
Without a schedule…
Our research plan did not include the schedule you might expect, with slots and breaks and participants all in a row. Instead of making sure we were on the appropriate calendars, our staff in Rwanda responded to our requests with “Let’s wait.”
You may have heard the phrase “Africa time.” It’s often spoken in an exasperated voice and as a euphemism for “nothing ever runs on time.”
I found Africa time, however, to mean something else. Instead of worrying over schedule, most folks we encountered were focused on what was happening at that moment with the people around them.
So, it wasn’t until a couple days after we arrived in Kigali that we knew which four schools we would visit.
…do spontaneous research.
We could do exploratory research:
- Non-leading interviews
- Follow-your-curiosity explorations
Or evaluative research:
- Usability tests on laptops
- Web training observations
We prepared scripts and materials for interviews, usability tests, and training observations. The “follow your curiosity” research was the most fun, exciting, and—as it turns out—informative.
We had the opportunity to follow our curiosity a few times throughout our stay:
- My fellow researcher walked all over one campus with a teacher trainer, even seeing where the trainers lived and prepared when not in the classroom.
- I ducked out of a training session to see the school library and talk to its librarian.
- Another day, the two of us sat among a half dozen teacher trainers at the lunch prepared by teachers and staff, and lead a half dozen micro-interviews.
Thanks to the permission we gave ourselves to explore, we learned much more than we could have if we had attempted to follow a schedule.
Without advanced recruiting…
With the lack of schedule, we could not follow a formal process for recruiting participants.
When we showed up at a school, we would tell the principal what we aimed to do. After thanking him for his hospitality and turning down his tea, we would meet whoever he thought we should talk to first.
…do recruitment interviews on the spot.
What subject did they teach? English teachers were good participants since they were especially comfortable speaking with us.
How often did they use a computer? Computer teachers were disqualified, since they’d likely have a different understanding of the web than the average teacher. On the other hand, folks who rarely used computers were disqualified from usability sessions, in case they were too unfamiliar with using a mouse.
Did they have time to talk to us now? If we were satisfied with their first two answers, we’d ask if the teacher trainer had time to talk more.
If a candidate seemed like a good person to talk to and they had time, we’d find a quiet place to meet.
Without a formal lab…
Though we knew enough to bring any equipment we would need with us, we had to rely on the school’s classrooms and offices to stage our sessions.
On one occasion, a designer and I happily accepted the Dean of Discipline’s office for a usability test. I set up at his desk with the participant, and the designer sat a few feet away so she could take notes.
And then, 10 minutes into a focused session, the Dean walked in and began to look through his filing cabinet. My teammate raised her eyebrows in surprise. The participant continued to think aloud.
A minute later, in came another teacher, who started a chatting with the Dean while they stood at the filing cabinet. My teammate raised her eyebrows again and shook her hand in the direction of the conversation. I nodded.
But the participant stayed focused and continued to think aloud, not phased even by our silent conversation.
…work with what’s around you.
A phrase I learned earlier in the trip should have prepared us for this perspective. Our Rwandan team invited us to a lake near Kigali. We had dinner there and even took a rowboat on the water. While we ate, I asked my colleagues to teach me more Kinyarwandan. I told them I knew how to say hello, goodbye, please, and thank you.* What should I learn next?
One person immediately replied, “Nta kibazo.” Everyone laughed and nodded in agreement.
I tried to say it the way he did, with the first word sounding like “na” and the second sounding like “chi-bah-zo.” “What does it mean?” I asked.
“Don’t worry about it,” he said.
It was a phrase I’d heard before, whether in English, “no worries,” or in Swahili, “hakuna matata.”**
As it turns out, even when you don’t have the Dean of Discipline walking into your usability session, not worrying helps when doing research or design of any kind.
Although we certainly do and should care about how things are done and what decisions are made, not everything will go as we expect it to go. If the changed expectation disrupts a participant or creates unreliable results, I do something about it. But if everything is going fine, then nta kibazo.
Many research techniques, however, work against finding where serendipity takes you. Many techniques pursue the goal of asking many people the same questions with the intention of verifying with numbers.
One method to increase repeatability is to read a script aloud to each participant.
In the US, if you read aloud to someone, most will quietly endure it. This is not what happened when I attempted to read questions to a Rwandan teacher trainer.
Each participant would wait a sentence or two before interrupting me. “But, we do not have connectivity,” one said, pointing out that he could not use this website now or in the future he foresaw.
He had a good point, and so did others who interrupted my script reading.
…let your curiosity lead you.
It can be easy to be focused on numbers and reproducibility in a place like the US where it is (comparatively) easy to do such things. By doing research in a place where the culture fought against techniques like reading aloud, I was reminded that genuinely engaging with participants can often reveal more insights than a script that was written before we knew what our participants had to say.
We weren’t looking for numbers; we were looking for real stories from real people. Following our intuition got us the information we needed.
Bringing it back to the US (and similar places)
The approaches described here might make you or your team nervous.
Four years later, I occasionally catch myself questioning whether I should try to be more structured. And yet, if for the right topic with the right participants, easing up on rigor can be exactly what a project needs.
Non-reproducible research can work for two reasons.
- First, the team has to be engaged in the topic. The team that went to Rwanda was dedicated to helping the teacher trainers. We were curious about what they had to say, and we followed our curiosity.
- Second, we watched for biases and were practiced in non-leading techniques. Because we had strong interviewing skills, improvised sessions or tours around campuses and libraries provided qualitative data worth analyzing.
If you’re able to bring both of these aspects to a research project, a less structured research plan—with a flexible schedule, on-the-spot recruiting, informal environment, and natural conversations—can work for you, too.
I hope you try it out and let me know what you find!
* To look up these words and more, check out this Kinyarwanda phrase book.
** Which, as it turns out, was not coined by a certain warthog and meerkat.