What you need to know about conducting research with subject matter experts

When we cross over from building products in the private sector tech world into the civic tech world, we bring our experience and knowledge from consumer research along with us. Most of my experience in startups and corporate roles was researching with consumers — the people trying to buy or use a product or service. One of the biggest changes I experienced moving into civic tech was the pivot to doing the bulk of my research projects with subject matter experts (SMEs). You might also hear folks call them “smees”.

An SME might be a stakeholder. More likely they are a policy expert, or someone who works in an area of government that is related to the product you are building. They might be a leader in the division your product lives under. They might even be a user of your product. SMEs have vast institutional knowledge, and may have seen a product or process evolve over time. They also can connect you with the right folks to fill in any knowledge gaps they or you may have. SMEs are one of your best, most reliable assets in building a product for the public sector. This is not to discount the role of constituents or consumers and their direct experience, of course.

There are some similarities between researching with consumers and researching with SMEs. The similarities are mostly in methodology — meeting people where they are, creating context, and making connections. With both consumers and SMEs, you want to guide a conversation while listening for great tidbits of information that you might not have known to ask about. With SMEs they have a level of expertise and depth of knowledge that you will not be able to anticipate, so deep listening and asking follow up questions on the fly becomes your greatest research skill.

Where conducting research with SMEs tends to differ from conducting research with consumers is around the SMEs depth of knowledge, their motivations to participate in studies, and the level of trust that needs to be established to open their Mary Poppins bag of good stuff.

Here are three things to pay attention to when conducting research with SMEs.

Supporting SMEs in sharing their knowledge across a product ecosystem requires deep trust.

• This level of trust goes beyond building a quick rapport with consumers at the start of an interview and then saying goodbye to the person forever at the end of the session. The objective in establishing trust with SMEs is to establish a long term organizational trust between your SMEs, your product team, other product teams, and any other folks working on the project. Your SMEs will be folks you return to for knowledge and advice, and they’ll need to know how they fit into the ecosystem and what this knowledge will be used for.
• A way to help build this trust is to prepare your team and your product folks for what an interview will be. Work with the team to set goals for conversation. Share out the protocol for review. There’s no quicker way to derail a conversation with an SME than to have stakeholders on a call asking contradictory questions, or introducing any level of people/team politics into a conversation because they weren’t included in the planning.
• Your SME wants to feel confident that your team is building a product that is going to purposefully and thoughtfully implement policy. You’ll help with this by really truly listening to what they have to say and deepening these relationships over time. You can also boost this trust by gaining a basic understanding of the program you’re working on by reading the history of the program or articles about the program, reading through related agency websites, learning from your PO and teammates, and remaining confident in interviews. If you aren’t sure where the SME is going with something, I like using the line, “What I’m hearing you say is X. As someone a little newer to this space, can you go into a little more detail for me?”

SMEs will be more knowledgeable than a consumer research participant.

• While a consumer may sometimes veer off topic, an SME will definitely take you into unfamiliar level of depth on a topic. Stay focused on determining if it’s time to reel in the conversation or if it’s time to let an SME keep talking or even to encourage them to go deeper into a line of thinking. A good PO or PM on the call can help facilitate this depth over a backchannel like Slack while the interview is being conducted. Or if you’re a long standing team with a great rapport, your product folks can be trained to give a signal and step in to the conversation to help guide someone back to the protocol, or they can encourage them with additional questions on a topic you may be unfamiliar with.
• SMEs will have stronger opinions than consumers, especially if they have worked for a long time in the space you’re researching. Watch out for basing product direction off one or two opinions. Walk the line between honoring that expertise while maintaining a strong product vision and always including the lived experiences of constituents in your analysis.
• Conducting research with SMEs is a really a great time to dive into a beginner’s mindset. Our old stand by “tell me more about that” is something you’ll want to always be ready to ask. But also consider trying the question “Is there something I might not know about this subject that is important for us to explore”. I’ve gotten great insights with that question, and it helps build trust as well.

SMEs will have different motivations to participate in research than consumers.

• In building government products or services, everything starts at the top with policy. Policy expert SMEs want to know that the implementation of policy will be useful and usable as well as accurate to the regulations and statutes.
• Stakeholders who are also users will want to participate to help guide what is likely an already difficult offline process to make sure it doesn’t become an even more daunting task online.
• While a consumer will generally let you run the protocol and follow your lead without questioning much, an SME will want to know why you’re speaking with them. It’s a fine line between leading and having open conversations with SMEs. Otherwise they might feel like you’re digging for a “gotcha” or that you just don’t understand what they are saying. You’ll know this is happening when the answers are short, or the participant seems confused by the questions you’re asking of them. If it does, it’s ok to be more open about what you’re studying than you might be with a consumer.

There is one last surprise to be prepared for.

• SMEs will invite other people to sessions. Be prepared to handle multiple opinions and facilitate multiple unexpected voices on the fly. I highly recommend having one or two dedicated note takers to capture all the information and someone to help determine which questions or opinions should be followed up on to get the best information out of a session in the event you have many people on the call (I’ve had up to 8!).

SMEs are the people who know what you need to know to build a fantastic government product. They are civil servants keeping our society and government services running smoothly. It’s an exciting challenge for a researcher to help share the knowledge of an SME with your product team. Let them be your information guide to building excellent government products and services.