User Interview Guide

One of the most important aspect of our product discovery was a series of user interviews. We've re-posted the interview guide here for reproducibility, including interview best practices and our actual interview guide template. After the interviews, we compiled what we heard into themes that helped us prioritize the features that would address users' most pressing issues. 

User Interview Guide - NYC 311 Open Data

This guide is intended for one-on-one or two-on-one interviews with researchers that use 311 data in their work.The goal of these interviews are to unpack use cases by (1) identifying user challenges when using NYC 311 Open Data; (2) identifying user motivations for using the data; and (3) brainstorming potential opportunities/features. Insights from these interviews will be clustered to inform potential product opportunities with the stakeholders, after which the team can map these user insights to functional features.

Interview design

Logistics and key recommendations

 

  • 45 minutes (book a space for one hour)
  • Room should be quiet, one-on-one environment like a conference room (it’s harder to take notes and get the information you need in more informal environments like coffee shops or public tables) 
  • Take paper notes; don’t be on your computer for note-taking unless it's essential for tech demo
  • Consider asking to record the interview so you can pull out direct quotes later, or experiment with transcription services like Cassette 
  • Have paper or a whiteboard ready to help map out / write down their user journeys with them
  • Be okay with letting them fill silence!

Interview template

  • Introduce yourself! This seems obvious, but start with a formal intro of your experience and the goals for the 311 capstone project. Frame this in a way that makes it clear why what they share will be useful. Make your spiel so the user knows when you’re shifting into “interview mode.”
    • “We want to understand how real people use this information and the existing tools they use to do so. That way, we can build something that can help what you’re actually trying to solve with these data.”
  • Begin with introduction about the goals of the session. Emphasize that the goal is to get information about the problems they’re trying to solve with this information, not necessarily how their using 311 data in its current form.
    • Encourage them to use language and examples from their personal experience/ personal research projects, e.g.,“I use OpenData portal to filter the data to my community district of choice, then download as a csv” instead of “People at Stern generally download the data and do what they need from there to answer their questions.”

One Stop Shop Question Guide

  1. What questions does 311 data help you to solve?

    • Step back and think of the larger implications of your study: What are you trying to do?

      • Identify a specific example that you can use as a case study for the rest of the questioning.
    • How do you access the data you use (311 and beyond)?

      • Outline all touch points. Consider drawing/ mapping them out as they’re described.

    • What are the pain points?

      • List pain points under the touchpoints discussed.

      • Pain points could be things like data munging, getting data for a specific area, missing data, or data quality issues.

      • Think of two types of pain points:

        • Technical pain points

          • Data understanding/ identification pain points

      • What did you do to resolve the issue?  Any lessons learned?

    • When working with 311 data, are there tasks that you do repeatedly?

      • Here, we’re getting at what type of analysis could be automated. But, at this stage it’s more about how they’re ultimately using that analysis and data, and less about their current workflow.

    • What’s the best data format to answer your particular questions?

      • Use a specific example and walk through the formats that are most useful, and why, e.g. are some things best as: a visualization/plot? a map? A filtered csv? Why?

        • Formats could also be defined as dimensions/bins of data, granularity of attributes, or density of filter

        • How would you structure these in priority order?

  2. What other data sets do you use to answer your research question? 

    • If they use other datasets in conjunction with 311 often, ask about the purpose as well.

    • If there’s time, consider asking them to show you how they interact with other tools/datasets that provide supplementary info for this research that tool during the second half of the interview. Ask them how they might merge with 311.

  3. Conclusion: If they could snap their fingers and have one thing to help integrate 311 findings into their research, what would it be?

    • End with an open-ended “blue sky” session

 

Sample Data Log

To ensure consistency between interviews over time, use a data log to record main takeaways. The idea is not to record ever detail, but to highlight the major themes within specific use cases. These should be more bullet points than complete sentences. Later, these may be themes that are used as the basis for functional items on a features roadmap or MVP.


 

 

Analyze NYC 311 Data with Google Bigquery

311: The little big data that could