User Research Framework

We used an agile product development method as we put together our solution for a one-stop shop for 311 research. Keystone to our process is user research. We conducted interviews with both potential users and domain experts using human-centered design principles, and continued to check in with users throughout the design process to incorporate their feedback. We prioritized three main user personas after surveying our primary users:

Screen Shot 2018-06-13 at 10.15.03 PM.png
  • The Non-Technical Analyst: This user that has domain knowledge of the 311 dataset and why it’s useful, but is not comfortable munging or merging the data independently. This user may be a policy worker, government employee, or graduate researcher in the social sciences.
  • The Technical Analyst: This user is comfortable working with data in a variety of forms, including some coding or spatial analysis, but their work requires them to do some of the same analysis repeatedly. While they may be comfortable wielding python, they may not be as familiar with the use case or the dataset itself. This user may be a data analyst at a non-profit or a CUSP student.

  • The Community Member/ Hacker: This user may have a range of technical abilities, but may lack specific domain knowledge about the 311 dataset. This user may be a hackathon member, active urbanist, or an individual looking for complaint information about their neighborhood.

Speaking to users in each of these groups helped us understand what features are most valuable, and for what purpose. The intention of these interviews is always to get a sense of the problems these groups are trying to solve with 311 data, as opposed to specific, technical features to build. For example, we heard from our users that demographic data is often used to solve aspects of research questions. After hearing how, why, and in what formats researchers use these data, we decided that adding the ability to filter by census tracts could help address this issue. That way, users could break data into a format that could be merged with American Community Survey data instead of needing to query the Socrata API. The functional item is the census tract toggle, but it’s tied to a broader user question regarding demographic data. Our interview guide is a template for these conversations with city employees, researchers, and hackers.  

Over the course of our project, we spoke to nearly ten users, including people from Downtown Brooklyn Partnership, the Furman Center, and the City of Cambridge. Some of the finding surprised us; others we happy to have as evidence to validate assumptions we had about how people tend to use the data. 

For example, we heard time and again that while maps help to see a high-level snapshot of an area, they're not as valuable for actual analysis. Instead, adding additional spatial filters would be more useful to solve our users questions about specific geographies. Through a lit review and market scan, we found several tools that provide maps; we wanted to go farther and make it easier for users to make their own project-specific map separately.

Most notably, we learned that playing with the data was not one of the primary questions from our users. Instead, interpreting -- or trusting -- the data was a main issue researchers faced. With this in mind, we focused on documenting how the dataset is maintained and has evolved over time, as much as on the tool itself. 

As we built the tool and manual, we continued to show users and stakeholders new features to ensure utility. We will continue to do so as this evolves. 

Do you have something to add to the user manual or tool? Contact us

Case Study: NYC 311 Requests in Business Improvement Districts

Our 311 User Manual Approach