NYC311 does provide some categorization of service requests. However, these categories are not part of the data. They are only visible when submitting a request through the mobile app or website, or when looking at NYC311's map of recent service requests. Furthermore, the categories provided to the user differ for each of these. In order to assist analysis and understanding of the data, we created our own category column that we added to the data.
Categorizing all the service requests was an iterative process. We originally started with just 12 categories, and had 185 complaint_types as Other. By delving deeper into the distinctions between service requests and soliciting feedback from stakeholders and users when we made changes, we were able to arrive at 16 different categories with just 85 Other complaint_types. You can see all the categories we created, as well as the complaint_types which fall into these categories, on our Categories in NYC 311 Open Data page. If you have any suggestions for additional categories or a re-classification of a complaint_type, please contact us.
We considered the following columns when categorizing the service requests:
In the case of the categories Your Home and Schools & Kids, we placed (almost) every service request assigned to the Housing and Preservation Department (HPD) and the Department of Education (DOE) to that category, respectively. This was because the objectives of these two agencies align precisely with our categories.
For all other service requests, we first checked the complaint_type column and if the appropriate category was not clear, we looked at the descriptor, agency, or, in some cases, location_type of the request. For example, some of the requests with complaint_type "Damaged Tree" had a descriptor which stated that it was leaning or would fall, whereas others had a descriptor saying it had already fallen. We placed the former of these service requests into the Public Health & Safety category. For the latter, if the request also had a location_type which specified the sidewalk or street, we placed it in the Streets & Sidewalks category.
Requests were categorized as Internal Agency Request if either the complaint_type or descriptor specified that the request was for internal agency use. The category Other Information Request was created to distinguish some of the requests for information from the requests for a service. This category includes requests for literature, status updates, and questions for various agencies.
The SQL code we used to generate categories are listed below:
Using Bigquery you can make your own categories super easily in a short amount of time. If you are new to Google Bigquery project, read our tutorial on how to Analyze NYC 311 data using Google Bigquery
Check out our code samples for changing categories below: