NYC311 Data 101

So what's the 411 on the 311?

The 311 data set available on the NYC Open Data portal is a collection of the service requests submitted through the NYC 311 service. The dataset consists of an aggregated file for 2010 through the present, which is updated daily. The data table consists of 41 columns and, as of May 8th, 2018, 17,538,526 rows. Each row represents a single service request. The attributes for each request include a unique identifier, the date of the request, a first-order complaint type as well as a more specific complaint description, the city government agency responsible for responding to the complaint, a description of the resolution and the date the service request was closed by the responding agency, and numerous locational fields. These location fields include an xy-coordinate and lat/lon-coordinate, address information, cross and intersection streets, and zip code and community board, and various additional fields that are filled in or left null depending on the location of the request. 

Who puts it together and how?

Requests are logged either through a call center, website, or mobile app. Based on complaint type, they are then sent to a specific city agency for resolution. These agencies include the departments of Transportation, Parks and Recreation, Housing and Preservation, the NYPD, and others. Information from all responsible agencies on all the complaints is then sent to the Department of Information Technology and Telecommunications (DoITT) which compiles the data and pushes it up to the Open Data portal daily. The data is available for download in CSV, XLX, and XML formats, as well as RSS or through an API. There are no limits to how many calls can be made through the API. There is a default to how many records are returned per API call, but this can be modified through the search query.

Where to access and explore the data?

Socrata serves as the engine behind NYC OpenData where users have access to a vast library of resources at the Socrata Customer Center which provides tutorials on how to create visualizations, maps, charts, filters and exploration tools.  Outside of this environment, Google Cloud’s BigQuery provides a dynamic web UI and command line platform with sample SQL queries that pulls directly from the NYCOpenData.  Reto Meier’s The Rise and Fall of New York City (311 complaints)  showcases how BigQuery can be utilized to create an effective summary analysis by examining 311 data from 2010. Not to be outdone, IBM proposes the use of Cognos coupled with Watson Analytics for more advanced data visualizations, discovery and predictive analytics.   Datawatch further explains how IBM’s tools can be further optimized with their data processing tool: Monarch. For more advanced technical geocoded analysis, the Department of  City Planning offers the GeoClient API that maps six types of location requests  such as an addresses, block or intersection to the BYTES of the BIG APPLE which lists all geographic data in the city.  While these tools are highly advanced, the requirements for technical know-how can be a deterrent for some if not most users.

Learn more about NYC311 on the city's website.

311: The little big data that could

Case Study: NYC 311 Requests in Business Improvement Districts