Award-winning project SmartData Platform: solving problems before they start

Chicago, United States

The SmartData Platform is an open source predictive analytics tool that enables data-driven decision making to ensure city operations are smarter and more efficient. 

City governments make decisions that impact citizen life every day. How can we protect children from danger? How can we safeguard our citizens’ health? How can we keep animal populations under control? As urban populations grow and demand for public services increases, governments need to find new ways to make the best and most efficient decisions.

The SmartData Platform project is the first open-source, predictive analytics platform for municipal government – aggregating and analyzing information to help leaders make smarter, faster decisions to solve important urban problems. The SmartData Platform will give city leaders a tool to search for relevant data and detect relationships, analyzing millions of lines of data in real-time. The predictive analytics platform will provide city personnel with the ability to more efficiently deploy city resources and deliver services to address public safety and social challenges more proactively.

Award-winning project

This project was awarded the 'Bloomberg Philanthropies’ Mayors Challenge' in 2013. Learn more about the award.


Tags

City information
City
Chicago

Size and population development
As of the 2010 census, there were 2,695,598 people with 1,045,560 households living in Chicago. Chicago is one of the United States' most densely populated major cities- with 12,750 people per square mile. More than half the population of the state of Illinois lives in the Chicago metropolitan area.

Population composition
As of the 2010 census, the racial composition of the city was: 45.0% White (31.7% non-Hispanic whites); 32.9% Black or African American; 28.9% Hispanic or Latino (of any race); 13.4% from some other race; 5.5% Asian; 2.7% from two or more races; 0.5% American Indian

Main functions
The city of Chicago covers an area of 60,000 hectares and sits 176 meters above sea level on the south-western shore of Lake Michigan. At 190km wide and 495km long, it is the 5th largest body of fresh water in the world. The city is traversed by the Chicago and Calumet rivers. Chicago’s extensive parklands include 3,000 hectares of city parks. Chicago is famed for its bold architecture, the city skyline is punctuated with many skyscapers such as the iconic John Hancock Centre, Willis Tower and the neo-gothic Tribune Tower.

Main industries / business
Chicago is a major world financial center, with the second-largest central business district in the United States and the third-largest gross metropolitan product in the United States. The city and its surrounding metropolitan area contain the third-largest labor pool in the United States with about 4.48 million workers, as of 2014. The city of Chicago hosts 12 Fortune Global 500 companies and 17 Financial Times 500 companies and three Dow 30 companies: aerospace giant Boeing, Kraft Foods and McDonald's. Additionally, tourism, education, technology, manufacturing, publishing and food processing play major roles in the city's economy.

Sources for city budget
National, State and City governments taxation revenue.

Political structure
The government of the City of Chicago is divided into executive and legislative branches. The Mayor of Chicago is the chief executive, elected by general election for a term of four years, with no term limits. The mayor appoints commissioners and other officials who oversee the various departments. As well as the mayor, Chicago's clerk and treasurer are also elected citywide.

Administrative structure
The City Council is the legislative branch and is made up of 50 aldermen, one elected from each ward in the city. The council takes official action through the passage of ordinances and resolutions and approves the city budget.

Is this city profile not up to date? Suggest changes.
Background and objectives
Chicago is the third largest city in the United States covering 237 square miles and housing 2.7 million residents. The City of Chicago collects 7 million rows of data a day - on everything from weather to traffic patterns to the location of libraries, schools, and public parks, criminal activity and maintenance management. Most of the data exists in separate systems often in conflicting and confusing formats that limit the ability to predict outcomes and allocate resources. 
 
The SmartData Platform allows policymakers to make sense of the city’s billions of lines of data stored in disparate systems. Managers are able to find answers to specific questions without having to manually search for data, or even know where or how the data is stored. End users do not see the millions of lines of code, or the billions of records stored and searched for their benefit. All a user sees is a simple search screen to perform the query. Results are presented in easy-to-read formats, including geographical plots, customized to the user’s preferences (police beats, school districts, sanitation districts, etc.).
 
Key objectives of the SmartData Platform are:
  • enable city departments to engage in predictive problem-solving
  • allow policy makers to visualize and make sense of trends and patterns in the billions of lines of data stored in city systems
  • share open source platform and tools with cities that cannot build the capacity themselves so they can develop their own predictive analytics 
Implementation

After winning the Mayors Challenge, the SmartData Platform team (SPD team) generated additional resources in the form of pro bono agreements with corporations and civic tech organizations, as well as university partnerships, to develop the complex data modelling and analytical software that sits behind the SmartData Platform.

The project leadership conducted meetings and planning sessions with commissioners and cross-departmental teams to identify over 100 potential projects that the SmartData Platform could support. Detailed follow up conversations with departments identified the operational areas that would most benefit from data-driven models.

The SDP team works with city departments to collect and analyze the data at their disposal.

The team’s data scientists use advanced analytics and machine learning to uncover many new ways of tackling the big issues faced by city governments. Based on this analysis, the team produces a detailed plan of action which they work on in conjunction with departments to implement.

The predictive power of the tool is its ability to analyze relationships in the data at a speed and on a scale not previously possible. For example, the SmartData Platform could query data on traffic patterns and pedestrian activity for a certain section of the city, and then compare it against other city data, such as weather patterns, traffic signal times and streetlight access. By doing so, SmartData might develop a prediction of where intervention is needed to reduce pedestrian-traffic collisions. The city could optimize services of all kinds in this way, benefitting citizens and reducing costs. 

To aid eventual replication, the team created a public repository of source code for data models along with explanatory documentation. This also allowed other data scientists to suggest improvements to the models. 

Financing and resources

The City of Chicago is the lead agency for the SmartData Platform and provides recurrent funding for the project.

Project partners include the Urban Centre for Computation and Data (UCCD) and Carnegie Melon University.

Bloomberg Philanthropies through the Mayors Challenge, awarded the City of Chicago $1 million for the development of the SDP.

Results and impacts

The City of Chicago’s investment in analytics has already improved the way the city operates.

Both rat baiting and restaurant inspection models are prime example of how the new platform improves efficacy and safety for the population

  • The rodent abatement model made city rat baiting teams 20% more efficient, leading to improved health and wellbeing for residents in affected areas. 

  • the restaurant inspection algorithm helps inspectors find critical violations of health code 7 days faster, preventing foodborne illness and making restaurants safer. 

The platform’s potential impact also includes crime reduction and prevention, preventative human service strategies on subjects ranging from public health to homelessness, infrastructure maintenance, and the delivery of core services.

Barriers and challenges

The biggest surprise and biggest challenge were the importance of middle managers. Without their buy-in, the project would not have moved forward. At first, some departments didn’t understand the technology or its purpose. This made them cautious. Other middle managers were resistant because they felt that the platform undermined their decision-making authority. To tackle this challenge going forward, middle manager buy-in has now been included in the criteria for working with a new department.

Lessons learned and transferability
  • many departments must focus on day-to-day operations to the exclusion of data analysis. To develop a predictive analytics model for the department, the team must gain a deep understanding of their processes. A manager directing the work of inspectors has a very different mind-set to data scientists. Communicating effectively across these cultures is essential for success.
  • showcase positive results through a real example, both internally and externally to build credibility and demand.
  • leveraging pro bono resources requires investing time to nurture and manage these relationships. Clearly defining roles is important.
  • proactive project management by the team helps keep problem definition on track. 
  • the SmartData Platform utilizes an open-source data infrastructure and set of algorithms that other cities can re-use with no startup software development costs.  Other cities can import the architecture and the predictive algorithms (both logic and source code) and adjust it to their own related data sets.  An archive of instructive documents and templates gives other cities a roadmap to develop their own predictive and analytics projects. 
References

External links / documents