Chicago, United States
The SmartData Platform is an open source predictive analytics tool that enables data-driven decision making to ensure city operations are smarter and more efficient.
City governments make decisions that impact citizen life every day. How can we protect children from danger? How can we safeguard our citizens’ health? How can we keep animal populations under control? As urban populations grow and demand for public services increases, governments need to find new ways to make the best and most efficient decisions.
The SmartData Platform project is the first open-source, predictive analytics platform for municipal government – aggregating and analyzing information to help leaders make smarter, faster decisions to solve important urban problems. The SmartData Platform will give city leaders a tool to search for relevant data and detect relationships, analyzing millions of lines of data in real-time. The predictive analytics platform will provide city personnel with the ability to more efficiently deploy city resources and deliver services to address public safety and social challenges more proactively.
This project was awarded the 'Bloomberg Philanthropies’ Mayors Challenge' in 2013. Learn more about the award.
Background and objectives
- enable city departments to engage in predictive problem-solving
- allow policy makers to visualize and make sense of trends and patterns in the billions of lines of data stored in city systems
- share open source platform and tools with cities that cannot build the capacity themselves so they can develop their own predictive analytics
After winning the Mayors Challenge, the SmartData Platform team (SPD team) generated additional resources in the form of pro bono agreements with corporations and civic tech organizations, as well as university partnerships, to develop the complex data modelling and analytical software that sits behind the SmartData Platform.
The project leadership conducted meetings and planning sessions with commissioners and cross-departmental teams to identify over 100 potential projects that the SmartData Platform could support. Detailed follow up conversations with departments identified the operational areas that would most benefit from data-driven models.
The SDP team works with city departments to collect and analyze the data at their disposal.
The team’s data scientists use advanced analytics and machine learning to uncover many new ways of tackling the big issues faced by city governments. Based on this analysis, the team produces a detailed plan of action which they work on in conjunction with departments to implement.
The predictive power of the tool is its ability to analyze relationships in the data at a speed and on a scale not previously possible. For example, the SmartData Platform could query data on traffic patterns and pedestrian activity for a certain section of the city, and then compare it against other city data, such as weather patterns, traffic signal times and streetlight access. By doing so, SmartData might develop a prediction of where intervention is needed to reduce pedestrian-traffic collisions. The city could optimize services of all kinds in this way, benefitting citizens and reducing costs.
To aid eventual replication, the team created a public repository of source code for data models along with explanatory documentation. This also allowed other data scientists to suggest improvements to the models.
Financing and resources
The City of Chicago is the lead agency for the SmartData Platform and provides recurrent funding for the project.
Project partners include the Urban Centre for Computation and Data (UCCD) and Carnegie Melon University.
Bloomberg Philanthropies through the Mayors Challenge, awarded the City of Chicago $1 million for the development of the SDP.
Results and impacts
The City of Chicago’s investment in analytics has already improved the way the city operates.
Both rat baiting and restaurant inspection models are prime example of how the new platform improves efficacy and safety for the population
- The rodent abatement model made city rat baiting teams 20% more efficient, leading to improved health and wellbeing for residents in affected areas.
- the restaurant inspection algorithm helps inspectors find critical violations of health code 7 days faster, preventing foodborne illness and making restaurants safer.
The platform’s potential impact also includes crime reduction and prevention, preventative human service strategies on subjects ranging from public health to homelessness, infrastructure maintenance, and the delivery of core services.
Barriers and challenges
The biggest surprise and biggest challenge were the importance of middle managers. Without their buy-in, the project would not have moved forward. At first, some departments didn’t understand the technology or its purpose. This made them cautious. Other middle managers were resistant because they felt that the platform undermined their decision-making authority. To tackle this challenge going forward, middle manager buy-in has now been included in the criteria for working with a new department.
Lessons learned and transferability
- many departments must focus on day-to-day operations to the exclusion of data analysis. To develop a predictive analytics model for the department, the team must gain a deep understanding of their processes. A manager directing the work of inspectors has a very different mind-set to data scientists. Communicating effectively across these cultures is essential for success.
- showcase positive results through a real example, both internally and externally to build credibility and demand.
- leveraging pro bono resources requires investing time to nurture and manage these relationships. Clearly defining roles is important.
- proactive project management by the team helps keep problem definition on track.
- the SmartData Platform utilizes an open-source data infrastructure and set of algorithms that other cities can re-use with no startup software development costs. Other cities can import the architecture and the predictive algorithms (both logic and source code) and adjust it to their own related data sets. An archive of instructive documents and templates gives other cities a roadmap to develop their own predictive and analytics projects.