The management of big data is one of the most critical components of any future city.

Cities generate data. By analysing this data, agencies can gain the knowledge to improve services and infrastructure – and make life better for residents.

Professor Peter TriantafillouAccording to Professor Peter Triantafillou, the new Chair of Data Systems Engineering in the School of Computing Science, you can only analyse the data when you have the infrastructure and systems in place to manage it effectively. His research is focused on unlocking the potential of big data by improving the systems that capture it, store it and return knowledge from it.

“Most people, when they talk about big data, they talk about being able to analyse it and get answers. Analyse what? The data? Well where is the data?

“Somebody has to store it, curate it, manage it and make sure it’s consistent. Someone has to make sure that the access to it is reliable and fault tolerant. Someone needs to ensure that the systems can scale to remain functional as the datasets grow with time.”

The data is vast and comes from many sources. Councils accumulate detailed information about housing and residents. Environmental monitoring provides data on water and air quality. Sensor systems capture energy use and traffic movements. Hospitals collect information on disease. Retailers capture shopping trends. CCTV cameras record a host of behaviours. Social media use identifies many other issues important to residents.

The potential for using this data to improve the lives of residents is remarkable, but there are also many challenges.

According to Professor Triantafillou, “the problem is that our information needs are always changing and they only pertain to a tiny, minute amount of this huge massive data set. My research tries to make sure that data is consistent and that it can answer all the queries of the user community in an efficient manner. Datasets grow with time and the user community grows with time. I build systems that will scale to accommodate that.”

Cities need to draw upon a level of data systems infrastructure and management that they may not be prepared for or used to. For example, they need the ability to associate disparate datasets on-the-fly from sensors deployed for environmental, traffic or personal health applications. This ability supports the testing of hypotheses, finding positive/negative correlations  between traffic patterns and health issues for example, or climate changes and environmental quality.

“Then, how do you ensure scalability in this setting where datasets and the user communities grow? How do you use all of your resources so to have a good return of IT investment? How do you manage your costs? How do you manage security issues and privacy issues? These are really at the top of the agenda.

“We are making progress on these questions, but the exponential growth of big data means it is always a moving target and we have to keep chasing it.

 “Typical small-to-medium enterprises or local governments do not have this know-how, they need data management people to come in and help.”

We, as a community, need to come together and invest in such efforts.

Related pages

 

First published: 5 November 2013

<< Projects