"> Collection: Harnessing data - Introduction from the authors » Socitm

Become a Socitm member today

Join a vibrant community of digital leaders who share a passion for transforming local, regional and national public services.

Find Out More

Become a member  


Harnessing data for better public service outcomes:

An introduction from our authors

The changing role of data

Some of the world’s wealthiest companies have made their fortunes with data. Many people in powerful positions use data to extend their influence and profile in ways never seen before. As individual citizens, we both generate and consume ever increasing amounts of data – physiological data to keep us fit and healthy, transport data to keep us moving, meteorological data to tell us what to wear and when to go out, financial data to manage our spending…

Data opens up new possibilities and futures for public services and for local, place-based public services, in particular. It sheds light on the many complex problems and opportunities that people, communities and businesses face, and it helps to achieve more sophisticated targeting and prioritisation of public policy interventions. 

Data underpins the technologies on which digital government depends – artificial intelligence (AI), robotic process automation (RPA), Internet of Things (IoT) and many other emerging technologies such as drones, autonomous vehicles, virtual reality and blockchain.  

Data plays a critical role in optimising and automating processes, delivering improved outcomes, greater efficiency and more personalised services. 

Data is the key to unlock decision-making processes that previously relied on human intervention. Even where people are still central to the operation of complex service delivery processes, they depend on timely, accurate, linkable, relevant and shareable data in order to perform their tasks. 

None of these opportunities come without risks. Some of the downsides of harnessing data lie in its deliberate and often flagrant abuse for personal or business gain. There are also many other subtle risks in overly relying on data where quality, provenance and purpose are questionable.  

Data in public services

None of these opportunities come without risks. Some of the downsides of harnessing data lie in its deliberate and often flagrant abuse for personal or business gain. There are also many other subtle risks in overly relying on data where quality, provenance and purpose are questionable.

Public services in general and local authorities in particular have a responsibility to harness the power of data for public good and in ways that do not cause unintended, negative consequences. As the dependency on data increases so does public scrutiny of how that data is used and managed. This is particularly true in how data is shared in a growing inter-connectivity between public service partners and agencies.

Data abuse or misuse, intentional or unintentional, can fundamentally undermine public trust in public services as they strive to become more digital. Without public trust in how public bodies use personal data there will be resistance to sharing it.  Such a loss of trust would be tragic. It would hold back the development of integrated services such as health and social care integration. It would undermine the challenge of dealing with troubled families and the causes of crime. And it would compromise the ability of public services to use data to improve performance and efficiency.

The Covid-19 pandemic is changing the perception of citizens and politicians towards the ownership and sharing of personal data for public benefit. Data offers the potential to fine tune the contribution made by our local public institutions to our health and to our economic, social and environmental well-being – critical in a post-Covid-19 and post-Brexit period. Consequently, data is no longer the domain of the experts; today, everyone needs to be competent in understanding data potential, limits and risks, and to play their part in maximising value and protecting data integrity and trust. This applies to politicians, executive leaders and everyone using data in their respective roles to take decisions and to support the work of others.

This collection aims to explore these issues and opportunities in a practical way to aid local public services and councils as they journey towards harnessing data for better public service outcomes. Many public service organisations see value in data, especially to improve services. Fewer see the opportunities to repurpose public services around the outcomes that matter to people and businesses locally, and few are yet managing data well. Too often, data is trapped in departmental silos or in proprietary systems, in different and incompatible formats. Frequently, data are managed with too little understanding of information risks. 

The purpose of this collection

This collection aims to explore these issues and opportunities in a practical way to aid local public services and councils as they journey towards harnessing data for better public service outcomes. Many public service organisations see value in data, especially to improve services. Fewer see the opportunities to repurpose public services around the outcomes that matter to people and businesses locally, and few are yet managing data well. Too often, data is trapped in departmental silos or in proprietary systems, in different and incompatible formats. Frequently, data are managed with too little understanding of information risks. 

The collection is designed to help an organisation and those with responsibilities for data handling to identify the areas where they need support and to confirm where they may have already display good practice. It is also aimed at different audiences – from the specialists who ‘live and breathe data’ in ICT and other technical teams, to those who just rely on good data practice to do their jobs. 

The collection offers insights into: 

  • Founding principles on which good data practice is built 
  • Good practice guidance providing insight and resources for organisations on their harnessing data projects 
  • References and case studies – materials that member organisations can use and contribute to over time. 

How this Harnessing Data guide came about

This collection and the resources that it contains support each of Socitm’s five key policy themes. The collection has been developed jointly with Jisc (formerly Eduserv), a Socitm partner with strong interests in the possibilities offered by data to transform public services and to help improve public services.  Further contributions have come from Splunk, one of Socitm’s Executive Partners. 

The collection is based on extensive research, a survey, two national roundtable events, interviews with experts and discussions at the annual Socitm President’s conference in 2019 where this harnessing data project was formally launched. It also builds on some of the foundation work from others such as Nesta and the Local Government Association (LGA). 

Feedback from Socitm members indicates that a collection of practical resources will be more useful than research into the potential of data to transform services per se. Local public service organisations generally know how important data is but often lack actionable advice and practical guidance about where to start, how to overcome obstacles and risks, and how to access and adopt shareable best practice – Simplify, Standardise and Share. This is why this collection is designed as a web resource – easier to target specific areas than thumbing through a long publication to find the specific advice you need. 

This collection also focuses on some of the biggest challenges and opportunities reported by local public service organisations; it is valuable to know that you are not alone in tackling these intractable challenges and to see how others are approaching them, often quietly and gradually rather than to a fanfare of awards and publicity. 

Local public service organisations are not uniform. There are significant and valid variations in form, function, form and governance. Those who argue that all councils should be the same, for example, because they largely do the same things, do not understand the need for variability to reflect demography, scale, organisation type, politics, and even geography. City councils and a rural borough will have differing challenges. However, there will also be commonalities. This collection seeks to address these similarities in order that there can be a consistent basis for harnessing data and managing risks wherever possible. This is particularly true when considering ethical use of data and methods for data sharing. 

In general, feedback for this collection has indicated that local public services organisations vary considerably in the maturity of their approach to harnessing data. Some are advanced in their thinking and practice, already employing senior data specialists and even Chief Data Officers, with considerable expertise and capacity in data exploitation. Others are more cautious, aware of and enthusiastic about the potential of data, but also fearful of the inherent risks. Many are just not sure where to start. It is for them that this collection is particularly aimed, helping to share best practice and stimulate innovative leadership. 


This collection seeks to help local public service organisations unlock the potential of data, whilst protecting the people and communities that they are there to serve. It seeks to help and inform organisations on their data projects, to anticipate and to avoid risks, and to harness and exploit value from data. It is not a traditional research report (although it contains research findings). Rather, it is intended as a practical ‘living and breathing’ resource that is interactive and updated as local public services use it and contribute to it. In that way it will be a resource for sharing best practice and stimulating innovative leadership. 

Please do offer your experiences and ideas. Contact: Martin Ferguson at martin.ferguson@socitm.net or Nadira Hussain at nadira.hussain@socitm.net. 

Explore this collection


About Jisc:  We provide digital solutions for UK education and research. Our vision is for the UK to be the most digitally-advanced higher education, further education and research nation in the world. We provide UK universities and colleges with shared digital infrastructure and services, such as the superfast Janet Network. We help the sector save time and money by negotiating sector-wide deals with IT vendors and commercial publishers. We offer expert and trusted advice on digital technology for education and research, built from over 30 years’ experience.

Visit the Jisc website

black Splunk logo

About Splunk: We turn data into doing with the Data-to-Everything Platform. Splunk technology is designed to investigate, monitor, analyze and act on data at any scale, from any source over any time period. The Data-to-Everything platform removes the barriers between data and action, so our customers – regardless of size or business – have the freedom to deliver meaningful outcomes across their entire organization. Our unique approach to data has empowered companies to improve service levels, reduce operations costs, mitigate risk, enhance DevOps collaboration and create new product and service offerings.

Visit the Splunk website

About Creese Consulting: Jos Creese is an independent digital consultant, researcher and analyst. As an Associate Director for Socitm and a past president, he has undertaken a range of activities for Socitm. Since founding CCL in 2015, after 30 years as a public sector CIO, he has helped nearly 200 public and private organisations on their digital journey, as a business consultant, mentor and problem solver. 

Visit the CreeseConsulting website

Become a Socitm Member today