Get set to use everything in your membership in 2025

Data skills

Practical insights and inspiration for achieving better public service outcomes. Data lies at the heart of what we do. This collection seeks to help local public service organisations unlock the potential of data, whilst protecting the people and communities that they are there to serve.

Authors and contributors: Martin Ferguson, Jos Creese

Although data science isn’t a new profession, it has evolved considerably over the last 50 years from its early origins grounded in statistical analysis. As early as 1962, mathematician John W. Tukey predicted the effect of modern-day electronic computing on ‘data analysis‘, which resembles what we now call data science. The term “data science” has been traced back to 1974, when Peter Naur proposed it as an alternative name for computer science.

Yet, the data science of today is a far cry from the one that Tukey imagined. Tukey’s predictions occurred well before the explosion of data and the ability to perform complex and large-scale analyses. After all, it wasn’t until 1964 that the first desktop computer—Programma 101—was unveiled to the public at the New York World’s Fair. Any analyses that took place were far more rudimentary than the ones that are possible today.

Step forward to the 21st century where the exponential growth of data and its ubiquity has helped to accelerate the take-up of machine learning and artificial intelligence, extending the power of data analytics. With this has come a boom in data science degrees and professional opportunities for data scientists in businesses and public services.

The need for the application of data skills extends across most services, not just ICT and Finance, to shed insights on the underlying causes of so-called wicked problems facing many of our communities and providing intelligence to re-think service design and delivery. Data can drive productivity, efficiency, performance, and risk management, as well as the delivery of new service models and the extraction of new value from data. The Covid-19 pandemic is a case in point.

For example, being able to transform information into helpful visual presentations makes information more digestible and usable. With that comes responsibility to ensure that data linkages and interpretations are carried out ethically and free of bias. Public services in a place need to start building their data skills capacity and capability now, especially given that the growing demand is likely to drive up the scarcity and cost of recruiting and retaining staff. Apprenticeships can be a valuable method for growing and developing the skills most relevant to the sector.

Opportunities to develop and deploy a data analytics capability come with a word of caution from McKinsey:

“…as analytics comes of age, there are some growing pains. While investments in analytics are booming, many companies aren’t seeing the ROI they expected. They struggle to move from employing analytics in a few successful use cases to scaling it across the enterprise, embedding it in organizational culture and everyday decision-making.

Empowering people with analytics—that’s where the real value creation occurs. And simply having the best data or writing the most cutting-edge algorithm won’t make it happen. It requires a wholesale organizational transformation, complete with robust change management and analytics/AI education programs.”

Source: McKinsey

Integrated into organisations and working with partners across places, data scientists can offer a range of technical and business skills for modern, place-based, digital democracy and services. Good examples include Data Mill North, the London Borough of Barking and Dagenham and the Greater Manchester Combined Authority. At its heart this is about the ability to analyse and interpret data, often using sophisticated digital tools to do so. Data scientists in today’s public services have to make sense of large amounts of data, often in highly complex scenarios. They have to be able to diagnose patterns and trends and from this be able to build valuable insight, business cases and risk profiles of different public services’ strategies and policies. However, in order to make effective use of these insights, data skills will be required across a wide range of public service responsibilities, as set out in the following diagram.

Data skills
Image: Data skills

Some of the roles

There is a wide range of new data roles becoming prevalent in data-driven places. Whilst these roles may be relevant in larger organisations, smaller organisations should seek to build collaborative arrangements with others to gain access to the relevant skills and opportunities to harness data for their purposes. Titles and job descriptions vary. Some common roles and job titles are emerging, while many data responsibilities lie in related roles such as ‘solutions architect’ or ‘big data developer’. Alongside these roles, it will be important for organisational leaders to understand the potential risks and opportunities that come with harnessing data and information assets.

Data skills roles
Image: Data skills roles

The roles with data responsibilities in local public services are many and varied. Some hold recognised specialist roles, others have some data functions within a wider role.

Each of these teams and individuals will have data responsibilities in a data mature public service organisation, with a governance model that joins them together. Data culture and the effective use of data depends on a wide range of individuals and functional areas aware of the risks and benefits, skilled in the tools and methods, and alert to the possibilities.

Data skills for the future

In the past, organisations have typically hired people with skills who can extract data from a system such as the ERP, making sense of the data, entering it into reports and dashboards, generating key performance indicators and producing analysis.

Today, the diversity, complexity and volume of data (and systems) has increased considerably, with data amalgamated from social media, wearable devices, third-party data sets and many other data sources. Skills are required beyond extracting data and compiling reports, from specialist IT roles to decision-makers.  And in future, this will extend to the ability to integrate cognitive machine learning capabilities to augment judgement and derive new insights from data.

Whilst organisations may understand that data analytics can deliver great value, they often struggle to embed analytics into existing roles and responsibilities, since this requires job changes, restructuring and potentially new rewards systems. Understanding the changing interaction between service platforms, their use and the data that flows between them can help with structuring place-based governance models, skills profiles and digital planning in general.

Data circle graph
Image: Data circle graph

Realising the full potential of data takes more than intelligent platforms and processes. It requires new structures for data assets and analytics, governance and prioritisation.

Using SFIA

A variety of information skills frameworks exist to help to classify different levels of the information and data skills. These can help to grade roles as well as to plan for skills development, following industry best practice. One of the best known is the ‘Skills for the Information Age’ framework which lists a range of accredited roles and measurable skills in this area. The framework is particularly relevant for leveraging specialist data skills in IT, Finance, and systems design roles.

Data skills SFIA foundation
Image source: SFIA Foundation

Data scientists and auditors

As public service organisations become increasingly dependent on the value of the data they collect and use across places, auditors will need to keep pace.

With each new generation of smart and connected data tools that mine, link and present data, so data value increases, as well as becoming more distributed and difficult to track. Emerging technologies such as artificial intelligence are only just beginning to be used and, in conjunction with burgeoning data sources including machine learning, robotic process automation and the ‘Internet of Things’, the role of the auditor has never been more challenging.

Auditors will need a multidisciplinary knowledge of data management, inference, analytical and forensic tools. They will grapple with an increasingly distributed network of systems and cloud services, with critical data spread across multiple systems, providers, organisations and physical locations. These make it much harder to track and monitor data flows and the associated risks. In the future, as machines themselves start to determine how data should be shared and linked in order to meet individual citizen needs, with no or minimal intervention necessary, auditors need to be sure the data flows are appropriate, and that all the necessary ‘checks and balances’ are in place to avoid unintended or deliberate bias and misuse.

Auditors will also need to get to grips with so-called ‘dark data’. According to new global research for Splunk by TRUE Global Intelligence, 55% of an organization’s data is “dark” – unquantified and untapped. Such data represent both a risk and a cost in IT processing. This data needs to be exposed and dealt with to remove consequent costs and vulnerabilities.

Data science circle graph
Image: Data science graph

Digital literacy and citizens

We are experiencing growing levels of technical knowledge, digital literacy and understanding, thanks to the ubiquitous nature of smartphones, online shopping and social media. But many are still being left behind – the very term ‘citizen’ is an exclusive one that excludes many who do not enjoy citizens’ rights and responsibilities. For a Socitm perspective on bridging the digital divide, see our three-part series.

The public’s level of concern regarding online data privacy is often not reflected in the steps they take to secure their personal information according to a report from the Carnegie UK Trust. Furthermore, the behaviour that they consider to be secure is not always so.

Many people are happy to accept risks they can’t see, in favour of a range of service benefits, for example when they shop online or use social media. Perhaps wrongly, they will tend to trust private companies with their data more that they trust public bodies. A lack of trust results in citizens being reluctant to use electronic services or to share their data or allow it to be linked to enable service integration.

In a digital world we all need to be able to take more responsibility for our own data. This is particularly true in the UK where the size of the State has reduced by at least 1/3 in the last decade, and many services previously run by the public sector are now delivered only by the private sector.

We all have access to an enormous amount of data on the internet, and that has increased in terms of personal data collected about us, much of which is automatically collected from apps on our smartphones and wearable devices.

A smart phone connected to a biometric smart watch can connect medical data, location, habits, activity, interests and buying preferences. This allows organisations, whether public or private, to deliver more personalised services, but it also opens up the possibility of fraud and abuse.

The National Cyber Security Centre offers a range of advice, including some targeted specifically at individuals and families. There is a role for public services in helping the public understand the risks as well as the benefits when, for example, our phones become our personal health and wellbeing clinic, monitoring our body’s chemistry, with data and triggers for interventions, links to GPs, specialist health checks and connections to equipment in the home.

Public service organisations need to engage their marketing and communications teams in this, explaining how data is used, as well as helping the public to have a mature approach to managing and protecting their own data.

Case Studies

Helpful external resources