Artificial intelligence (AI) and intelligence automation (IA) are no longer future technologies for the public sector. They’re already embedded in systems being used every day at work. And they’re also increasingly shaping how people experience services.
From AI-powered virtual assistants to chatbots to data-driven services, AI is becoming part of the digital fabric of local government.
The potential benefits are significant. So too are the risks. Particularly if equality, inclusion, and accessibility are treated as secondary considerations, rather than at the core of design and implementation.
📌 AI and IA can improve equality, inclusion and accessibility. But without careful and continual scrutiny there's a risk they can also unintentionally reinforce existing inequalities or create new barriers for disabled people.
The legal context still applies
An Equality Impact Assessment (EqIA) for using AI and IA, carried out by Norfolk County Council, revealed some key considerations. Public bodies operate within a clear legal framework:
- The Equality Act 2010 places duties on public bodies as employers and service providers to ensure people are not discriminated against. It requires us to “give due regard” to equality and make reasonable adjustments to remove barriers for disabled people so they can access services and employment.
- Digital services must meet public sector accessibility regulations, with websites and systems aligned to WCAG 2.2 Level AA.
These regulations ensure that online services are becoming much more accessible to disabled people.
AI-enabled services are not exempt from these requirements. Whether they’re delivered through third-party platforms or embedded into existing products.
As AI becomes more integrated, local authorities must continue to take careful account of their legal responsibilities and consider what they must do. Not just at the point of the interface with users, but also with regards to the data, logic, and assumptions that sit behind and enable automated processes.
Adding real value
When designed and deployed well, AI and IA can offer genuine benefits for employees and residents.
| For employees | For residents |
|---|---|
| IA can: – reduce the administrative burden – free up time for more direct work with service users and colleagues | AI can: – support independence at home – help reduce social isolation – enable tailored services that can quickly understand and improve user experience |
| There is growing evidence that AI-enabled tools can better support disabled employees to perform tasks effectively. | For example, through Smart Home applications which can enable people to easily and safely control their living environment. |
Data-led systems can quickly identify vulnerable people earlier, enabling faster, tailored preventative interventions.
At a population level, IA can build a more detailed understanding of local communities’ needs and experiences, supporting better more inclusive and tailored services.
This explains why adoption is accelerating. However, this value is only fully realised when equality, inclusion and accessibility are built in from the beginning.
Risks beneath the surface
📌 When AI is viewed as a fast, standalone solution, rather than a tool requiring human judgement and oversight, it increases risks to equality, inclusion and accessibility.
Biased or incomplete data can distort outputs
When designing AI and IA systems, if demographic data is missing, such as information on disabilities, health conditions, ethnicity or culture, some groups can effectively become invisible in automated decision-making.
Practical barriers
People may be excluded if they cannot interact with chatbots or where automated interfaces that fail to account of people’s different access needs. Incorrect assumptions about digital literacy, device access or physical ability can significantly limit people’s ability to use services designed to support them.
AI may fail to recognise human cues of distress
Creating greater safeguarding risks by giving incomplete or incorrect advice to people at risk of harm. It may contribute to unhealthy dependency for those already socially isolated, who could form overly personal connections with AI systems or become more vulnerable to individuals seeking to exploit or harass them through AI mediated interactions.
No human in the loop
When AI is used to prepare briefings or communications, without clear guidance on inappropriate or exclusionary language or imagery, it can reproduce stereotypes or negative assumptions about groups already at risk of marginalisation.
When design decisions exclude people
Some hypothetical examples:
An AI-enabled system is procured to support people with learning disabilities to manage day-to-day tasks.
- The design team has not considered that many service users also have long-term health conditions affecting their mobility, sight and dexterity. Or that some are deaf and use BSL.
- Because the system offers only one type of interface, some users cannot interact with it at all, while others can access only limited support.
- The AI’s suggested solutions are not tailored to the group’s diverse needs.
AI is used to analyse population experiences to shape preventative services.
However, gaps in data about local ethnic, cultural and faith diversity mean the needs of some communities are not fully reflected in the system’s design, reducing the cultural competence of the services and support on offer.
AI is increasingly used on websites to provide immediate customer support.
Where customers have complex queries or cannot recognise or interact with chatbots and online reporting systems because they are not accessible by design, it becomes much harder to understand the information or support available. Some people risk being excluded altogether because they cannot penetrate through the system.
Taking a leadership role for digital and ICT
The task might be to slow down just enough to ask the right questions, even as innovation accelerates. Where this work is done well it can widen access and improve outcomes and create greater equality.
AI done carelessly risks entrenching exclusion behind a veneer of efficiency. The foundations matter, and they must be laid early.
In summary… build equality and accessibility in from the start
Bias in, bias out
One of the strongest messages from the EqIA is that AI and IA systems must be scrutinised in the same way as any other process.
Human biases, which we all have, can significantly affect both inputs and outputs. If inequalities are enabled in data or design, technology is likely to reproduce these and potentially do so at scale.
So how do we address this? The EqIA highlights some core principles:
1. Data gaps should be acknowledged and addressed where possible.
2. Diverse service users should be present. If this is not possible, their experiences should be fully represented from the very beginning of a design or procurement process. Not retrofitted at the end. Involving diverse service users only at the testing or rollout stage is often too late. By then, key decisions may already have locked out some groups of people.
3. Equality, inclusion, and accessibility are not optional extras. They're legal duties, ethical obligations, and essential to maintaining public trust.
4. Scrutiny should be continuous, not a one-off exercise. Inputs and outputs must be checked with the same rigour as any other system.
5. Equality in outcomes should be measurable and accessibility for all should be a core feature of the system.
AI and IA can bring real benefits, but it should never replace professional judgement or human accountability. It works best as an enabler, not an answer in itself.
This technology must be consciously and responsibly adopted.
You might also like to read
- AI@Socitm
- Guide: Guidelines for elected members on the use of AI
- Template: Sample terms of reference for an Artificial Intelligence Governance Board
- Template: Policy for the use of generative artificial intelligence
- Blog: Shadow AI in the public sector: innovation without oversight?
- Blog: The threats and opportunities of ChatGPT and artificial intelligence applications
- Guide: Using generative AI large language models – Dos and dont’s
- Whitepaper: AI: what senior leaders in local government should know
