There has been a fair bit of noise recently about the “Big Four” racing to provide AI audit services, verifying the effectiveness, safety, and accuracy of artificial intelligence tools. But we’re hesitant to call it “news”. AI has long needed scrutiny and verification from a safety, accuracy, and, most importantly, an ethical standpoint.
We developed our ethical framework before the mass adoption of LLMs, as it was very clear that companies were using machine learning models to deliver outputs based on biased data. This meant their users were exposed to potentially unfair data, disadvantaging certain groups, and creating reputational issues for the companies involved.
How LLMs learn compounds the problem. They’re trained on vast datasets (like the internet) that aren’t always the most ethically sound or factually correct sources. And since these systems continually ingest new data and learn from their users, the challenge of auditing is both complex and ongoing. Governance and regulation will struggle to keep pace in a world where companies are racing to implement AI technology to gain a competitive edge.
So, how can firms take advantage of this tech, whilst still providing a user experience that is true to their values and mitigating risk?
Given the hurdles in controlling an LLM’s output, the most important thing to do is to look at the methodology of the prompts ,and assess how to verify the LLM’s groundedness (a measure of whether the LLM’s output is factually correct, assessing the ability of an AI model to connect its outputs to verifiable, real-world sources) so that users obtain accurate and reliable information.
At Elemental Concept, we’ve been doing this work for some time. But this is about more than just the technical side of machine learning; we bring an ethical lens to AI. We include an ethical framework to ensure the outputs are looking at the right data and actually represent your company values. This way, the tech works for you, not the other way round.
You can learn more about our AI consultancy services here.
