As financial institutions deploy AI to interact directly with customers, proving that these systems deliver appropriate outcomes is becoming a growing compliance challenge. Aveni is addressing this through an approach that combines pre-deployment testing, real-time monitoring and post-interaction quality assurance to help regulated firms identify risks throughout the AI lifecycle.
As part of the AI in Risk and Compliance 2026 report, co-authored by RegTech Analyst and Parker & Lawrence Research, the research team examined how firms can strengthen oversight as AI becomes more deeply embedded in regulated financial services.
Parker & Lawrence Research interviewed Aveni, a provider of AI-powered oversight and assurance for customer interactions, about the challenges of evaluating AI systems that operate directly with customers.
Traditional AI testing can establish whether a model performs against a benchmark or withstands a particular technical failure, but these measures do not necessarily demonstrate how a system behaves during a real customer interaction. An AI agent might perform well in controlled testing while still failing to recognise vulnerability, responding inappropriately to a customer’s circumstances or contributing to an unsuitable outcome.
The challenge is becoming more urgent as deployment accelerates. Research cited in the report found that 65.6% of reported AI activity had reached production or beyond, while 69.3% of institutions said deployment had outpaced their measured control environment. Existing model risk and quality assurance frameworks may struggle to keep pace with AI agents that conduct extended, dynamic conversations. Pre-deployment testing can miss unusual combinations of language and customer circumstances, while manual reviews and sample-based quality assurance may fail to identify serious but infrequent conduct risks.
Aveni argues that financial institutions need oversight that operates across the entire customer interaction, without introducing excessive latency or operating costs. Its approach is built around three products: Approve, Assure and Detect, covering testing before deployment, monitoring during live conversations and quality assurance after interactions have concluded.
Approve connects to customer-facing AI through an API, allowing firms to simulate conversations using selected customer personas and risk scenarios. The resulting dialogues are assessed against criteria including vulnerability, advice boundaries, harmful language, customer understanding and escalation. This gives model and risk committees evidence of where a system passes or fails and why, both before its initial release and following subsequent model changes.
Assure focuses on monitoring live conversations through a real-time conduct risk guardrail. Faster, task-specific models carry out initial checks within the available latency budget, while higher-risk exchanges can receive more detailed analysis. Depending on the circumstances, the system can allow a conversation to continue, alert a human reviewer, redact content or stop the interaction.
Detect extends oversight into post-interaction quality assurance, assessing voice, chat, email, documents and related records. Rather than relying solely on samples, it applies a firm’s own quality assurance and Consumer Duty framework across entire cases, retaining evidence for review and remediation. The product can also identify potential upselling opportunities and early indications of customer churn.
Aveni reports that one quality assurance use case reduced assessment time from 90 minutes to 15 minutes. The company also reported 61% faster time to first token, a 3% improvement in F1 accuracy and 92% lower running costs than Gemini Flash 2.5 Lite in benchmarked use cases. These figures are supplied by Aveni and relate to its specific tests.
Aveni chief commercial officer Kent Mackenzie said, “The bank that solves the oversight problem will be the bank that wins the race to become the most agentic bank.”
Parker & Lawrence Research highlights Aveni’s combination of customer outcome-focused oversight and specialist engineering. The company has trained its own small language models on financial services data and orchestrates more than 150 models, ranging from low-latency classifiers to specialist financial services models and selectively deployed frontier models.
By making testing across entire populations of interactions more practical, Aveni’s approach could help financial institutions strengthen oversight without relying entirely on manual reviews. As banks explore more autonomous customer-facing AI, demonstrating that systems behave appropriately in real-world situations may become as important as their technical performance.
Read the original post from Parker & Lawrence Research here.
Copyright © 2026 FinTech Global
