RBI mandates AI kill switch for banks in sweeping new governance rules

The Reserve Bank of India (RBI) has dropped a bombshell on the financial sector, proposing a mandatory AI governance structure that forces banks and regulated entities to install kill switches for machine-learning models. The draft guidelines, released today, require institutions to immediately shut down any artificial intelligence system that produces harmful or erroneous outputs, marking a historic regulatory intervention into algorithmic risk management.

The RBI’s interests

The proposed rules and guidelines to mitigate these AI risks drawn up by the RBI are open for public feedback until July 24. These guidelines cover every model used in bank operations, ranging from basic spreadsheet calculators to complex generative AI systems.

Banks would have to keep inventories of all models in their operation, assess risk at both the individual and enterprise level regularly, and take corrective action when risks cross acceptable thresholds. The modes of corrective action can range from tighter controls to complete decommissioning of the model, with well-detailed reports of the risks leading to such actions.

The framework also involves a risk classification system divided into multiple tiers. Every model, regardless of complexity, must be assigned a particular risk level, along with human oversight. High-risk models will require approval from the Risk Management Committee at the board level.

Kill-switches and human oversight for AI models

Banks are expected to build override, suspension, and deactivation mechanisms into every AI model deployment so that any model can be shut down immediately if it produces erroneous or harmful results.

The RBI also pointed to what it called an “automation bias,” which is the risk that bank employees will okay AI outputs without exercising any form of independent judgment. For customer-facing AI systems, the guidelines will mandate banks to let their customers know when they are interacting with an AI model, and give them the option to speak with a human at any point.

Generative AI models that interact with customers or external users will face additional cybersecurity requirements under the drafted proposal, according to Reuters.

Third-party AI providers come under scrutiny

Banks remain fully accountable for any AI model they choose to employ, regardless of if built internally or otherwise purchased from a vendor. The RBI also stated that supply chain concentration was one of its concerns, pointing to the risk that banks become overly dependent on a small number of global AI providers.

Independent validation is required for all third-party AI models. Banks must also define thresholds for AI and machine-learning models, ensuring they can articulate in plain terms why a model produced a given output.

The RBI has been tightening its technology governance position for a while. These draft guidelines represent the first time India’s central bank has proposed a comprehensive, board-level accountability structure specifically for AI model risks across the entire financial sector.

If you're reading this, you’re already ahead. Stay there with our newsletter.