AI Explainability is Crucial

Financial institutions around the world are making large-scale investments in AI, while governments and regulators struggle with the significant uncertainties and growing public trepidation as AI becomes central to the fabric of institutions, markets and individuals.

There is little dispute about the significant accuracy/precision increases of advanced machine learning and deep neural networks. However, the adoption of them in production environments for increased revenues, customer satisfaction, and the attainment of business objectives requires an articulation of their intricacies to model validators, internal audit, end-users, customers, compliance and regulators that the data scientists, quants and/or model developers have a challenge to provide.

Does this mean one should only use AI models for decisions which are clear but less accurate, or can we use significantly more accurate ‘black boxes’?

It is clear that at the centre of this is the importance of AI explainability. Retrenching is not an option if one wishes to still be relevant in the near, medium and long-term future.

Do not retrench but adopt explainability

Machine Learning explainability refers to the ability to make the behaviour and predictions of Machine Learning systems understandable and helps explain crucial aspects of our models:

  • What drives the model predictions?
  • Why did the model take a certain decision?
  • Can we trust the model prediction?

Be Regulatory Compliant

Regulators want to know how these AI models work and understand ‘what happened’. This partly accounts for the expansion of regulations in scope and severity over the past several years. In finance alone:

  • The Federal Reserve Board’s SR 11-7
  • Targeted Review of Internal Models
  • Anti-Money Laundering (AML) regulations
  • Know Your Customer (KYC) regulations
  • Various anti-fraud mandates
  • The Bank Secrecy Act 

.. and more regulation which either compel organizations towards complex AI models whilst demanding explainability in order to do so.

There are numerous other regulations like:

  • General Data Protection Regulation (GDPR), which covers customer privacy.
  • There are similar regulations in verticals like healthcare and insurance pertaining to explainable model results.