Transparency, explainability and fairness in approaches to AI regulation: Takeaways from the Executive Order on the Safe, Secure, and Trustworthy Development and Use of Artificial Intelligence

a Financial Regulation Innovation Lab, Strathclyde Business School, University of Strathclyde, Glasgow, Scotland

b Michael Smurfit Graduate Business School, University College Dublin, Dublin, Ireland

Introduction and Purpose

AI offers amazing opportunities, but has the potential for both harm and good. Used responsibly it can perhaps redress urgent concerns. Conversely, careless use may worsen societal harms – fraud, discrimination, bias, and disinformation among others.  AI deployment for good and towards achieving its many benefits necessitates mitigation of its considerable risks, demanding efforts from government, the private sector, academia, and civil society (Biden Jr., 2023).

Thus, on the 30^th of October 2023 an Executive Order on the Safe, Secure, and Trustworthy Development and Use of Artificial Intelligence (AI) was issued from the White House’s Briefing Room under the authority of President Biden (Biden Jr., 2023). Through the order’s authority, the utmost priority was placed on AI development and use governance via a coordinated, Federal Government-wide approach. The pace of AI capability advancements compelled this action (Biden Jr., 2023).

The order’s impact is assured by the force of law, and federal/executive departments and agencies[1] were made accountable for several duties within it. The aim is to achieve a more innovative, secure, productive, and prosperous future for equitable AI governance (Biden Jr., 2023). Consequently, they have undertaken initiatives to assist in shaping AI policy and advance the safe and responsible development and utilization of AI.[2]

The US’s systematic importance in shaping the global economic landscape makes it interesting to explore its approach to AI regulation (Jain, 2024). Thus, aspects centred around transparency, fairness and explainability within the Executive Order are outlined and form the basis of this piece. A particular emphasis is placed on Sections 7 (Advancing Equity and Civil Rights) and Section 8 (Protecting Consumers, Patients, Passengers, and Students), given the relevance of their respective content to explainability, transparency, and fairness in the context of this article. Finally, a juxtaposition against EU and UK regulatory approaches is made to draw out similarities and differences.

Executive Order Structure

The executive order is structured into the following sections:

1. Purpose.
2. Policy and Principles.
3. Definitions.
4. Ensuring the Safety and Security of AI Technology.
5. Promoting Innovation and Competition.
6. Supporting Workers.
7. Advancing Equity and Civil Rights.
8. Protecting Consumers, Patients, Passengers, and Students.
9. Protecting Privacy.
10. Advancing Federal Government Use of AI.
11. Strengthening American Leadership Abroad.
12. Implementation.
13. General Provisions.

Policy and principles

Eight guiding priorities and adhering principles are outlined for agencies, to comply with the order’s mandate, as appropriate and consistent with applicable law, while, where feasible, considering the views of other agencies, industry, academia, civil society, labor unions, international allies and partners, and other relevant organizations (Biden Jr., 2023). In synopsis, they are:[3]

(a) Safe and secure AI, requiring robust, reliable, repeatable, and standardized AI system evaluations, as well as policies, institutions, and other mechanisms to test, understand, and mitigate risks before use. This includes addressing the most pressing security risks of AI systems, while navigating AI’s opacity and complexity (Biden Jr., 2023).

(b) Promote responsible innovation, competition, and collaboration for AI leadership, and unlock potential for society’s most difficult challenges, through related education, training, development, research, and capacity investments. Concurrently, tackle novel intellectual property (IP) questions and other problems to shield inventors and creators (Biden Jr., 2023).

(c) Responsible AI development and use requiring commitment to supporting workers.  As new jobs and industries are created, workers need a seat at the table, including collective bargaining, so they benefit from opportunities. Job training and education to be adapted for a diverse workforce and providing access to AI-created opportunities (Biden Jr., 2023).

(d)  AI policies consistent with the Administration’s dedication to advancing equity and civil rights.  AI use to disadvantage those already too often denied equal opportunity and justice should not be tolerated. From hiring to housing to healthcare, AI use can deepen discrimination and bias, rather than improving quality of life (Biden Jr., 2023).

(e)  Protect interest of those increasingly using, interacting with, or purchasing AI and enabled products in daily lives. New technology usage does not excuse organizations from legal obligations, and hard-won consumer protections are more important in moments of technological change (Biden Jr., 2023).

(f)  Protect privacy and civil liberties as AI continues advancing. AI makes it easier to extract, re-identify, link, infer, and act on sensitive information about people’s identities, locations, habits, and desires. AI’s capabilities in these areas can increase the risk that personal data is exploited and exposed (Biden Jr., 2023).

(g)  Manage the risks from Federal Government’s own AI use and increase its internal capacity to regulate, govern, and support responsible AI use for better results. Steps are to be taken to attract, retain, and develop public service-oriented AI professionals, including from underserved communities, across disciplines and ease AI professionals’ path into the Federal Government to help harness and govern AI (Biden Jr., 2023).

(h)  Lead the way to global societal, economic, and technological progress, as in previous eras of disruptive innovation and change. This is not measured solely by technological advancements the country makes.  Effective leadership also means pioneering systems and safeguards to deploy technology responsibly — and building and promoting safeguards with the rest of the world (Biden Jr., 2023).

Definitions

“Artificial intelligence” or “AI” is defined in the order as a machine-based system that can, for a given set of human-defined objectives, make predictions, recommendations, or decisions influencing real or virtual environments.  Artificial intelligence systems use machine- and human-based inputs to perceive real and virtual environments; abstract such perceptions into models through analysis in an automated manner; and use model inference to formulate options for information or action (Biden Jr., 2023).

Further, “AI model” in the order means a component of an information system that implements AI technology and uses computational, statistical, or machine-learning techniques to produce outputs from a given set of inputs (Biden Jr., 2023).

Finally, the order’s “AI system” definition is any data system, software, hardware, application, tool, or utility that operates in whole or in part using AI (Biden Jr., 2023).

Transparency, explainability and fairness

While some notable elements of transparency, explainability and fairness are present, directly or indirectly, in other sections of the order, given their emphasised pertinence for human, consumer, and fundamental rights implications (Jain, 2024), over and above the guiding principles and policies discussed earlier, Section 7 and Section 8 delve into the greatest detail on these areas of particular interest.

Section 7 Advancing Equity and Civil Rights provides edification and guidance predominantly in relation to bias and discrimination from an AI perspective. This is in the context of varied rights including those related to the dispensation of criminal justice, and government benefits and programs. Finally, this is also done in the context of the broader economy: specifically, in so far as AI decision making is concerned, whether for disabilities, hiring, housing, consumer financial markets, tenant screening, among others (Biden Jr., 2023).[4]

Section 8 Protecting Consumers, Patients, Passengers, and Students illustrates, from the lens of AI, the direction and principles in relation to aspects of healthcare, public health, and human services. It also clarifies in relation to facets of bias and discrimination in such contexts. Moreover, it details guidance on transportation, education, and communication insofar as AI is concerned (Biden Jr., 2023).[5]

Disparities and parities viz-a-viz the UK and EU

Unlike the UK, and like the EU, explicit definitions for AI are mapped out within the order as highlighted earlier (Jain, 2024). For the most part, the order is phrased in the context of the US and its applicability is for the most part confined to the US, but similar to both the UK and EU, instances exist where international applicability comes into play (Jain, 2024). Notably however, the onus is largely laid upon existing regulatory bodies for the implementation of the order like the UK, albeit with the distinction that some existing US bodies (for example, TechCongress) mostly, if not entirely, have AI within their remits. Thus, in the latter respect, approach of the US is more similar to that of the EU, and perhaps most accurately defined as a combination of the two (Jain, 2024).

In so far as fairness, explainability and transparency are concerned, there is a very holistic emphasis from US lawmakers along several unique considerations. In this, the approach is more akin to that of the EU. As far as caveats and advantages are concerned, a comparison between the US and the UK can be drawn that is broadly parallel to the contrast between the EU and the UK. Specifically, due to its stricter approach, and bureaucratic structure, it will necessitate expending significantly more compliance time, cost, and effort. However, such regulatory guidelines have stronger ethical grounding, possibly ensuring the best interests of relevant stakeholders, and avoiding dark innovation, bad players, reputational damage, and insidious misuse (Jain, 2024). Lastly as seen for the EU and UK (Jain, 2024), fairness, explainability, and transparency once again come to the fore as key considerations in regulating AI within the order. They are also ubiquitously present principles in the approach of the US as evidenced above, underlining their importance and salience in lawmakers’ minds.

Future topics

Expounding upon and assessing the evolution of this regulatory space may be compelling subjects for future articles, as they could hold manifold implications for explainability, transparency and fairness. Further iterations or final versions of specific draft guidance (referenced in footnotes earlier in this piece) created in response to this order could be analysed in further detail (for instance, see here), and comparisons with other similar frameworks (for instance, see here) may be of interest.

References

Biden Jr., J. R. (2023, October 30). Executive Order on the Safe, Secure, and Trustworthy Development and Use of Artificial Intelligence. Retrieved from The White House’s Official Website – Briefing Room – Presidential Actions: https://www.whitehouse.gov/briefing-room/presidential-actions/2023/10/30/executive-order-on-the-safe-secure-and-trustworthy-development-and-use-of-artificial-intelligence/

Jain, K. (2024, April 03). How transparency, explainability and fairness are being connected under UK and EU approaches to AI regulation. Retrieved from FinTech Scotland: https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.fintechscotland.com%2Fhow-transparency-explainability-and-fairness-are-being-connected-under-uk-and-eu-approaches-to-ai-regulation%2F&data=05%7C02%7Ckushagra.jain%40strath.ac.uk%7C1f806I

Image created by OpenAI’s DALL·E, based on an article summary provided by ChatGPT.

The Consumer Duty at One Year – Can we have a more active and engaged view of consumers?

John Finch and Chuks Otioma draw on their research with the Financial Regulation Innovation Lab, arguing that the Duty would benefit from a clearer understanding of consumers being active, engaged and innovative

The UK Financial Conduct Authority held webinar at the end of July marking twelve months since introducing the Consumer Duty.  The webinar provides a welcome opportunity to reflect on the ways in which the Duty has changed our understandings of financial products and services as consumers experience these. While there are antecedents to the Consumer Duty, and it exists alongside other regulations such as Advertising Standards and General Data Protection, the simple force of the term Consumer Duty is remarkable. The landscape is changed.  Financial services providers have a duty to show how their products and services provide good quality outcomes for their consumers. We argue that the next steps for the Consumer Duty should embrace a more nuanced view of consumers as engaged, active, and co-developing, partners in innovation.

Outcomes-based regulation

The Consumer Duty is an example of outcomes regulation. In other words, regulation and corporate activity meet in compliance as much around how they have met performance expectations and have in place processes to continue to do so. The Consumer Duty sets out four dimensions or qualities of good outcomes for consumers: price and value, consumer understanding, consumer support, and governance of products and services. Additionally, providers should pay attention specifically to vulnerable consumers, that suppliers – often to include FinTechs – are included where having a material effect, and with the outcomes of understanding and support comes an implication for enhancing consumers’ financial literacy.

As engaged researchers, we have had an interesting year following the Consumer Duty. We have organised workshops internationally where researchers and practitioners have discussed their approaches. We also highlight presentations at the Market Research Society’s annual financial services research conference. The November 2023 meeting was particularly good.  Fair4All Finance focussed on financial inclusion and the more effective uses of market segmentation techniques to draw out consumer experiences in enchaining financial inclusion (https://fair4allfinance.org.uk/segmentation/).  Cowry Consulting reported on applying behavioural science to providers’ product service development in complex financial products, and in enhancing consumer understanding and support (https://2547826.fs1.hubspotusercontent-na1.net/hubfs/2547826/On%20The%20Brain%20-%20ESG%20Edition.pdf).

Both these themes (segmentation in support of financial inclusion, and applications of behavioural science) speak to a need for a more thorough integration of consumers’ experiences and practices, of how they go about engaging with and using financial services products. As the FCA webinar highlighted, one source of data is complaints and resolution through the Financial Ombudsman. This vital work is though something of a last resort for consumers and providers where things could have gone wrong. Otherwise, drawing in consumers and understanding their experiences, is a longstanding challenge, reflected in stakeholder theory, regulation, and responsible innovation and science, of how to gather consumers and consumer practices into groups in a way that has some comparability with service providers.

Insights can be gained by catching some of the hints about consumers’ process and capabilities reflected into providers through principles of outcomes regulation. If we accept consumers and consumption as active, requiring capability, literacy, understanding subject to behavioural bias, contending with complexity, and varied across individual – even if proxied through segmenting – this rich experience should find a way into the Consumer Duty and our reflections of its implementation.

Active, engaged consumers

Let’s sketch this in a little more detail. Consumer Duty implies a rebalancing of power towards that age-old principle in economics of consumer sovereignty. This is not easily achieved. The Consumer Duty has four dimensions of good outcomes, and only one is price and value. So, as with many consumer activities, outcomes of Consumer Duty are not simply of consumption being a purchase then ‘value sink’.  Consumers need to work hard to access the value designed into and intended in their financial products and services. Just as an example, and with relevance to FinTech too, financial inclusion is tensioned against the investments expected of consumers to participate in the market, in financial services, of a mobile phone with an up-to-date operating system and perhaps paid-for apps. Consumers co-invest and through their personalised adaptions in use, modify the digital infrastructure presumed for many financial services.

New technologies can support drawing insights from consumers. Leveraging AI solutions that provide capabilities for richer data, real time analysis and consumer insights, Fintech and financial service providers have a vantage potin from which to understand customer characteristics and preferences, engage with and offer personalised products that match consumer expectations. This way, financial products and services can extend beyond segmentation and stand a better chance to be directed towards good consumer outcomes. As a note of caution, sole deployment of cutting-edge technologies and data per se can narrow the scope for improved consumer experiences and good outcomes. This sounds the importance of business models that promote co-creation with consumers through product design, development and delivery that allow consumers to interact with and tinker around both products and services, and the platforms on which they are delivered.

Furthermore, consumers often make their own financial bundles and portfolios. With respect to financial inclusion and vulnerability, they do so with great ingenuity, in mind-occupying and time-consuming ways, through trial and error, shared experiences, and, crucially, cutting across providers and product categories. This can perhaps include juggling credit cards, short-term loans, welfare payments, and overdrafts. These are consumer experiences: innovative, imaginative, adaptive, ingenious, sometimes urgent and under pressure. Our challenge is to draw consumer experiences and actions into the view of the Consumer Duty. To vary our focus or unit of analysis to, of course, include individual products and services from providers, but also recognising the additional consumption activities among consumers, and how these vary significantly. This can also allow us to reflect on the definitions of, and contributions to, Consumer Duty categories of – taken together – price and value, consumer understanding, and consumer support. And such an approach can be part of and arguably improve the governance of products and services.

Professor John Finch and Dr Chuks Otioma are at the University of Glasgow’s Adam Smith Business School and the Financial Regulation Innovation Lab (a partnership between FinTech Scotland, University of Strathclyde, and University of Glasgow).  We can be contact at john.finch@glasgow.ac.uk, and chuks.otioma@glasgow.ac.uk.

We acknowledge funding from Innovate UK, award 10055559.

Open Access. Some rights reserved. 

Open Access. Some rights reserved. The publishers, the University of Glasgow and FinTech Scotland, and the authors, John Finch and Chuks Otioma, want to encourage the circulation of our work as widely as possible while retaining the copyright. We therefore have an open access policy which enables anyone to access our content online without charge. Anyone can download, save, perform or distribute this work in any format, including translation, without written permission. This is subject to the terms of the Creative Commons By Share Alike licence. The main conditions are:

• The University of Glasgow, FinTech Scotland, and the authors are credited, including our web addresses www.gla.ac.uk, and www.fintechscotland.com
• If you use our work, you share the results under a similar licence

A full copy of the licence can be found at

You are welcome to ask for permission to use this work for purposes other than those covered by the licence.

We gratefully acknowledge the work of Creative Commons in inspiring our approach to copyright. To find out more go to www.creativecommons.org 

AI and RegTech: Industry Insights on AI in Financial Regulation

Article written by Alessio Azzutti (University of Glasgow), Mark Cummins (University of Strathclyde),
Iain McNeil (University of Glasgow).
Note: Segments of this blog were generated by ChatGPT using notes taken on the day capturing the
presentations and discussions. The authors edited this generated content accordingly.

The integration of Artificial Intelligence (AI) into financial practice and regulatory processes represents a pivotal shift, promising enhanced efficiency, accuracy, and innovation across regulatory compliance and supervision. Our recent discussions explored various facets of AI’s role in financial regulation, revealing a landscape rich with opportunities and challenges. This synthesis (generated by ChatGPT from our discussions with industry partners in our AI & RegTech Workshop on 10 May 2024 and revised by the authors) aims to distill key insights from the discussion across themes, providing an overview of the promises that AI bears on regulatory practices. It will be followed soon by a white paper as part of the White Paper Series published by the Financial Regulation Innovation Lab. This white paper will set out the issues in more detail, linking them to prior research and evolving practice.

Transformation of Regulatory Compliance through AI

AI’s integration into regulatory compliance processes marks a significant evolution in how financial institutions deal with complex regulatory environments. Discussions highlighted AI’s potential to revolutionise compliance by automating and augmenting tasks such as data collection, management and analysis, especially in relation to vast datasets in order to generate actionable insights with unprecedented speed and accuracy. At the same time, the efficacy of AI in compliance hinges on several critical factors. Financial regulations encompass a spectrum of rules ranging from overarching principles to specific quantitative benchmarks and qualitative guidelines.

AI applications must move among these diverse regulatory requirements, which vary in complexity and scope across jurisdictions. Participants underscored the importance of regulatory clarity in fostering AI adoption in compliance (RegTech). Uncertainties about regulatory expectations can stifle innovation in RegTech solutions. Standardisation of data formats, communication protocols, and other AI-related requirements emerged as essential prerequisites to streamline AI integration and enhance compliance efficiency. AI can be integrated into compliance through comprehensive system-wide approaches or targeted solutions for specific regulatory challenges. In addressing the relationship between AI and the various layers of regulation, the roundtable emphasised the need to view compliance as a dynamic process and activity rather than a static framework.

Participants agreed on some crucial aspects related to AI governance. Despite AI’s capabilities, human oversight remains indispensable, for instance, to validate AI outputs, ensure ethical decision-making, and interpret regulatory requirements accurately. Indeed, the discussion highlighted the ongoing need for human experts to manage AI-augmented compliance effectively. Greater standardisation in AI-related technologies and regulatory frameworks are also seen as future catalysts for innovation in the RegTech sector. Standardised practices can enable financial institutions and technology providers to focus on enhancing their AI solutions rather than coping with disparate regulatory landscapes.

Design and Governance of AI-Enabled Compliance Systems

The application of AI, particularly its subfield of Machine Learning (ML) methods, in compliance systems was explored as a transformative force reshaping business strategies and operations within financial institutions. AI systems empower organisations to analyse complex datasets rapidly and derive insights that inform decision-making. Among other things, participants highlighted AI’s role in improving risk management, fraud detection, and overall operational efficiency. However, the design of AI systems in compliance is seen as extending beyond automation to facilitate strategic alignment with business goals and regulatory objectives.

In this context, ensuring effective model governance emerged as a critical priority for organisations deploying AI in regulatory compliance. Robust governance frameworks can help ensure transparency, accountability, and compliance with regulatory standards. The emerging field of Explainable AI (XAI) is deemed critical in financial services to ensure transparency and build trust among stakeholders. Clear explanations of AI processes and decisions enhance user confidence and facilitate regulatory compliance.

Addressing biases—whether in data, models, and their inherent human assumptions—was highlighted as essential to ensure fair outcomes and mitigate risks. Robust governance frameworks include mechanisms for bias detection, mitigation, and continuous monitoring to uphold ethical and legal standards. Discussions emphasised the need for clear policies and procedures to monitor AI models in and along the entire AI lifecycle.

Moreover, the debate between in-house AI development versus third-party vendor solutions highlighted some organisational preferences and challenges. Large financial institutions often opt for in-house development to tailor AI solutions to their specific needs and maintain control over data integrity and security.

Broadly speaking, legal and ethical considerations in AI deployment include data privacy, intellectual property rights, and liability for AI-based decisions. The roundtable discussions emphasised the need for clear regulatory frameworks to address all the complexities embedded in AI governance, which necessitates shared responsibility across stakeholders—developers, users, regulators, and consumers. Clear delineation of roles and responsibilities was deemed crucial by participants to mitigate risks and ensure responsible AI deployment.

The specific exploration of Generative AI in compliance identified its potential in automating routine tasks and enhancing productivity. Despite its benefits, concerns about data privacy, security, and the ethical implications of AI-generated content remain paramount. We heard further calls for human-in-the-loop solutions. Human oversight ensures the credibility and accuracy of Generative AI outputs.

Mutual Reinforcement of RegTech and SupTech

RegTech (regulatory technology) and SupTech (supervisory technology) represent two sides of the same coin, namely the adoption of innovative technology to bring greater effectiveness and efficiency to financial regulation and its enforcement. RegTech tools powered by AI enhance regulatory compliance by improving understanding of regulations, managing business activities, and achieving higher-quality compliance outcomes. However, regulatory fragmentation and differing compliance requirements pose challenges to widespread and trustworthy adoption. In parallel, financial supervisors are researching and gradually adopting AI-based SupTech solutions to enhance their ability to achieve supervisory objectives in an efficient and effective manner. Interoperability between RegTech and SupTech systems is essential for frictionless and secure data exchange between regulators/supervisors and regulated entities in order to improve regulatory oversight. Standardised data formats, communication protocols, and AI-related requirements promote collaboration between financial institutions and regulatory authorities. Greater regulatory clarity and standardisation are seen as catalysts for innovation in the RegTech space. Clear guidelines on regulatory requirements targeting AI applications facilitate technological advancements while ensuring compliance with regulatory standards. Collaboration between public authorities, financial institutions, and technology providers is expected to foster a conducive ecosystem for AI innovation in financial regulation. The roundtable discussions emphasised the importance of collaborative efforts to overcome regulatory challenges and promote technological convergence.

Conclusion

The synthesis of the roundtable discussions provides a comprehensive overview of AI’s transformative impact on financial practice and regulatory processes, underscoring opportunities for innovation, efficiency gains, and enhanced compliance. Key themes include (i) AI’s role in dealing with complex regulatory frameworks and (ii) advancing analytical capabilities in compliance systems, but also (iii) requirements for the design of ethical and law-compliant AI solutions, as well as (iv) the mutual reinforcement of RegTech and SupTech. While AI presents unprecedented opportunities, challenges such as regulatory fragmentation, technical robustness and reliability, and ethical considerations require careful consideration. Moving forward, collaboration between stakeholders, regulatory clarity, and robust governance frameworks will be critical in harnessing AI’s full potential while safeguarding against the associated risks. The insights from the roundtable discussions offer a starting point for various stakeholders, including financial institutions, technology providers, and financial regulators, to delve into issues related to AI and the resulting evolving landscape of financial regulation in a responsible and effective manner. With continued advances in AI, its integration into regulatory practices, if supported by adequate governance, holds promise for shaping a more resilient, efficient, and tech-savvy financial ecosystem.

Image by, https://www.freepik.com/free-photo/standard-quality-control-collage-concept_30589261.htm#fromView=search&page=1&position=0&uuid=5ac75fc0-aba0-4b75-96b8-13559c1795a8