I. Introduction

Every generation has its topic: The topic of our generation is digitalization. At present, we are all witnessing the so-called industrial revolution 4.0.Footnote ¹ This revolution is characterized by the use of a whole range of new digital technologies that can be combined in a variety of ways. Keywords are self-learning algorithms, Artificial Intelligence (AI), autonomous systems, Big Data, biometrics, cloud computing, Internet of Things, mobile internet, robotics, and social media.Footnote ²

The use of digital technologies challenges the law and those applying it. The range of questions and problems is tremendously broad.Footnote ³ Widely discussed examples are self-driving cars,Footnote ⁴ the use of digital technologies in corporate finance, credit financing and credit protection,Footnote ⁵ the digital estate,Footnote ⁶ or online dispute resolution.Footnote ⁷ In fact, digital technologies challenge the entire national legal system including public and criminal law as well as EU and international law. Some even say we may face ‘the beginning of the end for the law’.Footnote ⁸ In fact, this is not the end, but rather the time for a digital initiative. This chapter focuses on the changes that AI brings about in corporate law and corporate governance, especially in terms of the challenges for corporations and their executives.

From a conceptual perspective, AI applications will have a major impact on corporate law in general and corporate governance in particular. In practice, AI poses a tremendous challenge for corporations and their executives. As algorithms have already entered the boardroom, lawmakers must consider legally recognizing e-persons as directors and managers. The applicable law must deal with effects of AI on corporate duties of boards and their liabilities. The interdependencies of AI, delegation of leadership tasks, and the business judgement rule as a safe harbor for executives are of particular importance. A further issue to be addressed is how AI will change the decision-making process in corporations as a whole. This topic is closely connected with the board’s duties in Big Data and Data Governance as well as the qualifications and responsibilities of directors and managers.

By referring to AI, I mean information technology systems that reproduce or approximate various cognitive abilities of humans.Footnote ⁹ In the same breath, we need to distinguish between strong AI and weak AI. Currently, strong AI does not exist.Footnote ¹⁰ There is no system really imitating a human being, such as a so-called superintelligence. Only weak AI is applied today. These are single technologies for smart human–machine interactions, such as machine learning or deep learning. Weak AI focuses on the solution of specific application problems based on the methods from math and computer science, whereby the systems are capable of self-optimization.Footnote ¹¹

By referring to corporate governance, I mean a system by which companies are directed and controlled.Footnote ¹² In continental European jurisdictions, such as Germany, a dual board structure is the prevailing system with a management board running the day-to-day business of the firm and a supervisory board monitoring the business decisions of the management board. In Anglo-American jurisdictions, such as the United States (US) and the United Kingdom (UK), the two functions of management and supervision are combined within one unitary board – the board of directors.Footnote ¹³

II. Algorithms As Directors

The first question is, “Could and should algorithms act as directors?” In 2014, newspapers reported that a venture capital firm had just appointed an algorithm to its board of directors. The Hong Kong based VC firm Deep Knowledge Ventures was supposed to have appointed an algorithm called Vital (an abbreviation for Validating Investment Tool for Advancing Life Sciences) to serve as a director with full voting rights and full decision-making power over corporate measures.Footnote ¹⁴ In fact, Vital only had an observer and adviser status with regard to the board members, which are all natural persons.Footnote ¹⁵

Under German law according to sections 76(3) and 100(1)(1) AktG,Footnote ¹⁶ the members of the management board and the supervisory board must be natural persons with full legal capacity. Not even corporations are allowed to serve as board members. That means, in order to appoint algorithms as directors, the law must be changed.Footnote ¹⁷ Actually, the lawmaker could legally recognize e-persons as directors. However, the lawmaker should not do so, because there is a reason for the exclusion of legal persons and algorithms under German law. Both lack personal liability and personal accountability for the management and the supervision of the company.Footnote ¹⁸

Nevertheless, the European Parliament enacted a resolution with recommendations to the Commission on Civil Law Rules on Robotics, and suggested therein

The most fundamental requirement for legally recognizing an e-person would be its own liability – either based on an ownership fund or based on a mandatory liability insurance. In case corporations are appointing AI entities as directors (or apply it otherwise), they should be strictly liable for damages caused by AI applications in order to mitigate the particular challenges and potential risks of AI.Footnote ²⁰ This is because strict liability would not only delegate the risk assessment and thus control the level of care and activity, but would also create an incentive for further developing this technology.Footnote ²¹ At the same time, creditors of the company should be protected by a compulsory liability insurance, whereas piercing the corporate veil, that is, a personal liability of the shareholders, must remain a rare exception.Footnote ²² However, at an international level, regulatory competition makes it difficult to guarantee comparable standards. Harmonization can only be expected (if ever) in supranational legal systems, such as the European Union.Footnote ²³ In this context, it is noteworthy that the EU Commission’s White Paper on AI presented in 2020 does not address the questions of the legal status of algorithms at all.Footnote ²⁴

However, even if we were to establish such a liability safeguard, there is no self-interested action of an algorithm as long as there is no strong AI. True, circumstances may change in the future due to technological progress. However, there is a long and winding road to the notorious superintelligence.Footnote ²⁵ Conversely, weak AI only carries out actions in the third-party interest of people or organizations, and is currently not in a position to make its own value decisions and judgemental considerations.Footnote ²⁶ In the end, current algorithms are nothing more than digital slaves, albeit slaves with superhuman abilities. In addition, the currently applicable incentive system of corporate law and governance would have to be adapted to AI directors, because – unlike human directors – duties of loyalty can hardly be applied to them, but rather they decide according to algorithmic models.Footnote ²⁷ At present, only humans have original creative power, only they are capable of making decisions and acting in the true sense of the word.Footnote ²⁸

III. Management Board

Given the current limitations of AI, we will continue to have to get by with human directors for the next few decades. Although algorithms do not currently appear suitable for making independent corporate decisions, AI can nonetheless support human directors in their management and monitoring tasks. AI is already used in practice to analyze and forecast the financial development of a company, but also to identify the need for optimization in an entrepreneurial value chain.Footnote ²⁹ In addition, AI applications are used in the run-up to mergers and acquisitions (M&A) transactions,Footnote ³⁰ namely as part of due diligence, in order to simplify particularly labor-intensive processes when checking documents. Algorithms are also able to recognize unusual contract clauses and to summarize essential parameters of contracts, even to create contract templates themselves.Footnote ³¹ Further examples for the use of AI applications are cybersecurityFootnote ³² and compliance management systems.Footnote ³³

IV. Supervisory Board

For this reason, it will also be of fundamental importance in the future to have an institutional monitoring body in the form of the supervisory board, which enforces the interests of the company as an internal corporate governance system. With regard to the monitoring function, there is a distinction to be made as to whether the supervisory board makes use of AI itself while monitoring and advising the management of the company, or whether the supervisory board is monitoring and advising with regard to the use of AI by the management board.

V. Conclusion

AI is not the end of corporate governance as some authors predicted.Footnote ¹¹⁶ Rather, AI has the potential to change the overall corporate governance system significantly. As this chapter has shown, AI has the potential to improve corporate governance structures, especially when it comes to handling big data sets. At the same time, it poses challenges to the corporate management system, which must be met by carefully adapting the governance framework.Footnote ¹¹⁷ However, currently, there is no need for a strict AI regulation with a specific focus on corporations.Footnote ¹¹⁸ Rather, we see a creeping change from corporate governance to algorithm governance that has the potential to enhance, but also the risks to destabilize the current system. What we really need is the disclosure of information about a company’s practices with regard to AI application, organization, and oversight as well as potentials and risks.Footnote ¹¹⁹ This kind of transparency would help to raise awareness and to enhance the overall algorithm governance system. For that purpose, the already mandatory corporate governance report that many jurisdictions require, such as the US,Footnote ¹²⁰ the UKFootnote ¹²¹ and Germany,Footnote ¹²² should be supplemented with additional explanations on AI.Footnote ¹²³

In this report, the management board and the supervisory board should report on their overall strategy with regard to the use, organization, and monitoring of AI applications. This specifically relates to the responsibilities, competencies, and protective measures they established to prevent damage to the corporation. In addition, the boards should also be obliged to report on the ethical guidelines for a trustworthy use of AI.Footnote ¹²⁴ In this regard, they may rely on the proposals drawn up on an international level. Of particular importance in this respect are the principles of the European Commission in its communication on ‘Building Trust in Human-Centric Artificial Intelligence’,Footnote ¹²⁵ as well as the ‘Principles on Artificial Intelligence’ published by the OECD.Footnote ¹²⁶ These principles require users to comply with organizational precautions in order to prevent incorrect AI decisions, provide a minimum of technical proficiency, and ensure the preservation of human final decision-making authority. In addition, there is a safeguarding of individual rights, such as privacy, diversity, non-discrimination, fairness, and an orientation of AI to the common good, including sustainability, ecological responsibility, and overall societal and social impact. Even if these principles are not legally binding, a reporting obligation requires the management board and supervisory board to deal with the corresponding questions and to explain how they relate to them. It will make a difference and may lead to improvements if companies and their executives are aware of the importance of these principles in dealing with responsible AI.

I. Introduction

The use of algorithms is associated with a risk of collusion. This bears on the construal of the cartel prohibition, on which the present chapter focuses. The hypothesis is that algorithms may achieve a collusive equilibrium without any involvement of natural persons. Against this backdrop, it is questionable whether and to what extent such an outcome can be qualified as a concerted practice in terms of the law.

The analysis will be structured as follows: first, it will be assessed in what way algorithms can influence competition on markets (Section II). Subsequently, the article will deal with the traditional criteria of distinction between explicit and tacit collusion, which might reveal a potential gap in the existing legal framework with respect to algorithmic collusion (Section III). Finally, it must be analyzed whether the cartel prohibition can be construed in a way that captures the phenomenon appropriately (Section IV). The chapter will close with a summary (Section V).

II. Algorithmic Collusion As a Phenomenon on Markets

It is widely accepted that the use of algorithms can precipitate collusive outcomes, at least in theory. There is no lack of attempts to systematize the different ways algorithms can be involved here. Since the first groundbreaking publications by Ariel Ezrachi, Maurice Stucke, and Salil Mehra, as well as other authors in the following, the matter has come into the focus of antitrust scholarship and practice.Footnote ¹ Agencies have started to look into it.Footnote ² First cases have emerged about restraints implemented on platforms involving computer technology. With respect to the United States (US) TopkinsFootnote ³ must be mentioned, which involved alleged horizontal price fixing on a digital platform based on algorithms. For the European Union (EU), the EturasFootnote ⁴ case comes to mind. The operator of a travel booking platform had informed the travel agencies using that platform that it intended to cap rebates granted to end-consumers. The European Court of Justice (ECJ) held that this amounted to a horizontally concerted practice among the travel agencies to the extent that these had not objected to this proposal. The Luxembourg competition authority in 2018 found the taxi booking app Webtaxi to be exempt from the cartel prohibition although the system involved an algorithmic horizontal alignment of prices. The agency found offsetting efficiencies to the benefit of consumers.Footnote ⁵ These cases have in common that the natural persons representing the companies involved were aware of the restrictions, or that they at least ought to have known about them. Algorithms were an element of implementing the restriction, yet the ultimate decision about the competitive restraint was taken by human beings. Under these conditions the cases did not pose severe difficulties in establishing explicit collusion. There is not a fundamental difference to cases in which parties communicate, for example, by way of hub-and-spoke cartelization through traditional means and forms of communication.Footnote ⁶

A greater legal challenge is caused by the risk of autonomous algorithmic collusion. Computers with machine learning capabilities can possibly achieve or sustain a collusive equilibrium without any involvement of human knowledge or intent. The underlying scholarly discussion usually orbits around q-learning mechanisms.Footnote ⁷ The hypothesis is that algorithms with machine learning capabilities can act as computer agents exploring the success of their own actions, from which a collusive strategy can emerge as the optimum. In this event, it is conceivable that even the programmer of the algorithm was not aware of this potential outcome.Footnote ⁸ The market effect, therefore, can be a collusive equilibrium, albeit absent any human involvement.

Lawyers, economists, and computer scientists are still at odds over the likeliness and actual occurrence of autonomous algorithmic collusion. Some consider it a realistic scenario that tends to be underestimated.Footnote ⁹ The German Bundeskartellamt and the French Autorité de la Concurrence have refrained from a definitive conclusion so far.Footnote ¹⁰ Ulrich Schwalbe, in his seminal article, points out that the game theoretical dilemma that has to be solved by autonomous computer agents to achieve a stable collusive equilibrium is huge and cannot be easily overcome in practice.Footnote ¹¹ A more recent study by Emilio Calvano and othersFootnote ¹², however, concludes that q-learning algorithms, in fact, can autonomously collude. The EU Commission, in its proposal for a ‘New Competition Tool’, mentions the risk that digital platforms can create ecosystems in which collusion arises, which can be read as a recognition of the phenomenon as a matter of concern.Footnote ¹³ Against this backdrop, it is expedient to elaborate further on the application of the cartel prohibition in such cases.

III. On the Scope of the Cartel Prohibition and Its Traditional Construal

The conceptual problem behind the traditional construal of the cartel provision is the difference in the structure of the law on the one hand and the economic determinants for collusive equilibria on the other.Footnote ¹⁴ Anticompetitive collusive equilibria are characterized by the fact that the participants consider it individually rational to pursue such a strategy. That is the case for types of collusion that are usually referred to as explicit and which are illegal in the same way as it holds true for so-called tacit collusion, which is seen as not to fall within the scope of the prohibition. Agreements in breach of the cartel prohibition are null and void so that they cannot be enforced. Therefore, any cartel is only stable so long as the firms participating in it consider it rational to remain involved. All types of collusive equilibria can, therefore, be considered as non-cooperative games in terms of game theory.Footnote ¹⁵ Yet still, the law does not prohibit the achievement or sustaining of a collusive equilibrium as such. Instead, the provision is confined to certain types of measures, which are described as agreement, concerted practice, or decision. Mere tacit collusion is supposed to be distinct from these aforementioned types of anticompetitive conduct. While explicit collusion (i.e. achieved by agreement), concerted practice, or decision, is prohibited, tacit collusion is found to be legitimate.

As becomes obvious, the traditional construal of the law rests on a description of the means and forms by which firms interact when defining collusion. In the context at hand, the category of concerted practices has the greatest relevance. It is conceived of as something whereby a ‘practical cooperation’ is substituted for the risks of competition. Such practical cooperation, in turn, is supposed to be different from merely observing a rival’s conduct and reacting to it.Footnote ¹⁶ Similar approaches of distinction apply under section 1 of the US Sherman Act.Footnote ¹⁷ There, so-called conscious parallelism is deemed not to fall within the scope of the prohibition.Footnote ¹⁸ For the finding of a cartel, so-called plus factorsFootnote ¹⁹, or ‘facilitating practices’/‘facilitating devices’ need to be established.Footnote ²⁰

It is argued that, whereas firms when tacitly colluding merely observe each other and react independently, the mechanism allegedly differs if they opt for a practical cooperation. A private exchange of pricing information can serve as an example for the latter. The jurisprudence of the courts requires that such practical cooperation must be substituted ‘knowingly’ for the risks of competition for it to amount to a concerted practice.Footnote ²¹ The concept, therefore, hinges on the inner sphere of the firms involved.

Against the afore, it is questionable whether autonomous algorithmic collusion is prohibited under the traditional enforcement paradigms. If the firms lack of knowledge or intent with respect to the fact that their computer agents pursue a collusive strategy, it cannot be said that these firms ‘knowingly’ substitute a practical cooperation for competition. Several authors, therefore, point to the risk that the cartel prohibition might stop short of preventing such outcomes. Ezrachi and Stucke argue that collusion achieved by machine learning systems can fall outside the scope of the cartel prohibition for ‘lack of evidence of an anticompetitive agreement or intent.’Footnote ²² In a similar vein, Calvano and others conclude from their studyFootnote ²³:

Some authors, therefore, highlight that the enforcement paradigms might warrant amendments to close such regulatory gaps.Footnote ²⁴

IV. Approaches for Closing Legal Gaps

V. Conclusion

This article is a conceptual sketch of a way to deal with the intricacies coming along with autonomous algorithmic collusion. Such risk is being discussed, especially, with respect to q-learning algorithms. Even though practical cases have not yet emerged, there is sufficient reason to address potential issues as a precautionary measure at this point in the scholarly debate. It was the intention to demonstrate that the traditional construal of the law, which relies on a description of human behavior, appears inapt for effectively tackling machine-induced equilibria. Applying the established criteria in such cases would very likely lead to a harping on words without substance. There is simply no point in venturing to assess whether algorithms ‘practically cooperated’ as opposed to ‘merely observed each other’. The present proposal, therefore, is intended to serve as a conceptualization of the notion of concerted practices for cases that would otherwise elude the cartel prohibition. To conclude, the entire problem of autonomous algorithmic collusion is an example of the necessity for interdisciplinary research between lawyers, economists, and computer scientists. At the same time, the problem highlights how enforcement paradigms, that hinge on descriptions of the inner sphere and conduct of human beings, may collapse when applied to the effects precipitated by independent computer agents. The subject matter of this chapter is, therefore, an example for the greater challenges that the entire legal order faces in light of the progress of machine learning.

I. Introduction

The financial services industry has been at the forefront of digitization and big data usage for decades. For the most part, data processing has been automized by information management systems. Not surprisingly, Artificial Intelligence (AI) applications, capturing the more intelligent ways of handling financial activities and information, have increasingly found their way into the financial services industry over the last years; from algorithmic trading, smart automized credit decisions, intelligent credit card fraud detection processes, personalized banking applications and even into areas like so-called robo-advisory services and quantitative investment and asset management more recently.Footnote ¹

The financial industry has also been one of the most regulated industries in the world. In particular, since the collapse of Lehmann Brothers in 2008, leading into one of the most severe financial crises in history, regulation efforts of all kinds of finance-related activities and financial organizations as a whole by the different regulators around the world have significantly increased. In general, most regulations relating to the financial industry, in particular those put into place after the financial crisis in 2008, have focused on areas like safeguarding the financial institutions themselves, safeguarding the customers of financial institutions, and making sure the institutions comply with general laws overall and on a global scale, given the truly global nature of the financial industry.

More recently, authors have argued that with the emergence of AI-based applications in the financial industry, new kinds of risks have emerged that require additional regulations.Footnote ² They have pointed for instance to increased data processing risk, cybersecurity risks, additional challenges to financial stability, and even to general ethical risks stemming from AI in financial services. Some regulators like the Monetary Authority of Singapore (MAS) have proposed an AI governance framework for financial institutions.Footnote ³ The EU also explored this topic and published a report on big data risks for the financial sector, including AI, stressing appropriate control and monitoring mechanisms.Footnote ⁴ Scholars have developed this topic further by adopting so-called personal responsibility frameworks to regulate any new emerging AI-based applications in the financial industry.Footnote ⁵ In its recent draft regulation, the EU has presented a general risk-based regulatory approach of AI which regulates and even prohibits certain so-called high risk AI system; and some of them can supposedly also be found in the financial industry.Footnote ⁶

This chapter will explore this entire topic of AI in the financial industry (which will also be referred to as robo-finance) further. One focus of the article will be on whether AI in the financial industry gives rise to new kinds of risks or merely increases existing risks already present in the industry. Further, the article will review one prominent general regulatory approach many scholars and regulators have put forward to limit or mitigate these alleged new risks, namely the so-called (personal) responsibility frameworks. In the final section of this chapter, a different proposal will be presented on how and to what extent best to regulate robo-finance, which will take up key elements and concepts from the recent Draft EU AIA.Footnote ⁷ To lay the groundwork for the discussion of these topics, the nature of AI, in particular as a general-purpose technology, will be explored first. In addition, an overview of the current state of AI applications in financial services will be given, and the different regulatory layers or focus areas for regulations that are present in the financial industry today will be presented. Based on these introductory discussions, the main topics of the chapter can then be spelled out.

II. AI As a New General Purpose Technology

Electricity is a technology or technology domain which came into life more than 150 years ago, and it still drives a lot of change today. It comprises different concepts like electrical current, electrical charge, electric field, electromagnetics etc. which have led to many different application areas in their own right; from the light ball to electrical telegraphs or to electric engines, to mention only a few. It is fair to say that electricity as a technology field or domain has revolutionized the world in many ways, and it still does. And it has changed and transformed whole industries as it transforms the automotive industry with the transition from combustion engines to electric cars.

Given its wide range of underlying concepts with multiple specific application areas of their own right, several authors have referred to electricity as a general-purpose technology (GPT).Footnote ⁸ What is characteristic of GPTs is that there exists a wide range of different use cases in different industries, thus GPTs are not use-case-specific or industry-specific technologies but have applications across industries and across many types of use cases. Other examples of GPTs which scholars have identified are the wheel, printing, the steam engine, and the combustion engine, to mention a few.Footnote ⁹ As such, GPTs are seen as technologies that can have a wide-ranging impact on an entire economy and, therefore, have the potential to drastically alter societies through their impact on economic and social structures.Footnote ¹⁰

Several authors have claimed or argued in recent years that AI can or should also be considered a GPT, ‘the most important one of our era’ in fact.Footnote ¹¹ Or as Andrew Ng says: ‘AI will transform multiple industries’.Footnote ¹² AI’s impact on societies as a whole is seen as significant, for instance, changing the way we work, the way we interact with each other and with artificial devices, how we drive, how wars might be conducted, etc. Further, like in the case of electricity, there are many different concepts underlying AI today, from classical logic or rule-based AI to machine learning and deep learning based AI, as employed so successfully today in many areas. Some hybrid applications combine both concepts.Footnote ¹³ These concepts have allowed for many new types of AI applications, similar to the case of electricity, where different concepts have been merged together as well.

In fact, because the use cases for AI technologies are so enormous today, companies like Facebook have created their internal AI labs or what they have called their ‘AI workshop’ where many different applications of AI technologies, in particular machine learning applications, get explored and developed.Footnote ¹⁴ The underlying assumption of such companies is that AI can be applied to so many different areas and tasks that they need to find good ways to leverage their technological expertise in all such different areas.

Clearly, AI is still in its early stages of technological development, with fewer implementations in widespread operation than in the case of electricity. But there have been language and speech processing applications, visual recognition applications like face recognition in smartphones, photo optimization algorithms in digital cameras, many kinds of big data analytics applications, etc. AI technologies have also changed the interface between humans and machines, some turn machines into helpful assistants, others allow for intelligent ways of automating processes and so on. The applications of AI are already widespread today, and we seem to be just at the beginning of a long journey of bringing more applications to life.Footnote ¹⁵

In the following, we will look at the financial industry as one major application area for AI as a general-purpose technology. The financial industry is interesting in so far as it is heavily regulated on the one hand, but also highly digitalized and technologically advanced on the other hand, with many kinds of AI use cases operational already today.

III. Robo-Finance: From Automation to the Wide-Spread Use of AI Applications

The financial industry has been one of the most data-intensive and digitized industries for decades. In 1973, SWIFT was founded and launched, the so-called Society for Worldwide Interbank Financial Telecommunication, bringing together 239 banks from 15 countries worldwide with the aim of handling the communication of cross-border payments. The main components of the original system included a computer-based messaging platform and a standard message system.Footnote ¹⁶ This system disrupted the manual processes of the past, and today more than 11,000 financial institutions from more than 200 countries are connected through SWIFT’s financial global technology infrastructure. Nasdaq, to give another example, the world’s first electronic stock exchange, began its operations even earlier, in 1971, leading the way to fully digitized exchanges for the trading of any kinds of financial securities, which are the standard and norm today. And real-time financial market data and news, probably the first big data sets used in history, were made available in the early 1980s by companies such as Thomson Reuters and Bloomberg through their market data feeds and terminal services.Footnote ¹⁷

In the years to follow, the financial industry has been at the forefront of leveraging information (management) systems to manage and process the vast amounts of data and information available.Footnote ¹⁸ In fact, today, many financial institutions resemble technology companies more than traditional banking houses, and it is no surprise that companies like Paypal or, more recently, many new fintech players were able to further transform this traditional industry by leveraging new technologies like the Internet or mobile services, platforms, and infrastructures.Footnote ¹⁹

This development of digitizing financial information and financial transactions has made the automation of data handling and processing, not just a possibility but rather a necessity to maintain and defend one’s competitiveness and to deal with and manage the various kinds of risks inherent in the financial industry. The execution of payments within the international banking system or the execution of buying or selling orders on the exchanges can be fully automized today based on simple parameters (such as dates and amounts, or stop-loss orders to manage risk, etc.). Clearly, these ways of automizing financial transactions and processes are in no way intelligent, nevertheless they have helped the investment banks and other actors in the financial industry tremendously to increase process speed, accuracy, and also improve risk management.Footnote ²⁰ Therefore, it is no surprise that many actors in the industry have constantly searched and tried to develop more sophisticated processes, which has opened the doors for AI applications in the financial industry.

Today, there is a wide range of AI applications present in the financial industry of which the following are just key application areas with multiple kinds of use cases:Footnote ²¹

1. (1) Customer Related Processes:

   1. a. new ways of segmenting customers based on the use of so-called cluster algorithms or analyses,Footnote ²²

   2. b. personalized banking services and offers based, for instance, on profiling algorithms,Footnote ²³

   3. c. robo-advisory services replacing human financial advisory with machines,Footnote ²⁴

   4. d. intelligent chatbots advising or providing information to clients in different areas of their financial decision making.Footnote ²⁵

2. (2) Operations and Risk Management:

   1. a. underwriting automation in credit decisions and algorithmic credit scoring,Footnote ²⁶

   2. b. automized stress testing.

3. (3) Trading and Investment Management:

   1. a. algorithmic trading – from simple rule-based AI to more sophisticated machine learning based algorithms,Footnote ²⁷

   2. b. automatic portfolio rebalancing in asset management adjusting the portfolio to the predefined asset allocation scheme based on simple rule-based algorithms,

   3. c. big data and machine learning–based (assisted or fully automized) asset management.Footnote ²⁸

4. (4) Payment Processes:

   • fraud detection algorithms in credit card payments using big data analytics and learning algorithms.Footnote ²⁹

5. (5) Data Security and Cybersecurity:Footnote ³⁰

   1. a. data security – algorithms protecting the data from inside a financial institution,

   2. b. cybersecurity – algorithms protecting the data from outside attacks.Footnote ³¹

6. (6) General Regulatory Services and Compliance Requirements:Footnote ³²

   1. a. Anti Money Laundering (AML) automation and protection algorithms helping to identify politically exposed people (so-called Peps) or criminals involved in certain financial transactions,

   2. b. detection of compliance breaches in case of insider trading etc.

As shown here, AI is already employed today in many areas of the financial industry, and new applications are emerging every day. The question is whether additional or increased risks stem from these applications, which might require additional regulations, as argued by some authors.Footnote ³³ This line of argument will be reviewed in more detail in the following sections. But first, it is important to understand from a high-level perspective the main areas and layers of regulations in the financial industry today.

IV. A Short Overview of Regulation in the Financial Services Industry

The financial services industry is probably one of the most regulated industries. Regulation of trading practices for instance dates back to the seventeenth century when in 1610 in Holland, some first forms of short selling became prohibited.Footnote ³⁴ At the same time, the first central banks were created, such as the Swedish Riksbank in 1668, to regulate payment transactions on a national level and establish national currencies by issuing banking notes. Some of the early regulation was ‘private self-regulation’, in other words, bottom up norm creation,Footnote ³⁵ as in the case of regulatory practices around many of the emerging exchanges, but some regulation was already at these early times government- or state-driven (top down) as in the case of the establishment of central banks and their key role in establishing standardized payments practices based on backed up currencies.Footnote ³⁶

Today the financial industry is heavily regulated by national or supranational bodies, for instance, by the ESMAFootnote ³⁷ in the EU or by the SECFootnote ³⁸ in the US in regard to activities on the different financial markets. Some of the regulations are financial-industry-specific, others are general regulations that severely impact the financial industry. Overall, the different types or layers of regulations in the financial industry can be classified by their underlying aims, namely: (i) regulations meant to safeguard overall financial stability, (ii) regulations for the protection of consumers of financial services, and (iii) regulations that are meant to make sure financial services can operate in a challenging and diverse international environment with sometimes conflicting rules and principles.Footnote ³⁹

The following overview tries to capture the main regulation areas or layers and their specific purpose or aim as they are present in the financial industry today. Some of the layers directly link up to the categories just mentioned, some are cutting across the different categories, and some are also mirroring the classification of the previous Section of AI-impacted application domains in the financial industry:

1. (1) Equity and liquidity requirements for banks and financial institutions to adhere to minimum capital ratios and liquid asset holdings to prevent financial stress, improve risk management, and promote transparency. Examples are the Basel I, Basel II, Basel III regulations which are global voluntary regulatory frameworks adhered to by most financial institutions today;Footnote ⁴⁰

2. (2) Infrastructure regulations, many still in the proposal stage, to improve financial services firms’ operational resilience (in case of major disasters, for instance), and their responses to cyberattacks;Footnote ⁴¹

3. (3) Pre- and post-trading regulations to strengthen investor protection and improve the functioning of financial markets, making them more efficient, resilient, and transparent like banning certain trading practices or making kickbacks by product issuers transparent. The MiFID I and II regulations in the EU are examples of such kinds of regulations;Footnote ⁴²

4. (4) Payment services regulations, like the PSD II directive (2015) in the EU, with the aim of creating more integrated payments markets, making payments safer and more secure and also protecting consumers, for instance, from the financial damage resulting from fraudulent credit card payments;

5. (5) Various kinds of compliance regulations, for instance anti-money-laundering or terrorist financing regulations etc., to ensure that financial institutions obey the treaties and laws and do not enter any illegal transactions or practices, also regarding cross-border transactions;Footnote ⁴³

6. (6) General data privacy protections like the GDPRFootnote ⁴⁴ in the EU, which is highly relevant as financial transactions involve much sensitive personal data.

As we can see, there are no specific AI regulations of financial services – although many of the regulations will also impact AI-based financial services. In fact, there are even very few regulations regarding the underlying technologies in financial services, but most of them focus on the use cases or financial activities, processes, and on the outcomes themselves. Yet, recently some scholars and some regulators have argued that there might be new risks stemming from AI applications and technologies in financial services which require additional regulation. In the following, we will look at some of the alleged risks as pointed out by scholars in the field and explore to what extent they might be covered by the above regulations already or whether there is a need for new regulatory frameworks.

V. New Risk Categories in Robo-Finance Stemming from AI?

Dirk Zetzsche and others, in their recent paper, have identified the following four risk categories or risk areas allegedly related to AI applications in the financial industry:

1. (1) Data risks

2. (2) Cybersecurity risks

3. (3) Financial stability risks

4. (4) Ethical risks.Footnote ⁴⁵

Although I agree with the authors that all these kinds of risks are related to AI applications in financial services, it appears that these risks already existed before the emergence of robo-finance, given the advanced stage of the industry in terms of digitization and data dependency and usage. In fact, some of these risks might even be reduced or vanish when AI comes into place. Let us look at the different risk areas one by one.

Firstly, starting with the data risks of AI applications, Zetzsche and others bring up the following more specific arguments: (i) Because the data quality might be poor, there can be deficiencies stemming from AI applications. As a matter of fact, data quality has often been poor in many parts of the financial industry, for instance, outages at the data centers of the exchanges or of the market data providers leading to the misstating of prices of securities, which can have negative effects on investors’ decisions and the markets overall. AI could actually be used to deal with the data issues in terms of detecting and even resolving them.Footnote ⁴⁶ (ii) Besides, they argue that data used for AI analyses might suffer from biases, for instance, relating to what they call ‘oversight’ in a financial organization. Again, biases have already been influencing decision making in the financial industry even before the emergence of AI applications, maybe not in the form of what they call data-biases, but biases residing more generally in human decision, for instance in the making of credit decisions or consumer lending.Footnote ⁴⁷ AI application might free us from certain biases by providing a more neutral stance if programmed accordingly or at least be sensitive to such kinds of biases. (iii) And it is claimed that AI interdependency can lead to what they call ‘herding’, for instance all systems selling securities triggered by certain market events, which can lead to what has been referred to as ‘flash crashes’.Footnote ⁴⁸ Again ‘herding’ behavior has existed in the financial markets for a long time, and whether the emergence of AI in electronic trading systems has been the cause of what has been called ‘flash crashes’ seems rather questionable. Simple rule-based algorithms, which today by industry experts would rather not be classified as AI systems can give rise to such behavior in contrast to more sophisticated systems trained on historical data relating to such events.

Secondly, let us look at cybersecurity risks. Obviously, they have also existed before the arrival of AI, with most attacks initiated and conducted by human individuals directly or by simple processes, methods, or algorithms. Examples are emails carrying malware that, after it has installed itself on someone’s computer, can silently send all sorts of confidential data from the computer or computer network to the attacker; a similar case of phishing attacks through links to websites – for instance, of online banks that mimic the log-in pages one is familiar with; or finally, simply the reuse of a user’s credentials which the attackers have somehow got hold of – for instance, by one of the already mentioned measures or by simply spying on people in combination with our carelessness in setting passwords. That ‘algorithms can be manipulated in an effort to transfer wealth’ has nothing to do with the presence of AI systems because this could be done already before such systems were in place and it currently happens every day in many different ways within traditional information system environments.Footnote ⁴⁹ It rather seems plausible that AI might provide some help in identifying and preventing cybersecurity attacks.Footnote ⁵⁰ Many services are offered today in this regard, and this seems to be one of the areas where the financial industry could benefit from employing AI-based solutions and thereby reduce potentially harmful cybersecurity risks.

Thirdly, when Zetzsche and others talk about financial stability risks, it is fairly unclear what they have in mind since they mention almost all areas of AI applications in financial services – as laid out above – from consumer facing and supporting applications, to trading and portfolio management systems, to general regulatory and compliance systems. Overall, their main concern here seems to be the emergence of ‘additional third-party dependencies’ to AI technology providers up to the ‘level of oligopoly or monopoly’. Since many of these third-party technology providers are unregulated today, as they point out, and it might even be hard to regulate them as ‘AI-related expertise beyond those developing the AI is limited’, there appears to be a major risk. As they say, ‘these third-party dependencies […] could have systemic effects’.Footnote ⁵¹

What can be said against this last part of their arguments is that many of the actors at the forefront of using AI in financial services today develop their applications inhouse, like the dominant hedge funds or algo trading shops set up by IT specialists.Footnote ⁵² AI-based technology has become such a core asset these days and a competitive factor of financial services that many financial institutions resemble IT companies more and more today to keep all that knowledge inhouse or with high IT expertise inside the organizations to manage their IT service providers or outsourcing partners – far from being entirely dependent or in the hands of monopolistic or oligopolistic structured IT providers.Footnote ⁵³ Hence, their worry about systemic effects stemming from such dependencies seems to be overstated, at least in certain critical banking areas. Moreover, in many instances there are quite a few technology providers that offer similar services to the financial industries, for instance market data providers which increasingly have started to use AI technologies to organize and manage the quality of their market data feeds.Footnote ⁵⁴ Financial institutions, at least in critical areas like trading, often make use of different providers at the same time, which also helps them to reduce their third-party-dependencies. Furthermore, with higher education AI or machine learning programs popping up at many educational institutions around the world, new graduates are also increasingly being educated and trained in these key areas. Thus, knowledge is building up quickly and will also be more widely available, reducing the fear of there being a kind of ‘mystery science’ only a few people have access to and can take advantage of.

Finally, let us focus on what Zetzsche and others refer to as new ‘ethical risks’ stemming from AI applications in financial services. The starting point of their argument is that algorithms do not feel anything, nor do they have values which the authors equate with a lack of ethical foundation in AI-decision making. For instance, they point out that such ‘unethical’ AI systems might nudge people to purchase unsuitable financial products, which might further be facilitated by the fact humans would easily develop a higher level of trust in the AI-based systems because with them, human–machine communication can nowadays be quite sophisticated. What this ultimately can and will lead to is reputational risk for the financial institution employing such systems, for example, when people are driven to make the wrong financial decisions and this becomes public or will be reported in the media or brought up to the courts.

There are quite a few problems with this line of reasoning as there are many financial institutions that do not have much direct interaction with human consumers, like mutual funds, hedge funds, credit card companies, etc. Besides, it is also conceivable that AI systems can have an ethical foundation, for instance thinking of utilitarian approaches which are less focused on being able to feel anything or have values. Such aligned AI systems might still be able to calculate the best outcome for society as a whole. But the main counter argument seems to be that the financial industry has not been a role model for ethical behavior to start with. Quite to the contrary, over many decades, financial institutions have been prone to all kinds of ethical misconduct. Just to give a few examples: (i) consumers have been pushed by financial advisors, humans with feelings and values, employed by financial institutions, to buy financial products which were often not suitable or beneficial to them, yet by selling them, the advisors were able to boost their commission payments, and the financial institutions could thereby boost their profits;Footnote ⁵⁵ (ii) insider trading has happened frequently,Footnote ⁵⁶ (iii) market manipulation has occurred, for instance in the case of the Libor scandal, and many other examples in different areas of the financial industry.Footnote ⁵⁷ Thus, it is far from clear why AI-based systems and processes would make the industry less ethical than it has been in the past. In fact, the case could be made that AI-based systems and processes might allow society to create and control financial institutions and make them less driven by greed but more by higher motives to bring benefits to consumers and install fairness within the systems.

But this line of reasoning might sound overly naïve, given how many actors in the financial industry have successfully used technology over the last decades to their advantage, and to the disadvantage of other actors. One example has been the area of high frequency trading and the so-called dark pools where ‘fast moving robot trading machines were front-running long term investors on exchanges’.Footnote ⁵⁸ Darks pools are markets established by the financial actors themselves for trading securities outside of the exchanges, usually virtually unregulated. The benefits for the market actors were faster processing of orders with less or even no fees from the exchanges. But the real benefits for the involved high frequency trading firms were obviously financial: with their algorithms and high frequency trading infrastructures, they were able to read the directions markets were going, and being able to buy securities before the real investors could do it and then selling the securities back to them at a higher price only milliseconds after the initial orders by the investors had been made. This practice allowed them to make huge profits stealing away money from the long-term investors like pension funds, etc.

This is a very sophisticated version of an old, mostly considered illegal practice of so-called front-running – in other words, someone trading a stock or any other financial asset based on insider knowledge of a future transaction that is about to affect its price. One important point is that this practice has been around before the emergence of AI and the high frequency data processing infrastructures. But it needs to be acknowledged that the new technologies have allowed for a more sophisticated and harder-to-control form of front-running. Yet the problem here is not that the employed AI algorithms are unethical. The problem is that the actors have used the technologies in an unethical way which obviously needs to be prevented for the benefits of the wider investor community and for society as a whole. Again, this is not a new risk, but it shows that AI and technology can be an accelerator of existing risks inherent in the financial industry.

In sum, then, the arguments by Zetzsche and others that there are many new risks stemming from AI-based applications in financial services are not fully convincing. To the contrary, it is feasible that the employment of AI applications in the financial industry might provide a route of managing existing and inherent risks in a better way, or even being able to reduce or eliminate some of these risks.Footnote ⁵⁹ But clearly, there are also cases like front-running based on algorithmic high frequency trading, where it seems obvious that through the employment of AI, existing inherent risks in the financial industry have increased and can cause additional damage. Therefore, it is also important to look at ways how such damages resulting from the use of the new technologies can be avoided. In this regard, in the following section one prominent regulatory approach, the so-called responsibility frameworks, will be discussed.

VI. Responsibility Frameworks As a Solution for Managing AI Risks in financial services?

In regulating the financial industry, many regulators have moved to so-called responsibility frameworks in recent years, like the EU’s EBA/ESMA guidelines or the FCA in the UK.Footnote ⁶⁰ The proposed measures focus on personal managerial responsibility, for example, the personal responsibility of directors, senior management, and individual line managers. Initially, such frameworks were meant to be applied to mitigate the risks of financial services in general, but recently authors have argued that they can also be applied to the emerging AI-based processes in the financial industry.Footnote ⁶¹

The responsibility-driven regulations by the EU, published by EBA and ESMA, focus mainly on the management bodies of financial institutions, in particular on their role in conducting their overall operational duties, but with a particular focus on risk management conduct. They are meant to ensure that a sound risk culture has been implemented in their respective organizations consistent with the individual risk profile and the overall business model of the institution. The UK’s senior management regulatory framework for financial institutions has evolved from the overall EU framework but it has strengthened the establishment of clear conduct rules for senior managers. These rules specify in more detail the steps necessary to ensure that the business of the financial institution is controlled effectively and is in compliance with existing regulatory frameworks. Also, requirements are made on the delegation of responsibilities and on the disclosure of relevant information for the regulators. Other States like the US or Singapore have issued similar guidelines.Footnote ⁶²

Although these responsibility frameworks have been very general in nature, meant to capture all kinds of aspects of risk management in financial institutions, Zetzsche has argued that they will give us the right framework to address and manage any new risks stemming from AI applications in financial services. They write: ‘personal responsibility frameworks provide the basis for an appropriate system to address issues arising from AI in financial services’.Footnote ⁶³ They have suggested the following three distinct instruments or measures for regulating activities related to the development and use of AI applications in financial services:

1. 1. AI Review Committees: the installation of AI review committees is meant to address what they call the information asymmetry as to the function and limits of an AI system, namely the problem that third party vendors or inhouse AI developers understand the algorithms far better than the financial institutions that acquire and use them, and the supervisors of the institutions. These committees are meant to augment decision making and should not ‘detract from the ultimate responsibility vested in management […] regarding AI governance.Footnote ⁶⁴

2. 2. AI Due Diligence: mandatory AI due diligence should be put in place, which should be done prior to any AI employment and should include what they call “a full stock of all the characteristics of the AI […] in particular the mapping of the data set used by AI”, including an analysis of data gaps and data quality.Footnote ⁶⁵

3. 3. AI Explainability: the explainability requirement is proposed to be necessary as a minimum standard ‘demanding that the function, limits and risks of AI can be explained to someone at a level of granularity that enables remanufacturing of the code’. And this someone ‘should be a member of the executive board responsible for the AI’.Footnote ⁶⁶

Before we review this proposal, it is fair to mention that the authors themselves note a few limitations, of which I want to focus on the main one, namely the inability of their responsibility framework to control what they call ‘autonomous AI’. What they mean by this are cases in which developers lose control over self-learning AI, not understanding anymore what the algorithms are doing.Footnote ⁶⁷ What they propose is the concept of being able to always switch off the AI (as a kind of human oversight) while the provided services would still be functioning. This seems, prima facie, to be a reasonable request, looking for instance at the example of self-learning AI application in payment fraud detection based on the analysis of large transaction data sets. The system might modify its outlier detection algorithm in a way which might force the financial institution to switch it off, maybe because fraudsters have fed the system with data to facilitate fraudulent transactions. In this setting, switching the AI system off would make sense, but the delivery of the basic payment services should not be impacted by this – for instance, in the case of a credit card company. Yet, there will be applications where such a switch-off mechanism might be more difficult to realize without causing any further damage, as in the case of trading financial securities where orders or transactions ‘might get lost’ by switching off applications.Footnote ⁶⁸

Overall, I agree with the authors that their approach is important and should be part of any software-based technology development in financial services. In fact, many elements have been in place in the industry for years already, for instance in terms of regular due diligence audits of the financial services’ technology providers.Footnote ⁶⁹ Hence, the financial industry is already prepared and experienced in conducting due diligence audits on a regular basis, and they do this frequently before the release of new technology and software systems or installations, irrespective of whether these systems would include AI technology or not.

Yet, on the other hand, the authors propose some specific requirements for their AI due diligence and also in regard to their explainability requirement for AI. As will be argued, these requirements are not entirely clear and potentially will also be hard to fulfill given the nature of many AI applications.

Firstly, they argue that any AI due diligence should comprise taking full stock of all the characteristics of the AI, ‘in particular the mapping of the data set used by AI, including an analysis of data gaps and data quality. It is not clear what is meant by ‘a full stock of all the characteristics of the AI’. Besides talking about the different functionalities of the AI system, they also seem to focus on the underlying data set used by it. The problem here is that data sets might be potentially infinite and/or not be fully determined at the outset. In the case of its employment, a self-learning AI might discover new data (sets), and as it is often the case with big data, there can be quality issues and gaps. What does this mean for the AI system: should it not be launched under such circumstances? Or is it just enough to be aware of such limitations?

Secondly, their explainability requirement seems even harder to deal with in the case of AI applications. Even in regard to existing non-AI applications, it is questionable whether this requirement can be met given the complexity of many software solutions in the financial sector with millions of lines of code and often old legacy systems.Footnote ⁷⁰ In the case of AI-based application, the situation is even more complex because learning AI systems are less static but more dynamic in nature, which could mean that the system might even rewrite its code in the course of its operations. Making explainability a minimum standard in the sense defined above could be the end of many applications in financial services, not only AI-driven ones. But what might be argued for is that a reduced version of this principle can be applied, not to require the remanufacturing of the actual code but, at a minimum, the possibility for the functions, limits, and risks of the AI systems to be understood on a higher functional level.

Thus, to conclude this section, the so-called responsibility frameworks can provide a basis for limiting the risks of AI applications in financial services, given that new risks really emerge. In a way, they have been present in the financial industry before the rise of AI applications, such as regular due diligence audits of technical systems and key software applications. But they also have their limitations, in particular, when certain requirements are taken in a very strict sense, as it was the case above in regard to the discussed explainability requirement. Yet reducing such requirements to a lower level raises the question to what extent the risk stemming from any kinds of new potential high risk AI applications in financial services can be contained. A slightly different approach will be presented in the final section, which builds on the approach put forward recently by the EU.

VII. Standardization, High Risk AI Applications, and the New EU AI Regulation

As noted before in this chapter, AI, being a general-purpose technology, will impact not just one industry but will also have many different kinds of use cases in different industries. There are already a lot of AI applications in the financial industry today, as pointed out earlier, and more are being added constantly by different financial institutions, their IT service providers, and innovative fintech companies. Many of the solutions might be simple or fairly basic, like the use of face recognition as an identification method giving users access to their financial accounts or applications. The others will be more complex, like developing so-called robo advisors, with AI systems engaging with users in natural languages trying to understand their financial needs and giving them suitable financial advice.

What will be important going forward is that regarding some kinds of key AI applications – such as identification processes or human–machine interaction – there can and will be standards across industries which companies need to comply with, like there are standards and norms for the use of electricity irrespective of their specific domain of use.Footnote ⁷¹ Looking at the example of AI systems intended to interact with humans, providers of such systems might require that they be transparent to the user, that they are not communicating with humans but with a machine. Such notification obligations can then become part of the standard for such human–machine interaction enabling AI systems.Footnote ⁷²

Besides such general standard AI applications used across industries, there might also be ones very specific to certain industries like the financial industry which need to be dealt with outside of the model of standardization. In particular, when these specific applications give rise to higher or new risks, additional specific regulations might need to be put into place. For instance, there have been attempts to contain the risks of algorithmic trading applications in the financial industry, which can cause (and probably have already caused to some extent) significant financial damage in the form of leading markets to crash, thereby diminishing or blowing away investors’ money in the course of seconds.Footnote ⁷³ Such flash crashes have been at the center of some debates over the last years, and regulators such as the EU have tried to contain this risk by putting additional obligations into place through the MiFID II framework as discussed earlier.Footnote ⁷⁴

In its recent Draft EU AIA the EU has also distinguished between ‘high risk’ and non-high-risk (standard) AI applications.Footnote ⁷⁵ The new proposed regulation starts with the assumption that AI applications are ultimately and potentially just tools to increase human well-being. Thus, the technology development of AI should not be hindered by any unnecessary constraints, but the rules should be balanced and proportionate. The regulation is centered on a ‘risk based regulatory approach, whereby legal intervention should be tailored to those concrete situations where there is a justified cause of concern’.Footnote ⁷⁶ A key distinction is made between the so-called high risk AI systems for which special requirements and obligations then apply and other AI systems with much more limited requirements and obligations. The classification of AI systems as high-risk is thereby mainly based on their intended purpose and their harmful impact on health and safety and human rights. High risk systems are more or less identified in a two-step process, namely whether they can cause certain harms to protected goods or rights and by the severity of the harm caused and the probability of occurrence.

Given this approach, it seems obvious that there cannot be one final list of high-risk AI applications because the technology is still emerging and new applications are being launched every day. The EU acknowledges this as it lists in its Draft EU AIA only a limited number of high-risk AI applications (Annex III). Further, it allows the EU Commission to amend this list over time based on criteria spelled out in Article 7.

Interestingly, many of the high-risk applications listed by the Draft EU AIA are not specific to one industry but are general AI applications that can be present in many industries. Examples are applications that embody what is called ‘manipulative AI practices’ and a second group with ‘indiscriminate surveillance’ practices. But there are also many very specific high risk AI applications listed in the draft. When it comes to high-risk AI applications in financial services, the EU draft of the regulation lists, prima facie, only one class, namely AI systems that evaluate the creditworthiness of persons (Annex III No 5 lit. b).Footnote ⁷⁷ This class of applications is included in the high-risk list because of (i) possible discrimination of persons of certain ethnic or racial origin based on the potential perpetuation of historical patterns by the AI algorithms, and (ii) the potential severity of such acts of discrimination, as in the way such discriminating credit decisions can significantly affect the course of life of people.Footnote ⁷⁸

The second kind of AI application that can be associated with the financial services industry, listed in the Draft EU AIA, is the one written about above, namely AI systems intended to interact with natural persons or generate content consumed by such person. Such systems do not necessarily classify as high-risk systems, for instance they might just help someone to enter information or explain a product, but they pose the specific risk of impersonation and deception, and therefore they are subject to specific transparency obligations according to the Draft EU AIA that means that the natural persons have to be informed that they are interacting with an AI system (Article 52).

Overall, the risk-based regulatory approach which underpins the Draft EU AIA makes much more sense than any kind of generalized approach of regulating AI applications as a whole as embodied in the responsibility frameworks discussed above. As a general-purpose technology, there will be so many kinds of applications that not one standard set of rules can be applied across the board. A rigorous case by case approach is required, which also allows for amendments and revisions, as embodied in the outline of the EU regulation.

VIII. Conclusion

What has been shown in this chapter is first, that it is questionable whether there are many new additional risks stemming from AI applications in financial services today. The risks that have emerged recently, like data risks, cybersecurity risks, financial stability risks, and ethical risks have been inherent in the financial industry as a highly digitized and also complex global industry for decades. The author has taken the more positive view that by using AI these risks will not necessarily increase, but on the contrary, AI might help to mitigate and reduce them. Second, the responsibility frameworks as developed over the last few years, which are meant to deal with and limit the risks of AI in the financial industry overall, do not provide a suitable framework beyond what has been put in place already to manage the risks with more standard IT and software systems and applications in the financial industry. Furthermore, overseeing all AI applications in financial services will quickly become as complex as overseeing all types of applications in the area of electricity, to mention another general-purpose technology. What has been argued in this chapter is that for some kinds of key applications – like identification processes or human–machine interaction – there should be standards defined across industries with which companies need to comply. But for other very specific, potentially new high-risk financial AI applications, in case they emerge, there might be the need for additional very specific regulation, as in the case of certain algorithmic high frequency trading applications. But this will be less a regulation of the technology but more of the practices and intended uses of the technology, which has also been the core thinking underlying the recent Draft EU AIA of AI applications. In fact, this new EU regulation, like the GDPR a few years ago in regard to data privacy protection, in many ways points to the right direction of how to deal with AI and potential risks arising from it.