The case of the U.S. youth who committed suicide abetted by a chat bot that was so human-like that the adolescent literally romantically fell in love with it illustrates how important it is to regulate the AI industry to prevent abuses of this otherwise very beneficial technology.  We cannot afford to throw the baby with the bath water.  Before we start demanding stronger and stronger state control, considering the principle of subsidiarity, we have to find ways that the private sector—both business and civil society—can regulate the profit-making activities of AI enterprises.  In fairness to Character.AI that developed the bot used by the hapless youth, in response to the tragic event, this technology enterprise immediately adopted safety measures to prevent a similar happening in the future.  As one example of responsible leadership in the age of AI, let us describe in detail the response of Charater.AI

First, it declared what is its business mission:  Our goal is to offer the fun and engaging experience our users have come to expect while enabling the safe exploration of the topics our users want to discuss with Characters.  Our policies do not allow non-consensual sexual content, graphic or specific descriptions of sexual acts, or the promotion or depiction of self-harm or suicide.  We are continually training the large language model (LLM) that powers the Characters on the platform to adhere to these policies.  As a specific response to the tragic case of suicide, Character.AI management started investing heavily in its trust and safety processes and internal team.  Being a relatively new company, it hired a Head of Trust and Safety and a Head of Content Policy.  In addition, engineering safety support team members were employed.  The firm plans to continue growing and evolving in this area of safety.  Management recently put in place a pop-up resource that is triggered when a user inputs certain phrases related to self-harm or suicide, directing the user to the National Suicide Prevention Lifeline. 

Worthy of emulation by similar AI enterprises are new features that Character.AI intends to roll out.  These are new safety and product features that will strengthen the security of its platform without compromising the entertaining and engaging experience users have come to expect from the company.   Among these features are:

-Changes to its models for minors (under the age of 18) that are designed to reduce the likelihood of encountering sensitive or suggestive content.

-Improved detection, response, and intervention related to user inputs that violate the firm’s Terms of Community Guidelines.

-A revised disclaimer on every Chat to remind users that AI is not a real person.

-Notification when a user has spent an hour on the platform, with additional user flexibility in progress.

            To prevent AI characters from going on their own, Character.AI engages in proactive detection and the moderation of user-created Characters, including using industry-standard and custom blocklists that are regularly updated.  Proactively and in response to feedback from users, the company also removes Characters that violate its “Terms of Service.” Character.AI also adheres to Digital Millenium Copyright Act (DMCA) requirements and take swift action to remove reported Characters that violate copyright law or the company’s policies.  The Company recently removed a group of Characters that have been flagged as violative, and these will be added to its custom blocklists moving forward.  This means that users also won’t have access to their own chat history with the Characters in question.  This case clearly illustrates how responsible leadership in the private sector should be the main way to guarantee that AI can be purged of its harmful effects following the principle of subsidiarity:  what can be accomplished at the lower level of society should not be coopted by higher authority.   

            Another moral dilemma that we studied at the workshop on Responsible Leadership in the Age of AI at the IESE Business School had to do with who picks up the errors of artificial intelligence.  The economic sector that can benefit enormously from AI is health care, considering  the huge increase in healthcare costs in recent years.  The case we studied was that of a 43-year old lady who went in for a check-up mammogram.  A nurse performed the mammography within a department coordinated by a specialist in radiology with 30 years of experience.  After it was performed, the lady’s file was automaticall passed to an AI system which concluded that the patient was “healthy” and that there were no signs off morphological abnormalities.  To cut the story short, when the patient returned for her next check up about a year later, the nurse performing the mammogram found the presence of a mass that needs attention.  After two visits to specialists, the patient was diagnosed with stage 2 to 3 cancer.   She had to undergo chemotherapy which led her to bewail “If someone had carefully checked that mammogram last year, I wouldn’t have to have chemotherapy and go through this hell.”

            The response given by the specialist treating her was, however, sobering: “If every doctor had to check every mammogram in such detail, we would be able to do 10% of the mammogram checks we are able to do today in this department.I truly understand it’s terrible…but it’s the price that must be  paid to carry out large-scale checks? to save as many people as possible.” Fortunately, the lady’s cancer was successfully treated, but she couldn’t help but feel that the hospital’s treatment system puts a price on some people’s lives and the sufferings of others.  Possibly using the so-called principle of double effect, ethics expert may help answer the question “how many people will suffer, or even die, to entrust checks to an inexperienced nurse and an AI system that fails to be as precise as an expert doctor?”  The Industrial Revolution 4.0 will bring with it many such moral dilemmas that can only be resolved by seeking the advice of Moral Authorities such as  Pope Leo XIV.

            As a CPA, I am also interested in less life-or-death cases as in the use of AI systems in businesses processes.   A recent report in the Financial Times by Ellesheva Kissin referred to efforts of the Big Four accountants to create audits that verify the effectiveness of AI tools so as to profit from the increasing demand for  proof that client AI systems work and are safe.  The audit firms are hoping that demand for AI assurance will be buoyed by the need to develop greater trust in the technology and companies’ desire for confirmation that they are complying with regulations.   The audits would open a revenue stream similar to that when firms cashed in companies buying assurance for environmental, social and governance metrics.  This move is coming at a time when insurers begin offering cover for losses caused by malfunctioning AI tools, such as customer service chatbots.  For comments, my email address is bernardo.villegas@uap.asia.