New York State Department of Financial Services Issues Proposed Artificial Intelligence Circular Letter

On January 17, 2024, the New York State Department of Financial Services (“NYSDFS”) released a proposed circular letter addressing the use of external consumer data and information sources (“ECDIS”) and artificial intelligence systems (“AIS”) in insurance underwriting and pricing (the “Proposed Circular Letter”). The Proposed Circular Letter sets out rules and principles under the following headings: Fairness Principles, Governance and Risk Management, and Transparency. The NYSDFS acknowledges that AIS and ECDIS can “benefit insurers and consumers alike,” but also highlights “significant concerns” about increased “risks of inaccurate, arbitrary, capricious, or unfairly discriminatory outcomes that may disproportionately affect vulnerable communities and individuals or otherwise undermine the insurance marketplace in New York.” The NYSDFS will accept comments on the Proposed Circular Letter until March 17, 2024. Comments may be submitted to innovation@dfs.ny.gov.

Who’s Covered?

The Proposed Circular Letter would apply to any insurer authorized to write insurance in New York State, any licensed fraternal benefit society, and the New York State Insurance Fund (collectively referred to herein as “insurers”).

Fairness Principles

The Proposed Circular Letter sets out specific fairness principles that insurers would be required to adhere to in the use of ECDIS or AIS for underwriting or pricing:

• The data source or model supporting the ECDIS or AIS must not use, and must not be based in any way on, any class protected pursuant to Article 26 of the New York Insurance Law;
• Such use must not result in or permit any unfair discrimination, or otherwise violate the New York Insurance Law;
• Any ECDIS to be used must be supported by generally accepted actuarial standards of practice and based on actual or reasonably anticipated experience;
• Any ECDIS to be used must not be prohibited by the New York Insurance Law and must not serve as a proxy for any protected classes that may result in unfair or unlawful discrimination; and
• The ECDIS or AIS must not collect or use criteria that would constitute unfair or unlawful discrimination or an unfair trade practice.

The Proposed Circular Letter would require an insurer using ECDIS or AIS in underwriting or pricing to conduct a comprehensive assessment to determine that such use would not be unfairly or unlawfully discriminatory in violation of the New York Insurance Law. The Proposed Circular Letter sets out a three-step process for such a comprehensive assessment:

• First, the insurer would be required to assess whether the use of ECDIS or AIS would produce disproportionate adverse effects in underwriting and/or pricing on similarly situated insureds or insureds of a protected class.
• Second, if there is a prima facie showing of a disproportionate adverse effect, then a further assessment would be required to determine whether there is a legitimate, lawful, and fair explanation or rationale for such effect. If no such explanation or rationale can be determined, then the insurer would be obligated to modify its planned use of ECDIS or AIS.
• Third, even if a legitimate, lawful, and fair explanation or rationale exists, the insurer would be required to conduct and document a search for less discriminatory alternative variables or methodology that would still reasonably meet the insurer’s business needs.

To demonstrate compliance with the above requirements, the Proposed Circular Letter would require insurers to conduct this assessment before an AIS is launched and on a “regular cadence thereafter” and after material updates. Insurers should also appropriately document the processes and reasoning behind their testing methodologies and analysis. The Proposed Circular Letter encourages insurers to use multiple statistical metrics in evaluating data and model outputs. These metrics may include: adverse impact ratio; denials odds ratios; marginal effects; standardized mean differences; Z-tests and T-tests; and drivers of disparity.

The Proposed Circular Letter also provides that the ultimate responsibility for proper use of ECDIS or AIS rests with the insurers, even if such ECDIS or AIS was developed or deployed by third-party vendors.

Governance and Risk Management

The Proposed Circular Letter notes that an existing NYSDFS regulation requires an insurer to have a corporate governance framework that is appropriate for the nature, scale, and complexity of the insurer. It then goes on to describe key expectations for an insurer’s governance and risk management framework with respect to ECDIS and AIS:

• Such a framework should provide appropriate oversight of the insurer’s use of ECDIS and AIS, including at the board of directors and senior management levels.
• An insurer that uses ECDIS or AIS should have written policies and procedures that clearly define appropriate roles and responsibilities, outline monitoring and reporting requirements, provide for training of relevant personnel and set standards for the acquisition, use of, or reliance on ECDIS and AIS developed or deployed by third-party vendors.
• The insurer’s board of directors or senior management should review and approve such policies and procedures at least annually.
• The insurer’s internal audit function should be appropriately engaged with the insurer’s use of ECDIS and AIS, consistent with the financial, operational, and compliance risk.
• An insurer should maintain comprehensive documentation regarding its use of ECDIS or AIS.
• An insurer must establish a system for receiving and addressing consumer complaints and inquiries about the insurer’s use of ECDIS and/or AIS.

Transparency

The Proposed Circular Letter also reminds insurers that the failure to adequately disclose to an insured or potential insured any specific reason or reasons for its refusal of coverage, limitation of coverage, or charging a different rate for coverage may be deemed an unfair trade practice. When an insurer is using ECDIS and/or AIS, the notice regarding an adverse underwriting or pricing decision is expected to include the following:

• The specific source of the information upon which the insurer based its decision;
• Whether the insurer uses AIS in its underwriting or pricing process;
• Whether the insurer uses ECDIS; and
• A description of the process for the insured or potential insured to request information about the specific data that resulted in the decision.

The Proposed Circular Letter warns that the failure to disclose such information could constitute an unfair trade practice under Article 24 of the New York Insurance Law, and that an insurer may not rely on the proprietary nature of a third-party vendor’s algorithmic processes to justify the lack of specificity in such an adverse decision notice. Also, to the extent that an accelerated underwriting process is available only to certain persons, an insurer must disclose the objective criteria for using the accelerated process at the outset. Further, if the accelerated process determines that an applicant will not be approved for insurance under the accelerated process, and can only obtain insurance by submitting to the traditional underwriting process, the reason for such a decision must be disclosed to the applicant.

Conclusion

The Proposed Circular Letter is the latest state-based effort to regulate the use of ECDIS and AIS in the insurance industry. It follows upon the adoption by the National Association of Insurance Commissioners in December 2023 of a model bulletin regarding the “Use of Artificial Intelligence Systems in Insurance” and the adoption by the Colorado Division of Insurance of an artificial intelligence regulation focused on governance and risk management for life insurers and a proposed regulation regarding quantitative testing of ECDIS, algorithms, and predictive models used for life insurance underwriting for unfairly discriminatory outcomes.¹ The Proposed Circular Letter, however, is more expansive in terms of the level of detailed requirements it would impose. We will continue to report on the NYSDFS’ effort to issue a final version of the Circular Letter, while monitoring efforts by other states to regulate the use of artificial intelligence in insurance.

¹ For further information, see our Legal Updates: “US NAIC Fall 2023 National Meeting Highlights: Innovation, Cybersecurity, and Technology (H) Committee” (December 13, 2023); “Colorado Releases Draft Regulation on AI Testing for Life Insurers” (October 16, 2023); and “Colorado Adopts Artificial Intelligence Regulation for Life Insurers” (September 28, 2023).