Cybersecurity Policy Report, PRIVACY NEWS: Public-benefits fraud detection system accused of privacy violations, unsafe AI practices, (Jan 12, 2024)

By Thomas Long, J.D.

Privacy watchdog EPIC’s complaint to FTC urges investigation of Thomson Reuters over Fraud Detect product that allegedly caused many recipients to wrongfully lose benefits.

In developing and operating its fraud detection system known as “Fraud Detect,” Thomson Reuters Corporation and its subsidiaries, West Publishing Corporation and Pondera Solutions, LLC, engaged in harmful data collection and artificial intelligence (AI) practices, according to a complaint and request for investigation filed with the Federal Trade Commission by the Electronic Privacy Information Center (EPIC) on January 3. Via the Fraud Detect system, “Thomson Reuters purports to accurately detect public benefits fraud through an opaque, proprietary algorithm and commercial data derived from sources like social media, credit reports, and housing records,” the complaint states. EPIC called on the FTC to investigate the information services company for its allegedly unlawful use of sensitive data gleaned from government and commercial sources. EPIC asserts that Fraud Detect generated false fraud alerts, causing hundreds of thousands of legitimate claimants to be deprived of public benefits. Additionally, “Thomson Reuters has also failed to show that its Fraud Detect product meets accepted standards for responsible automated decision-making systems,” including President Biden’s Executive Order 14110 on safe use of AI and the White House’s Blueprint for an AI Bill of Rights. The company’s conduct amounts to unfair and deceptive practices in violation of Section 5 of the FTC Act, the complaint alleges. EPIC seeks an injunction against the offending business practices, along with model deletion or destruction of the Fraud Detect system.

Based in Washington, D.C., EPIC is a public interest organization focused on emerging privacy and civil liberties issues. EPIC’s complaint states that Thomson Reuters markets various algorithmic and automated risk analytics products to various healthcare, unemployment, and social services entities throughout the United States, including its Fraud Detect product. The product also has been sold under the names “FraudCaster,” “ID Risk Analytics,” and “Fraud Detection as a Service.” Under most contracts with the state entities, Thomson Reuters maintains control and ownership over the Fraud Detect system, including control over its source code, proprietary data, operation, and maintenance. “Often, this control meant that neither agencies nor public benefits recipients knew how Fraud Detect generated fraud alerts or whether Thomson Reuters was meeting minimum standards for responsible AI development and use,” EPIC said in a media release.

The complaint points out that principles for AI use promulgated by the Organization for Economic Cooperation and Development (OECD)—which have been endorsed by the U.S. government—stress the need for producers of AI tools to respect privacy and data protection rights and to practice transparency, so that members of the public can be aware of their interactions with AI systems and challenge outcomes if adversely affected. According to the complaint, the OECD Principles on Artificial Intelligence are “established public policies” within the meaning of the FTC Act. The complaint further cites the White House’s AI safety blueprint, which calls for pre-deployment testing, risk identification and mitigation, and ongoing monitoring to ensure safety and effectiveness. The blueprint contains its own transparency recommendations, urging developers to provide generally accessible plain language documentation with clear descriptions of the overall system functioning and the role automation plays, as well as notice that such systems are in use. EPIC further asserts that Thomson Reuters’ practices run afoul of the AI Risk Management Framework published in January 2023 by the National Institute of Standards and Technology.

Thomson Reuters acquired the automated fraud detection and prevention system, then marketed as FraudCaster, along with its developer, Pondera Systems, in March 2020. A Thomson Reuters media release at the time claimed that the system was intended to help government and private healthcare programs to detect and prevent improper payments. Thomson Reuters subsequently integrated its CLEAR investigation platform into the product; CLEAR is a searchable database of billions of public and proprietary commercial records from over 60 data sources. EPIC’s complaint notes that Thomson Reuters has been accused of scraping personal and sensitive information to create a “cradle-to-grave dossier” for each person in its database, which it sells to third parties. Involved data includes individual and business records, identity information, criminal backgrounds, incarcerations, deceased status, best known address, affiliates, linkages, and social media information. EPIC also accuses Thomson Reuters of collecting such information as recipients’ home addresses and how far recipients travel to buy groceries.

According to the complaint, the Fraud Detect system uses public benefits program data and proprietary commercial data about benefits recipients to train rule-based algorithms that purportedly can predict “fraud or other collusive activities.” When the algorithm detects “anomalous behavior,” it produces alerts designed to trigger further review. Fraud Detect also generates Risk Scores and Scorecard rankings based on the alerts. EPIC alleges that Fraud Detect regularly flags legitimate public benefits claims as fraudulent, leading to the wrongful reduction, denial, and recollection of public benefits for eligible recipients. For example, “Fraud Detect has been used to claw back purported years-old improper overpayments from public benefits recipients who did not commit fraud, falsely flagging consumers as fraudsters when agency officials erred,” the complaint states. The system’s fraud predictions are “frequently incorrect,” EPIC says. Moreover, the company allegedly has wrongfully withheld information concerning the design, evaluation, and operation of Fraud Detect from government agencies and the general public.

EPIC’s complaint asks the FTC to investigate Thomson Reuters to determine if the company has engaged in unfair and deceptive trade practices under Section 5 of the FTC Act and violations of the Fair Credit Reporting Act. Among the requested injunctive relief, EPIC seeks FTC action to require Thomson Reuters to delete or destroy any data, models, or algorithms related to the Fraud Detect system that are derived from illegally collected, retained, or used consumer data or that were deployed in ways that impose an impermissible risk of errors, biases, or other consumer harms.

News: AINews ConsumerProtection FederalTradeCommissionNews Privacy CyberPrivacyFeed DataPrivacy DataSecurity