A. Class Action Lawsuits Over Insurance Claim Denials

(1) Estate of Lokken v. UnitedHealth Group: The nH Predict Algorithm

On May 5, 2022, Gene B. Lokken, a 91-year-old man living in Wisconsin, fell at his home. He broke his leg and ankle. An ambulance transported him to Aspirus Tomahawk Hospital.

After admission, his condition began to deteriorate. His attending physician recommended hospice care. On May 11, Lokken was transferred to a skilled nursing facility at Tomahawk Health Services. There, he began receiving rehabilitation therapy.

On June 24, an orthopedic surgeon examined his broken leg. The surgeon removed the splint and fitted a removable ankle brace. The doctor instructed the physical therapists to begin weight-bearing walking and mobility training with the brace on. The therapists reported that Lokken was slowly regaining strength and mobility, but that intensive physical therapy remained medically necessary.

From July 1 through July 20, UnitedHealthcare covered Lokken's nursing facility costs. Then on July 20, coverage was abruptly terminated. The notice from UnitedHealthcare stated: "Additional days in the skilled nursing facility are not medically necessary."

Lokken's attending physician said continued physical therapy was necessary. UnitedHealthcare's algorithm said it was not. Whose judgment was correct?

To answer that question, you need to know a name: nH Predict.

nH Predict is an AI program developed by naviHealth, a subsidiary of UnitedHealth Group. The algorithm compares a specific patient against similar patients to predict the required duration of post-acute care. In plain terms, it says: "People like you received this much treatment on average, so that should be enough for you too."

The problem is that patients are not averages. A 91-year-old does not recover at the same pace as a 40-year-old. A patient with diabetes heals differently from one without. But the algorithm ran its own calculations regardless of what the patient's own physician recommended.

On November 14, 2023, the family of Lokken and the family of another deceased patient, Dale Henry Tetzloff, filed a class action lawsuit in the United States District Court for the District of Minnesota. The defendants were UnitedHealth Group, UnitedHealthcare, and naviHealth.

The complaint contained striking allegations.

UnitedHealthcare knew that nH Predict was inaccurate.

Because more than 90% of denial decisions were overturned on appeal.

More than 80% of prior authorization denials were also reversed.

The company knew these facts and continued using the algorithm anyway.

The plaintiffs asserted seven causes of action: breach of contract, breach of the duty of good faith and fair dealing, unjust enrichment, insurance bad faith, Oregon wrongful death, Minnesota unfair insurance practices, and violation of California's Unfair Competition Law.

UnitedHealthcare's rebuttal was predictable. We did not use nH Predict for coverage decisions. The tool was used only to provide guidance to healthcare providers, families, and caregivers about the help and treatment a patient might need. We continue to make coverage decisions based on Medicare coverage criteria and plan terms.

The word Medicare appears here. This is what makes the case complicated. Medicare is a federal health insurance program for Americans aged 65 and older. Medicare Advantage is a program in which private insurers deliver Medicare benefits on the government's behalf. UnitedHealthcare is the largest Medicare Advantage provider in the United States. It provides health insurance to 52.9 million Americans.

The problem is that Medicare law is federal law. Federal law generally takes precedence over state law. This is called federal preemption. UnitedHealthcare's lawyers invoked this principle and moved to dismiss the lawsuit. They argued that all of the plaintiffs' state-law claims were preempted by the Medicare Act, and that the court therefore lacked jurisdiction.

There was another barrier as well: the exhaustion of administrative remedies requirement. If a Medicare Advantage enrollee disagrees with a coverage denial, they must first go through four levels of administrative appeals. They must "exhaust" all these steps before filing suit in federal court. The plaintiffs had not satisfied this requirement.

On February 13, 2025, Judge John R. Tunheim issued his ruling. This federal judge, appointed by President Clinton, faced two legal barriers erected by UnitedHealthcare's defense team.

The first barrier: "These people are not yet entitled to come to court."

Korean legal professionals will find this logic familiar. It is the doctrine requiring exhaustion of administrative proceedings before judicial review. You must complete all internal appeal procedures within the administrative agency before knocking on the courthouse door. Korean Administrative Litigation Act contains the same concept.

What should you do when an insurer refuses to pay for treatment? First, you must file an internal appeal with the insurer. If that is denied, you must proceed to Medicare administrative hearings. Only after completing every step can you file a lawsuit in court. Skip even one step, and the case is dismissed for failure to exhaust.

The problem was time. Completing these procedures can take two years. For a dementia patient expelled from a nursing home, two years is an eternity. Judge Tunheim recognized an exception. The logic paralleled Article 18, Paragraph 2 of Korea's Administrative Litigation Act: an urgent need to prevent serious harm. The plaintiffs had been forced to forgo essential treatment. There was an irreparable threat to life and body.

There was a second argument as well: futility. Even when appeals were filed, UnitedHealthcare used the nH Predict algorithm to issue another denial. It was mechanical rejection on repeat. Going through the preliminary procedures was nothing more than a formality. The process had become an empty shell. The judge acknowledged this point.

The plaintiffs cleared the first barrier.

The second barrier was more complex: federal preemption.

Korea has a unitary legal system. Laws are made by the National Assembly and apply uniformly across the country. The United States is different. Each of the 50 states makes its own laws. The federal government also makes laws. What happens when the two conflict?

The principle is straightforward. The higher law prevails. Federal law wins. But the question is how far it wins. When federal law begins to regulate a particular field, does state law get displaced entirely? Or can state law still operate in areas that federal law does not address?

UnitedHealthcare's lawyers argued as follows. Medicare law governs everything about elderly health insurance. Which treatments are covered, how to challenge denial decisions, all of it is set by federal law. The claims that plaintiffs raised under state law invade the exclusive domain of federal law. They must be dismissed.

Much of this was correct.

Judge Tunheim split the causes of action cleanly in two: the adequacy of benefits and the fairness of process.

The unjust enrichment claim was dismissed. The argument that "more insurance payments should have been made" is a question of benefit standards set by the Medicare Act. It belongs to the federal domain. Claims based on Oregon state law, Minnesota state law, and California state law were dismissed for the same reason. State law cannot encroach upon federal jurisdiction.

But the breach of contract claim survived. The judge's reasoning went like this: Medicare law determines "what is covered." But what UnitedHealthcare promised its own customers is not determined by Medicare law. That lives in the insurance contract.

That contract contained clear language. Physician-led medical review. Decisions considering the individual patient's circumstances. The plaintiffs' argument was straightforward: you broke the promises you wrote in your own contract. An algorithm decided, not a physician. You denied coverage based on statistical averages without examining individual circumstances.

This is not a question of benefit adequacy. It is a question of contractual performance, a matter of private autonomy. It is a breach of obligation. It falls outside the domain regulated by Medicare law.

The contract that UnitedHealthcare itself drafted. The procedures it promised to follow. Whether it honored those commitments was territory that federal law had not preempted.

The plaintiffs cleared the second barrier as well. The gate was narrow, but it was open. It became possible to challenge in court the very manner in which AI determines people's fates.

The ruling carries significant implications. A path has opened for Medicare Advantage enrollees to legally challenge AI-based coverage denials. The shield of federal preemption has been shown to be incomplete. If an insurer promised physician review in its contract, it must keep that promise.

The case has not yet reached trial on the merits. Discovery into how UnitedHealthcare actually used nH Predict will proceed. More time is needed before the truth comes to light.

One question remains. If 90% of denials are overturned on appeal, why does the insurer keep denying? The answer is simple: most patients never appeal. According to 2022 data, only 0.2% of denied patients actually file an appeal. 998 out of 999 just give up. This is the economics of the algorithm. Deny, and some will appeal and win, but the vast majority will surrender. The treatment costs saved from those who give up become profit. In 2023, for-profit health insurers earned a combined $70.7 billion.

While the Lokken case was proceeding, another lawsuit in California was posing a similar question.

(2) Kisting-Leung v. Cigna: The 1.2-Second Automatic Denial

In July 2023, a woman named Suzanne Kisting-Leung filed a complaint in a California federal court.

The defendant was Cigna.

Her story went like this. She had two ultrasound exams to screen for ovarian cancer. A cyst was found on her left ovary. The tests cost $723. Cigna denied the claim.

Under Cigna's medical coverage policy, transvaginal ultrasounds for screening or surveillance of women at high risk for ovarian or endometrial cancer are considered 'medically necessary.' She fell squarely into that category. So why was her claim denied?

The answer came from a ProPublica investigation.

In March 2023, ProPublica published a bombshell article based on Cigna's internal documents and interviews with former employees.

Cigna had been using a system called PxDx. The system allowed its doctors to automatically deny claims without ever opening a patient's file.

Look at the numbers. Over a two-month period in 2022, Cigna's doctors used PxDx to deny more than 300,000 claims. The average review time per claim was 1.2 seconds.

1.2 seconds. Not nearly enough time to read a patient's medical records. Not even enough time to open the file.

One former Cigna doctor told ProPublica: "We literally click and submit. It takes ten seconds to process fifty claims at once." Another doctor, Dr. Cheryl Dopke, processed 60,000 claims in a single month in 2022. Dr. Alan Muney, the developer of PxDx and Cigna's former chief medical officer, confirmed it to ProPublica: "No doctor, no nurse, no one reviews PxDx cases. This has undoubtedly saved billions of dollars."

Billions of dollars. That was the point.

Kisting-Leung and other plaintiffs initially brought four state-law claims: breach of the duty of good faith and fair dealing, violation of California's Unfair Competition Law, interference with contractual relations, and unjust enrichment.

A year later, in June 2024, their legal strategy shifted. In the Third Amended Complaint, they pivoted to claims under ERISA, the Employee Retirement Income Security Act, the federal law governing employer-sponsored health insurance plans.

Why the change? ERISA is a double-edged sword. On one side, it preempts state-law claims, narrowing the plaintiffs' options. On the other, it offers a powerful concept: fiduciary duty. An insurer has a fiduciary obligation to act in the interest of its enrollees. Does denying claims through an algorithm, without looking at a patient's file, meet that obligation?

On March 31, 2025, Judge Dale Drozd ruled on Cigna's motion to dismiss. The result was a partial victory.

First, there was the question of standing.

Cigna argued that three plaintiffs, Kisting-Leung, Thornhill, and Bredlow, had not actually been denied through PxDx. Cigna submitted a sworn declaration from Dr. Julie B. Kessel, a medical officer in Cigna's Clinical Performance and Quality division. According to her statement, the claims of these three plaintiffs were not processed through PxDx.

Here is what made it interesting. Cigna claimed it sent a disclosure notice to the doctor and patient every time PxDx was used. These three plaintiffs never received such a notice. Therefore, their claims were not processed through PxDx. The judge accepted this logic. He ruled that Kisting-Leung, Thornhill, and Bredlow lacked standing for PxDx-related claims. But he allowed them to proceed with their general claims of improper benefit denial.

What mattered more was that the ERISA fiduciary duty claim survived. The judge rejected Cigna's argument. Cigna had asserted it held discretionary authority to interpret the terms of its insurance policies and that using an algorithm fell within that discretion. The judge found this to be an abuse of discretion.

The core reasoning went like this. The insurance contract specified that medical necessity determinations would be made by a 'medical director.' The plaintiffs argued that delegating medical necessity decisions to the PxDx system violated this contractual term. The judge found that the plaintiffs had alleged sufficient facts to support this claim.

Cigna's response is worth noting. A Cigna spokesperson claimed that PxDx does not use AI. The company said it resembles software that other health insurers and the Centers for Medicare and Medicaid Services (CMS) have used for years. Cigna added that PxDx applies only to about 50 'low-cost tests and procedures,' that reviews occur after treatment, and that notifications are sent to doctors and patients.

Cigna's lawyers criticized the ProPublica article as 'misleading and inflammatory.' But the judge said that at this stage, he did not need to determine the accuracy of the reporting. The facts would come out through discovery.

On June 9, 2025, an initial scheduling conference was held in the Sacramento federal court. It went ahead as planned.

After Judge Drozd's March ruling, the plaintiffs' side faced a choice. File a Fourth Amended Complaint within 21 days, or declare they would fight with the claims that survived. They chose the latter. No more revisions to the complaint. They would fight with what they had. Two weapons remained. ERISA fiduciary duty breach. And violation of California's Unfair Competition Law. Both aimed at the same question: did Cigna abuse its discretion by replacing the 'medical director review' promised in the contract with the PxDx algorithm?

Summer passed and fall arrived. On September 2, 2025, Judge Chi Soo Kim approved a 'Modified Stipulated Protective Order.'

In American litigation, there is a phase before trial where both sides show each other their cards. It is called 'discovery.' The problem is that this process inevitably exposes sensitive information: trade secrets, internal emails, technical documents.

A protective order is a safeguard for situations like this. In plain terms, it says: 'Show your secrets, but don't spread them around.' Documents designated as confidential can be viewed only by people involved in the case. They cannot be copied or disclosed outside the courtroom. When the trial ends, they must be returned or destroyed.

The word 'stipulated' means both plaintiff and defendant agreed to these rules. 'Modified' means they revised the original version later.

Discovery. The most expensive stage of American litigation, and the stage where the most comes to light.

The plaintiffs' lawyers could now demand Cigna's internal documents. How PxDx works. What criteria it uses to decide denials. What the doctors actually see and what they don't. What happens in those 1.2 seconds.

Cigna keeps repeating the same talking points. PxDx is not AI. It is industry-standard software. The Centers for Medicare and Medicaid Services (CMS) has used similar systems for years. It applies only to about 50 low-cost tests and procedures. Reviews happen after treatment. Notifications go to doctors and patients.

But what matters in court is not whether PxDx qualifies as AI. What matters is whether Cigna kept its promise to its customers. The insurance contract said 'a medical director determines medical necessity.' It did not say an algorithm could take over that role. The lawsuit has now entered its most tedious and most critical phase.

Lawyers will demand documents, the other side will resist, and the judge will mediate. This process will continue for months. Depositions will begin. Dr. Alan Muney, the former chief medical officer who developed PxDx, may take the witness stand. So may Dr. Julie B. Kessel.

Cigna's lawyers, who called the ProPublica article 'misleading and inflammatory,' may now have to produce the raw data behind it. The data showing more than 300,000 claims denied over two months in 2022.

There is an irony here. Cigna claimed it sent a disclosure notice to the doctor and patient every time PxDx was used. That claim succeeded in blocking three plaintiffs' PxDx-related causes of action. They never received a notice, so they were never denied through PxDx, the logic went. But that same logic is tightening around Cigna.

If a notification system exists, it means Cigna knows exactly who was denied through PxDx. It means records exist. Those records will surface during discovery.

Meanwhile, in Minnesota, the Lokken case is following a similar path. UnitedHealthcare's nH Predict. Cigna's PxDx. The two lawsuits pose the same question: can an algorithm make coverage decisions in place of a human doctor?

The Lokken case concerns Medicare Advantage plans, and the Kisting-Leung case concerns employer-sponsored health insurance. The former falls under Medicare law; the latter under ERISA. The statutes differ, but the substance is the same. When an insurer replaces the 'human judgment' it promised with a machine's calculation, is that a breach of contract?

The answer has not arrived yet. But the question has already reached the courtroom. There is an important difference between the Lokken case and the Kisting-Leung case. Lokken involves a Medicare Advantage plan; Kisting-Leung involves employer-sponsored health insurance. One is governed by Medicare law, the other by ERISA. Yet both cases ask the same question: can an algorithm make coverage decisions in place of a human doctor?

(3) The Rise in AI Denial Rates: From 10.9% to 22.7%

Numbers don't lie.

In October 2024, the U.S. Senate Committee on Investigations released a damning report. It examined prior authorization denial rates at three major insurers: UnitedHealthcare, Humana, and CVS.

In 2020, the average prior authorization denial rate across these three companies was 10.9%. By 2023, it had jumped to 22.7%, more than doubling. The denial rate doubled in three years.

What changed? Did patients suddenly start requesting more unnecessary treatments? That seems unlikely. What changed was how decisions were being made. AI and algorithms had entered the picture.

Consider what these numbers mean. Prior authorization is a system in which an insurer must approve a specific treatment or procedure before a patient can receive it. Even if a doctor determines that an MRI is necessary, the patient must pay the full cost out of pocket or forgo treatment entirely if the insurer does not approve it.

The denial rate rose from 10.9% to 22.7%. That means a significant portion of treatments that would previously have been approved are now being denied. Cancer screenings, physical therapy, pre-surgical tests, specialist referrals. These are being denied.

According to a November 2024 Gallup survey, only 44% of Americans rated the quality of U.S. healthcare as 'excellent' or 'good.' This is the lowest since 2001. In the same survey, 36% of adults reported experiencing an insurance claim denial. Of those, 60% had been denied multiple times. The appeals data is telling. Only 0.2% of denied patients actually file an appeal. But more than 80% of those who do appeal win. This suggests that a large share of the original denial decisions were not justified.

Why don't patients appeal? Because the process is complex and time-consuming. Medicare Advantage has a four-stage administrative appeals process. Each stage requires paperwork, waiting, and if denied, advancing to the next level. For a sick patient, enduring this process is difficult.

Insurers know this. It is their business model. Deny first, pay only the few who appeal. The majority will give up.

In December 2024, UnitedHealthcare CEO Brian Thompson was fatally shot in Manhattan, New York. This tragic event ignited nationwide fury against the health insurance industry. Stories poured across social media from people who had suffered insurance denials.

Violence cannot be justified under any circumstances. But this event revealed how deep the frustration Americans feel toward the health insurance system runs.

AI has made this system more efficient. The question is: efficient for whom? For insurers, AI is a cost-cutting tool. For patients, AI is one more barrier.

The Lokken lawsuit and the Kisting-Lung lawsuit are the first attempts to challenge this system. Whatever the courts decide, these lawsuits have brought an important question into public debate: when an algorithm makes decisions about human health, who should be held accountable?

The next section examines another dimension of this question. When AI commits an error in diagnosis or treatment, who bears liability for medical malpractice?

B. AI Misdiagnosis: Who Is Liable?

(1) AI Diagnostic Errors and Establishing Negligence

In 2024, three out of five American physicians were using AI in clinical practice. This was according to a survey by the American Medical Association (AMA).

The number is striking. Just a few years ago, AI was a research topic confined to laboratories. Now it sits in exam rooms facing patients. It reads X-rays, analyzes skin lesions, and detects cancer.

A question arises. What happens when AI is wrong?

According to 2024 data, medical malpractice claims involving AI tools increased 14% compared to 2022. Most occurred in radiology, cardiology, and oncology. Missed cancer diagnoses were the most frequent type.

The core concept of medical malpractice law is the 'standard of care.' Put simply, it asks: 'What would a reasonable physician have done in the same situation?' If a physician falls short of this standard and the patient suffers harm as a result, negligence is established.

The problem is that the equation becomes complicated once AI enters it.

Consider Scenario 1. A physician blindly followed an AI system's recommendation, the AI was wrong, and the patient was harmed. Whose fault is it?

Scenario 2. A physician ignored an AI system's recommendation, the AI was correct, and the patient was harmed. Whose fault is it?

Scenario 3. A physician did not use an AI system at all, the error could have been avoided if AI had been used, and the patient was harmed. Whose fault is it?

Under the current legal framework, the answer in all cases is 'the physician.' In April 2024, the Federation of State Medical Boards (FSMB) issued recommendations. State medical boards are the bodies that regulate physician licensing and discipline. The FSMB's recommendation was clear: when AI technology causes a medical error, the clinician, not the AI manufacturer, should bear responsibility.

Their reasoning went like this: 'As with any other tool or differential diagnosis used in diagnosis or treatment, the medical professional is responsible for ensuring the accuracy and veracity of evidence-based conclusions.'

In other words, AI is just a tool, like a stethoscope or an MRI machine. Even if the tool provides incorrect information, the physician interprets it and makes the decision. The final responsibility therefore rests with the physician.

But there is a flaw in this logic. A stethoscope does not render judgments on its own. An MRI machine does not say 'this is cancer.' AI does. AI makes recommendations. Sometimes those recommendations are highly specific and delivered with confidence.

There is a phenomenon called 'automation bias.' Humans tend to accept information provided by computers uncritically, especially when they believe the computer processes more data and is more accurate than they are. Physicians are not exempt.

A study conducted by Johns Hopkins researchers found that physicians were more likely to consult AI in straightforward cases but tended to avoid AI in complex scenarios where outcomes were less predictable. The reason was malpractice concerns.

This creates a paradoxical situation. If physicians avoid AI in the complex cases where it could help most and use it only in simple cases where it is less needed, AI's potential value goes unrealized.

There have been no major rulings on AI-related medical malpractice yet. Cases are only now beginning to reach the courts. But legal scholars have proposed several theories of liability.

First, physician negligence. A physician may be found negligent for blindly following an AI recommendation, or for failing to use a validated AI tool when the standard of care requires it.

Second, institutional liability. This includes respondeat superior, negligent credentialing, and failure to validate, train, or update AI systems.

Third, developer product liability. This could cover defective design, failure to warn, or inadequate training data.

Most AI medical device terms of service include disclaimer clauses. 'Final responsibility rests with the physician.' This is a mechanism for manufacturers to avoid liability.

But how far courts will accept these disclaimers remains uncertain.

There is one exception. A company called Digital Diagnostics developed IDx-DR, a diagnostic system for diabetic retinopathy.

The company carries its own medical malpractice insurance and assumes liability for injuries caused by its system. This is a rare case in the industry.

The standard of care itself is also shifting.

In May 2024, the American Law Institute approved its first Restatement on medical malpractice law. The document reflects a move away from strict reliance on customary practice toward a more patient-centered concept of 'reasonable care.' Courts can now consider evidence-based guidelines and modern standards. The implication for AI is this: as AI-based devices and workflows become widespread and demonstrate proven usefulness, what a 'reasonable physician' is expected to do will shift accordingly. Not using AI could itself become negligence.

The medical malpractice insurance industry is adapting as well. Some insurers have introduced AI-related exclusion clauses. Others require physicians to undergo AI training in order to maintain coverage.

Jared Kaplan, CEO of Indigo, said AI is a 'net positive' and will 'lower malpractice rates in the long run.' But he also acknowledged that AI presents a 'new threat' legally, and that there is 'no black-and-white answer' to who is at fault.

(2) Black-Box AI and the Physician's Duty to Explain

When an AI system says 'this is a malignant tumor,' what must the physician explain to the patient?

Traditionally, a physician must explain the diagnosis and treatment options to the patient and obtain consent. This is called 'informed consent.' The physician must explain risks and benefits so the patient can make a decision based on adequate information.

When AI is involved in the diagnostic process, new questions arise. Must the physician inform the patient that AI was used? Must the physician explain how the AI reached its conclusion?

The problem is that many AI systems are 'black boxes.' Input goes in, output comes out, but what happens in between is difficult to know. Deep learning algorithms adjust millions of parameters to recognize patterns. Explaining why they reached a particular conclusion is hard.

The physician may not understand how the AI works. In such a situation, what can they explain to the patient?

A research team led by Associate Professor Sara Gerke of the University of Illinois Urbana-Champaign conducted a focus group study with 18 surgeons from the United States and the EU. The results were published in Annals of Surgery Open.

The surgeons generally accepted that even when using AI, ultimate responsibility rests with them. They viewed AI as not currently part of the standard of care but expected it would become so in the future. Most were skeptical that manufacturers would bear significant liability unless there was a clear defect. Some called for shared responsibility when a surgeon properly followed AI instructions.

Patient consent emerged as another topic. The surgeons felt that when AI is used, patients should be informed, especially when following or rejecting AI advice could change outcomes. California has begun legislating in this direction. AB 3030 requires, starting January 1, 2025, that healthcare providers disclose when generative AI is used to send communications containing clinical information to patients. The message must include a disclaimer that it was generated by AI and clear instructions on how to communicate with the provider without an AI response.

But this concerns communications. Disclosure obligations regarding AI use in diagnostic or treatment decisions remain unclear.

Technical solutions to the black box problem are also being explored. There is a field called Explainable AI, or XAI. It is technology designed to make AI capable of explaining why it reached a particular decision. For example, if AI detected pneumonia on a chest X-ray, it can visually highlight which regions contributed to that conclusion.

Explainable AI has its limits, though. Fully explaining the decision-making process of complex deep learning models remains difficult. And even when explanation is possible, whether that explanation is medically meaningful is a separate question.

The FDA's January 2025 guidance emphasizes transparency. Manufacturers must document how their AI works, what data it was trained on, and what its known limitations are. But whether this information actually reaches patients depends on the judgment of healthcare providers.

One core principle of medical ethics is patient autonomy. Patients have the right to make informed decisions about their own treatment. If AI influences that decision, don't patients have the right to know?

The legal answer to this question is still evolving.

(3) Allocating Liability Between Physician Judgment and AI Recommendations

A final question.

When AI says one thing and the physician chooses another, who should be held liable?

Under current U.S. medical malpractice law, liability turns on the standard of a "reasonable physician in similar circumstances." Whether AI was used or not, courts judge the physician's actions. There is no legal doctrine assigning shared liability to an AI system.

Aviation takes a different approach. When automation fails, responsibility is distributed among the pilot, the system, and the manufacturer. Could a similar approach work in medicine?

Legal scholar W. Nicholson Price has proposed a framework for distributing liability more equitably. The European Union's AI Liability Directive is also moving toward applying no-fault liability to high-risk AI failures.

A scholar named Chan proposed a "common enterprise" model. The idea is to treat the physician, manufacturer, and hospital as a common enterprise for purposes of liability. This approach represents a shift "away from the individualistic concept of responsibility embodied in negligence and products liability toward a more distributed concept."

Another proposal follows the vaccine injury compensation program model. The United States has a national program providing no-fault compensation for vaccine-related injuries. Could a similar program be created for AI medical devices?

The most radical proposal is granting AI legal personhood. That would allow injured patients to sue the AI device directly. But this raises numerous philosophical and legal problems.

Realistically, the most likely near-term change is the evolution of the standard of care. If following an AI recommendation becomes what a "reasonable physician" would do, then a physician who followed AI could be protected. Conversely, if not using AI comes to be seen as unreasonable, a physician who did not use AI could face liability.

The problem is this transition period. While AI is not yet part of the standard of care, physicians must make decisions amid uncertainty. Follow the AI, or ignore it? Either way, if things go wrong, they may be held responsible.

An experimental study published in 2021 surveyed 2,000 U.S. adults on jury judgments in AI-use scenarios. The results were interesting.

When AI recommended standard treatment and the physician followed it, liability decreased even if harm occurred. When AI recommended non-standard treatment and the physician rejected it in favor of standard care, there was no similar protective effect.

The study's conclusion is encouraging: "The tort system is unlikely to deter the use of AI precision medicine tools and may even encourage their use."

But this reflects juror intuition only. Actual verdicts may differ.

The deeper AI integrates into medicine, the more complex questions of liability will become. Current legal frameworks were not designed with AI in mind. Adaptation is necessary.

When an insurance algorithm denies treatment, when a diagnostic AI misses cancer, when a prescribing AI recommends the wrong drug, someone must be held accountable. Whether that "someone" should always be the physician, or whether liability should be distributed more equitably, is the question medical law must answer over the next decade.

The next section examines California's SB 1120 and the FDA's regulation of AI medical devices. It covers how legislators and regulators are responding to this new reality.

C. The Transparency California Demanded

(1) California SB 1120: AI Disclosure Requirements

On September 28, 2024, California Governor Gavin Newsom signed a bill sitting on his desk.

SB 1120. Nicknamed the "Physicians Make Decisions Act," the bill could be summarized in a single sentence: humans, not algorithms, must make medical decisions.

The reason this law was needed is straightforward.

Insurers had begun using AI to cut costs, and as a result, patients were being denied necessary treatment. The California Medical Association, representing 50,000 members, sponsored the bill. Their argument was clear: AI tools lack the ability to recognize and accommodate the unique circumstances of individual patients.

The law's provisions are surprisingly specific.

Starting January 1, 2025, all health plans and disability insurers doing business in California must follow these rules. Coverage decisions based on medical necessity must be made by a licensed physician or qualified medical professional. AI tools cannot decide alone.

A more important provision exists. The law restricts the data AI systems may use when making decisions. Decisions cannot be based solely on group datasets.

The system must consider the individual patient's medical records, clinical history, and the specific clinical circumstances provided by the treating physician. The approach nH Predict took, making decisions by comparing patients to "similar patients," is no longer permitted.

Anti-discrimination provisions were included as well. AI tools must not discriminate based on race, gender, age, disability, or other protected characteristics. This aligns with the federal Section 1557 final rule revised in July 2024, which also prohibits AI tools and algorithms from discriminating against marginalized patients. Transparency requirements are included too.

AI algorithms must be open to inspection for audit or compliance review. Insurers must disclose written policies regarding AI use to healthcare providers, subscribers, and members of the public who request them. The era of black boxes is over.

The law's definitions are also notable. It defines "artificial intelligence" as "an engineered or machine-based system that varies in its level of autonomy and that can, for explicit or implicit objectives, infer from the input it receives how to generate outputs that can influence physical or virtual environments." However, it does not define "algorithm" or "other software tools." Legal experts point out this could be interpreted broadly. The excuse Cigna claimed, that "PxDx is not AI," may no longer hold.

The enforcement mechanism is strong. The California Department of Managed Health Care (DMHC) and the Department of Insurance (CDI) have authority to impose administrative penalties. Willful violations are treated as criminal offenses.

This law does not prohibit insurers from using AI. It says only that AI cannot be the final decision-maker.

But if every AI decision requires human review, what is the point of using AI in the first place? The core benefit of AI, improved efficiency, could be nullified. This is why we must watch how insurers comply with this law.

On the same day, Governor Newsom also signed AB 3030. This law imposes disclosure obligations when generative AI is used to send communications containing clinical information to patients.

Patients have the right to know whether the message they received was generated by AI.

California is the largest state in the United States. Other states are likely to use this law as a model. Several states are already reviewing similar bills. A patchwork of AI medical regulation is taking shape. At the federal level, a different wind is blowing. The Trump administration revoked Biden's AI Executive Order 14110 on January 20, 2025. Three days later, it issued a new executive order titled "Removing Barriers to American Leadership in Artificial Intelligence." The goal: cut regulation and promote innovation. Tension between the federal and state governments is mounting.

The FDA's regulation of AI medical devices continues to evolve amid this tension.

(2) FDA AI Medical Device Regulation

On January 7, 2025, the FDA released a long-awaited guidance document.

It carried a lengthy title: "Artificial Intelligence-Enabled Device Software Functions: Lifecycle Management and Marketing Submission Recommendations."

The 85-page draft guidance opened a new chapter in AI medical device regulation.

Start with the numbers. As of July 2025, the FDA has authorized more than 1,250 AI-enabled medical devices. That is an increase of over 300 from the 950 recorded in August 2024, in less than a year. Most fall in the radiology field. They read X-rays, analyze MRIs, and detect cancer.

The problem is that these devices keep learning and changing. Traditional medical devices stay the same once approved. A pacemaker works the same way it did ten years ago. AI is different. It gets retrained on new data. Its algorithms get updated. Yesterday's AI and today's AI can be two different things.

The FDA's solution is the Predetermined Change Control Plan, or PCCP. If a manufacturer submits a plan in advance saying, "We will make these kinds of changes in this way," then changes within that scope can proceed without a new FDA review. A final guidance published in December 2024 established this framework.

A PCCP must include three core elements.

First, a description of the planned modifications. The manufacturer must detail what changes it intends to make, such as algorithm retraining or integration of new datasets.

Second, a modification protocol. This must specify data management practices, algorithm retraining methods, and performance evaluation procedures.

Third, an impact assessment. It must evaluate the risks and benefits of the proposed changes and present risk mitigation strategies. The January 2025 guidance goes further. It emphasizes a Total Product Life Cycle, or TPLC, approach. Safety and effectiveness must be considered across the device's entire lifespan, from design through retirement. Transparency and bias mitigation are the central themes.

On the transparency front, manufacturers must explain how their AI works: what data it was trained on, which patient populations it was tested in, and what its known limitations are. The "Transparency Guiding Principles for Machine Learning-Enabled Medical Devices," published in June 2024, set this direction.

Bias mitigation is also becoming mandatory. Manufacturers must collect evidence showing that their AI devices perform comparably across all relevant demographic groups, including race, ethnicity, sex, and age. An AI trained on data from white males must be proven accurate for Black women as well.

But large language models like ChatGPT are entering the medical space. They draft clinical documents, assist with diagnoses, and converse with patients. This guidance does not directly address special considerations for generative AI. In its request for public comments, the FDA asked for feedback on "the appropriateness of recommendations for emerging technologies such as generative AI."

There is also a staffing problem. As of September 2025, the FDA's workforce had shrunk by roughly 15% compared to 2023, a reduction of about 2,500 employees. That constrains the agency's capacity to evaluate AI medical devices quickly and thoroughly.

Most AI medical devices are cleared through the 510(k) pathway. That accounts for 97% of the total. Under this pathway, a new device only needs to show it is "substantially equivalent" to an already-cleared device. No new clinical trials are required. Critics argue that this fails to adequately assess the unique risks posed by AI devices.

There is a connection to medical malpractice liability. For FDA-authorized AI products, state tort claims against manufacturers may be limited by federal preemption. This can make it harder for patients to sue a manufacturer when a defective AI device causes harm. On the other hand, for AI tools not subject to FDA regulation, a physician's clinical judgment becomes the key factor in determining liability. On February 18, 2025, the FDA held a public webinar on the guidance. The comment period closed on April 7, 2025. When the final guidance will be issued remains uncertain. Whether the Trump administration's deregulatory posture will affect the guidance's fate is another open question.

One thing is clear. AI is already deeply embedded in medical practice. The question is whether regulation can keep pace with the technology. California put the brakes on insurers' use of AI, and the FDA is trying to manage medical devices across their full lifecycle. But technology moves faster than rules.

Mrs. Lokken's husband Gene was 91 years old and receiving care at a nursing facility after a fall. His orthopedic surgeon said he needed continued physical therapy. But the nH Predict algorithm reached a different conclusion. "No additional inpatient days are medically necessary." That was the machine's decision.

The new regulations are designed to prevent this from happening again. But passing a law does not solve the problem by itself. Enforcement has to follow. Whether insurers will honor the spirit of the law or look for loopholes is something only time will tell.

The next chapter examines financial services and algorithmic collusion. It looks at how AI can become a tool for discrimination and collusion in lending decisions and pricing, and how regulators are responding.