A. Algorithmic Lending Discrimination (Same Credit, Different Rates)

(1) Student Loan AI Discrimination Class Action

On July 10, 2025, Massachusetts Attorney General Andrea Joy Campbell stood at a press conference podium. In her hand was a document worth $2.5 million. The target was a student loan refinancing company called Earnest Operations. Attorney General Campbell stepped up to the microphone and said: "Earnest's AI model unfairly placed historically marginalized student borrowers at risk."

What had happened?

The story goes back to 2014. Earnest had the typical Silicon Valley startup origin narrative. Its founders believed traditional credit scoring was outdated.

Judging people by FICO scores alone? FICO stands for Fair Isaac Corporation. It is the most widely used credit scoring system in the United States. It is expressed as a number between 300 and 850. The higher the score, the better the credit. Lenders check this score for loan applications, credit card approvals, even apartment lease agreements. It is similar in concept to Korea's credit rating system.

They thought more data needed to be examined. Alma mater. Major. Employment history. Feed these variables to an AI, and the AI could predict more accurately who would repay their debts.

The problem lay in one of those variables. It was something called the Cohort Default Rate, or CDR.

This is a number published by the U.S. Department of Education for each university. It indicates how many graduates from that school failed to repay their federal student loans.

Earnest's AI looked at this number. And it made a straightforward calculation. If the CDR of your alma mater is high, you are also likely to default on your debt. So your interest rate goes up, or your loan application is rejected outright. On the surface, it seems reasonable. If you graduated from a school whose alumni have poor repayment records, isn't caution justified?

But prosecutors saw something else. The United States has what are known as Historically Black Colleges and Universities, or HBCUs. Places like Howard University and Spelman College. These schools are attended predominantly by African American students. And these schools tend to have CDRs above average. Why? Because those students' families have not been wealthy for generations. Late repayment of student loans is not a matter of individual creditworthiness; it is the result of economic inequality accumulated across generations.

Earnest's AI did not understand this context. The AI only saw patterns.

Graduates from this school repay late. Therefore, dock this person's score. But that "this person" could be a Black young man who graduated top of his class from Howard University and landed a job at Google. His credit score might be perfect. His salary might be $150,000. Yet the AI still docks his score. Because the alumni of his school had poor repayment records.

In legal terms, this is called "Disparate Impact." Put simply: even if you did not intend to discriminate against Black people, if your system produces results that are systematically unfavorable to them, it is illegal.

A university's CDR is not a race variable. But it is closely correlated with race. Lawyers call this a "Proxy Variable." It is a detour that allows you to infer race without asking directly.

Earnest had another problem. Something called a "Knockout Rule." If a non-citizen applicant did not hold a green card (permanent residency), the AI was configured to automatically reject them at an early stage of review. An Indian engineer on a work visa earning $200,000 a year could be turned away at the door solely because they lacked a green card.

Prosecutors determined this constituted discrimination based on national origin.

The conditions were strict. Earnest must immediately stop using the CDR variable. It must abolish the automatic rejection rule based on immigration status. It must conduct mandatory fairness testing on all AI models. It must establish a documented corporate governance system and regularly report compliance to prosecutors.

Earnest denied the allegations. The company maintained it had not violated any laws. It said it agreed to the settlement "to avoid protracted litigation." This is the standard phrasing American corporations use when announcing penalty settlements. A way to pay money without admitting fault.

The reason this case matters lies elsewhere. While the federal government was stepping back from AI discrimination enforcement, state governments began filling that vacuum. After the Trump administration's return to power, the Consumer Financial Protection Bureau's aggressive enforcement slowed. The Federal Trade Commission's monitoring of algorithmic discrimination loosened. State attorneys general in Massachusetts, California, Oregon, and New Jersey pushed into the gap.

Attorney General Campbell said at the end of the press conference: "No matter how advanced technology becomes, it cannot serve as an excuse to circumvent civil rights and consumer protections."

That statement was a warning to the entire financial technology industry.

(2) UC Berkeley/Urban Institute Research Findings

In 2018, an unusual tension hung in the air at the UC Berkeley Haas School of Business research lab.

Finance professor Adair Morse and law professor Robert Bartlett were staring at their screens. They were analyzing millions of mortgage loan records.

The original purpose of their research had been to praise fintech. They expected to find that algorithms had eliminated the biases of human loan officers. They believed that cold mathematics had solved the chronic problem of unconscious discrimination when a Black customer walked in.

The data showed the opposite.

The researchers analyzed mortgage data from 2008 through 2015.

They found that algorithm-based fintech lenders were charging Black and Latino borrowers an average of 7.9 basis points (0.079 percentage points) more in interest than white borrowers.

0.079 percentage points. It sounds like a small number. But multiply it across the entire U.S. lending market and a different story emerges. According to the researchers' calculations, minority borrowers were paying approximately $765 million extra per year because of this rate differential.

Professor Morse put it this way: "The mode of lending discrimination has shifted from human bias to algorithmic bias." There was a bitter smile in those words. "Even when the people writing algorithms have the intention of creating a fair system, their programming is having a discriminatory impact on minority borrowers."

How is this possible? Algorithms do not look at race. At least not directly. Under U.S. law, using race as a variable in loan underwriting is illegal. But algorithms look at everything except race. Residential ZIP code. Shopping patterns. Bank account transfer habits. Smartphone model used. These variables are highly correlated with race. The researchers called this "Algorithmic Strategic Pricing."

Fintech companies' AI predicts who will comparison-shop. Some customers check rates at multiple banks and chase the best terms. These customers must be offered competitive rates. Otherwise they will go to another bank.

On the other hand, there are customers unlikely to comparison-shop. People living in areas underserved by financial institutions. People with limited internet access. People too busy with daily life to visit multiple banks. These customers can be offered slightly higher rates. They won't compare anyway.

The problem is that these characteristics overlap with race. Areas underserved by financial institutions are historically neighborhoods where people of color live. People who lack time to compare rates across banks are often low-income workers. The algorithm does not ask about race. But it figures it out through a back door.

The researchers made one interesting finding as well. There was a way in which algorithms were better than humans. In loan approval and rejection decisions, algorithms were less discriminatory than human underwriters.

The rate at which Black or Latino applicants were denied loans outright because of their race decreased. But discrimination in the interest rates applied after loan approval actually increased.

This is a subtle distinction. Opening the door for someone and how they are treated once inside are two different matters.

The Urban Institute's 2024 analysis produced even more striking numbers. In AI-based lending models, Black applicants and applicants of color were more than twice as likely to be denied a loan compared to white applicants. This was a gap not explained by differences in credit history. These studies were directly cited in the CFPB's August 2024 guidelines. The Consumer Financial Protection Bureau stated: "New technology does not create an exception to federal consumer financial protection law." The mere fact of using AI does not exempt a company from legal liability for discriminatory impact.

The financial industry was bewildered. They had genuinely believed AI would eliminate bias. Where could racism possibly hide in cold mathematics? But the data the mathematics learned from was already contaminated. For the past 50 years, human bankers had been reluctant to lend to Black people. As a result, Black borrowers' credit histories in the data were thin or poor. The AI looked at this data and learned. Black people (or people with patterns similar to Black people) are risky.

The algorithm was a mirror that mathematically justified the biases of the past and projected them into the future.

B. The Apple/Goldman Sachs Case

(1) CFPB $89 Million Penalty

On August 20, 2019, Apple partnered with Goldman Sachs to launch the Apple Card. The advertising was dazzling. A physical card made of titanium. A clean design. "The most consumer-friendly credit card." Wall Street's titan Goldman Sachs meeting Silicon Valley's icon Apple; everyone expected a revolutionary financial product to be born.

Four days before this splashy launch, on August 16, 2019, a report went up to the Goldman Sachs board of directors. The title was simple.

The dispute resolution system was "not fully ready." There were technical problems. The system that receives, investigates, and refunds when a customer reports a billing error. It wasn't working properly.

The board received the report. Four days later, they pushed ahead with the launch.

Why? The answer was in the partnership contract. Every time Goldman Sachs delayed the launch by 90 days, Apple could impose a $25 million penalty.

Pushing back the launch date meant paying tens of millions of dollars. Goldman Sachs did the math. The cost of problems later because the system was incomplete, versus the penalty owed to Apple right now. Which was greater? They chose to launch.

Five years later, that calculation proved wrong.

On October 23, 2024, the Consumer Financial Protection Bureau (CFPB) imposed more than $89 million in fines and restitution on Apple and Goldman Sachs. Goldman Sachs received a $45 million fine and $19.8 million in consumer restitution. Apple received a $25 million fine. And Goldman Sachs faced an even more devastating sanction: it was banned from launching any new credit card products until it submitted a "credible plan to ensure legal compliance." The king of Wall Street had its hands tied in the credit card business. The CFPB's findings were damning. Thousands of customer disputes were not properly transmitted from Apple to Goldman Sachs. Even disputes that were transmitted were not properly investigated by Goldman Sachs. Customers waited months for refunds.

Meanwhile, their credit scores were marked "delinquent." Goldman Sachs's automated system had classified disputed transactions as overdue. The truth was a billing error, but the customer's credit was ruined.

There was another problem. Apple advertised that "interest-free installments are automatically applied for certain device purchases." But in reality, many customers were automatically enrolled in standard revolving payments with interest. They bought iPhones thinking they were interest-free, only to receive bills with interest charges months later.

CFPB Director Rohit Chopra said: "Apple and Goldman Sachs illegally dodged their legal obligations to Apple Card borrowers. Big Tech companies and large Wall Street banks should not act as though they are exceptions to federal law."

Neither company admitted wrongdoing. The standard language appeared: "without admitting or denying the allegations." Goldman Sachs stated that it had "worked diligently to address certain technical and operational issues that arose after launch." Apple said it "strongly disagreed with CFPB's characterizations but agreed to the settlement."

But the market had already reached its verdict. Goldman Sachs tried to exit the Apple Card business. It offered the partnership acquisition to other banks. No one stepped forward willingly. Who would want to take on this bomb?

In January 2026, JPMorgan Chase decided to acquire Apple Card. It was a $2.2 billion deal. The transition period is expected to take 24 months. Goldman Sachs's consumer finance experiment ended with losses exceeding $1 billion.

(2) Apple Card Algorithm Discrimination Controversy

Five years before the $89 million fine, Apple Card faced a different kind of crisis. In November 2019, Danish software developer David Heinemeier Hansson (DHH) posted a furious message on Twitter.

"The Apple Card is a sexist program."

Hansson is a famous figure in the programming world. He created a web framework called Ruby on Rails. He had over 350,000 Twitter followers. His post spread instantly.

His claim was this.

He and his wife Jamie file their taxes jointly. They share the same assets. His wife's credit score is higher than his. Yet the Apple Card algorithm gave him a credit limit 20 times higher than his wife's.

Hansson called Apple customer service. "Why is my wife's limit so low?" The representative couldn't answer. "That's just what the algorithm decided."

A few days later, Apple co-founder Steve Wozniak chimed in. "The same thing happened to me." Wozniak and his wife share all their accounts. They file the same tax return. Yet he received a limit 10 times higher than his wife's. Even the man who built Apple couldn't understand Apple's algorithm.

New York State Department of Financial Services (NYDFS) Superintendent Linda Lacewell immediately launched an investigation. She posted on Medium: "This is not just about investigating one algorithm. Consumers across the country should have confidence that algorithms affecting access to financial services treat all individuals equally and fairly."

The investigation continued through 2021. The result was unexpected. "No evidence of intentional gender discrimination was found." Goldman Sachs's algorithm did not include gender as a variable at all. Under U.S. law, that would be illegal. The differences in credit limits appeared to stem from other variables: how income was shared, debt histories, and similar factors.

Legally, Apple and Goldman Sachs were cleared. But the case left deeper questions behind.

The first problem was inexplicability. When Hansson asked "why?," no one could answer. Not the customer service representative. Not Goldman Sachs's credit officers. Not even the engineers who designed the algorithm. The AI made a decision, but it couldn't explain why it made that decision. It was a "black box."

The second problem was the possibility of proxy discrimination. Even without directly including gender as a variable, variables like shopping patterns or spending habits could have served as proxies for gender. Women's shopping patterns differ from men's. The algorithm may have observed those differences and inferred gender. It was never legally proven, but the suspicion remained.

The AI Now Institute analyzed this case and wrote: "In the Apple Card controversy, bias is not a bug but a feature." Algorithmic discrimination is not an accidental error; it is a structural problem embedded in data and design.

The lesson from this case is clear. If AI is deployed in financial services, it must be able to explain why it made its decisions. "The algorithm did it" doesn't hold up in court, in front of customers, or before regulators.

The partnership between Apple and Goldman Sachs began its slow collapse after this incident. It started with the gender discrimination controversy, continued with dispute resolution failures, and culminated in the $89 million fine. The alliance once called "the marriage of the century" ended in divorce proceedings.

C. Pricing Algorithms and Antitrust Law

(1) The RealPage Case: Rent Algorithm Collusion

A tenant living in a Seattle apartment received a 2022 lease renewal notice. The rent had gone up 30%. Furious, the tenant checked rates at the building next door, and the one beyond that. Every apartment's rent had risen by the same amount. Vacancies were everywhere, yet prices weren't falling.

"This is collusion."

It was. Just not the kind of collusion the tenant imagined. There was no scene of landlords gathering in a secret location to agree, "Let's raise prices by this much."

They had never even met each other. Instead, they were all using the same software. An algorithm called YieldStar, made by a company called RealPage.

YieldStar worked like this. Landlords send their sensitive data, including actual contract rents, vacancy rates, and lease terms, to RealPage's servers. RealPage's AI runs an integrated analysis of this data. Then it presents each landlord with the "optimal rent." "Charge this price."

Why is this a problem? In a market economy, competitors must set prices independently. If Apartment A raises its price, Apartment B can lower its price to steal tenants. That is competition. It's good for tenants. Prices go down.

But when A and B use the same algorithm, something different happens. The algorithm tells both of them to "raise prices." A raises. B raises. Tenants have nowhere to run. The entire market's prices have gone up.

There's a subtler point. In the past, landlords feared vacancies. Vacancies are losses. So they would lower prices to find tenants. But RealPage offered a new calculation. "Some vacancies are fine. If you keep prices high, your total revenue will be higher." It was a strategy of tolerating vacancies to raise prices.

In August 2024, the U.S. Department of Justice (DOJ) and eight state attorneys general filed an antitrust lawsuit against RealPage. The charges were violations of Section 1 (conspiracy to restrain trade) and Section 2 (monopolization) of the Sherman Act.

The DOJ's logic worked like this. RealPage is the "hub," and the landlords are the "spokes." Think of a bicycle wheel. The spokes aren't directly connected to each other. But they're all connected to the central hub. When the hub turns, the spokes turn with it. When RealPage, the hub, coordinates prices, the landlords' prices move together. They never made a single phone call to each other. But the effect is the same as if they had colluded.

On November 24, 2025, the DOJ and RealPage reached a settlement. The terms were detailed.

RealPage cannot use competitors' non-public data in real-time pricing.

Only data at least 12 months old can be used to train AI models.

Geographic analysis cannot be more granular than the state level.

This is to prevent micro-level collusion at the individual apartment complex level. The "auto-accept" feature, which automatically follows the price suggested by the algorithm, must be removed.

The 'Governor' function must be made symmetrical.

The settings that were generous with price increases but stingy with price decreases must be changed.

A court-appointed monitor will oversee compliance for three years.

The consent decree lasts seven years. However, it can be terminated early if the DOJ determines after four years that it is no longer necessary. There was no monetary fine. There was no admission of wrongdoing. RealPage maintained that it "never violated the law." It said it agreed to the settlement only "to avoid prolonged litigation."

Abigail Slater, the DOJ's antitrust chief, said this: "Competing companies must make independent pricing decisions. As algorithms and artificial intelligence technology advance, we will continue to stand at the forefront of vigorous antitrust enforcement."

The settlement did not end everything. Ten states (California, Colorado, Connecticut, Illinois, Massachusetts, Minnesota, North Carolina, Oregon, Tennessee, and Washington) did not sign the agreement. They are continuing separate lawsuits. Several private class action suits are also proceeding.

New York and California enacted separate laws prohibiting algorithmic rent-fixing. The New York law took effect on December 15, 2025. RealPage filed suit in the Southern District of New York federal court, arguing that the law violates the First Amendment (freedom of speech). The fight continues.

(2) Yardi Systems Lawsuit: Sherman Act Violation

RealPage was not the only company in trouble. Its competitor Yardi Systems also became entangled in a similar lawsuit. Yardi's software 'RENTmaximizer' (now rebranded as Revenue IQ) operated the same way as RealPage. It collected data from landlords, and the AI suggested prices.

In 2023, a class action was filed in the Western District of Washington federal court. The plaintiffs alleged a violation of Section 1 of the Sherman Act and sought treble damages and injunctive relief.

The defendants (Yardi and the landlords) moved to dismiss the case. "We never colluded. We just used software." In December 2024, the court denied the motion to dismiss. The case moved to the discovery phase.

The court's reasoning in this decision matters. The court held: "If competitors delegate pricing decisions to a common algorithm and participate knowing that the algorithm uses competitors' non-public data, this can constitute evidence of collusion even without an explicit agreement."

"Participated knowing." That phrase is the key. The landlords never called each other. They never exchanged emails. They never agreed to "raise prices." But they knew. They knew that using the same software would cause prices to move in tandem. That they could raise prices without competition. And they chose that software.

Under the court's reasoning, this constitutes an "implicit agreement." Even without saying it directly, they agreed through their actions.

The plaintiffs argued this falls under the Sherman Act's 'per se illegality' standard. Per se illegality means a practice is so clearly harmful that it is deemed illegal without needing to examine its specific effects on the market. Price-fixing is the classic per se illegal act. The court has not yet made a final ruling on per se illegality. But the fact that the case advanced to discovery is itself a bad sign for the defendants. During discovery, internal emails, meeting minutes, and financial data are disclosed. Unfavorable evidence may emerge.

The significance of the Yardi lawsuit extends beyond the RealPage case. Hotel pricing algorithms. Airline pricing algorithms. Ride-sharing pricing algorithms. Every AI wrapped in the label of 'dynamic pricing' could face scrutiny under the same logic. If an algorithm that shares data among competitors coordinates prices, that is collusion.

The DOJ sent a clear message. You cannot escape liability by saying "the algorithm set the price."

D. Australia's Robodebt Scandal

(1) Automated Welfare Debt Recovery

Cass lived in a small apartment in Melbourne, Australia. One day in 2016, she found a letter in her mailbox. It was from Centrelink, the government agency. Her hands trembled as she read it. It demanded she repay 3,000 Australian dollars (roughly 2,600,000 Korean won). The letter claimed that welfare payments she received five years earlier, when she was unemployed, had been issued in error.

Cass was confused. Five years ago? What does this mean? She tried to search her memory. She had thrown away pay slips from five years ago long before. The government was firm. "If you don't repay, you'll be blacklisted. Your tax refund will be garnished."

Cass was not alone. Between 2016 and 2019, approximately 440,000 people across Australia received similar letters. Behind this massive debt collection drive was an automated system called 'Robodebt.'

The system's logic was simple. Almost embarrassingly simple to call AI. The algorithm pulled annual income data from the Australian Taxation Office (ATO). Then it divided it by 26 (the number of fortnights in a year). This became the 'average fortnightly income.' It then compared this figure to the fortnightly income reported to Centrelink. If there was a difference? "You concealed your income and collected welfare. Pay your debt."

The problem is that real people don't work like machines. There are university students who only work part-time during school holidays. There are seasonal workers who only work half the year. Freelancers only earn money when they have work. If you divide their annual income by 26, you get numbers that bear no resemblance to reality.

Here is an example. University student Tom earned 6,000 dollars working part-time over three months of summer break. He didn't work during the remaining nine months because he was in school. The Robodebt algorithm calculated it this way: 6,000 divided by 26 equals 231 dollars. "Tom earned 231 dollars every two weeks. But Centrelink records show zero income for nine months. He lied!" In reality, Tom did not lie. He genuinely did not work for nine months. He was entitled to receive welfare. But the algorithm could not understand this context.

The cruelest aspect of this system was the 'reversal of the burden of proof.' In the past, the government had to prove fraudulent claims before it could recover money. A case officer would review records, give the person a chance to explain, conduct an investigation, and then reach a conclusion. But under Robodebt, once the algorithm declared "you have a debt," the citizen had to prove "I don't have a debt."

You had to find pay slips from seven years ago. You had to contact your former employer for records. What if the company had closed? What if the documents were gone? That was your problem. If you couldn't prove it, the debt stood.

The Australian government projected this system would save 4.77 billion dollars. By catching welfare fraud and cutting public servant costs.

They underestimated the costs.

(2) The 1.8 Billion Dollar Settlement and Lessons Learned

Tragedy followed. Vulnerable people who could not bear the sudden debt demands took their own lives. The exact number is unknown, but the Royal Commission investigation directly linked at least two suicides to Robodebt. Some estimates claim that stress-related deaths caused by this system exceeded 2,000.

In 2019, a Victorian court ruled that Robodebt's core calculation method, 'income averaging,' was illegal. The judge was unequivocal. "Averaging is not evidence." This meant that simply dividing annual income to estimate fortnightly earnings had no legal basis.

A law firm called Gordon Legal filed a class action on behalf of the victims. In November 2020, the Australian government surrendered. A settlement of 1.2 billion Australian dollars (approximately 1 trillion Korean won) was reached. It included refunds, debt write-offs, and interest on the 746 million dollars illegally collected.

That was not the end. In July 2023, the Royal Commission released its final report. Spanning over 900 pages, the report called Robodebt "a crude and cruel mechanism." It wrote: "Robodebt was neither fair nor legal. It made many people feel like criminals. In essence, people were traumatized merely by the possibility that they might owe money."

The Royal Commission recommended criminal referrals for senior officials including Scott Morrison, who was Social Services Minister at the time (and later became Prime Minister). They knew the algorithm was illegal, yet pushed the system forward for political purposes (publicizing budget savings).

Gordon Legal found new evidence in the Royal Commission report. Evidence that could prove 'misfeasance in public office,' a legal doctrine that applies when public officials abuse their authority and cause harm to citizens. They filed a new lawsuit. In September 2025, the Australian government agreed to a second settlement. It would pay an additional 475 million Australian dollars (approximately 400 billion Korean won). Attorney-General Michelle Rowland said: "Settling this claim is the just and fair thing to do."

The total settlement exceeded 2.4 billion dollars. It was the largest class action settlement in Australian history.

The lessons Robodebt left behind are clear.

First, automation does not mean accuracy. When you reduce complex human lives to a simple formula, errors occur on a massive scale.

Second, the burden of proof matters. Demanding that citizens disprove an algorithm's conclusions violates due process. For vulnerable people with limited resources, it is an impossible demand.

Third, human oversight (human-in-the-loop) is essential. When decisions directly tied to people's livelihoods are fully automated, errors spread beyond control.

Fourth, someone must be held accountable. The excuse that "the system did it" does not hold.

The Robodebt scandal became a permanent cautionary tale for public AI adoption worldwide. When an algorithm introduced in the name of efficiency attacked the most vulnerable people, its cost amounted to half of what the government had hoped to save. But the real cost cannot be measured in money. Lost lives. Broken families. And trust in government and technology. No settlement can bring those back.