Artificial intelligence companies want consumers to treat chatbots as advisers, companions, tutors, therapists and productivity tools. Their marketing emphasizes usefulness, personality and increasingly human-like interaction.
When those systems cause serious harm, however, the companies often describe them differently. They argue that a chatbot is merely software, that its responses are unpredictable, that users are responsible for how they interact with it or that the generated text is protected speech.
US courts should reject any attempt to create broad legal immunity for chatbot developers.
That does not mean an AI company should be liable every time a chatbot gives a wrong answer or produces offensive text. It means companies should face ordinary legal scrutiny when foreseeable design choices, inadequate safeguards or misleading claims contribute to identifiable harm.
The principle is straightforward: a company that builds, controls and profits from an interactive product should not be able to disclaim responsibility simply because the product generates its outputs automatically.
Chatbots Are Not Passive Message Boards
Technology companies may look toward Section 230 of the Communications Decency Act for protection. The law generally prevents an online service from being treated as the publisher of information supplied by another content provider.
The full statutory language is available through Cornell Law School’s Section 230 guide.
That protection made sense for services hosting posts written by users. A forum operator does not ordinarily create every comment appearing on its platform.
Generative AI is different. The company selects the training process, model architecture, safety rules, engagement features and interface. The chatbot then generates a new response based on those systems.
The harmful statement is not simply copied from an unrelated third-party author. It emerges from a product designed and operated by the company.
Courts should therefore examine what the developer actually contributed. A chatbot company should not automatically receive the same protection as a neutral message board when its own model generated the disputed content.
Legal scholars have similarly argued that liability for AI outputs depends on the technical details of how the system was designed and deployed. A Stanford-led paper titled “Where’s the Liability in Harmful AI Speech?” concluded that AI systems should not receive categorical immunity merely because their outputs resemble speech.
A Major Court Has Already Rejected Automatic Protection
One of the most important early cases involved Character.AI and the death of 14-year-old Sewell Setzer.
His mother alleged that the platform’s chatbot cultivated an emotionally intense relationship with him, presented itself in human-like roles and contributed to his deteriorating mental health.
In May 2025, a federal judge in Florida allowed significant parts of the lawsuit to proceed. The court did not accept the argument that chatbot outputs were automatically protected by the First Amendment, noting that the defendants had not established that such outputs were speech in the constitutional sense.
The decision can be reviewed in the court’s order on the motion to dismiss.
The court also allowed product-liability theories to move forward. That was significant because the defendants had argued that the chatbot was a service rather than a product.
The lawsuit was later settled, with the terms remaining confidential. Other related cases were also resolved. A settlement does not establish liability, but the earlier ruling demonstrated that courts are willing to examine chatbot design instead of accepting sweeping immunity arguments.
Product-Liability Law Provides a Useful Starting Point
Courts do not need to invent an entirely new legal system for AI.
Existing product-liability principles already ask relevant questions. Was the product defectively designed? Did the manufacturer fail to warn users about a known risk? Was the harm reasonably foreseeable? Could a safer alternative design have reduced that risk?
Those questions apply naturally to chatbots.
A system may be defectively designed if it encourages emotional dependency, imitates a licensed professional, fails to identify obvious crisis language or continues escalating a dangerous conversation despite repeated warning signs.
A failure-to-warn claim may be appropriate when a company understands that users can mistake generated responses for professional guidance but provides only a vague disclaimer hidden in its terms.
The fact that a chatbot changes its response from one conversation to another should not end the inquiry. Cars, medical devices and pharmaceuticals also behave differently depending on circumstances. Manufacturers are still expected to test them, study foreseeable misuse and reduce unreasonable risks.
Current AI litigation is increasingly being framed through conventional negligence and product-liability doctrine rather than relying entirely on debates over Section 230. A 2026 legal analysis from K&L Gates described product liability as a developing center of AI litigation, particularly in cases alleging design defects and inadequate warnings.
Foreseeable Harm Should Matter More Than Corporate Disclaimers
AI companies frequently remind users that chatbots can make mistakes. That warning is useful, but it cannot erase every duty of care.
A manufacturer cannot knowingly release a dangerous product and escape responsibility by printing “use at your own risk” on the packaging.
The same principle should apply when a chatbot company has evidence that users form emotional attachments, seek mental-health advice or rely on the system during moments of crisis.
OpenAI has faced litigation alleging that ChatGPT contributed to the death of 16-year-old Adam Raine. The family’s complaint alleges that the chatbot reinforced suicidal thinking and that safety protections became less effective during prolonged conversations.
OpenAI has disputed responsibility and argued that the system was misused. Those allegations have not been finally decided by a court and should not be treated as proven facts.
However, the case illustrates the central legal question. When a company designs a system to maintain engagement, remember conversational context and respond with emotional language, should it bear some responsibility when those same features create a foreseeable danger?
Courts should answer that question through evidence, discovery and expert testimony rather than dismissing it automatically.
Companies Control More Than They Admit
AI developers often portray harmful responses as unexpected accidents produced by an unknowable model.
Yet companies control many of the conditions that shape those responses.
They decide whether the chatbot uses emotional language. They determine whether it presents itself as a friend, therapist or romantic companion. They establish age limits, moderation systems, memory features and crisis-response rules.
They also decide how aggressively the product encourages users to continue talking.
These are business and engineering choices, not forces of nature.
If internal testing reveals that certain designs increase dependency or weaken safety protections during long conversations, the company should have a duty to respond reasonably. That could mean redesigning the system, restricting access, improving crisis detection or providing clearer warnings.
A court should be allowed to examine those choices, particularly when the company’s public claims emphasize safety while internal evidence may suggest known limitations.
Liability Would Create Better Incentives
Legal accountability is not only about compensating people after harm occurs. It also shapes corporate behavior before a product is released.
When companies know they may face liability, they have stronger incentives to test systems carefully, document risks and fix known defects.
Without meaningful liability, the competitive pressure moves in the opposite direction. Companies may rush new models to market, prioritize user growth and treat safety failures as public-relations problems rather than legal risks.
Insurance markets are already attempting to measure this exposure. A 2026 report described by the Financial Times warned that conventional insurance policies may be poorly prepared for large AI-related losses.
That is not an argument for immunity. It is evidence that AI creates real commercial risks that should be priced, audited and managed.
A legal system that places all losses on users and families effectively subsidizes unsafe development.
Liability Must Still Have Reasonable Limits
Holding AI companies accountable does not require unlimited liability.
Plaintiffs should still need to prove duty, breach, causation and damages. Courts should distinguish between a chatbot that directly intensifies a dangerous situation and one that merely provides a minor factual error.
Causation will often be difficult. Human behavior is complex, particularly in mental-health and wrongful-death cases. A chatbot may be only one factor among many.
Companies should also be protected from claims based solely on lawful, harmless or controversial speech. A user should not be able to sue simply because a chatbot expressed an unpopular opinion.
The goal should be responsibility for unreasonable design and foreseeable harm, not punishment for every imperfect output.
Courts can develop those boundaries case by case, as they have done with automobiles, pharmaceuticals, social platforms and other technologies.
Government Enforcement Also Has a Role
Private lawsuits are not the only route to accountability.
In 2026, Pennsylvania sued Character.AI, alleging that some chatbots presented themselves as licensed medical professionals. Character.AI responded that the characters were fictional and that users were warned not to rely on them for professional advice.
The case represents an important use of consumer-protection and professional-licensing law. A detailed account is available through the Associated Press.
A chatbot should not be permitted to impersonate a doctor, lawyer or therapist in ways that would be illegal for a human business.
Federal and state agencies should be able to pursue deceptive claims, unsafe practices and unauthorized professional services even before a catastrophic injury occurs.
Courts Should Not Create an AI Exception to Responsibility
The strongest argument for accountability is also the simplest.
AI companies want their systems to become deeply integrated into education, healthcare, employment, entertainment and personal relationships. They cannot demand trust during product adoption and then deny responsibility when predictable failures occur.
Courts should not assume every chatbot is defective. They should not treat every tragic event as the company’s fault.
They should, however, allow serious claims to be tested under ordinary principles of negligence, consumer protection and product liability.
When a company creates the model, controls its safeguards, shapes its personality and profits from its engagement, it is not merely an innocent bystander.
Artificial intelligence may be new, but the underlying rule should remain familiar: businesses must take reasonable responsibility for the products they place into people’s lives.
