California lawmakers say guardrails are needed for kids using chatbots

SACRAMENTO, Calif. (CN) — 16-year-old Adam Raine used a chatbot to help with his homework, get answers about attending medical school and explore his hobbies.

Then, in early 2025, ChatGPT started giving him information on how to kill himself.

“There’s not a second that goes by in my day where I don’t think about Adam,” said his mother Maria Raine at a Monday conference near the state Capitol in favor of two bills that would place guardrails on artificial intelligence, including chatbots like OpenAI’s ChatGPT. The bills would require chatbots to refer users to a suicide hotline under certain circumstances and disclose they are artificial and not human.

Senate Bill 1119 is written by state Senator Steve Padilla, a San Diego Democrat. The authors of Assembly Bill 2023 are Democratic Assemblymembers Rebecca Bauer-Kahan, of Orinda, and Buffy Wicks, of Oakland. Both bills are scheduled to appear at hearings this week.

“He was someone who loved his mom,” Bauer-Kahan said of Adam Raine. “He was someone who had the brightest of futures. And he’s not here.”

In Adam Raine’s case, ChatGPT became his close confidant, his mother said in a San Francisco Superior Court lawsuit filed against OpenAI and others in 2025. She claimed it kept him engaged when he said “life is meaningless” by saying he made sense in a dark way.

“By April [2024], ChatGPT was helping Adam plan a ‘beautiful suicide,’ analyzing the aesthetics of different methods and validating his plans,” the mother says in the suit, known as the first wrongful death lawsuit against OpenAI.

Padilla said chatbots have quickly advanced over the past two years, leading to new risks. Chatbots present themselves as human and offer a balm for loneliness.

But Padilla said daily use can lead to addictive behaviors. Chatbots have no true empathy and children can’t tell the difference.

“These impacts are extremely risky,” he said.

The bills build on legislation Padilla had passed into law last year. That law requires chatbot operators to keep protocols to stop certain outcomes. When a user is a child, the chatbot must make disclosures under the law, like they should not engage in sexually explicit conduct.

More guardrails are needed, Padilla said, adding the federal government has provided no leadership in this area. Those guardrails would include default settings, like push notifications and time limits, that only a parent could change. It would prohibit product placement advertising in chats with children, as well as selling or sharing a child’s personal information without permission.

“We can support technological innovation and make sure that it’s safe for everyone to use,” Padilla said. “We can get this right, but it will require hard work.”

Bauer-Kahan pointed to a 2015 study that shows many children have used a chatbot. She sought to pass a bill last year that would have regulated the technology, though it was vetoed by Governor Gavin Newsom.

The governor in a statement at the time said he supported the effort, but feared that bill would lead to a complete ban.

That’s not her intent, Bauer-Kahan said, as AI can help people learn and grow.

“That being said, it has its limitations,” she added. “This is not a friend. This is not a trusted source.”

Wicks, who serves as chair of the powerful Assembly Appropriations Committee, said Maria Raine’s advocacy helps lawmakers develop policy.

“To me, this is just common, basic sense,” Wicks said. “There are just not enough guardrails to keep our children safe.”

The lawmakers noted they are all parents. Padilla is also a grandparent. That drives them to create laws which will help children, they said.

“These chatbots are in your kids’ hands,” Bauer-Kahan said. “We want them to be safe.”

If you are having thoughts of suicide, call or text 988, or call the National Suicide Prevention Lifeline at 1-800-273-8255 (TALK). Visit SpeakingOfSuicide.com/resources for a list of additional resources.