California is setting a precedent for AI use in scenarios where workers’ jobs are at stake.
Governor Gavin Newsom this week signed the No Robo Bosses Act (SB 947), which bans employers from relying solely on AI-powered automated decision-making systems (ADS), also known as “bossware,” to discipline or terminate workers.
Introduced by Senator Jerry McNerney, the act is the first of its kind in the US. It is set to take effect on July 1, 2027.
While applauded by many, on a wider scale, No Robo Bosses raises questions about what adequate human oversight looks like in this age of AI.
“A machine can recommend, but a human must decide,” noted Frank Dickson of Dickson Research. “The hard part is defining what ‘decide’ means.”
Combating AI bias and ‘troubling’ errors
ADS platforms are becoming ever more prevalent in hiring, as they promise to speed up and streamline HR processes, maximize productivity, and reduce costs. According to reports, more than half of employers use ADS to help make decisions around restructuring and role planning.
But, McNerney and others contended, AI is prone to bias and misjudgment, and there are examples of ADS making “troubling” errors, including mistaken firings. Meanwhile, on-device “bossware” or “tattleware” employee monitoring and productivity tracking software could potentially be used in reprimanding or firing decisions.
AI recruitment tools have similarly come under scrutiny of late, with job seekers alleging that platforms like Workday discriminate based on age, race, and disability.
To combat these issues, the bill, SB 947, seeks to set guardrails around AI use in the workplace. The legislation mandates human oversight and verification when enterprises apply ADS tools, and employers must inform workers if they have used them when making decisions about their jobs.
Further, SB 947 prohibits the use of ADS and bossware systems that analyze personal information and behaviors to essentially predict how employees will perform or act in the future. And at a worker’s request, employers must provide at least one year of the data used by an ADS system to make a disciplinary, termination, or deactivation decision.
Enterprises found violating any of these provisions are subject to a $500 civil penalty per violation.
“Employers are devastating workers’ livelihoods and taking no responsibility for the callous decisions of this unchecked technology,” said Lorena Gonzalez, president of the California Federation of Labor Unions, AFL-CIO. “This is unacceptable. We need stronger guardrails to make sure there is human review and oversight of any decision made by a machine that impacts a worker’s job and paycheck.”
Promoting decisions based on context, evidence
Dickson pointed out that the law is narrower than its name implies: ‘No Robo Bosses’ sounds like a ban, but it is not. “The law does not keep AI out of the decision. It keeps a human in it,” he said.
According to the legislation, a human must corroborate the system’s output using the data behind it and other supporting information like performance evaluations, personnel files, work product, peer reviews, or witness interviews. If the reviewer finds the output inaccurate, incomplete, or misleading, the employer cannot use it, Dickson said.
“The test is whether the human checked the evidence and had the power to say no [to the AI’s decision],” he said, observing that a manager simply forwarding the bot’s verdict is a mail carrier, not a decision maker. In a sense, it’s like an instant replay in sporting events: The umpire watches the video and makes a judgment call, rather than just rereading the call from the field.
“What matters is that they examined the evidence, not just the conclusion,” Dickson said. An employee’s disciplinary or termination notice must disclose whether an AI system was used, and confirm that a human reviewed and corroborated the decision. Human accountability is the point here; an employer owns the final decision.
“Accountability can’t be shifted to the AI,” said Valence Howden, advisory fellow at Info-Tech Research Group. How it comes to its conclusions must be continuously assessed for bias, error, and drift.
“The bot has no human compass for rationale, and context is defined by who controls its model,” Howden said. “Without context, the human has no impact on the end result, and, given the rate of failure here, there are many (legal and other) implications.”
What enterprises can take away from California’s example
The grey area in the bill is the phrase “primarily relies,” Dickson pointed out; it does not specifically define what that means. “A prudent employer may assume that if the system’s output started the conversation, the decision is in scope, and document accordingly,” he said.
Enterprises should consider five factors when developing a plan around discipline and termination, he said:
- Inventory: Know which tools score, rank, or flag employees. McNerney’s office cites hundreds of “bossware” products on the market, and many enterprises may not know which ones their managers use.
- Reviewers: Name who corroborates decisions, train them, and give them real authority to overrule the system.
- Data access: A reviewer cannot corroborate what they cannot see. If a vendor cannot show the inputs behind an output, the tool may be unusable for these decisions. That condition belongs in the contract.
- Records: Keep the evidence that the reviewer examined, as employees have the right to request the personal data used.
- Notice: Build a plain-language, standalone notice template now.
Dickson said that this is an IT problem as much as an HR one.
“Corroboration requires that the inputs to every automated decision are logged, retained, and retrievable by someone outside IT,” he said. “That is an architecture decision, and the CIO makes it.” And, while the $500 penalty might sound small, larger exposure may come later during wrongful termination disputes in which “the algorithm said so” proves a weak defense.
Info-Tech’s Howden advised enterprises to identify their high-risk, high-human-impact scenarios and cases, clarify which decisions, activities, and actions AI can take, and tag those that require human validation and involvement. They should also look at the volume and scale of these types of action so that volume alone doesn’t become a challenge to human involvement.
This is AI governance at a foundational level, Howden noted. “AI shouldn’t be allowed to make irreversible or material decisions and changes without oversight,” he said.
Ultimately, this move may drive enterprises to actually understand the criteria for hiring and firing, and look for issues and biases in the way decisions are made before an AI exposes poor decision logic and removes critical people. “Legal implications are more likely to shape the direction,” Howden said, “especially with the disclosure requirement.”