A shooting in Tumbler Ridge, Canada, has sparked a critical discussion about the responsibility of AI developers to report threats of violence, versus their duty to protect user confidentiality. Before the attack, in which several people were killed, including students and family members, the perpetrator is said to have used OpenAI’s ChatGPT to obtain tactical advice on the effectiveness of shotguns in confined spaces.
This incident has led to a civil lawsuit against OpenAI and its CEO, Sam Altman, with the victims’ families and the government of British Columbia claiming that the company failed to act despite being aware of the dangerous conversations.
The central conflict lies in the difficulty of distinguishing between a real threat and an innocent interaction. Legal experts, such as civil rights attorney Jenin Younes, argue that making the monitoring of private chats mandatory could infringe on fundamental constitutional rights and privacy.
Conversely, there is growing concern that AI provides a degree of detailed information that could make it easier for individuals to plan large-scale attacks.
The technical challenge of assessing intent is significant. Fazl Barez, an AI safety researcher at the University of Oxford, notes that current technology cannot reliably distinguish between a creative writer working out a fictional murder, a scientist researching dangerous pathogens for safety reasons, and someone planning a real crime.
He argues that a credible threat can only be identified by analysing behavioural patterns across multiple sessions, rather than focusing on individual statements.
Comparisons have been drawn between AI companies and mental-health professionals, who are legally required to warn potential victims of credible threats.
However, Professor Chinmayi Sharma of Fordham University has pointed to a striking difference: therapists are licensed professionals trained to assess threats and bound by strict legal standards, whereas AI companies lack both the expertise and the professional accountability.
Further risks concern the possibility of systemic bias. Prof Sharma warneds that AI monitoring tools could disproportionately flag the expressions of minority groups because of biases embedded in the training data, leading to unfair targeting.
In addition, the prospect of continuous monitoring could discourage people from using chatbots for mental-health support or personal reflection, out of fear that their innermost thoughts might be passed on to the authorities.
From a legal standpoint, the duty to warn has not yet been clearly extended to AI providers.
Prof Bryan Choi of the University of Colorado Boulder explained that, under current law, a chatbot does not have the same “special relationship” as a therapist with a patient, which makes it difficult to hold companies like OpenAI legally liable for failing to alert the police.
To resolve these tensions, some are proposing the creation of independent oversight bodies to establish reporting protocols. Such organisations could assess threats and manage data sharing without compromising user anonymity.
Supporters argue that users should at the very least be explicitly informed if their data are being monitored and under which specific conditions these might be shared with law-enforcement agencies. As AI continues to evolve, the balance between public safety and personal freedom remains an unresolved legal and ethical challenge.
Business AM
Ltd
