Nope nope nope nope nope.
If this happened to me I’d sue for sharing my personal information with an unofficial 3rd party.
According to the article, the tool is internal. Which means that at the very least it’s been vetted, and potentially is a self-developed, self-hosted service, in which case your data doesn’t even leave the police.
That would be OK, but the fact they’re using a tool that can hallucinate laws and rights that don’t exist is a huge problem.
Wait till you learn that the lawmakers used AI to write the laws
Ugh… fucking hell.
Fuck it. Burn it all down.
… a FOREIGN party. That’s data sovereignty and unchecked spreading of P-I.
We don’t sue much for that - thats for Americans - but it should be a career-limiting decision.
The article calls it an internal tool, meaning it’s probably sanctioned by leadership. I don’t know, everything after the first paragraph is behind a paywall.
Approval by leadership of a tool is not necessarily reflective of the governance requirements of the organization. Data classification exists but in VERY few orgs is it respected and followed.
“You can use ChatGPT to write Facebook posts” is sanctioning by leadership.
“You can use ChatGPT” is what people often hear.
15 years ago it was “my manager got us Dropbox”.
Correct. This is actually against the Privacy Act (PII distributed or stored outside of the borders by an entity holding Australian citizen data), so no doubt the AFP will be looking into this one.
The city of Victoria, British Columbia, Canada, not the state of Victoria, Australia.
💯💯💯
It sounds like they are running self-hosted Ollama with custom RAG.
If I believe Vic PD description of what they’ve made available, its probably one of the best possible situations. Someone who knows LLMs and understand what they are has created a relatively thoughtful implementation, its not just because some braindead executive wants to get in bed with Google.
It still has a lot of the problems and should be scrutinized heavily, but at least they aren’t piping their data into Grok or anyone else for that matter.
I think there is still a significant issue with accountability and accuracy.
Are they more or less accurate than a trained human legal expert? Maybe equivalent, but they’re non-deterministic so will frequently have different answers to the same question asked at different times. They will typically bull ahead with incorrect information rather than output that they’re unsure and need to investigate which you would expect a human expert to say.
LLMs also have no accountability so all decisions must be verified by an accountable expert. Given the time it takes to read and absorb a potentially wrong advice, and given you need to ask the legal expert anyway, you might as well ask the expert in the first instance.
yeah, well, this is what I mean by the inherent problems. The open models still hallucinate, they are still probability-driven. Even when guided by RAG and other precautions, it still is only as good as those supporting elements can offer (assuming they are high-quality themselves).
On balance, I do not understand how this kind of tool will help officers in the field unless it is to advise on procedures for the officer to follow. Not trying to interpret situations as offenses or not, not trying to be a pocket legal interpreter.
I only suggest that whomever decided to try this for VicPD is being supported by a LLM geek that probably at least cares how well the models are performing. They seem to understand what kind of safeguards should exist for this type of thing to exist in this space… I personally doubt its good enough, but I rather see this than ChatGPT.
You’re wrong on the first count primarily because you presume RAG retrieval isn’t deterministic. Which it is, even if you format the question differently - as long as the request is coherent and covers the same topic, RAG lookup should be roughly the same, and that should result in the same RAG entries surfaced.
At which point it’s the prompt (the hidden/system prompt, not the end user’s basic question) that determines just how well that RAG data gets displayed and potentially reworded. But as long as it’s essentially used as a human language lookup system + summarisation, its reliability should be generally pretty good, on par with having an on-site paralegal.
