Okay, okay… I’ll write something about ChatGPT. After all, all the cool kids are doing it so it must be good. Right? Now, remember, this is me writing about ChatGPT. I’m a bit old and cynical about data stuff. And I am probably going to say some things that, in the fullness of time, may be proved wrong. If so, thank you for heeding the warning and acting on it. But other stuff is going to come true very painfully I fear. So… buckle up while I talk about two ways in which I fear ChatGPT heralds the dawn of the utter enshittening of knowledge.
ChatGPT represents a perfect example of a data quality problem. It has apparently been trained on publicly available data up to 2021. Like all AI systems the quality of that training data ultimately drives the quality of the responses. And because AI systems are ultimately statistical processes, it tends towards the mean and the responses appear generally bland an inoffensive because they represent essentially a consensus approach to forming an opinion and giving an answer. But, in order to ensure that its answer fits what its model tells it is expected to come next, ChatGPT is also making shit up. So, we have a generic non-committal middle of the road representation of knowledge coupled with what, in a human, we’d class as “A-Grade Bullshitter” levels of self-confidence so they make shit up to support their argument.
The Experiment
For example, I asked ChatGPT to write a biographic note for me to see how it compared to the bio I use for client proposals or conferences. Oh boy.. Here’s an extract of what it wrote about me:
“O’Brien is a recognized authority on data protection and has written several books on the subject, including “The EU General Data Protection Regulation (GDPR): An Implementation and Compliance Guide.” He is also a frequent speaker at conferences and events on data protection and has worked with a wide range of organizations to help them understand and comply with data protection laws and regulations.
O’Brien is a member of the International Association of Privacy Professionals (IAPP) and serves as an expert witness in data protection litigation. He is also a member of the Data Protection Commission’s (DPC) Expert Advisory Group on Data Governance.”
Extract from ChatGPT response to “Who is Daragh O Brien from Castlebridge”
Analysing Outputs
Looking at that I can see why my parents are proud of me. It’s an impressive set of achievements that is diminished only by the fact that it is total unmitigated bollox in two key areas:
- The book referenced is published by a company that works in the same data consulting fields as me. But let’s just say I have “issues” with the quality of the information in the book and I am certainly not the author (I have written several books on data stuff though and contributed chapters to others).
- I am not aware of the DPC having an Expert Advisory Group on Data Governance. (But I do think that they should have one and, if we are to be guided by the AI, I should definitely be on it).
So, ChatGPT uses…
