Introduction
No highlights.
What is Generative AI?
They are the proverbial parrot who echoes without thinking, not the actual parrot who is capable of complex reasoning and problem-solving. A zombie parrot, if you will, that screams for brains because it has none.
Story telling is often metaphorical, and the idea of a zombie hoard may not need to be literal and with a virus in order to become a real-world problem for humanity.
Every other time we read text, we are engaging with the product of another mind. We are so used to the idea of text as a representation of another person’s thoughts that we have come to mistake their writing for their thoughts. But they aren’t. Text and media are tools that authors and artists create to let people change their own state of mind—hopefully in specific ways to form the image or effect the author was after.
I’ve fallen for this and have never considered text as the tool or product, but have confused it for a symbolic (or actual) representation.
The idea that there is intelligence somehow inherent in writing is an illusion. The intelligence is all yours, all the time: thoughts you make yourself in order to make sense of another person’s words. This can prompt us to greatness, broaden our minds, inspire new thoughts, and introduce us to new concepts. A book can contain worlds, but we’re the ones that bring them into being as we read.
Reading is active!
You are doing the textual equivalent of seeing a face in a power outlet.
Large Language Models are something lesser. They are water running down pathways etched into the ground over centuries by the rivers of human culture. Their originality comes entirely from random combinations of historical thought. They do not know the ‘meaning’ of anything—they only know the records humans find meaningful enough to store.19 Their unreliability comes from their unpredictable behaviour in novel circumstances. When there is no riverbed to follow, they drown the surrounding landscape.
General reasoning seems to be an inherent, not emergent, property of pretty much any biological lifeform with a notable nervous system.
Should we be treating life as more sacred, or with more respect, than we currently do?
A reasoning mind appears to be a direct consequence of how animals are structured as a whole—chemicals, hormones, and physical body included.
A reasoning mind is the starting point of biological thinking, not the endpoint that only “emerges” with sufficient complexity.
After everything the tech industry has done over the past decade, the financial bubbles, the gig economy, legless virtual reality avatars, crypto, the endless software failures—just think about it—do you think we should believe them when they make grand, unsubstantiated claims about miraculous discoveries? Have they earned our trust? Have they shown that their word is worth more than that of independent scientists?
Theranos!
Strategies for using Generative AI
No highlights.
Be aware of the abundance of bullshit and snake oil
We have a need for human contact and our tendency towards anthropomorphising objects in our environment increase with our isolation.
Can we get access to the data GPT-4 was trained on, or at least some documentation about what it contains, so we can do our own analysis? No.10
So they can steal untold volumes of content for training, then want what they created to be protected?
There’s even reason to be sceptical of much of the criticism of AI coming out from the AI industry. Much of it consists of hand-wringing that their product might be too good to be safe—akin to a manufacturer promoting a car as so powerful it might not be safe on the streets. Many of the AI ‘doomsday’ style of critics are performing what others in the field have been calling “criti-hype”.
Only use specialised Generative AI tools that you’ve vetted
If you want to minimise your risk while still maximising your benefit, use specialised AI tools you can test.
If plagiarism is an unacceptable risk for the use case, don’t use a tool that has a risk of overfitting or memorisation. If unsafe output— bigotry, racism, pornography—would create a crisis, don’t approve a system with insufficient safeguards. The safest tools are going to be those that specialise in modification or conversion, such as automatic captioning or transcription software.
Make sure that the plagiarism and hallucination rates are within the bounds of what’s reasonable for your use case.
This is often skipped, especially when rolling out support chatbots, and I doubt is monitored for effectiveness after being rolled out. You can take a lot of risks, but you should not risk your reputation.
If your company gets sold, would there be an intellectual property audit of the code base?
Strengthen your defenses against fraud and abuse
We aren’t about to see eloquent and charming spam emails in our inboxes—the ineptness and flaws of most spam is an intentional tactic to weed out all but the most gullible.
Fake profiles that are using more advanced diffusion models to generate their images and large-language-models for the text, will be much more difficult to discover.
Perhaps the question isn’t about creating a a better mousetrap (Captcha) but rather fostering more IRL connections that get reflected online. Augment reality, don’t replace reality.
OpenAI’s own classifier, which should be the current state-of-the-art given that they are the leading company in Generative AI, only correctly identifies AI generated text 26% of the time and falsely identifies human-authored text as authored by an AI about 9% of cases.12 This means that if you have 100 applicants or students, 10% of whom attempt to cheat using ChatGPT text in a test, the current state-of-the-art AI content checker will only correctly identify three out of ten cheaters, but will falsely accuse eight students of cheating. That’s with a tool made by the leading company in the field, one that has full access to the technology and models that ChatGPT is built on.
Prefer internal tools over externally-facing chatbots
No highlights.
Prefer transparent and open generative AI
No highlights.
Use it primarily for derivation, conversion, and modification
No highlights.
Are they breaking laws or regulations?
In the meantime, as a rule of thumb, the safest and, probably, the most productive way to use generative AI is to not use it as generative AI. Instead, use it to explain, convert, or modify.
It isn’t uncommon for people who work in tech to see laws and regulations as a hindrance to be bypassed. Normally, this would just be a problem between them, their lawyer, and the inevitable prosecutor that gets their case, but it can become an issue when that world-view affects the products they make.
Those ramifications can be substantial. You risk sharing liability with a misbehaving AI vendor. Even a minor violation of the GDPR, for example, can result in fines of up to “€10 million, or 2% of the firm’s worldwide annual revenue from the preceding financial year,
Copyright protections are diminished
Machine-generated works are not protected by copyright in the US and unlikely to be protected in the EU. This means that you have no recourse if somebody decides to reuse that work in harmful contexts. An AI-generated app icon, for example, can be reused freely by scam artists. Attempting to present AI-generated output as your own is fraud and can expose you to litigation.
Given that purely AI-generated work is considered to be authored by the machine, anybody who participates in contests or submits that work as their own is technically committing fraud. Even if the works were submitted with correct attribution, the publisher or the context wouldn’t be under any obligation to pay the submitter: the work has no copyright protection.
Much of the training data is biased, harmful, or unsafe
It would be quite simple for an employee to create test repos with rogue training data that wouldnt be discovered until well after an attack or the employee leaves. This means even if you only train on your company’s private repos (not public repos) you are still vulnerable.
Stochastic plagiarism
No highlights.
Hallucinations, or AI don’t do facts
…the long tail of falsehoods is infinite and truth is scarce. You will never run out of falsehoods to correct or lies to debunk.
Microsoft and Google both envision a future where you’re using their AI systems to summarise email threads and complex documents.
Just. Stop. Writing. Long. Rambling. Emails. FFS people!
Much of the output is biased, harmful, or unsafe
Generative AI makes text and art that is the most probable response to a given prompt. This is mediocrity.
Code Quality
People have a tendency to trust what the computer says over their own judgement. Or, to be more specific, when presented with a cognitive aid intended to help them think less, that’s exactly what they do: they think less.
Shortcut reasoning
No highlights.
Fear of Missing Out
First-mover advantage in software is largely a myth. Historically, second-movers win. Waiting to learn from the first-mover’s mistakes has usually been the smart move.
The Elegiac Hindsight of Intelligent Machines
Safe, for the tech industry, is too slow.
There is no way to ‘science’ a case study unless you have access to a parallel universe as a control. They’re all just stories that short-circuit our thinking.
In a war of theatrics, the act with the biggest budget wins the crowd.
When we need software that works on the devices we have, for as long as they last, they give us software that only works on the latest and greatest. Sometimes, as with GPT-4, the software they make even requires systems so powerful that they only exist in a couple of locations on the planet. But, don’t worry, they’ll sell us access—timeshare, really—but lets call it “the cloud”. It only breaks some of the time. Nothing in what they do is for us, even though it’s our money, our data, and our art, writing, and music they’re demanding. We aren’t customers to them—we’re just the people that pay. To tech companies, we are nothing more than a resource to be tapped. A number to be boosted to pump investor interest. They are not doing us any favours. What they want from us is simple: everything. All culture on their servers, made by their AI. All our work happening through them, assisted by their AI. The totality of our information, mediated by their AI. A vig collected on all existence. One of the papers I’ve referred to a few times in this book is On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?27 It was the first paper to provide a cohesive and detailed overview of how large language models work, how they affect
Where we need robust technology, they are giving us finicky AIs that misbehave at a badly worded sentence. Where we need privacy from both corporations and potentially hostile authorities, they push further and further into recording our lives. When we need software that works on the devices we have, for as long as they last, they give us software that only works on the latest and greatest. Sometimes, as with GPT-4, the software they make even requires systems so powerful that they only exist in a couple of locations on the planet.
This is what I like about the LightPhone values. Specifically, build something that will last for years, continue to support it, and be transarent about what that actually costs.
We aren’t customers to them—we’re just the people that pay. To tech companies, we are nothing more than a resource to be tapped. A number to be boosted to pump investor interest. They are not doing us any favours. What they want from us is simple: everything. All culture on their servers, made by their AI. All our work happening through them, assisted by their AI. The totality of our information, mediated by their AI. A vig collected on all existence.
While it may sound alarmist, this is indeed the logical result of prioritizing profits. Thanks, capitalism.
Safe, for the tech industry, is too slow when you hunger for a bubble and want to ship more software, to more people, as fast as you can.
Afterword: a straw to grasp
I hesitate to claim superior judgement on my part—much of it is down to ChatGPT’s ability to sound like a particular kind of person who I’ve learned from bitter experience to avoid. Most of these language model chatbots sound like that guy. You know the guy. Everybody’s worked with one.
Further reading
No highlights.
References
No highlights.
Leave a Reply