high angle photo of robot

The Danger With ChatGPT Isn’t What You Think. It’s Worse.

There’s a scene in the recent Top Gun film where Tom Cruise is getting a rollicking for an audacious stunt he pulled.

The General tells him that soon they won’t need pilots anymore. Pilots who eat, sleep.. who disobey orders.

He says, “Your kind is headed for extinction.”

At which point Tom Cruise turns to the General and says, “That maybe so sir, but not today.”

Let me tell you, as a pilot that line stirs my loins in a way few women ever could. And not just because it’s Tom Cruise!

Because it’s true.

We know that one-day automation will take over the reins. One day the pilot will be made redundant.

But not today.

Not Today.

Today, the industry still needs pilots.

Despite the extraordinary technological advances in aviation, the machines we fly are far from perfect.

Most aircraft are programmed to maximise efficiency, but in doing so they fly very close to the margins.

I’ve seen the autopilot exceed those margins on several occasions.

The truth is pilot oversight and intervention is still required. We have to monitor the automation like a hawk.

This is what’s lost on many people — what many in the industry want to sweep under the carpet.

Increasingly automated machines require pilots with more experience and training, not less.


  • First, they require a greater amount of knowledge to understand. (Duh!)
  • Second, if something does go wrong, because of the sheer level of complexity, it invariably requires a particularly skillful response.

Yet, an over-reliance on automation has made the so-called “weak link in the machine” even weaker.

This is what’s known as the automation paradox.

The very advances that have significantly improved safety standards have made us worse at our jobs.

It’s perhaps the aviation industry’s most significant issue  today.

The Automation Paradox

“I say your civilization, because as soon as we started thinking for you it really became our civilization.” — Agent Smith, The Matrix.

Now, why am I telling you this?

Well, because we face the same threat as writers with the advent of ChatGPT and the rest of it.

It’s the automation paradox on steroids.

Let’s be upfront about what that means.

AI is hot on our heels in terms of being able to outwrite us. It can already do it faster and with fewer mistakes (at least, grammatically speaking).

I had Chat GPT write a children’s book the other day using an idea I’d been sitting on. I fed it the idea and said, “Sing for me, baby!”

And honestly, it wasn’t terrible.

I gave her a few more prompts — told her to write it as a rhyme book, and voila! I had something approximating what I wanted in a matter of minutes. All it required was a few tweaks and I was away.

At first, I was elated. I had dollar signs in my eyes. But then I felt this crushing wave of depression wash over me. (As you do.)

I thought, why bother taking the autopilot out? I mean, if the robot can sh*t this out in a matter of seconds, why sit down and do it manually?

Even if it can’t fly better than me yet — it won’t be long before it can, right?

Why not head over to Myjourney, get some AI-generated artwork, and then format the lot in a quick and dirty ebook? Why not put it on amazon kindle and watch as the meaningless dollars roll in?

Other people are going to. They are going to make good money doing so. At the same time, they will flood the market, making it harder for other folks who have done it the hard way.

Right there — while asking these questions — I had my answer. It’s wrong.

Uncanny Valley

There are serious ethical considerations here.

In case you didn’t realise, writers and artists are the very sources from which AI feeds. Using AI-generated content may amount to plagiarism.

That’s point one.

Point two is this: When — if — many artists/writers are forced to look for other work, what do you think Chat GPT and the rest of these AI tools will ultimately feed on?


There’s a term I came across in this article on the subject called “uncanny valley”. It that refers to the “unsettling relationship between the human-like appearance of an artificial object and the emotional response it provokes.”

It’s that gut instinct that something feels off. In literature, this effect is particularly disconcerting given AI’s inability to imitate what we might call the human soul.

This is what’s at stake. 

If we allow AI to replace the real artists of this world — who have always acted as a mirror for the human soul —we may lose sight of ourselves altogether.

The Value of the Human Soul.

Here’s the good news. The value of the human soul just went up 1000-fold.

There’s a tweet by Kent Beck that highlights this point well. Following the first time he used Chat GPT he had this to say,

“The value of 90% of my skills just dropped to $0. The leverage for the remaining 10% went up 1000x. I need to recalibrate.”

This is the million-dollar question: What is the 10%?

Well, I’ll tell you what it isn’t. It isn’t some highly structured post with perfect grammar. It’s not 6 simple ways to lose belly fat this summer.


It’s writing that bleeds onto the page from a person’s heart. It’s writing that comes straight from the gut and punches you in the face .

AI is miles away from being able to replicate this kind of writing. Why?

Because it’s an algorithm that predicts what word should come next. It’s not writing. It’s doing math. As a result, it lacks the ingredients that really make a piece of writing sing: depth, nuance, and personality.

Maybe one day it will be able to replicate this kind of thing. That maybe be so sir, but not today.

Not today.

And then, I ask, what happens when all is said and done?

I don’t know about you, but I couldn’t care less about what a robot has to tell me when it comes to the things that really matter.

I care about the person with real-life experience. About the soul brave enough to place their heart on a page. That’s what I care about. That’s who I want to hear from.

You, dear reader.


Why We Write

“I write for the same reason I breathe — because if I didn’t, I would die.” — Isaac Asimov

Does this mean we should avoid using automation? No, not at all.

As a pilot, I love having the autopilot engaged. I think, “Thank the Lord I don’t have to hand fly this aircraft for over 12 hours straight.”

I get to take it out for the best bits and let the machine do the rest.

That’s how we should look at AI — as a tool to help us do some of the heavier lifting while we get to take our pens out for the best bits.

We should embrace that. If we want to take advantage of the opportunities that AI presents we must remain open. But we need to remain extremely cognisant at the same time.

We must do our homework and understand how it works — what it’s suitable for and most certainly isn’t. We need to make sure we are using it ethically. More than that, we need to be careful not to place too much faith in it.

If there is one thing I want you to take away it’s this: The skills we automate are the skills we lose.

The great danger is that we rely on automation to think for us. The great danger is we give up on writing altogether because the bot can do it better.

Because guess what?

The real rewards from writing have nothing to do with money or ego-inflating (or deflating, in my case) stats.

It’s what goes on internally that matters.

It’s how it helps us organise our thoughts. It’s how it helps challenge and update our limiting beliefs. It’s how it helps us process our emotions and grow as human beings. 

It’s how it helps us see who we really are.

These are the benefits you get from writing that no amount of AI will ever be able to do for you.

Not today.

Not ever.

For a weekly collection of meditations to help you navigate life,  join my Stuck in the Clouds newsletter here

46 thoughts on “The Danger With ChatGPT Isn’t What You Think. It’s Worse.

    1. Haha 😂. I look for mistakes now. Lets me know I’m reading work from a human. Thanks for taking the time to read/comment David🙏

  1. I don’t know why it was invented in the first place. You can reach way past Top Gun and The Matrix to Frankenstein. Sure, it’s fiction but is it? In my opinion, it still applies.

    And why do we assume that the thing that is “helping ” us or in the case of Frankenstein, the creature that Dr. Frankenstein wanted to brag about “creating” won’t turn on us?

    1. Great point. If AI does gain consciousness it’s hard to tell what it will ultimately do. Thank you for sharing your thoughts 🙏

  2. Thought provoking post AP. There’s a lot of concern and panic about AI but we have to figure out how to use it, as you said. We need to think of it as a tool and use it as such. It’s there to help and support, but I agree others may let it do the work for them.

    1. It is a tool we must embrace. I agree Brenda. I think for the really good writers out there it’s a tool that can help elevate their writing even further. But for those who don’t have the foundations in place to lean on it heavily could do more harm than good. Thank you for sharing your thoughts Brenda. Hope you’re feeling better/on the mend?

      1. Slowly getting there. Getting frustrated because I keep forgetting my wrist is sore, use it and hurt it again. I’m going to see the doctor next week … a month in, I would have hoped it would be better than this

      2. Sorry to hear that it’s healing slower than you would liked. I remember when I injured my back training for a marathon. Instead of waiting for it to heal I ran through the pain. This caused back issues that remain to this day. Patience is key. You’ll get there!

      3. But to cheer me up, I’m having a weekend away in Edinburgh… Friday til Monday evening 😁

    1. It’s been an issue in the industry for a while. Made worse by the desire to cut costs – using advances in automation as an excuse to do – giving pilots less training when they should have been giving them more. Thanks for stopping by 🙏

  3. AI generated content is missing the essence a human mind injects into a written piece.

    A guy I follow wrote an entire blog post using AI and almost all the comments on that post saw it instantly. (It was a test, or an experiment.)Why? Because we have gotten to know his essence, the snark and humor and wordplay he uses when he tells the story. It was all missing. The AI post was clinical and sounded like a grade 10 English essay, but touched on all the subject matter and was, as you say, not terrible. It was just not enjoyable. I would not follow a blog written by AI if the content sounded the way it did in his AI-generated example.

    However, this whole thing with AI needs to be embraced because as you say it’s not going to go away. It’s going to be interesting from the writer’s perspective how all of this is going to transmit going forward.

    My website is under construction but I will touch on this topic when I go live again, so thank you for the inspiration. 😊

    1. I’ve seen a few of these experiments myself. Like you say – it’s not that the writing is bad but it lacks – for lack of a better word – soul. I agree with you though AI needs to be embraced. It’s not going away. Those who don’t may be left in the dust. I just worry that for those who don’t have the foundations in place it may do more harm than good.

      Let us know when you write this post about it – you can leave a link here. I’d be keen to read it. Thank you for sharing your thoughts 🙏

    1. No I haven’t although I’ve heard snippets of it. I’ll be sure to give it a listen during my next bike ride. Thanks for sharing 🙏

  4. Things can definitely go sideways with AI, and if recent history has anything to show us just in how humans have been reacting and creating drama, we may be in for a bumpy ride if AI is combing through everything online to learn more about how to operate better. I’ve been tempted to jump on the AI bandwagon to have easy content created for kids books to create easy money, but I prefer doing the work myself, even if I struggle to figure out what would sell well! As for writing posts and books, ditto for writing it myself. AI might one day be able to create seamless copy, but as a writer, I work out so many of my thoughts through the writing process, and having AI create for me just won’t give me the catharsism I need!

    1. That, to me, is the biggest reward that comes from writing. Nothing to do with making money or getting likes. It’s the wonderful community of like minded people and the way it helps to organise ones thoughts and process ones emotions. I realised that using AI to write a children’s book would stripe the whole project of its meaning for me. Thank you Tamara 🙏

  5. Really insightful! We should learn how to effectively work with AI since it is here to stay.

    1. Indeed we should. But we need to make sure we have the foundations in place first. AI has the capacity to make us better or worse depending on our current skillset. If we lean on AI before we’ve learnt the ropes it won’t do us any good. Thank you Heidi-Marie. Appreciate your comment. 🙏

  6. I remember hearing years ago that there were two things safe from automation: creativity and relationship-building. With ChatGPT and AI, it sometimes feels like creativity is being automated but, as you point out, it isn’t. It’s an algorithm—plain and simple. It is not creative at all. And it also isn’t always accurate.

    You raise a good point about plagiarism. Should writers be required to disclose they used ChatGPT or other AI tools to write, or even assist, with their material? It feels like they should so we, as readers, can decide if we want to read something written by an algorithm or a living, thinking, feeling, breathing human being.

    1. Exactly Michelle. If I bought a book only to later find out it was written by an algorithm I’d be furious. I want to know that information upfront. Great point. Thank you 🙏

  7. Thanks for this AP2- an excellent, thought-provoking, and (ultimately) inspirational post. I’m bookmarking this one to read again in the future- I think I’ll need the reminder.

    1. Thanks Todd – it’s easy to get downbeat as a writer/creator at the moment. I think we need to support one another as much as we can going forward. It’s a brave new world we’re entering into. 🙏

  8. Very interesting post David! I read an article recently about AI that reassured me somehow. The author asked ChatGPT “What jobs will AI be unable to replace?”. ChatGPT replied that jobs that require human judgement, decision-making and that require complex and nuanced communication won’t be replaced. Moreover, AI won’t replace jobs requiring social and emotional intelligence. creativity and innovation, and physical dexterity and mobility. I think that to some extent we may end up with better jobs. You know that especially in administration we use to deal with very boring tasks that are more and more taken over by IT systems. So now we can focus on more meaningful tasks, such as writing reports. What governments would need to do very soon is establish strict rules on ethics.

    1. I think there is a lot to be positive about. It’s not all doom and gloom. There are certainly opportunities here. I think you’re right to point out that these are jobs AI won’t be able to replace – at least not adequately. But that’s also the danger – that certain large corporations attempt to use AI to do exactly that – all to save money. In the process ending up hurting all of us. The government needs to step in and apply rules. I worry that bureaucracy won’t be able to keep up with the increasing pace of change. Thank you for sharing your thoughts Cristiana 🙏

  9. AI is misnamed. There is no intelligent thought. It merely assembles pieces of ideas that already exist and, as a couple of lawyers discovered to their own humiliation just recently, it can assemble those pieces into citations of things that never existed.

    Train an AI routine to focus on security and there’s a risk it will tag its human handlers as threats and lock them out or attack them. AI has no real ability to evaluate what it does in terms of what users want.

    AI performs sophisticated tasks in a very dumb way, without the ability to evaluate whether it’s meeting the user’s objectives or not. We have the recent example of a cabin depressurization and the autopilot flying in the wrong direction and altitude, insuring the death of all on board. AI can do the equivalent of that in any field in which its employed. It will produce something, but whether that something is good, bad or catastrophic, it doesn’t know.

    By the way, did you see the recent disclosure of 19 deaths and 730-something accidents caused by the Tesla self-driving software. As people get burned using it, they’re going to be gunshy for quite some time.

    1. Thanks for sharing your thoughts Vic – I agree. This is why we need to be cognisant. That’s not to say we shouldn’t use this technology – but we need to make sure the tool is working as intended. And be sure to take the automation out when it isn’t functioning as it should.🙏

  10. Large boy has been playing with AI to make art and I keep telling him to stop feeding the beast. What the one he uses generates is obviously not from human hand. I saw a reel on FB yesterday tho that I thought was AI but all the comments were appreciating the beauty and asking where the beach was; I assume anyone questioning the provenance had their comments removed.
    So yeah, I totally agree is that AI for art, music etc is a terrible idea.

    1. Stop feeding the beast is a great way to put it. The more it’s fed the more difficult it will be to determine what’s made by a human and what isn’t. If we sideline the real artists of the world as a result we all be worse off. Thank you for your comment 🙏

Leave a Reply