David Trinko: Testing the limits of accuracy with ChatGPT

If you ever need a good laugh, ask artificial intelligence to tell you about your life.

Like so many people, I’m intrigued by ChatGPT, the online artificial intelligence that has people all worked up with its ability to write passable English sentences.

In a world of deep-fake videos and Photoshopped images, the possibility of a machine replacing people certainly has a folktale sound to it, like John Henry going up against the steam drill.

It’s impressive, being able to type in your request and get something that looks like it could be right. For instance, I asked it to write 500 words about my life.

I learned a lot of interesting things about myself. It was nice to hear that I’m “an accomplished journalist, author and speaker” known for “his insightful and engaging work across a wide range of topics and for his commitment to journalism ethics and delivering accurate, well-researched and thought-provoking stories.”

Maybe this technology isn’t so bad. It’s good at flattery.

Then I read on a bit more, and I learned some things I didn’t know about myself. For instance, I didn’t know I was married to someone named Sarah and had two children with her. I hope ChatGPT doesn’t tell the wife I know about, Jessica, or any of our four children.

I also read that I grew up in Lima, which was news to me and probably to my hometown of Arlington, in Hancock County. I apparently have also written two books, which I didn’t, although I look forward to receiving payments for those. I also apparently graduated from Ohio Northern University, which was a surprise to me since my diploma and memories say Ohio University.

I suppose it’s a word to the wise that you should consider anything you read from an AI to be partially artificial. I’ve learned that internal research found these stories to be about 90% accurate. That’s also to say it’s 10% inaccurate, which isn’t acceptable in journalism.

That’s not to say it can’t be entertaining to run things through its machine-mind. We asked it to write a pirate drinking song about one of our dogs, telling it a few things:

“Well shiver me timbers, there’s a dog named Mack,” ChatGPT wrote. “A Golden Doodle with a bark that could scare off a pack / He loves to hear himself, oh his bark is quite grand / But after a while, it gets a bit out of hand.

“He’s got an infatuation with Jessica, his lass / Follows her around like a true pirate’s first mate, no sass / Though his love is pure, it can be a bit much / Sometimes she just needs a break, a moment to clutch.

“But don’t you worry, Mack’s still a good dog / Loyal and friendly like any proper pirate’s log / He’ll keep you company through thick and thin / And make sure your spirits stay high with a grin.”

I have to give the AI credit on that one. No human would ever write something like this. Then again, no human would ever try.

The trick — and the very human part of writing — is being able to tell what’s interesting and what’s not. After all, ChatGPT wouldn’t have delivered this refrain to a song about our other dog, Brother, without getting the right information from the right people.

“Brother, my faithful canine friend / You may be blind and deaf, but my love for you won’t end / You’re a big dog, with a heart so true / And even when you fart, I’m still in love with you.”

To be fair, the technology is willing to admit it can’t replace me just yet.

“While I can generate text-based responses to questions and provide information, I don’t have the ability to conduct interviews, gather sources or conduct original research,” ChatGPT responded. “Journalists play an important role in society by providing accurate and impartial reporting on current events, conducting in-depth investigations, and holding those in power accountable. While AI language models like myself can be helpful tools for generating text-based content, we can’t replace the unique perspective and skills that journalists bring to their work.”


See past columns by David Trinko at LimaOhio.com/tag/trinko.

Subscribe to the Trinko Thinks So podcast at LimaOhio.com/podcasts.

David Trinko is editor of The Lima News. Reach him at 567-242-0467, by email at [email protected] or on Twitter @Lima_Trinko.