If the Value Is Words, Skip AI - Jon Gillham of Originality.ai

Jon Gillham built an AI detector, and this is his rule for content: "if the value that is being provided on that page is words, then AI shouldn't be used." The founder of Originality.ai tells Jeremy Rivera why data plus words with a human in the loop is the use case he backs. Recorded in September 2024, so every number here is a September 2024 number.

Read the full episode recap or watch the conversation on YouTube.

Jon ran a portfolio of his own sites with a team of writers and editors, sold the extra capacity as a content marketing business, and ended up selling that business. When GPT-3 writing tools like Jasper arrived, the question kept coming up: did your writers use AI or not? "We really had no certainty about who was or wasn't using it." So he built a content QA tool and launched it the weekend before ChatGPT came out.

In this episode

• A toolkit for anybody who functions as a copy editor: AI detection, plagiarism, fact checking, grammar, spelling and readability, every step before they can "hit publish with confidence and with integrity."
• Why a detector can't explain itself. Ask why a piece got flagged and Jon says "our answer is we don't know." A classifier finds patterns its builders don't fully understand, and blind studies put educators at about 60% when they try it by eye.
• A fact checker still in beta at recording. It runs a RAG process over Google results, so it depends on what those results hold.
• Readability. Jon's readability study looked at the top 20 results in Google, and he calls "grade eight for everything" wrong. He also sent a guide to checking readability in Google Docs.
• Predicting rankings. Originality trains a model on one question, "does this article rank higher than this article?" Jon reported a correlation score of 60% to 70%, with no modeling over time yet.
• False positives. Originality open-sourced data sets and a testing tool so you can benchmark detectors on your own data.
• AI in Google results. Jon's monthly study of AI content in Google results read about 13% of the top 20 at the time of recording. He sees the March 5, 2024 manual actions as "more of a psyops than an update."

Jon also sent his AI bot blocking page. For another angle on the same question, read does Google penalize AI content?, which quotes Jon on keeping a human in the loop.

Chapters

• (00:00) Meet Jon Gillham
• (01:21) From content marketing to AI detection
• (03:08) Derivative content and the copy editor toolkit
• (05:26) Fact checking with RAG
• (08:08) When the facts depend on the SERP
• (10:18) Why did it get flagged? We don't know
• (12:36) Why humans are bad at spotting AI
• (15:10) The readability study
• (17:07) Predicting which article ranks higher
• (23:46) Correlation and false positives
• (27:07) The snake eating its tail
• (28:29) About 13% of the top 20 in September 2024
• (30:50) Jeremy's $2 million budget story
• (31:44) Helpful content and the March 5 manual actions
• (34:16) The right mix of AI in content
• (37:50) Where to find Jon

Notable quotes

"For the rest of humanity, all known large human data sets have been created." - Jon Gillham

"If their search results are filled with nothing but AI, why would anyone go to Google?" - Jon Gillham

Unscripted SEO is hosted by Jeremy Rivera. More episodes at unscriptedseo.com.

This episode is supported by our sponsors uSERP, ClearLead Digital, Oleno and OppAlerts.

Sponsor disclosure: uSERP, Oleno, ClearLead Digital and OppAlerts are paid sponsors of the Unscripted podcast network. Links to them are marked as sponsored.