By Christopher Acheson
I have been thinking a lot about AI and its ability to automate content this year. By this point, there is no question that AI can generate content in ways that were unimaginable just a year ago. However, the question I have been asking myself is: “Can AI produce content of equal quality to a human writer?” To find this out, I have experimented with this myself and written up my findings.
There are two questions to answer as part of this:
- Is there a difference between an AI writer and a human one?
- How can you compare the quality of AI-generated writing and human writing?
What is the difference between humans and AI?
When talking about the quality of what AI produces, we need to recognise how AI “thinks” – or rather how it doesn’t think.
When a Large Language Model (LLM) AI responds to a prompt, the response that is produced is the statistically most-likely response according to the data it has available. In the case of ChatGPT, this data is the entire internet. As a result, there is a risk that the resulting content and its tone will be very generic and repetitive – particularly without very clear and restrictive prompting.
The second issue is due to how humans see the world. Too often, we measure something’s intelligence and reasoning ability by how well it communicates. If something talks like us, we assume that it can think like we do as well.
Think of a parrot that can imitate human speech. Is it talking intelligently and understanding what it says, or is it repeating the noises that provide the result it wants? The danger with AI is assuming that a clearly expressed answer is both accurate and well-written.
An AI is able to automate the process of content creation – creating in minutes what could take a human writer hours to write from scratch. However, an experienced human writer is able to write content which has been carefully considered and targeted for a specific audience. Provided, of course, they know what they are talking about.
How can we test the difference in quality?
There is a lot that goes into writing a quality blog. In our archives, we have many blogs discussing this and how to up your blog-writing skills. For testing the quality of a human-written blog against an AI-generated one, there were three specific elements I wanted to examine: the content, the writing style and readability, and the search engine optimisation (SEO).
Content
When writing a blog, it is not enough to share information without deeper analysis and thought. Ensuring that your content is people-first and tailored to your target audience and niche is important. Not only does focused, people-first content rank better on search engines, but being useful to your reader means that they are also more likely to share it with others and trust you as an expert who can help with their issues.
Readability
Part of the power of a blog is its ability to share detailed information in a bite-sized and digestible way. A natural-sounding voice that makes its point without overcomplicating things makes the experience more supportive for the reader. The harder the reader has to work, the less likely they are to get what they need from reading. A good blog will use a clear structure and a writing style that guides the reader through the topic without overcomplicating or oversimplifying the information.
SEO
It doesn’t matter how good your content is if nobody can find it online. Strong SEO is what gives content its visibility by helping search engines discover it and rank it. A blog with good SEO can help raise your website’s visibility and build a community around you and your brand.
How we ran our experiment
Earlier this year, I wrote a blog on the difference between memoirs and autobiographies. As the topic was one that required less specialised and niche knowledge, I decided that this would be a good candidate for this experiment.
Using OpenAI’s GPT4 chatbot interface, I prompted the AI to generate a suggested structure for the blog, then generate content for each section. In order to test the quality of the content without human intervention, I did not focus or reprompt the results. The result was a 1,600 word blog on the topic.
I then assessed both blogs based on the three criteria I’ve just outlined (content, readability and SEO).
Results
Writing a blog is an involved task. It takes time and research to do. As expected, ChatGPT was able to do in minutes what had taken me hours. However, writing a blog as part of a content strategy means that your writing has specific goals. Can an AI achieve those goals as well as a human can?
When we looked at the quality of the content, readability, and SEO of both blogs, the human-written blog performed better every time. Based on the instruction “write a blog on biographies versus memoirs”, the human writer was able to extrapolate the needs and search intent of the reader. As a result, the human-written content better served the precise needs of the readers and the business.
The issue for the AI is that purely AI-generated content is more likely to be generic and less tailored to the reader. Without human oversight and expertise working to focus and refine the content, it failed to address the reader’s questions or needs clearly or concisely. You can read a more thorough analysis here.
This is not to say that AI is useless in content creation, however. The power of AI is not to generate content without any human input but rather to assist humans in expressing their ideas clearly.
Rather than seeing things as “either-or”, we recommend a synergistic “AI-assisted approach” to content creation. Rather than using AI as our replacement, we can instead use it to take the roadblocks that used to slow us down out of our way. This allows us to produce quality content quickly and easily while preserving our unique voice, perspective, and expertise.
Are you using AI to streamline or automate your content creation? How do you wish you could use AI to automate tasks while not compromising on quality?
For more information on how you can harness the power of AI ethically and legally in your content, book a call today.