How I Learned to Use ChatGPT as a Legal Content Creation Tool
Since ChatGPT’s emergence, We Do Web has incorporated it as a useful tool. It helps us create quality content more efficiently, which is part of what makes us a leader in legal content creation. However, ChatGPT is not without its limitations, something we have to remember every time we use it.
ChatGPT is NOT a true AI. Not yet.
In the time I have trained with this tool, I have developed efficient prompts, key phrases, and tricks to write quality content, but ultimately, ChatGPT is only as good as the person using it. Putting your faith in it alone is like asking a hammer to build your house. It’s a tool, not a replacement.
ChatGPT Is a Tool
Even though ChatGPT is marketed as an AI, or an “evolving” AI, it’s more akin to a very powerful predictive text generator. Instead of just generating the next word, it creates the next phrase based on its training as an LLM, a large language model.
It can generate outlines for entire essays and even provide scripted feedback. It can adhere to parts of speech, grammatical rules, and syntax, but it doesn’t understand value. This is where it falls short.
I’m reminded of the 1986 film The Fly, starring Jeff Goldblum. In the movie, Doctor Seth Brundle invents a teleporter, but items teleported never fully retain all their inherent quality or essence. For example, a teleported steak tastes “synthetic” and unappealing. As Doctor Brundle says, he has to teach his computer how to taste.
ChatGPT can’t taste. It doesn’t understand quality. The content it generates lacks human insight and experience. It can state basic facts but fails to reflect first-hand expertise and knowledge. It fails to create compelling, informative content or to convey the value of a law firm’s expertise. It struggles to connect. It needs a human touch.
Probabilistic vs Deterministic
When prompted, ChatGPT generates content based on the most likely response with some degree of variation (probabilistic). And because of that, it does not give a predictable response (deterministic). For example, if you ask ChatGPT to,
“Write me an essay that’s 1000 words long.”
It doesn’t know the different values we place on that prompt. It only knows the most common response. We understand that this prompt has two criteria that are not the same value:
- Be creative: write me an essay
- Be accurate: that’s 1000 words long
For ChatGPT, it doesn’t see a difference between those two phrases. It’s one statement, one prompt. It is incapable of understanding the different values we place on each phrase. It is simply providing the most likely response to the whole statement.
As a result of this limitation, ChatGPT will generate content, but it will estimate (aka hallucinate) a word count and then claim it is true. And ChatGPT hallucinates about a lot of things. You simply cannot trust it for accurate responses.
We Will Always Need Experts
Technological advancement has always changed trades and professions. The buggy whip maker is an old metaphor for this. As automobiles began to replace horse-drawn carriages, buggy whip makers either adapted to the new times or went out of business.
Change is inevitable. We must embrace change for growth. Even though most buggy whip makers are gone, we never lost the need to get from point A to point B.
ChatGPT is a paradigm shift in how SEO providers will manage a company’s marketing and digital presence. Content writers, strategists, and SEO specialists will either adapt to this new tool or go out of business, just like the inflexible buggy whip makers. That’s why I embrace ChatGPT as an excellent tool for day-to-day efficiency when I write new, original content.
However, no matter how sophisticated AI becomes, we will always need experts. AI cannot eliminate the need for human experience and expertise because it doesn’t create; it only copies.
The Compare and Contrast Project
I had an idea. I’d been working with ChatGPT and thought about creating a custom GPT. The goal was to identify how we could improve our client’s content. I set up the GPT to:
- Search for the top 5 law firm pages for a particular practice area in a specific target location
- Compare those top 5 pages for what they had in common and what set them apart
- Present the findings in a spreadsheet format
The project started out well enough and had the support of my team, but it never really worked out. For example, I could never get ChatGPT’s search results to match my own.
If I searched for “Miami car accident lawyer,” we would find two completely different sets of results. Through multiple attempts and queries, I eventually found that to even get close to matching results would require coding an API (application programming interface) key to an Ahrefs or Semrush account.
Additionally, when it did generate the report, it would use sources other than the top 5 pages it claimed to be comparing. It was a frustrating experience.
What Did I Learn Working With ChatGPT?
Although the project didn’t work out as intended, and others had already created something similar, I did learn some things working with it.
Strict Adherence Rules
You can tell your custom GPT to strictly adhere to an instruction. We frequently use these instructions to ensure branding consistency for our clients. It’s also handy to exclude specific phrases and topics our clients have banned from their content.
Conditional Rules
It is also possible to tell the custom GPT to only take an action if certain conditions exist. It’s basically if/then programming. For example, if the topic is a blog article, don’t mention the client’s name until the call to action at the end.
Integrated Balance
This rule is also fascinating to me. Take, for example, that your client has several unique selling points that you want to include in your content. If you’d like to spread these marketing points throughout the piece, you can add them to your custom GPT and instruct it to include them, but balance it with the legal content. That way, it can sprinkle them through the introduction, body, and closing.
However, I still had to give it some if/then examples to understand what I wanted, such as mentioning when the law firm was founded in the introduction or including how much the client has recovered when discussing damages.
AI Needs Adult Supervision
Here’s the downside. Even with those rules, ChatGPT will not understand the value of your instructions. You can tell it to follow every rule and instruction, but it’s in its programming to hallucinate. It will take it upon itself to present what is the most likely response. It is not capable of understanding what content is most valuable to a client.
However, if you want to control how much it deviates from your instructions, you could include a quality check rule. You could instruct the custom GPT to pause and check its response against all the guidelines and instructions you’ve given it. Even then, it’s still not foolproof. You will have to check the work.
What’s the Take Away?
ChatGPT is a tool, but like any tool, it is only as good as the person who is wielding it. It makes my job easier by improving day-to-day efficiency, but it doesn’t replace the need for manual proofreading, editing, and research.
