Using ChatGPT for Customer Service
If you’ve landed here looking for a quick answer to the question “Can ChatGPT replace my human customer service team?” then here is what you need to know, directly from ChatGPT’s makers:
ChatGPT sometimes writes plausible-sounding but incorrect or nonsensical answers.
So … no. If you want to give your customers accurate, consistently good service experiences, then you can’t have ChatGPT answer them directly without human intervention. Not yet, at least, and maybe never. You can stop reading now; check back in a couple of years.
Although to be fair to ChatGPT, producing plausible-sounding but nonsensical answers was sometimes a successful essay writing strategy for me at school, so perhaps I judge too hastily.
Still, anyone with internet access can today use AI tools to produce paintings, create portraits, generate musical works, and write code, scripts, poetry, and prose, all with little initial effort compared to the traditional approaches.
Could ChatGPT — or something like it, — be used to at least help answer customer service queries? How good is it and in what other ways could it be used in customer service situations?
Let ’s find out.
Author’s note: This is not one of those articles where, right at the end, I reveal that ChatGPT generated the entire text for me. I wrote this myself, and any errors are therefore the fault of my meat-based brain and the pale and aging body that carries it around.
What is ChatGPT?
A while back we wrote about GPT–3, a language model that we tested with real customer service queries, with … mixed results. ChatGPT is another language model based on the newer GPT–3.5, and it’s built specifically for conversational interaction.
It was fine-tuned using human AI trainers submitting conversations where they “played both sides—the user and an AI assistant.” To me, this feels a bit like being fired and then forced to train your own cheaper replacement before you leave.
ChatGPT can be genuinely impressive. You can ask a question, receive a natural-sounding answer, and then hold a follow-up conversation referring back to earlier information asking for re-phrasings, clarification, summaries, or additional details.
At its best, ChatGPT can save you time, enhance your knowledge, and provide inspiration for new approaches. When it works, it feels like a sentient sci-fi computer — one of the good ones, not an evil one. Hopefully.
That is how it feels. That is not what is happening.
What ChatGPT is not
ChatGPT is not a knowledge base or an encyclopedia. It always sounds confident about its answers, even when they are utterly wrong, and it cannot differentiate between “facts” and made up information. That’s why Stack Overflow temporarily banned the use of ChatGPT-generated answers.
We have all been at a party, trapped in a conversation with an overconfident person (let’s be real, it’s almost always a man) who may have qualifications in one field but who holds equally strong opinions on every other topic, all presented as “facts.”
When you know enough about the topic to identify their error, it is irritating. But it’s worse if you can’t spot the error and are left with an understanding that is misleading or outright wrong. Those are not the staff you want on your customer service team, human or AI.
ChatGPT, the unreliable customer service agent
I tested ChatGPT with questions we commonly receive from new Help Scout customers. Note that ChatGPT has no special training in Help Scout and no access to internal information, so it is working from publicly accessible data like our website and knowledge base.
Here is the short version: It worked really well — sometimes. For example, I asked how to get my email from Outlook into Help Scout. The correct answer is to set up a redirect rule in Outlook to send emails to your Help Scout mailbox address.
ChatGPT is sensitive to quite small changes in questions, and in three attempts asking only slightly varied questions, ChatGPT gave me:
The correct instructions for setting up a rule in Outlook.
The correct instructions again, but also a confusing and pointless reference to using Zapier.
Completely incorrect instructions about using IMAP to send outgoing email via Outlook.
Even using the exact same prompt sometimes gave me the correct answer and sometimes an incorrect one. That was enough to confirm that OpenAI’s warning, quoted at the top of this article, was accurate: ChatGPT cannot be relied on for consistently accurate answers to specific questions.
ChatGPT, during my testing, did better with more carefully crafted questions, but every customer service pro knows that a huge part of their job is translating vague and confusing questions into the actual problem that needs to be solved.
This does not mean the technology is useless for customer service, however. There are many options to consider.
Opportunities for AI to improve customer service
How might artificial intelligence software and large language models be used behind the scenes to create better customer experiences?
1. Provide potential answers for human review
We know AI can often predict exactly the right answer, and other times it can offer something that is close to what is needed. Offering up the most likely answers to a customer service person could save a lot of time, but getting to that point would take some real work. ChatGPT itself does not currently allow users to tune the model to their specific content, but you can do so with GPT-3 and other tools.
There are clear privacy and security concerns to consider before allowing a third party access to the customer and internal data that would be needed. Understanding and controlling the data used to train large language models is a huge concern amongst artists today, and text-based inputs will be no different.
Even after training, quality outcomes will require team members who can tell the difference between “that sounds right” and “that is actually right.” In the short term, AI tools that offer predictive text to customer service agents (similar to that built into Gmail) may provide some benefit without the need for customization.
2. Speed up onboarding for new team members
Getting started on a customer service team can be hard work. There are so many places to look for answers and so much background context to learn.
An AI tool could shortcut a lot of searching by suggesting the most useful internal documents and next steps. It may also reduce the burden on the rest of the team members who would otherwise be answering the questions.
3. Summarize long discussions
ChatGPT shows a lot of promise in providing a concise summary of a given set of information. If AI tools can be constrained to just review known good information, they could automatically generate a helpful summary that saves time across the team.
4. Categorize and prioritize
Customer service teams spend a lot of time trying to organize incoming questions, sort them into relevant groups, and understand where problems are being created.
AI tools can save a ton of time by learning from past categorizations and suggesting new and more consistent taxonomies.
5. Monitor quality
As customer service volume increases, keeping an eye on quality and consistency is challenging. AI tools are already effective in sentiment analysis and could be used to identify the conversations that need to be reviewed.
6. Proactively offer help
A quick support response time is excellent, but even better is not having to ask for help at all. Applying AI to determine when a customer is stuck (or about to be stuck) and offering the most relevant help article could really improve customer experiences at a scale that’s harder to do manually.
The future of AI in customer service
“How can AI be used in customer service?” is a perpetual question with an answer that seems to change constantly.
What is true today about the capabilities (or lack of them) will probably not be true next year. We will not see AI-only support any time soon, but the AI-augmenting-humans model is here to stay.
When considering your own usage of AI, keep these points in mind.
1. The best answer is often a question
ChatGPT is never stumped; it will always generate an answer. But customer service agents know that the best first response is often a follow-up question to clarify the customer’s need or a request for more specific details. The best question, and the phrasing of that question, will depend on the type of customer, their past history, their language skills, and so much else.
AI, today, is not ready for that level of conversation.
2. Natural conversations make mistakes harder to find
A lot of what makes ChatGPT impressive is how natural it sounds, at least at first glance. A factual error is easier to spot in a bullet list than it is when buried inside three paragraphs of chatty conversation.
The closer we get to human-sounding AI, the more likely it is that mistakes won’t be noticed until it is too late.
3. Trust really matters
When everything is going well, an AI providing the answer is fine. But when things go wrong and mistakes are made, customers will want accountability and assistance. AI does not care about mistakes, and it can’t feel responsible.
Your customers will always want to know there is a person who cares about their experience and wants to make it better.
The thoughtful application of technology
ChatGPT is not going to replace human customer service staff, though that will not stop some people from trying to make it happen.
For small and medium-sized companies, the time has not yet arrived that AI is essential, or even desirable, for delivering great service. However, continued improvement in language models and the interfaces to them will inevitably become part of the customer service experience in the future.
The companies that succeed with these tools will be those that find ways to use the technology to amplify the things their staff can do best and to add capabilities that people just cannot do (or cannot do at scale).
The more you understand exactly what your customers value, the better decisions you can make about when and where to apply technology in providing customer service.