AI in customer service: when it should answer, and when it should hand off
When customers ask the same things over and over, an AI can handle that part for you and reply within seconds. What separates a strong setup from a weak one is knowing when the AI should stay silent. This guide walks you through picking which questions to give the machine, ordering your hand-off rules, prepping your knowledge base, and measuring how well it all works.
What AI does in customer service
AI handles four jobs in customer service: understanding the question, finding answers in your documents, handing off to the right person when there is no answer, and summarizing conversations. It does not fully replace your agents. It reduces the time they spend on repeat questions. Store-specific questions get separate coverage in our guide to AI in ecommerce.
Older chatbots ran on a pre-written decision tree. You pressed a button, and the bot showed you a canned reply. An assistant built on a language model understands free-form questions, but that flexibility carries a risk. It can write a fluent answer to a question it doesn't know.
Which questions to leave to AI
| Question type | Who answers | Why |
|---|---|---|
| Opening hours, address, shipping time | AI | The answer is written down and stable |
| Product details, sizing, usage | AI | Read from product documents |
| Order status | AI, if the data is connected | Otherwise it would guess; hand it off |
| Quotes, discounts | Agent | Creates a commitment |
| Return approval, damaged items | Agent | Needs a decision and a record |
| Complaints, legal matters | Agent | Tone and accountability are human work |
| Card, ID, password | Nobody asks for these in chat | Personal data security |
You write the split down once, and it is turned into rules. Live chat stores the split as 'topics that need human approval'. When such a topic comes up, the AI generates no reply. The chat is passed straight to an agent.
Hand-off rules: when AI should stay quiet
The advice 'hand complex questions to a human' cannot be measured unless you define complex. When the order of hand-off checks is a rule, the hand-off becomes measurable. Our site chat checks every message in the order below.
- 1Hand-off word
If the customer types a word like agent, complaint or refund, the chat goes to a person.
- 2Approval topic
If a topic the business marked for human approval comes up, no reply is generated.
- 3Reply cap
By default the AI sends at most 12 replies in one conversation.
- 4Knowledge base
If the documents have no match, the AI does not guess; it hands off.
- 5Reply
The reply draws only on the documents; if unsure, it still hands off.
On WhatsApp, the rules adapt to the channel, so the AI gives no reply in group chats and waits 10 minutes when a team member starts typing. Anyone who writes outside working hours receives the after-hours message. After 6 AI replies, the conversation is handed off to your team. See the WhatsApp AI assistant page for details.
Preparing the knowledge base
The quality of answers can't exceed the quality of your knowledge base. You can't expect correct answers from the AI about what your documents don't say. When two documents disagree, the AI picks one of them.
- Write FAQs in the customer's own words: not "delivery" but "when will my order arrive".
- Put shipping, return and warranty terms with dates and durations into one document.
- Remove old promotions and price lists; the AI reads old documents too.
- Upload files with real text instead of scanned PDFs; a scanned page has no text to read.
- Read the questions that caused hand-offs every month and add their answers to the documents.
Channels: site chat, WhatsApp, comments and DMs
Your customers aren't limited to one channel. They can reach you on WhatsApp, site chat, Instagram comments, and direct messages. With separate screens for each channel, the same customer can get two different answers. Our panel puts all channels in the same inbox.
- Site chat is added with one line of code and uses the same knowledge base.
- A WhatsApp number connects as a linked device by scanning a QR code; an official WhatsApp Business API application is not part of this setup.
- Bulk WhatsApp messages only go to people who wrote in the last 24 hours and haven't asked to opt out.
- Moving from an Instagram comment to a private message follows rules the business writes.
We run the setup for you, and we maintain the rules. Our WhatsApp chatbot service handles the chatbot side. Our lead management service follows incoming requests through to a sale.
Notice and transparency
The European Commission publishes an AI Act overview you can read for yourself. If your business uses a chatbot, you must ensure people know they are interacting with a machine. The Act's transparency rules took effect in August 2026.
Asking for a name, phone number, or email in a chat request counts as personal data collection. You must provide a privacy notice stating why you collect it. Put the notice link in your pre-chat form.
- Say in the welcome message that the customer is talking to an AI assistant.
- Never ask for card numbers, ID numbers or passwords in chat.
- Let the customer reach an agent with a single word.
How to measure success
A claim that your AI resolves most questions is meaningful only when the rate is correctly calculated. A customer can open a chat and ask for an agent transfer minutes later. If that conversation is counted as resolved, the rate is inflated.
| Metric | How it's calculated | Watch out |
|---|---|---|
| Resolution rate | Share of closed chats that never reached an agent | Open chats are excluded |
| First response time | Average time from customer message to first reply | Read after-hours separately |
| Hand-off reason | Keyword, approval topic, cap, no information | The most common reason is the first fix |
| Knowledge gaps | Questions handed off because documents had no answer | Add them to the documents monthly |
| Savings | Interventions avoided × the business's own unit cost | Never estimate the unit cost for them |
Frequently asked questions
Will AI replace customer service agents?
No. It handles the routine information questions that come up again and again, and this frees up time for your agents. Your agents keep handling pricing, refunds, complaints, and legal matters.
What does AI do with a question it can't answer?
In a well-built setup there is no guessing. A question without a knowledge base match gets handed to an agent through our panel. Hand-off happens by default.
What happens if the customer wants a person?
Typing "agent" is enough, and so is typing "complaint" or "refund". The chat then moves to a person. Your business can edit the hand-off words.
What should go into the knowledge base?
Start with your FAQs, shipping, return and warranty terms, product details, and opening hours. Remove old promotion documents before you upload anything. Then upload text files, not scanned PDFs.
Do I need the official API to use an AI assistant on WhatsApp?
In our setup, your number works as a linked device and connects via QR code. The official WhatsApp Business API application is not part of the service. The difference is explained on our WhatsApp chatbot page.
Do I have to tell customers they're talking to AI?
Under the EU AI Act, you must tell people in the EU that they are interacting with a machine. This transparency rule took effect in August 2026. Outside the EU, we recommend saying it in your welcome message.
How do I measure AI's success in customer service?
Calculate your resolution rate from closed chats only, and track first response time and hand-off reasons. Look at the questions causing hand-offs and add them to your knowledge base. Do this every month.
Sources
The official documentation behind the technical points on this page:
- [1]WhatsApp Business Messaging Policy
Businesses using automation must offer a prompt, clear and direct escalation path.
- [2]European Commission: AI Act
Chatbot users must know they are interacting with a machine; transparency rules in effect from August 2026.
Related pages
If you'd like us to run this for you
The WhatsApp chatbot service page lists the scope and the monthly fee. If you'd rather use the panel yourself, you can start with a free account.