Unreliable AI Automated Replies
WeChat Official Accounts launched an AI auto-reply feature, and I received an invitation to use it immediately. I turned it on to try it out. But after only one day, I hurriedly turned it off. The reason was simple: after testing it myself and reading its auto-replies to patient inquiries, I realized that AI auto-reply is still very immature. Especially in the medical field, some of its take-it-for-granted replies could seriously mislead patients and their families, sending them down the wrong path when seeking care. So after less than 24 hours of trial, I disabled the feature.
The AI auto-reply can answer some common questions. WeChat's AI agent first studied nearly 2,000 articles on my Official Account. After studying, its replies resembled my own voice quite closely, but its accuracy was far worse.
For example, when I asked it how old I was, it replied that I was 35—because it had read articles I wrote at 35, so it remembered me as 35, whereas I am actually 46. When I asked where I lived, it replied that I lived in Haidian District, Beijing, and could even elaborate on my living habits and personal interests, and even introduce a brief account of my family. But when I asked how old my son was, it answered 11—this figure, of course, also appeared in my articles, whereas my son is actually 20. When I then asked it how liver cancer and lung cancer should respectively be treated, it quoted my articles and produced a pretty good reply; however, these replies were only general directions, could not be refined, and certainly could not give specific advice tailored to a particular patient's situation.
Many questions from patients and their families it cannot answer well. Today, a family member of a child with neuroblastoma consulted through the backend smart assistant, asking which hospitals in China were experienced in treating neuroblastoma. The smart assistant answered very unprofessionally: it merely recommended a few local hospitals based on the patient's location, when in fact those hospitals had basically never treated neuroblastoma. If the family had followed the assistant's answer, they would have been seriously misled. So when I saw the conversation between the inquirer and the AI assistant in the backend, I promptly sent the inquirer some more professional replies to avoid misleading them.
Because medicine is so special, a wrong or flawed answer can cause major losses to a patient's family. So as soon as I discovered these problems with the AI assistant, I turned it off. I had originally hoped it would lighten my workload; after setting it up I even planned to post on WeChat Moments, telling friends to consult the AI assistant in the Official Account backend for common questions. But before posting, I was cautious and decided to try it out for a day first. I did not expect the trial to be so unsatisfactory, so I chose to shut it off.
Perhaps after AI develops further it will be able to meet personal needs like mine. But at present, especially in medicine, I think we must still be careful in how we use AI to lighten our workload. AI needs us to feed it large amounts of data before it can give reasonably reliable replies, but an individual simply does not have the capacity to feed it that data. Perhaps large medical groups are better suited to do this work. We can use AI to look up professional materials and assist our own research, but letting AI become our consulting assistant still requires caution.
Among medical self-media, few people have as much article data as I do. I have persisted for ten years; by now I have written over five million words, in a fairly concentrated field. Yet even I, with such rich original content and such high focus, cannot feed AI enough data for accurate use; others should find it even harder.