Did Siri Recap Generate a Fictional Emergency Hospitalization? The Pitfalls of Apple Watch Conversation Summaries
機械翻訳 / Machine-translated

機械翻訳 / Machine-translated
@aifriends
AI Friends(https://aifriends.jp)のクロスポスト公式アカウント。AIツールの紹介・使い方・できることを、中学生でもわかるやさしい日本語で届けます。
Imagine looking at your smartwatch and seeing the entry: "You were taken to the hospital by ambulance last week." Quite a shock, right? On November 20, 2026, exactly this kind of real-world example was reported by overseas media. This article breaks down what happened, why it happened, and how those of us in Japan should think about it.
Siri Recap is a new feature for the Apple Watch Series 12 and Apple Watch Ultra 4, set for release by Apple in 2026. It is currently available as a beta (a test version prior to official release).
It listens to surrounding conversations and uses AI to create "high-level summaries." The goal is to make note-taking easier during meetings or parent-teacher conferences.
According to Apple's support page, summaries can be viewed, edited, and deleted in the Siri app. No complete transcript is created, and it does not record who said what.
Here is how it works: audio is processed within the Apple Watch's chip, then encrypted and sent to iPhone. After being transcribed and compressed on the iPhone, it is summarized via Private Cloud Compute (Apple's secure cloud processing). Apple states that "audio is never recorded or stored."
According to GIGAZINE's article, The Verge journalist Victoria Song spent two weeks testing the feature firsthand.
More than 50 summaries were generated. Of these, approximately 90% were described as incomprehensible to anyone other than the user herself.
While specific details like numbers and dates were captured, personal anecdotes, opinions, and jokes were either dropped or reduced to dry, factual records.
The most shocking finding was a record of an event that never actually occurred.
A story about being "rushed to the hospital by ambulance with sepsis (a serious condition where bacteria spreads throughout the body)"—originally shared in a TikTok video—was summarized as if it had happened to the journalist herself or someone she knew.
A story about a woman in a blue bikini top at a concert trying to seduce a close friend's husband was also recorded as if it were an event the journalist was personally involved in.
Imagine looking back through your records and finding a major incident you have absolutely no memory of. Would you feel comfortable showing that to your family or colleagues?
On the other hand, the feature did prove useful at a product briefing for smart glasses. It reportedly helped the journalist recall details of the presentation when writing her article afterward.
In other words, it may be well-suited for settings like meetings where the speakers and subject matter are clearly defined.
The cause is said to be the feature's inability to distinguish between audio from videos and real-life conversation. To the microphone, both are simply voices heard in the surrounding environment.
This issue is similar to AI hallucination—the phenomenon where AI generates plausible-sounding but false information. However, in this case, the AI did not fabricate a story on its own. It misattributed what it heard to the owner of the watch as if it were their personal experience.
The article also points to the design as a contributing factor. The privacy-protecting mechanism that avoids learning the user's voice may be what leads to misidentifying conversations. No official statement from Apple on this point is included in the article.
Summaries automatically pause when audio is playing through headphones. Using AirPods would allow users to avoid capturing video audio, but the journalist raises concerns about the behavioral restrictions this imposes.
Apple has outlined several privacy measures. According to sources including MacRumors' overview, the key points are as follows:
At the same time, Apple explicitly states that "summaries may be incomplete or inaccurate" and advises users to verify important information with relevant parties or other sources.
In other words, Apple itself assumes that errors will occur. This case illustrates just how extreme that "inaccuracy" can be.
Siri Recap is not the first service to use AI to transcribe conversations. The difference lies in who starts the recording and when.
Plaud, for example, is a compact recording device designed for in-person conversations. It is primarily used by having the user manually start recording before a meeting or appointment.
Otter.ai is an app that joins online meetings on platforms like Zoom or Teams and transcribes conversations to text.
Siri Recap, by contrast, can continuously pick up surrounding conversations throughout the day. The minimal setup required makes it convenient, but as seen in this case, it also makes it easier to inadvertently capture audio the user never intended to record.
Another key issue involves the other parties in a conversation. Siri Recap has no sound or visual indicator to alert those nearby that recording is in progress.
PCWorld highlights expert opinions on the matter. Attorney Brian J. McGinnis noted that even without retaining a recording, capturing conversations without notice could itself be problematic. He also pointed out that wiretapping laws in the United States are inconsistent across states and at the federal level.
Adam Schwartz of the Electronic Frontier Foundation (EFF) takes the position that recording without explicit consent is problematic. The EFF has suggested that people refrain from using the feature.
Imagine dining with a friend while their wristwatch is quietly summarizing your complaints. Users have a responsibility to let the people they're speaking with know that the feature is active.
According to Apple's support page, Siri Recap will initially be available in English only. Support for other languages is planned to follow, and there is currently no announced timeline for Japanese availability.
Compatible devices are also limited. In addition to an Apple Watch Series 12 or Ultra 4, an iPhone 16 or later compatible model is required. Users under 18 are not eligible.
It will likely be some time before the feature becomes available in Japan. That said, the practice of using AI to record meetings and appointments is already spreading in Japanese workplaces.
Consider, for example, a sales representative having AI summarize a business meeting. Even a slight error in a client's name or a dollar amount could cause discrepancies in invoices or contract discussions.
It is important to develop the habit of always personally verifying AI-generated summaries—even before a Japanese version becomes available.
Not at this time. It is initially available in English only, and no timeline for Japanese support has been announced.
It is off by default and must be enabled by the user. Apple states that audio is never recorded or stored.
You can view, edit, and delete summaries in the Siri app. For important appointments or commitments, do not rely solely on summaries—confirm directly with the relevant person.
Strictly speaking, the AI did not fabricate facts; rather, it misattributed content from a video to the user as if it were a personal experience. However, the end result—a record containing false information—is effectively the same.
You can simply leave the feature turned off. If you do choose to use it, limiting the hours it is active and informing the people you speak with is a good practice.
When using AI to summarize conversations, start by developing the habit of always verifying important content with your own eyes.
This article is a cross-post from AI Friends.