Privacy regulators quiz Microsoft over Windows’ incoming Recall AI feature

Yesterday I wrote about the potential winners and losers in Microsoft’s big push to sell powerful and energy-efficient AI PCs. But, while there was a lot to chew on with the hardware aspects, I failed to mention a major angle to the AI-ification of Windows: privacy.

One of Microsoft’s most eye-catching announcements was that of a feature called Recall, which gives the operating system’s Copilot AI assistant “photographic memory” of what the user does on their PC. This involves taking snapshots of the active screen every few seconds, which are then encrypted and kept on the machine. The aim is to help people remember, for example, which website they found stuff on or which folder they stored stuff in. Microsoft will gradually refine the tool, ultimately allowing users to quickly open the document, website, or email shown in the snapshot.

This could clearly be useful to many people, but some privacy experts have concerns about what Microsoft is doing here—and so do data protection authorities.

The BBC reported today that the U.K. Information Commissioner’s Office has asked Microsoft for information to help it “understand the safeguards in place to protect user privacy.” And the Irish Data Protection Commission, which is Microsoft’s main privacy enforcer in the EU, told me it’s also on the case.

“We too have reached out to Microsoft this morning and are looking for further information,” said the Irish watchdog’s communications chief, Graham Doyle. (There's no word yet on any regulatory pushback in the U.S.)

Now, there’s clearly a good deal of privacy built into Windows Recall, as Microsoft explained in a statement: “Recall data is only stored locally and not accessed by Microsoft or anyone who does not have device access. The security to protect a user's Recall data is the same as for anything on the endpoint. With available built-in security and identity features, an attacker would need to get physical access to the device, be able to unlock it and sign-in, and Microsoft already builds in defenses, such as Windows Hello, right into devices.”

Also, per Microsoft’s FAQ on the subject, Recall is designed to avoid taking snapshots of private browsing sessions in the company's Edge browser, and users will be able to click on a special icon to “make choices about what snapshots Recall collects and stores.”

However, the same page warns that “Recall will not hide information such as passwords or financial account numbers” from those viewing the snapshots. And that’s far from being the only potential pitfall for Microsoft here.

The most immediate issue is whether Recall’s implementation complies with the General Data Protection Regulation (GDPR)—an EU law, but one that’s also still replicated in the legal system of post-Brexit Britain. As Keystone Law partner Daniel Tozer told the BBC, the GDPR means “Microsoft will need a lawful basis to record and re-display the user’s personal information.” And with these snapshots often containing sensitive personal information, that basis may need to involve the user’s active consent.

The big question here is whether Microsoft will be getting that consent in a GDPR-compliant (and ethically sound) way. European regulators have generally maintained that you can't claim to have received consent in a situation where the user is automatically opted into the processing of their data. And that is what's happening here.

Although Microsoft insists that people setting up one of the incoming Copilot+ AI PCs will be "informed about Recall and given the option to manage [their] Recall and snapshots preferences"—a spokesperson told me this means "you have the option from the start to opt in or opt out"—the fact remains that not clicking through the Recall settings will automatically authorize the default saving of snapshots.

Then there’s the question of who has access to your computer. Again, keeping everything local comes with clear benefits, but people often share their computers and logins with others, particularly family members. As cybersecurity expert Eva Galperin noted in an X post: “I’ve got some news for Microsoft about how domestic abuse works.”

It’s too early to say that Recall has irredeemable privacy flaws, but Microsoft definitely has to demonstrate that it has answers to all the questions that are rightly being raised, otherwise it may find itself in trouble with both regulators and Windows users.

More news below.

David Meyer

Want to send thoughts or suggestions to Data Sheet? Drop a line here.

This story was originally featured on Fortune.com