MosaicLeaks: Can your research agent keep a secret?
Which summary reads better? Pick one — models revealed after.Both summaries are AI-generated.
Research agents designed to combine private documents with web searches risk leaking sensitive information through seemingly harmless queries, a vulnerability known as the mosaic effect. A new benchmark, MosaicLeaks, reveals that even advanced models frequently expose private data, with training for performance often worsening the issue. A proposed solution, Privacy-Aware Deep Research (PA-DR), improves accuracy while significantly reducing leakage.
MosaicLeaks is a Hugging Face/ServiceNow research benchmark examining whether deep-research agents leak private information through external search queries when combining local documents with web retrieval. Tests found agents often exposed sensitive details via cumulative query logs, while a proposed Privacy-Aware Deep Research training method improved strict chain success and sharply reduced answer/full-information leakage.