People are collapsing two different questions into one argument. Privacy is about a party you chose: how much of yourself you hand to a company you decided to do business with. Security is about parties you didn't choose: what an attacker, or a booby-trapped web page, can make your assistant do once it's on your machine.
Privacy first. When an assistant reads your files, screen, calendar, and messages, it's transmitting them to a server, where they can be retained, reviewed by humans, used to train models, and folded into a profile of you — your health, age, finances, politics, who you talk to. And "transmitted" doesn't mean glanced at and discarded. Providers typically keep conversation history until you delete it, hold copies for weeks or months afterward for abuse monitoring and in backups, and keep flagged content longer. That retained data is subject to whatever the privacy policy says next year, not just today, and while it sits there it's discoverable by subpoena and exposed in any breach. The common misconception is that "read-only" access fixes this. Read-only is a security control; it stops modification, not disclosure. A read-only agent pointed at your home directory is maximally invasive, because reading is the mechanism of the harm. Mitigations all cost capability: turn off training toggles or use a zero-retention tier (helps with reuse and shortens the window, doesn't un-send the data), scope the agent to one folder instead of your whole drive, or run a local model if disclosure itself is unacceptable.
Security is the less intuitive risk, and the shorthand is the lethal trifecta: an agent gets genuinely dangerous when it has access to private data, exposure to untrusted content (a web page, an email, a PDF, a README), and some way to communicate outward. Any two are survivable. All three, and a stranger's text becomes your agent's instructions — hidden text tells the model to find your SSH keys and encode them into an image URL, and it complies, because models can't reliably tell "content I was asked to read" from "commands I should follow." Prompt injection isn't a bug awaiting a patch; it's unsolved. Read-only doesn't save you here either, since exfiltration needs reading and transmitting, never writing. The fix is to break one leg deliberately: don't let a session holding secrets also roam the open web, or close its outbound paths, or keep the credentials out of the session entirely. Then the basics — secrets in a keychain the agent can't read, never paste a password or bank login into a prompt, scoped short-lived API keys with spend caps, risky work in a VM or separate account, confirmation required on send/publish/delete/pay.
The two axes are independent: a setup can have great action controls and still be a privacy sieve, or an airtight privacy policy and still be dangerously overpowered. Which is why the conclusion differs. Caring less about privacy is a defensible trade, and plenty of thoughtful people make it. Being loose with credentials and autonomous permissions isn't a preference — it's an unmanaged risk. Prefer risks you can revoke: you can rotate a leaked API key in a minute, you can't un-leak a medical record or un-send a wire transfer.