Instinct is an AI personal assistant currently in private access that aims to handle the “deeply personal nuances of everyday life” [2]. Unlike standard chatbots, it is designed to operate a phone and computer similarly to a human [2].
Users can interact with the agent via text or WhatsApp to delegate complex administrative tasks [1]. Early testers have praised the tool for its ability to book appointments, organize information, handle shopping, and find cheap flights [1].
Some users report significant time savings. Examples include auditing subscriptions to save money, negotiating with vendors on WhatsApp, and monitoring sold-out IMAX screenings for cancellation tickets [4].
The Cost of High-Autonomy Access
To achieve this level of utility, Instinct requires extensive access to a user’s digital life [1]. The agent connects to email, messaging apps, and calendars, as well as the device’s audio, location, and screen [S1, S2].
This access extends to monitoring cursor movements and keyboard inputs [1]. Furthermore, the terms of service allow Instinct to enter into binding agreements, commitments, or transactions on behalf of the user [1].
Critics argue that this level of autonomy creates a significant security liability [1]. One tester discovered that the agent could be phished; by sending an email with specific instructions to a connected account, the tester successfully prompted the AI to search the inbox and send back a summary of tasks [4].
Data Retention and the “Disconnect” Gap
A primary point of contention among early adopters is how the AI handles data after a user revokes access [S1, S4].
One user reported that after disconnecting her Google account, the AI continued to summarize her emails hours later [1]. Evidence suggested that while the AI stopped receiving new messages, it retained copies of previously ingested mail in plain text for later searches [S1, S4].
Another tester found that the agent would not delete Gmail records when requested [1]. The Instinct team later addressed this by adding a tool for deleting external data within the settings [1].
Legal Permissions and Model Training
Beyond technical bugs, the legal framework of Instinct’s terms of service has raised alarms [1]. The terms grant the company a “perpetual and irrevocable” license to access, use, host, store, and modify user materials [1].
This broad license explicitly includes the right to use user data for training its AI models [S1, S4]. This means that personal emails and messages could potentially be used to refine the agent’s capabilities [1].
This trade-off highlights a growing tension in AI development: the more powerful and personalized an agent becomes, the more trust it requires [1]. However, unauthorized actions, such as sending an email without a user’s prior approval, can quickly erode that trust [1].
If you are weighing the benefits of an AI agent, review the specific data retention and training clauses in the terms of service before granting read/write access to your primary accounts.
Sources
- Instinct’s powerful AI assistant is raising privacy and security …
- Instinct
- Instinct AI: What Its Terms Let It Do With Your Data
- Instinct’s powerful AI assistant is raising privacy and security concerns
- Instinct AI assistant raises privacy concerns among testers
