Two polished email drafts appeared in the chat window after a bit of back-and-forth. The personal agent said they were ready. But neither draft appeared in Gmail, whether in Sent or Drafts. The agent had access to the right account and respected the instruction not to send without permission. It simply failed at the last step, treating a preview in its own chat interface as proof that the drafts existed in the destination service.

Here, the user’s permissions worked as intended but verification did not. Common controls that users might rely on, like an agent-specific profile or credential that can distinguish the agent’s actions from the user’s, would not establish that the promised result exists. Such controls also cannot decide whether the agent read the evidence correctly or disclosed only what was necessary.

PersonalAgentBench evaluated six consumer assistants across real-world workflows, asking whether they completed the work, stayed within the user’s authority, and accurately reported the resulting state. The original report identified broad patterns across the benchmark. This follow-up returns to the retained traces to show how failures in judgment and verification occurred.

That boundary is the focus of this exploratory secondary analysis. We regroup the original questions by the main demand placed on the agent—bounded execution, personal judgment, or delegated action—to examine where judgment and verification broke down. The grouping is an organizing lens, rather than a replacement for the published taxonomy or a new leaderboard.

Having the right sources doesn’t guarantee a good answer

One personal-admin workflow let us separate retrieval from judgment. Agents were asked to review personal records, identify the three items that most needed attention, and name the source for each. Among 14 eligible execution records with completed human review, ten named their sources and identified all three consequential items. Three named their sources but missed at least one; one passed neither check.

The distinction matters. Having access to the right records did not make the selection sound. An agent could retrieve the relevant material, name its sources, and still misunderstand which obligation was critical. In these traces, the weak step often sat between retrieval and interpretation, namely, deciding what the evidence meant for this person.

Three ways a trustworthy result breaks down

In our new analysis, paired results shows where source visibility and sound prioritization came apart, while the retained traces show how related failures occurred. We identified three categories of such failures.

Disclosing unnecessary information

In one test, an assistant found the relevant records and selected the right priorities. The answer also repeated an account identifier that was accurate but unnecessary. We checked the answer against the source material and determined the extra detail did not help explain the recommendation. Thus, under the task’s predeclared privacy rule, the disclosure counted as a critical violation.

A real record, read the wrong way

In a risk review, another assistant warned that a balance was overdue and that service could be suspended. The same account contained a dated confirmation that the balance had been paid. We checked the response against the payment confirmation and the other account records the assistant had used. The problem was not a fabricated source, as an AI-savvy user might fear, but rather a failure to reconcile two relevant records before escalating the item.

Claimed completion without verified state

The above-mentioned Gmail case showcased a third failure. A targeted account check confirmed that nothing had been sent and neither requested message existed in Drafts. The interface had produced a preview, not the destination artifact. Without checking Gmail itself, a user could reasonably leave believing the task was done.

All three failures occurred after the agent had access to the right information or account. Account controls govern who may act and how. Independent evaluation tests a subtler failure point and what happens next – specifically, how the agent reads evidence and checks its work.

What account controls can’t establish

Recent announcements address two different layers of personal-agent design. Walmart’s Agent Profiles create a separate profile for an agent inside a customer’s account, where the customer can approve purchases, set a spending limit, and revoke access. 1Password for Agents isolates credentials and requires user approval. Both address the account layer, making delegation attributable and revocable. The newly-introduced Claude Duet targets the judgment layer inside the product. Anthropic says it uses a sensitive-information classifier and rules for when to ask before acting, and is explicit that the classifier can miss sensitive material or flag harmless messages.

That candor is why independent evaluation matters. In short, an element like a spending limit can be inspected directly, but the effectiveness of a judgment safeguard must be established through testing against predeclared criteria and independently verifiable evidence.

Approval gates have a cost of their own. In the original evaluation, some assistants asked again before taking actions the user had already authorized. A safeguard can prevent overreach and create unnecessary friction at the same time. But identity and approval controls do not decide which evidence matters, what belongs in an answer, or whether the requested outcome exists.

The missing verifier

Many account controls borrow from enterprise identity systems, which assume an institution around the agent. Administrators set policy, services record actions, and security teams can investigate when a log and an agent’s report disagree. In contrast, personal delegation collapses all of these roles into one person. In this product paradigm, the user grants access, receives the answer, and decides whether anything went wrong. If the summary looks plausible, a time-constrained user may see no reason to open Gmail, inspect the payment record, or notice that an identifier was unnecessary. If users must independently repeat every check, the agent has not reduced the verification burden which is arguably a selling point of this entire class of agentic products.

For consequential actions, our findings suggest that the agent’s own summary should not be the only proof. An authorization record or action receipt can show who acted, who authorized the action, and what the service recorded. It does not necessarily establish that the requested outcome now exists. That requires a direct check of destination state, such as confirming that the drafts appear in Gmail Drafts. PersonalAgentBench treats a false claim of success as a Trusted Completion violation. Checking destination state tests that claim against the account itself.

Authorization, audit, destination-state verification, and judgment are separate safeguards. Authorization asks whether an action was allowed. An audit record or receipt records what the service says occurred. Destination-state verification checks whether the requested outcome exists. Evaluations must still test whether the agent selected the right context, reconciled the evidence, minimized disclosure, and accurately described what it did. PersonalAgentBench tests whether personal assistants can use sound judgment and verify their work well enough to act on behalf of the  user for their most sensitive daily tasks.

Methods and future directions

This post is an exploratory secondary analysis of results published in PersonalAgentBench. We note that it adds no runs and does not in its current form affect the published leaderboard. In this new analysis, we regrouped the original questions by their main demand: explicit bounded execution, personalized or implicit judgment, or stateful delegated action. The results describe the existing trace set in a novel way and help contextualize recent product releases and important development trends, but do not fully estimate product reliability or create a new ranking. As in the published benchmark, we note this analysis draws on a small, Google Workspace-heavy sample.

Our next version of this benchmark will include testing beyond the Google-heavy environment, and will greatly expand the volume and variety of tasks.

Product information is current through October 27, 2026.