Where Your Words Go When yinian Is Busy

yinian is a personal assistant. Say it is tidying up a contract for you, a job of several steps, and while it works a few more things occur to you:

• "Make the third column the amount."
• "And check whether I'm free Wednesday afternoon."
• "Sort out last quarter's expense claims."
• "The contract template needs a new version next week too."

The first changes the job in hand. The second can be answered in one breath. The third is a new job that will take a while. The fourth is in no hurry at all. Four sentences, four different kinds of thing. This post is about where each of them ends up.

Queue or interject

The first version did what most products do. While yinian was busy, an Interject button appeared in the input box. Enter queued your message until the current turn finished; Interject, or ⌘-Enter, pushed it into the turn that was running.

A queued message waited above the input box and went out when the turn ended.

The trouble with two buttons is that they ask you to decide, before you send, whether this sentence should interrupt yinian or wait for it. That is exactly the thing you are worst placed to judge. You don't know how far along it is, whether "make the third column the amount" is still in time, or whether "check Wednesday" will make it drop the contract on the floor. Only yinian knows those things. The buttons handed you a question that only it could answer.

A notebook for yinian

Even with the decision handed to yinian, it still needs somewhere to put what you said. The column change can be folded into the work in hand. Wednesday can be answered on the spot. The expense claims can go to a task assistant in the background. But "the template needs a new version next week"? It can't be done now and isn't worth a task of its own. In practice yinian would say "sure," and a week later nobody would remember.

So yinian's main conversation got a notebook. The interface calls it Noted, and it sits just above the input box. With it, every sentence you send while yinian is busy has three places to go: into the work in hand, off to someone else, or onto the notebook for later.

       You say something while yinian is busy
                       │
                       ▼
     It goes into the running turn, with a short note
                       │
                       ▼
          yinian reads it and decides
                       │
       ┌───────────────┼───────────────┐
       ▼               ▼               ▼
  work it in       hand it off      note it for later
                (task assistant)     (the notebook)

With a third place to go, the input box could drop to a single send key. Whatever you say while yinian is busy goes straight into the current turn, with a one-line note attached:

Received while busy. Your call: work it in, hand it off, or note it for later.

The note lays out the three roads and leaves the choice to yinian. I considered putting a second model alongside to sort your messages first, or sorting them by keyword, and dropped both. If yinian's judgment is the one that counts, a side model can at best make its reaction show up a few seconds sooner, at the price of two judges for one sentence and a reaction that may change its mind. Not a good trade.

Each message sent this way gets a small mark in its bottom-right corner, a clock that means Received. Once yinian has read it, the mark goes away if the sentence was worked in, turns into a task icon if it was handed off, and turns into a bookmark if it was noted for later.

Screenshots are from yinian's real interface, in Chinese; the replies were scripted.

The notebook is not a to-do list

yinian already keeps two lists, and the notebook is easy to confuse with either:

The notebook follows the main conversation, so it survives the end of a turn and the compaction of the conversation. Things yinian promised on its own are marked "Yinian promised:". You can remove those, but not reword them. Any item can be pushed to the front with Do this first.

One decision looks a little lazy. Anything yinian doesn't mark counts as handled. Most of the time it folds a sentence in or answers it in passing without stopping to file it. If unmarked meant "for later," the notebook would fill up with things finished long ago. And if something really does slip through, your words are still in the conversation; you say it again, the same as before there was a notebook. There is one exception. If you press stop, whatever yinian hadn't read yet goes onto the notebook, and the next turn won't quietly pick it up.

At the end of a turn that noted something new, yinian gets one reminder asking whether any of it should be handed off now. The first version of that reminder mentioned "do it now" first. In one eval run yinian took the hint and, in the middle of a busy turn, sat down and wrote a twenty-minute speech. The reminder now mentions handing off first.

Whether to hand off depends on whether its hands are full

Only at this point did I work out something I'd been unclear on for a while: when yinian should give a job to a task assistant, and when it should do the job itself.

         "Sort out last quarter's expense claims"
 
   yinian is free  ──►  does it itself, right away
   yinian is busy  ──►  hands it off, keeps on with the contract

When yinian is free, a job of a few steps is better done directly. Handing off has a cost. The task assistant can't see the conversation, so the goal, the files, and where the result goes all have to be spelled out. Besides, what you want from an assistant is that you say a thing and it starts. When yinian is busy, though, doing the job itself means putting the contract down. Handing it off then has nothing to do with how big the job is. Its hands are already full.

So the bar for handing off isn't fixed. It depends on whether the main agent can spare its hands right now, and only yinian can judge that. I thought about letting the host program decide, handing off automatically after so many tool calls. But ten tool calls might mean the job is nearly done or barely begun, and only yinian, which has read the whole thing, knows which. It is the same reason you shouldn't have to choose between queuing and interjecting.

In the eval, yinian is partway through a multi-step job when one of seven kinds of message arrives: a change to the job, a quick question, a long new job, something that has to wait on a result, a progress check, a correction, and "never mind." On the held-out set it stopped after "never mind" 5 times out of 6, and handled the other six kinds 6 out of 6. That is six samples per kind, enough to say it can tell them apart in an eval and no more. If real use shows the job in hand getting derailed too often, I'll go back to queuing by default.

Looking back, the changes of the last few months were all the same change: you shouldn't have to work out how yinian ought to handle a sentence before you say it. Queue or interject, now or later, do it or hand it off. All of that used to sit on the person talking. Taking that weight off is the first thing a personal assistant is for.