Who is the user when an AI tool never leaves the building?

Saleema Amershi and colleagues published 18 design guidelines for human-AI interaction at CHI 2019, in Glasgow, that May. They had condensed more than 150 recommendations from academic and industry sources. A user study in the paper asked 49 design practitioners to test the set against 20 popular products that put an AI feature in front of a person. The paper's name for that object is an AI-infused system: the capability is exposed directly to the end user. The authors are plain about what happens when the behaviour is inconsistent. People get confused, confidence drops, and they abandon the technology.

An internal drafting tool meets the definition. The end user is the employee who has to finish the task. The function that signed the order is a constraint on data, cost, and risk. It is not the user the guidelines were written for.

The brief usually names the wrong room

Internal tools are chosen by people who will barely type into them. Security wants a control. A sponsor wants a launch. Learning is asked, late, to "drive adoption" of an interface that has already frozen. Staff are told the tool is available, as if availability were a design.

The first two guidelines are about that opening moment. Make clear what the system can do. Make clear how well it can do it, including how often it may be wrong. Internal rollouts often invert both. The note lists the licence, the classes of data you must not paste, and a link to a portal. It does not say what the tool is bad at. People meet the miss in a meeting, after a draft has already gone out.

That is a design failure. It is not a shortage of enthusiasm. Design thinking that survives the board pack is about keeping a problem honest once a committee is in the room. An internal tool can fail earlier than that. The committee wrote the brief, and the person who does the work on Tuesday was not in it.

Dismissal has to be cheaper than obedience

Two later guidelines matter once the tool is in the day. Support efficient dismissal, so an unwanted suggestion can be ignored. Support efficient correction, so a wrong result can be edited in the place it appeared. If accepting the paragraph is one click and fixing it is a ticket, people will use the tool, and they will use it badly. A usage chart will look healthy. The work will not.

A further guideline is about timing. Offer the service when the person's task and context call for it. A rewrite that lands on a sentence they were content with is not help. It is an interruption with a logo on it. Applied to a staff tool, the same idea is a decision about whose minute you are spending. It is not a visual style you add before launch.

Collective Campus runs design thinking with intact teams so the work, not the licence, sets the problem. A session does not replace sitting with the person who has to live in the tool. It is a way to make that sitting happen before the interface hardens.

What you can check without a new survey

Watch one real task. Do not watch the demo the vendor rehearsed.

1. Can the user say, in one sentence, what the tool will not do? If the only answer is a data rule, you have a prohibition, not a statement of capability.
2. Can they dismiss a suggestion without leaving the task?
3. Can they correct a wrong line without starting the task again?
4. Did the tool wait for a moment of need, or did it decorate work that was already going well?

The study behind the guidelines used popular products, not your tenancy. The transfer is still ordinary. Your employee is an end user. If the four checks fail, another round of instruction on the same screen will teach people to click through. Change the screen, or stop calling the rollout a design.

Source: Saleema Amershi and colleagues, "Guidelines for Human-AI Interaction", CHI 2019. https://www.microsoft.com/en-us/research/publication/guidelines-for-human-ai-interaction/