Where to put the model in a workflow

A model call is one step in a larger process. Before choosing a model, decide where the result is needed and what should happen if it does not arrive.

Start with the waiting time

If a person needs the result to continue, a direct call may be enough. Show that the request is in progress, set a time limit, and keep their input if the request fails.

If the result can arrive later, put the work in a queue. This adds another component, but the request no longer depends on an open browser tab.

Decision	Direct call	Queue
Result needed now	Usually a better fit	Requires a status view
Long tasks	Limited by request timeout	Can continue in the background
Retry handling	Caller must manage it	Worker can manage it
Components to operate	Fewer	More

Keep the boundary small

The rest of the application should depend on a task result, not a provider response. Validate the result at the boundary and store enough context to understand a failure.

interface TaskResult {
  summary: string;
  sourceIds: string[];
}

This boundary makes it easier to change a provider or test the workflow without calling a model.

Write down the trade-off

A queue does not make the task more accurate. It changes how the system waits and recovers. A useful decision record says which problem the extra component solves and which costs it adds.

The first question is simple: does the person need this answer now?