Swarm

Ask a model a hard question and you get an answer. Ask again an hour later and you get a different one. Nothing in either reply tells you which parts were solid and which the model made up to fill the shape of an answer.

That bothered me more than a wrong answer would. So I built a bench where the unit of work is the run, and runs stack up instead of scrolling away.

The front screen offers six ways in. Two build a crew of agents: either I describe the goal and the app designs the lineup, or I write every agent myself and wire who reads whose output. Three fire the same prompt at the model over and over, 3 to 30 runs, ten in parallel at a time, then read the runs against each other. One of those three invents a panel of expert personas first, picked to disagree. The sixth turns a prompt into rules and a starting state and steps it forward, wiping the context between steps so the model can't quietly smooth the story out.

Before anything expensive starts, the app asks a short list of questions it thinks are missing from my brief. It aims for three to six, each with suggested answers. I answer them or skip. The answers go to every agent as shared context.

The screen I open first

Not the report. The consensus map. Ten runs come back and the map lays them out as a grid: one row per topic, one column per run, and the number in a cell is where that topic appeared in that run's order. Rows are banded by how many runs contained them, from universal core at the top to the low-frequency tail at the bottom. It aims for 12 to 25 rows.

A model does the clustering, so the rows are its reading of the runs, not a string match. I still trust the grid more than the synthesis report underneath it. A glance tells me which parts every run reached on its own, and which part one run invented.

Two failures the code expects

In manager-led runs, every agent reads the full raw text of every agent before it. No summary in between. Compressing between hops was quietly deleting the detail I ran the swarm to get.

And when the last agent returns under a hundred characters, or text like [complete response], the run notices and retries the synthesis directly against the earlier agents' output. Multi-agent pipelines fail that way often enough to be worth the code.

Where it's thin

There's no index over the saved runs. History search matches titles, modes and prompts, not the body of the reports, so finding a half-remembered result still means opening files. Saving is a button I press, not something that happens on its own. The cost readout only appears on crew runs, and it applies OpenAI's prices whatever provider actually ran, so the modes that fire up to thirty calls tell me nothing about what they cost. None of that is a decision. I just haven't got to it.

The simulation mode is the honest outlier. It works. Not one saved run from it. Saving is manual, so that's weak evidence, and the only evidence I have.

It's a local Streamlit app on my own API keys, one user, nothing to sign up for. 36 saved runs between March and July: 16 from crews, 15 from the two repeat-sampling modes, 5 from expert panels, none from the simulation. SEO strategy and client proposals are the biggest group, then buying decisions. Every run costs real money, which is its own kind of rate limit.

Stack: Python and Streamlit, because the distance from idea to usable tool is short and I'm the only user. CrewAI runs the agents, Tavily does the web search, and each run happens in a subprocess so the stop button actually stops it. Four providers sit behind one switch, OpenAI, Gemini, Kimi and DeepSeek, choosable per agent as well as per run, and a run with an image attached gets moved onto a provider that can see it, because two of the four can't.