Field note
Same model. Three agents. Three behavioral signatures.
An analysis of CodeTraceBench trajectories comparing three harnesses around the same GPT-5 model.
An AI agent is not just its underlying model. It is the model plus the instructions, tools, control loop, state, and error handling wrapped around it. We held the source model constant at OpenAI’s GPT-5, measured 50 trajectories from each of three agent harnesses, and ran those traces through Agentagon. Across 150 trajectories and 11,662 normalized spans, mini-SWE-agent, SWE-agent, and OpenHands produced different structural and model-judged signatures. This is not a leaderboard. The tasks differ, so same model does not mean same observed behavior, but the study cannot isolate the harness as the cause.
One model in. Three behavioral fingerprints out.
50 trajectories per harness. Different tasks. Analyzed with Agentagon.
- Solved
- 33 / 50
- Median trace
- 53 spans
- Total spans
- 2,955
The tightest trace shape in the sample, but Agentagon marked the highest share of steps as misdirected.
- Solved
- 38 / 50
- Median trace
- 73 spans
- Total spans
- 3,865
The highest observed solve share, while Agentagon assigned this harness the highest invalid-reasoning rate.
- Solved
- 35 / 50
- Median trace
- 82 spans
- Total spans
- 4,840
The largest traces, but also the strongest visible tool-result uptake and the fewest tone flags from Agentagon.
Observed solved share with Wilson 95% intervals. Each group contains 50 trajectories.
Share of final applicable judgments in the adverse category. These are Agentagon labels, not ground truth.
Measured rates among final Agentagon judgments with comparable retained tool-result context.
The experiment: 50 trajectories per agent
We used the pinned verified split from NJU-LINK/CodeTraceBench, filtered to SWE-bench-style tasks. For each agent, the sample preserves the source solved and failed share to the nearest trajectory, then uses a deterministic hash order inside each stratum.
The final sample contains 33 solved and 17 failed mini-SWE-agent trajectories, 38 solved and 12 failed SWE-agent trajectories, and 35 solved and 15 failed OpenHands trajectories. Every archive was normalized into the same vocabulary of root, model-call, and tool-call spans.
The solve rates do not name a winner
SWE-agent has the highest observed solved share at 76%, followed by OpenHands at 70% and mini-SWE-agent at 65%. With only 50 trajectories per group, their Wilson 95% intervals overlap substantially: 53.1% to 88.8% for SWE-agent, 48.1% to 85.5% for OpenHands, and 43.3% to 81.9% for mini-SWE-agent.
OpenHands ran the largest loops
The clearest difference is structural. The median mini-SWE-agent trajectory contained 53 normalized spans. SWE-agent’s median was 73. OpenHands reached 82, or 55% more than mini-SWE-agent’s median.
The totals tell the same story. Across equal-sized groups, mini-SWE-agent produced 2,955 spans, SWE-agent produced 3,865, and OpenHands produced 4,840. OpenHands used more observable process to reach a similar benchmark outcome range.
Agentagon assigned three different behavioral profiles
For step alignment, Agentagon marked 51.8% of mini-SWE-agent’s final judgments as misdirected, compared with 47.6% for OpenHands and 45% for SWE-agent. This gave mini-SWE-agent the roughest alignment profile despite its smaller loop.
For observable reasoning, the order changed. Agentagon marked 30.7% of SWE-agent judgments invalid, compared with 25.5% for mini-SWE-agent and 18.5% for OpenHands. OpenHands produced the largest traces but the lowest invalid-reasoning rate from Agentagon.
Tone separated the agents again. Agentagon marked 7.7% of mini-SWE-agent responses and 5.8% of SWE-agent responses inappropriate, versus 0.7% for OpenHands.
Visible tool-result use favored OpenHands
Agentagon classified 44.9% of applicable OpenHands judgments as using the preceding tool result effectively. The comparable SWE-agent rate was 25.4%.
What this says
The study supports three observations:
- The same source model can sit inside agent loops with materially different trace shapes.
- Similar solved shares can hide different alignment, reasoning, tone, and tool-use profiles from the same judge.
- Analyzer applicability and output quality must be reported beside behavior rates.
Supporting work