Loading...
Loading...
Contributors:
Dataset URI:
10.5281/zenodo.23808696Raw per-run records behind four small benchmarks on how agent tooling fails: a streamed tool-call argument truncated mid-token that gets repaired into a call the user never wrote; a degenerate repetition loop streamed to the end of the budget; the same tool call executed twice after a retry; and a retrieval system that answers where it should abstain. 721 files, copied byte-for-byte from four public repositories at the commits named in the README. Each record names the client, its version, the scenario and the outcome, so every number in the corresponding write-up can be re-derived instead of trusted. Nothing here is analysis, and nothing here is a ranking: the clients were run at different versions on different days and the scenario counts differ per benchmark.
Keyword(s):
agents; benchmarking; failure modes; reproducibility; tool calling.
MeSH Term(s):
Grants and Funding:
No Grants or Funding Data Available
Associated Publication(s):
No Associated Publication Data Available
Supplementary Data:
No Supplementary Data Available
Related Dataset(s):
No Related Dataset Data Available
The National Library of Medicine (NLM) provides information about biomedical datasets. Inclusion on an NLM Web page does not imply endorsement of, or agreement with, the contents by NLM or the National Institutes of Health. Learn more: NLM Web Policies (nih.gov)