# neomonic benchmark evidence — MANIFEST

Harness: mem0's open-source benchmark harness, https://github.com/mem0ai/memory-benchmarks, commit `4b61c5d31b9c668a12b4f5e78064248a02c82d2b`.
Only the memory search was replaced with neomonic's; answer prompt, judge prompt and scoring are the harness's own. Answerer and judge: openai/gpt-5.

## File kinds

- `*_scores.json` — the run's final scores. For LoCoMo / LongMemEval: the harness result file's `metadata` and `metrics_by_cutoff`, verbatim. For BEAM: the harness's own `_metrics.json` (field `harness_metrics_file`, verbatim) plus a recount from the per-question rows.
- `*_judged.json(.gz)` — every question of the run: question, gold answer, GPT-5's answer, the judge's verdict, score and reason. The retrieved memories' **text** was removed for size (the originals are 23–170 MB per run); each retrieved memory's id, score and timestamp is kept. Gzipped when over 2 MB.
- `logs/drive_*.log` — the BEAM driver's console output, ending in the harness's score summary. `drive_beam1m_k50_r2.log` is the first leg of that run, which was interrupted; `drive_beam1m_k50_r2_resume.log` resumed it and carries the final score (700 questions, no duplicates).

Every score below was recounted from the per-question rows in the judged file and matches the harness's reported figure exactly. Pass (BEAM) = per-question score >= 0.5.

## Files

| file | bytes | sha256 | backs | run | harness source |
|---|---|---|---|---|---|
| `beam10m_top200_run1_judged.json.gz` | 734594 | `6bb396760d3f88e8af7f57e9d833d1784664922362b30eca84907bc876aa823b` | BEAM 10M, 200 q, top-200 | run 1: 105/200 pass (52.5%), avg 0.482 | `predicted_beam10m_k200/` |
| `beam10m_top200_run1_scores.json` | 2425 | `0d33314d0815a164162c485a428ce45a5afe7534bebf19902ef0998b5d1bd1f5` | BEAM 10M, 200 q, top-200 | run 1: 105/200 pass (52.5%), avg 0.482 | `predicted_beam10m_k200/` |
| `beam10m_top200_run2_judged.json.gz` | 735372 | `40f0ed60741d7842fcdaef7ca5e25e69b055a86c7be95368c4490c142829319d` | BEAM 10M, 200 q, top-200 | run 2: 102/200 pass (51.0%), avg 0.492 | `predicted_beam10m_k200_r2/` |
| `beam10m_top200_run2_scores.json` | 2453 | `80abf63d94876b17f029ab34355e7ea21bc05b320dbaa19ec3efe9604edbd789` | BEAM 10M, 200 q, top-200 | run 2: 102/200 pass (51.0%), avg 0.492 | `predicted_beam10m_k200_r2/` |
| `beam10m_top50_run1_judged.json.gz` | 704627 | `e24d81e1d2b076249faf97fee45a68bb6ac7f06fb17ca2639e15b6527c6e11e2` | BEAM 10M, 200 q, top-50 | run 1: 97/200 pass (48.5%), avg 0.452 | `predicted_beam10m_k50/` |
| `beam10m_top50_run1_scores.json` | 2431 | `9ba75b3f22cc444709cb2d90b7101e2724e9cc27e77215e2b64cf2cf828c3cae` | BEAM 10M, 200 q, top-50 | run 1: 97/200 pass (48.5%), avg 0.452 | `predicted_beam10m_k50/` |
| `beam10m_top50_run2_judged.json.gz` | 703053 | `786fe10637bc06156de8587daf4b0cbdb60c6d50197cf4f266259cc0a66e5ec2` | BEAM 10M, 200 q, top-50 | run 2: 99/200 pass (49.5%), avg 0.443 | `predicted_beam10m_k50_r2/` |
| `beam10m_top50_run2_scores.json` | 2450 | `166490929994a059e0c05d70c7b9bafe30601940525eaf06de8788d4b0c48b1d` | BEAM 10M, 200 q, top-50 | run 2: 99/200 pass (49.5%), avg 0.443 | `predicted_beam10m_k50_r2/` |
| `beam1m_top200_run1_judged.json.gz` | 2514481 | `9ae787cf54511af6669f00da074636a94822b2776f145e3e93c25ce14bd8e87c` | BEAM 1M, 700 q, top-200 | run 1: 504/700 pass (72.0%), avg 0.656 | `predicted_beam1m_k200/` |
| `beam1m_top200_run1_scores.json` | 2682 | `b7a4abc0c8da7ec40a5efce48364f6576ac744f1230a1e4652a206ab621e629c` | BEAM 1M, 700 q, top-200 | run 1: 504/700 pass (72.0%), avg 0.656 | `predicted_beam1m_k200/` |
| `beam1m_top200_run2_judged.json.gz` | 2507916 | `4e911bca3a04531e9773a9fae3ff630d30e38103aac5f2ddbfafa754aa77d064` | BEAM 1M, 700 q, top-200 | run 2: 508/700 pass (72.6%), avg 0.669 | `predicted_beam1m_k200_r2/` |
| `beam1m_top200_run2_scores.json` | 2715 | `d0bb47c7339b384468c0d7496eb10271141a63c30ca53f4f09f2bf689cd8cea6` | BEAM 1M, 700 q, top-200 | run 2: 508/700 pass (72.6%), avg 0.669 | `predicted_beam1m_k200_r2/` |
| `beam1m_top50_run1_judged.json.gz` | 2430318 | `865f01d3ce3f202eb0e5486839a02aece66585733bcf3e2bc35bb10f4a124047` | BEAM 1M, 700 q, top-50 | run 1: 472/700 pass (67.4%), avg 0.625 | `predicted_beam1m_k50/` |
| `beam1m_top50_run1_scores.json` | 2681 | `5748bedba1af4e9f63d7a081b056418c3f18f8a6e631d42cab4e8d59e64cdbf3` | BEAM 1M, 700 q, top-50 | run 1: 472/700 pass (67.4%), avg 0.625 | `predicted_beam1m_k50/` |
| `beam1m_top50_run2_judged.json.gz` | 2418081 | `e8928988f7e774b0d83b4565ad045ebaddd2eb322427e96fb378287f8c2d85be` | BEAM 1M, 700 q, top-50 | run 2: 471/700 pass (67.3%), avg 0.624 | `predicted_beam1m_k50_r2/` |
| `beam1m_top50_run2_scores.json` | 2660 | `63ed67d6c3e5183ce4b08ca68c97c9e33071bb817e0b0dd962ae5dda08bdae12` | BEAM 1M, 700 q, top-50 | run 2: 471/700 pass (67.3%), avg 0.624 | `predicted_beam1m_k50_r2/` |
| `locomo_top100_run1_judged.json.gz` | 1245776 | `0b66ef2e8a2b6ffa11e5fb0b9ab455ff6f431746a1dce31826a50dfe8bcaa049` | LoCoMo, 1,540 q, top-100 | run 1: 1436/1540 (93.2%) | `locomo_results_20261021_050549.json` |
| `locomo_top100_run1_scores.json` | 1201 | `9eec489f84c09424b3db6a92e8be60f4c86ac2fcd00472986e2f2a6fb4f30f4d` | LoCoMo, 1,540 q, top-100 | run 1: 1436/1540 (93.2%) | `locomo_results_20261021_050549.json` |
| `locomo_top50+100_run2_judged.json.gz` | 1356442 | `279e0c01cfecf4bd154c10e28d97dcd8e94ae5a8be684f4b9a6487729d68ad43` | LoCoMo, 1,540 q, top-50 AND top-100 | run 2: top-50 1413/1540 (91.8%); top-100 1425/1540 (92.5%) — one retrieval pass judged at both cutoffs | `locomo_results_20261021_155113.json` |
| `locomo_top50+100_run2_scores.json` | 1909 | `7ba08c98ea94b5dfcbb8c0fe611b5e63e639dcd2af877b70b078a1cf6283cb88` | LoCoMo, 1,540 q, top-50 AND top-100 | run 2: top-50 1413/1540 (91.8%); top-100 1425/1540 (92.5%) — one retrieval pass judged at both cutoffs | `locomo_results_20261021_155113.json` |
| `locomo_top50_run1_judged.json.gz` | 1244388 | `803b7ddb29695b23a83bbc681a2836df48dd8dffc8d8e6082c44f6e49c2d85ff` | LoCoMo, 1,540 q, top-50 | run 1: 1421/1540 (92.3%) | `locomo_results_20261021_024242.json` |
| `locomo_top50_run1_scores.json` | 1167 | `d5341c80bdb7c3d41c7d2283aa9ac7a88deacaafa5a4af67664b2f188c40d6a5` | LoCoMo, 1,540 q, top-50 | run 1: 1421/1540 (92.3%) | `locomo_results_20261021_024242.json` |
| `longmemeval_top50_run1_judged.json.gz` | 342340 | `5cf800486322ab2d9e34fcd0d55783ab7d7a30e0e388038def4ec9ca9052a734` | LongMemEval, 500 q, top-50 | run 1: 467/500 (93.4%) | `longmemeval_results_20261020_234116.json` |
| `longmemeval_top50_run1_scores.json` | 1711 | `10140d308076d661fdac51a7f0d7035ce1d7f79bcaea07bb007acd032b4a271a` | LongMemEval, 500 q, top-50 | run 1: 467/500 (93.4%) | `longmemeval_results_20261020_234116.json` |
| `longmemeval_top50_run2_judged.json.gz` | 343122 | `f03fd8034cb298b5cb541beabd54e8272211bd222c0a8a778643c9c8144a0584` | LongMemEval, 500 q, top-50 | run 2: 472/500 (94.4%) | `longmemeval_results_20261021_003715.json` |
| `longmemeval_top50_run2_scores.json` | 1737 | `95754a2b3de19044e9d64b65ebfc73874e1d73d33715df81de78a0b319d32824` | LongMemEval, 500 q, top-50 | run 2: 472/500 (94.4%) | `longmemeval_results_20261021_003715.json` |
| `longmemeval_top50_run3_judged.json.gz` | 343962 | `2a87fc8794bfb264c8274faf4c32ee93ed757e32d77acfea95e4caf11d9411c7` | LongMemEval, 500 q, top-50 | run 3: 466/500 (93.2%) | `longmemeval_results_20261021_010418.json` |
| `longmemeval_top50_run3_scores.json` | 1687 | `e6ee8393c341449624199ccb30ebdfbe9db3f73caf3918c72a0b46cd008ff621` | LongMemEval, 500 q, top-50 | run 3: 466/500 (93.2%) | `longmemeval_results_20261021_010418.json` |
| `logs/drive_10M_k200.log` | 2880 | `c58431e3c7ebe839df0f99c96de8ab6d76b668f062ac610a84705598e4ebc579` | BEAM 10M top-200, run 1 | driver console log | `logs/drive_10M_k200.log` |
| `logs/drive_10M_k50.log` | 1551 | `94b24174e9662a1aafbceb398acfc886b0568f95c9c2caaf076764b8f5ecec86` | BEAM 10M top-50, run 1 | driver console log | `logs/drive_10M_k50.log` |
| `logs/drive_1M_k200.log` | 4528 | `273c84c71731d4c3694700f1b5ba3b706ca89fba9c1c76cdbeec387673fb6d86` | BEAM 1M top-200, run 1 | driver console log | `logs/drive_1M_k200.log` |
| `logs/drive_1M_k50.log` | 3069 | `dc0e8d42b0c30a6bfc35d37dfa2e4edd07803a65a1df3594dadcf5c1a64d5225` | BEAM 1M top-50, run 1 | driver console log | `logs/drive_1M_k50.log` |
| `logs/drive_beam10m_k200_r2.log` | 2205 | `42c7e439fdaa64e9b2990c65a8b4e6b4ff52defbf7b46d7fa2eed7bf472dc523` | BEAM 10M top-200, run 2 | driver console log | `logs/drive_beam10m_k200_r2.log` |
| `logs/drive_beam10m_k50_r2.log` | 2656 | `b513f60723d3d667fab895108f71ec217632d70d962c569d1b0ffe55a7f083e3` | BEAM 10M top-50, run 2 | driver console log | `logs/drive_beam10m_k50_r2.log` |
| `logs/drive_beam1m_k200_r2.log` | 2701 | `b18fb1f7e1bb6d768dfebc374211e87424b0de5d8da5fb9d0e921408f2ab738c` | BEAM 1M top-200, run 2 | driver console log | `logs/drive_beam1m_k200_r2.log` |
| `logs/drive_beam1m_k50_r2.log` | 450 | `7ebf3f34f8ab7be158727eb913eb495db9b5d97876acc1fcaf1d2c2e3e6a1138` | BEAM 1M top-50, run 2 (first leg, interrupted) | driver console log | `logs/drive_beam1m_k50_r2.log` |
| `logs/drive_beam1m_k50_r2_resume.log` | 2686 | `3506f0bfd648dc546df75a30943014bba96709f8b09bb632dc857e6980cee4fa` | BEAM 1M top-50, run 2 (resumed leg, final score) | driver console log | `logs/drive_beam1m_k50_r2_resume.log` |

## Not included

- The full harness result files and per-question prediction directories with retrieved memory text (about 1.4 GB in total). Available on request.
- No API keys, tokens or credentials appear in any file here (scanned). Email-shaped strings that do appear are synthetic persona text inside the public BEAM dataset itself.