vibe-lm
Public
Project Files
benchmark / results / run-remote.log
[2026-07-11T00:11:39.612Z] Benchmarking currently-loaded model: remote:qwen3.6-35b-a3b (modelKey=qwen/qwen3.6-35b-a3b, arch=qwen35moe)
[2026-07-11T00:11:39.613Z] running arith-1...
[2026-07-11T00:12:02.087Z] running arith-2...
[2026-07-11T00:12:22.256Z] running logic-1...
[2026-07-11T00:12:31.459Z] running needle-haystack...
[2026-07-11T00:12:50.172Z] running agentic-file-roundtrip...
[2026-07-11T00:13:08.209Z] Done with remote:qwen3.6-35b-a3b.
[2026-07-11T00:13:55.088Z] Benchmarking currently-loaded model: remote:nemotron-3-nano (modelKey=nvidia/nemotron-3-nano-omni, arch=nemotron_h_moe)
[2026-07-11T00:13:55.089Z] running arith-1...
[2026-07-11T00:13:59.320Z] running arith-2...
[2026-07-11T00:14:04.680Z] running logic-1...
[2026-07-11T00:14:08.036Z] running needle-haystack...
[2026-07-11T00:14:23.815Z] running agentic-file-roundtrip...
[2026-07-11T00:18:37.691Z] agentic error:
┌ Error ──────────────────────────────────────────────────────────────────────┐
│ │
│ Context size has been exceeded. │
│ │
│ │
│ </> STACK TRACE │
│ │
│ at runAgenticPrompt (file:///Users/drunkktoys/Desktop/vibeLM/benchmark/ │
│ run-remote.mjs:108:21) │
│ at main (file:///Users/drunkktoys/Desktop/vibeLM/benchmark/run-remote.m │
│ js:158:28) │
│ at process.processTicksAndRejections │
│ (node:internal/process/task_queues:103:5) │
│ │
└─────────────────────────────────────────────────────────────────────────────┘
[2026-07-11T00:18:37.691Z] Done with remote:nemotron-3-nano.
[2026-07-11T00:20:07.187Z] Benchmarking currently-loaded model: remote:qwen3-coder-30b (modelKey=qwen/qwen3-coder-30b, arch=qwen3moe)
[2026-07-11T00:20:07.188Z] running arith-1...
[2026-07-11T00:20:07.962Z] running arith-2...
[2026-07-11T00:20:08.648Z] running logic-1...
[2026-07-11T00:20:09.227Z] running needle-haystack...
[2026-07-11T00:20:12.599Z] running agentic-file-roundtrip...
[2026-07-11T00:20:17.441Z] Done with remote:qwen3-coder-30b.
[2026-07-11T00:21:11.407Z] Benchmarking currently-loaded model: remote:glm-4.7-flash (modelKey=zai-org/glm-4.7-flash, arch=deepseek2)
[2026-07-11T00:21:11.408Z] running arith-1...
[2026-07-11T00:21:18.429Z] running arith-2...
[2026-07-11T00:21:27.425Z] running logic-1...
[2026-07-11T00:21:32.745Z] running needle-haystack...
[2026-07-11T00:21:43.983Z] running agentic-file-roundtrip...
[2026-07-11T00:21:50.121Z] Done with remote:glm-4.7-flash.
[2026-07-11T00:22:51.759Z] Benchmarking currently-loaded model: remote:gemma-4-26b-a4b (modelKey=google/gemma-4-26b-a4b-qat, arch=gemma4)
[2026-07-11T00:22:51.761Z] running arith-1...
[2026-07-11T00:22:58.376Z] running arith-2...
[2026-07-11T00:23:04.934Z] running logic-1...
[2026-07-11T00:23:10.023Z] running needle-haystack...
[2026-07-11T00:23:17.261Z] running agentic-file-roundtrip...
[2026-07-11T00:23:25.230Z] Done with remote:gemma-4-26b-a4b.
[2026-07-11T00:24:38.836Z] Benchmarking currently-loaded model: remote:gpt-oss-20b (modelKey=openai/gpt-oss-20b, arch=gpt-oss)
[2026-07-11T00:24:38.837Z] running arith-1...
[2026-07-11T00:24:40.259Z] running arith-2...
[2026-07-11T00:24:42.216Z] running logic-1...
[2026-07-11T00:24:43.571Z] running needle-haystack...
[2026-07-11T00:24:46.431Z] running agentic-file-roundtrip...
[2026-07-11T00:24:50.674Z] Done with remote:gpt-oss-20b.
[2026-07-11T00:25:56.106Z] Benchmarking currently-loaded model: remote:qwen3.6-27b (modelKey=qwen/qwen3.6-27b, arch=qwen35)
[2026-07-11T00:25:56.107Z] running arith-1...
[2026-07-11T00:26:30.826Z] running arith-2...
[2026-07-11T00:28:30.829Z] arith-2 error: timeout after 120000ms: arith-2
[2026-07-11T00:28:30.829Z] running logic-1...
[2026-07-11T00:30:05.104Z] running needle-haystack...
[2026-07-11T00:32:27.943Z] running agentic-file-roundtrip...
[2026-07-11T00:34:00.196Z] Done with remote:qwen3.6-27b.