AgentOps replays your agent's run. We measure your agent and your product.
Straight comparison: AgentOps is better than us at step-by-step session replay of Python agent frameworks and at tracking what each run cost in LLM spend. smolanalytics is one MIT Go binary that does product analytics and computed agent observability from the same events, sent from any stack, with no Python SDK required and no metered AI.
smolanalytics vs AgentOps, which one do I want?
They answer different questions, and some teams run both. AgentOps is an agent observability platform built around an MIT-licensed Python SDK: initialize it and you get session replays with step-by-step execution graphs, LLM cost and spend tracking across providers, and native integrations with agent frameworks like CrewAI, AG2 (AutoGen), the OpenAI Agents SDK, Agno, LangChain and LangGraph; its site says it covers 400-plus LLMs and frameworks. If you build Python agents on those frameworks and want to rewind a run call by call or see what it cost, AgentOps is the better tool. Two things to know before you pick it: the SDK is MIT, but the app directory (dashboard plus API backend) ships under the Elastic License 2.0, and self-hosting that app means Supabase, ClickHouse, an OpenTelemetry collector, and separate dashboard and API services via Docker Compose. smolanalytics is a different shape: a single MIT Go binary, all of it MIT, that does web and product analytics (funnels, retention, paths, cohorts, breakdowns, heatmaps, surveys, feature flags and A/B) and computed agent observability off the same event stream, tool-call volume, error rate, latency p50/p90/p99, client split, an error taxonomy, and conversation health like turns, re-ask rate, abandonment and resolution. It takes plain events from any language, not a Python SDK, so it works whatever your agent is written in. Every number is computed from your events, never labelled by a model, and a CI test pins /v1, MCP and the dashboard to the same answer. You ask it questions in plain English from your editor: one MCP connection with an org token operates every project, 79 tools, answered by your own coding agent on your own model, so the AI is never metered by us. Pricing is a 14-day full trial with no card, Pro $49/mo for 1M events, Scale $149/mo for 10M, $8 per extra million, self-host free forever.
what each one actually is
“Python SDK for AI agent monitoring, LLM cost tracking, benchmarking, and more. Integrates with most LLMs and agent frameworks including CrewAI, Agno, OpenAI Agents SDK, Langchain, Autogen, AG2, and CamelAI”
github.com/AgentOps-AI/agentops, repo description; the SDK is MIT
Self-hosting the AgentOps app (dashboard + API) means Supabase, ClickHouse and an OpenTelemetry collector via Docker Compose, and the app directory ships under the Elastic License 2.0, not MIT.
agentops app/README.md and app/LICENSE
smolanalytics is one MIT Go binary and one data file, MIT end to end. Product analytics and agent observability come out of the same events, from any language, with no SDK requirement and no per-run replay store.
what we do instead
AgentOps vs smolanalytics
| AgentOps | smolanalytics | |
|---|---|---|
| The job it is built for | Python agent replay + LLM spend | product analytics + agent health |
| Step-by-step session replay | yes, execution graphs per run | no, we compute rollups |
| LLM cost and spend tracking | per-session, across providers | not offered, we never meter models |
| Framework integrations | CrewAI, AG2, OpenAI Agents SDK, LangGraph | none needed, plain events from any stack |
| Instrumentation | Python SDK (JS/TS in alpha) | events from any language |
| Funnels, retention, paths, cohorts | not its category | yes, same binary, same events |
| Agent numbers | replays, spend, session metrics | computed: p50/p90/p99, error taxonomy, re-ask, resolution |
| Self-host footprint | Supabase + ClickHouse + OTel collector + 2 services | one static binary, one file |
| Dashboard vs API drift | separate read paths | CI pins /v1 == MCP == dashboard |
| License | MIT SDK, Elastic License 2.0 app | MIT, all of it |
when to use AgentOps (often alongside us)
- You need to rewind one agent run step by step. AgentOps gives you session replays with execution graphs of every LLM call, tool call and multi-agent interaction. We compute rollups over your events and do not store per-run replays, so if the question is “what exactly did the crew do on run 4f2a”, that is AgentOps, not us.
- You want to know what each run cost in model spend. LLM cost tracking across providers is a core AgentOps feature. We never see your model calls and do not track LLM spend at all.
- You are all-in on a Python agent framework. If your stack is CrewAI, AG2 (AutoGen), the OpenAI Agents SDK or LangGraph, AgentOps instruments it natively in a couple of lines, and its Basic tier is free up to 5,000 events a month with Pro from $40/mo. Running both is a normal setup: AgentOps for replaying the agent's runs, smolanalytics for the product usage and the computed agent health beside it.
Any stack, one binary
# no Python SDK, no Supabase, no ClickHouse, no collector docker run -p 8080:8080 -v $PWD/data:/data \ -e SMOLANALYTICS_DB=/data/smolanalytics.data \ ghcr.io/arjun0606/smolanalytics # dashboard + MCP at http://localhost:8080 # then, in your editor: "re-ask rate and tool error taxonomy for the last 7 days"
Keep AgentOps for replaying Python agent runs and tracking spend. Put this next to it for funnels, retention and the computed agent numbers, from any stack, on your own box.
questions
Is smolanalytics an AgentOps alternative?
Only partly, and we would rather be straight about it. AgentOps is built around a Python SDK and is better than us at step-by-step session replay of agent runs and at LLM cost tracking, with native integrations for CrewAI, AG2 (AutoGen), the OpenAI Agents SDK and more. smolanalytics is one MIT Go binary that gives you web and product analytics (funnels, retention, paths, cohorts, heatmaps, surveys, flags and A/B) plus computed agent observability from the same events: tool-call volume, error rate, latency p50/p90/p99, client split, an error taxonomy, and conversation health like turns, re-ask rate, abandonment and resolution. If replaying Python agent runs is what you need, use AgentOps. If you want product usage and agent health in one place, from any language, use us. Some teams run both.
Can smolanalytics replay an agent session like AgentOps?
No. We do not store per-run replays or execution graphs, and we are not planning to. Our agent observability is computed rollups over the events your app already sends, which answers “is the agent healthy and where is it failing”, not “show me every step of that one run”. For that second question, AgentOps.
Can I self-host AgentOps, and what license is it under?
You can. The repo says you can run the full app, dashboard plus API backend, on your own machine, and the Enterprise tier lists on-premise deployment. Check the footprint first: the app's own setup guide calls for Supabase, ClickHouse and an OpenTelemetry collector alongside the dashboard and API services via Docker Compose. On license, the Python SDK's LICENSE file is MIT, while the app directory's LICENSE file is the Elastic License 2.0, which is source-available rather than OSI open source and restricts offering it as a hosted service. Read both files yourself rather than taking our word for it. Self-hosting smolanalytics is one MIT Go binary and one data file, and everything is MIT.
Do I need a Python SDK to use smolanalytics with my agent?
No. smolanalytics takes plain events over HTTP from any language, so it works whether your agent is Python, TypeScript, Go, Rust or a shell script wrapping a coding agent. The agent metrics, tool-call counts, error rate, latency percentiles, client split, error taxonomy, turns, re-ask rate, abandonment and resolution, are all computed deterministically from that event stream, never labelled by a model, and a CI test asserts the /v1 API, the MCP answer and the dashboard return the same number for the same question.
What does smolanalytics cost, and can I self-host it?
14-day full trial, no card. Pro is $49/mo for 1M events, Scale is $149/mo for 10M, and extra usage is $8 per million. Self-hosting is free forever under MIT: one Go binary, one data file, no external services. There is no free cloud tier. For comparison, AgentOps' site lists a free Basic tier up to 5,000 events a month and Pro starting at $40/mo.
Get the verdict, not another dashboard.
Drop one snippet, ask in your editor, own your data. 14-day full trial, no credit card.