SCItation: structured agentic development on a problem that actually mattered
How building an open-source citation pattern auditor became a forcing function for learning structured agentic development with Claude Code — and why it worked.
The hardest part of learning agentic workflows is finding the right problem. A todo list or a script wrapper doesn't reveal anything — when the requirements are trivial, the workflow is trivial too. The discipline shows up when the domain pushes back: ambiguous requirements, multi-step data dependencies, acceptance criteria that can't be hand-waved. SCItation — an open-source tool for auditing citation patterns in academic research networks — was chosen as a forcing function precisely because it wasn't trivial.
The problem
Citation manipulation is harder to prove than it looks. High self-citation rates alone mean very little. Every researcher cites their own prior work; baseline rates vary by field, career stage, and publication venue. A rate of 20% could be entirely normal in one context and a red flag in another.
What actually matters is a subtler signal: the normalisation effect. As a paper accumulates external citations over time, the self-citation share should decline — other researchers are citing it, so the author's own citations become a smaller fraction of the total. When that decline doesn't happen, when the self-citation share stays elevated regardless of how many external citations the paper receives, that's a signal worth examining.
The second signal is network-level. A tightly connected group of authors citing each other at rates that diverge significantly from comparable researchers outside the network compounds the first signal. Neither pattern alone is conclusive. Together, they form the basis for a structured review.
The methodology
SCItation uses OpenAlex as its data source — free, CC0-licensed, no API key required. It covers the majority of published academic literature, though it's important to note that OpenAlex covers less than Google Scholar. All rates the tool produces are lower bounds. A finding is a flag for review, not a verdict.
From an author search, the tool builds a co-author network and selects a control group: comparable researchers in the same field and career stage, outside the audited network. This baseline is what makes the statistics meaningful.
The core analysis uses two tests. Spearman correlation measures whether a paper's self-citation share declines as its total citation count grows. Spearman (rather than Pearson) because the expected relationship is monotonic — self-citation share should fall as external citations accumulate — not necessarily linear. A positive or flat correlation where a negative one is expected flags the normalisation effect failure.
Mann-Whitney U compares the distribution of self-citation rates between the audited group and the control group. It's non-parametric, which matters: citation data is typically skewed, and assuming a normal distribution would produce unreliable results.
For specific flagged articles, a citation target analysis goes further: it measures the internal citation share (what fraction of citing papers come from within the author's own network) and the topical relevance of those citing papers. A paper cited heavily by researchers in unrelated fields, from within a tightly connected network, is a different kind of signal than one cited for its content.
The agentic workflow
Structured agentic development means something specific: each task has a defined input, a defined output, and acceptance criteria that can be checked before the next task begins. It sounds obvious. It isn't, until you try to do it.
The discipline that SCItation forced was in the decomposition. You can't hand-wave requirements when an agent is going to implement them literally. Ambiguity that a human developer resolves by asking a question, an agent resolves by making an assumption — and assumptions compound across a pipeline. Writing the spec before dispatch, committing it to git, and treating the committed prompt as the source of truth is what prevents the compounding.
Claude Code handled implementation throughout: working from committed specs, on isolated branches, with human sign-off before merge. The workflow is the same pattern described in Building this site with a governance toolkit — SCItation was a larger, harder test of the same approach.
What the real complexity of the problem exposed: the statistical pipeline, the network data model, the report generation, the optional LLM narrative layer — each of these had enough moving parts that shortcut prompts surfaced immediately as broken outputs. The problem's resistance to shortcuts was exactly what made it a useful learning environment.
The tool
SCItation ships with:
- Author search via OpenAlex
- Co-author network builder with matched control group
- Spearman correlation and Mann-Whitney U analysis
- Citation target analysis for flagged articles
- PDF and DOCX report generation with charts
- Optional LLM narrative via Ollama (qwen2.5:7b or compatible)
Everything runs locally. No data leaves your machine. Sessions are saved as reproducible JSON files — shareable, re-runnable audit records.
It's MIT-licensed and available at github.com/Witchd0ct0r/SCItation.
Why this
Real problems make better learning environments because they resist shortcuts. The methodology had to be right because the domain demanded it — a wrong statistical test produces a meaningless result, and a meaningless result is worse than no tool at all. The workflow had to be disciplined because the methodology was complex enough that undisciplined prompting produced exactly that.
Both outputs — a working tool and a repeatable development practice — were the point.