Developer Documentation

Backend

Python / Flask / gunicorn

Frontend

Next.js 16.2.12 / React 19 / TypeScript

Database

Redis (in-memory store + work streams)

Library DB

SQLite (public-domain book corpus)

Search

Apache Solr (full-text)

AI Inference

Ollama — local model + cloud escalation

Speech

Kokoro neural TTS — local synthesis, MP3 output

Metrics

Prometheus + Grafana (corpus and pipeline telemetry)

Auth

Auth.js v5 beta — Google + GitHub OAuth, JWT sessions

Proxy

Caddy (automatic TLS via Let's Encrypt)

Process Mgr

arc.sh + systemd (auto-starts on boot)

The stack is managed by a single control script and auto-starts on boot. A watchdog supervises the services at runtime and restarts any that crash, distinguishing a deliberately-stopped service from a failed one.

What runs: RSS ingestion and the full A.R.C. analysis pipeline (the Scribe); on-demand and manual publishing of user submissions; background analysis workers; automated posting to Bluesky, Mastodon, and Facebook (each toggleable at runtime with no restart); an email digest; and the Next.js frontend.

Posting is fail-safe, not fire-and-forget: the posters track which articles they have published separately, so a mid-publish failure re-tries on the next cycle rather than double-posting or silently dropping.

A Caddy reverse proxy terminates TLS and routes requests: authentication and user-preference calls are handled by the Next.js application server, and the remaining API traffic goes to the Flask backend. Everything else renders from Next.js. Automatic certificate management is handled by Caddy.

Arc Codex exposes a read-only public HTTP API over the same domain. It serves the article feed, individual articles with their full analysis, full-text search, an RSS feed, the wiki directive pages, the public-domain library, and the machine-readable sitemap. Signed-in users can additionally submit content for processing and manage their own preferences.

Machine-readable discovery surfaces — /rss.xml, /sitemap.xml,/news-sitemap.xml, and /opensearch.xml — are published and kept current automatically.

Live application data is held in Redis for speed; the public-domain book corpus behind the Library lives in SQLite; and full-text search is served from Solr. At a conceptual level the system stores:

Soft auth — the site is fully public. Signing in with Google or GitHub is optional and unlocks preferences, publishing, and private articles. There is no username/password fallback and no third-party tracking.

Sessions are JWT-based (stateless), and preference writes are accepted only from the application server itself — never directly from the public internet — so a user can only ever change their own settings.

Inference is tiered and demand-gated: a compact local model handles the bulk of the work, and a larger cloud model is reached only on escalation, within a weekly budget. The Red / Blue / Purple analyses are computed lazily — on an article’s first view rather than at ingest — so inference cost tracks readership, not ingest volume. Published articles are retained for roughly a month before they are pruned. Translation degrades gracefully when a model is unavailable: “model unavailable” is shown rather than a hard failure.

Translation is a click, not an auto-fire in the feed.A scrolled feed holds many mounted cards; firing translation on each mount would overwhelm the inference tier. A preferred language is a shortcut that skips the picker — it does not translate the whole feed automatically.

Every published article is also spoken. A neural text-to-speech model (Kokoro) renders the article body to audio on the same hardware that runs the rest of the pipeline — there is no cloud speech service, no per-character billing, and no third party receives the text. Long pieces are split into chunks, synthesised in sequence, then concatenated and encoded to a compact mono MP3 sized for slow connections rather than for fidelity.

Narration is opportunistic rather than blocking. Publishing never waits on audio: a pass runs each cycle, picks the newest article still lacking a recording, and defers if the machine is busy. A deferred article is simply retried next time round. Recent narrations are also concatenated into a rolling bulletin — a single continuous audio stream of the day’s reporting, intended for listeners who want the news without a screen.

Synthesis yields to analysis. Speech generation is memory-hungry, so a pre-flight check confirms there is genuine headroom before a run starts. If there is not, narration steps aside rather than competing with the analysis pipeline for the same machine. Audio is the part of the system that can afford to be late.

The pipeline is instrumented rather than trusted. Metrics are scraped continuously and rendered as dashboards covering ingest rate, analysis latency, inference tiering, and corpus-level qualities — the average reading difficulty and objectivity of what has actually been published, not merely how much of it there is.

Alerting distinguishes liveness from output. A worker publishes a heartbeat on a short expiry, so its silence is itself the signal; that is a separate question from whether the day produced many articles or few. Conflating the two produces an alarm that fires on every quiet afternoon and is therefore ignored when it matters.

An alert that cannot clear is not an alert.Conditions are edge-triggered and paired with an explicit all-clear, so a fault that resolves itself says so. Without that, a recovered incident and an ongoing one look identical from the outside.

• Feed rendering

  The lazy-loading feed structure is load-bearing — changes are surgical, never structural.

• Theme layer

  A single stylesheet layer is the source of truth for colours and overrides everything else.

• Preferences

  One context is the single source of truth for user preferences across the app.

• App Router

  Next.js 16 App Router with Turbopack. Not the pages router.

• No ads

  Fully ad-free by design. No ad networks, no analytics beacons.

Full-text search is served by Apache Solr, indexed over the article corpus (title, content, source, directive, and the reading-difficulty score). Search reconnects lazily so a restart of either the search engine or the application resolves itself without manual intervention.