Skip to content

Runbook — persistent Claude↔Telegram agent cutover (Phase 4)

Takes the scaffolded listener + sender + dispatcher live, replacing the current session-bound plugin pollers. Design + rationale: docs/ai/research/2026-06-17-persistent-claude-telegram-agent.md.

⚠️ Blast radius — read first

After cutover the listener is the sole getUpdates consumer of the bot token. All Telegram I/O (José, Cristina, Vigia lanes) then flows through notification_db.telegram_messages. Stopping the current pollers takes the live Cristina↔Vigia channel offline until the new pipeline is verified up. Do this: - in a low-traffic window, - with Vigia coordinated (its session's plugin poller must stop too), - after the dry run on a test bot (Step 2) has passed, - with the rollback (bottom) staged and understood (~1 min back to status quo).


Decision gates — resolve ALL before touching anything

  • G1 · Bot-token inventory. Confirm how many bot tokens are actually in play and which one the listener will own. List live consumers: pgrep -af "server.ts" (expect ~5 bun server.ts plugin pollers). The listener takes over the token those share. If José / Cristina-via-Vigia are on different tokens, decide whether to consolidate onto one (the lane-fan-out design) or run a listener per token.
  • G2 · Dispatcher placement (locked candidate: VPS). Either:
  • VPS — postgres is local on po-shared-network, but Claude Code must be installed + authenticated there; or
  • laptop — Claude Code is present, but needs a tunnel to po-postgres for the dispatcher's DATABASE_URL.
  • G3 · claude -p flags. Verify src/dispatch.js's invocation against the installed Claude Code version (--allowedTools, --output-format text, stdin prompt). Do a manual dry run (Step 2.3).
  • G4 · Vigia coordinated. Agreed window; Vigia knows its plugin poller stops and that its lane's messages will arrive via the DB/dispatcher afterward.

Step 0 — capture current state (for rollback)

# On each host running a poller (laptop + wherever Vigia runs):
pgrep -af "server.ts" | tee /tmp/telegram-pollers-precutover.txt
# Note which Claude sessions own them so you can restart them on rollback.
Nothing to back up in the DB (tables are empty / unused pre-cutover).


Step 1 — apply migrations to prod (safe; additive, no traffic impact)

make db-migrate-prod        # applies notification/20260617160807_telegram_messages.sql
                            #     + notification/20260617182239_telegram_messages_attempts.sql
# verify on notification_prod:
#   \d telegram_messages   -> table + attempts column + 3 indexes present
These are forward-only and idempotent. The table is unused until the listener runs, so applying early is harmless and can be done ahead of the window.


Step 2 — DRY RUN on a throwaway test bot (de-risk; does NOT touch production)

Validate the whole loop without going near the production token or Cristina↔Vigia.

  1. Create a separate test bot via @BotFather; get its token + your own test chat_id.
  2. On the VPS, write .env.telegram with the test token and TELEGRAM_LANE_OPERATOR=<your test chat_id>, DATABASE_URL → notification_prod. Build + start:
    docker compose -f infrastructure/compose/telegram-listener.yml --env-file .env.telegram up -d --build
    docker logs -f po-telegram-listener po-telegram-sender   # expect "online as @<testbot>; draining backlog"
    
  3. On the dispatcher host, write .env.dispatch and dry-run the agent once by hand to confirm G3 flags:
    echo "É o assistente. Responda 'ok' em português." | claude -p --model claude-sonnet-4-6 --output-format text --allowedTools ""
    
    Then install the timer: sudo bash infrastructure/scripts/install-telegram-dispatch.sh
  4. DM the test bot: confirm ack → row direction='in' → dispatcher writes 'out' → sender delivers the reply. (Verification SQL in the Appendix.)
  5. Tear down the test: docker compose -f infrastructure/compose/telegram-listener.yml down
  6. sudo systemctl disable --now telegram-dispatch.timer. Delete the test bot.

Only proceed to Step 3 if the dry run delivered a real reply end-to-end.


Step 3 — THE FLIP (the brief outage window; do with Vigia present)

Order matters — Telegram allows one getUpdates consumer per token, so all old pollers must stop before the listener starts, or it 409s.

# 3a. STOP every current poller (laptop + Vigia's host). This is the outage start.
#     Stop the Claude sessions running the telegram plugin, then CONFIRM none remain:
pgrep -af "server.ts"            # must return NOTHING before continuing

# 3b. Point .env.telegram at the PRODUCTION token + real lane chat_ids:
#       TELEGRAM_BOT_TOKEN=<the shared production token>
#       TELEGRAM_LANE_SA=<Cristina chat_id>
#       TELEGRAM_LANE_OPERATOR=<José chat_id>
#       TELEGRAM_LANE_VIGIA=<Vigia-side contact chat_id(s)>
#     (chmod 600; never commit.) Then start the always-on plane:
docker compose -f infrastructure/compose/telegram-listener.yml --env-file .env.telegram up -d --build
docker logs --since 1m po-telegram-listener   # "online as @<prodbot>; N chat(s) mapped; draining backlog" — NO 409

# 3c. Start the dispatcher timer on the chosen host (if not already from a prior run):
sudo bash infrastructure/scripts/install-telegram-dispatch.sh
systemctl list-timers telegram-dispatch.timer --no-pager

Step 4 — end-to-end smoke (per lane)

For each lane (Cristina = sa, José = operator, a Vigia contact = vigia): send a real DM and confirm the full round-trip. Outage ends when the sa + vigia lanes both answer.

# After a test DM, watch it flow (run against notification_prod):
psql "$DATABASE_URL" -c "SELECT id,direction,lane,left(body,40) AS body,
  to_char(created_at,'HH24:MI:SS') t, processed_at IS NOT NULL done
  FROM telegram_messages ORDER BY created_at DESC LIMIT 8;"
Expect, per inbound: an instant ack in Telegram → one direction='in' row (correct lane, done=t once dispatched) → one direction='out' row (done=t once sent) → the substantive reply arriving in Telegram. Also send from an unknown chat_id and confirm it is stored lane='unrouted' with no reply.


Step 5 — post-cutover wiring

  • Session Telegram plugins stay OFF. With the token owned by the listener, no Claude session should run the telegram plugin (it would 409). Any session that wants to send to Telegram now writes a direction='out' row (the sender delivers) instead of calling the plugin reply tool.
  • Autonomous José pings (e.g. the Cristina↔Claude comms-poll summary) must be rewired to write an 'out' row in the operator lane rather than using the plugin reply tool. Track as a follow-up; until then, ping José out-of-band.
  • The PO-misc Riff comms-poll is unaffected — it reads the task board (cc_prod), not Telegram. Keep it.
  • Update docs/ai/sessions/active.md + the research doc Phase 4 status to "live".

Rollback (if any smoke step fails) — ~1 minute

# 1. Free the token: stop the new always-on plane.
docker compose -f infrastructure/compose/telegram-listener.yml down
sudo systemctl disable --now telegram-dispatch.timer

# 2. Restart the previous pollers — re-open the Claude sessions that ran the
#    telegram plugin (per /tmp/telegram-pollers-precutover.txt). Cristina↔Vigia
#    is back as soon as Vigia's poller is up again.
DB rows are harmless — leave them; they replay no traffic once the listener is down. Diagnose from docker logs po-telegram-listener / journalctl -u telegram-dispatch before re-attempting.


Appendix — verification SQL

-- pending inbound (dispatcher backlog)
SELECT count(*) FROM telegram_messages WHERE direction='in' AND processed_at IS NULL AND lane<>'unrouted';
-- pending outbound (sender backlog)
SELECT count(*) FROM telegram_messages WHERE direction='out' AND processed_at IS NULL;
-- unrouted (needs operator allowlisting)
SELECT id,chat_id,left(body,60) FROM telegram_messages WHERE lane='unrouted' ORDER BY created_at DESC;
-- dead-lettered outbound (hit the retry cap)
SELECT id,chat_id,attempts,error_message FROM telegram_messages
  WHERE direction='out' AND attempts>=5 AND error_message IS NOT NULL;