Ch. 9: Schedule maintenance

Some work isn’t triggered by a request; it runs on a clock. In this chapter you add a link-sweeper worker that gives every new link an expiry and deletes the stale ones on a schedule with iii-cron. As with the safety agent in Chapter 8, the sweeper stays decoupled from the link worker: it tracks expiry in its own database by subscribing to link.created, and removes expired links through the link::delete you already wrote. The link worker needs no changes.

Add the workers

iii-cron fires functions on a schedule. The link-sweeper worker is yours, so scaffold it the same way you scaffolded link in Chapter 1:

iii worker add iii-cron
iii worker init link-sweeper --language typescript

Give the sweeper its own database

The sweeper tracks expiry itself, so it needs somewhere to keep it. Add a sweeper database to the database worker’s config, alongside the ones from earlier chapters:

config.yaml

workers:
  # ...
  - name: database
    config:
      databases:
        # ...primary, harness, and safety from earlier chapters
        sweeper:
          url: sqlite:./data/sweeper.db

Build the link-sweeper worker

The worker leverages two other workers: iii-pubsub and iii-cron. First we’ll subscribe to link.created and record an expiry for each new link in the sweeper database. The TTL defaults to 30 seconds so you can watch links expire during the tutorial; in production you’d set SWEEP_TTL_SECONDS to something like a year. Replace the generated link-sweeper/src/index.ts with:

link-sweeper/src/index.ts

import { registerWorker, Logger } from "iii-sdk";

const worker = registerWorker(process.env.III_URL ?? "ws://localhost:49134", {
  workerName: "link-sweeper",
});
const logger = new Logger();
const DB = "sweeper";

// 30 seconds by default so expiry is observable in the tutorial. In production
// set SWEEP_TTL_SECONDS to something like 31536000 (365 days).
const TTL_SECONDS = Number(process.env.SWEEP_TTL_SECONDS ?? "30");

async function ensureSchema(): Promise<void> {
  await worker.trigger({
    function_id: "database::execute",
    payload: {
      db: DB,
      sql: "CREATE TABLE IF NOT EXISTS expirations (code TEXT PRIMARY KEY, expires_at TEXT NOT NULL)",
    },
  });
}

// The sweeper is decoupled from the link worker and records its own data by
// subscribing to the same link.created event everything else does.
worker.registerFunction("link-sweeper::on_link_created", async (data: { code: string }) => {
  const expiresAt = new Date(Date.now() + TTL_SECONDS * 1000).toISOString();
  await worker.trigger({
    function_id: "database::execute",
    payload: {
      db: DB,
      sql: "INSERT INTO expirations (code, expires_at) VALUES (?, ?) ON CONFLICT(code) DO UPDATE SET expires_at = excluded.expires_at",
      params: [data.code, expiresAt],
    },
  });
});

worker.registerTrigger({
  type: "subscribe",
  function_id: "link-sweeper::on_link_created",
  config: { topic: "link.created" },
});

Schedule sweeping

Then it sweeps on a schedule: it queries its own table for expired codes (ie. expires_at < TTL), deletes each from the link worker through link::delete, and removes its local entry. The cron expression has seven fields, starting with seconds and ending with year (0 0 3 * * * * is 03:00 every day, any year):

link-sweeper/src/index.ts

worker.registerFunction("link-sweeper::sweep_expired", async () => {
  const now = new Date().toISOString();
  const { rows } = await worker.trigger<
    { db: string; sql: string; params: string[] },
    { rows: Array<{ code: string }> }
  >({
    function_id: "database::query",
    payload: {
      db: DB,
      sql: "SELECT code FROM expirations WHERE expires_at < ?",
      params: [now],
    },
  });
  let swept = 0;
  for (const { code } of rows) {
    // Only forget the link locally once link::delete actually succeeds; on
    // failure keep the row so the next sweep retries it.
    await worker
      .trigger({ function_id: "link::delete", payload: { code } })
      .then(() =>
        worker.trigger({
          function_id: "database::execute",
          payload: { db: DB, sql: "DELETE FROM expirations WHERE code = ?", params: [code] },
        }),
      )
      .then(() => {
        swept += 1;
      })
      .catch((err) => {
        logger.error("link::delete failed; will retry next sweep", { code, error: String(err) });
      });
  }
  logger.info("swept expired links", { count: swept });
  return { swept };
});

worker.registerTrigger({
  type: "cron",
  function_id: "link-sweeper::sweep_expired",
  // For the tutorial, sweep every 15 seconds so you can watch it fire.
  // In production use "0 0 3 * * * *" (03:00 every day).
  config: { expression: "*/15 * * * * * *" },
});

ensureSchema().then(() => logger.info("link-sweeper ready"));

iii worker add ./link-sweeper

sweep_expired is an ordinary function, so you can run it on demand to test rather than waiting for the schedule to fire.

See it work

Create a link. The sweeper records a 30-second expiry for it behind the scenes, off the link.created event:

iii trigger link::create url=https://example.com code=temp

Run the sweep right away and nothing happens, because the 30 seconds haven’t passed:

iii trigger link-sweeper::sweep_expired

{ "swept": 0 }

Wait 30 seconds (the cron also fires every 15 seconds on its own), then resolve the code again: The link has been swept from the link worker:

iii trigger link::resolve code=temp

{ "url": null }

Conclusion

Linkly now maintains itself: a dedicated link-sweeper worker tracks expiry by subscribing to link.created and deletes stale links on a schedule, all without touching the link worker. That ends the tutorial. You started with a single-worker URL shortener in Chapter 1 and finished with a real system: HTTP, durable storage, an event bus, a queue, a live click stream, bulk uploads over a channel, a browser worker, an autonomous safety agent that investigates links in sandboxes, and scheduled maintenance, all on the same iii bus and all visible in the same console.

​Add the workers

​Give the sweeper its own database

​Build the link-sweeper worker

​Subscribe to link.created events

​Schedule sweeping

​See it work

​Conclusion