Index of every page on this site: llms.txt
All skills
bluesky-social avatar

/lex-setup

@d9cdc54
by bluesky-socialbluesky-social/atproto9.7k stars
941
Run it yourself

Wire up, maintain, or troubleshoot `@atproto/lex` code generation and the per-package `lexicons.json` manifest. Use when a package needs generated Lexicon schemas; when adding or editing a `lexicons.json`, its `resolvers` (`directory` / `http` / `repo`) or a `lexicons-overrides` directory; when migrating a package off compiling the repo-root `lexicons` directory; when adding a `codegen:lex` / `lex:install` / `prebuild` script; when `./src/lexicons/` or `./lexicons/` is missing, stale, or gitignored wrong; after editing anything under `lexicons/`; when adding an NSID; when `pnpm codegen`, `lex build`, or `lex install` fails; when `lex` resolves to the wrong binary; when a post-lexicon-edit build reports "type X is not assignable to Y"; or when ripping out legacy `@atproto/lex-cli` codegen. Reach for it before hand-editing anything under a `lexicons/`, `lexicons-overrides/`, or `src/lexicons/` directory, or any `lexicons.json`.

  • 1 file
  • 17.4 KB
  • Updated 3 weeks ago
  • GitHub
Use this Skill: https://skilld.dev/gh/bluesky-social/atproto/lex-setup

Nothing lands on disk. Nothing to clean up.

Fork this Skill

Edit a local copy. It keeps the author and licence.

SKILL.md

≈238 tokens for metadata: the name and description. ≈4.2k when used: this file.

Description uses 11.9% of example budget

Before choosing a Skill, your Agent reads its name and description. All available Skills share that space.

  • A shorter description leaves more room for other Skills. This entry exceeds our 1% size suggestion.

In our Claude Code example, all Skill names and descriptions share 8,000 characters. This Skill uses ≈951 characters, or 11.9%.

The 1% threshold is a size suggestion. Longer descriptions can still fit.

Your model, settings, and other Skills decide how much text your Agent can read.

Example settings and source

The example uses a 200k-token context and default Claude Code settings. The count includes the name, description, separators, and when_to_use when present. Codex also counts local file paths.

Skit's source and limits: Codex 0.160.1, Claude Code 2.1.292.

@atproto/lex codegen setup

Two commands do everything:

  • lex build — compiles Lexicon JSON into a TypeScript schema tree. Every package that uses @atproto/lex runs this.
  • lex install — resolves the Lexicons a package's lexicons.json lists into ./lexicons/, recording each one's source + CID, from the manifest's resolvers and the network.

Both ship in the lex bin of @atproto/lex, which re-exports @atproto/lex-client, -schema, -data, -json, and -cbor, but also depends on the codegen tooling the bin needs. Services must not ship that tooling, so packages in this monorepo list @atproto/lex as a devDependency only: production code imports each name from the sub-package that re-exports it, and depends on that package (eslint rejects @atproto/lex imports under packages/*/src). lex build takes --lib @atproto/lex-schema for the same reason, as generated files import l from that package.

Wiring a package in this monorepo

lexicons/ at the repo root is the canonical source for the com.atproto and tools.ozone namespaces only. Every package manages its Lexicons through its own committed lexicons.json manifest and resolves them with lex install — no package compiles the root directory directly. The manifest's resolvers decide where each NSID comes from; see Manifest resolvers.

Copy the shape consuming packages use (packages/pds, packages/sync, packages/bsky):

{
  "scripts": {
    // Generate TypeScript schema from Lexicon JSON
    "lex:build": "lex build --clear --indexFile --lib @atproto/lex-schema",
    // Install Lexicon JSON into ./lexicons/ and update the manifest. Use `pnpm lex:install foo.bar.baz` to install new lexicons
    "lex:install": "lex install",
    // Keep the TypeScript schema up to date after installing new lexicons
    "postlex:install": "pnpm run lex:build",
    // Allows to keep the manifest.json up-to-date when a canonical lexicon is updated. Also allows to fetch new lexicons from the network.
    "lex:update": "lex update",
    // Run the build step after updating the lexicons
    "postlex:update": "pnpm run lex:build",
    // When performing a regular build (eg. from CI), ensure that the lexicons are installed and match the manifest.
    "codegen:lex": "pnpm run lex:install --ci",
    "prebuild": "pnpm run '/^(codegen:.+)$/'",
    "build": "tsc --build tsconfig.build.json"
  }
}

Then gitignore the build artifacts. The generated TypeScript (src/lexicons) is never committed, and the canonical schemas symlinked into ./lexicons from the repo root (com.atproto / tools.ozone) are gitignored too — they can't drift and are recreated on every install. Everything else resolved into ./lexicons is committed: the at:// network copies and the app.bsky.* / chat.bsky.* documents fetched from the bsky GitHub repo, so builds and CI work offline (the network can't serve old record versions). The shared repo-root lexicons-overrides/ is committed too. See packages/bsky/.gitignore for the shape:

# @atproto/lex
/src/lexicons
# Canonical schemas are symlinked from the repo-wide source; everything else
# under ./lexicons is a committed copy kept for offline builds and CI.
/lexicons/com/atproto
/lexicons/tools/ozone

A package whose Lexicons are all symlinked canonical (e.g. packages/lex/lex-password-session) has nothing to commit under ./lexicons and can gitignore the whole /lexicons tree instead.

The indirection through codegen:lex is deliberate:

  • prebuild runs every codegen:* script the package declares, so a package that also generates protobuf or templates needs no change to this line. enable-pre-post-scripts = true in .npmrc is what makes a build fire prebuild at all.
  • Root pnpm codegen runs build:tooling and then every codegen:* script across the workspace in parallel. A generator hidden inside prebuild would be skipped by it.

Which lex binary you get

@atproto/lex-cli (legacy) and @atproto/lex both claim the lex bin name, and resolution is per-package. In packages/api — which still depends on lex-cli for gen-api — pnpm exec lex gives you the legacy CLI, not lex build. api therefore installs its manifest with the lex-installer bin (@atproto/lex-installer) and feeds the installed ./lexicons to gen-api. Everywhere else lex is @atproto/lex.

gen-api is slow, so api's lex:gen is mtime-guarded: it only re-runs when ./src/client/index.ts is missing or some lexicon content is newer than it. Use find -L ./lexicons -type f (not the symlinks themselves) — lex install re-creates the canonical symlinks and rewrites lexicons.json on every run, so their mtimes are meaningless; -L follows each symlink to its (stable) target, and the regular files are the committed/fetched copies.

If a lex build invocation errors about an unknown command, that collision is why. @atproto/lex also registers ts-lex, which is unambiguous. Outside a workspace, npx @atproto/lex avoids pulling the unrelated unscoped lex package that bare npx lex would fetch.

Manifest resolvers (lexicons.json)

The manifest's lexicons array lists the NSID entry points the package needs; lex install pulls in schemas they reference transitively, so you only list the roots. resolvers is an ordered list deciding where each NSID is fetched from — earlier entries win, with the AT Protocol network as the implicit last fallback. The kinds:

  • { "type": "directory", "path": "../../lexicons", "include": ["com.atproto.*", "tools.ozone.*"] } — resolve from a local directory. The canonical set lives at the repo root; matches are symlinked into ./lexicons and locked with a file:// URI, so they can't drift. Adjust path for the package's depth.
  • { "type": "http", "base": "https://raw.githubusercontent.com/bluesky-social/bsky/refs/heads/main/lexicons/", "include": ["app.bsky.*", "chat.bsky.*"] } — fetch <base>/<…nsid.segments>.json (e.g. a raw Git host). http falls through on 404 but throws on other errors.
  • { "type": "repo", "repo": "<handle|did>" } — resolve each NSID from that repo's com.atproto.lexicon.schema records; never falls through.
  • A directory resolver pointing at the shared, committed repo-root ../../lexicons-overrides directory, listed after the canonical resolver but before the http/network ones, so it overrides the lexicons they would otherwise return (it cannot override canonical, which resolves first). Put any document that must differ here: unpublished NSIDs (internal.*, place.stream.badge.defs, a not-yet-published app.bsky.unspecced.*), documents whose local version is ahead of the published one, and minimal compatibility relaxations of published third-party lexicons that network records violate (see Compatibility overrides; the PDS must exclude them so writes keep validating against the published schema). All packages share the one lexicons-overrides/ at the repo root.

packages/bsky/lexicons.json is the full reference (overrides + canonical directory + GitHub http).

include/exclude globs accept * as a wildcard; exclude is applied after include.

Packages under packages/lex/

@atproto/lex depends on the SDK sub-packages, so they cannot depend back on it (it would be circular). They generate with @atproto/lex-builder's lex-builder build --lib @atproto/lex-schema (that is where the generated files import l from) instead of lex build, and install their manifest with @atproto/lex-installer's lex-installer install instead of lex install:

"lex:build": "lex-builder build --out ./src/lexicons --clear --lib @atproto/lex-schema --index-file",
"lex:install": "lex-installer install",
"postlex:install": "pnpm run lex:build",
"codegen:lex": "pnpm run lex:install --ci",

See packages/lex/lex-password-session. Add --default-export to lex:build for a browser-facing client (packages/lex/lex-client).

lex build flags

Defaults: --lexicons ./lexicons, --out ./src/lexicons, --no-pretty, --default-export. yargs accepts either kebab-case or camelCase (--index-file == --indexFile).

  • --clear — wipe the output dir first, so deleted lexicons don't linger
  • --override — overwrite existing files (no-op alongside --clear)
  • --indexFile — emit index.ts re-exporting root namespaces (app, com, …)
  • --include / --exclude <patterns…> — filter by NSID
  • --lib <package> — where generated files import l from
  • --importExt <ext> — import specifier extension (.js; "" for extensionless)
  • --fileExt <ext> — emitted file extension (.ts)
  • --no-pretty — skip prettier
  • --no-defaultExport — drop default re-exports (see Tree-shaking)
  • --defsExport — also expose defs under $defs, for when a child namespace shadows a sibling definition (com.example.foo vs com.example.foo.bar)
  • --ignore-errors / --ignore-invalid-lexicons — skip bad inputs instead of failing

Using the generated tree

--indexFile gives one entry point for all root namespaces; schemas are addressed by NSID dot-path:

import { app, com, chat } from '../lexicons/index.js'

app.bsky.feed.post // record schema
app.bsky.feed.defs.postView // object def
com.atproto.repo.getRecord // query/procedure schema

Each NSID compiles to a <name>.defs.ts holding the schema plus a thin <name>.ts re-exporting it (and main as default). Nothing under src/lexicons/ is hand-editable — it is deleted and rewritten on every --clear build.

Tree-shaking

The generated tree is tree-shakeable, but the reference style decides how much survives. A bundler can't tell that client.call() only consumes .main, so naming the namespace retains every sibling def inside it:

// Smallest — default import; preferred for browser bundles
import getRecord from '../lexicons/com/atproto/repo/getRecord.js'
await client.call(getRecord, {})

// Same size, but leaks the `main` identifier into your source
import { main as getRecord } from '../lexicons/com/atproto/repo/getRecord.js'
await client.call(com.atproto.repo.getRecord.main, {})

// Largest — namespace notation drags in sibling defs
await client.call(com.atproto.repo.getRecord, {})

Services, scripts, and tests here use namespace notation, which reads closest to the NSID and costs nothing server-side. Libraries and browser apps should prefer the default import.

After editing lexicon JSON

Generated TypeScript is not rebuilt automatically. Regenerate before building or testing anything downstream — a stale tree is by far the most common cause of "type X is not assignable to Y" right after a lexicon edit:

pnpm codegen        # repo root: every codegen:* script in the workspace
pnpm run prebuild   # inside one package

pnpm codegen builds @atproto/lex-builder first via build:tooling. If codegen output still looks stale, that build is the thing to re-run.

Adding a new external NSID means listing it in the package's lexicons.json lexicons array (or pnpm lex:install <nsid>), then re-running codegen. A new canonical (com.atproto.* / tools.ozone.*) schema is added at the repo root and picked up by each dependent package's next lex:install; run pnpm lex:update-local from the root to propagate a changed canonical CID into every manifest.

Network-installed schemas (outside this repo)

For a package whose Lexicons are not maintained alongside it, lex install fetches them from the network into ./lexicons/ and records each resolution (AT URI + CID) in ./lexicons.json.

lex install app.bsky.feed.post   # add NSIDs; dependencies resolve automatically
lex install                      # install everything the manifest lists
lex install --ci                 # fail if installed files drift from manifest CIDs
lex install --no-save <nsid>     # install without touching the manifest
lex update                       # re-resolve all to latest
lex update 'chat.bsky.*'         # re-resolve only matching NSIDs

Flags: --manifest <path> (default ./lexicons.json), --lexicons <dir> (default ./lexicons), --ci, --no-save. lex install --update still works but is deprecated in favor of lex update.

Commit both the manifest and the fetched JSON — they are the input contract, and the recorded CIDs are what makes --ci meaningful. The upstream-recommended wiring adds "postinstall": "lex install --ci" to catch drift on every install and "update-lexicons": "lex update" as the deliberate refresh.

Don't commit a second copy of schemas that already live in lexicons/ — it would silently diverge. In-repo, vendor by symlink instead (next section), which can't drift.

Vendoring the canonical lexicons (in-repo)

Consuming lexicons/ through lex install (rather than compiling it directly) is the standard setup — it makes a package's lexicon set an explicit, CID-locked contract. A directory resolver resolves each canonical NSID from the root; those resolutions are symlinked into ./lexicons/ (not copied) and locked with a file:// URI relative to the manifest, so they can't diverge. packages/pds is the reference:

// lexicons.json (committed) — roots + resolvers + CID locks
{
  "version": 2,
  "lexicons": ["com.atproto.repo.getRecord", "…every vendored NSID…"],
  "resolvers": [
    {
      "type": "directory",
      "path": "../../lexicons",
      "include": ["com.atproto.*", "tools.ozone.*"],
    },
  ],
  "resolutions": {
    "com.atproto.repo.getRecord": {
      "uri": "file://../../lexicons/com/atproto/repo/getRecord.json",
      "cid": "…",
    },
  },
}
// package.json — install (creating symlinks + verifying the lock), then build
"codegen:lex": "pnpm run lex:install --ci",
"lex:install": "lex install",     // regenerate the manifest after adding/removing a lexicon
"postlex:install": "pnpm run lex:build",
"lex:build": "lex build --clear --indexFile --lib @atproto/lex-schema",

Gitignore the symlinked canonical dirs under /lexicons (com/atproto, tools/ozone — build artifacts) but commit lexicons.json (the contract), along with any network- or GitHub-resolved copies. Generate/refresh the manifest with pnpm run lex:install; --ci then fails the build if /lexicons drifts from the locked CIDs. Tradeoff: the lexicons roots list is explicit, so adding a new canonical schema requires re-running lex:install to include it — unlike a plain lex build --lexicons ../../lexicons, which picks up every file implicitly. For the full resolver vocabulary (directory / http / repo, ordering, and the lexicons-overrides directory) see Manifest resolvers above.

Removing legacy @atproto/lex-cli codegen

The legacy output directory depends on which generator ran — lex gen-server emits ./src/lexicon (singular), lex gen-api emits ./src/client. Both are fully generated and safe to delete. Leave lexicons/ alone; it is input, not output.

Then drop @atproto/lex-cli from devDependencies, add a lexicons.json (resolving the canonical set from the root plus whatever external namespaces the package used), replace the old codegen:lex with the lex:install / lex:build form above, and repoint .gitignore at src/lexicons plus the symlinked canonical dirs under /lexicons. (packages/api is the exception — it keeps gen-api but now feeds it a locally-installed ./lexicons; see Which lex binary you get.)

Related skills

Once the package generates code: lex-schema ($-accessors, validation), lex-data (values, blobs, CBOR, branded strings), lex-client (calling out), xrpc-server (defining routes), and lexification-client / lexification-server (migrating off the legacy stack).

Source: SKILL.md on GitHub

skilld matched fixed text patterns in SKILL.md and file names. Patterns miss obfuscated code.

skilld run checks every file with the same patterns. It asks for approval before it loads a Skill with a behavior marked Needs approval.

No third-party reports yet.

Signed by skilld at d9cdc54. This ties the file your Agent reads to that commit on GitHub. It does not review the instructions.

Last checked against GitHub yesterday.

Activeupdated 3 weeks ago
disable-model-invocation
false

README badge

README badge for bluesky-social/atproto/lex-setup