Index of every page on this site: llms.txt
All skills
neondatabase avatar

/neon-postgres

@d3601d9 official
Run it yourself

Guides and best practices for working with Lakebase Postgres on Neon: connections, pooled vs direct, schema migrations, branching (including schema-only, reset-from-parent, and branch-per-PR workflows), autoscaling, scale-to-zero, instant restore, read replicas, IP allow lists, logical replication, egress optimization, and Lakebase Search. Use when the work is an existing DATABASE_URL, SQL, schema, inspect, or search. New backends, Auth, files, Functions, and LLM calls go to the parent `neon` skill. Live queries, reactive UIs, and realtime data go to `neon-realtime`. Also use for "@neondatabase/serverless", "@neon/neon-js", "neon inspect db", "neon postgres computes", "Neon branch", "schema-only branch", "reset branch", "branch per PR", high database bills, network transfer or egress, Free plan limits or a compute suspended over quota (SQLSTATE 53000, HTTP 402), SELECT * optimization, query overfetching, "semantic search", "vector search", "full-text search", "BM25", or "hybrid search".

Use this Skill: https://skilld.dev/gh/neondatabase/agent-skills/neon-postgres

Nothing lands on disk. Nothing to clean up.

Fork this Skill

Edit a local copy. It keeps the author and licence.

referencesegress-optimization.md

≈2.4k tokens on demand. Your agent reads this file only when SKILL.md points to it.

Postgres Egress Optimizer

Guide the user through diagnosing and fixing application-side query patterns that cause excessive data transfer (egress) from their Postgres database. Most high egress bills come from the application fetching more data than it uses.

Work the four steps in order: diagnose which queries transfer the most data, analyze the codebase behind them, fix the anti-patterns, then verify nothing broke and the transfer actually dropped.

Step 1: Diagnose

Identify which queries transfer the most data. The primary tool is the pg_stat_statements extension.

Check if pg_stat_statements is available

SELECT 1 FROM pg_stat_statements LIMIT 1;

If this errors, the extension needs to be created:

CREATE EXTENSION IF NOT EXISTS pg_stat_statements;

On Neon the extension is available by default, but it may still need this CREATE EXTENSION step. Ask before creating it because installing an extension modifies the database.

Handle empty stats

Stats are cleared when a Neon compute scales to zero and restarts. If the stats are empty or the compute recently woke up:

  1. Ask before resetting the stats, then start a clean measurement window with SELECT pg_stat_statements_reset();
  2. Let the application run under representative traffic for at least an hour.
  3. Return and run the diagnostic queries below.

If the user has stats from a production database, use those. If they have no access to production stats, proceed to Step 2 and analyze the codebase directly — code-level patterns are often sufficient to identify the worst offenders.

Diagnostic queries

Run these to identify the top egress contributors. Focus on queries that return many rows, return wide rows (JSONB, TEXT, BYTEA columns), or are called very frequently.

Queries returning the most total rows:

SELECT query, calls, rows AS total_rows, rows / calls AS avg_rows_per_call
FROM pg_stat_statements
WHERE calls > 0
ORDER BY rows DESC
LIMIT 10;

Queries returning the most rows per execution (poorly scoped SELECTs, missing pagination):

SELECT query, calls, rows AS total_rows, rows / calls AS avg_rows_per_call
FROM pg_stat_statements
WHERE calls > 0
ORDER BY avg_rows_per_call DESC
LIMIT 10;

Most frequently called queries (candidates for caching):

SELECT query, calls, rows AS total_rows, rows / calls AS avg_rows_per_call
FROM pg_stat_statements
WHERE calls > 0
ORDER BY calls DESC
LIMIT 10;

Longest running queries (not a direct egress measure, but helps identify problem queries during a spike):

SELECT query, calls, rows AS total_rows,
  round(total_exec_time::numeric, 2) AS total_exec_time_ms
FROM pg_stat_statements
WHERE calls > 0
ORDER BY total_exec_time DESC
LIMIT 10;

Interpret the results

Rank findings by estimated egress impact:

  • High row count + wide rows = biggest egress. A query returning 1,000 rows where each row includes a 50KB JSONB column transfers ~50MB per call.
  • Extreme call frequency on even small queries adds up. A query called 50,000 times/day returning 10 rows each = 500,000 rows/day.
  • Cross-reference with the schema to identify which columns are wide. Look for JSONB, TEXT, BYTEA, and large VARCHAR columns.

Step 2: Analyze the Codebase

For each query identified in Step 1, or for each database query in the codebase if no stats are available, check:

  • Does it select only the columns the response needs?
  • Does it return a bounded number of rows (LIMIT/pagination)?
  • Is it called frequently enough to benefit from caching?
  • Does it fetch raw data that gets aggregated in application code?
  • Does it use a JOIN that duplicates parent data across child rows?

Step 3: Fix

Apply the appropriate fix for each problem found. Below are the most common egress anti-patterns and how to fix them.

Unused columns (SELECT *)

Problem: The query fetches all columns but the application only uses a few. Large columns (JSONB blobs, TEXT fields) get transferred over the wire and discarded.

Fix: Name only the columns the response needs.

Before:

SELECT * FROM products;

After:

SELECT id, name, price, image_urls FROM products;

Missing pagination

Problem: A list endpoint returns all rows with no LIMIT. This is an unbounded egress risk — every new row in the table increases data transfer on every request. Flag this regardless of current table size.

This is easy to miss because the application may work fine with small datasets. But at scale, an unpaginated endpoint returning 10,000 rows with even moderate column widths can transfer hundreds of megabytes per day.

Fix: Bound the result set with ORDER BY plus LIMIT/OFFSET.

Before:

SELECT id, name, price FROM products;

After:

SELECT id, name, price FROM products
ORDER BY id
LIMIT 50 OFFSET 0;

When adding pagination, check whether the consuming client already supports paginated responses. If not, pick sensible defaults and document the pagination parameters in the API.

High-frequency queries on static data

Problem: A query is called thousands of times per day but returns data that rarely changes. Every call transfers the same rows from the database. This pattern is only visible from pg_stat_statements — the code itself looks normal.

Look for queries with extremely high call counts relative to other queries. Common examples: configuration tables, category lists, feature flags, user role definitions.

Fix: Add a caching layer between the application and the database so it avoids hitting the database on every request. Reuse a cache the app already has. Otherwise, a serverless Redis such as Upstash Redis works from serverless and edge runtimes without connection management.

Application-side aggregation

Problem: The application fetches all rows from a table and then computes aggregates (averages, counts, sums, groupings) in application code. The full dataset transfers over the wire even though the result is a small summary.

Fix: Push the aggregation into SQL.

Before: The application fetches entire tables and aggregates in code with loops or .reduce().

After:

SELECT p.category_id,
       AVG(r.rating) AS avg_rating,
       COUNT(r.id) AS review_count
FROM reviews r
INNER JOIN products p ON r.product_id = p.id
GROUP BY p.category_id;

JOIN duplication

Problem: A JOIN between a wide parent table and a child table duplicates all parent columns across every child row. If a product has 200 reviews and the product row includes a 50KB JSONB column, the join sends that 50KB × 200 = ~10MB for a single request.

This is distinct from the SELECT * problem. Even if you select only needed columns, a JOIN still repeats the parent data for every child row. The fix is structural: avoid the join entirely.

Fix: Split the join into two queries, one per table.

Before:

SELECT * FROM products
LEFT JOIN reviews ON reviews.product_id = products.id
WHERE products.id = 1;

After (two separate queries):

SELECT id, name, price, description, image_urls FROM products WHERE id = 1;
SELECT id, user_name, rating, body FROM reviews WHERE product_id = 1;

Two queries instead of one JOIN. The product data is fetched once. The reviews are fetched once. No duplication.

Step 4: Verify

After applying fixes:

  1. Run existing tests to confirm nothing broke.
  2. Check the responses — make sure the API still returns the same data shape. Column selection and pagination changes can break clients that depend on specific fields or full result sets.
  3. Measure the improvement — if pg_stat_statements data is available, ask before resetting it (SELECT pg_stat_statements_reset();), let traffic run, then re-run the diagnostic queries to compare before and after.

Neon Infrastructure as Code (neon.ts)

The fixes above cut egress (data transferred out of Postgres). The other big non-prod cost lever is compute, and you can codify it durably in neon.ts — Neon's infrastructure-as-code file (see the neon skill for the full reference) — so dev, preview, and CI branches stay cheap by default instead of relying on per-branch flags:

npm i @neon/config
// neon.ts
import { defineConfig } from "@neon/config/v1";

export default defineConfig({
  branch: (branch) => {
    if (branch.exists || branch.isDefault) return {}; // don't touch prod
    return {
      ttl: "7d", // ephemeral branches auto-expire instead of accruing storage
      postgres: {
        computeSettings: {
          autoscalingLimitMinCu: 0.25, // scale to zero when idle
          autoscalingLimitMaxCu: 1, // cap autoscaling on throwaway branches
          suspendTimeout: "5m",
        },
      },
    };
  },
});
neon config apply   # apply to the current branch (neon deploy is an alias)

This is complementary, not a substitute: query-pattern fixes are what actually reduce egress charges, while these settings keep non-production compute and storage from quietly inflating the same bill. Because neon checkout applies the policy when it creates a branch, new dev/preview branches inherit the cheap profile automatically.

Further Reading

Source: SKILL.md on GitHub

skilld matched fixed text patterns in SKILL.md and file names. Patterns miss obfuscated code.

skilld run checks every file with the same patterns. It asks for approval before it loads a Skill with a behavior marked Needs approval.

No alerts8d5 checks · Risk SAFE
  • Gen Agent Trust Hub8d

    This skill is generally safe and provides official guidance for Neon Postgres. It includes a reranking feature for search results that processes data from the database, which presents a surface for indirect prompt injection, though the impact is limited to search relevance. It also installs standard packages and executes official CLI tools.

  • Socket8d

    No alerts

  • Snyk8d

    Risk: LOW · No issues

  • Runlayer8mo

    4/14 files flagged

  • ZeroLeaks7mo

    Score: 93/100 · 2 sections analyzed

Signed by skilld at d3601d9. This ties the file your Agent reads to that commit on GitHub. It does not review the instructions.

Last checked against GitHub yesterday.

Activeupdated last week
Other metadata
metadata
{
  "parent": "neon",
  "source": "https://github.com/neondatabase/agent-skills/tree/main/skills/neon-postgres"
}

README badge

README badge for neondatabase/agent-skills/neon-postgres

Guides setup, connections, branching, and advanced features for Neon Serverless Postgres. Covers the Neon CLI, MCP server, REST API, TypeScript and Python SDKs, connection pooling strategies for different runtimes (node-postgres vs. serverless driver), Neon Auth, and infrastructure-as-code via neon.ts. Use when a user asks about Neon setup, DATABASE_URL, scale-to-zero, autoscaling, read replicas, or connecting an ORM like Drizzle.

Generated from the current SKILL.md.

Does Neon work with my ORM or database client?
Yes. Neon is fully compatible with Postgres, so it works with any language, framework, or ORM that supports Postgres — including Drizzle, Prisma, TypeORM, node-postgres, and the Neon serverless driver. Pick the driver based on your runtime: use node-postgres for long-running environments and the serverless driver for fully-isolated serverless platforms like Netlify.
What's the difference between the serverless driver and node-postgres?
The serverless driver (@neondatabase/serverless) queries over HTTP and is designed for fully-isolated serverless runtimes like Netlify where a persistent TCP connection can't be reused per request. node-postgres (pg) maintains a persistent connection pool and is better for long-running or shared-runtime environments like Vercel Fluid compute or Neon Functions.
Can I use this skill to set up a new Neon project from scratch?
Yes. Run `npx -y neonctl@latest init --agent <agent-name>` to automatically create an API key, set up the MCP server or extension, install the skill, and guide you through project selection or creation. This is the recommended starting point.
Does this skill cover Neon Auth setup?
Yes. The skill covers Neon Auth setup, including provisioning it via the MCP server and integrating it with the Neon JS SDK for combined auth and data API workflows. Skip it if your app doesn't need user authentication.
Can I use infrastructure-as-code to configure Neon branches?
Yes, via `neon.ts` (from @neondatabase/config). Declare which services (Neon Auth, Data API) your branches need and per-branch compute settings like autoscaling and scale-to-zero, then use `neonctl config apply` to provision them.

Generated from the current SKILL.md. These answers refresh after source changes.