Index of every page on this site: llms.txt
All skills
inference-sh avatar

/javascript-sdk

@c895a47
by inference.shinference-sh/skills819 stars
108
Run it yourself

JavaScript/TypeScript SDK for inference.sh - run AI apps, build agents, integrate with all models. Package: @inferencesh/sdk (npm install). Full TypeScript support, streaming, file uploads. Build agents with template or ad-hoc patterns, tool builder API, skills, human approval. Use for: JavaScript integration, TypeScript, Node.js, React, Next.js, frontend apps. Triggers: javascript sdk, typescript sdk, npm install, node.js api, js client, react ai, next.js ai, frontend sdk, @inferencesh/sdk, typescript agent, browser sdk, js integration

Use this Skill: https://skilld.dev/gh/inference-sh/skills/javascript-sdk

Nothing lands on disk. Nothing to clean up.

Fork this Skill

Edit a local copy. It keeps the author and licence.

SKILL.md

≈140 tokens for metadata: the name and description. ≈3.1k when used: this file. ≈20k more on demand in 8 files.

Description uses 7.0% of example budget

Before choosing a Skill, your Agent reads its name and description. All available Skills share that space.

  • A shorter description leaves more room for other Skills. This entry exceeds our 1% size suggestion.

In our Claude Code example, all Skill names and descriptions share 8,000 characters. This Skill uses ≈559 characters, or 7.0%.

The 1% threshold is a size suggestion. Longer descriptions can still fit.

Your model, settings, and other Skills decide how much text your Agent can read.

Example settings and source

The example uses a 200k-token context and default Claude Code settings. The count includes the name, description, separators, and when_to_use when present. Codex also counts local file paths.

Skit's source and limits: Codex 0.160.1, Claude Code 2.1.292.

Install the belt CLI skill: npx skills add belt-sh/cli

JavaScript SDK

Build AI applications with the inference.sh JavaScript/TypeScript SDK.

JavaScript SDK

Quick Start

npm install @inferencesh/sdk
import { inference } from '@inferencesh/sdk';

const client = inference({ apiKey: 'inf_your_key' });

// Run an AI app
const result = await client.run({
  app: 'pruna/p-image',
  input: { prompt: 'A sunset over mountains' }
});
console.log(result.output);

Installation

npm install @inferencesh/sdk
# or
yarn add @inferencesh/sdk
# or
pnpm add @inferencesh/sdk

Requirements: Node.js 18.0.0+ (or modern browser with fetch)

Authentication

import { inference } from '@inferencesh/sdk';

// Direct API key
const client = inference({ apiKey: 'inf_your_key' });

// From environment variable (recommended)
const client = inference({ apiKey: process.env.INFERENCE_API_KEY });

// For frontend apps (use proxy)
const client = inference({ proxyUrl: '/api/inference/proxy' });

Get your API key: Settings → API Keys → Create API Key

Running Apps

Basic Execution

const result = await client.run({
  app: 'pruna/p-image',
  input: { prompt: 'A cat astronaut' }
});

console.log(result.status);  // "completed"
console.log(result.output);  // Output data

Fire and Forget

const task = await client.run({
  app: 'google/veo-3-1-fast',
  input: { prompt: 'Drone flying over mountains' }
}, { wait: false });

console.log(`Task ID: ${task.id}`);
// Check later with client.getTask(task.id)

Streaming Progress

const stream = await client.run({
  app: 'google/veo-3-1-fast',
  input: { prompt: 'Ocean waves at sunset' }
}, { stream: true });

for await (const update of stream) {
  console.log(`Status: ${update.status}`);
  if (update.logs?.length) {
    console.log(update.logs.at(-1));
  }
}

Run Parameters

Parameter Type Description
app string App ID (namespace/name@version)
input object Input matching app schema
setup object Hidden setup configuration
infra string 'cloud' or 'private'
session string Session ID for stateful execution
session_timeout number Idle timeout (1-3600 seconds)

File Handling

Automatic Upload

const result = await client.run({
  app: 'image-processor',
  input: {
    image: '/path/to/image.png'  // Auto-uploaded
  }
});

Manual Upload

// Basic upload
const file = await client.uploadFile('/path/to/image.png');

// With options
const file = await client.uploadFile('/path/to/image.png', {
  filename: 'custom_name.png',
  contentType: 'image/png',
  public: true
});

const result = await client.run({
  app: 'image-processor',
  input: { image: file.uri }
});

Browser File Upload

const input = document.querySelector('input[type="file"]');
const file = await client.uploadFile(input.files[0]);

Sessions (Stateful Execution)

Keep workers warm across multiple calls:

// Start new session
const result = await client.run({
  app: 'my-app',
  input: { action: 'init' },
  session: 'new',
  session_timeout: 300  // 5 minutes
});
const sessionId = result.session_id;

// Continue in same session
const result2 = await client.run({
  app: 'my-app',
  input: { action: 'process' },
  session: sessionId
});

Agent SDK

Template Agents

Use pre-built agents from your workspace:

const agent = client.agent('my-team/support-agent@latest');

// Send message
const response = await agent.sendMessage('Hello!');
console.log(response.text);

// Multi-turn conversation
const response2 = await agent.sendMessage('Tell me more');

// Reset conversation
agent.reset();

// Get chat history
const chat = await agent.getChat();

Ad-hoc Agents

Create custom agents programmatically:

import { tool, string, number, appTool } from '@inferencesh/sdk';

// Define tools
const calculator = tool('calculate')
  .describe('Perform a calculation')
  .param('expression', string('Math expression'))
  .build();

const imageGen = appTool('generate_image', 'pruna/p-image@latest')
  .describe('Generate an image')
  .param('prompt', string('Image description'))
  .build();

// Create agent
const agent = client.agent({
  core_app: { ref: 'anthropic/claude-sonnet-5-5@latest' },
  system_prompt: 'You are a helpful assistant.',
  tools: [calculator, imageGen],
  temperature: 0.7,
  max_tokens: 4096
});

const response = await agent.sendMessage('What is 25 * 4?');

Available Core Apps

Model App Reference
Claude Sonnet 5.5 anthropic/claude-sonnet-5-5@latest
Claude Haiku 5.5 anthropic/claude-haiku-5-5@latest
GPT-6.1 Sol openai/gpt-6-1-sol@latest
GPT-6 Luna openai/gpt-6-luna@latest

Tool Builder API

Parameter Types

import {
  string, number, integer, boolean,
  enumOf, array, obj, optional
} from '@inferencesh/sdk';

const name = string('User\'s name');
const age = integer('Age in years');
const score = number('Score 0-1');
const active = boolean('Is active');
const priority = enumOf(['low', 'medium', 'high'], 'Priority');
const tags = array(string('Tag'), 'List of tags');
const address = obj({
  street: string('Street'),
  city: string('City'),
  zip: optional(string('ZIP'))
}, 'Address');

Client Tools (Run in Your Code)

const greet = tool('greet')
  .display('Greet User')
  .describe('Greets a user by name')
  .param('name', string('Name to greet'))
  .requireApproval()
  .build();

App Tools (Call AI Apps)

const generate = appTool('generate_image', 'pruna/flux-dev@latest')
  .describe('Generate an image from text')
  .param('prompt', string('Image description'))
  .input({ num_inference_steps: 20 })
  .requireApproval()
  .build();

Agent Tools (Delegate to Sub-agents)

import { agentTool } from '@inferencesh/sdk';

const researcher = agentTool('research', 'my-org/researcher@v1')
  .describe('Research a topic')
  .param('topic', string('Topic to research'))
  .build();

Webhook Tools (Call External APIs)

import { webhookTool } from '@inferencesh/sdk';

const notify = webhookTool('slack', 'https://hooks.slack.com/...')
  .describe('Send Slack notification')
  .secret('SLACK_SECRET')
  .param('channel', string('Channel'))
  .param('message', string('Message'))
  .build();

Internal Tools (Built-in Capabilities)

import { internalTools } from '@inferencesh/sdk';

const config = internalTools()
  .plan()
  .memory()
  .webSearch(true)
  .codeExecution(true)
  .imageGeneration({
    enabled: true,
    appRef: 'pruna/flux-dev@latest'
  })
  .build();

const agent = client.agent({
  core_app: { ref: 'anthropic/claude-sonnet-5-5@latest' },
  internal_tools: config
});

Streaming Agent Responses

const response = await agent.sendMessage('Explain quantum computing', {
  onMessage: (msg) => {
    if (msg.content) {
      process.stdout.write(msg.content);
    }
  },
  onToolCall: async (call) => {
    console.log(`\n[Tool: ${call.name}]`);
    const result = await executeTool(call.name, call.args);
    agent.submitToolResult(call.id, result);
  }
});

File Attachments

// From file path (Node.js)
import { readFileSync } from 'fs';
const response = await agent.sendMessage('What\'s in this image?', {
  files: [readFileSync('image.png')]
});

// From base64
const response = await agent.sendMessage('Analyze this', {
  files: ['data:image/png;base64,iVBORw0KGgo...']
});

// From browser File object
const input = document.querySelector('input[type="file"]');
const response = await agent.sendMessage('Describe this', {
  files: [input.files[0]]
});

Skills (Reusable Context)

const agent = client.agent({
  core_app: { ref: 'anthropic/claude-sonnet-5-5@latest' },
  skills: [
    {
      name: 'code-review',
      description: 'Code review guidelines',
      content: '# Code Review\n\n1. Check security\n2. Check performance...'
    },
    {
      name: 'api-docs',
      description: 'API documentation',
      url: 'https://example.com/skills/api-docs.md'
    }
  ]
});

Server Proxy (Frontend Apps)

For browser apps, proxy through your backend to keep API keys secure:

Client Setup

const client = inference({
  proxyUrl: '/api/inference/proxy'
  // No apiKey needed on frontend
});

Next.js Proxy (App Router)

// app/api/inference/proxy/route.ts
import { createRouteHandler } from '@inferencesh/sdk/proxy/nextjs';

const route = createRouteHandler({
  apiKey: process.env.INFERENCE_API_KEY
});

export const POST = route.POST;

Express Proxy

import express from 'express';
import { createProxyMiddleware } from '@inferencesh/sdk/proxy/express';

const app = express();
app.use('/api/inference/proxy', createProxyMiddleware({
  apiKey: process.env.INFERENCE_API_KEY
}));

Supported Frameworks

  • Next.js (App Router & Pages Router)
  • Express
  • Hono
  • Remix
  • SvelteKit

TypeScript Support

Full type definitions included:

import type {
  TaskDTO,
  ChatDTO,
  ChatMessageDTO,
  AgentTool,
  TaskStatusCompleted,
  TaskStatusFailed
} from '@inferencesh/sdk';

if (result.status === TaskStatusCompleted) {
  console.log('Done!');
} else if (result.status === TaskStatusFailed) {
  console.log('Failed:', result.error);
}

Error Handling

import { RequirementsNotMetException, InferenceError } from '@inferencesh/sdk';

try {
  const result = await client.run({ app: 'my-app', input: {...} });
} catch (e) {
  if (e instanceof RequirementsNotMetException) {
    console.log('Missing requirements:');
    for (const err of e.errors) {
      console.log(`  - ${err.type}: ${err.key}`);
    }
  } else if (e instanceof InferenceError) {
    console.log('API error:', e.message);
  }
}

Human Approval Workflows

const response = await agent.sendMessage('Delete all temp files', {
  onToolCall: async (call) => {
    if (call.requiresApproval) {
      const approved = await promptUser(`Allow ${call.name}?`);
      if (approved) {
        const result = await executeTool(call.name, call.args);
        agent.submitToolResult(call.id, result);
      } else {
        agent.submitToolResult(call.id, { error: 'Denied by user' });
      }
    }
  }
});

CommonJS Support

const { inference, tool, string } = require('@inferencesh/sdk');

const client = inference({ apiKey: 'inf_...' });
const result = await client.run({...});

Reference Files

Related Skills

# Python SDK
npx skills add inference-sh/skills@python-sdk

# Full platform skill (all apps via CLI)
npx skills add inference-sh/skills@infsh-cli

# LLM models
npx skills add inference-sh/skills@llm-models

# Image generation
npx skills add inference-sh/skills@ai-image-generation

Documentation

Source: SKILL.md on GitHub

skilld matched fixed text patterns in SKILL.md and file names. Patterns miss obfuscated code.

skilld run checks every file with the same patterns. It asks for approval before it loads a Skill with a behavior marked Needs approval.

1 alert3d5 checks · Risk SAFE
  • Gen Agent Trust Hub3d

    The skill provides comprehensive documentation and code examples for the inference.sh JavaScript/TypeScript SDK, covering agent building, tool creation, file handling, and secure server-side proxy integration with security best practices.

  • Socket3d

    No alerts

  • Snyk3d

    Risk: CRITICAL · 2 issues

  • Runlayer8mo

    6/9 files flagged

  • ZeroLeaks7mo

    3 findings · Score: 69/100

Signed by skilld at c895a47. This ties the file your Agent reads to that commit on GitHub. It does not review the instructions.

Last checked against GitHub 3 days ago.

Activeupdated 4 days ago
What it can do
Runs commands
All 5 allowed tools
Bash(npm *)Bash(npx *)Bash(node *)Bash(pnpm *)Bash(yarn *)

README badge

README badge for inference-sh/skills/javascript-sdk

Provides a JavaScript/TypeScript SDK for building AI applications on inference.sh, supporting 250+ models, streaming, file uploads, and multi-turn agents with tool integration. Use for Node.js backends, React/Next.js frontends, or ad-hoc agent creation with Claude, GPT-4o, or custom core models via the @inferencesh/sdk npm package.

Generated from the current SKILL.md.

Does this SDK support TypeScript?
Yes. The SDK includes full TypeScript type definitions and works with Node.js 18.0.0+. It also supports CommonJS and ESM.
Can I use this in a browser or frontend app?
Yes. For frontend apps, you proxy API calls through your backend (Next.js, Express, Hono, Remix, or SvelteKit) to keep your API key secure.
What models are available for agents?
The SDK supports Claude Sonnet 4, Claude 3.5 Haiku, GPT-4o, and GPT-4o Mini as core agent models, plus 250+ other AI apps accessible through the platform.
Does this support file uploads?
Yes. The SDK handles automatic file uploads for inputs and manual uploads via uploadFile(). It works with Node.js file paths and browser File objects.
Can I build multi-turn conversations?
Yes. The agent SDK supports multi-turn chat with sendMessage(), conversation reset, and chat history retrieval. You can also use sessions to keep workers warm across multiple calls.

Generated from the current SKILL.md. These answers refresh after source changes.