Index of every page on this site: llms.txt
All skills
huggingface avatar

/transformers-js

@205e9c2
Run it yourself

Run state-of-the-art machine learning models directly in JavaScript. `@huggingface/transformers` supports text, vision, audio, and multimodal tasks in browsers and Node.js / Bun / Deno, with WebGPU or WASM execution.

Use this Skill: https://skilld.dev/gh/huggingface/transformers.js/transformers-js

Nothing lands on disk. Nothing to clean up.

Fork this Skill

Edit a local copy. It keeps the author and licence.

referencesPIPELINE_OPTIONS.md

≈5.1k tokens on demand. Your agent reads this file only when SKILL.md points to it.

Pipeline options

The pipeline(task, model?, options?) factory accepts a common set of options across every task. Task-specific call options are documented on each pipeline class in the API reference.

Common options

<!-- @generated:start id=typedef:PretrainedModelOptions -->
Option Type Description
progress_callback? ProgressCallback If specified, this function is called during model construction with progress updates. (default: null)
config? PretrainedConfig Configuration to use for the model instead of an automatically loaded configuration. Configuration can be automatically loaded when:<br />- The model is provided by the library and loaded with the model ID string of a pretrained model.<br />- The model is loaded by supplying a local directory as pretrained_model_name_or_path and a configuration JSON file named config.json is found in the directory. (default: null)
cache_dir? string Path to a directory where downloaded model files should be cached if the standard cache should not be used. (default: null)
local_files_only? boolean Whether to only look at local files (e.g., not try downloading the model). (default: false)
revision? string The model revision to use. This can be a branch name, tag name, or commit ID. Because the Hub uses Git-based storage, revision can be any identifier accepted by Git. Ignored for local requests. (default: 'main')
subfolder? string In case the relevant files are located inside a subfolder of the model repo on huggingface.co, you can specify the folder name here. (default: 'onnx')
model_file_name? string Override the base ONNX model file name, excluding dtype and .onnx suffixes. This is most useful for single-session models. (default: null)
device? DeviceType|Record<string, DeviceType> The device to run the model on. If not specified, the device will be chosen from the environment settings. (default: null)
dtype? DataType|Record<string, DataType> The data type to use for the model. If not specified, the data type will be chosen from the environment settings. (default: null)
use_external_data_format? ExternalData|Record<string, ExternalData> Whether to load external data files. null uses the model configuration; false disables external data; true uses one chunk; a number selects the chunk count. (default: null)
session_options? InferenceSession.SessionOptions User-specified ONNX Runtime session options. Suitable defaults are filled in when omitted. (default: {})
<!-- @generated:end id=typedef:PretrainedModelOptions -->

Progress tracking

The progress_total event reports aggregate progress across every file the model needs, which is what a loading bar usually wants:

const pipe = await pipeline("sentiment-analysis", null, {
  progress_callback: (info) => {
    if (info.status === "progress_total") {
      console.log(`Loading model: ${info.progress.toFixed(1)}%`);
    }
  },
});

For per-file detail, listen for progress events instead:

const pipe = await pipeline("sentiment-analysis", null, {
  progress_callback: (info) => {
    if (info.status === "progress") {
      console.log(`${info.file}: ${info.progress.toFixed(1)}%`);
    }
  },
});

ProgressInfo shape

<!-- @generated:start id=typedef:ProgressInfo -->

InitiateProgressInfo

Option Type Description
status 'initiate' A file load is about to start.
name string The model ID or directory path.
file string The name of the file.

DownloadProgressInfo

Option Type Description
status 'download' A file download has started.
name string The model ID or directory path.
file string The name of the file.

ProgressStatusInfo

Option Type Description
status 'progress' A file download has reported byte progress.
name string The model ID or directory path.
file string The name of the file.
progress number A number between 0 and 100.
loaded number The number of bytes loaded.
total number The total number of bytes to be loaded.

DoneProgressInfo

Option Type Description
status 'done' A file has finished loading.
name string The model ID or directory path.
file string The name of the file.

ReadyProgressInfo

Option Type Description
status 'ready' The requested pipeline is ready to use.
task string The loaded task.
model string The loaded model.

TotalProgressInfo

Option Type Description
status 'progress_total' Aggregate progress across all files being loaded.
name string The model ID or directory path.
progress number A number between 0 and 100.
loaded number The number of bytes loaded.
total number The total number of bytes to be loaded.
files FilesLoadingMap A mapping of file names to their loading progress.
<!-- @generated:end id=typedef:ProgressInfo -->

Device selection

// Default — CPU (onnxruntime-node in Node.js, WASM in browsers)
await pipeline("sentiment-analysis", null);

// GPU via WebGPU (Chrome 113+, fastest for big models)
await pipeline("text-generation", "onnx-community/Qwen3-0.6B-ONNX", {
  device: "webgpu",
  dtype: "fp16",
});

// Node.js with onnxruntime-node
await pipeline("sentiment-analysis", null, { device: "cpu" });

Detect WebGPU support before attempting to use it:

const hasWebGPU = typeof navigator !== "undefined" && "gpu" in navigator;
const pipe = await pipeline("text-generation", "onnx-community/Qwen3-0.6B-ONNX", {
  device: hasWebGPU ? "webgpu" : "wasm",
  dtype: hasWebGPU ? "fp16" : "q8",
});

Task-specific call options

Once created, each pipeline is itself callable with task-specific options:

const classifier = await pipeline("text-classification");
await classifier("great movie", { top_k: 3 });

const generator = await pipeline("text-generation");
await generator("Once upon a time", { max_new_tokens: 100, do_sample: true, temperature: 0.7 });

const translator = await pipeline("translation", "Xenova/nllb-200-distilled-600M");
await translator("Hello", { src_lang: "eng_Latn", tgt_lang: "fra_Latn" });

The per-task recipes in TASKS.md show the most common call options in use. For the exhaustive option types of a specific pipeline, see the API reference.

Generation parameters

Every field below can be passed as an option to a text-generation, text2text-generation, translation, or summarization pipeline call, or anywhere generate() is invoked directly.

Pairs of fields with overlapping meanings:

  • max_new_tokens overrides max_length. Pass one, not both — max_new_tokens is usually what you want because it doesn't depend on the prompt length.
  • min_new_tokens overrides min_length for the same reason.
  • Sampling vs. greedy: temperature and top_k only take effect with do_sample: true. Under greedy decoding (the default) they are silently ignored.
  • Beam search is not yet implemented: num_beams > 1 is accepted but generation still keeps a single sequence, so leave it at the default.
<!-- @generated:start id=fields:GenerationConfig -->
Option Type Description
max_length? number The maximum length the generated tokens can have. Corresponds to the length of the input prompt + max_new_tokens. Its effect is overridden by max_new_tokens, if also set. (default: 20)
max_new_tokens? number The maximum number of tokens to generate, ignoring the number of tokens in the prompt. (default: null)
min_length? number The minimum length of the sequence to be generated. Corresponds to the length of the input prompt + min_new_tokens. Its effect is overridden by min_new_tokens, if also set. (default: 0)
min_new_tokens? number The minimum number of tokens to generate, ignoring the number of tokens in the prompt. (default: null)
early_stopping? boolean|"never" Controls the stopping condition for beam-based methods, like beam-search. It accepts the following values:<br />- true, where the generation stops as soon as there are num_beams complete candidates;<br />- false, where a heuristic is applied and the generation stops when it is very unlikely to find better candidates;<br />- "never", where the beam search procedure only stops when there cannot be better candidates (canonical beam search algorithm). (default: false)
max_time? number The maximum time, in seconds, allowed for generation. Generation will still finish the current pass after the allocated time has passed. (default: null)
do_sample? boolean Whether to use sampling; use greedy decoding otherwise. (default: false)
num_beams? number Number of beams for beam search. 1 means no beam search. (default: 1)
num_beam_groups? number Number of groups to divide num_beams into to encourage diversity among different groups of beams. See this paper for more details. (default: 1)
penalty_alpha? number Balance model confidence against the degeneration penalty during contrastive search decoding. (default: null)
use_cache? boolean Whether the model should reuse past key/value states, when supported, to speed up decoding. (default: true)
temperature? number The value used to modulate the next token probabilities. (default: 1.0)
top_k? number The number of highest-probability vocabulary tokens to keep for top-k filtering. (default: 50)
top_p? number If set to a float below 1, only the smallest set of most probable tokens with probabilities that add up to top_p or higher are kept for generation. (default: 1.0)
typical_p? number Local typicality measures how similar the conditional probability of predicting a target token next is to the expected conditional probability of predicting a random token next, given the partial text already generated. If set to a float below 1, the smallest set of the most locally typical tokens with probabilities that add up to typical_p or higher are kept for generation. See this paper for more details. (default: 1.0)
epsilon_cutoff? number If set to a float strictly between 0 and 1, only tokens with a conditional probability greater than epsilon_cutoff will be sampled. In the paper, suggested values range from 3e-4 to 9e-4, depending on the size of the model. See Truncation Sampling as Language Model Desmoothing for more details. (default: 0.0)
eta_cutoff? number Eta sampling is a hybrid of locally typical sampling and epsilon sampling. If set to a float strictly between 0 and 1, a token is only considered if it is greater than either eta_cutoff or sqrt(eta_cutoff) * exp(-entropy(softmax(next_token_logits))). The latter term is intuitively the expected next token probability, scaled by sqrt(eta_cutoff). In the paper, suggested values range from 3e-4 to 2e-3, depending on the size of the model. See Truncation Sampling as Language Model Desmoothing for more details. (default: 0.0)
diversity_penalty? number This value is subtracted from a beam's score if it generates the same token as any beam from another group at a particular time. Note that diversity_penalty is only effective if group beam search is enabled. (default: 0.0)
repetition_penalty? number Penalty applied to repeated tokens. 1.0 means no penalty. See this paper for more details. (default: 1.0)
encoder_repetition_penalty? number Penalty applied to sequences that are not in the original input. 1.0 means no penalty. (default: 1.0)
length_penalty? number Exponential penalty applied to sequence length during beam-based generation. It is applied as an exponent to the sequence length, which in turn is used to divide the score of the sequence. Since the score is the log likelihood of the sequence (i.e. negative), length_penalty > 0.0 promotes longer sequences, while length_penalty < 0.0 encourages shorter sequences. (default: 1.0)
no_repeat_ngram_size? number If set to an integer greater than 0, all n-grams of that size can only occur once. (default: 0)
bad_words_ids? number[][] List of token IDs that are not allowed to be generated. In order to get the token IDs of the words that should not appear in the generated text, use tokenizer(bad_words, { add_prefix_space: true, add_special_tokens: false }).input_ids. (default: null)
force_words_ids? number[][]|number[][][] List of token IDs that must be generated. If given a number[][], this is treated as a simple list of words that must be included, the opposite of bad_words_ids. If given number[][][], this triggers a disjunctive constraint, which allows different forms of each word. (default: null)
renormalize_logits? boolean Whether to renormalize the logits after applying all the logits processors or warpers (including the custom ones). It's highly recommended to set this flag to true because search algorithms assume the score logits are normalized, but some logit processors or warpers break the normalization. (default: false)
constraints? Object[] Custom constraints that guide generation to include certain tokens as defined by Constraint objects. (default: null)
forced_bos_token_id? number The ID of the token to force as the first generated token after the decoder_start_token_id. Useful for multilingual models like mBART where the first generated token needs to be the target language token. (default: null)
forced_eos_token_id? number|number[] The ID of the token to force as the last generated token when max_length is reached. Optionally, use a list to set multiple end-of-sequence tokens. (default: null)
remove_invalid_values? boolean Whether to remove possible nan and inf outputs of the model to prevent the generation method from crashing. Note that using remove_invalid_values can slow down generation. (default: false)
exponential_decay_length_penalty? [number, number] This tuple adds an exponentially increasing length penalty after a certain number of tokens have been generated. The tuple consists of: (start_index, decay_factor) where start_index indicates where the penalty starts and decay_factor represents the factor of exponential decay. (default: null)
suppress_tokens? number[] A list of tokens to suppress during generation. The SuppressTokens logit processor sets their log probabilities to -inf so that they are not sampled. (default: null)
streamer? TextStreamer Streamer used to yield generated text incrementally. (default: null)
begin_suppress_tokens? number[] A list of tokens to suppress at the beginning of the generation. The SuppressBeginTokens logit processor sets their log probabilities to -inf so that they are not sampled. (default: null)
forced_decoder_ids? [number, number][] A list of integer pairs that maps generation indices to token indices that will be forced before sampling. For example, [[1, 123]] means the second generated token will always be a token of index 123. (default: null)
guidance_scale? number The guidance scale for classifier-free guidance (CFG). CFG is enabled by setting guidance_scale > 1. Higher guidance scale encourages the model to generate samples that are more closely tied to the input prompt, usually at the expense of poorer quality. (default: null)
num_return_sequences? number The number of independently computed returned sequences for each element in the batch. (default: 1)
output_attentions? boolean Whether to return attention tensors from all attention layers. See attentions under returned tensors for more details. (default: false)
output_hidden_states? boolean Whether to return the hidden states of all layers. See hidden_states under returned tensors for more details. (default: false)
output_scores? boolean Whether to return the prediction scores. See scores under returned tensors for more details. (default: false)
return_dict_in_generate? boolean Whether to return a ModelOutput instead of a plain tuple. (default: false)
pad_token_id? number The ID of the padding token. (default: null)
bos_token_id? number The ID of the beginning-of-sequence token. (default: null)
eos_token_id? number|number[] The ID of the end-of-sequence token. Optionally, use a list to set multiple end-of-sequence tokens. (default: null)
encoder_no_repeat_ngram_size? number If set to an integer greater than 0, all n-grams of that size that occur in the encoder_input_ids cannot occur in the decoder_input_ids. (default: 0)
decoder_start_token_id? number If an encoder-decoder model starts decoding with a token other than bos, the ID of that token. (default: null)
generation_kwargs? Object Additional generation kwargs forwarded to the model's generate function. Kwargs that are not present in generate's signature are used in the model forward pass. (default: {})
<!-- @generated:end id=fields:GenerationConfig -->

Streaming tokens as they're produced

import { pipeline, TextStreamer } from "@huggingface/transformers";

const generator = await pipeline("text-generation", "onnx-community/Qwen3-0.6B-ONNX");
const streamer = new TextStreamer(generator.tokenizer, {
  skip_prompt: true,
  skip_special_tokens: true,
  callback_function: (text) => process.stdout.write(text),
});

await generator("Tell me a short story.", { max_new_tokens: 200, streamer });

For whisper-style chunk / token / finalize callbacks, use WhisperTextStreamer.

KV cache reuse across calls

For multi-turn chat, pass the same DynamicCache to every call and re-send the full conversation each turn. The pipeline uses the cache to skip re-encoding the tokens it has already processed (the shared prefix). Do not pass only the new message: the model needs the full sequence to compute token positions, so incremental-only input produces wrong results.

import { pipeline, DynamicCache } from "@huggingface/transformers";

const generator = await pipeline("text-generation", "onnx-community/Qwen3-0.6B-ONNX");
const past_key_values = new DynamicCache();
const messages = [{ role: "user", content: "What is the capital of France?" }];

// Turn 1
const turn1 = await generator(messages, { max_new_tokens: 128, past_key_values });
messages.push(turn1[0].generated_text.at(-1)); // append the assistant reply

// Turn 2: send the full conversation again — only the new tokens are encoded
messages.push({ role: "user", content: "What about Germany?" });
const turn2 = await generator(messages, { max_new_tokens: 128, past_key_values });
console.log(turn2[0].generated_text.at(-1).content);

await past_key_values.dispose(); // free the cache tensors when the conversation ends

Source: SKILL.md on GitHub

skilld matched fixed text patterns in SKILL.md and file names. Patterns miss obfuscated code.

skilld run checks every file with the same patterns. It asks for approval before it loads a Skill with a behavior marked Needs approval.

No alerts2mo3 checks · Risk SAFE
  • Gen Agent Trust Hub2mo

    This skill provides a JavaScript library for running machine learning models via Transformers.js. It includes standard security considerations such as downloading model weights from the Hugging Face Hub and processing external data. These features are documented with configurable safety options, such as local model loading and secure token management. See the detailed analysis for further context.

  • Socket2mo

    No alerts

  • Snyk2mo

    Risk: LOW · No issues

Signed by skilld at 205e9c2. This ties the file your Agent reads to that commit on GitHub. It does not review the instructions.

Last checked against GitHub last week.

Activeupdated 2 months ago
Other metadata
metadata
{
  "author": "huggingface",
  "repository": "https://github.com/huggingface/transformers.js"
}
compatibility
Node.js 20+ (or equivalent Bun / Deno), or a modern browser with ES modules. WebGPU requires runtime and hardware support; WASM is the fallback. Model downloads from the Hugging Face Hub require network access unless you ship models locally.

README badge

README badge for huggingface/transformers.js