skilld
All skills
huggingface avatar

/transformers-js

@4db6015 official
by Hugging Facehuggingface/skills11k stars
753

Use Transformers.js to run state-of-the-art machine learning models directly in JavaScript/TypeScript. Supports NLP (text classification, translation, summarization), computer vision (image classification, object detection), audio (speech recognition, audio classification), and multimodal tasks. Works in browsers and server-side runtimes (Node.js, Bun, Deno) with WebGPU/WASM using pre-trained models from Hugging Face Hub.

Ask your Agent

Use this Skill: https://skilld.dev/gh/huggingface/skills/transformers-js

This session only. Nothing lands on disk.

referencesMODEL_ARCHITECTURES.md

≈1.4k tokens on demand. Your agent reads this file only when SKILL.md points to it.

Supported Model Architectures

This document lists the model architectures currently supported by Transformers.js.

Natural Language Processing

Text Models

  • ALBERT - A Lite BERT for Self-supervised Learning
  • BERT - Bidirectional Encoder Representations from Transformers
  • CamemBERT - French language model based on RoBERTa
  • CodeGen - Code generation models
  • CodeLlama - Code-focused Llama models
  • Cohere - Command-R models for RAG
  • DeBERTa - Decoding-enhanced BERT with Disentangled Attention
  • DeBERTa-v2 - Improved version of DeBERTa
  • DistilBERT - Distilled version of BERT (smaller, faster)
  • GPT-2 - Generative Pre-trained Transformer 2
  • GPT-Neo - Open source GPT-3 alternative
  • GPT-NeoX - Larger GPT-Neo models
  • LLaMA - Large Language Model Meta AI
  • Mistral - Mistral AI language models
  • MPNet - Masked and Permuted Pre-training
  • MobileBERT - Compressed BERT for mobile devices
  • RoBERTa - Robustly Optimized BERT
  • T5 - Text-to-Text Transfer Transformer
  • XLM-RoBERTa - Multilingual RoBERTa

Sequence-to-Sequence

  • BART - Denoising Sequence-to-Sequence Pre-training
  • Blenderbot - Open-domain chatbot
  • BlenderbotSmall - Smaller Blenderbot variant
  • M2M100 - Many-to-Many multilingual translation
  • MarianMT - Neural machine translation
  • mBART - Multilingual BART
  • NLLB - No Language Left Behind (200 languages)
  • Pegasus - Pre-training with extracted gap-sentences

Computer Vision

Image Classification

  • BEiT - BERT Pre-Training of Image Transformers
  • ConvNeXT - Modern ConvNet architecture
  • ConvNeXTV2 - Improved ConvNeXT
  • DeiT - Data-efficient Image Transformers
  • DINOv2 - Self-supervised Vision Transformer
  • DINOv3 - Latest DINO iteration
  • EfficientNet - Efficient convolutional networks
  • MobileNet - Lightweight models for mobile
  • MobileViT - Mobile Vision Transformer
  • ResNet - Residual Networks
  • SegFormer - Semantic segmentation transformer
  • Swin - Shifted Window Transformer
  • ViT - Vision Transformer

Object Detection

  • DETR - Detection Transformer
  • D-FINE - Fine-grained Distribution Refinement for object detection
  • DINO - DETR with Improved deNoising anchOr boxes
  • Grounding DINO - Open-set object detection
  • YOLOS - You Only Look at One Sequence

Segmentation

  • CLIPSeg - Image segmentation with text prompts
  • Mask2Former - Universal image segmentation
  • SAM - Segment Anything Model
  • EdgeTAM - On-Device Track Anything Model

Depth & Pose

  • DPT - Dense Prediction Transformer
  • Depth Anything - Monocular depth estimation
  • Depth Pro - Sharp monocular metric depth
  • GLPN - Global-Local Path Networks for depth

Audio

Speech Recognition

  • Wav2Vec2 - Self-supervised speech representations
  • Whisper - Robust speech recognition (multilingual)
  • HuBERT - Self-supervised speech representation learning

Audio Processing

  • Audio Spectrogram Transformer - Audio classification
  • DAC - Descript Audio Codec

Text-to-Speech

  • SpeechT5 - Unified speech and text pre-training
  • VITS - Conditional Variational Autoencoder with adversarial learning

Multimodal

Vision-Language

  • CLIP - Contrastive Language-Image Pre-training
  • Chinese-CLIP - Chinese version of CLIP
  • ALIGN - Large-scale noisy image-text pairs
  • BLIP - Bootstrapping Language-Image Pre-training
  • Florence-2 - Unified vision foundation model
  • LLaVA - Large Language and Vision Assistant
  • Moondream - Tiny vision-language model

Document Understanding

  • DiT - Document Image Transformer
  • Donut - OCR-free Document Understanding
  • LayoutLM - Pre-training for document understanding
  • TrOCR - Transformer-based OCR

Audio-Language

  • CLAP - Contrastive Language-Audio Pre-training

Embeddings & Similarity

  • Sentence Transformers - Sentence embeddings
  • all-MiniLM - Efficient sentence embeddings
  • all-mpnet-base - High-quality sentence embeddings
  • E5 - Text embeddings by Microsoft
  • BGE - General embedding models
  • nomic-embed - Long context embeddings

Specialized Models

Code

  • CodeBERT - Pre-trained model for code
  • GraphCodeBERT - Code structure understanding
  • StarCoder - Code generation

Scientific

  • SciBERT - Scientific text
  • BioBERT - Biomedical text

Retrieval

  • ColBERT - Contextualized late interaction over BERT
  • DPR - Dense Passage Retrieval

Model Selection Tips

For Text Tasks

  • Small & Fast: DistilBERT, MobileBERT
  • Balanced: BERT-base, RoBERTa-base
  • High Accuracy: RoBERTa-large, DeBERTa-v3-large
  • Multilingual: XLM-RoBERTa, mBERT

For Vision Tasks

  • Mobile/Browser: MobileNet, EfficientNet-B0
  • Balanced: DeiT-base, ConvNeXT-tiny
  • High Accuracy: Swin-large, DINOv2-large

For Audio Tasks

  • Speech Recognition: Whisper-tiny (fast), Whisper-large (accurate)
  • Audio Classification: Audio Spectrogram Transformer

For Multimodal

  • Vision-Language: CLIP (general), Florence-2 (comprehensive)
  • Document AI: Donut, LayoutLM
  • OCR: TrOCR

Finding Models on Hugging Face Hub

Search for compatible models:

https://huggingface.co/models?library=transformers.js

Filter by task:

https://huggingface.co/models?pipeline_tag=text-classification&library=transformers.js

Check for ONNX support by looking for onnx/ folder in model repository.

Source: SKILL.md on GitHub

Third-party checks

No alerts16d5 checks · Risk SAFE
  • Gen Agent Trust Hub16d

    This skill provides comprehensive documentation and examples for using Transformers.js to run machine learning models. All external dependencies and model resources originate from well-known registries or trusted vendor domains, following industry-standard practices for client-side machine learning.

  • Socket16d

    No alerts

  • Snyk16d

    Risk: LOW · No issues

  • Runlayer6mo

    5/7 files flagged

  • ZeroLeaks5mo

    Score: 93/100 · 2 sections analyzed

Provenance

Signed by skilld at 4db6015. This ties the file your Agent reads to that commit on GitHub. It does not review the instructions.

Last checked against GitHub last week.

Activeupdated 6 months ago

Capability

Other metadata
metadata
{
  "author": "huggingface",
  "version": "4.x",
  "category": "machine-learning",
  "repository": "https://github.com/huggingface/transformers.js"
}
compatibility
Requires Node.js 18+ (or compatible Bun/Deno runtime) or modern browser with ES modules support. WebGPU requires runtime and hardware support; WASM is the broad fallback. Internet access is needed for downloading models from Hugging Face Hub (optional if using local models).

Topics

  • TypeScript
  • transformers.js
  • machine-learning
  • nlp
  • computer-vision
  • javascript
  • huggingface
  • text-classification
  • object-detection
  • speech-recognition
  • image-classification

README badge

README badge for huggingface/skills/transformers-js

What it does

Runs state-of-the-art ML models directly in JavaScript and TypeScript across browsers and Node.js/Bun/Deno using Transformers.js. Supports NLP tasks (text classification, translation, summarization), computer vision (image classification, object detection), audio (speech recognition), and multimodal workflows with models from Hugging Face Hub.

Generated from the current SKILL.md.

Frequently asked

Does this work in the browser?
Yes. Transformers.js runs in modern browsers with ES modules support and can be loaded via CDN. Models run client-side using WebGPU (if available) or WASM as a fallback, with no backend server required.
What runtimes does this support?
Node.js 18+, Bun, Deno, and modern browsers. WebGPU requires runtime and hardware support; WASM is the broad fallback for CPU inference.
Do I need to download models ahead of time?
No. Models are downloaded automatically from Hugging Face Hub on first use. Internet access is required unless you configure local models.
What ML tasks does this support?
NLP (text classification, translation, summarization, question-answering, token classification), computer vision (image classification, object detection, segmentation, depth estimation), audio (speech recognition, audio classification, text-to-speech), and multimodal tasks (image captioning, document QA).
Is memory management required?
Yes. All pipelines must be disposed with `pipe.dispose()` when finished to prevent memory leaks.

Generated from the current SKILL.md. These answers refresh after source changes.