omniserve-native

Native C inference: embedded llama.cpp text/ModernBERT embeddings and stable-diffusion.cpp, or pooled Gemma, image, 3D, multimodal, TTS and STT workers behind weighted tier-priority admission. Spec: openapi.json

Auth

Optional. When the server is started with OMNISERVE_NATIVE_SECRET, send it as a secret header or Authorization: Bearer. Priority via X-Omniserve-Tier: paid | sub | free | background.

Chat

POST/v1/chat/completions
curl localhost:8791/v1/chat/completions -d '{
  "messages": [{"role": "user", "content": "hello"}],
  "max_tokens": 128, "stream": true
}'

Streaming uses SSE chunks in the OpenAI chat.completion.chunk shape, ending with data: [DONE]. Non-streaming returns chat.completion with usage, cached_prompt_tokens, and timings.elapsed_ms.

POST/v1/completions
POST/api/v1/generate — text-generator.io compatibility
POST/api/v1/generate-large — large-generation alias
POST/api/v1/autocomplete — low-latency alias
POST/api/v1/summarization

Embeddings

POST/api/v1/feature-extraction — flat ModernBERT vector
POST/v1/embeddings — OpenAI response shape

Images

POST/v1/images/generations
curl localhost:8791/v1/images/generations -d '{
  "prompt": "a cabin in a snowstorm", "negative_prompt": "blurry", "size": "768x768", "steps": 9, "seed": 42,
  "teleport": true, "teleport_start_step": 8
}'
POST/v1/images/segmentations — SAM2 click-based object mask via the scale-to-zero image-editor pool
POST/v1/images/text-layers — OCR a text box, estimate its typography, and return an erase mask
POST/v1/images/edits — regenerate only a supplied mask via the same isolated pool
POST/v1/images/backgrounds — scene art via Z-Image; always background tier
curl localhost:8791/v1/images/backgrounds -o scene.webp -d '{
  "prompt": "flat vector reef background, no text", "size": "1280x720"
}'
GET/v1/images/backgrounds/style — restyle an existing backdrop; background tier
POST/v1/images/background-removals — BiRefNet transparent WebP/PNG via pooled C relay
curl localhost:8791/v1/images/background-removals -o cutout.png -d '{
  "image_url": "https://example.com/chair.webp"
}'

Ask for the backdrop the matte solve already computed, or swap it for a new one. More than one image means a JSON answer instead of raw bytes.

curl localhost:8791/v1/images/background-removals -d '{
  "image_url": "https://example.com/chair.webp",
  "return_background": true, "background": "#ffffff"
}'
POST/v1/images/background-removals/jobs — queue + poll; identical inputs are answered from the WebP cache

A generated backdrop only runs here: diffusion holds a permit for its whole duration, so the synchronous route refuses background_prompt rather than parking on an interactive slot.

curl localhost:8791/v1/images/background-removals/jobs -d '{
  "image_url": "https://example.com/chair.webp",
  "background_prompt": "a sunlit loft, no subject"
}'
GET/v1/images/background-removals/jobs/{id} — job status
POST/v1/images/hair-layers — SAM2/silhouette front and back hair layers
curl localhost:8791/v1/images/hair-layers -d '{
  "image_url": "https://example.com/waifu.webp"
}'
POST/v1/images/hair-layers/jobs — queue + poll hair split
GET/v1/images/hair-layers/jobs/{id} — hair job status
POST/v1/depth-estimations — Depth Anything V2 Small, 16-bit PNG plus preview, 1 credit
curl localhost:8791/v1/depth-estimations -d '{
  "image_url": "https://example.com/room.webp", "output_format": "png16"
}'
POST/v1/images/foreground-generations/jobs — text-to-image + BiRefNet in one queued stage
curl localhost:8791/v1/images/foreground-generations/jobs -d '{
  "prompt": "full-body deckhand on a plain grey backdrop",
  "width": 768, "height": 1024, "seed": 42
}'
GET/v1/images/foreground-generations/jobs/{id} — composite job status and cutout

3D

POST/v1/3d/generations — TRELLIS.2/Pixal3D worker relay; always background tier
GET/v1/3d/assets/{job}/model.glb — locally retained result
curl localhost:8791/v1/3d/generations -d '{
  "image_url": "https://example.com/object.webp",
  "model": "microsoft/TRELLIS.2-4B", "resolution": 512,
  "texture_size": 1024, "publish": {"r2": true, "searchable": true}
}'

Motion

POST/v1/motion/video — GEM-X video-to-motion worker relay; returns a 27-joint ARDY-format clip, optional native SOMA rotations, stage timings and gpu_seconds
curl localhost:8791/v1/motion/video -d '{
  "video_url": "https://example.com/backflip.mp4", "static_cam": true
}'

Animation

POST/v1/animations/generations — NVIDIA ACE/Animation Graph worker relay; always background tier
POST/v1/expression-pack — LivePortrait six-frame blink and mouth pack through its own background lane
curl localhost:8791/v1/animations/generations -d '{
  "prompt": "cautious walk and wave", "rig_type": "biped",
  "bones": ["Hips", "Spine", "Chest"],
  "publish": {"r2": true, "searchable": true}
}'

Audio

POST/v1/audio/speech — TTS bytes
POST/v1/audio/transcriptions — multipart STT
POST/v1/training/jobs — queue consent-gated ASR training
POST/v1/training/jobs/run — exclusive background segment; checkpoints on serving demand
POST/api/v1/generate_speech — text-generator.io TTS
GET/api/v1/speech/catalog — voices and languages
POST/api/v1/audio/transcribe — worker STT
POST/api/v1/audio-file-extraction — multipart STT
POST/api/v1/audio-extraction — URL STT

Multimodal

POST/api/v1/image-caption
POST/api/v1/video-question
POST/api/v1/multimodal-generate
POST/api/v1/voice-chat

Ops

GET/health
GET/status — VRAM, permits, queues, and upstream connection reuse
GET/v1/models

Run

OMNISERVE_NATIVE_LLM_GGUF=/models/gemma-roleplay-v2-q8_0.gguf \
OMNISERVE_NATIVE_EMBEDDING_GGUF=/models/modernbert-base-q8_0.gguf \
OMNISERVE_NATIVE_SD_MODEL=/models/sdxl-turbo.safetensors \
./omniserve-native --port 8791
OMNISERVE_NATIVE_LLM_UPSTREAM=http://127.0.0.1:8300 \
OMNISERVE_NATIVE_IMAGE_UPSTREAM=http://127.0.0.1:8100 \
OMNISERVE_NATIVE_TTS_UPSTREAM=http://127.0.0.1:9083 \
OMNISERVE_NATIVE_STT_UPSTREAM=http://127.0.0.1:9083 \
OMNISERVE_NATIVE_SLOTS=4 ./omniserve-native --port 8791

Front with omniserve (Python) by setting OMNISERVE_PROXY_PROXY_LLM=http://127.0.0.1:8791 to keep its catalog, eviction, and fleet features while this server owns the hot path.