Skip to main content

Deployment Overview

Once you have a recipe or custom configuration ready, Emmy handles provisioning the GPU server and launching the inference backend.

Backends​

vLLM​

High-throughput inference engine with PagedAttention. Best for maximizing throughput on single or multi-GPU setups.

SGLang​

Structured generation engine. Best for complex multi-turn workloads and structured outputs.

Deployment flow​

  1. Select a recipe or custom configuration
  2. Choose a GPU type (or let Emmy recommend one based on your benchmark results)
  3. Launch — Emmy provisions the server and starts the inference endpoint
  4. Get an OpenAI-compatible API endpoint to send requests to