One-Click Deployment

Float16's One-Click Deployment lets you deploy vLLM models instantly. Select from preset models or add custom HuggingFace models.

Overview

One-Click Deployment provides:

• vLLM Framework: High-throughput LLM serving with PagedAttention
• Preset Models: Optimized settings for popular models
• Custom Models: Deploy any HuggingFace model
• Endpoint Proxy: Secure access via proxy URLs

Deploying a Model

1. Navigate to GPU Instance > Create Instance
2. Select the One-Click Deployment tab
3. Enter a project name (optional)
4. Select instance type (e.g., H100)
5. Choose a model:
   • Preset Models: Select from the model catalog
   • Custom Model: Enter a HuggingFace model ID
6. Configure volume size (50-10000 GB)
7. Click Create Instance

Preset Models

Choose from pre-configured models optimized for vLLM:

Model Capabilities

• text: Text generation and chat
• vision: Image understanding
• reasoning: Advanced reasoning capabilities
• tools: Function/tool calling support
• grammar: Structured output support
• typhoon-ocr: Thai/English document OCR

Custom Models

Deploy any compatible HuggingFace model:

1. Select the Custom Model tab
2. Enter the model ID in format: organization/model-name
   • Example: meta-llama/Llama-3.3-70B-Instruct
3. Click Verify Model to check compatibility
4. Configure volume size (ensure sufficient space for model weights)
5. Click Create Instance

Endpoint Proxy

Access your deployed model via secure proxy endpoints:

• Format: https://proxy-instance.float16.cloud/{task_id}/{port}/{path}
• Ports: 3000-4000 supported
• Compatible with: vLLM, custom APIs, Jupyter

Using with OpenAI SDK

from openai import OpenAI

client = OpenAI(
    api_key="YOUR_API_KEY",
    base_url="https://proxy-instance.float16.cloud/{task_id}/3000/v1"
)

response = client.chat.completions.create(
    model="your-model-name",
    messages=[
        {"role": "user", "content": "Hello!"}
    ]
)

print(response.choices[0].message.content)

vLLM Playground

Test your deployed models with the interactive playground:

• Tool Calling: Test function calling with example tools
• Structured Outputs: JSON Schema, Regex patterns, Choice constraints
• Typhoon OCR: Extract text from Thai/English documents
• View Code: Copy Python, cURL, or JSON examples

Pricing

View current pricing at GPU Instance > Pricing.

Instance Lifecycle

Manage your deployment:

View and manage instances at GPU Instance > Instances.

Next Steps

• GPU Platform Overview - Learn about Base VM instances
• Volumes & Storage - Manage persistent storage