Scaling

Scale your applications horizontally with manual replica counts or automatic scaling rules based on resource usage.

Manual Scaling

The simplest way to scale is to set a fixed number of replicas. Each replica is an identical container running your application. sh0 distributes incoming requests across all healthy replicas.

1. Navigate to your app in the dashboard.
2. Open Settings → Scaling.
3. Set the Replica Count to your desired number.
4. Click Apply. sh0 will spin up or tear down containers to match.

Auto-Scaling Rules

Auto-scaling dynamically adjusts the number of replicas based on real-time resource usage. When usage exceeds a threshold, sh0 adds replicas. When it drops, replicas are removed.

CPU Threshold

Set a target CPU utilization percentage. When the average CPU usage across all replicas exceeds this threshold for a sustained period (default: 60 seconds), sh0 adds a new replica.

Memory Threshold

Similar to CPU, you can set a memory utilization threshold. When average memory usage exceeds the threshold, sh0 scales up. This is especially useful for memory-intensive applications like Node.js or Java services.

Min & Max Replicas

Auto-scaling operates within bounds that you define:

• Min Replicas: The minimum number of containers always running. Set to at least 1 for high availability, or 2 for zero-downtime deployments.
• Max Replicas: The upper limit for scaling. This prevents runaway scaling from consuming all server resources.

{
  "min_replicas": 2,
  "max_replicas": 10,
  "cpu_threshold": 70,
  "memory_threshold": 80,
  "scale_up_cooldown_seconds": 60,
  "scale_down_cooldown_seconds": 300
}

Load Balancing Across Replicas

sh0 uses Caddy as its reverse proxy to distribute traffic across replicas. The load balancing strategy is round-robin by default, ensuring even distribution of requests.

Key behaviors:

• Health checks: Unhealthy replicas are automatically removed from the load balancer pool.
• Graceful drain: When scaling down, existing connections are drained before the container is stopped.
• Session affinity: Not enabled by default. If your app requires sticky sessions, configure it in the app settings.
• WebSocket support: WebSocket connections are properly proxied and maintained across replicas.

Scaling via API

You can manage scaling programmatically through the REST API. This is useful for CI/CD pipelines or custom automation.

Set replica count:

curl -X PUT https://your-server:9000/api/apps/my-app/scaling \
  -H "Authorization: Bearer YOUR_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"replicas": 4}'

Configure auto-scaling:

curl -X PUT https://your-server:9000/api/apps/my-app/scaling \
  -H "Authorization: Bearer YOUR_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "auto_scaling": true,
    "min_replicas": 2,
    "max_replicas": 8,
    "cpu_threshold": 75,
    "memory_threshold": 85
  }'

Get current scaling status:

curl https://your-server:9000/api/apps/my-app/scaling \
  -H "Authorization: Bearer YOUR_TOKEN"

Best Practices

• Make your app stateless: Store sessions in Redis or a database, not in memory. Stateless apps scale effortlessly.
• Set resource limits: Define CPU and memory limits per container so one replica cannot starve others.
• Use health checks: Configure a health check endpoint so sh0 can detect and replace unhealthy replicas automatically.
• Start with min replicas = 2: This ensures zero-downtime during deployments (blue-green pattern).
• Monitor before scaling: Use the metrics dashboard to understand your baseline resource usage before setting auto-scaling thresholds.
• Test scale-down behavior: Ensure your app handles graceful shutdown (SIGTERM) properly to avoid dropped requests.