Fly Proxy autostop/autostart

With Fly Proxy autostop/autostart, you can scale your app for peak loads and run those Machines only when needed to save on resource costs; you don’t pay for Machine CPU and RAM when they’re in a stopped or suspended state. Fly Proxy automatically stops (or suspends) and starts existing Machines based on incoming requests to your app, and you have the option to keep a minimum number of machines running at all times in your primary region.

Autostop/autostart works for Fly Apps with a service configured in the fly.toml file, which generally covers publicly available apps that accept requests from the internet. But you can also use autostart/autostart for private apps by setting up services and using a Flycast private IPv6 address. See Flycast - Private Fly Proxy Services and Autostop/autostart private apps.

For apps that shut down automatically when idle and don’t need autostop, the Fly Proxy can still restart your app’s Machines when there’s traffic.

Learn how to configure autostop/autostart.

How Fly Proxy autostop/autostart works

Fly Proxy runs a process to determine whether an app has excess capacity every few minutes.

Fly Proxy process to stop or suspend Machines

When auto_stop_machines is set to "stop" or "suspend" in your fly.toml, the proxy looks at Machines running in a single region and uses the concurrency soft_limit setting for each Machine to determine if there’s excess capacity. If the proxy decides there’s excess capacity, it stops or suspends exactly one Machine. The proxy repeats this process every few minutes, stopping or suspending only one Machine per region, if needed, each time.

If you have the kill_signal and kill_timeout options configured in your fly.toml file, then Fly Proxy uses those settings when it stops a Machine.

Fly Proxy determines excess capacity per region as follows:

• If there’s more than one Machine in the region:
  • the proxy determines how many running Machines are over their soft_limit setting and then calculates excess capacity: excess capacity = num of machines - (num machines over soft limit + 1)
  • if excess capacity is 1 or greater, then the proxy stops or suspends one Machine
• If there’s only one Machine in the region:
  • the proxy checks if the Machine has any traffic
  • if the Machine has no traffic (a load of 0), then the proxy stops or suspends the Machine

Fly Proxy process to start Machines

When auto_start_machines = true in your fly.toml, the Fly Proxy restarts a Machine in the nearest region when required.

Fly Proxy determines when to start a Machine as follows:

• The proxy waits for a request to your app.
• If all the running Machines are above their soft_limit setting, then the proxy starts a stopped or suspended Machine in the nearest region (if there are any stopped or suspended Machines).
• The proxy routes the request to the newly started Machine.

How Fly Proxy behaves without autostart/autostop

Fly Proxy still routes requests to your app’s Machines even when auto_stop_machines is not set, or when auto_start_machines is explicitly set to false in your fly.toml file.

• Machines stay running unless they exit on their own or are stopped during a deploy or scaling operation.
• Fly Proxy does not check for excess capacity or stop Machines automatically, even if they are idle.
• If auto_start_machines is explicitly set to false, the proxy only routes traffic to Machines that are already running. If no Machines are running, the proxy does not start one and the request will fail.
• After a Machine starts, there may be a short delay before all proxy nodes recognize that it is running. During this time, requests sent to the Machine might fail if the proxy tries to route traffic too early.
• If a Machine becomes unreachable, such as failing repeated internal pings over Fly’s private IPv6 network, it may be marked as unhealthy and stopped. Any requests already routed to that Machine will fail, since it’s no longer considered valid for the app.

We recommend setting auto_start_machines = true and auto_stop_machines = "stop" or "suspend" in your fly.toml file to reduce idle resource usage and improve availability for apps that need to scale based on demand.

Related topics

• Configure autostop/autostart for Fly Launch apps
• Autostop/autostart private apps
• Fly Proxy features