App configuration (fly.toml)

You can configure an app for deployment on Fly.io using a fly.toml file. Configuration of builds, environment variables, internet-exposed services, disk mounts, and release commands go here. fly.toml configuration applies for apps managed by Fly Launch commands.

App configuration files

Most of the time you’ll use the fly.toml file that you get automatically when you run fly launch to create an app. There are also ways to re-use config files and to generate them in different formats.

Ways to get a config file

  • Run flyctl launch in your project source directory to create a new app and fly.toml file automatically.
  • Run flyctl config save -a <existing-app-name> to download a fly.toml file from an existing app.
  • Create one by hand with help from this reference, or copy and modify an example file.

Config file formats

The default configuration file generated by Fly Launch is called fly.toml.

TOML is a simple configuration file format. If you’re a VSCode user, then you can install the Even Better TOML extension for automatic fly.toml validation and hints drawn from this documentation.

If you want your config file in JSON or YAML format, use the --json and --yaml options in flyctl commands:

  • Add the --yaml or --json options to fly launch.
  • For existing apps, run fly config save -a <existing-app-name and specify --yaml or --json. Remove your fly.toml file when done as it takes precedence.
  • Run fly config show --local to preview different formats in your terminal. fly config show defaults to JSON, but you can also specify --toml or --yaml options.

The app name

The first key/value in any fly.toml file is the application name. This will also be used to create the host name that the application will use by default. For example:

app = "restless-fire-6276"

Whenever flyctl is run, it will look for a fly.toml file in the current directory and use the application name in that file. This behavior can be overridden by using the -a flag to set the application name, or on some commands (such as deploy) by using a -c flag to point to a different fly.toml file.

Primary region

fly deploy uses the primary_region option to determine where to create new Machines, as well as to set the PRIMARY_REGION environment variable within the Machines it deploys. Replace ord with the three-letter code for your Fly Region of choice.

primary_region = "ord"

Runtime options

The following options are available to control the lifecycle of a running application. These are optional and can be placed at the top level of the fly.toml file:

kill_signal option

When shutting down a Fly Machine, Fly.io sends a SIGINT signal to the running process by default. Typically this triggers a hard shutdown option on most applications. The kill_signal option lets you override that with a different signal so that you can trigger a softer, less disruptive shutdown: SIGTERM, SIGQUIT, SIGUSR1, SIGUSR2, SIGKILL, or SIGSTOP.

For example, to set the kill signal to SIGTERM, you would add:

kill_signal = "SIGTERM"

kill_timeout option

Note: kill_timeout settings should be considered best-effort, and your app needs to be prepared to handle shorter stopping times

This sets how long Fly.io waits (in seconds) after sending the kill_signal before moving on to a forced shutdown. Set kill_timeout to a value that gives your app enough time to exit gracefully. The default is 5 seconds. You can set it up to a maximum of 300 seconds (5 minutes).

For example, to set the timeout to two minutes:

kill_timeout = 120

How the shutdown sequence works

These options come into play during controlled shutdowns such as:

  • When using fly deploy
  • When using fly machine stop
  • When a machine stops due to auto_stop_machines

The sequence looks like this:

  1. Fly sends the kill_signal to the main process (defined by your Dockerfile’s ENTRYPOINT or CMD, or overridden via --entrypoint).

  2. It waits for up to kill_timeout seconds.

  3. Then it sends SIGTERM unconditionally.

This gives your app a chance to shut down cleanly: finish requests, close connections, stop taking new work. It’s designed for apps that can wrap up in under five minutes.

Many workers will re-queue unfinished jobs so another machine can pick them up. If your app doesn’t support that, or needs longer to shut down, you’ll want to plan around that.

You can watch shutdown behavior by watching fly logs during a deploy or idle stop.

Console command

The command to run when you run the fly console command. Configure the console_command field with the command that opens your framework’s console, and then fly console will run that command automatically in a new, dedicated Machine. The new Machine is configured with the image and environment from your app’s latest release, but your app isn’t started, and no traffic will be routed to it. The Machine gets destroyed when you exit the console.

Here’s an example of a console command for Django:

console_command = "/code/manage.py shell"

swap_size_mb option

Setting this option enables Linux swap on Machines. A swap partition is created with size swap_size_mb, expressed in megabytes.

Swapping to disk can help avoid out-of-memory crashes on brief spikes in memory use. Swap is much slower than RAM, so if performance is important, a better solution is to increase the Machine memory with fly scale memory.

swap_size_mb = 512

The build section

The optional build section contains key/values concerned with how the application should be built. You can read more about builders in Builders and Fly

builder

[build]
  builder = "paketobuildpacks/builder:base"
  buildpacks = ["gcr.io/paketo-buildpacks/nodejs"]

The builder “builder” uses CNB Buildpacks and Builders to create the application image. These are third party toolkits which can use Heroku compatible build processes or other tools. The tooling is all managed by the buildpacks and buildpacks are assembled into CNB Builders - images complete with the buildpacks and OS to run the tool chains.

In our example above, the builder is being set to use Paketo’s all-purpose builder with the NodeJS buildpack.

Specify a Docker image

[build]
  image = "flyio/hellofly:latest"

The image builder is used when you want to immediately deploy an existing public image. When deployed, there will be no build process; the image will be prepared and uploaded to Fly.io infrastructure as is. This option is useful if you already have a working Docker image you want to deploy on Fly.io or you want a well known Docker image from a repository to be run.

Specify a Dockerfile

[build]
  dockerfile = "Dockerfile.test"

dockerfile accepts a relative path to a Dockerfile, or a URL. By default, flyctl looks for Dockerfile in the application root.

Gotchas:

  • This option will not change the Docker context path, which is set to the project root directory by default. If you want the Dockerfile to use its containing directory as the context root, use fly deploy <directory>.
  • When specifying a local Dockerfile, make sure it’s not excluded from the Docker build context in your .dockerignore.

Specify a Docker ignore file

[build]
  ignorefile = "/path/.dockerignore"

ignorefile accepts a relative path to a .dockerignore file. By default, flyctl looks for the .dockerignore file in the working directory.

Specify a multistage Docker build target

[build]
  build-target = "test"

If your Dockerfile has multiple stages, you can specify one as the target for deployment. The target stage must have a CMD or ENTRYPOINT set.

Specify Docker build arguments

You can pass build-time arguments to both Dockerfile and Buildpack builds using the [build.args] sub-section:

[build.args]
  USER="plugh"
  MODE="production"

You can also pass build arguments to flyctl deploy using --build-arg. Command line args take priority over args with the same name in fly.toml.

Note that build arguments are not available in the runtime container. If you need build information at runtime - like a Git revision - store it in a file at build time, like:

RUN echo $GIT_REVISION > REVISION

Likewise, application environment variables and secrets are not available to builds.

The deploy section

This section configures deployment-related settings such as the release command or deployment strategy.

Run one-off commands before releasing a deployment

The release_command lets you run a one-off task, like a database migration, before any of your deployed Machines are created or updated. It runs in a temporary Machine using the newly built image, once per deploy attempt. This is a common pattern for frameworks like Rails or Phoenix, but it’s not limited to them. You can use it for any setup task that needs to run once, not on every app boot.

If the command fails, the deploy will stop. The goal is to make deployment safer: do the risky part first, and only continue if it succeeds.

To run a command in a temporary Machine before the release is deployed, define a release_command:

[deploy]
  release_command = "bin/rails db:prepare"

The release_command value replaces CMD in the temporary Machine. This is most commonly used for database migrations, but can be any task that prepares your app for the new release. Note that the Docker image’s ENTRYPOINT is not overridden by the release_command; ENTRYPOINT always runs.

By default, the temporary Machine has full access to the network, environment variables and secrets, but not to persistent volumes. Changes made to the file system on the temporary Machine will not persist. It has no volumes attached, and the Machine is destroyed after the command completes. The building/compiling of your project should be done in your Dockerfile. If you need to modify persistent volumes or configure your application, consider making use of CMD or ENTRYPOINT in your Dockerfile, or of a process group command.

The temporary Machine inherits the size from the largest Machine in the default process group of the app as of flyctl v0.0.508 (they also default larger on empty/new apps, using the Machine shared-cpu-2x preset). The default process group is defined as either the app process, or the first process group in alphabetical order (if an app process is not present).

It is also possible to explicitly define the Machine size by setting [deploy.release_command_vm], for example:

[deploy]
  release_command = "bin/rails db:prepare"

  [deploy.release_command_vm]
    size = "performance-1x"
    memory = "8gb"

A non-zero exit status from this command will stop the deployment. fly deploy will display logs from the command. Logs are available via fly logs as well.

To ensure the command runs in a specific region, such as dfw, set PRIMARY_REGION = 'dfw' in your application environment in fly.toml or with fly deploy -e PRIMARY_REGION=dfw. Setting PRIMARY_REGION is important if you’re running database replicas in multiple regions.

The environment variable RELEASE_COMMAND=1 is set within the temporary release Machine. You can use this to define behavior in your Dockerfile’s ENTRYPOINT that’s conditional on being run on a release Machine.

By default, the release command has 5 minutes to complete before it is killed. It is possible to increase or decrease the timeout with fly deploy --release-command-timeout=10m or by setting release_command_timeout = "10m" in [deploy] section of fly.toml.

For more context, see our Seamless Deployments guide.

Things to know about release_command:

  • Replaces CMD, but not ENTRYPOINT
  • Runs in a temporary Machine with no volumes
  • Inherits environment, secrets, and network config
  • Fails the deploy on a non-zero exit code
  • Times out after 5 minutes by default (configurable)
  • Logs are streamed to fly deploy and fly logs

Picking a deployment strategy

[deploy]
  strategy = "bluegreen"

strategy controls the way a new release replaces the previous release. Different strategies offer trade-offs between downtime and reliability.

strategy may also be specified at deploy time with flyctl deploy --strategy.

The available strategies are:

rolling: The default strategy for apps with or without volumes. One by one, each running Machine is taken down and replaced by a new release Machine.

immediate: Replace all Machines with new releases immediately without waiting for health checks to pass. This is useful in emergency situations where you’re confident about release health and can’t wait for health checks.

canary: Boots a single, new Machine with the new release, verifies its health, and then proceeds with a rolling restart strategy. The canary strategy cannot be used for Machines with attached volumes.

bluegreen: For every running Machine, a new one is booted alongside it in the same region. Once all of the new Machines pass health checks, traffic gets migrated to the new Machines and the old Machines are destroyed. If your app is scaled to 2 or more Machines, this strategy can reduce deploy time by running tasks in parallel. You need to configure at least one health check to use the bluegreen strategy. The bluegreen strategy cannot be used for Machines with attached volumes.

Note: If max-per-region is set to 1, then the default strategy is set to rolling. This happens because canary needs to temporarily run more than one Machine to work correctly. The bluegreen strategy will behave similarly with max-per-region set to 1.

Rolling deploy configuration

[deploy]
  strategy = "rolling"
  max_unavailable = 1

For rolling deploys, you can use max_unavailable to control how many Machines can be down at a time.

Whole number values, greater than or equal to one, specify exactly how many Machines can be down at a time. For example, a value of “1” would update one-at-a-time.

Fractional values, between zero and one, provide a percentage of Machines that can be down at a time. For example, the default value of 0.33 means that a maximum of one third of your Machines can be down during a deploy.

Dealing with timeout errors while deploying a new version

Every time fly deploy runs, it builds a new Docker image, pushes it to an internal registry, and then updates or creates Machines for the app. Sometimes the image is too large, or the platform is slower than usual pulling the new image layers, triggering the default 5 minute timeout waiting for the Machine to be in a started state.

The good thing is that this timeout can be controlled in two ways, with the --wait-timeout option, for example fly deploy --wait-timeout=10m, or as a more permanent fly.toml setting:

[deploy]
wait_timeout = "10m"

The env variables section

[env]
  LOG_LEVEL = "debug"
  RAILS_ENV = "production"
  S3_BUCKET = "my-app-production"

This optional section allows setting non-sensitive information as environment variables in the application’s runtime environment.

For sensitive information, such as credentials or passwords, use the secrets CLI command.

Env variable names are strictly case-sensitive and cannot begin with FLY_ (as this could clash with the runtime environment). Env values can only be strings.

Secrets take precedence over env variables with the same name.

Note: In Apps V2, the primary_region option sets the PRIMARY_REGION environment variable within a Machine and overrides any value set in the [env] section.

The http_service section

For apps that only need HTTP and HTTPS services, you can replace the [[services]] section with this simpler alternative.

An [http_service] section defines a service that listens on ports 80 and 443. Port 80 will have an HTTP handler. Port 443 will have a TLS and HTTP handler. You can configure additional services on different ports by adding [[services]] sections.

As with the more verbose [[services]], you can specify a list of processes to limit the service definition to Machines belonging to process groups that should receive HTTP requests.

[http_service]
  internal_port = 8080
  force_https = true
  auto_stop_machines = "stop"
  auto_start_machines = true
  min_machines_running = 0
  [http_service.concurrency]
    type = "requests"
    soft_limit = 200
    hard_limit = 250
  • processes: For apps with multiple processes. The process group that this service belongs to. Define process groups in the processes section.
  • internal_port: The port this service (and application) will use to communicate with clients. The default is 8080. We recommend applications use the default.
  • force_https: A Boolean which determines whether to enforce HTTP to HTTPS redirects.
  • auto_stop_machines: The action, if any, that Fly Proxy should take when the app is idle for several minutes. Options are "off", "stop", or "suspend". If "off", Fly Proxy will never stop Machines. If "stop", Fly Proxy stops Machines when the app has excess capacity. If "suspend", Fly Proxy suspends Machines (if possible) when the app has excess capacity. Starting a Machine from a suspended state is faster than starting a Machine from a stopped state, but there are some caveats. If there’s only one Machine in a region, then the Machine is stopped or suspended if it has no traffic. The Fly Proxy runs a process to automatically stop Machines every few minutes. The default if not set is "off".
  • auto_start_machines: Whether to automatically start an application’s Machines when a new request is made to the application and there’s no excess capacity, per region. If there’s only one Machine in a region, then it’s started whenever a request is made to the application. The default if not set is true.
  • min_machines_running: The number of Machines to keep running, in the primary region only, when auto_stop_machines is "stop" or "suspend". The default if not set is 0.

We recommend configuring auto_stop_machines and auto_start_machines so that they are both enabled or both disabled, to avoid having Machines that either never start or never stop. Learn more about how Fly Proxy autostop/autostart works, including how Fly Proxy determines excess capacity in a region using the soft_limit setting.

http_service.concurrency

The http_service concurrency section configures how to measure load for an application to inform Fly Proxy load balancing and autostop/autostart. The [http_service.concurrency] section has the same settings as [services.concurrency], which are:

  • type specifies what metric is used to determine when to auto start or stop, or when a given Machine should receive more or less traffic (load balancing). The two supported values are connections and requests.

connections: Load balance and scale based on the number of concurrent TCP connections. This is the default when unspecified. This is also the default when fly.toml is created with fly launch. A new connection is established for each HTTP request; consider the requests type for HTTP apps.

requests: Load balance and scale based on the number of HTTP requests. This is recommended for web services, because the Fly Proxy can pool and reuse connections for requests.

  • hard_limit : When a Fly Machine is at or over this number of concurrent connections or requests, the system will stop sending new traffic to that Machine. If unset, no hard_limit is enforced.
  • soft_limit : When a Fly Machine is at or over this number of concurrent connections or requests, the system will deprioritize sending new traffic to that Machine and only send it to that Machine if all other Machines are also at or above their soft_limit. This setting is also used to determine excess capacity for the autostop/autostart feature. If unset, the default is 20.

For more information about concurrency settings, see Guidelines for concurrency settings.

http_service.http_options.idle_timeout

Configure an idle-timeout for connections to your app.

  [http_service.http_options]
    idle_timeout = 600

http_service.http_options.response.pristine

Configures Fly Proxy to not add any Fly headers to HTTP responses. The following response headers won’t be added and won’t be modified if returned by the app:

  • Server
  • Via
  • Fly-Request-Id
  • Fly-Cache-Status

This is useful if you are building a platform on top of Fly and have your own load balancer that already adds similar response headers.

  [http_service.http_options.response]
    pristine = true

http_service.http_options.response.headers

Add or remove HTTP response headers.

In the following example, Example-Header will be removed from the final response that Fly Proxy sends, while Example-Header-1 and Example-Multi-Value and their values will be added to the response header.

  [http_service.http_options.response.headers]
    Example-Header = false
    Example-Header-1 = "value"
    Example-Multi-Value = ["value1", "value2"]

http_service.http_options.h2_backend

Indicates whether the app supports HTTP/2 cleartext (H2C) with prior knowledge or not.

In the following example, h2_backend is set to true, which allows the app to expose HTTP/2-only service, like gRPC, via http handler.

  [http_service.http_options]
    h2_backend = true

http_service.tls_options

Configure the TLS versions and ALPN protocols that Fly’s edge will use to terminate TLS for your application with:

[http_service.tls_options]
  alpn = ["h2", "http/1.1"]
  versions = ["TLSv1.2", "TLSv1.3"]
  default_self_signed = false
  • alpn: Array of strings indicating how to handle ALPN negotiations with clients.
  • versions: Array of strings indicating which TLS versions are allowed.
  • default_self_signed: When true, serve a self-signed certificate if no certificate exists. Default is false.

Fly.io can also terminate TLS only and pass through directly to your service. For more information, refer to services.ports.tls_options

http_service.checks

To configure health checks for your HTTP service, you can use the http_service.checks section. These checks expect a successful HTTP status in response (i.e, 2xx). Here is an example of a http_service.checks section:

[[http_service.checks]]
  grace_period = "10s"
  interval = "30s"
  method = "GET"
  timeout = "5s"
  path = "/"

Roughly translated, this section says every thirty seconds, perform an HTTP GET on the root path (e.g. http://appname.fly.dev/) looking for it to return an HTTP 200 status within five seconds. The parameters of a check are listed below.

Times are in milliseconds unless units are specified.

  • grace_period: The time to wait after a Machine starts before checking its health. Make sure this is long enough for your app to start up. For example, if your app takes 2 seconds to start up, give it some runway by setting grace_period to at least 3 seconds.
  • interval: The time between connectivity checks. There should be a balance between the interval and the grace period. If interval is long and grace_period is shorter than your app’s startup time, the health check will take too long, adding to your deployment time.
  • timeout: The maximum time a connection can take before being reported as failing its health check.
  • method: The HTTP method to be used for the check. If omitted, the default is get.
  • path: The path of the URL to be requested.
  • protocol: The protocol to be used (http or https). If omitted, the default is http.
  • tls_server_name: If the protocol is https, the hostname to use for TLS certificate validation.
  • tls_skip_verify: When true (and using HTTPS protocol) skip verifying the certificates sent by the server.
  • http_service.checks.headers: This is a sub-section of http_service.checks. It uses the key/value pairs as a specification of header and header values that will get passed with the check call.

The health check will not automatically follow any HTTP 301 or 302 redirect, so it will fail if it receives anything other than a 200 OK response. This may be an issue if your heath check specifies protocol = http (the default) but your app forces https. You may be able to avoid this redirect and continue using http for your health check by adding X-Forwarded-Proto = "https" to the [http_service.checks.headers] sub-section.

The http_service.machine_checks section

You can use the same machine_checks section for HTTP services as for services. The machine_checks section defines parameters for checking the health of a service.

[[http_service.machine_checks]]
  image = "curlimages/curl"
  entrypoint = ["/bin/sh", "-c"]
  command = ["curl http://[$FLY_TEST_MACHINE_IP] | grep 'Hello, World!'"]
  kill_signal = "SIGKILL"
  kill_timeout = "5s"

The services sections

The services sections define how Fly Proxy connects requests that hit an app’s public Anycast or private Flycast address to services running within Machines, and configure other Fly Proxy behavior for a service. If a service only needs HTTP and HTTPS on standard ports, you can use the less verbose http_service.

A services section itself is a table of tables in TOML, so it is delimited with double square brackets. Tables within [[services]] configure additional features and behaviors for the service, and are described in subsequent sections.

At least one services.ports entry is required for each services section, to set a port and handler for incoming requests.

An app can have:

  • No services section (and no http_service section): The application cannot be reached from the public internet, nor by a Flycast address; this is typical of apps that talk only over 6PN private networking to other apps on the same network.
  • One services section (or an http_service section): One internal port mapped to one or more external ports.
  • Multiple services sections (or an http_service section and one or more services sections): Mapping multiple internal ports to multiple external ports.

Example (note the double square brackets):

[[services]]
  internal_port = 8080
  protocol = "tcp"
  auto_stop_machines = "stop"
  auto_start_machines = true
  min_machines_running = 0

Settings:

  • processes: For apps with multiple process groups, the process group or groups that this service belongs to. Define process groups in the processes section.
  • internal_port : The port this service (and application) will use to communicate with clients. The default is 8080. We recommend applications use the default.
  • protocol : The protocol that this service will use to communicate. Typically tcp for most applications, but can also be udp.
  • auto_stop_machines: The action, if any, that Fly Proxy should take when the app is idle for several minutes. Options are "off", "stop", or "suspend". If "off", Fly Proxy will never stop Machines. If "stop", Fly Proxy stops Machines when the app has excess capacity. If "suspend", Fly Proxy suspends Machines (if possible) when the app has excess capacity. Starting a Machine from a suspended state is faster than starting a Machine from a stopped state, but there are some caveats. If there’s only one Machine in a region, then the Machine is stopped or suspended if it has no traffic. The Fly Proxy runs a process to automatically stop Machines every few minutes. The default if not set is "off".
  • auto_start_machines: Whether to automatically start an application’s Machines when a new request is made to the application and there’s no excess capacity, per region. If there’s only one Machine in a region, then it’s started whenever a request is made to the application. The default if not set is true.
  • min_machines_running: The number of Machines to keep running, in the primary region only, when auto_stop_machines is "stop" or "suspend". The default if not set is 0.

We recommend configuring auto_stop_machines and auto_start_machines so that they are both enabled or both disabled, to avoid having Machines that either never start or never stop. Learn more about how Fly Proxy autostop/autostart works, including how Fly Proxy determines excess capacity in a region using the soft_limit setting.

services.ports

For each services section in your fly.toml, i.e. for each external port you want to accept connections on, you need a services.ports section. The section is denoted by double square brackets like this:

  [[services.ports]]
    handlers = ["http"]
    port = 80
    force_https = true  # optional

This example defines an HTTP handler on port 80.

  • handlers : An array of strings, each string selecting a handler process to terminate the connection with at the edge. Here, the http handler will accept HTTP traffic and pass it on to the internal_port of the application, which we defined in the services section above. For the list of available handlers, and how they manage network traffic, see the Public Network Services documentation.
  • port : An integer representing the external port to listen on (ports 1-65535).
  • start_port : For a port range, the first port to listen on.
  • end_port : For a port range, the last port to listen on.
  • force_https: A Boolean which determines whether to enforce HTTP to HTTPS redirects.

You can have more than one services.ports section. The default configuration, for example, contains two. We’ve already seen one above. The second one defines an external port 443 for secure connections, using the tls handler.

  [[services.ports]]
    handlers = ["tls", "http"]
    port = 443

For UDP applications, make sure to bind the application to the same port as defined in the relevant services.ports section. For example, the configuration below will listen on port 5000 and the application will need to bind to fly-global-services:5000 to receive traffic. Leave handlers unset for UDP services.

[[services]]
  internal_port = 5000
  protocol = "udp"
  [[services.ports]]
    port = 5000

Instead of using multiple port definitions, you can specify a range of ports. For example, the configuration below will listen on the ports 8080, 8081, 8082, 8083, 8084 and 8085:

[[services.ports]]
  handlers = ["tls"]
  start_port = 8080
  end_port = 8085

services.ports.http_options.idle_timeout

Configure an idle-timeout for connections to your app.

  [services.ports.http_options]
    idle_timeout = 600

services.ports.http_options.response.pristine

Configures Fly Proxy to not add any Fly headers to HTTP responses. The following response headers won’t be added and won’t be modified if returned by the app:

  • Server
  • Via
  • Fly-Request-Id
  • Fly-Cache-Status

This is useful if you are building a platform on top of Fly and have your own load balancer that already adds similar response headers.

  [services.ports.http_options.response]
    pristine = true

services.ports.http_options.response.headers

Add or remove HTTP response headers.

In the following example, Example-Header will be removed from the final response that Fly Proxy sends, while Example-Header-1 and Example-Multi-Value and their values will be added to the response header.

  [services.ports.http_options.response.headers]
    Example-Header = false
    Example-Header-1 = "value"
    Example-Multi-Value = ["value1", "value2"]

services.ports.http_options.h2_backend

Indicates whether the app supports HTTP/2 cleartext (H2C) with prior knowledge or not.

In the following example, h2_backend is set to true, which allows the app to expose HTTP/2-only service, like gRPC, via http handler.

  [services.ports.http_options]
    h2_backend = true

services.ports.proxy_proto_options

Configure the version of the PROXY protocol that your app accepts. Version 1 is the default.

For example:

[[services.ports]]
  handlers = ["proxy_proto"]
  port = 5000
  proxy_proto_options = { version = "v2" }
  • version : A string to indicate that the TCP connection uses PROXY protocol version 2. The default when not set is version 1.

services.ports.tls_options

Configure the TLS versions and ALPN protocols that Fly’s edge will use to terminate TLS for your application with:

  [[services.ports]]
    handlers = ["tls", "http"]
    port = 443
    tls_options = { "alpn" = ["h2", "http/1.1"], "versions" = ["TLSv1.2", "TLSv1.3"] }
  • alpn : Array of strings indicating how to handle ALPN negotiations with clients.
  • versions : Array of strings indicating which TLS versions are allowed.
  • default_self_signed: When true, serve a self-signed certificate if no certificate exists. Default is false.

Fly.io can also terminate TLS only and pass through directly to your service. This works for a variety of applications that can benefit from offloading TLS termination and accept the unencrypted connection.

One use case is applications using HTTP/2, like gRPC. Fly’s edge terminates TLS and sends h2c (HTTP/2 without TLS) directly to your application through our backhaul. The config below will negotiate HTTP/2 with clients, and then send h2c to the application:

  [[services.ports]]
    handlers = ["tls"]
    port = 443
    tls_options = { "alpn" = ["h2"] }

services.concurrency

The services concurrency sub-section configures how to measure load for an application to inform Fly Proxy load balancing and autostop/autostart.

This section is a simple list of key/values, so the section is denoted with single square brackets like so:

  [services.concurrency]
    type = "connections"
    hard_limit = 25
    soft_limit = 20

type specifies what metric is used to determine when a given Machine has reached a concurrency limit. The two supported values are connections and requests.

connections: Load balance and scale based on the number of concurrent TCP connections. This is the default when unspecified. This is also the default when fly.toml is created with fly launch. A new connection is established for each HTTP request; consider the requests type for HTTP apps.

requests: Load balance and scale based on the number of HTTP requests. This is recommended for web services, because the Fly Proxy can pool and reuse connections for requests.

  • hard_limit : When a Fly Machine is at or over this number of concurrent connections or requests, the system will stop sending new traffic to that Machine. If unset, no hard_limit is enforced.
  • soft_limit : When a Fly Machine is at or over this number of concurrent connections or requests, the system will deprioritize sending new traffic to that Machine and only send it to that Machine if all other Machines are also at or above their soft_limit. This setting is also used to determine excess capacity for the autostop/autostart feature. If unset, the default is 20.

For more information about concurrency settings, see Guidelines for concurrency settings.

services.tcp_checks

When a service is running, Fly Proxy can check up on it by connecting to a port. The services.tcp_checks section defines parameters for those checks. For example, the default tcp_checks looks like this:

  [[services.tcp_checks]]
    grace_period = "1s"
    interval = "15s"
    timeout = "2s"

Times are in milliseconds unless units are specified.

  • grace_period: The time to wait after a Machine starts before checking its health. Make sure this is long enough for your application to start up. For example if your app takes 2 seconds to start up, give it some runway by setting this to at least 3 seconds.
  • interval: The time between connectivity checks. If the interval is long, and the grace_period is shorter than your app’s startup time, then the health check will take longer and will add to your deployment time.
  • timeout: The maximum time a connection can take before being reported as failing its health check.

services.http_checks

Another way of checking a service is running is through HTTP checks as defined in the services.http_checks section. These checks are more thorough than services.tcp_checks as they require not just a connection but a successful HTTP status in response (i.e, 2xx). Here is an example of a services.http_checks section:

  [[services.http_checks]]
    interval = 10000
    grace_period = "5s"
    method = "get"
    path = "/"
    protocol = "http"
    timeout = 2000
    tls_skip_verify = false
    [services.http_checks.headers]

Roughly translated, this section says every ten seconds, perform an HTTP GET on the root path (e.g. http://appname.fly.dev/) looking for it to return an HTTP 200 status within two seconds. The parameters of a http_check are listed below.

Times are in milliseconds unless units are specified.

  • grace_period: The time to wait after a Machine starts before checking its health. Make sure this is long enough for your app to start up. For example, if your app takes 2 seconds to start up, give it some runway by setting grace_period to at least 3 seconds.
  • interval: The time between connectivity checks. There should be a balance between the interval and the grace period. If it’s long and your grace_period shorter than your app’s startup time, health check will take too long adding you your deployment time.
  • timeout: The maximum time a connection can take before being reported as failing its health check.
  • method: The HTTP method to be used for the check.
  • path: The path of the URL to be requested.
  • protocol: The protocol to be used (http or https).
  • tls_server_name: If the protocol is https, the hostname to use for TLS certificate validation.
  • tls_skip_verify: When true (and using HTTPS protocol) skip verifying the certificates sent by the server.
  • services.http_checks.headers: This is a sub-section of services.http_checks. It uses the key/value pairs as a specification of header and header values that will get passed with the http_check call.

Note: The services.http_checks section is optional and not generated in the default fly.toml file.

services.machine_checks

Machine checks work a bit differently than the other checks. They run on each deploy, and if they fail, the deploy is stopped. A new Machine is spawned with the environment variable FLY_TEST_MACHINE_IP set to the 6PN IPv6 address of the Machine being tested. This is useful for running integration tests on Machines before a deployment. Here’s an example:

  [[services.machine_checks]]
    image = "curlimages/curl"
    entrypoint = ["/bin/sh", "-c"]
    command = ["curl http://[$FLY_TEST_MACHINE_IP] | grep 'Hello, World!'"]
    kill_signal = "SIGKILL"
    kill_timeout = "5s"

This example uses the curl image to run a test. It checks that the Machine is serving a page with the text “Hello, World!”.

  • image: The Docker image to use for the test. We default to the one used for the process group the Machine is in.
  • entrypoint: The entrypoint for the test. Defaults to the entrypoint of the Machine being tested if not set
  • command: The command to run for the test.
  • kill_signal: The signal to send to the test process if it runs too long. Defaults to the signal of the image, if a custom image is set.
  • kill_timeout: The time to wait before sending the kill signal. Defaults to the timeout of the image, if a custom image is set.

Machine checks are especially useful for canary deploys. flyctl will spawn a new Machine, ensure all machine capabilities are functional, and then deploy the rest of the Machines in your app.

Machine checks are supported for apps using the rolling and canary deployment strategies.

The mounts section

This section supports the Volumes feature for persistent storage. The section has two required entries: source and destination.

[mounts]
  source = "myapp_data"
  destination = "/data"
  processes = ["disk"] # optional - attach volumes to Machines that belong to one or more process groups
  initial_size = "20gb" # optional - the size of the volume created on first deploy

source

The source is a volume name that this app should mount. Any volume with this name, in the same region as the app and that isn’t already mounted, may be mounted. A volume of this name must exist in some region for the application to deploy.

destination

The destination is the directory where the source volume should be mounted on the running app. This cannot be / since that’s where the root file system is mounted.

processes

Optionally, you can specify a list of processes to limit volume mounts by process group. You can even specify two different [[mounts]] sections to mount different source volumes to Machines in different process groups. Note the need to use double brackets if there are multiple [[mounts]] defined.

initial_size

Optionally, you can specify the size of the volume to be created when launching an app for the first time.

This value is only used by fly launch and fly deploy commands in case they need to create a new volume; it is ignored by the fly volume create command.

The value can be a string with units, like initial_size = "30GB" for example. The value can also be an integer, like initial_size = 30 for example, with GB as the default unit.

snapshot_retention

Optionally, you can specify the number of days snapshots of the volumes in this app are retained.

The default is 5 days when unset. The minimum is 1 day. The maximum is 60 days.

For example, the following config retains snapshots of volumes for 14 days:

[mounts]
  source = "myapp_data"
  destination = "/data"
  snapshot_retention = 14

scheduled_snapshots

Optional. Enable or disable automatic daily snapshots for a volume.

The default is true (enabled) when unset.

For example, the following config disables automatic daily snapshots:

[mounts]
  source = "myapp_cache"
  destination = "/cache"
  scheduled_snapshots = false

auto_extend_size_threshold

Optional. The threshold of storage used on a volume, by percentage, that triggers extending the volume’s size by the value of auto_extend_size_increment, unless the resulting size would exceed auto_extend_size_limit.

auto_extend_size_increment

The increment, in GB, by which to extend the volume after reaching the auto_extend_size_threshold.

auto_extend_size_limit

Maximum size, in GB, of a volume after an extension is completed.

When using the auto-extension feature for volumes, both auto_extend_size_increment and auto_extend_size_limit are required. Without these values, the volume won’t resize automatically, even if the usage threshold is reached.

Auto extend volume size configuration

You can optionally configure volumes to automatically extend when they reach a usage threshold.

For example, the following config extends the size of the volume by 1GB when it reaches 80% capacity. The volume extensions are limited to 5GB in total.

[[mounts]]
  source = "data"
  destination = "/data"
  auto_extend_size_threshold = 80
  auto_extend_size_increment = "1GB"
  auto_extend_size_limit = "5GB"

The vm section

This section defines the compute requirements for the Machines used for the application.

The default Machine size is shared-cpu-1x but it is not enforced if not specified in fly.toml. It means commands like fly deploy won’t attempt to update the compute requirements (or size) for a Machine. fly scale commands will use existing Machines to infer new Machine sizes, and in the event that there are no Machines, they will use shared-cpu-1x.

If you care about a predictable scaling up from zero case, or if you have different compute requirements for different process groups, then it is highly recommended to add this section to fly.toml. Once compute requirements are set, fly deploy and fly scale count commands will enforce them. If you change the size of the Machines using fly scale vm (or its sibling fly scale memory) then it will be reset on next fly deploy unless fly.toml is updated accordingly. Learn more about Machine size configuration precedence.

If you update your Machine size using fly scale vm or fly scale memory but you still have a [[vm]] section in your fly.toml file, the next fly deploy will reset your Machines to the configuration in the file. If you want to manually scale your Machines and keep those settings, remove the [[vm]] section from your fly.toml file.

All keys are optional and size has lower precedence than all other keys.

[[vm]]
  size = "shared-cpu-2x"
  memory = "2gb"
  cpus = 4
  cpu_kind = "performance"
  gpus = 1
  gpu_kind = "a100-pcie-40gb"
  kernel_args = "no-hlt=true"
  host_dedication_id = "customer-id"
  persist_rootfs = "never"
  processes = ["app"]

size

The Machine preset size to use as base, like size = "shared-cpu-1x" for example.

To get the full list, run fly platform vm-sizes or check the pricing page.

memory

The memory to request. It can be expressed with units, like memory = "2GB" or memory = "512mb" for example. The value can also be an integer, like memory = 1024 for example, with MB as the default unit.

cpus

The number of vCPUs to request. Valid values are 1, 2, 4, 8, or 16, but depends on cpu_kind.

cpu_kind

The kind of CPU to request. Valid values are shared and performance.

gpus

The number of GPUs to request. Valid values are 1, 2, 4, 8, but depends on gpu_kind.

Setting this value requires also setting gpu_kind.

gpu_kind

The kind of GPU to request. Valid values are a10, l40s, a100-pcie-40gb and a100-sxm4-80gb.

Setting this value assumes gpus = 1 if not present.

kernel_args

Additional kernel parameters provided to the kernel when booting the VM.

host_dedication_id

In the case of having a dedicated host, this is the value to set so Machines are launched on dedicated hardware.

persist_rootfs

The persistence of the root filesystem across restarts and updates. Valid values are never (default), restart, and always.

  • never: The root filesystem is ephemeral and will not be persisted across restarts or updates.
  • restart: The root filesystem is persisted across restarts, but not across updates.
  • always: The root filesystem is persisted across restarts and updates.

processes

Optionally, you can specify a list of processes to limit by process group. You can even specify two different [[vm]] sections to have different compute requirements per process group. Note the need to use double brackets if there are multiple [[vm]] sections defined.

A [[vm]] section without a processes key applies to all process groups, except when another [[vm]] section targets a specific process group. For example the following snippets uses shared-cpu-4x for app and other process groups but performance-1x for worker group.

[processes]
app = "/start-app"
worker = "/start-worker"
other = "/start-other"

[[vm]]
size = "shared-cpu-4x"

[[vm]]
size = "performance-1x"
processes = ["worker"]

The restart section

[[restart]]
  policy = "never"
  retries = 10
  processes = ["app"]

When a machine unexpectedly exits, our scheduling machinery follows a machine’s restart policy to restart it without your intervention. These policies are quite similar to Docker’s container restart policies:

  • always: we’ll attempt to restart the machine no matter the exit code.
  • never: we won’t restart the machine even if it exits with a non-zero exit code.
  • on-failure: we’ll only restart the machine if it exited with a non-zero exit code (due to a failure or crash). This is the default policy if one is not explicitly set.

A restart policy can be targeted to a specific process group. If a group is not specified, all machines in an app will have the same default restart policy. You can also specify the number of times we should retry restarting the machine before giving up.

The checks section

If your app doesn’t have public-facing services, or you want independent health checks that don’t affect request routing, use this top-level checks section instead of [[services.checks]].

Unlike service-level checks, top-level checks require:

  • A port number
  • A unique name
  • The type of check (http or tcp) 
[checks]
  [checks.name_of_your_http_check]
    grace_period = "30s"
    interval = "15s"
    method = "get"
    path = "/path/to/status"
    port = 5500
    timeout = "10s"
    type = "http"
    [checks.name_of_your_http_check.headers]
      Content-Type = "application/json"
      Authorization = "super-secret"

  [checks.name_of_your_tcp_check]
    grace_period = "30s"
    interval = "15s"
    port = 1234
    timeout = "10s"
    type = "tcp"

Fields are very similar to [[services.checks]]:

  • port: Internal port to connect to. Needs to be available on 0.0.0.0. Required.
  • type: Either tcp or http. Required.
  • grace_period: The time to wait after a Machine starts before checking its health. Make sure you give your app enough time to start up. For example if your app takes 2 seconds to start up, give it some runway by setting this to at least 3 seconds.
  • interval: The time between check runs. If your grace_period is shorter than your app’s startup time, and interval is too long, checks will increase deployment times.
  • processes: For apps with multiple processes. The process group to apply the health checks to. Define process groups in the processes section.

For HTTP checks only:

  • method: The HTTP method to be used for the check.
  • path: The path of the URL to be requested.
  • timeout: The maximum time a connection can take before being reported as failing its health check.
  • headers: Specify key/value pairs will get passed as HTTP headers on the check request.

Again, times are in milliseconds unless units are specified.

The processes section

The processes section allows you to define process groups to be run on separate Machines within a single app. Learn more about running multiple process groups in an app.

Each Machine can run a different command on start, allowing you to re-use your code base for different tasks (web server, queue worker, etc).

To run multiple processes, you’ll need a [processes] block containing a map of a name and command to start the application.

[processes]
  web = "bundle exec rails server -b [::] -p 8080"
  worker = "bundle exec sidekiqswarm"

Furthermore, you can “match” a specific process (or processes) to an [http_service] or [[services]] configuration. For example:

[http_service]
  processes = ["web"] # this service only applies to the web process
  internal_port = 8080
  force_https = true

Volumes can also be assigned to specific processes; use double brackets to include more than one [[mounts]] section, and mount differently named volumes to Machines in different process groups:

[[mounts]]
  source = "data"
  destination = "/data"
  processes = ["app"]

The metrics section

Fly apps can be configured to export custom metrics to the Fly.io-hosted Prometheus service.

[metrics]
  port = 9091
  path = "/metrics" # default for most prometheus exporters

Custom metrics can be also configured for specific processes, using double brackets to define multiple [[metrics]] tables:

[[metrics]]
  port = 9394
  path = "/metrics"
  processes = ["web"]

[[metrics]]
  port = 9113
  path = "/metrics"
  processes = ["proxy"]

Check out Metrics on Fly.io for more information about collecting metrics for your apps.

The statics sections

You can use the [[statics]] section in your fly.toml to serve static files such as images, CSS, or JavaScript for your application. There are two ways to serve statics:

  • From a running Fly Machine
  • From a Tigris object storage bucket.

Serving statics from a Fly Machine

When you add a [[statics]] section to fly.toml, the Fly Proxy intercepts requests matching your url_prefix. If the requested file exists in your guest_path, the request is routed to a static file server running inside your machine. This bypasses your main web server process. Your machine must be running for static files to be served.

If your machine is stopped, static files will not be served. If autostart is enabled, the machine will be started during statics requests as well as during normal requests to the app. Static assets are always served directly from your VM, not from the Fly edge or proxy host.

[[statics]]
  guest_path = "/app/public"
  url_prefix = "/public"

In this example, a request to /public/image.png is served from /app/public/image.png inside your running machine. You can define up to 10 [[statics]] mappings per application.

The guest_path is the directory in your Docker image to serve files from. It may overlap across mappings, but url_prefix values must not overlap. Only one static mapping is honored for a given prefix. For example, two mappings to /public/foo and /public/bar are allowed, but two mappings to /public are not.

Serving statics from Tigris object storage

You can serve static files directly from a Tigris object storage bucket. This lets you host assets or full static sites from a bucket instead of baking them into your Docker image.

Static files in Tigris buckets are always delivered through your application’s running VM, not directly from the Fly edge. If no machine is running, requests for Tigris statics will not be served. However, if autostart is enabled for your app, incoming requests for static assets will automatically start a machine as needed, the same as for normal application traffic.

[[statics]]
  guest_path = "/app/public"
  url_prefix = "/public"
  tigris_bucket = "my-bucket"

In this configuration, requests under /public are served from files stored in my-bucket under the /app/public path. Your machine must be running for these files to be served. If your machine scales to zero, static assets will not be accessible.

When handling trailing slashes, requests to a directory path such as /public/dir/ look for an index.html file, if you specify one. Requests without a trailing slash such as /public/dir may not serve as expected. Consider using redirects or documenting this for your users. Tigris statics support the index_document key, which allows you to serve a default file for a directory path.

[[statics]]
  guest_path = "/app/public"
  url_prefix = "/public"
  tigris_bucket = "my-bucket"
  index_document = "index.html"

In this configuration, a request to https://example.com/public/ serves /app/public/index.html from the bucket. Similarly, https://example.com/public/assets/ serves /app/public/assets/index.html.

Tigris statics are billed at Tigris pricing. Data egress from Tigris may incur both Tigris and Fly Proxy egress charges.

Do not overlap url_prefix values; only one static mapping will be honored for a given prefix.

Caveats

This feature should not be compared directly with a CDN, for the following reasons:

  • This feature does not exempt you from having to run a web service in your Machine.
  • You must have a running machine for statics to be served. If your machine is stopped, static assets will not be served. If auto_start_machines is enabled, the machine will be started automatically in response to statics requests, the same as with normal application requests.
  • The Fly Proxy routes matching requests directly to the statics file server running inside your machine. Static requests do not go through your main application process, but are handled by the file server within the same VM.
  • Serving an index document is only supported for statics served directly from Tigris. The index document must have the same name in all the relevant locations in your Tigris bucket.
  • For statics served from your container, it will not find index.html at the root. The full path must be requested.
  • You cannot set Cache-Control or any other headers on assets. If you need those, deliver them from your application and set the relevant headers.
  • statics does not honor symlinks. For example, if /app/public in your container is a symlink, such as /app-39403/public, use the absolute original path in your statics configuration.

The files section

When a files section is set, the contents from one of raw_value, local_path, or secret_name will be written to the Machine at the provided guest_path.

  • raw_value: The raw file content to be written. Must be base64 encoded.
  • local_path: The path to a local file in your system that should be used. Does not need to be base64 encoded.
  • secret_name: The name of an app secret that’s been set with fly secrets set. The value of the secret will be written to the file. The referenced secret’s value must be base64 encoded.

Examples:

[[files]]
  guest_path = "/path/to/hello.txt"
  raw_value = "aGVsbG8gd29ybGQK"

[[files]]
  guest_path = "/path/to/config.yaml"
  local_path = "/local/path/config.yaml"

[[files]]
  guest_path = "/path/to/secret.txt"
  secret_name = "SUPER_SECRET"

You can optionally restrict which Machine(s) contain the file using the processes field. For example:

[[files]]
  guest_path = "/path/to/config.yaml"
  local_path = "/local/path/config.yaml"
  processes = ["web"]

To remove a previously configured file on the next deployment, either delete the files entry entirely, or set its source to an empty string like this:

[[files]]
  guest_path = "/path/to/config.yaml"
  local_path = ""

The experimental section

This section is for flags and feature settings which have yet to be promoted into the main configuration.

[experimental]
cmd = ["path/to/command", "arg1", "arg2"]
entrypoint = ["path/to/command", "arg1", "arg2"]
exec = ["path/to/command", "arg1", "arg2"]

cmd

This overrides the CMD set by the Dockerfile. It should be specified as an array of strings, as seen in the example above.

entrypoint

This overrides the ENTRYPOINT set by the Dockerfile. It should be specified as an array of strings, as seen in the example above.

exec

This overrides both the CMD and the ENTRYPOINT set by the Dockerfile. It should be specified as an array of strings, as seen in the example above.

Open in ChatGPT
Report an issue or edit this page on GitHub