Updating Scale

We have been recently looking at how scaling commands work on Fly and we’ve come up with a new system with a simpler model and more compact command set to work with. We’d like to introduce it to you today.

The Scaling System

Previously, at least in the user interface, we offered regions as a fixed list with a number of instances per region set as part of a scaling plan. This worked but introduced some operations that were not intuitive, such as having to reset that list or elements in the list to remove regions. It was also fairly rigid and we wanted to automate the process more and let scaling manage more of the global load.

The new system is based on a pool of regions where the application can be run. Using a selected model, the system will then create at least the minimum number of application instances across those regions. The model will then be able create instances up to the maximum count. The min and max are global parameters for the scaling. There are two scaling models, Standard and Balanced.

• Standard: Instances of the application, up to the minimum count, are evenly distributed among the regions in the pool. They are not relocated in response to traffic. New instances are added where there is demand, up to the maximum count.

• Balanced: Instances of the application are, at first, evenly distributed among the regions in the pool up to the minimum count. Where traffic is high in a particular region, new instances will be created there and then, when the maximum count of instances has been used, instances will be moved from other regions to that region. This movement of instances is designed to balance supply of compute power with demand for it.

It’s worth noting that the scaling model, in conjunction with the platform, may not deploy in a predictable way. For example, if an allowed region is unable to allocate new instances, the platform will fallback to creating one in a nearby region. As this fallback operation is a feature of the system, it’s not something to worry about when an region that is not in the pool shows up in the currently running instances (as displayed with flyctl status).

flyctl regions

This is a new flyctl command, taking over from the previous flyctl scale regions command. It has its own subcommands:

• list : lists out the regions that are currently in the application’s pool of regions:

flyctl regions list

Allowed Regions:
  ams  Amsterdam, Netherlands
  ewr  Parsippany, NJ (US)
  iad  Ashburn, Virginia (US)
  mrs  Marseille, France
  yyz  Toronto, Canada

This list shows that a pool with five regions in it.

• add : takes a space-separated list of regions, adds those regions to the pool of regions and then applies that new pool to the application.
• remove : takes a space-separated list of regions, removes those regions from the pool of regions and then applies that new pool to the application.

This pool reflects the possible locations where the application may be run. A region being in the pool doesn’t mean there will be an instance in that location. That all depends on the scaling model and min/max count settings which are selectable with flyctl scale.

flyctl scale

This command has been expanded to give you more control over your scaling.

The show subcommand will display the current configuration of an application:

flyctl scale show

Scale Mode: Standard
Min Count: 5
Max Count: 10
VM Size: micro-2x

Here, the standard model is being used, with a min of 5 instances and a max of 10. All the instances are created with a micro-2x VM.

The balanced and standard subcommands select the model and can optionally set the min and max values as key=value settings. If we wanted to just switch to a balanced model, we would enter:

flyctl scale balanced

Scale Mode: Balanced
Min Count: 5
Max Count: 10

Or, if we wanted to switch to a standard model with a max of 20 we could enter:

flyctl scale standard max=20

Scale Mode: Standard
Min Count: 5
Max Count: 20

There is also a set subcommand which lets you vary the min and max without changing the model.

flyctl scale set min=6 max=10

Scale Mode: Standard
Min Count: 6
Max Count: 10

You can, in just in one line, set the model and all the parameters too.

flyctl scale balanced min=5 max=10

Scale Mode: Balanced
Min Count: 5
Max Count: 10

The vm command remains as it was, showing and setting the size of the virtual machine - in terms of cpu and memory - that each instance of the application will be run with. flyctl scale vm will display detailed information about the currently selected vm size. Specifying a vm size on the command line flyctl scale vm cpu1mem1 will switch the application to using that size of vm globally.

Going Forward

Scaling will always be a constantly evolving feature on Fly as we match more models with use cases and find new ways to fine tune models and commands.

Do let us know what you think of the changes by dropping a line to support or leaving a message on our Spectrum Community.