Running apps on UDP and TCP

You can run any containerized app on our platform, regardless of what protocols it talks over, including apps that rely on UDP in addition to TCP. This guide walks you through how to set up your UDP app on Fly.io.

Before getting started, here’s the summary of important points about using up UDP on Fly.io (that we’ll expand on below):

• You need a dedicated IPv4 address. You can’t use a shared IPv4 address or an IPv6 address for UDP. We support public IPv6 and shared IPv4 for TCP.

• The UDP side of your app needs to bind to the special fly-global-services address. But the TCP side of your app can’t; it should bind to 0.0.0.0.

• The UDP side of your app needs to bind to the same port that is used externally. Fly will not rewrite the port; Fly only rewrites the IP address for UDP packets.

• We swipe a couple dozen bytes from your MTU for UDP, which usually doesn’t matter, but in rare cases might.

You need a dedicated IPv4 address

You’ll need a dedicated IPv4 address for your app to accept UDP packets. We don’t support UDP over public IPv6. Every Fly.io app gets a public IPv6 address and a shared IPv4 address that will work for the TCP side of your app.

Your app needs to bind to the fly-global-services address

To receive UDP packets, your app needs to bind to the special fly-global-services address (that’s it; that’s the address). Just as importantly: it needs to reply to messages using that address.

You usually need to explicitly bind your UDP service to fly-global-services. Sorry, but 0.0.0.0, *, and INADDR_ANY generally won’t do: Linux will use the wrong source address in replies if you use those.

But: TCP can’t use fly-global-services

You can’t bind your TCP service to fly-global-services; the TCP side of your app should probably bind to *.

Why the fly-global-services address?

For the most part, UDP “just works” on Fly.io, the way you’d expect it to.

The thing about forwarding UDP that makes it different from HTTP is that UDP messages don’t have HTTP headers. When a standard proxy receives an HTTP message and sends it to its target, it stashes the original source address in the headers, so the target knows who it’s talking to. You can’t generally do that with UDP messages. But you need to know who you’re talking to in order to respond to a UDP message!

We use ⚡️eBPF magic⚡️ to shuttle UDP packets across our forwarding fabric while transparently mapping source addresses. What that means for you is that you can generally just write a standard socket program that reads UDP messages and replies to them based on the source address of the packet.

Your app needs to listen on the same port externally and internally

Fly will only rewrite the IP addresses for UDP packets. The destination port for inbound packets will not be rewritten. That means if your app receives UDP traffic on port 5000 externally, the app needs to bind to fly-global-services:5000 to receive that UDP traffic.

You might need to be mindful of MTUs

An invariant on almost every network is the maximum packet size, the MTU. Typically, the MTU is 1500 bytes. If you exceed 1500 bytes, your packets will fragment. You don’t want to fragment.

Fly.io takes a bite out of your packets for two purposes:

1. We forward all traffic across a WireGuard mesh, and WireGuard eats 60 bytes of every packet.

2. The UDP proxy forwarding system we use grabs another dozen bytes to record the original addresses of the packets — the source address we saw when your client’s packet hit our edge. You don’t see these bytes; they’re stripped off the packet before it’s delivered to your app. But they’re used in transit.

In most cases this won’t matter to you at all. Modern UDP protocols are designed to assume they have far less than 1500 bytes to work with, because there are all sorts of weird links on the Internet with smaller MTUs.

But if you’re doing something super-custom, it’s best to assume you’ve got something more like 1300 bytes to work with, not 1500.

Example UDP app

Let’s build a simple echo service. You can play the home version at this GitHub repository. We’ll listen on port 5000, UDP and TCP, and just bat back whatever clients send us.

Use flyctl launch to create a fly.toml file. Use it to wire up the ports.

[[services]]
  internal_port = 5000
  protocol = "udp"

  [[services.ports]]
    port = "5000"

[[services]]
  internal_port = 5000
  protocol = "tcp"

  [[services.ports]]
    port = "5000"

We’re going to build this app in Go (don’t worry, it’s simple, you’ll be able to follow it even if you aren’t a gopher). It’s going to be vanilla socket code.

Let’s See Some Code

You light up UDP and TCP with standard socket code. Usually, the only fussy bit will be the fly-global-services bind for UDP. Here’s what that looks like in Go:

func main() {
  // in other programming languages, this might look like:
    //    s = socket(AF_INET, SOCK_DGRAM, IPPROTO_UDP)
    //    s.bind("fly-global-services", port)

    udp, err := net.ListenPacket("udp", fmt.Sprintf("fly-global-services:%d", port))
    if err != nil {
        log.Fatalf("can't listen on %d/udp: %s", port, err)
    }

    // in other programming languages, this might look like:
    //    s = socket(AF_INET, SOCK_STREAM, IPPROTO_TCP)
    //    s.bind("0.0.0.0", port)
    //    s.listen()

    tcp, err := net.Listen("tcp", fmt.Sprintf(":%d", port))
    if err != nil {
        log.Fatalf("can't listen on %d/tcp: %s", port, err)
    }

    go handleTCP(tcp)

    handleUDP(udp)
}

The handlers for this app are textbook Go code; after the UDP bind, nothing else in your app will care about Fly.io.

Here’s TCP:

func handleTCP(l net.Listener) {
    for {
        conn, err := l.Accept()
        if err != nil {
            if errors.Is(err, net.ErrClosed) {
                return
            }

            log.Printf("error accepting on %d/tcp: %s", port, err)
            continue
        }

        go handleConnection(conn)
    }
}

func handleConnection(c net.Conn) {
    defer c.Close()

    lines := bufio.NewReader(c)

    for {
        line, err := lines.ReadString('\n')
        if err != nil {
            return
        }

        c.Write([]byte(line))
    }
}

And here’s UDP:

func handleUDP(c net.PacketConn) {
    packet := make([]byte, 2000)

    for {
        n, addr, err := c.ReadFrom(packet)
        if err != nil {
            if errors.Is(err, net.ErrClosed) {
                return
            }

            log.Printf("error reading on %d/udp: %s", port, err)
            continue
        }

        c.WriteTo(packet[:n], addr)
    }
}

In a real Go app doing something like DNS, you might have a single thread reading messages off the UDP socket and passing them to a pool of workers over a buffered channel; you might not want to fork off a thread for every UDP message you receive. This is just an echo service, so we’ll dispense with that detail.

This program, with a simple Dockerfile and the fly.toml configuration above, should just work on Fly.io. Boot it up and throw some UDP packets at it with netcat:

echo hello world | nc -w1 -u udp-echo-sample.fly.dev 5000

hello world