Migrating Heroku Database Apps To Fly

I was recently asked how the database migration in the original Turboku demo worked. For that original demo, the migration was practically automatic. We already knew that you could access a Heroku database from outside Heroku. Since then, Changes in how Heroku and the Postgres driver handle things have meant you need to be a little more proactive with your database connecting code. So, if we’re setting out to make an app that migrates without changes to Fly, what do we need to do?

Start With The DATABASE_URL

When you attach a Postgres addon to your Heroku app, it also creates a URL which contains everything you need to connect to the database. You can get the value of DATABASE_URL from the environment and that has all the credentials you need to connect to the Postgres database. It also contains the host, port and database name to connect to. For most situations, it can be used without modification. The demo uses Massivejs here as a layer on top of the node-postures (aka pg) package and connecting looks something like this:

const massive = require('massive');

var connectionString = process.env.DATABASE_URL;


(async () => {
    try {
        const db = await massive(connectionString);
     } catch (e) {
        console.log("No DB",e)
    }
})();

Now, if we wanted to connect to Heroku Postgres from outside Heroku, the number one requirement is that you switch to SSL. Now, you could do that by appending a string with the required change to the end of the connection string. But experience tells me that connection strings can change and change in sometimes unpredictable ways. So…

Turning on SSL

The first thing we need to do is to parse the connection string. To handle that, we’ll use the pg-connection-string package and we’ll extract the parse function:

var parse = require('pg-connection-string').parse;

This is the same package the pg library uses to parse connection strings so you shouldn’t have any compatibility issues with it. We can now take the connection string and parse it:

var connectionString = process.env.DATABASE_URL

connector=parse(connectionString);

And now we can set the ssl field:

connector.ssl={ "sslmode": "require", "rejectUnauthorized":false };

Ah! You may say, all you need to do is set ssl to true. And in a perfect world, that would be correct. Firstly, any value in the ssl setting equates to ssl being set to true. Secondly, we need to tweak the SSL configuration because Heroku Postgres has servers with a self-signed certificate in their chain of server certificates. If we just had SSL turned on, the client driver would try and verify the server’s identity and throw an error when it saw that self-signed certificate. You can read more about this in this github issue.

The fix, now, is to set the sslmode to require which requires SSL to be enabled and then rejectUnauthorized to false to stop that certificate verification check. The best part is we don’t need to do anything else to connect; we can use our parsed connector rather than the connection string on most connection calls to Postgres:

const db = await massive(connector);

Putting It Together

Putting all this together, we get this code:

const massive = require('massive');

var connectionString = process.env.DATABASE_URL

if (connectionString==undefined) {
    // Fallback to a local Postgres
    connector=parse("postgres://postgres@localhost/postgres");
    connector.ssl=false;
} else {
    connector=parse(connectionString);
    connector.ssl={ "sslmode": "require", "rejectUnauthorized":false };
}

(async () => {
    try {
        const db = await massive(connector);
        app.set('db', db);
        tables=db.listTables()
    } catch (e) {
        console.log("No DB",e)
    }
})();

There’s a little extra code to help out when testing locally against a local Postgres - don’t define DATABASE_URL and it’ll fall back to a non-SSL localhost connection. In our example code, flydictionary, the rest of the code a simple dictionary with the ability to search or add words and save them in the database.

Pushing to Heroku

If you want to install this app, grab the flydictionary code, log in to Heroku and deploy the app with heroku create then push the app up to Heroku with git push heroku master.

The app will start running but stop immediately as there’s no database for it to take to - Use:

heroku addons:create heroku-postgresql

This will create a Postgres database and attach it to your app. Now you can run heroku open to open a browser onto your newly created app. Add some words to the dictionary.

Coming to Fly

Bringing Heroku web apps to Fly is an uncomplicated process thanks to the specialized Heroku migration support.

All you need to do is go to the https://fly.io/heroku. Log in to Heroku there. That will show the available apps on the page and you can select the app you just created on Heroku.

Then you press the Turboku! button and the app will be automatically migrated over to Fly. That includes migrating the Heroku DATABASE_URL environment variable which will be turned into a Fly secret.

Things to remember with a Heroku database connection: Heroku only has two regions: the US and Europe. Database performance will be a function of how close your selected Fly regions are to your Heroku database region. Consider caching within Fly regions if performance is mission-critical.

Fly Closer

Thanks to the connection changes we made to the application, it’ll just start running on Fly and we can treat it like other Fly applications, adding and removing regions, scaling up, scaling down and more, while getting the benefit of being closer to the user. And remember that with Turboku, apps migrated this way are automatically updated when changes are pushed to Heroku.