Challenge #5b: Multi-Node Kafka-Style Log

In this challenge, you’ll need to take your Single-Node Kafka system and distribute it out to multiple nodes.

Your nodes can use the linearizable key/value store provided by Maelstrom to implement your distributed, replicated log. This challenge is about correctness and not efficiency. You only need to keep up with a reasonable request rate. It’s important to consider which components require linearizability versus sequential consistency.

Specification

This challenge works the same as the single-node except that it’s now running with two nodes. All correctness checks in Maelstrom should pass.

Service: lin-kv

You’ve used the seq-kv service in the Grow-only Counter challenge, however, in this challenge you can use the linearizable version called lin-kv. The API is the same but they have different consistency guarantees.

You can instantiate the Go client in the library by using the NewLinKV() function:

node := maelstrom.NewNode()
kv := maelstrom.NewLinKV(node)

Evaluation

Build your Go binary as maelstrom-kafka and run it against Maelstrom with the following command:

./maelstrom test -w kafka --bin ~/go/bin/maelstrom-kafka --node-count 2 --concurrency 2n --time-limit 20 --rate 1000

This will run a two-node system for 20 seconds with 4 clients (2n). It will validate the system for correctness.

If you’re successful, that’s great! Continue on to the Efficient Kafka challenge. If you’re having trouble, jump over to the Fly.io Community forum for help.