Phoenix LiveView Zipped Uploads

File and image uploads in LiveView are already easy and elegant. But what if the user wants to upload an entire directory with more nested directories and files? How should we handle that? Chris McCord walks us through what it takes to make that a good experience in LiveView.

Problem

LiveView makes interactive file uploads a breeze. A user can select multiple files to upload, one-by-one, and LiveView will dutifully upload the files in parallel, show previews, and even allow the user to prune their upload selections before you commit the uploads to their final destination. This flow works great, but sometimes we want to allow users to upload entire directories of nested files.

How can we upload entire directories of files, including nested directories, in LiveView?

Solution

Fortunately, we have a great starting point with the file input’s webkitdirectory attribute, but we need a little more work to make this sing with LiveView.

The webkitdirectory attribute allows the user to select entire directories for upload, but we don’t want to send each file (potentially hundreds or thousands) as an individual upload entry. This is way more expensive for the server to handle as you need to consume each file one-by-one, and it also sends up more bandwidth than necessary.

Instead, we can perform the following steps:

• Compress the files on the client into a zip archive.
• Upload the single zip file to LiveView.

Once the zip file is uploaded, we will:

• Unpack the zip archive on the server.
• Move the tree of files to their final location.

This approach saves on bandwidth and server resources by processing the files in a single step.

To handle zipping the files on the client, let’s pull in the JSZip library.

From your phoenix project directory, vendor the jszip library to assets/vendor :

$ curl -o assets/vendor/jszip.js https://raw.githubusercontent.com/Stuk/jszip/main/dist/jszip.min.js
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 97630  100 97630    0     0   366k      0 --:--:-- --:--:-- --:--:--  372k

chris@m2a ~/playground/zip

Next, import the JSZip object in your assets/js/app.js file:

import {LiveSocket} from "phoenix_live_view"
import JSZip from "../vendor/jszip"

We’ll need a sprinkle of custom JavaScript to handle zipping the collection of files on the client, before passing it off to LiveView as the upload. We’ll use a Hook for this. In your assets/js/app.js add the following hook:

let Hooks = {}
Hooks.ZipUpload = {
  mounted(){
    this.el.addEventListener("input", e => {
      e.preventDefault()
      let zip = new JSZip()
      Array.from(e.target.files).forEach(file => {
        zip.file(file.webkitRelativePath || file.name, file, {binary: true})
      })
      zip.generateAsync({type: "blob"}).then(blob => this.upload("dir", [blob]))
    })
  }
}

Here we listen for the input event of the file input element, which means a directory has been selected. When fired, we create a new JSZip instance and add each file to the object. Once all files are added, we call zip.generateAsync to generate the zip archive and finish by calling the built-in this.upload("dir", [blob]) to upload the zipped blob to LiveView.

We also need to let our liveSocket instance know how to find the Hook. Update your liveSocket instantiation to look like this:

let liveSocket = new LiveSocket("/live", Socket, {
  params: {_csrf_token: csrfToken},
  hooks: Hooks
})

Next, let’s replace our landing page route with a new LiveView route in lib/zip_web/router.ex:

-   get "/", PageController, :home
+   live "/", UploadLive

Now, let’s create a new LiveView in lib/zip_web/live/upload_live.ex and key this in:

defmodule ZipWeb.UploadLive do
  use ZipWeb, :live_view

  @uploads_dir Path.expand("../../../priv/uploads", __DIR__)

  def render(assigns) do
    ~H"""
    <form phx-change="files-selected">
      <p :if={@status}><%= @status %></p>
      <div class={@status && "hidden"}>
        <.live_file_input upload={@uploads.dir} class="hidden" />
        <input id="dir" type="file" webkitdirectory={true} phx-hook="Zip" />
      </div>
      <%= for entry <- @uploads.dir.entries do %>
        <%= entry.progress %>%
      <% end %>
    </form>
    """
  end

  def mount(_params, _session, socket) do
    {:ok,
     socket
     |> assign(files: [], status: nil)
     |> allow_upload(:dir,
       accept: :any,
       max_entries: 1,
       auto_upload: true,
       max_file_size: 1_000_000_000,
       progress: &handle_progress/3
     )}
  end

  def handle_event("files-selected", _, socket) do
    {:noreply, assign(socket, status: "compressing files...")}
  end

  def handle_progress(:dir, entry, socket) do
    raise "TODO"
  end
end

We started with the skeleton of what we need to accept file uploads. First, we defined a template inside render that builds a basic upload form. It uses the <live_file_input/> component to build the necessary LiveView input, but we hide it with the hidden class because we’ll be using our own input to build the zip file. Once we zip the file on the client with our Hook, we’ll use the regular LiveView input to perform the upload.

Our template is basic. When we have a @status assigned, it means our zipped upload is either in progress or complete, and we display a message to the user. Otherwise, we show the input and current file progress.

Next, in mount, we start with an empty status, then we allow an upload for the user’s directory. We mark it as max_entries: 1 because we’ll be taking all the files in the user’s selected directory and zipping it as a single entry on the client. We also set auto_upload: true to the upload as soon as the directory is selected. Next, we allow up to a 1gb file size and finally tell LiveView to call our handle_progress/3 callback on progress events.

With this in place, we can check that our LiveView renders things appropriately:

It’s not much to look at yet, but we know we’re on the right track. To complete the loop, we need to unzip the archive once it has been completely uploaded and move the contents to the final destination path:

def handle_progress(:dir, entry, socket) do
  if entry.done? do
    File.mkdir_p!(@uploads_dir)

    [{dest, _paths}] =
      consume_uploaded_entries(socket, :dir, fn %{path: path}, _entry ->
        {:ok, [{:zip_comment, []}, {:zip_file, first, _, _, _, _} | _]} =
          :zip.list_dir(~c"#{path}")

        dest_path = Path.join(@uploads_dir, Path.basename(to_string(first)))
        {:ok, paths} = :zip.unzip(~c"#{path}", cwd: ~c"#{@uploads_dir}")
        {:ok, {dest_path, paths}}
      end)

    {:noreply, assign(socket, status: "\"#{Path.basename(dest)}\" uploaded!")}
  else
    {:noreply, assign(socket, status: "uploading...")}
  end
end

Fortunately Erlang makes this part easy. The Erlang standard library includes a :zip module, which can unzip an archive to a final destination. In our handle_progress callback, we check if the zipped file is done being uploaded with entry.done?, if so we use LiveView’s consume_uploaded_entries function to process the temporary files that have been uploaded.

Inside the consume function, we have a complex pattern match on the results of the :zip.list_dir(~c"#{path}") call. This match simply grabs the first file in the user’s selection, which should be the root directory name. Next, we build a final destination path where we want to unzip the temporary file to. We complete the operation by calling :zip.unzip to move the files to the final location on disk.

We return the dest path and update our socket assigns to show the new status.

Let’s see it all in action:

That’s it!

Discussion

We see here that Elixir inherits powerful features through OTP like the :zip module. It’s fun to dig into some of those features and see how we can use them.

This solution could work also well for uploading a directory of images, for generating business reports from a directory of CSV files, and many other uses!