Extracting GPS/Exif Data From a Photo in Phoenix

Note: In this post I’ve added “digressions”. They contain a little extra commentary on what I talk about in the post but aren’t necessarily vital. Give them a try and tell me what you think on twitter.

The Context

I’ve been learning Elixir on and off for a few months now and recently started working on a toy web app using the Phoenix framework. The app is called Cable Car Spotter. Its purpose is to allow people to track which of the 40 historic cable cars they’ve seen around San Francisco[1].

The Problem

One of the first features I added was the ability for users to upload a photo when they see (spot) a cable car. Then, I thought it would be fun to plot a user’s sightings on a map - I am apparently addicted to mapping photos[2]. In order to do this, I need to know where the photo was taken. Nowadays the simplest way to do that is by pulling the latitude and longitude from the exif metadata of the uploaded photo.

Extracting the lat/lng from a photo and saving it to the database also seemed like a reasonably-size feature to practice my Elixir chops.

The Solution

At a high level, the flow looks like this:

1. A user submits a photo through the “sightings” form.
2. We pull out the exif data from the photo.
3. We save the date the photo was taken and the lat/lng to our database.
4. We save the image to an S3 bucket.

To make this post more digestible, I’m starting with an app that already does steps 1 & 4 - here I’ll be adding steps 2 & 3. But fret not, all the code for this project is on GitHub if you want to dive deeper.

Ok, with that plan in mind, let’s push forward, starting with the create function in our sightings_controller:

def create(conn, %{"sighting" => sighting_params}, user) do
  changeset =
    user
    |> build_assoc(:sightings)
    |> Sighting.changeset(sighting_params)

  case Repo.insert(changeset) do
    {:ok, _sighting} ->
      conn
      |> put_flash(:info, gettext("Sighting created successfully."))
      |> redirect(to: sighting_path(conn, :index, conn.assigns.locale))
    {:error, changeset} ->
      render(conn, "new.html", changeset: changeset)
  end
end

It’s a pretty straight-forward Phoenix controller action, right? We build up a changeset associating the current user with a sighting based on the given parameters and insert it into our repo.

To access the exif data on the uploaded image we’re going to use the Exexif package. I chose this package because it does what we want, is pure Elixir (no shelling out to exiftool), and is maintained by Dave Thomas (who literally wrote the book on Programming Elixir).

Here is what I learned from playing with Exexif in iex:

> iex -S mix
Erlang/OTP 20 [erts-9.0] [source] [64-bit] [smp:4:4] [ds:4:4:10] [async-threads:10] [hipe] [kernel-poll:false] [dtrace]

Interactive Elixir (1.4.5) - press Ctrl+C to exit (type h() ENTER for help)
iex(1)> {:ok, info} = Exexif.exif_from_jpeg_file("test/fixtures/cable_car_with_exif.jpg")
{:ok,
 %{:exif => %{:datetime_original => "2011:04:18 02:11:29",
     :component_configuration => "Cb,Y,Cr,Y", :exposure_program => "Program AE",

  #... and on and on; cut for brevity ...

iex(2)> IO.puts info.gps
 37°47´44˝N,122°24´36˝W
 :ok
iex(3)> IO.puts info.exif.datetime_original
 2011:04:18 02:11:29
 :ok

At this point we want to get our code to a place where we can add the exif data to the changeset in our controller so we can persist it to the database with the rest of the submitted form.

iex(4)> info.gps.gps_longitude
 [122, 24, 36]
iex(5)> info.gps.gps_latitude
 '%/,'
# ^^^ -- I don't know what that means...

Being a relative Elixir n00b, I updated my controller action to look like this:

def create(conn, %{"sighting" => sighting_params}, user) do
  base_struct = user |> build_assoc(:sightings)

  changeset = case Map.has_key?(sighting_params, "photo") do
    true -> Sighting.changeset_with_photo(
      base_struct,
      sighting_params,
      ExifExtractor.extract_metadata_from_photo(sighting_params["photo"].path)
    )
    false -> Sighting.changeset(base_struct, sighting_params)
  end

  case Repo.insert(changeset) do
    # ... nothing changed here
  end
end

source on GitHub

The relevant changes here are that

1. if the form is submitted with a photo, we use a new changeset, changeset_with_photo
2. we’re passing that new changeset a value from a new function, ExifExtractor.extract_metadata_from_photo

The new changeset is super simple and I’ll get into it later. Instead, let’s look at the new function:

defmodule CableCarSpotter.ExifExtractor do
  def extract_metadata_from_photo(image_path) do
    case Exexif.exif_from_jpeg_file(image_path) do
      {:ok, info} -> extract_from_valid_exif(info)
      {:error, _} -> %{}
    end
  end

  defp extract_from_valid_exif(exif_data) do
    %{
      geom: %Geo.Point{
        coordinates: {
          from_dms_to_decimal(exif_data.gps.gps_latitude, exif_data.gps.gps_latitude_ref),
          from_dms_to_decimal(exif_data.gps.gps_longitude, exif_data.gps.gps_longitude_ref)
        },
        srid: 4326
      },
      photo_taken_at: datetime_original(exif_data.exif)
    }
  end

  # ... truncated for brevity
end

The one public function, extract_metadata_from_photo just runs the Exexif function exif_from_jpeg_file. If the image has valid exif data in it, we return the map generated by extract_from_valid_exif. If not, we just return an empty map. Why no more error handling? Well, in the context of our use case, we don’t want the lack of exif data to stop the form submission process.

Looking at the map returned by extract_from_valid_exif, you might be wondering, “wtf is a %Geo.Point{}?” And my answer is, “thanks for the segue into talking about how we store the latitude and longitude data.”

%Geo.Point is a type that comes from the geo package, which is required by the geo_postgis package, which we are using to talk to our PostGIS-enabled Postgres database. Whoa, let me unpack that by repeating it backwards:

Phoenix uses Postgres as its default database via the Postgrex package. PostGIS is an extension for Postgres that enables handling of geospatial data. GeoPostGIS is a package that extends Postgrex so we can use PostGIS data types. Under the hood, GeoPostGIS uses the structs and functions from the Geo package. Whew.

What all this means is that in the schema for our sightings we can do:

schema "sightings" do
  field :comment, :string
  field :photo, CableCarSpotter.Photo.Type
  belongs_to :user, CableCarSpotter.User
  belongs_to :cable_car, CableCarSpotter.CableCar
  field :photo_taken_at, :utc_datetime
  field :geom, Geo.Point  # <-- This right here

  timestamps()
end

The catch here is that you need to add the Geo types to the list of types Postgrex knows about. According to the docs, we need to define the types somewhere (the docs were unclear as to where, so I put the definition in a new file in lib/) and then add them to your repo configs (see my config/dev.exs).

Ok, so with that in place, the map returned by extract_from_valid_exif can be used in our new changeset - which I held off showing until now:

def changeset_with_photo(struct, params, metadata) do
  struct
  |> changeset(params)
  |> cast(metadata, [:photo_taken_at, :geom])
end

source on GitHub

And that’s it more or less. We updated our app to be geo-aware, then used the Exexif package to pull out GPS data from uploaded photos and save them. Huzzah!

The Caveats

Oh yeah, this is the first non-trivial Elixir app I’ve ever written and the first time I’ve used Phoenix, so don’t take this as cannon. Hit me up on twitter if you have any thoughts on how I could do this better.

Also, if you’re a little confused looking through the code on GitHub, bear in mind that I started this project on Phoenix 1.2, got this feature working, then upgraded to 1.3 and switched to the new folder structure…but I haven’t converted the models into bounded contexts yet.

I plan on writing more about the build of this app, so if you liked this, stay tuned for more.

The References

Here are some posts I found helpful in building this feature:

• Exploring Geospatial data in Elixir with Phoenix, D3, and PostGIS
• The answer to How to use Geo library to create valid Ecto Model changeset?
• Testing a file upload in elixir/phoenix

[1] My family and I lived on Nob Hill for 6+ years and saw Cable Cars all the time. It was then that my wife made the pen & paper version of this app - a piece of posterboard on the back of the front door where my daughter would cross off the cars she saw whenever we were out. 💚

[2] See Flat Ben, Zamar Map, and A Data Liberation Walkthrough for examples…