Planet-Scale OSM Routing on a Raspberry Pi

Serving OpenStreetMap routing for the entire planet from home hardware sounds absurd until you realize most of the work happens once, offline, on machines with fast NVMe and enough RAM to survive the scratch-file storm. The Raspberry Pi’s job is simpler: load a pre-built tile archive and answer HTTP requests.

This post documents how I run Valhalla for the whole planet on a Pi 5, what failed along the way, and the counter-intuitive I/O lesson that saved hours of build time.

The shape of the problem

Valhalla builds routing graphs as a global tile grid. Each tile is a .gph file. A planet build touches scratch files (way_nodes.bin alone can exceed 120 GB) that dwarf available RAM. On a Ryzen 9950X with 60 GB RAM and 32 build threads, I watched the process enter a death spiral: 67% iowait, 7 GB swapped to zram, tile rate collapsing to ~180 tiles/hour.

The Pi cannot build the planet. It can serve it — if you ship it a merged tile tarball (~95 GB) and keep memory limits sane in valhalla.json.

Architecture: no runtime sharding

An early plan used three Valhalla containers behind an nginx geo-router. That worked in theory but added operational complexity. The production design is cleaner:

  1. Build Americas+Oceania on one x86 host.
  2. Build Eurasia+Africa on another.
  3. Merge at the tile-file level — copy both tile trees into one directory; on boundary overlaps, keep the larger .gph (more detail wins).
  4. Tar and deploy a single valhalla_tiles.tar to the Pi.
  5. One container, one endpoint.

Tile IDs are global. Two regional builds covering disjoint areas compose into a valid planet set without a runtime router.

flowchart LR
  subgraph build["x86 build hosts"]
    PBF["planet-latest.osm.pbf"]
    EA["Eurasia+Africa tiles"]
    AO["Americas+Oceania tiles"]
    PBF --> EA
    PBF --> AO
  end
  MERGE["merge-tiles.py\nlarger .gph wins"]
  TAR["valhalla_tiles.tar\n~95 GB"]
  PI["Raspberry Pi 5\nvalhalla-scripted:3.7.0"]
  EA --> MERGE
  AO --> MERGE
  MERGE --> TAR
  TAR --> PI

Public traffic hits TLS on a Pi 4B front proxy; the actual Valhalla container runs on the Pi 5. Same tile archive, different roles.

The build pipeline that survived contact with reality

The canonical pipeline uses osmium extract from a single planet PBF per region — never osmium merge of separate Geofabrik downloads. One source file keeps topology consistent across regional cuts.

valhalla-build.sh runs seven checkpointed phases:

Phase What Threads
parse_ways Initialize scratch (way_nodes.bin, ways.bin) 16
io_split Move way_nodes.bin to a second NVMe —
parse_tail Relations, nodes, edges (random reads) 16
build Tile generation (I/O bound) 4
enhance Hierarchy, shortcuts, restrictions 16
elevation Bake SRTM into edges 2
validate Spatial index restore 16

Checkpoints mean a failed phase resumes without redoing completed work.

Thread count is the dominant knob

Dropping build threads from 32 to 4 eliminated the thrashing spiral. At 32 threads, most workers sat in uninterruptible sleep waiting on NVMe queue depth. At 4, the drive could actually finish requests.

This sounds slow. It is not — because the alternative is effectively stalled.

I/O split reverses at planet scale

A Czech micro-benchmark (1.7 GB scratch) showed zero benefit from splitting way_nodes.bin onto a second drive at 4 threads. At Eurasia+Africa scale (188 GB scratch, 3× RAM), the same trick delivered +41.8% on the build stage:

Layout Build time
4 threads, single drive 13h 07m
4 threads, I/O split 7h 38m

The split run had more page faults, not fewer — but two NVMe controllers serve them in parallel. Micro-benchmarks lie when the bottleneck shifts from CPU cache contention to drive queue depth.

Rule of thumb: parse at full threads, build at 4, put way_nodes.bin on your fastest drive, split it to a second physical NVMe before the random-read phases.

Serving on the Pi

The Pi 5 runs ghcr.io/valhalla/valhalla-scripted:3.7.0 (native arm64). Tiles load from valhalla_tiles.tar via tile_extract — no rebuild in the container.

Memory tuning matters on 8 GB-class hardware. I reduced Thor label reservations and matrix limits in valhalla.json so concurrent route requests do not OOM the box.

Verification is scripted: known regression cases plus continental sanity routes (NYC↔LA, Berlin↔Tokyo, Sydney↔Melbourne, etc.). Deploy only passes canary on a spare instance before swapping production tiles.

Elevation: two different things

This trips people up:

I had routing elevation globally but /height returning null outside Europe because only a 40 GB regional SRTM subset was on the Pi. Fix: rsync the full dataset from the build host. No tile rebuild, no container restart — Valhalla reads SRTM tiles on demand.

Should you do this?

Do it if you want a private, global routing API and already have:

Do not do it if you just need routing for one country — use Geofabrik extracts and a single-machine build. Planet scale is a research project that happens to serve HTTP.

Resources

The orchestration scripts, merge tool, env templates, and benchmark write-ups live in a private repo for now. A sanitized public release may follow; the ideas above are the portable part.

Self-hosting at planet scale is mostly about refusing to run 32 threads against a 273 GB working set. The Pi is the easy part.