camera-webui/README.md

# camera-webui

A camera service for single-board computers: a live video feed served through a small web UI, with
face recognition as a later stage.

**Target hardware is a Jetson Orin Nano.** A Raspberry Pi 5 is the development and test platform —
convenient, and available first — but it is not where this is meant to end up. That distinction is
load-bearing: the two boards do not share a camera stack, and only one of them can realistically run
face recognition on a live stream.

## Platforms

| | Jetson Orin Nano | Raspberry Pi 5 |
|---|---|---|
| Role | **target** | test / development |
| Arch | aarch64 | aarch64 |
| Stack | L4T / JetPack, CUDA, TensorRT | Raspberry Pi OS (Debian 12) |
| Camera path | V4L2 / GStreamer (Argus for Bayer CSI sensors) | libcamera / `rpicam` / `picamera2` |
| Inference | GPU + DLA | CPU only |

Recorded for `mikkeli-orin-nano-2` (`192.168.2.209`): L4T 36.4.4, CUDA 12.6, TensorRT 10.7. **Confirm
against whichever unit is actually used** — there is more than one Orin here, and the camera is going
to whichever one gets it wired first.

## ⚠ The camera stacks are not the same, and that is the main design constraint

This is the thing to get right early, because retrofitting it is expensive:

- **Pi 5** uses libcamera. `picamera2` is the idiomatic Python entry point.
- **Orin Nano** uses V4L2 and GStreamer. CSI Bayer sensors go through NVIDIA's Argus stack
  (`nvarguscamerasrc`); USB/UVC cameras are plain V4L2.
- Code written directly against `picamera2` **will not run on the Orin at all.**

So: **put capture behind an interface** with one backend per platform, and let everything above it —
streaming, the web UI, recognition — depend only on "a source of frames". Pick the backend at
runtime from what the machine actually has, not from a build flag.

The same applies to inference. On the Orin, face recognition should go through TensorRT and can use
the GPU or DLA. On the Pi 5 it is CPU-only and will not keep up with a live stream at full
resolution. Treat the Pi as proof the *pipeline* works, never as evidence the *performance* works.

## Current state: working camera on the Orin

**An IMX219 is live on the Orin Nano's CAM0**, confirmed by capture, not just by a bound driver:

```console
$ v4l2-ctl --list-devices
vi-output, imx219 9-0010 (platform:tegra-capture-vi:1):
	/dev/video0

$ v4l2-ctl -d /dev/video0 --stream-mmap --stream-count=1 --stream-to=/tmp/frame.raw
$ ls -l /tmp/frame.raw
16163840        # 3280 x 2464 x 2 bytes — full sensor, 10-bit Bayer
```

The frame was 99.8% non-zero across 40 distinct values, so it is real sensor data rather than an
allocated buffer. Formats: `RG10` (10-bit Bayer) at 3280x2464/21fps, 1920x1080/30fps, 1640x1232/30fps.

### ⚠ The Orin needed a device-tree overlay — it does NOT auto-detect

This cost a whole debugging session, because the symptom is identical to a dead camera: no
`/dev/video0`, no sensor lines in `dmesg`, nothing on i2c.

**The Pi auto-detects cameras. The Jetson does not.** `camera_auto_detect=1` has no equivalent — the
sensor's overlay must be selected explicitly, and until it is, a perfectly good camera is invisible:

```bash
sudo /opt/nvidia/jetson-io/config-by-hardware.py -l                     # list modules per header
sudo /opt/nvidia/jetson-io/config-by-hardware.py -n 2="Camera IMX219-A" # header 2 = 24-pin CSI
sudo reboot                                                            # required
```

⚠ **`-n` needs the header number** (`2=` for the CSI connector). Without it the tool defaults to the
40-pin header and fails with `No configuration found for Camera IMX219-A on Jetson 40pin Header!`,
which reads like the module is unsupported.

Naming: **`-A` is CAM0, `-C` is CAM1.** `extlinux.conf` is backed up before the change, and the
result is one `OVERLAYS` line — remove it and reboot to undo.

⚠ **`v4l2-ctl` is not installed by default** (`sudo apt install v4l-utils`), and its absence reports
as `command not found`, which is easy to misread as "no camera".

### The Pi 5 camera is still dead

Separate fault, genuinely hardware: not detected, traced to a bad cable with the module possibly
damaged too. **Do not generalise the Orin's fix to it** — the Pi's auto-detect had nothing to
configure, so an overlay is not the answer there.

Evidence, for contrast with the Orin's *configuration* fault above:

```console
$ rpicam-hello --list-cameras
No cameras available!
```

The imaging pipeline was up (`pisp_be` loaded, `/dev/media0-2` present), but
`/sys/bus/i2c/devices/` held only `i2c-13` and `i2c-14` — **no camera i2c bus and no CFE bound** —
and `dmesg` had no sensor probe lines. On the Pi, `camera_auto_detect=1` loads an overlay when it
finds something, so an absent bus means the firmware found nothing to probe. Unchanged across a
reboot, with the module's IR LEDs lit.

⚠ **Lit IR LEDs prove power, not a working sensor.** The illuminator is wired to the 3.3V rail
independently of the i2c and CSI lanes, so it lights whenever the ribbon is seated well enough to
carry power — while a creased or partly-seated cable can still have broken exactly the i2c traces
detection depends on. Which is what happened.

⚠ **Do not treat `/dev/video*` as evidence of a camera.** Those nodes exist on both boards with
nothing attached — on the Pi 5 they are the codec and ISP blocks, on the Orin
`tegra-camrtc-ca`/`/dev/media0` is the VI platform block.

### Checking a connection

```bash
# Pi 5
rpicam-hello --list-cameras
dmesg | grep -iE 'imx|ov5647|cfe'
ls /sys/bus/i2c/devices/          # a camera bus should appear

# Orin Nano
grep -i OVERLAYS /boot/extlinux/extlinux.conf   # ⚠ CHECK THIS FIRST — no line, no camera
v4l2-ctl --list-devices
dmesg | grep -iE 'imx|camera|argus|vi:'
```

On the Orin, check the overlay **before** suspecting hardware. An unconfigured Jetson and a dead
camera look exactly alike.

Cable notes worth keeping, since they cost a module here:

- The **Pi 5 uses the narrow 22-pin FPC**; Pi 4-era modules ship with a **15-pin** cable and need the
  adapter. Both Pi 5 connectors (`CAM/DISP 0` and `1`) are dual-purpose, so either accepts a camera.
- **Jetson carrier boards use their own pinout** — a cable that fits a Pi does not necessarily carry
  the same signals. Match the cable to the carrier, not to the sensor.
- Ribbon orientation differs at each end. Always power off first.

## Planned stages

1. ~~**Capture** — get a sensor detected on the target, grab a still, establish resolution and
   format.~~ **Done on the Orin** (IMX219, `/dev/video0`, `RG10`, full frame captured).
2. **Capture abstraction** — one interface, a backend per platform, chosen at runtime.
3. **Live feed** — MJPEG first, because it works in any browser with no negotiation. WebRTC later
   only if latency demands it.
4. **Web UI** — one page: live feed and basic controls.
5. **Face recognition** — detection before recognition, on downscaled frames, off the capture thread.
   TensorRT on the Orin.

Each stage should be usable on its own before the next begins.

## Development

Codex runs on the boards themselves, so development happens on the target rather than cross-compiled
or deployed. The model endpoint and MCP gateway live on `halogen` and are reachable from both boards
by name. See [AGENTS.md](AGENTS.md).