XiaoZhi Device

GPIO Pin Map

Button Behaviour

Audio Pipeline

Partition Table

Assets partition is NOT updated by OTA — font/sound changes require a full USB flash.

Deploy Workflow (Server Build — preferred)

# 1. Bump version — do this BEFORE building
sed -i 's/set(PROJECT_VER ".*")/set(PROJECT_VER "2.2.X")/' \
    /home/ubuntu/danscodellaro.com/xiaozhi-esp32/CMakeLists.txt

# 2. Build
source /home/ubuntu/esp-idf/export.sh
cd /home/ubuntu/danscodellaro.com/xiaozhi-esp32
idf.py build

# 3. Verify the embedded version matches
strings build/xiaozhi.bin | grep '^2\.'
# If wrong version — STOP. Fix CMakeLists.txt and rebuild.

# 4. Copy binary to web root
cp build/xiaozhi.bin \
    /home/ubuntu/danscodellaro.com/frontend/esp32/xiaozhi/xiaozhi.bin

# 5. Update server version constant (must match exactly)
sed -i "s/_OTA_FIRMWARE_VERSION = '.*'/_OTA_FIRMWARE_VERSION = '2.2.X'/" \
    /home/ubuntu/danscodellaro.com/api/server.py

# 6. Restart Flask
sudo systemctl restart travel-api

# 7. Free ~350 MB disk space
rm -rf build/

Version constant and binary MUST be in sync — mismatch causes an infinite OTA loop.

Deploy Workflow (Windows Build → Server)

# Build on Windows:
cd C:\Users\dansc\xiaozhi-esp32
python idf_build.py build

# SCP to server:
scp -i C:/Keys/lightsail-cos.pem build/xiaozhi.bin \
    ubuntu@foundry.danscodellaro.com:/home/ubuntu/danscodellaro.com/frontend/esp32/xiaozhi/xiaozhi.bin

# Then run steps 5-6 above on the server.

OTA Response Format

The OTA endpoint (POST /esp32/xiaozhi/ota.php) returns:

{
  "firmware": {
    "version": "2.2.26",
    "url": "https://www.danscodellaro.com/esp32/xiaozhi/xiaozhi.bin"
  },
  "websocket": { "url": "wss://api.tenclass.net/xiaozhi/v1/" },
  "server_time": { "timestamp": 1775386136101, "timezone_offset": 600 }
}

The device compares firmware.version against its running version. If server is newer → download + flash + reboot. On success the app calls MarkCurrentVersionValid() to prevent OTA rollback.

Pointing Device at This Server (First Time)

# Device in download mode (hold BOOT + power cycle via USB)
python set_ota_url.py
# Patches NVS at offset 0x9000 only — no firmware reflash needed

Custom Files (untracked — required for build)

These files are not committed to git. They are required for a correct build — never delete them, and always restore them when cloning to a new machine. The committed main/CMakeLists.txt lists both telemetry.cc and media_player.cc as source files; a missing file causes a "no such file" build error.

⚠️ Known Footgun: Circular GetInstance() Deadlock

If a singleton's constructor calls GetInstance() on another singleton that also calls back into the first, the C++11 magic-static initialisation guard will deadlock the main task permanently. The device hangs at that point on every boot with no panic — it just silently freezes.

This hit us in v2.2.6 and v2.2.7: McpServer() constructor called MediaPlayer::RegisterMcpTools(), which called McpServer::GetInstance() — re-entering the already-held static-init mutex. Fixed in v2.2.8 by moving the RegisterMcpTools() call from the constructor to AddCommonTools().

Rule: never call GetInstance() on a singleton from within another singleton's constructor.

.axv Binary Frame Format

Each frame (video or audio) is prefixed with a 16-byte header. All four fields are big-endian (network byte order). The firmware rejects any file with wrong byte order — it logs "Frame sync lost" and stops silently.

Common mistake: Python's struct.pack uses native (little-endian) byte order by default. Always use struct.pack(">IIII", ...) — the ">" prefix is required. Type constants are also easy to get wrong: video=0, audio=1 (not 1/2). The encoder script is pack_axv.py in the same directory as each .axv file.

Frames are interleaved in presentation-timestamp order. At 8 fps video and 60 ms audio, the ratio is approximately 1 video frame per 2 audio packets (125 ms vs 60 ms).

Creating .axv Files

Use scripts/make_axv.py on the server. It requires ffmpeg in PATH and handles aspect ratio padding automatically.

From a video file

python3 scripts/make_axv.py --input video.mp4 --output out.axv

Generate a test pattern (30 s SMPTE bars + 440 Hz tone)

python3 scripts/make_axv.py --test --output test_video.axv

The script extracts video at 240×240 with black-bar padding, re-encodes audio to Opus at 24 kHz mono, then interleaves both streams by presentation timestamp.

ffmpeg internals

# Video frames (extracted as individual JPEGs):
ffmpeg -i input.mp4 \
  -vf "scale=240:240:force_original_aspect_ratio=decrease,pad=240:240:(ow-iw)/2:(oh-ih)/2:black" \
  -r 8 -q:v 5 frames/%06d.jpg

# Audio (Opus in OGG, then raw packets demuxed):
ffmpeg -i input.mp4 -vn -ar 24000 -ac 1 \
  -c:a libopus -b:a 24k -frame_duration 60 -vbr off audio.ogg

Media Library JSON Schema

Each item in the library (api/media_library.json) follows this schema:

Add an item via API

curl -X POST https://www.danscodellaro.com/esp32/xiaozhi/media/library \
  -H "Content-Type: application/json" \
  -d '{
    "id": "my-track",
    "title": "My Track",
    "description": "An Opus audio file.",
    "type": "audio",
    "tags": ["music"],
    "duration_s": 120,
    "url": "https://www.danscodellaro.com/esp32/xiaozhi/media/files/my-track.ogg"
  }'

Posting an item with an existing id performs an upsert (update). Delete via DELETE /esp32/xiaozhi/media/library/<id>.

Server Infrastructure

Host: AWS Lightsail Ubuntu (foundry.danscodellaro.com) · Web: Caddy · API: Python Flask (travel-api.service, port 5001)

Firmware Version History

† 2.2.5 "running on device" is the Windows English rebuild — different binary from the first Linux 2.2.5 build.

Known Issues