GPS via gpsd, Spotify OAuth, barge-in, audio ducking, diarization, intent gate, Pi config.
Date: 2026-06-11 (Session 6) Commits: 3020b87, 008fc97, b6058f5, 4329bf4, 9590315, 0c87d7d, 1f19602, a3b2066, ee082a4, d12711f - plus the still-uncommitted audio/diarization.py, core/intent_gate.py, and the wiring edits to pipeline.py, factory.py, app.py, config.py, pyproject.toml, and the mobile manifests. Theme: Give EXCAR senses and reflexes - real GPS, music control, the ability to be interrupted, social awareness about *who* is speaking, and a production config for the car.
Reports 01–05 built the body, the app, and the data. This session added perception (where am I, what's playing, who's talking) and reflexes (stop when interrupted, duck the music, skip the LLM for filler).
3020b87)What. GpsdLocationProvider (services/gps.py) - precise position from a GPS module on the Pi, via the gpsd daemon.
Why. IPLocationProvider is city-accurate at best - fine for laptop dev, useless for a car that needs to know which road it's on. The car has a real GPS module; EXCAR has to read it.
How. gpsd speaks a line-oriented JSON protocol on a local TCP socket (127.0.0.1:2947). The provider opens a raw socket (no gpsd Python dependency - the protocol is simple), enables watch mode (?WATCH={"enable":true,"json":true}), polls (?POLL;), and reads reports until a TPV (Time-Position-Velocity) message carries an actual lat/lon fix, with a fix_timeout (default 5s). It returns a Location(..., source="gps") after reverse-geocoding. factory.build_location_provider selects it on location.provider: gpsd.
Purpose / goal. Accurate, in-car positioning - the foundation for real navigation and any location-aware behaviour.
008fc97, b6058f5, 1f19602)What. A three-layer Spotify integration (services/spotify.py): SpotifyAuth (OAuth 2.0 PKCE), SpotifyClient (Web API wrapper), SpotifyTool (the LLM-callable tool); plus device routes /spotify/auth, /spotify/callback, /spotify/status (api/routes/spotify.py) registered in the FastAPI app.
Why. Music is the single most-used in-car feature. The driver should say "play some jazz" and have it happen - which means EXCAR needs an authenticated Spotify session and the model needs a tool to drive it.
How.
get_auth_url() generates a code_verifier + S256 challenge; the user authorises in a browser; exchange_code() swaps the code for tokens, persisted to data/spotify_token.json. get_valid_token() auto-refreshes 30s before expiry. Implemented with urllib.request only - no requests dependency. PKCE (not a client secret) is the right flow because EXCAR is a public client (device/app), with no safe place to keep a secret.
play(query) (searches then plays), pause, skip, now_playing. HTTP errors are mapped to *spoken-friendly* messages - 401 → "reconnect in the EXCAR app", 403/404 → "open Spotify on your phone first".
SpotifyTool exposes one action enum (play / pause / skip / now_playing) to the LLM; factory.build_tools adds it only if configured (get_spotify_tool() returns None without a client ID, so the assistant still boots without Spotify).
Purpose / goal. Hands-free, voice-driven music - the headline everyday feature - with auth that's secure for a public client and failure messages a driver can act on without looking.
a3b2066)What. The driver can talk over EXCAR and it stops within ~60ms, mid-sentence.
Why. A co-pilot that finishes its paragraph while you're trying to correct it is infuriating and unsafe. Real conversation lets you cut in. The interrupt signal was already threaded through playback from Report 01 - this session activated it.
How. When barge_in=True, _respond (core/pipeline.py) spawns a daemon _barge_in_monitor thread that opens a 16kHz input stream and watches block RMS; above an energy threshold (0.035) it set()s the shared threading.Event that playback already honours. The _speak_worker checks the event and drains the sentence queue *without speaking* once interrupted, so EXCAR goes quiet immediately but the turn still completes cleanly (no deadlock on the worker join).
Purpose / goal. Natural, interruptible turn-taking - a large jump in how human the assistant feels.
4329bf4, 0c87d7d)What. AudioDucker + DuckingContext (audio/ducking.py); SpeakerPlayer ducks system/music volume while EXCAR speaks (audio/player.py).
Why. If music is playing and EXCAR talks over it at full volume, you can't understand it. Every good in-car assistant lowers the music while it speaks, then restores it.
How. AudioDucker is best-effort, fire-and-forget OS volume control: pactl set-sink-volume on Linux (the Pi), osascript ... set volume on macOS (dev). DuckingContext is a context manager that ducks on enter and restores on exit; a None ducker makes it a no-op so callers never branch. SpeakerPlayer wraps playback in it. factory injects an AudioDucker() into the player.
Purpose / goal. Intelligible speech over music without the driver reaching for the volume - table-stakes polish for a car product.
audio/diarization.py, uncommitted)What. is_owner(audio) verifies captured audio matches the enrolled owner's voiceprint *before* STT runs; called at the top of process_utterance.
Why. Wake mode (Report 01) gates on *what* was said. Diarization gates on *who* said it - so a passenger saying "EXCAR" can't hijack the assistant, and the driver gets a personal co-pilot. It spends the voiceprint enrolled in Report 05.
How. Loads the d-vector from data/voice_profile.npy, encodes the utterance with resemblyzer, and compares via cosine similarity against a 0.70 threshold. Two deliberate design choices:
must never go deaf because enrollment wasn't done or a library hiccupped.
the owner rather than ever triggering on a passenger - a missed wake is a minor annoyance; answering the wrong person is a serious one.
Purpose / goal. Owner-only attention - the privacy and personalisation backbone, and the answer to "what if a passenger talks?"
core/intent_gate.py, uncommitted)What. should_respond(text) runs *after* STT, *before* the LLM, and drops filler: empty strings, single words, and acknowledgement tokens ("ok", "yeah", "thanks", "bye"…).
Why. Sending "uh huh" to the LLM wastes ~300–700ms and a token spend on a turn that deserves no reply. Cheap, deterministic filtering belongs before the expensive model.
How. Normalise (strip trailing punctuation, lowercase); reject if empty, under _MIN_WORDS (2), or in a frozenset of ack tokens. Pure function, no dependencies, trivially testable.
Purpose / goal. Lower latency and cost, and stop EXCAR from chattily replying to every backchannel - it should answer questions, not acknowledge acknowledgements.
a3b2066, ee082a4)What. build_pipeline (factory.py) now wires GPS, Spotify, PersistentMemory, the AudioDucker, barge_in=True, diarization and the intent gate into one loop.
How. process_utterance becomes: is_owner? → transcribe → should_respond? → _respond. _respond injects PersistentMemory.get_recent_summaries() as a synthetic opening exchange before live history, then appends each turn back to the daily log. This is where the persistent memory built in Report 02 finally reaches the prompt.
Purpose / goal. Compose every sense and reflex into the single turn flow from Report 00's one-line diagram - without the pipeline knowing the concrete classes.
9590315)What. config/config.pi.yaml - the in-car deployment profile.
Why. Laptop defaults (English, small Whisper, IP location, cloud-only) are wrong for the car (Turkish, lighter Whisper, gpsd, hybrid brain). The car needs its own config without editing code.
How. A parallel YAML tuned for the Pi: lighter STT model, location.provider: gpsd, Turkish voice/replies, hybrid LLM. Selected with excar -c config/config.pi.yaml.
Purpose / goal. One binary, two deployment profiles - clean separation of dev and production, the last piece of "built for commercialisation".
_speak_worker keeps draining the queue after interrupt (silently) so worker.join() in _respond always returns; a naive "stop the thread" would hang the turn.
get_spotify_tool() returns None and the tool is simply omitted - the assistant degrades, it doesn't fail.
EXCAR *more* responsive, never mute.
EXCAR now perceives (GPS, now-playing, owner-vs-passenger), reflexes (barge-in, ducking, filler-skipping), controls music by voice, and has a production Pi config. Backend Session-6 work is pushed; diarization, intent gate, and the wiring edits remain to be committed (handled in Session 7).
Still pending into Session 7: app icon (1024×1024) → TestFlight build; Spotify mobile integration (Connect button, Now Playing widget); commit the 8 pending files. Tracked in docs/STATUS.md.