Gurkirat Singh

Focused window on the virtual desktop (multi-monitor)
  │  Win32: GetForegroundWindow, GetWindowRect,
  │  monitor union (P/Invoke, per-monitor DPI aware)
  ▼
WindowTracker ── azimuth θ, distance ──┐
                                      ▼
WASAPI loopback capture ── PCM chunks ──► Spatializer (DSP)
  (48 kHz float stereo)                    ├ ITD: ring-buffer delay, far ear (max 630 µs)
                                           ├ ILD: equal-power cos/sin panning
                                           └ HRTF: 72-direction KEMAR table
                                                · split-complex FFT (from scratch)
                                                · overlap-add convolution
                                                · azimuth crossfade + smoothing
  ▼
Output device (feedback-loop safe, headphones)

• Window-following audio: azimuth and distance computed from the focused window's rect across the virtual desktop (±70° front-arc model)
• DSP from scratch: split-complex FFT verified against a direct DFT (N=512, max difference 1.5e-5), overlap-add HRTF convolution, and per-window azimuth filter crossfades
• HRTF: 72-direction KEMAR table; all 1420 raw HRIRs parsed and self-verified, resampled 44.1 → 48 kHz
• Measured latency: 2.8 / 29.4 / 55.3 ms (min/avg/max) end to end through the capture → spatialize → render path
• Tested: xUnit DSP suite running on GitHub Actions CI
• Low-level Win32: P/Invoke into user32.dll, per-monitor DPI awareness, asymmetric-monitor virtual desktop union
• Feedback-loop safe: capture and output devices must differ, so the mix cannot re-enter its own loopback

Visualizer + telemetry interface (M3.5), then elevation and HRTF naturalness work.

PC host (Python mock → C# later)
  │  USB serial, frozen frame format:
  │  [AA 55][ver][type][seq u16][len u16][payload][crc16]
  ▼
RP2350 firmware (C++, Pico SDK)
  ├ frame parser + CRC-16/CCITT-FALSE + resync
  ├ dashboard state machine
  └ fallback: no valid frame in 2 s →
       passive mode (onboard temp + LINK LOST)
  ▼
ST7789 1.14" LCD (240×135 visible window, SPI @ 10 MHz)

• Driver from scratch: init sequence, gamma tables, MADCTL, and the panel's real RAM window (the visible area sits at x 0x28–0x117, y 0x35–0xBB inside the 240×320 die — writing 0–239/0–134 shows nothing)
• Frozen protocol v1: frame layout documented, CRC-16/CCITT-FALSE with known ground truth, resync on bad frames, ~1.3 KB/s cadence budget
• Fault tolerance: a 2-second link timeout drops the dashboard into a passive mode showing onboard temperature + LINK LOST
• Sim before hardware: Wokwi RP2040 + ILI9341 stand-in for logic; the physical Waveshare panel is the source of truth
• Learning in public: C++ written by me from the SDK up as a deliberate learning project (no MicroPython in the shipped firmware)

M2.1 serial byte-pipe + host mock → framebuffer and font → dashboard v1 → real host integration → Cloudflare edge pipeline feeding live stats to this site.

Windows OS
  │  WH_MOUSE_LL hook (JNativeHook)
  ▼
Capture worker (40 ms pulse)
  ├ Win32: GetForegroundWindow, WindowFromPoint (JNA)
  ├ weighted-pulse CSV logging (5 px dead zone + dwell)
  └ Platform.runLater → UI updates
  ▼
JavaFX dashboard (thread-isolated)
  ├ live X/Y + active/hovered window titles
  └ resizable path-trail canvas (multi-monitor normalized)

[ M3 pipeline — in progress ]
  CSV writer → Kafka producer → Postgres consumer
  → DBSCAN / K-Means clustering (10–20 px jitter)
  → heatmap output · Docker Compose

• Weighted-pulse logging: writes CSV rows only on ≥5 px movement or a window change; accumulates dwell time while idle to minimize disk I/O
• Multi-monitor normalization: computes virtual desktop bounds from all monitors and maps raw hook coordinates to canvas percentages
• Thread isolation: hook message pump on a dedicated worker; all UI mutations marshaled through Platform.runLater()

• Kafka producer replacing the CSV writer (event schema first)
• Postgres warehouse consumer with warehouse tables
• DBSCAN / K-Means clustering to heatmap output
• Docker Compose for the whole stack

opencode (AI coding agent, in WSL)
  │  SSE event stream → completed replies
  ▼
voice_agent.py (Windows)
  ├ sanitize()   → tables/code → "[table]" / "[code]"
  ├ split_text() → ≤480-char sentence chunks
  └ POST /generate per chunk (prefetch next while playing)
      ├ s2.exe (s2.cpp)   → S2 Pro · Vulkan · 7 GB VRAM
      └ kokoro_server.py → Kokoro-82M · CPU · ~10× realtime
  ▼
WASAPI → speakers (2 s pre-buffer, self-healing stream)

• Dual engine: S2 for expressive speech with style tags, Kokoro for near-instant CPU narration; toggle via config
• Zero cloud: models run locally; the only network egress is the coding agent's own API
• Stutter-free playback: synthesized chunk-by-chunk with prefetch overlap; tables and code collapse to spoken cues
• Upstream contribution: PR #48 to s2.cpp — an unqualified max breaks builds under NOMINMAX; also avoided a 16 GB RAM OOM by moving to the C++/GGML engine

• Unbeatable AI (Hard): minimax over the full game tree; Easy plays random
• Online multiplayer: room-code matchmaking via a Cloudflare Worker relay, Workers KV for cross-instance state
• Session persistence: sessionStorage-backed rejoin across refreshes
• State-driven rendering: immutable game state with a single render path

• Two-stage local AI: Qwen3-VL reads the image; TranslateGemma translates; both on-device via Ollama
• Zero preprocessing: raw images go straight to the vision model
• Crash-resilient batch mode: streaming per-page writes survive mid-batch failures
• Small footprint: 563 lines of Python, no framework overhead

• Auto-detection: pom.xml, requirements.txt, package.json, Cargo.toml, go.mod → the right base image
• Multi-stage Dockerfiles with layer caching; dependencies installed before source
• Optional deploy.yaml: override name, port, env, volumes, and command
• Verified against: OCR Translate (Python) and MouseFlow (Java/Maven) builds