Jose Gibson
← All Projects

Sparshiq Driver Monitoring System

Commercial-grade Driver Monitoring System for vehicle platforms, combining face detection, landmarks, iris/gaze, head pose, drowsiness, distraction, unsafe-behavior alerts, telemetry, calibration, buzzer control, and dashboard integration.

Python 3.12+FastAPI/control APITFLite, INT8 PTQ, Vela-compiled Ethos-U modelsOpenCV/vision utilitiesFaceMesh, BlazeFace-style face detection, iris landmarks
⚡ Automotive DMS combines perception quality with deterministi…⚡ Head pose and gaze needed smoothing and robust geometry beca…⚡ Side-view occlusion and A-pillar mounting required special h…⚡ The project had to balance product requirements against inco…⚡ NPU deployment required model quantization, Vela compilation…

Outcomes

  • ●Product-level DMS repository with model assets, runtime code, dashboard, docs, deployment scripts, and packaged distributions.
  • ●Supports core DMS signals: EAR, MAR, PERCLOS, KSS, head pose, gaze zone, drowsiness, yawning, distraction, phone/smoking.
  • ●Clear gap analysis exists for seat belt and driver authentication work.

Portfolio Highlights

  • →Built a modular Driver Monitoring System for NXP i.MX93/Ethos-U, integrating face landmarks, iris tracking, head pose, gaze, PERCLOS/KSS, phone/smoking detection, telemetry, and dashboard controls.
  • →Refactored DMS execution from a linear script into a node-based architecture with perception, fusion, logic, action, configuration, and API layers.
  • →Improved alert reliability through PnP-based head pose, smoothing, temporal filtering, calibration, and false-positive handling.

Snapshot

  • Period: December 2025 to April 2026, with major activity February to March 2026
  • Source: `aspire7:D:\DMS_sparshiq`
  • Domain: Automotive AI, Driver Monitoring System, edge deployment
  • Target: NXP i.MX93 / Ethos-U65 NPU
  • Status: Product repository, versioned as `dms` v1.6.0

Portfolio Summary

This is one of the strongest portfolio projects on the Aspire7 drive. It is a full DMS product repo, not just an experiment. The project targets Tata Motors vehicle classes and NXP i.MX93-style edge deployment, with INT8/Vela-compiled models and a modular node-based architecture. The repo documents a transition from script-style execution to a more robust data-flow architecture with Input, Perception, Fusion, Logic, Action, Configuration, and API components.

Stack

  • Python 3.12+
  • FastAPI/control API
  • TFLite, INT8 PTQ, Vela-compiled Ethos-U models
  • OpenCV/vision utilities
  • FaceMesh, BlazeFace-style face detection, iris landmarks
  • WHENet/head pose, L2CS gaze exploration, MoveNet
  • YOLOv4-tiny smoking/calling detection
  • YAML configuration, persistent config editing
  • Dashboard templates, telemetry, buzzer utilities
  • Fabric/Makefile deployment workflow

What I Built

  • Modular node architecture: input, perception, fusion, logic, action.
  • Model suite with face detection, face landmarks, iris tracking, gaze estimation, head pose, EAR/MAR estimators, MoveNet, smoking/calling, WHENet, and face recognition modules.
  • Logic engine for drowsiness, yawning, PERCLOS, KSS, distraction, phone/smoking, talking, and temporal filtering.
  • Control API for runtime state/config changes.
  • Dashboard integration with live metrics and calibration controls.
  • Buzzer feedback for alerts and calibration.
  • NPU/deployment scripts and target launchers for i.MX-class deployment.

Key Decisions

  • Moved toward node-based architecture to separate capture, perception, fusion, logic, and action responsibilities.
  • Used PnP with canonical 3D face geometry for head pose in the alpha suite, improving stability over direct regression.
  • Used temporal filtering/latching for phone and smoking detection to reduce single-frame flicker.
  • Exposed calibration/config controls to dashboard so thresholds could be adjusted without rebuilding.
  • Designed telemetry levels and profiler concepts for production vs debug modes.

Development Timeline

  1. Progress report captured the DMS objective, TML requirements, model suite, and gaps.
  2. Mid-February 2026: Architecture spec defined data/control planes, nodes, sidecar config manager, watchdog, WebRTC strategy, and observability.
  3. February 2026: Built beta suite, L2CS gaze, MoveNet pose, model integration, and dashboard.
  4. February to March 2026: Refactored into modular nodes, consolidated config, result handlers, and perception pipeline.
  5. March to April 2026: Added WHENet, driver recognition, buzzer control, metrics chipbar, improved face processing, and packaged v1.6.0 cleanup.
  6. GoalAdd standards-led product and technical research report to docs
  7. ShippedAdded docs/standards_product_technical_report.md covering India-first DMS standards, global feature direction, competitor methods, research algorithms, product lines, and milestones.
  8. DoneAdd standards-led product and technical research report to docs