Jose Gibson
← All Projects

Railtrack OD on DMP Di1

Embedded dual-model railway obstruction pipeline running BiSeNetV2 and YOLOv8s on the DMP Di1/iCatch A3000 NPU with live camera streaming and integration into a fleet DMS/ODS dashboard.

C++ inference applicationDMP Di1 / iCatch A3000 NPU SDKBiSeNetV2 FP16 and YOLOv8s INT8 compiled model binariesOpenCV, V4L2, YUYV camera input, MJPEG streamingFlask, Waitress, PyYAML, PyInstaller for dashboard packaging
⚡ Hardware ISP letterboxing failed with `keep_aspect=true`, br…⚡ Headless ISP path depended on OSD/display behavior and faile…⚡ OpenCV `cv::resize` was unstable on the board's ARM build.⚡ All-hardware preprocessing was fast but damaged detection ac…

Outcomes

  • ●Stable 4.5 to 5 FPS dual-model pipeline on DI1 hardware.
  • ●Camera-to-dashboard latency documented under 200 ms.
  • ●YOLO detection confidence recovered from near-zero under distorted hardware stretch to above 0.87 for key objects after CPU letterboxing.
  • ●Long-run reliability documented as no crashes in 60+ minute runs.
  • ●Production handoff includes final binaries, models, bug reports, deployment docs, and setup scripts.

Portfolio Highlights

  • →Ported a railway obstruction detection pipeline to the DMP Di1 edge-AI SoC, running BiSeNetV2 and YOLOv8s concurrently on the A3000 NPU.
  • →Diagnosed vendor SDK and hardware-preprocessing limitations, then designed a hybrid hardware/software preprocessing path that restored YOLO accuracy while preserving segmentation speed.
  • →Built an end-to-end embedded workflow: cross-compile, deploy, live camera capture, MJPEG/SSE dashboard, device scripts, and handoff documentation.

Snapshot

  • Period: April 2026
  • Source: `/Users/jose/Developer/work/sparshiq/railtrack-od-dmp-di1`
  • Domain: Embedded AI, edge computer vision, hardware deployment
  • Target: DMP Di1 edge AI SoC / iCatch A3000 NPU
  • Status: Production-ready handoff documented

Portfolio Summary

This project turned the railtrack research pipeline into a deployable edge-AI system on real DI1 hardware. The final system runs BiSeNetV2 for rail/track-bed segmentation and YOLOv8s for obstacle detection on the A3000 NPU, supports live USB camera input, streams overlays to a dashboard, and includes build/deploy/run automation for the workstation, build machine, and device.

Stack

  • C++ inference application
  • DMP Di1 / iCatch A3000 NPU SDK
  • BiSeNetV2 FP16 and YOLOv8s INT8 compiled model binaries
  • OpenCV, V4L2, YUYV camera input, MJPEG streaming
  • Flask, Waitress, PyYAML, PyInstaller for dashboard packaging
  • Shell orchestration via `di1.sh` and `di1-direct.sh`
  • Cross-compilation through WSL/Linux and later local Docker toolchain
  • SSH, rsync/tar deployment, direct Ethernet device workflow

What I Built

  • C++ dual-model inference app for BiSeNetV2 segmentation and YOLOv8s detection.
  • Build/deploy/run orchestration across Mac, WSL/build machine, and DI1 device.
  • USB camera capture pipeline and live stream mode.
  • MJPEG/SSE dashboard integration for low-latency visualization.
  • Integration with the packaged fleet dashboard's ODS/APC panel for live object detection and segmentation overlays.
  • Device-side run scripts and final deployment handoff command.
  • Hardware validation docs and bug reports for SDK/vendor limitations.
  • Local Docker-based cross-compile workflow to reduce build-deploy iteration time.

Key Decisions

  • Created a hybrid preprocessing path: BiSeNet stayed on hardware ISP stretch for speed, while YOLO moved to CPU letterboxing for accuracy.
  • Implemented CMA repacking by staging USB YUYV data into DI1-owned contiguous memory so the ISP would behave like it had local sensor input.
  • Replaced unstable OpenCV resizing with manual C++ resizing loops.
  • Forced stable USB camera settings to reduce DMA and driver pressure.
  • Added dashboard streaming with MJPEG and SSE so status/events could update with low latency.
  • Moved toward local Docker cross-builds to avoid dependence on the offline build machine.

Development Timeline

  1. Added initial DI1 deployment source and `di1.sh` build/deploy/run orchestration.
  2. Integrated raw V4L2 MMAP capture and packaged a fleet dashboard.
  3. Added headless YOLOv8s demo, BiSeNet debug notes, deployment notes, and model conversion/probe scripts.
  4. Integrated MJPEG server, diagnostic tools, dashboard SSE/client-side overlays, and robust USB camera support.
  5. Restored live stream with software YUYV preprocessing and documented hardware preprocessing reproduction issues.
  6. Refactored Python research source into the production repo and optimized pipeline geometry.
  7. Stabilized the dual-model hybrid pipeline, created local Docker build tooling, finalized handoff backups/docs, and added final deployment command.
  8. Initialized devlog CLI for project management
  9. Stabilized dual-model hybrid pipeline (BiSeNetV2 + YOLOv8s) at 5 FPS on DI1 hardware
  10. GoalCapture side-by-side video of YOLO hardware-stretch failure vs CPU-letterbox success
  11. Integrated live object detection and segmentation overlays into the ODS/APC panel
  12. GoalVerify long-run stability (> 60 mins) of the MJPEG/SSE stream on the target hardware
  13. GoalResolve Disk Full Crash and Implement Prioritized Logging
  14. ShippedFixed 3.5GB log file issue by implementing Level-Based Logging in C++ and run.sh.
  15. DecidedSuppress per-frame profiling by default (INFO level) to reduce latency and disk I/O.
  16. FixedV4L2 camera devices hang after crashes requiring aggressive cleanup
  17. FixedV4L2 camera devices could hang after crashes and require aggressive cleanup.
  18. LearningMitigated 100% disk usage on device by truncating 3.5GB log and redirecting stdout to /dev/null in run.sh.
  19. DoneResolve Disk Full Crash and Implement Prioritized Logging
  20. LearningIdentified and verified local Docker workflow using di1-direct.sh. This bypasses the need for aspire7 by building inside the 'di1-toolchain:native' container via Rosetta.
  21. GoalVerify build and deploy using local Docker workflow
  22. ShippedSuccessfully built and deployed using local Docker workflow and sparshberry jump host.
  23. DoneVerify build and deploy using local Docker workflow
  24. ShippedImplemented 'net-up' and 'net-down' in di1-direct.sh to manage project-confined network alias for direct device access. Connectivity is now fully automated and transparent.
  25. ShippedMigrated noisy std::cout and profiling logs to use the level-based logger (LOG_DEBUG for per-frame logs).
  26. FixedHardware ISP letterboxing damages YOLO aspect ratio
  27. FixedMJPEG server could fail to bind after restarts due to lingering TCP state
  28. ShippedFully migrated all std::cerr and std::cout in src_di1/app to the level-based logger.
  29. GoalAdd standards-led ODS product and technical research report to docs
  30. ShippedAdded docs/standards_product_technical_report.md covering India-first ODS standards context, global railway safety standards, competitor methods, research algorithms, product lines, and milestones.
  31. DoneAdd standards-led ODS product and technical research report to docs