Jose Gibson
← All Projects

Railtrack Detector Research

Railway obstruction-detection research pipeline that evolved from rail-line detection to BiSeNetV2 segmentation plus dual-YOLO obstacle detection.

Python, PyTorch, torchvisionOpenCV, NumPy, SciPy, PillowUltralytics YOLOv8Matplotlib diagnosticsRoboflow/COCO/YOLO dataset conversion
⚡ L4R NLB annotations were in native 1920x1080 coordinate spac…⚡ MobileNetV2/Rail-Net polylines were brittle on Indian locomo…⚡ Segmentation masks jittered frame-to-frame, requiring tempor…⚡ Small railway obstacle dataset had too few car/rock examples…⚡ SynRailObs had useful obstacle masks but sparse rail-line la…

Outcomes

  • ●Working research pipeline at roughly 9 FPS on Apple MPS for BiSeNetV2 plus dual YOLO on 1920x1080 input.
  • ●Fine-tuned YOLOv8n obstacle model reached reported mAP50 of 0.857 overall, with weak classes identified as data-limited.
  • ●A clear path emerged from research prototype to embedded deployment on DI1 hardware.

Portfolio Highlights

  • →Built a railway track obstruction detection pipeline using BiSeNetV2 segmentation and dual YOLOv8 detectors for track-aware alerts.
  • →Diagnosed and fixed a dataset coordinate-space bug that inflated validation accuracy while destroying real label quality.
  • →Designed a 4-class obstruction segmentation training strategy by combining dense RailSem19 labels with SynRailObs obstacle masks.

Snapshot

  • Period: April 2026
  • Source: `/Users/jose/Developer/work/sparshiq/experiments/railtrack-detector`
  • Domain: Railway safety, computer vision, ML research prototype
  • Status: Research prototype and model pipeline foundation

Portfolio Summary

This project explored real-time railway track obstruction detection for forward-facing locomotive footage. The initial plan combined rail-line detection, corridor geometry, obstacle detection, and stereo depth. During development, the pipeline shifted from brittle Rail-Net/UFLD-style rail polylines to a segmentation-first BiSeNetV2 approach, then layered YOLO-based obstacle detection and a future 4-class obstruction segmentation strategy on top.

Stack

  • Python, PyTorch, torchvision
  • OpenCV, NumPy, SciPy, Pillow
  • Ultralytics YOLOv8
  • Matplotlib diagnostics
  • Roboflow/COCO/YOLO dataset conversion
  • Apple MPS and RunPod-oriented GPU workflows

What I Built

  • Rail corridor geometry modules for rail boundaries, zones, and bbox-vs-track intersection.
  • Dataset loaders for L4R NLB, RailSem19, RailSem19 4-class augmentation, and SynRailObs crops.
  • Training and visualization scripts for rail-line, segmentation, and obstruction experiments.
  • BiSeNetV2 inference path for rail/track-bed segmentation.
  • Dual-YOLO inference strategy combining railway-finetuned YOLO with original COCO YOLO for vehicle retention.
  • Temporal smoothing stack for video overlays.
  • Documentation covering plans, implementation, large files, RunPod setup, existing solutions, and a dated devlog.

Key Decisions

  • Pivoted from Rail-Net polylines to BiSeNetV2 segmentation because pretrained RailSem19 weights gave denser, more stable track masks.
  • Dropped extrapolation when it produced visual artifacts near the vanishing point.
  • Used dual YOLO models so railway-specific fine-tuning did not erase COCO vehicle detection.
  • Planned 4-class BiSeNetV2 obstruction segmentation by copy-pasting SynRailObs obstacle crops onto RailSem19 track regions.
  • Chose BiSeNetV2 over SegFormer because local texture anomaly detection did not require a heavier attention model and local weights were available.

Development Timeline

  1. Defined goal: detect animals, logs, people, and vehicles on railway tracks from locomotive footage.
  2. Built initial modules for rail detection, corridor construction, bbox intersection, depth stub, and pipeline orchestration.
  3. Fixed the L4R coordinate-space bug and added multi-task training/dataset infrastructure.
  4. Added a polynomial rail-regression architecture, perspective-aware corridor geometry, and tests, then pivoted to BiSeNetV2 segmentation.
  5. Added temporal smoothing to reduce jitter in video overlays.
  6. Added segmentation plus YOLO pipeline, railway obstacle dataset prep, dual-YOLO merge, SynRailObs exploration, and 4-class obstruction segmentation design.
  7. 2026-04-14 to 2026-04-17: Began moving research outputs toward DI1 deployment with model export, compiled binaries, and C++ app scaffolding.