Develop Once, Run Fast and Efficiently Across Every Chip
Unified Abstraction · Native SDK · Operator Fusion · Memory Reuse · Instruction Pipelining · Adaptive Quantization — Six Low-Level Technologies at the Core
Compute, energy efficiency, accuracy, latency, concurrency, and long-run stability advance together. The same hardware supports larger models, more streams, and lower power; unified abstraction lets applications be written once and adapted across chips; data, transport, operations, visualization, and iteration all accelerate in step.
Upper‑layer invocation, hardware/software adaptation, deployment integration, and data pipeline efficiency are the core foundations for AI box deployment.
Our AI Box · Core Advantages
Develop Once, Deploy Everywhere
We build a unified hardware acceleration interface, so applications are written once and self‑adapt to different chip capabilities, eliminating repeated multi‑chip porting.
Extreme Low‑Level Speed
Deep customization on native chip SDKs with efficient C/C++ development, avoiding inefficient general‑purpose libraries and pipelines.
Comprehensive Deep Optimization
Triple optimization across operator fusion, memory reuse, and instruction pipelining, plus a fully rebuilt high‑performance data pipeline, to extract the maximum compute from the chip.
Zero‑Dependency Lightweight Deployment
Zero third‑party library dependencies; the runtime plus dependencies stay under 1 MB, so it deploys directly onto resource‑constrained products.
Typical Industry AI Box · Pain Points
High Adaptation Cost
No unified hardware interface; every chip requires separate development, leaving reuse extremely low in multi‑chip scenarios.
Low Runtime Efficiency
Reliance on OpenCV and similar general‑purpose libraries, plus high‑level Python and inefficient ROS2 data pipelines, wastes significant resources.
No Low‑Level Optimization
Stock vendor SDKs run with default logic and no deep operator, memory, or instruction optimization.
Bloated Dependencies Block Deployment
Dependence on Python, OpenCV, and other third‑party libraries pushes the runtime plus dependencies to hundreds of MB or several GB, ruling out resource‑constrained products.
How We Do It
Operator‑Level Fusion
· Merge consecutive operators such as Conv, BN, and ReLU.
· Reduce repeated read/write of intermediate results and memory access overhead.
· Convert multiple operations into a single NPU instruction for higher inference efficiency.
Memory Reuse Strategy
· Manage cache space uniformly based on Tensor lifecycles.
· Released memory blocks can be reused by subsequent Tensors.
· Reduce repeated allocation and off‑chip memory usage.
Instruction Pipeline Parallelism
· Split CPU pre‑processing, NPU inference, and CPU post‑processing into pipeline stages.
· Overlap multi‑frame tasks to reduce wait time.
· Increase overall throughput and reduce perceived data transfer latency.
Adaptive Quantization
· Select quantization precision based on activation distributions per layer.
· Keep INT8 for precision‑sensitive layers and INT4 for less sensitive ones.
· Improve inference speed and reduce compute while preserving accuracy.
SDK‑Level Hardware Acceleration
· Use low‑level C/C++ for higher interface efficiency;
· Direct native SDK access maximizes hardware acceleration vs. OpenCV and third‑party libraries.
Unified Abstraction Layer
· Abstract and unify vendor SDK interfaces to isolate chip differences from upper‑layer algorithms;
· Run the same algorithm across chips—write once, deploy on multiple platforms.
Operator‑Level Fusion
· Merge consecutive operators such as Conv, BN, and ReLU.
· Reduce repeated read/write of intermediate results and memory access overhead.
· Convert multiple operations into a single NPU instruction for higher inference efficiency.
Memory Reuse Strategy
· Manage cache space uniformly based on Tensor lifecycles.
· Released memory blocks can be reused by subsequent Tensors.
· Reduce repeated allocation and off‑chip memory usage.
Instruction Pipeline Parallelism
· Split CPU pre‑processing, NPU inference, and CPU post‑processing into pipeline stages.
· Overlap multi‑frame tasks to reduce wait time.
· Increase overall throughput and reduce perceived data transfer latency.
Adaptive Quantization
· Select quantization precision based on activation distributions per layer.
· Keep INT8 for precision‑sensitive layers and INT4 for less sensitive ones.
· Improve inference speed and reduce compute while preserving accuracy.
SDK‑Level Hardware Acceleration
· Use low‑level C/C++ for higher interface efficiency;
· Direct native SDK access maximizes hardware acceleration vs. OpenCV and third‑party libraries.
Unified Abstraction Layer
· Abstract and unify vendor SDK interfaces to isolate chip differences from upper‑layer algorithms;
· Run the same algorithm across chips—write once, deploy on multiple platforms.
Operator‑Level Fusion
· Merge consecutive operators such as Conv, BN, and ReLU.
· Reduce repeated read/write of intermediate results and memory access overhead.
· Convert multiple operations into a single NPU instruction for higher inference efficiency.
Memory Reuse Strategy
· Manage cache space uniformly based on Tensor lifecycles.
· Released memory blocks can be reused by subsequent Tensors.
· Reduce repeated allocation and off‑chip memory usage.
Instruction Pipeline Parallelism
· Split CPU pre‑processing, NPU inference, and CPU post‑processing into pipeline stages.
· Overlap multi‑frame tasks to reduce wait time.
· Increase overall throughput and reduce perceived data transfer latency.
Adaptive Quantization
· Select quantization precision based on activation distributions per layer.
· Keep INT8 for precision‑sensitive layers and INT4 for less sensitive ones.
· Improve inference speed and reduce compute while preserving accuracy.
SDK‑Level Hardware Acceleration
· Use low‑level C/C++ for higher interface efficiency;
· Direct native SDK access maximizes hardware acceleration vs. OpenCV and third‑party libraries.
Unified Abstraction Layer
· Abstract and unify vendor SDK interfaces to isolate chip differences from upper‑layer algorithms;
· Run the same algorithm across chips—write once, deploy on multiple platforms.
Significant Gains in Efficiency and Results
Larger Models
Same Hardware, Larger Models
Lite series originally ran YOLOv8n; after optimization it runs YOLOv8s without FPS loss
More Streams
More Concurrent Streams
Handle more video channels at the same compute level, boosting edge concurrency.
Lower Power
Lower Power Consumption
Power scheduling tuned to model structure and chip characteristics reduces long‑run energy use.
Higher Performance
Higher Performance
Optimized memory, operators, and pipelines deliver higher inference efficiency per resource unit.
More Chips
More Chip Support
Unified low‑level adaptation covers more chip platforms and hardware form factors.
Test Environment
Hardware: Pro
Model: YOLOv8n
Resolution: 1080p
Camera /ISP/VI
Capture & signal processing
—
Pre-processing
Convert / resize / normalize
2.65 ms
AI Inference
NPU-accelerated compute
14.23 ms
Output Retrieval
Result copy / sync
2.44 ms
Post-processing
Decode / NMS / filter
1.15 ms
App / Display
Output & visualization
—
Core compute chain total≈ 20.46 ms
22-25 FPSEnd-to-end capability of the Pro series
Pipeline scheduling delivers higher throughput
Example: YOLOv5s / 224x224
Serial processing22FPS
XOS pipeline scheduling30FPS
+36%throughput gain
Better per-chip resource efficiency
Normalized equivalent throughput (same workload)
Baseline6.86
XOS8.42
+22.7%throughput gain
Note: 6.86 / 8.42 are normalized equivalent metrics and do not represent physical video channel counts.
Unified runtime
Build once, deploy across platforms
Efficient memory management
Buffer reuse, fewer data copies
Heterogeneous coordination
Parallel CPU/NPU/ISP scheduling
End-to-end optimization
Capture → process → infer → post-process in sync
Stable and reliable
Low latency, high stability
About these figures
Figures are measured in a typical test environment; real-world results vary with model, input size, and scenario.
Bar chart comparing Xeroptix vs stock SDK YOLOv8n FPS on Lite: stock SDK 10fps, Xeroptix 25fps
