Computer vision on the edge means running image and video models on the camera, a gateway or a local server, so that decisions are made where the pixels are captured. For enterprise teams the reason is practical: a defect, a safety breach or a blocked lane has to be acted on within seconds, and sending every frame to a cloud service adds delay, bandwidth cost and a privacy question.
This guide is for CTOs, CIOs, heads of engineering and operations leads who must decide what runs on the device, what stays central and how to govern it. It focuses on the model, runtime and hardware choices; the system view is in our guide to edge AI video analytics architecture, and sector examples are in vision analytics use cases, challenges and outlook.
There is no single right place. Our team's working rule is to decide model by model, not site by site, using the questions below.
| Decision factor | Run on the edge when | Keep in the cloud or data centre when |
|---|---|---|
| Response time | The action must follow the frame within the line speed or safety window | Results can wait minutes or hours |
| Connectivity | Links are weak, shared or metered, or sites must keep working offline | Sites have reliable, cheap bandwidth |
| Data sensitivity | Raw footage should stay on site and only events leave | Footage is not personal data and can be centralised |
| Model size and update rate | The model fits the device after quantisation and changes monthly, not daily | The model is large or changes often |
| Workload | Inference on live streams | Training, fleet-wide analytics, re-processing archives |
Two changes matter to enterprise buyers. First, the runtimes have consolidated. Google describes LiteRT as the framework formerly known as TensorFlow Lite, for running ML and generative AI models on-device on Android, iOS, web, Linux, macOS and Windows, with CPU, GPU and NPU acceleration and a post-training quantiser. PyTorch's ExecuTorch targets phones, embedded systems and microcontrollers, with backends that include XNNPACK, Core ML, Qualcomm, MediaTek, Arm Ethos-U, OpenVINO and NXP. Old TensorFlow Lite code should be reviewed against the new name and tooling.
Second, hardware choice is wider. NVIDIA's Jetson page lists the Orin Nano series at up to 67 TOPS and 7 to 15 W, Orin NX at up to 157 TOPS and 10 to 25 W, AGX Orin at up to 275 TOPS and 15 to 60 W, and AGX Thor at up to 2070 TFLOPS of sparse FP4 performance and 40 to 130 W. The Thor figure uses a different measure, so do not compare it directly with the TOPS figures.
| Tier | Typical device | What it suits | Check before you buy |
|---|---|---|---|
| Microcontroller or sensor | Low-power MCU with a small NPU | Wake-word style triggers, simple classification, presence | Model size, memory, whether ExecuTorch or LiteRT supports the chip |
| Camera or smartphone class | Camera SoC, phone NPU | One or two models per stream, single-camera inspection | Vendor NPU support in the runtime; quantisation accuracy loss on your images |
| Embedded GPU module | Jetson Orin Nano to AGX Orin class | Several streams, multi-model pipelines | Power and heat budget; supply and support period |
| Edge server or gateway | On-site server with a GPU | Many cameras per site, larger models | Maintenance, remote updates, spare parts |
Whatever you pick, test on the target device with your own images. A model that scores well on a benchmark can lose accuracy after quantisation, and the only reliable check is a held-out set from your own plant, store or site.
The strongest cases share one trait: the action is local and time-bound.
Cameras on the line can flag defects, missing parts or label errors before a product is packed, so the reject decision happens at line speed. This is usually the best first project because the success measure (escapes and false rejects) is easy to count.
Video: AI packaging inspection checks every seal, cap and label (XenonStack AI).
Vision systems can count stock and read pallet or label data on the move. Gartner predicted in June 2024 that by 2027, 50% of companies with warehouse operations will use AI-enabled vision systems, mostly replacing scan-based cycle counting.
Imaging models can run on the device that produces the image, which keeps patient data local. Any clinical use needs the relevant medical-device approval and clinician oversight, so treat these as regulated products, not pilots.
Drones and field cameras work where connectivity is poor, so crop, pest or asset checks need models that run onboard and send summaries when a link is available.
Local analytics can raise alerts for safety breaches, crowding or intrusion. Biometric use needs particular care; see the rules below and our post on vision analytics use cases.
Privacy is better when footage stays local, but it is not automatic: you still need a lawful basis, retention rules and access control for whatever the device stores or sends. Under the EU AI Act, the Commission page shows prohibited practices applying from 2 February 2025, Annex III high-risk use cases from 2 December 2027 and Annex I systems in regulated products from 2 August 2028, following the AI Omnibus, which entered into force on 27 July 2026. Check your own use case against the Act with legal counsel.
Security needs the same discipline as any fleet of remote computers: signed model updates, device identity, patching and a plan for lost or tampered devices. For cost, count the whole life: hardware refresh, remote management, labelling and retraining, not only the module price. Governance of the actions a vision alert triggers can be handled with an agent control layer such as AgentGRC, which keeps autonomous agent actions inside policy with an audit trail, and device fleets can be run with NexaStack, an agentic operating platform for physical AI that runs across on-premises, cloud and hybrid environments.
Agents that coordinate several vision tasks and automations can be built and governed on ElixirClaw, an agentic OS platform to build, orchestrate and govern AI agents and automation workflows while keeping humans in control. XenonStack also describes its own approach on the Edge AI Vision page.
Avoid the edge when the model changes daily, when a single central site can see all the cameras with cheap bandwidth, when nobody can maintain devices in the field, or when the task needs a model far larger than the device can hold. In those cases a regional data centre or cloud service is simpler and cheaper to run. A hybrid design, with a small model on the device and a larger model for second opinions, is often the sensible middle.
These are our best-supported expectations for 2026 to 2031. Each rests on a dated source or is labelled as XenonStack's view.
| Prediction | Why we expect it (evidence) | Timeframe | What leaders should do now |
|---|---|---|---|
| Edge budgets keep growing, and vision is a main workload | IDC (press release, 17 March 2025) estimated edge spending of nearly $261 billion in 2025 and about $380 billion by 2028, a 13.8% CAGR, across all edge use cases, not only vision. IDC names manufacturing and retail among the largest sectors. | To 2028 | Budget for fleet management, updates and retraining, not only devices |
| EU high-risk duties start to bite | Commission page: Annex III high-risk applies from 2 December 2027; Annex I product systems from 2 August 2028 (AI Omnibus in force 27 July 2026). | From Dec 2027 | Inventory vision systems that use biometrics or sit in regulated products |
| Warehouse and logistics vision becomes mainstream | Gartner (12 June 2024): by 2027, 50% of companies with warehouse operations will leverage AI-enabled vision systems. The press release cites a December 2023 survey in which 20% had already adopted them. | By 2027 | Run a counting or damage-detection pilot on one site |
| NPU acceleration becomes a default runtime feature | LiteRT lists NPU acceleration across Qualcomm, MediaTek, Google Tensor, Intel and Samsung; ExecuTorch lists backends for Qualcomm, MediaTek, Arm, NXP and Samsung. XenonStack's view: this makes models easier to move between devices. | 2027 to 2029 | Avoid locking models to one chip; keep export paths open |
| Vision language models verify alerts on local hardware | NVIDIA's VSS 3.3.0 documentation describes using vision language models to verify alert authenticity, with DGX Spark, IGX Thor and AGX Thor named as edge hardware. XenonStack's view: timing for smaller sites depends on device cost. | 2027 to 2029 | Test alert verification to cut false alarms before widening coverage |
| Alerts turn into governed agent actions | XenonStack's view, based on our team's work on agentic video intelligence: detection is becoming commodity, so value shifts to what happens next, with policy and an audit trail. | 2027 to 2030 | Map each alert to an owner, an action and an approval rule |
What would prove us wrong: a sharp fall in edge hardware supply, a regulatory pause on biometric or high-risk vision uses, or large models becoming so cheap to call over the network that local inference stops paying for itself. We will revise this table as those dates and sources change.
What this means for you: decide now which vision decisions must stay local, and build the update, governance and audit routines before you add cameras. To map one vision decision your agents should make, and the context and policy it depends on, book a Context Workshop. See what happens in the workshop.
It is running image and video models on the camera, a gateway or a local server, close to where the pixels are captured, instead of sending every frame to a cloud service. Only events, metadata or short clips then travel upstream.
Choose the edge when decisions must happen within the line speed or safety window, when links are weak or costly, or when raw footage should stay on site. Keep training, fleet analytics and heavy re-processing in the cloud or a regional data centre.
On the software side, Google's LiteRT (formerly TensorFlow Lite) and PyTorch's ExecuTorch both target CPU, GPU and NPU. On the hardware side, NVIDIA lists Jetson modules from Orin Nano (up to 67 TOPS, 7 to 15 W) to AGX Thor. Pick by model size, power budget and the vendor NPU you already use.
It can. Prohibited practices have applied since 2 February 2025. High-risk obligations for Annex III use cases apply from 2 December 2027 and for systems in regulated products (Annex I) from 2 August 2028, after the AI Omnibus entered into force on 27 July 2026. Running a model on the edge does not remove these duties.
Expect edge spending to keep growing (IDC projects about $380 billion by 2028 across all edge use), NPU support to become standard in runtimes, vision language models to verify alerts on local hardware, and EU high-risk duties to start applying from December 2027. Vision outputs are also likely to feed governed agent actions. Dates and sources are in the outlook table above.
Pick one measurable use case, measure it on the target device with your own images, set a retraining and rollback routine, and write down who owns each decision the model triggers. Scale to more sites only after that loop works.