Edge AI for industrial: Jetson, YOLO, and the real cost of leaving the cloud

tl;dr. Edge AI sounds like a way to save on cloud inference bills. In practice, it usually is not. The right reasons to go to the edge are latency, privacy, and bandwidth, in that order, and if your reason is not one of those three, you should probably stay in the cloud. The hardware has never been cheaper, a $249 Jetson developer kit now runs modern object detection at real-time framerates, but the hardware was never the expensive part. The operational tail is. Here is the honest cost picture, from evaluating this exact stack for AI-powered construction site monitoring.

Edge AI is not cheaper. It is different.

I want to lead with that, because the pitch that usually sells an edge deployment is a cost pitch: inference in the cloud costs money per frame, a box on site costs money once, therefore edge wins. That arithmetic is missing most of the terms, and the missing terms are where the projects go over budget. I have been working through this calculus while evaluating vision-based site monitoring, think PPE and safety compliance detection on construction sites, as a bundled offering for the connectivity rental business I run on the side, and the honest conclusion is that edge AI earns its place on three grounds only, none of which is the invoice.

The three legitimate reasons, ranked

Latency is the first and strongest. If the decision has to happen in tens of milliseconds, a robot arm stopping, a gate opening, an alert that must fire while the event is still happening, the cloud round trip is disqualified before the pricing conversation starts. Edge is not cheaper here, it is the only option.

Privacy and data sovereignty is the second. Video of a workplace is video of identifiable people, and in Europe that makes every frame you ship to a cloud a GDPR question. An architecture where raw video never leaves the site, where only events and metadata go up, "person without helmet detected, zone 3, timestamp", is dramatically easier to defend, both legally and in a works council meeting. For workplace monitoring in particular, this is often the decisive argument, and it was the one that mattered most in my own evaluation. What leaves the site is a fact, not a face.

Bandwidth is the third. A construction site on a 4G or satellite uplink cannot stream multiple continuous HD feeds to a cloud, and even where it physically can, the data plan says otherwise. Analyzing at the edge and uplinking kilobytes of events instead of gigabytes of video is what makes video analytics feasible at all on constrained links. If your sites have fiber, this argument weakens accordingly.

If none of the three applies to your use case, stop. Run it in the cloud, keep your fleet size at zero, and revisit when one of them starts to apply.

Hardware reality in 2026

The hardware story is genuinely good news, and it is the part of edge AI that has improved the most.

The reference point is NVIDIA's Jetson Orin Nano Super developer kit: $249, and with the current software stack it delivers 67 TOPS, which in practice means modern YOLO-class object detection at real-time framerates, comfortably 30 frames per second and often more depending on model size and precision. Notably, the "Super" uplift came as a free software update, the same silicon jumped from 40 to 67 TOPS with a JetPack flash, which tells you something about how much headroom lives in the software layer of these devices. Real-world power draw under inference sits around 10 to 15 watts, which matters for solar-and-battery-powered site installations.

One step up sits the Orin NX, at roughly 100 to 157 TOPS depending on the memory variant, with more CPU and more bandwidth. The honest sizing advice: the Nano class handles a single-model, few-camera deployment comfortably, and the NX is the correct step when you need many simultaneous streams or several models running at once. For fixed, single-purpose inference at very low power, dedicated accelerators like Hailo or Coral are worth a look, at the cost of a much narrower software ecosystem.

Two practical notes that do not appear in datasheets. First, memory, not TOPS, is usually the binding constraint on the smaller boards: 8 GB shared between the system and the models fills faster than you expect once you add a video pipeline. Second, the developer kit is a prototyping tool, not a product; production deployments are built on the bare module plus an industrial carrier board, and that carrier-board-and-enclosure work is a real line item people forget to budget.

Model reality

On the model side, the YOLO family remains the default for real-time detection, and the current generation is genuinely strong. The practical trade-off has not changed in a decade: smaller variants buy framerate at the cost of accuracy on small or distant objects, larger variants the reverse, and quantization down to INT8 via TensorRT is where much of the deployable performance comes from.

Two lessons from working through this for site monitoring. The generic pretrained classes get you a demo, not a product; the value is in fine-tuning on your actual domain, your camera angles, your lighting, your definition of "helmet", and that means owning a dataset and a retraining loop, which is an ongoing cost, not a one-time one. And the newest model release is not automatically the right choice for edge: the freshest architectures sometimes land with immature export paths and quirks on embedded targets, and the boring previous generation with a battle-tested TensorRT path often ships a better product. Let someone else find the sharp edges.

The quiet operational tax

Now the part the demo videos skip, which is everything that happens after the demo works.

A cloud model is one deployment you control centrally. An edge fleet is dozens or hundreds of Linux computers in the field, and they need what every fleet needs: an update mechanism for both the OS and the models, monitoring so you know a box is down before the customer does, remote access that works from behind whatever network the site has, and a plan for the unit that stops responding entirely. None of this is exotic, all of it is work, and it scales with fleet size forever. The rough rule I would give anyone budgeting: the model is a fifth of the engineering, the pipeline around it is another fifth, and the fleet operations layer is the remaining three fifths, amortized over the life of the deployment.

Physical environment is its own tax category. Outdoor industrial sites mean dust, vibration, temperature swings, and enclosures that must shed heat in summer without letting water in ever. A 15-watt computer in a sealed IP-rated box in direct sun is a small oven, and thermal throttling is a silent performance bug that only shows up in August. Power is the other physical constraint: many of the most valuable sites are exactly the ones without reliable mains, which pushes you into solar-battery sizing, and suddenly your 10-watt inference budget is a design driver rather than a footnote.

And then there is the unglamorous truth that the camera is part of the model. Mounting height, angle, lens, and cleaning schedule move your detection accuracy more than a model version bump does. A fleet of edge AI boxes is also a fleet of physical cameras that get dirty, get bumped, and get repositioned by well-meaning site staff.

The architecture that holds up

The pattern that survives contact with all of the above is a pragmatic hybrid, and it is the one I would build.

Inference runs at the edge, for the latency, privacy, and bandwidth reasons that justified the edge in the first place. Events, metadata, and carefully sampled frames flow up to the cloud, where they accumulate into the dataset that matters. Retraining happens in the cloud, where the GPUs and the MLOps tooling live, on that accumulating dataset. And improved models flow back down through the fleet update mechanism on a regular cadence. Edge inference, cloud learning, periodic model push. Each side does what it is structurally good at, and the feedback loop between them is the actual product asset, because six months in, your domain dataset is worth more than your model architecture.

The summary I would put on one slide: go to the edge for latency, privacy, or bandwidth, and for no other reason. Budget the hardware honestly, it is the cheap part. Budget the operational tail honestly, it is the expensive part. And design the cloud loop from day one, because an edge fleet without a learning loop is a depreciating asset, and one with it compounds.

If you are weighing an edge AI deployment, industrial vision, site monitoring, or anything in that family, and want help pressure-testing the architecture and the real cost model before the hardware order goes in, that is exactly the kind of engagement I take on. Details on the consulting page.