Context
I am building WearEdge Pro, a wearable industrial edge Agent runtime. The system takes first-person operator evidence from a wearable device, sends it to a local Jetson inference node, and returns structured action guidance for maintenance, quality inspection, changeover, work instruction, and hazard review.
The important design constraint is that the model is not allowed to be an unbounded chatbot. The model must fit into a workflow that includes tool calls, deterministic guards, action cards, follow-up plans, and audit logs.
To validate the current model baseline, I ran a horizontal comparison of five compact multimodal candidates on the same Jetson endpoint path:
- Gemma 4 E2B
- Qwen2.5-VL-3B-Instruct
- SmolVLM2-2.2B-Instruct
- InternVL3-2B-Instruct
- Qwen2.5-Omni-3B
What We Measured
Every candidate was run through the same five industrial agent prompts:
- predictive maintenance
- online quality inspection
- changeover guidance
- work-instruction assistance
- hazard exposure review
The primary test used 560/560 image tokens. Qwen2.5-VL also got a 1024/1024 pass because visual grounding sometimes improves with a higher image-token budget.
| Model | Result | Average latency | Product read |
|---|---|---|---|
| Gemma 4 E2B | 5/5 | 37.51s raw, 46.02s excluding hot-cache IQC | Baseline |
| Qwen2.5-VL-3B | 5/5 at 560 and 1024 | 39.72s / 63.48s | Best OCR challenger |
| SmolVLM2-2.2B | 5/5 | 12.84s | Fast but too generic |
| InternVL3-2B | 2/5 at ctx2048, 5/5 at ctx4096 | 80.35s at ctx4096 | Not current baseline material |
| Qwen2.5-Omni-3B | 5/5 | 50.09s | Future omni-modal track |
The Most Useful Finding
Qwen2.5-VL was clearly strong for OCR-heavy scenes. In a changeover image, it read the machine and SKU as LABELER-FL1 and SKU-C500, while Gemma made a small OCR error. If the product goal were only “read the visual label,” Qwen would have the edge on that sample.
But WearEdge Pro is not only an OCR system. It is a bounded industrial workflow. The model must be judged by how well it behaves inside that workflow.
Why Gemma 4 E2B Stayed the Baseline
Gemma 4 E2B stayed the baseline because it matched the system architecture better:
- it runs as the restored local Jetson baseline service
- it completed all five agent contracts
- it gave stronger human-confirmation behavior in maintenance
- it fits the current action-card and audit flow
- it aligns with the long-context, function-calling-oriented runtime design
The model does not have to be the best OCR model on every image to be the right current baseline. In industrial edge AI, the “winner” is the model that best survives the full workflow: evidence, reasoning, tool/action boundary, guard, and audit.
Where I Would Take This Next
My next technical branch would not be “replace Gemma.” It would be:
- Keep Gemma 4 E2B as the workflow baseline.
- Add Qwen2.5-VL as an A/B branch for IQC and changeover OCR.
- Put Qwen outputs behind the same WearEdge guards.
- Re-run the same five-agent matrix through the full product gateway rather than the raw endpoint.
- Explore Qwen2.5-Omni only for a future native speech/video branch.
This made the model decision more honest. Gemma is the baseline because it is the strongest current product fit, not because every single cell in a benchmark table belongs to it.
Full benchmark summary is included in this post. A public reproducibility artifact link can be added after the repository/artifact package is made public.