Edge AI Computing: The Shift from Cloud to Device

By Tanqory Engineering Team

The AI revolution is moving from the cloud to the edge. In 2025, we're witnessing a fundamental transformation in how artificial intelligence is deployed—shifting processing power from centralized data centers to billions of edge devices worldwide. This isn't just an architectural change; it's a complete reimagining of how commerce applications deliver intelligent experiences.

At Tanqory, Edge AI now powers over 50 million daily inferences across our global platform, enabling real-time personalization, instant fraud detection, and smart inventory management with latencies under 10 milliseconds. This article details how we built this infrastructure and why edge computing represents the future of commerce technology.

1. Why Edge AI Matters Now

The Latency Problem

Cloud-based AI introduces unavoidable delays:

Round Trip	Typical Latency	Impact on UX
User → Cloud → User	100-500ms	Noticeable delay
User → Regional Server → User	50-150ms	Slight hesitation
User → Edge Device	1-10ms	Instantaneous

For commerce applications where milliseconds impact conversion rates, these differences are significant. Research shows that every 100ms of latency reduces conversion by 1%. Edge AI eliminates this bottleneck entirely.

The Privacy Imperative

Data privacy regulations are tightening globally:

• GDPR in Europe requires data minimization
• CCPA gives California consumers data control rights
• LGPD in Brazil mandates explicit consent
• PIPL in China restricts cross-border data transfers

Edge AI addresses these challenges by processing sensitive data locally—personal information never leaves the user's device. This isn't just compliance; it's a competitive advantage in privacy-conscious markets.

The Cost Equation

Cloud inference costs scale linearly with usage:

Traditional Cloud AI (per 1M inferences):

• Compute: $15-50
• Network egress: $5-15
• Storage: $2-5
• Total: $22-70

Edge AI (per 1M inferences):

• Initial model deployment: One-time cost
• Inference: $0 (on-device)
• Updates: $0.50-2
• Total: $0.50-2

At scale, edge AI delivers 90%+ cost reduction for inference workloads.

2. Tanqory's Edge AI Architecture

System Overview

Our edge computing infrastructure spans three tiers:

┌───────────────────────────────────────────────────────────────────┐
│                          Cloud Layer                              │
│  ┌─────────────────────────────────────────────────────────────┐  │
│  │    Model training and optimization                          │  │
│  │    Global model registry and versioning                     │  │
│  │    Aggregate analytics and insights                         │  │
│  │    A/B testing coordination                                 │  │
│  └─────────────────────────────────────────────────────────────┘  │
└───────────────────────────────────────────────────────────────────┘
                              │
                    Model Distribution
                              │
                              ▼
┌───────────────────────────────────────────────────────────────────┐
│                          Edge Layer                               │
│  ┌──────────────┐  ┌──────────────┐  ┌──────────────┐             │
│  │ Edge Server  │  │ Edge Server  │  │ Edge Server  │             │
│  │ (Region A)   │  │ (Region B)   │  │ (Region C)   │             │
│  │              │  │              │  │              │             │
│  │   Model cache│  │   Model cache│  │   Model cache│             │
│  │   Aggregation│  │   Aggregation│  │   Aggregation│             │
│  │   Fallback   │  │   Fallback   │  │   Fallback   │             │
│  └──────────────┘  └──────────────┘  └──────────────┘             │
└───────────────────────────────────────────────────────────────────┘
                              │
                      Model Sync
                              │
                              ▼
┌───────────────────────────────────────────────────────────────────┐
│                         Device Layer                              │
│  ┌─────────┐ ┌─────────┐ ┌─────────┐ ┌─────────┐ ┌─────────┐      │
│  │ Mobile  │ │ Browser │ │ POS     │ │ IoT     │ │ Kiosk   │      │
│  │ App     │ │ (WASM)  │ │ Terminal│ │ Sensor  │ │ Display │      │
│  └─────────┘ └─────────┘ └─────────┘ └─────────┘ └─────────┘      │
│                                                                   │
│  Local inference Privacy preserved Offline capable            │
└───────────────────────────────────────────────────────────────────┘

Model Optimization Pipeline

Deploying AI models to edge devices requires aggressive optimization:

Step 1: Architecture Selection

• Start with efficient architectures (MobileNet, EfficientNet, DistilBERT)
• Custom architectures for specific commerce tasks
• Neural architecture search for optimal efficiency

Step 2: Quantization

• Float32 → Int8 quantization (4x size reduction)
• Mixed precision for accuracy-critical layers
• Post-training quantization for rapid deployment
• Quantization-aware training for best results

Step 3: Pruning

• Remove 60-80% of weights with minimal accuracy loss
• Structured pruning for hardware efficiency
• Iterative pruning with fine-tuning

Step 4: Compilation

• Target-specific optimization (ARM, x86, WebAssembly)
• Operator fusion for reduced memory access
• Memory layout optimization for cache efficiency

Results:

Model	Original Size	Optimized Size	Latency (Mobile)	Accuracy
Product Recommender	450MB	12MB	8ms	97.2%
Fraud Detector	280MB	8MB	3ms	99.1%
Search Ranker	1.2GB	45MB	15ms	96.8%
Image Classifier	180MB	6MB	12ms	94.5%

3. Real-World Applications

3.1 On-Device Personalization

The Challenge: Deliver personalized product recommendations without sending user behavior data to servers.

Our Solution:

On-device recommendation models that:

• Process browsing history locally
• Generate embeddings for user preferences
• Match against cached product embeddings
• Update in real-time as user browses

Technical Implementation:

• Model size: 12MB (fits in mobile app)
• Inference time: 8ms on average smartphone
• Update frequency: Daily model sync (2MB delta)
• Offline support: Full functionality without network

Results:

• Recommendation latency: 8ms (vs. 180ms cloud)
• Personalization accuracy: 97.2% (vs. 98.1% cloud)
• User engagement: +23% click-through rate
• Privacy: Zero user data transmitted

3.2 Real-Time Fraud Prevention

The Challenge: Detect fraudulent transactions instantly at point-of-sale without network dependency.

Our Solution:

Edge fraud detection that analyzes:

• Transaction patterns and anomalies
• Device fingerprinting signals
• Behavioral biometrics (typing patterns, touch dynamics)
• Location and time context

Architecture:

Transaction Input
       │
       ▼
┌──────────────────┐
│ Feature Extraction│ → 150+ signals in <1ms
└────────┬─────────┘
         │
         ▼
┌──────────────────┐
│ On-Device Model  │ → Risk score in 3ms
└────────┬─────────┘
         │
    ┌────┴────┐
    │         │
    ▼         ▼
 Low Risk   High Risk
 (Approve)  (Secondary Check)
              │
              ▼
       ┌──────────────┐
       │ Edge Server  │ → Deep analysis in 50ms
       │ Verification │
       └──────────────┘

Results:

• Decision latency: 3ms for 94% of transactions
• Fraud detection rate: 96.2%
• False positive rate: 0.08%
• Offline operation: Full protection without connectivity

3.3 Smart Inventory Management

The Challenge: Track inventory across thousands of retail locations in real-time with minimal infrastructure.

Our Solution:

Edge-powered inventory intelligence:

• Computer vision on store cameras
• RFID/barcode processing on mobile devices
• Predictive stock-out alerts
• Automated reorder triggers

Deployment:

• Camera-based detection on Raspberry Pi 4 ($75/unit)
• Mobile app for manual scanning with AI assist
• Central aggregation through edge servers
• Cloud sync for analytics and planning

Results:

• Stock accuracy: 99.4% (vs. 91% manual)
• Stockout reduction: 67%
• Labor cost savings: 40%
• Hardware cost: $75-200 per location

4. Technical Deep Dive: Model Serving at the Edge

Runtime Selection

Different edge environments require different inference runtimes:

Platform	Runtime	Model Format	Optimization
iOS	Core ML	.mlmodel	Neural Engine
Android	TensorFlow Lite	.tflite	GPU/NNAPI
Browser	ONNX.js / TF.js	.onnx / .json	WebGL/WASM
Linux Edge	TensorRT	.engine	CUDA
Embedded	TFLite Micro	.tflite	Microcontroller

Cross-Platform Model Distribution

Our model serving infrastructure ensures consistent behavior across platforms:

Model Registry:

• Versioned model storage with metadata
• Platform-specific compiled variants
• A/B testing configuration
• Rollback capabilities

Distribution Protocol:

• Delta updates (only changed weights)
• Compression: 85% bandwidth reduction
• Background downloads with retry
• Integrity verification (SHA-256)

Update Strategy:

• Gradual rollout (1% → 10% → 50% → 100%)
• Automatic rollback on accuracy degradation
• Device-specific targeting (by capability)
• Time-windowed updates (off-peak hours)

Handling Model Updates

Edge models require careful update management:

Challenges:

• Devices may be offline for extended periods
• Storage constraints limit model versions
• Updates must not interrupt user experience
• Rollback must be instant if issues detected

Our Approach:

1. Shadow deployment: New model runs alongside current
2. A/B comparison: Compare predictions silently
3. Gradual activation: Switch traffic incrementally
4. Health monitoring: Track accuracy, latency, crashes
5. Instant rollback: Previous model always cached

5. Performance Benchmarks

Latency Comparison

Real-world measurements across deployment scenarios:

Use Case	Cloud (P50)	Cloud (P99)	Edge (P50)	Edge (P99)
Product Recommendation	145ms	380ms	8ms	15ms
Search Autocomplete	85ms	220ms	4ms	8ms
Fraud Scoring	120ms	350ms	3ms	6ms
Image Search	280ms	650ms	45ms	80ms
Price Optimization	95ms	240ms	12ms	22ms

Accuracy Retention

Model accuracy after optimization pipeline:

Model Type	Cloud Accuracy	Edge Accuracy	Delta
Recommendation	98.1%	97.2%	-0.9%
Fraud Detection	99.4%	99.1%	-0.3%
Search Ranking	97.5%	96.8%	-0.7%
Classification	96.2%	94.5%	-1.7%

The small accuracy trade-off is vastly outweighed by latency and privacy benefits.

Cost Analysis

Monthly infrastructure costs for 100M inferences:

Component	Cloud-Only	Hybrid (Cloud+Edge)	Edge-First
Cloud Compute	$45,000	$8,000	$2,000
Network	$12,000	$3,000	$500
Edge Infrastructure	$0	$5,000	$8,000
Model Distribution	$0	$1,000	$2,000
Total	$57,000	$17,000	$12,500
Savings	Baseline	70%	78%

6. Building for Offline-First

Why Offline Matters

In global commerce, connectivity cannot be assumed:

• Emerging markets with unreliable networks
• Underground retail locations (malls, subways)
• Rural and remote areas
• High-traffic events causing network congestion

Our Offline Architecture

Data Synchronization:

• Local-first database (SQLite/IndexedDB)
• Conflict resolution for concurrent edits
• Background sync when connectivity returns
• Priority queue for critical operations

Model Inference:

• All models work without network
• Cached product data for recommendations
• Local transaction queue for payments
• Graceful degradation for missing data

User Experience:

• No loading spinners for AI features
• Instant responses regardless of connectivity
• Clear indicators when sync is pending
• Automatic retry with exponential backoff

7. Security Considerations

Model Protection

Edge deployment exposes models to potential extraction:

Threats:

• Model weight extraction
• Reverse engineering of business logic
• Adversarial attacks on local models
• Tampering with model files

Mitigations:

• Model encryption at rest
• Obfuscation of model architecture
• Integrity verification on load
• Secure enclave execution where available
• Regular model rotation

Data Security

Protecting user data on edge devices:

Approach:

• Minimal data retention (process and discard)
• Encryption for any persisted data
• No personally identifiable information in logs
• Secure deletion on app uninstall

8. Future Directions

Emerging Hardware

New silicon is purpose-built for edge AI:

Apple Neural Engine:

• 15.8 TOPS on iPhone 15 Pro
• Dedicated transformer acceleration
• On-device large language model support

Qualcomm Hexagon:

• 75 TOPS on Snapdragon 8 Gen 3
• INT4 support for extreme efficiency
• Always-on AI processing

Google Tensor:

• Custom TPU for Pixel devices
• Optimized for Google's AI models
• Edge TPU for IoT applications

On-Device Large Language Models

The next frontier: running LLMs locally:

Current Capabilities (2025):

• 3B parameter models on flagship phones
• 7B models on high-end tablets
• Specialized commerce LLMs under 1B parameters

Use Cases:

• Natural language product search
• Conversational shopping assistants
• Automated customer support
• Content generation for merchants

Tanqory's Approach:

• Custom small language models for commerce
• Hybrid: Simple queries local, complex to cloud
• Progressive enhancement based on device capability

Federated Learning Integration

Combining edge inference with federated learning:

Vision:

• Models improve from on-device usage
• No raw data leaves devices
• Collective intelligence without privacy compromise
• Personalization that respects user control

9. Implementation Guide

Getting Started with Edge AI

Step 1: Identify Candidates

• Latency-sensitive features
• Privacy-sensitive data processing
• High-volume, low-complexity inference
• Offline-required functionality

Step 2: Measure Baselines

• Current cloud latency (P50, P95, P99)
• Model accuracy metrics
• Infrastructure costs
• User experience impact

Step 3: Optimize Models

• Start with quantization (easiest win)
• Apply pruning if needed
• Consider architecture changes for major gains
• Validate accuracy at each step

Step 4: Deploy Incrementally

• Shadow mode first (run both, compare)
• A/B test with small user segment
• Monitor closely during rollout
• Have rollback plan ready

Step 5: Iterate

• Collect edge performance metrics
• Identify accuracy gaps
• Update models regularly
• Expand to new use cases

Key Takeaways

Aspect	Cloud AI	Edge AI
Latency	100-500ms	1-10ms
Privacy	Data transmitted	Data stays local
Cost at Scale	Linear growth	Near-zero marginal
Offline Support	None	Full capability
Model Size	Unlimited	Constrained (MB)
Accuracy	Maximum	Slightly reduced

Conclusion

Edge AI represents a fundamental shift in how we build intelligent commerce applications. By moving AI inference to the device, we achieve latencies that feel instantaneous, privacy guarantees that build trust, and economics that scale sustainably.

At Tanqory, our edge computing infrastructure now handles over 50 million AI inferences daily across recommendations, fraud detection, search, and inventory management. The results speak for themselves: 10x latency improvement, 78% cost reduction, and privacy protection that meets the strictest global standards.

The future of commerce AI is not in the cloud—it's in your pocket, your store, and every device that connects buyers and sellers. And that future is already here.

---

This article is part of our Engineering Series exploring cutting-edge AI implementation at Tanqory. For questions or collaboration opportunities, reach out to info@tanq.com.sg

Published by Tanqory Engineering Team | December 2025