On-Device Small Language Model Chatbot
Fully offline SLM chatbot running on-device with no cloud dependency — quantized inference tuned for constrained memory and latency budgets.
Pipeline
Architecture
End-to-end flow, stage by stage.
- 01
Model distillation
- 02
TFLite conversion
- 03
Quantization
- 04
Pruning
- 05
Local state
How it works
Converted and optimized a compact language model with TensorFlow Lite for private, offline inference across desktop and mobile targets.
Combined quantization, pruning, and knowledge distillation to reduce the runtime footprint while preserving intent recognition and contextual dialogue quality.
Kept multi-turn state locally, eliminating network latency and ensuring that conversation data never had to leave the device.
- 0
- Cloud calls — fully offline
- Quantized
- Runs on constrained memory
Stack
More projects
RAG / NLP
ChronosGuru
Persona-driven historical RAG platform hitting 92% factual accuracy, holding persona fidelity across 20+ turns, with hybrid keyword + semantic search cutting retrieval latency under 200ms.
RAG / NLP
FinSight
Real-time forex intelligence platform fusing live rate APIs with an NLP news aggregation pipeline, stateless zero-PII architecture, and programmatic RBI FEMA rate disclaimers.