When school WiFi died for three days, I fine-tuned Phi-3 mini on the curriculum and ran a fully offline reading tutor on a $280 mini-PC under the teacher's desk.
WiFi Goes Down. Learning Doesn't Have To.
After I built the AI tutor for my niece's algebra class last year — the one her teacher asked for the link to — I kept thinking about the schools that can't use cloud AI at all. Not because they don't want to, but because the infrastructure isn't there.
My daughter's school is one of them. Rural-ish, underfunded, and the WiFi goes down with some regularity. When it does, all the cloud-dependent ed-tech that the district paid for just… stops working. Three days last November, the reading program the second-graders were using was completely unavailable.
I decided to build something that would never need the cloud.
What I Picked and Why
Model: Microsoft Phi-3 mini (3.8B parameters, 4-bit quantized). This was an easy call — Phi-3 mini is specifically designed for edge deployment, has strong reasoning at its parameter class, and the fine-tuning documentation is excellent. At 1.4GB in its Q4_K_M quantized form, it fits comfortably on the mini-PC.
Hardware: A used Beelink Mini PC with an Intel N100 CPU and 16GB RAM — $280 on Amazon. No GPU. The N100 is not fast, but with llama.cpp's CPU optimization it runs the quantized Phi-3 at acceptable latency for a reading tutor (p95 480ms, which is fast enough that students don't notice a significant pause).
Fine-tuning: I used the teacher's existing curriculum materials — vocabulary lists, reading passages at each grade level (Lexile levels 200-800), and a set of Socratic-style question templates. I generated ~3,000 training examples: a reading passage + student question → tutor response that asks a guiding question rather than giving the answer directly.
|
# generate_training_data.py import openai, json
SOCRATIC_TEMPLATE = """Generate a tutoring exchange for a reading passage. Do NOT give the answer directly. Ask a guiding question. Student age: {age}, Reading level: Lexile {lexile} Passage: {passage} Student question: {question} Tutor response (Socratic, encouraging, 1-2 sentences):"""
# Fine-tune with Unsloth for efficiency # Then convert to GGUF format for llama.cpp deployment |
I fine-tuned on a free Google Colab T4 using Unsloth (which makes QLoRA fine-tuning about 2x faster than vanilla HuggingFace Transformers). Training took about 90 minutes for 3,000 examples. Then I converted the merged model to GGUF format for llama.cpp deployment on the mini-PC.
How It Works in the Classroom
The mini-PC sits under the teacher's desk, connected to the classroom LAN via ethernet. Students access the tutor through a simple web interface I built in Flask — any device on the classroom network can connect, including the school's aging Chromebooks.
The interface is deliberately simple: a reading passage is displayed on the left half of the screen, a chat window on the right. Students type questions or request help with vocabulary. The tutor responds in the voice of the fine-tuned model — Socratic, encouraging, grade-appropriate.
There's no user account system, no cloud calls, no data leaving the classroom. When the internet goes down: doesn't matter. When the server room has a problem: doesn't matter. When the district's SaaS contract expires: doesn't matter.
|
# deploy.sh — run on the mini-PC # Start llama.cpp server with quantized Phi-3 ./llama-server \ --model phi-3-mini-reading-tutor-Q4_K_M.gguf \ --host 0.0.0.0 \ --port 8080 \ --ctx-size 4096 \ --n-predict 256 \ --threads 4 \ --mlock # keep model in RAM
# Flask frontend on port 5000 python app.py --llm-url http://localhost:8080 |
What Broke
Two real problems during the first month of use:
Latency spikes during recess. When 6 students all hit the tutor at the same time (same passage, same break time), the CPU got saturated and latency jumped to 2-3 seconds. I added a simple request queue with a "your question is next" UI indicator. Students accepted this naturally — they're used to raising their hand and waiting.
The model hallucinated vocabulary definitions. On words not in the training data, Phi-3 mini occasionally made up plausible-sounding but wrong definitions. I added a vocabulary lookup layer: before sending any response containing a definition, the server checks a local dictionary (WordNet via NLTK) and substitutes the authoritative definition if the model's version differs by more than 20% word overlap.
What I Learned
Small, well-fine-tuned models are genuinely good for narrow educational tasks. Phi-3 mini on a $280 box produces responses that the teacher described as "better than most of the educational apps we've paid for." The narrow fine-tune (reading tutor, grades 2-4, Socratic style) is more important than raw model size.
Also: the lack of internet dependency is a feature, not a constraint. It means the teacher doesn't need to think about the tool's availability. It's just there, like the whiteboard.
If I Were Doing This Again
I'd look at Apple Silicon Mac Mini instead of the Beelink — the M-series GPU acceleration via Metal would give much better latency at similar price point, and Apple MLX makes deployment simpler. I'd also add a local curriculum-syncing mechanism so the teacher can update reading passages without me having to SSH in.
GitHub gist coming soon — fine-tuning script, llama.cpp deployment config, and the Flask frontend all included.
Figure 8. Cozy classroom isometric scene: mini-PC under teacher's desk connected to classroom LAN. Chromebooks on student desks showing the reading tutor UI (passage left, chat right). No internet connection…
REFERENCES
1. Microsoft Phi-3 Technical Report. Microsoft Research / arXiv (2024).
https://arxiv.org/abs/2404.14219
2. llama.cpp: Inference of LLaMA model in pure C/C++. GitHub / ggerganov (2024).
https://github.com/ggerganov/llama.cpp
3. Common Core State Standards for English Language Arts. Common Core State Standards Initiative (2010).
https://www.corestandards.org/ELA-Literacy/
4. Unsloth: Fast LLM Fine-Tuning Library. GitHub / unslothai (2024).
https://github.com/unslothai/unsloth
5. QLoRA: Efficient Finetuning of Quantized LLMs. arXiv (2023).
https://arxiv.org/abs/2305.14314
Comments (0)
Join the conversation!