EchoLens — Real-time in-browser audio classifier
CNN that classifies urban sounds instantly on your own device: trained in PyTorch, quantized to INT8, and running 100% on WebAssembly in the browser — no server, no audio ever leaving the device.
EchoLens is a sound classifier that runs entirely in the browser with no server: the user opens the page, makes a sound, and the system classifies it instantly on their own device — the audio never leaves it. Technically it combines training a CNN in PyTorch on UrbanSound8K, exporting it to ONNX and quantizing it to INT8 to reduce model size ~4× with less than 2 accuracy points lost, and running it in real time with onnxruntime-web on WebAssembly. Microphone capture and log-mel spectrogram extraction run live in the browser via the Web Audio API and an AudioWorklet. The project demonstrates a set of skills that rarely appear together: digital audio signal processing (STFT, mel filters), deep learning model training and optimization, edge ML deployment (quantization and model conversion), and browser ML engineering (on-device inference on WASM) — all with professional engineering practices: CI/CD, tests, and quantitative evaluation of latency and accuracy.
The problem
ML audio models typically require sending audio to a server for classification, introducing network latency, connectivity dependency, and above all privacy concerns: the user's audio leaves their device. Modern browsers offer the primitives to avoid this, but combining the full pipeline — microphone capture, feature extraction, and neural network inference — efficiently on WebAssembly is non-trivial.
The solution
I trained a CNN on UrbanSound8K (10 urban sound classes) in PyTorch with 10-fold cross-validation. I exported the model to ONNX and quantized it to INT8 — approximately 4× size reduction with less than 2 accuracy points lost — for efficient edge inference. Deployment is 100% client-side with onnxruntime-web on WebAssembly: microphone capture and log-mel spectrogram extraction run live via the Web Audio API and an AudioWorklet, with guaranteed numerical parity between the Python and JavaScript preprocessing pipelines. The project includes a comparative study of inference backends (WASM vs WebGPU) with documented latency metrics.