2025 · MACHINE LEARNING · SPEECH PROCESSING

FlowCheck

An end-to-end speech fluency analysis application that combines automatic speech recognition, acoustic signal processing and machine learning to detect stuttering patterns and provide personalised feedback.

1,383labelled audio clips

4acoustic feature types

3audio formats supported

83%accuracy · 0.90 F1

BUILT WITH

PythonFastAPIStreamlitWhisper AIWebRTC VADLibrosaRandom ForestNLTK WordNet

THE PROBLEM

Existing speech-to-text tools transcribe what someone says and analyse audio, but don't necessarily capture how fluently they say it or suggest alternative words where stuttering occurs. I wanted to explore whether machine learning could identify stuttering-related speech patterns and provide constructive, accessible feedback for people who stammer.

THE HARD PARTS

Built a multi-stage audio-processing pipeline combining speech recognition, voice activity detection and acoustic feature extraction. Engineered MFCC, zero-crossing rate, RMS energy and spectral-centroid features to train a speech-disfluency classifier, while handling the challenge of distinguishing natural pauses from potentially disfluent speech.

WHAT I BUILT

Developed a full-stack speech analysis application using FastAPI and Streamlit, integrating Whisper AI for word-level transcription, WebRTC VAD for speech segmentation and a Random Forest classifier trained on 1,383 SEP-28k audio clips. The application generates fluency scores, disfluency predictions, annotated transcripts and WordNet-based synonym suggestions, with inference running locally.

UNDER THE HOOD

Machine learning

Trained a Random Forest classifier on 1,383 curated audio clips from Apple's SEP-28k stuttering dataset, preprocessing labelled speech samples and balancing fluent and disfluent classes for model development and evaluation.
Engineered four categories of acoustic features using Librosa — Mel-Frequency Cepstral Coefficients (MFCCs), zero-crossing rate, RMS energy and spectral centroid — to capture speech characteristics for classification.

Speech pipeline

Integrated Whisper AI with word-level timestamps and WebRTC Voice Activity Detection, enabling transcript generation, segmentation of voiced and silent intervals, and analysis of pauses and potential disfluencies.
Implemented a FastAPI-based speech processing backend, combining transcription, acoustic feature extraction, ML inference and fluency scoring into a unified analysis workflow.

App & feedback

Developed an end-to-end speech fluency analysis application using Python, FastAPI and Streamlit, integrating automatic speech recognition, voice activity detection and supervised machine learning into a local inference pipeline.
Developed a Streamlit frontend supporting three audio formats (WAV, MP3 and M4A), displaying fluency scores, disfluency probabilities, transcriptions and word replacement suggestions.
Integrated NLTK WordNet for context-aware synonym suggestions, extending the application beyond disfluency classification to provide actionable feedback for speech practice.

Privacy

Designed the application for fully local execution, running inference without external API calls to keep uploaded speech recordings within the local environment.