MedSpeech
An AI-enabled medical speech processing platform for clinical toxicology: a Django/DRF backend service that took a recorded or typed consultation and carried it all the way to a finalized note — transcription, extracted clinical facts, AI analysis, treatment guidance, and safety alerts, with an audit trail and version history behind every step. Retired in 2026 once its implementation was ported into ToxLumen as the clinical case domain.
Overview
An AI-enabled medical speech processing platform for clinical toxicology: a Django/DRF backend service that took a recorded or typed consultation and carried it all the way to a finalized note — transcription, extracted clinical facts, AI analysis, treatment guidance, and safety alerts, with an audit trail and version history behind every step. Retired in 2026 once its implementation was ported into ToxLumen as the clinical case domain.
Problem
Toxicology consultations happen by phone and by voice, and the record of them was being written afterwards from memory. Commodity transcription solved only the first step: what clinicians actually needed was the consultation turned into structured clinical facts, checked for interaction and dosing risks, and closed out as a documented note that could be audited later.
Solution
Built a service organized around the consultation rather than around the audio file. A submitted source — audio or text — was transcribed, reduced to structured facts, analysed, and turned into guidance and a note, with each stage tracked as its own job so a failure never lost the work before it. Safety alerts ran against substance guidelines, every state change was written to an audit log, and finalizing a case locked it while keeping a full version history. Workspaces with invitations let a clinical team share cases instead of each clinician working alone.
Architecture
Django + DRF under a strict layered pattern — views → serializers → services/selectors — with one file per resource in every layer. The domain modelled the consultation directly: source, subject, audio asset, processing job, analysis, guidance, safety alert, note, version, audit event, folder, and substance guideline. Provider work sat behind an integrations layer with separate transcription, LLM and guideline clients, so Deepgram, OpenAI and Groq could be swapped or fallen back on without touching domain code. Stage execution ran on Huey with per-stage job records the frontend polled for progress; media went to S3; the React 19 + Vite frontend consumed the API. When a case was finalized it became immutable, and any later change to an upstream fact was surfaced as an affected-downstream signal rather than applied silently.
Challenges
Designing a multi-stage AI pipeline where each stage could fail independently: stages were modelled as job records with their own state, so a failed analysis could be retried without re-transcribing the audio or discarding the facts already extracted.
Keeping a clinical record trustworthy after the fact — an append-only audit event for every state change, versioning on the documents, and a hard lock on finalized cases, so what the note said at sign-off stays recoverable.
Handling edits to upstream facts on a case whose analysis was already generated: rather than silently regenerating, the system marked the downstream artifacts as affected and left the decision to the clinician.
Abstracting three AI providers with different latency, failure modes and rate limits behind one integrations layer, so provider choice became configuration rather than a code change.
Outcomes
Ran in production for Medical Toxicology clients from 2024 until its retirement in 2026, across roughly 1,900 commits.
Took the consultation record from post-hoc typing to a documented, audited, versioned artifact produced from the consultation itself.
Its implementation became the specification for ToxLumen: the case domain, AI pipeline, safety alerts, audit trail and finalized-lock were ported directly rather than redesigned.
Retired deliberately rather than abandoned — the service was shut down only once its successor had absorbed the domain, with the original repository kept as the migration reference.
Lessons Learned
Model the workflow, not the file. Organizing the system around the consultation rather than around the uploaded audio is what let transcription, analysis, guidance and documentation live in one coherent record.
AI latency is unpredictable, so every stage needs its own durable job record — async-first with per-stage state is the only pattern that survives a provider timing out halfway through.
Immutability has to be designed in from the start: an audit trail and a finalized lock retrofitted onto a clinical system are never fully trustworthy.
A well-layered service is worth more at the end of its life than at the start — the layering is precisely what made it portable into its successor instead of a rewrite.
