When I implemented a realtime speech transcript server to save lives.

Introduction

I am the developer of the MVP stage of a patients management and assessment system for hospitals, currently called "Bot2Talk".

This software contains an A.I.-based (GPT) chatbot for taking the relevant symptoms from the patients and assisting them temporarily.

This system also contains a section for doctors to log in and visualize their patients and their current status. When the doctors consider the info took by the chatbot to be sufficient, they can have the patient enter into a virtual meeting.

In this meeting, the doctor further analyzes the patient, and during their session, a live transcript of the patient's symptoms is generated, that is further used to automatically generate an editable S.O.A.P. note for the doctor.

A reports and analytics panel is included with each module, and a master analytics dashboard with all the relevant info presented exists as well.

I created a transcript server from scratch that I am specifically proud of implementing due to it being to analyze the conversation of the doctors / patients in real-time and organizing the data accordingly for the S.O.A.P. note generation.

How it Works

I developed this app using Python / Flask / MySQL, and Node.js for the transcript service.

I am providing below a brief description of the functionality that includes the patient's and the doctor's perspective in the form of steps. Do note that this list is not exhaustive and there are further features just in the M.V.P. stage of this software.

1. The patient appears on the bot after filling out a pre-session form with basic info.
2. The bot asks them a few questions and symptoms regarding their problem, and assures them that a doctor will be with them in a little while after getting its answers.
3. Patient answers all the questions.
4. Doctor can view this patient on their dashboard, and their current conversation with the bot. They can now decide if the provided info is enough for them to engage in a virtual meeting and so they click the large video-icon button and both, the doctor and the patient move into the meeting simultaneously.
5. During the meeting, both, the doctor and the patient have to consent to having a transcript generated of the meeting due to some security policies. Without consent of both parties, the transcript will not be generated.
6. The doctor discusses with the patient, and when the session ends, the doctor returns to their dashboard on that that specific visit's page and clicks on "View SOAP Note", and there, a S.O.A.P. note that considers the patient's session with the chatbot, and the transcript of their session with the doctor, is generated via GPT technology. The doctor can further edit this note if needed, and they can click save to have it saved with this visit forever.
7. The doctor can also choose to generate an S.O.A.P. note of the patient's conversation with the bot alone should they choose to not have a meeting for whatever reason. The said generated note should only cover the "Subjective" part of the note and others would be left blank.
8. Repeat 🔁

Some Screenshots