EMT Vision: Revolutionizing Emergency Medical Response with AR Technology
As my senior design project at Santa Clara University, developed in collaboration with the Santa Clara County Fire Department and funded by the School of Engineering, EMT Vision represents a breakthrough in emergency medical services technology. This project was awarded first place in the 2025 Senior Design Competition, recognizing its innovative approach to solving real-world challenges in emergency medical response.
Project Overview
EMT Vision offers automated and lightweight solutions for minimizing redundant workloads among first responders. Developed by Logan Calder, Grant Johnson, John Alvarado, and Jack Landers, under the guidance of Dr. Krishna Ramamoorthy, this project aims to transform emergency medical response through innovative technology. Our collaboration with the Santa Clara County Fire Department provided invaluable insights into the daily challenges faced by first responders.
The Problem
In emergency medical services, every second counts. Yet, current systems require a dedicated paramedic for form documentation during each call, significantly impacting response efficiency. This not only extends call times but also introduces potential errors in critical patient information. The manual documentation process consumes valuable resources that could be better utilized in direct patient care.
The Solution
EMT Vision introduces a lightweight, AI-powered Microsoft HoloLens 2 headset that revolutionizes the documentation process. By leveraging advanced audio processing and multimodal LLMs, the system automatically generates electronic Patient Care Report (ePCR) forms while paramedics focus on patient care. The headset also provides real-time protocol visualization and progress tracking, ensuring comprehensive care without compromising efficiency.
Technical Architecture
The project consists of two main components: an AR headset application and a web platform. The headset application, built with Unity and C#, integrates Microsoft Azure Speech Services for real-time transcription and OpenAI GPT-4 for intelligent form population. All patient data is securely stored using Supabase's HIPAA-compliant database system.
The web platform, developed with Next.js and TypeScript, provides a secure interface for medical professionals to access patient information. Authentication is handled through Google Cloud Platform OAuth, ensuring that only authorized personnel can access sensitive data. The entire system is deployed on Vercel, providing reliable and scalable hosting.
Testing & Results
Our rigorous testing phase demonstrated the system's reliability in real-world conditions. The headset achieved 89% accuracy in multi-speaker environments, 94% accuracy in noisy conditions, and 98% accuracy in handling corrected information. These results, combined with positive feedback from first responders, contributed to our project's success in the Senior Design Competition.
Future Development
Building on our success, we plan to enhance the system's HIPAA compliance and security measures. Future developments include implementing document and ID scanning capabilities, improving hardware with better microphones, and adding advanced UI features for medication tracking. These improvements will further streamline emergency response procedures while maintaining the highest standards of patient care and data security.
View our project demo here.
Visit our web platform here.
