SHF: Small: On-device Large Language Model Personalization with Algorithm-Hardware Co-design for Healthcare Applications

Personalized healthcare that considers individual differences in genetics, lifestyle, and medical history is more effective than one-size-fits-all solutions. This project utilizes advanced wearable and portable devices with large language models (LLMs) to enhance personalized healthcare by addressing the patient variability often overlooked by current methods. It focuses on real-time healthcare personalization through fast and accurate data searches in ever-growing personal databases, using novel memory semiconductor devices, advanced circuits, and custom architecture. This enables quick, personalized interactions on devices, potentially saving lives with timely interventions for emergencies like suicide attempts and strokes, thus advancing precision medicine and national health. Additionally, the project will support activities to enhance healthcare education for K-12 students, tech briefings on semiconductor technology for undergraduate students, and mentoring support for underrepresented students.

The project addresses on-device LLM personalization through Retrieval Augmented Generation (RAG) for healthcare applications, aiming to significantly reduce latency and hardware overhead via algorithm-hardware co-design. It will define healthcare scenarios for LLM applications, generate user prompt input datasets for benchmarking, and create personalized healthcare datasets for LLM personalization. Efficient RAG-based personalization will be explored, focusing on unsupervised data selection and optimal embedding dimension/bit-width selection. To mitigate computation-storage data transfer bottlenecks, custom compute-in-memory architectures and data search frameworks using Ferroelectric Field-Effect Transistors will be investigated. This approach aims for a 1000-fold latency reduction and a 100-fold increase in energy efficiency compared to state-of-the-art edge LLM embedded systems, setting new benchmarks for edge computing performance and sustainability. Successful implementation will enhance personalized healthcare interventions and advance AI-assisted personalized healthcare.