Hierarchical Modeling of ICD Codes in EHR Foundation Models

A new study proposes that explicitly encoding the hierarchical structure of ICD-10-CM diagnosis codes can improve how foundation models learn from electronic health records, a departure from standard methods that treat codes as flat, unrelated tokens [1]. The work, submitted on 13 Jun 2026, investigates the ICD-10-CM hierarchy as a general inductive bias for clinical representation learning [1]. Medical classification systems like ICD-10-CM transform descriptions of diagnoses into standardized statistical codes used for tracking diseases, processing reimbursement claims, and supporting decision-support systems [4]. The researchers note that existing electronic health record (EHR) foundation models typically treat these codes as flat tokens, overlooking the clinically meaningful structure that captures disease families, subcategories, and fine-grained diagnostic detail [1][2]. The study explores two complementary mechanisms for incorporating hierarchy. The first augments diagnosis sequences in a BERT-style transformer with tokens corresponding to different levels of the ICD hierarchy. The second injects hierarchy into graph-based code representations through hierarchy-aware edges combined with diagnosis co-occurrence structure [1][2]. These approaches stand in contrast to other data models used in clinical informatics, such as the entity–attribute–value model, which is optimized for sparse, ad-hoc property storage rather than leveraging pre-defined hierarchical relationships [3]. Experiments were conducted on two large-scale real-world clinical datasets: MIMIC-IV, used for pretraining and in-domain evaluation, and eICU, used to assess cross-dataset transfer via frozen encoder probing [1][2]. The findings show that explicitly encoding ICD hierarchy improves over flat code representations in both in-domain and cross-dataset settings [1][2]. The most useful level of hierarchy was found to depend on both the task and the modeling approach [1]. The research contributes to a broader effort to make clinical data more interoperable and semantically rich. Other standardized terminologies, such as SNOMED CT, provide a systematically organized collection of medical terms that can cross-map to classifications like ICD, supporting consistent information interchange and reducing variability in how data are captured and used for patient care and research [5]. By demonstrating that hierarchy-aware EHR representation learning generalizes across modeling settings and hierarchy levels, the study suggests a path toward foundation models that better reflect the structured nature of clinical knowledge [1][2].