FMplex: Model Virtualization for Serving Extensible Foundation Models

A new serving system called FMplex treats foundation-model backbones as a virtualization layer, allowing multiple downstream tasks to share a single physical model while maintaining logical isolation, its developers report. Foundation models are increasingly deployed as backbones for tasks spanning language, vision, time-series, and multimodal applications [1]. Current serving systems typically deploy each customized task as an independent model instance, duplicating heavyweight backbones, consuming accelerator memory, and missing opportunities to amortize batching and loading costs [1]. FMplex introduces a virtual-foundation-model abstraction that presents each task with a logically private instance backed by a shared physical backbone, preserving task-specific extensions, independent lifecycles, and task-level isolation [1]. The system incorporates a batch-aware fair-queueing scheduler that combines weighted task-level sharing with inter- and intra-task batching across co-located tasks [1]. Across seven foundation-model backbones comprising 16 variants and 92 downstream tasks, FMplex reduced latency by up to 80% over spatial partitioning and 33.3% over best-effort co-location, while hosting up to 6x more tasks at cluster scale [1]. The implementation spans task construction, sharing-aware deployment, and runtime execution [1]. The work arrives as the broader machine-learning infrastructure community continues to explore efficient multi-tenancy for large models; several recent preprints on arXiv similarly address co-location and resource-sharing strategies for transformer-based architectures, though their abstracts provide only tooling metadata rather than performance comparisons [3][4][5]. The FMplex authors argue that treating foundation-model backbones as a virtualization substrate can reduce accelerator waste and improve cluster utilization without sacrificing the independence that downstream developers require [1].