Self-attention-based non-linear basis transformations for compact latent space modelling of dynamic optical fibre transmission matrices

Researchers have made advancements in using multimode optical fibres for sub-cellular image resolution deep inside the body, overcoming challenges related to image scrambling during transmission.

Multimode optical fibres can transport light efficiently and provide sub-cellular image resolution deep inside the body, according to a study published on arxiv.org^[1]. However, images transmitted through these fibres are scrambled, requiring an unscrambling process that accounts for dynamic changes in the fibre's effect on light. A new approach using self-attention layers has been introduced to dynamically transform the coordinate representations of varying fibre matrices to a basis that admits compact, low-dimensional representations. This method has shown significant improvement in the sparsity of fibre bases, with a participation ratio between 0.01 and 0.11^[1]. The transformed representations also allow for reconstruction of the original matrices with less than 10% reconstruction error. In related research, neural networks for histopathology classification tasks rely on data encoding into latent space, and encoder networks can be pretrained on general image datasets or specifically on histopathological images^[2]. The choice of pretraining dataset can impact performance, and adapting training to downstream tasks can improve outcomes. Researchers used networks provided by Lunit Inc., Bioptimus, and Meta Research Team to investigate the effect of image transformation on latent space.