TL++: Accuracy and Privacy Preserving Traversal Learning for Distributed Intelligent Systems

Researchers have proposed TL++, a framework for distributed intelligent systems that preserves accuracy and privacy, achieving up to 91.41% accuracy on CIFAR-10 while reducing per-step communication by 13.1-fold^[1].

Distributed intelligent systems often need to train across data silos without centralizing raw data. Federated learning and split learning have limitations, including high communication costs and potential exposure of sensitive information^[1]. TL++ addresses these issues by constructing virtual batches across nodes and using a secure mode to secret-share cut-layer activations and gradients between an orchestrator and a non-colluding helper. This approach prevents either server from observing plaintext cut-layer tensors. The framework was evaluated against federated and split-learning baselines on CIFAR-10 and BioGPT/PubMedQA using full fine-tuning and LoRA. On CIFAR-10, TL++ achieved accuracies of 91.41% (standard deviation 0.19) and 90.93% (standard deviation 0.17) for base cut 1 and exact secure cut 3, respectively. In a related development, researchers explored the use of fully homomorphic encryption for privacy-preserving inference tasks, achieving minimal drop in classification accuracy for various datasets, including MNIST, Kuzushiji MNIST, Fashion-MNIST, and CIFAR-10^[2]. Microsoft SEAL was used for performing homomorphic encryption.