Cost-Aware Routing for Efficient Text-To-Image Generation

Researchers have proposed new methods to optimize text-to-image generation, balancing quality and computational cost. A framework allows computation to vary based on prompt complexity, while another method accelerates auto-regressive generation using a multi-sequence draft tree structure.

A new framework is proposed to optimize the trade-off between quality and computational cost in text-to-image generation by allowing the amount of computation to vary for each prompt based on its complexity^[1]. Diffusion models generate high-fidelity images but at a high computational cost due to their iterative denoising process. The framework learns to reserve expensive choices for complex prompts and employ more economical choices for less sophisticated prompts. It delivers an average quality higher than that achievable by any of the individual text-to-image models alone. Meanwhile, a new method, PathRelax, is proposed to accelerate auto-regressive text-to-image generation by enhancing efficiency through a multi-sequence draft tree structure^[2]. PathRelax expands the token search space, achieving a higher speedup ratio without sacrificing image quality. Its relaxation mechanism can be seamlessly integrated with other relaxation techniques, enabling further acceleration.