Safeguarding Text-to-Image Generation via Inference-Time Prompt-Noise Optimization

Researchers have developed two new methods to improve image generation: one for safe image generation and another for creating single-line drawings in vector format.

A team of researchers has proposed a novel approach called Prompt-Noise Optimization (PNO) to mitigate the generation of unsafe images by Text-to-Image (T2I) diffusion models. T2I models are prone to producing images containing sensitive or inappropriate content^[1]. Current methods to prevent this are vulnerable to adversarial attacks. PNO achieves state-of-the-art performance in suppressing toxic image generations and demonstrates robustness to adversarial attacks. It does so without requiring model parameter tuning and maintains comparable generation time. Separately, another research team has developed a semantics-driven approach for generating single-line drawings in vector format. This method uses score distillation sampling to optimize the parameters of a uniform rational B-spline (URBS) curve, resulting in a single continuous stroke by design^[2]. The generated drawings are more aesthetically pleasing and faithful to the style of continuous line drawing artists. They also directly support downstream fabrication processes such as embroidery, laser engraving, and wire bending.