A Unified Theory of Sinusoidal Activation Families for Implicit Neural Representations

Researchers have proposed a new framework for Implicit Neural Representations (INRs), a tool used in vision, graphics, and signal processing. The framework, called Sinusoidal Trainable Activation Functions (STAF), uses learned amplitudes, frequencies, and phases.

INRs model continuous signals with compact neural networks and have shown impressive results in various tasks. However, their fundamental capacity and scaling behavior remain poorly understood^[2]. Periodic activations, such as STAF, have emerged as a remedy for accurately capturing fine detail in INRs^[1]. STAF is competitive and often stronger on distortion-oriented reconstruction metrics such as PSNR/SSIM across evaluated INR tasks, according to a study published on arxiv.org^[1]. Another study on the same platform found that grids with interpolation train faster and to higher or comparable quality than INRs with the same number of parameters for many tasks involving dense signals^[2]. However, INRs outperform grids in limited settings, such as fitting binary signals like shape contours. The performance of INRs varies across different tasks and signal types, including 2D and 3D real and synthetic signals^[2].