arXiv AI recent: ITNet: A Learnable Integral Transform That Subsumes Convolution, Attention, and Recurrence
The article introduces ITNet, a learnable integral transform that unifies convolution, attention, and recurrence mechanisms.,ITNet uses a neural network (MLP) as a kernel to model pairwis...
ITNet is a unified architecture that subsumes convolution, self-attention (including multi-head), and autoregressive recurrence (including LSTM, GRU, S4, and Mamba) as special cases through parameterization.,The kernel in ITNet is implemented as a small MLP that adapts pairwise interactions based...