Skip to main content

ORIGINAL RESEARCH article

Front. Artif. Intell.
Sec. Machine Learning and Artificial Intelligence
Volume 7 - 2024 | doi: 10.3389/frai.2024.1462952

SineKAN: Kolmogorov-Arnold Networks Using Sinusoidal Activation Functions

Provisionally accepted
Eric Allen Friss Reinhardt Eric Allen Friss Reinhardt *Sergei Gleyzer Sergei Gleyzer
  • University of Alabama, Tuscaloosa, United States

The final, formatted version of the article will be published soon.

    Recent work has established an alternative to traditional multi-layer perceptron neural networks in the form of Kolmogorov-Arnold Networks (KAN). The general KAN framework uses learnable activation functions on the edges of the computational graph followed by summation on nodes. The learnable edge activation functions in the original implementation are basis spline functions (B-Spline). Here, we present a model in which learnable grids of B-Spline activation functions are replaced by grids of re-weighted sine functions. We show that this leads to better or comparable numerical performance than B-Spline KAN models on a benchmark vision task. Further, we compare to traditional multi-layer perceptron models and an alternative KAN implementation based on periodic cosine and sine functions representing a Fourier Series. We also show this model provides a substantial speed increase at all hidden layer sizes, batch sizes, and depths.

    Keywords: machine learning (ML), Periodic function, Kolmogorov - Arnold representation, Kolmogorov-Arnold Networks (KANs), Kolmogorov-Arnold Network, Sinusoidal activation function

    Received: 10 Jul 2024; Accepted: 16 Dec 2024.

    Copyright: © 2024 Reinhardt and Gleyzer. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

    * Correspondence: Eric Allen Friss Reinhardt, University of Alabama, Tuscaloosa, United States

    Disclaimer: All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.