AUTHOR=Sattelberg Ben , Cavalieri Renzo , Kirby Michael , Peterson Chris , Beveridge Ross TITLE=Locally linear attributes of ReLU neural networks JOURNAL=Frontiers in Artificial Intelligence VOLUME=6 YEAR=2023 URL=https://www.frontiersin.org/journals/artificial-intelligence/articles/10.3389/frai.2023.1255192 DOI=10.3389/frai.2023.1255192 ISSN=2624-8212 ABSTRACT=

A ReLU neural network functions as a continuous piecewise linear map from an input space to an output space. The weights in the neural network determine a partitioning of the input space into convex polytopes, where each polytope is associated with a distinct affine mapping. The structure of this partitioning, together with the affine map attached to each polytope, can be analyzed to investigate the behavior of the associated neural network. We investigate simple problems to build intuition on how these regions act and both how they can potentially be reduced in number and how similar structures occur across different networks. To validate these intuitions, we apply them to networks trained on MNIST to demonstrate similarity between those networks and the potential for them to be reduced in complexity.