The final, formatted version of the article will be published soon.
METHODS article
Front. Behav. Neurosci.
Sec. Individual and Social Behaviors
Volume 18 - 2024 |
doi: 10.3389/fnbeh.2024.1509369
SARTAB, a scalable system for automated real-time behavior detection based on animal tracking and region of interest (ROI) analysis: validation on fish courtship behavior
Provisionally accepted- Georgia Institute of Technology, Atlanta, United States
Methods from Machine Learning (ML) and Computer Vision (CV) have proven powerful tools for quickly and accurately analyzing behavioral recordings. The computational complexity of these techniques, however, often precludes applications that require real-time analysis: for example, experiments where a stimulus must be applied in response to a particular behavior or samples must be collected soon after the behavior occurs. Here, we describe SARTAB (Scalable Automated Real-Time Analysis of Behavior), a system that achieves automated real-time behavior detection by continuously monitoring animal positions relative to behaviorally relevant Regions Of Interest (ROIs). We then show how we used this system to detect infrequent courtship behaviors in Pseudotropheus demasoni (a species of Lake Malawi African cichlid fish) to collect neural tissue samples from actively behaving individuals for multiomic profiling at single nucleus resolution. Within this experimental context, we achieve high ROI and animal detection accuracies (mAP @[.5 : .95] of 0.969 and 0.718 respectively) and 100% classification accuracy on a set of 32 manually selected behavioral clips. SARTAB is unique in that all analysis runs on low-cost, edge-deployed hardware, making it a highly scalable and energy-efficient solution for real-time experimental feedback. Although our solution was developed specifically to study cichlid courtship behavior, the intrinsic flexibility of neural network analysis ensures that our approach can be adapted to novel species, behaviors, and environments.
Keywords: Behavior, computational ethology, Cichlidae fish, Computer Vision, machine learning, real-time analysis
Received: 10 Oct 2024; Accepted: 13 Nov 2024.
Copyright: © 2024 Lancaster, Leatherbury, Shilova, Streelman and McGrath. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
* Correspondence:
Jeffrey Todd Streelman, Georgia Institute of Technology, Atlanta, United States
Patrick McGrath, Georgia Institute of Technology, Atlanta, United States
Disclaimer: All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.