Beyond Single Discrete Responses: An Integrative and Multidimensional Analysis of Behavioral Dynamics Assisted by Machine Learning

The Spatial Dimension: A Relevant Feature Neglected by Standard Behavioral Science Paradigms

The main objective of behavioral science is to account for the principles that underlie the behavioral system, understanding it as an emergent and complex system comprising an environment and organism (Skinner, 1938; Kantor, 1970; Kuo, 1976; Gibson, 1979; Timberlake, 1994; Turvey, 2018). In simple words, the principal goal of behavioral science is to describe the principles and processes of natural behavior. The foundational works of behavioral science show that natural behavior includes the spatio-temporal dynamic as a fundamental dimension (e.g., approach-withdrawal patterns; Schneirla, 1959).

Nevertheless, for various reasons, in the history of experimental behavioral science, the temporal distribution of discrete-response analysis gained prominence over the analysis of spatial patterns and their dynamics. One reason for this was the affordable technology available at the end of the first half of the last century to make reliable and automatized behavior records and measures. These records were made primarily through the use of mechanical and electronic switches (Escobar, 2014). Thus, the methodological approach focused on computing the frequency and temporal distribution of the activation or deactivation of switches (e.g., the total number of responses to an operand, number of responses per unit of time, inter-response times, etc.). This approach is called the single-response paradigm (Henton and Iversen, 1978). Until now, it has been the standard in the experimental analysis of animal behavior (e.g., operant and Pavlovian paradigms).

The predominance of apparatus, measures, data analysis, and data representations based on discrete responses (e.g., lever press, food dispenser entrance) resulted in the spatial dimension of behavior being generally neglected. It follows that standard approaches in experimental behavioral science do not account for the natural behavior associated with the organism's movement and its embedded dynamics (León et al., 2020b).

Computational Animal Behavior Analysis and Integration of the Spatial Dimension To The Experimental Analysis Of Behavior (EAB)

Under operant and Pavlovian paradigms, behavioral systems include complex interactions between spatiotemporal patterns, discrete responses, and programmed stimuli, challenging to apprehend with the methodological standard approaches of the single-response record (see Henton and Iversen, 1978; Pear, 1985). Although this issue was pointed out a long time ago, it has not been easy to solve for Experimental Analysis of Behavior (EAB). Developing an integrative approach between environmental features, discrete responses, and spatiotemporal dynamics is still challenging.

Computational advances made in the last decade (i.e., computer vision, machine learning, and deep learning techniques) have made the recording, measurement, and analysis of spatial patterns of behavior affordable (Dell et al., 2014; Pérez-Escudero et al., 2014; Mathis and Mathis, 2020). Moreover, these technological advances to facilitate accurate and objective analysis of behavior have opened new methodological perspectives in behavioral science (Menaker et al., 2020), such as computational ecology. Nevertheless, EAB has so far benefited little from these developments.

Current computational methods (Mathis et al., 2018; Datta, 2019; Marshall et al., 2020; Torabi et al., 2020) provide invaluable tools to crystallize an integrative EAB approach for the analysis and understanding of the spatiotemporal dynamics (Maekawa et al., 2020; Loveless and Webb, 2021) associated with relevant behavioral phenomena (León et al., 2020a,b). This multidisciplinary approach could show behavioral features and processes, hidden until now to behavioral science and, more specifically, to EAB. Hence, this emergent multidisciplinary approach could be called computational-experimental analysis of behavior (CEAB).

How the Integrative Approach of CEAB Could Extend the Scope of Behavioral Science and EAB

EAB could be positively affected by CEAB in recording, measuring, analyzing, and representing the behavioral systems. In addition, CEAB could help to identify features or variables embedded in the spatiotemporal continuum of behavior under well-established methodological paradigms (e.g., operant and Pavlovian conditioning) hidden until now. If this is the case, the revealed features could extend our understanding of behavioral processes and the scope of behavioral science and eventually open new research possibilities.

Recording

The relevance of accurate and objective records to any empirical science is well-known. It is established that one of the main reasons for the success of the operant paradigm is its objective record of behavior (Escobar, 2014; León et al., 2020b). Under the single-response paradigm (e.g., pressing the lever or entering the dispenser) it is possible to identify ordered functional relations between different variables and procedures (e.g., schedules of reinforcement, deprivations or motivational operations) and temporal patterns of discrete responses. CEAB, recording multiple responses and spatial behavior, could close the gap to identify new interactions and determinants between environmental events and spatiotemporal patterns of behavior.

Measuring and Data Analysis

There is a strong relationship between recording, measuring, and data analysis. Under the single-response paradigm, the primary measure has been the response rate (Skinner, 1966; see any current issue of Journal of Experimental Analysis of Behavior). The analysis focuses on unidimensional changes in this measure. The multidimensional data obtained through CEAB (e.g., through sensing of discrete responses and spatial behavior with tracking systems based on computer vision) extend measurements and analyses coherently with an approach that assumes behavior as a spatiotemporal continuous system (Kantor, 1970; Henton and Iversen, 1978; Gibson, 1979; Pear, 1985; Timberlake, 1994; León et al., 2020b). Given the vast possibilities of behavior measuring and the considerable amount of data associated with the continuous spatiotemporal recording of behavior, the central issue is what we should measure and analyze and why (Menaker et al., 2020).

First, it is relevant to measure and analyze discrete responses (e.g., lever presses, dispenser entrances, “correct” responses, among others, depending on specific behavioral phenomena) to have a comparative and parsimonious approach covering the standard paradigms in the EAB.

On the other hand, given that some very plausible proposals on the relevant functions of spatial behavior (Elliott, 1934; De Valois, 1954; Berlyne, 1955; Duffy, 1957; Schneirla, 1959; Henton and Iversen, 1978; Pear, 1985) were gradually abandoned due to the lack of record systems, the technology available at the time, and the predominance of the single-response paradigm, it could be fertile to recover past insights about spatial behavior with current technology (Spruijt et al., 2014). The primary dimensions identified in those proposals were the direction, intensity (or vigor), and variation of behavior. The recording of both discrete responses and spatial behavior makes it possible to account for these dimensions. Thus, in second place, it could be relevant to measure and analyze the direction, for example, as approach–withdrawal patterns to relevant areas and stimuli (Duffy, 1957; Schneirla, 1959); intensity, for example, as traveled distance, velocity, and rate of response; and variation of spatial behavior, for example, as recurrence patterns and entropy. Considering these dimensions in CEAB could be a bridge to close the gap between EAB and other paradigms of behavioral science, facilitating seeing multiple aspects of behavioral phenomena more fully. In addition, these dimensions could be helpful to identify behaviorally meaningful patterns.

Data Representation

An additional challenge is to conduct an analysis and data representation that integrates, in a perspicuous way, both discrete responses and spatial behavior as a whole behavioral system. This integrative analysis should identify and represent the participation and relevance (e.g., ranking variables) of different behavioral features or dimensions in the system (e.g., comparing the weight between variables based on discrete responses and continuous spatial behavior). Until now, this has been a challenging task that can be resolved with methods for multidimensional analysis based on machine learning, such as t-distributed stochastic neighbor embedding (t-SNE).

A First Approach of the CEAB

Figure 1 shows the scheme of the general procedure used in this work. Each colored row depicts a component of our approach, assisted by different computational tools and procedures, namely, sensing/recording, measuring, data analysis, and representation. Columns depict continuous spatial behavior and discrete responses, respectively. The intersection between rows and columns exemplifies some applications of computational tools in a given component for each kind of response (discrete or spatial).

FIGURE 1

Figure 1. Graphical representation of the proposed integrative, multilevel, and multidimensional approach. It integrates recordings for continuous spatial behavior (based on machine vision), discrete responses of the organisms with multiple measures for each one, multilevel data analysis (within-subject, within-session, between subjects, between sessions), and a multidimensional characterization of the unitary system between spatiotemporal dynamics and discrete responses.

The proposed approach integrates (a) recording for continuous spatial behavior (based on machine vision) and discrete responses of the organisms; (b) multiple measures for each record (first order, such as velocity and distance to focal points, and non-first order, entropy and divergence); (c) multilevel data analysis (within-subject, within-session; between subjects, between sessions); and (d) multidimensional and integrative representations of the system, made up spatial behavior and discrete responses, based on machine learning.

Under the CEAB approach, we present two examples using a subset of data sets from our laboratory (León et al., 2020a; Hernández et al., 2021) related to relevant behavioral phenomena: (a) water-seeking behavior under temporal schedules and (b) motivational effects of water and food deprivation. These phenomena have well-documented dynamics of discrete responses but scarce findings concerning spatial dynamics and even less integrative analysis between discrete responses and spatial behavior patterns.

The purposes of the present work are (a) to provide behavioral measures based on spatial dynamics sensitive to paradigmatic procedures with well-known effects on discrete responses (e.g., stimulus schedules and motivational operations), (b) to reveal spatial behavioral features usually hidden under standard approaches based on the single-discrete response paradigm, (c) to illustrate helpful multidimensional representations based on machine learning for the integration of discrete responses and spatial behavior for a more comprehensive study of behavioral systems.

Although the general approach based on CEAB is the same for both examples, given the experiments correspond to different phenomena, a specific justification, methods, and results are presented for each one. Finally, a general discussion related to both examples and the purposes of the work is presented. The hypotheses in this work are the following: (1) the proposed measures based on spatial behavior are sensitive to EAB paradigmatic procedures; (2) the CEAB, assisted by machine learning, reveals that spatial features are at least as relevant as behavioral features based on discrete responses; (3) CEAB shows that discrete responses and spatial behavior integrate a whole behavioral system even under different experimental procedures and phenomena.

Example 1

Water-Seeking Behavior: Behavioral Dynamics Under Fixed and Variable Temporal Schedules

One of the most significant contributions of EAB to comprehend the variables underlying behavioral phenomena are stimuli schedules (e.g., schedules of reinforcement; Ferster and Skinner, 1957). A stimuli schedule is a rule, defined in a systematic and parametric way, to present stimuli (e.g., water, food; Reynolds, 1975). The stimuli schedules can be categorized according to several criteria; one of the most common and useful is contingent vs. non-contingent schedules. The first one, in methodological terms, is usually associated with operant contingencies, and the second one with Pavlovian contingencies. The difference between contingent and non-contingent schedules is that, in the first one, the occurrence of a given stimulus is dependent (or contingent) on a given response of the organism (e.g., lever presses). In contrast, in non-contingent schedules, the presentation of the stimuli does not depend on any organism's response but only on the temporal relation between the stimuli. These last are named time-based schedules.

There is a vast corpus of research with both kinds of schedules with several species and apparatus (Zuriff, 1970; Lachter et al., 1971; Boren et al., 1978; León et al., 2020a; Hernández et al., 2021). Most of this research is based on the recording and data analysis of a single discrete response, especially with “appetitive” stimulation. The primary data are head entries to a food or water dispenser and food pellets or drops of water consumed—in other words, the temporal distribution of a given discrete response. Only in a few studies, the data was extended to time spent in zones near a dispenser (Baum and Rachlin, 1969). Different behavioral phenomena are studied with time-based schedules, such as “superstitious behavior” (Skinner, 1948; Reberg et al., 1977) and “timing” (Drew et al., 2005; Sanabria et al., 2009), among others. These different phenomena could be characterized as behavioral systems and their corresponding spatial-temporal dynamics from a systemic and parametric approach. The effects of the two time-based schedules, fixed and variable, on behavior are scarcely studied comparatively. On the other hand, no comparative studies explicitly include the spatial dimension of behavior and its dynamics. Under these fixed and variable temporal schedules, neither integrates the standard data based on discrete responses with continuous data based on locomotion.

Under this rationale, we evaluate the spatial dynamics of behavior in Wistar rats under two temporal water-delivery schedules (fixed and variable-time schedules) in a modified open field system (MOFS) from a multidimensional analysis, using machine learning tools.

Method

Subjects

Four experimentally naïve female Wistar rats were used; two rats were assigned to a fixed-schedule condition and two rats to a variable-schedule condition. All rats were 3 months old at the beginning of the experiment. Rats were housed individually with a 12-h light and dark cycle and maintained under a daily schedule of 23 h of water deprivation with free access to water 1 h after experimental sessions. Food was freely available in their home cages. One session was conducted daily, 7 days a week. All procedures were conducted according to university regulations of animal use and care and followed the official Mexican norm NOM-062-ZOO-1999 for Technical Specification for Production, Use, and Care of Laboratory Animals.

Apparatus

A MOFS (model WEOF by Walden Modular Equipment) was used. A diagram of the apparatus can be found in León et al. (2020a). Dimensions of the chamber were 100 × 100 cm. All four walls of the chamber and the floor were made of black Plexiglas panels. A water dispenser (by Walden Modular Equipment), based in a servo system, when activated, delivered 0.1 cc of water on a water cup that protruded from the center of the MOFS. The MOFS was illuminated by two low-intensity lights (3 watts) located above the chamber and on opposite sides of the room to avoid shadowed zones. Once delivered, the water remained available for 3 s. A texturized black patch, 9 × 9 cm with 16 dots/cm, printed in a 3-D printer, was located close to the water dispenser to facilitate its location.

The experimental chamber was located in an isolated room on top of a table of 45 cm in height. The room served to isolate external noise. All programmed events were scheduled and recorded using a Walden Tracking System (v.0.1). In addition, a Logitech C920 web camera recorded rats' movement at the center, located 1.80 m above the experimental chamber. Tracking data was analyzed using the Walden Tracking System (v.0.1). This software recorded rats' location, by the center of mass, every 0.2 s in the experimental space using a system of X, Y coordinates. Data files obtained from this software were then analyzed using MOTUS© and Orange 3.26 Software.

Procedure

Subjects were exposed to one of two conditions of water delivery: (a) a fixed time (FT) 30 s schedule or (b) variable time (VT) 30 s schedule. Each condition lasted 20 sessions. Each session lasted 20 min. Rats were directly exposed to the conditions without any previous training. Two rats were assigned to Condition 1 (FT) and two to Condition 2 (VT).

Data Analysis

To have a complete representation of the behavioral system, we analyzed different dimensions and levels based on the record of spatial behavior in a bidimensional space at five frames/s; these are described below. Formal and computational descriptions of the measurements and methods of analysis are found in the Supplementary Material.

Analysis Between Subjects Within-Session

This level of analysis was conducted with representative subjects and thorough visual inspection of the data to identify changes in the spatial dynamics, moment to moment, related to water deliveries and the water dispenser location through the sessions. The measures and representations account for the changes in direction and variation of spatial behavior under the different experimental conditions (FT vs. VT). In addition, they allow depicting the evolution and process of the spatial behavior to compare the experiment's initial, intermediate, and final session. The specific analyses for this level were bidimensional routes and rat's location at the moment of water delivery per session; distance to the dispenser, moment to moment (five frames/s), and smoothed distance to the dispenser with a moving average of 200 frames (for a formal description, see Supplementary Material); and recurrence plot, depicting the change of regions of each rat in a matrixial configuration of 10 × 10 virtual zones (for a description, see Supplementary Material).

Analysis Between Subjects Throughout the Experiment

This level of analysis was conducted through visual inspection to identify the stability or variation of spatial behavior throughout the whole experiment. The used measures were entropy to indicate the variability of the organism's location and divergence to indicate the consistency or inconsistency in such variability between consecutive sessions (for a formal description of entropy and divergence, see Supplementary Material).

Analysis Between Conditions by Feature for All Sessions

This analysis and data representation level was conducted to identify the experimental condition's global effect on each spatial and discrete feature (FT vs. VT). The representation and analysis were based on measures of central tendency and variance. The analyzed features were traveled distance, entropy, divergence, maximum velocity, coincidence index, and mean distance to the dispenser. These features account for intensity, direction, and variation of behavior.

Multidimensional and Integrative Analysis Based on Machine Learning

The main level of analysis allows the integration of a complete comparison and representation of all features in a whole behavioral system. Ranking variables, t-SNE, and linear projection were conducted (for a formal description, see Supplementary Material) to identify the weight of each feature and the effect of the experimental condition on the multidimensional system as a whole. Data of all subjects and sessions were used.

Results

We conducted an integrative and multilevel analysis to characterize the behavioral continuum and compare the spatial dynamics of behavior under fixed and variable time schedules. First, we present representative within-subject results of the behavioral continuum within sessions for the first, intermediate, and last sessions of the experiment for one representative subject for each condition. Second, we show summary results for measures based on spatial behavior of first and non-first order (entropy and divergence) and a measure based on discrete responses (coincidence index). Third, we present a multidimensional analysis and integrative representation based on machine learning.

Figure 2 shows routes (Figure 2A), relative distance to the dispenser (Figure 2B), and recurrence plots (Figure 2C) for one rat under FT (Rat 1, left section) and one rat under VT (Rat 4, right section). Each column corresponds to a given session (1, 10, and 20). In Figure 2A, in a bidimensional representation of the MOFS, the routes of the rat (gray lines) for the whole session and the rat's location at the moment of water delivery (Location at Water Delivery, LWD) are presented (black dots). Three findings are worth mentioning: (1) routes were more extended under FT than with VT; (2) for both conditions, a progressive change in the direction of the routes was observed, toward the water-delivery zone, as the sessions progressed; and (3) the LWD gradually got closer to the water dispenser as the experimental sessions progressed. Figure 2B shows the relative value of the distance from the rat to the dispenser every 0.2 s (gray dots). In addition, to show the tendency of the distance function, we performed a smoothing of it (red line) by using a moving average of 200 frames (i.e., 40 s, see equation in the Supplementary Material). Values close to one indicate that the rat's distance to the dispenser was the maximum possible; values close to zero indicate that the rat was located close to the dispenser in a given time (frame). Under both conditions, a back-and-forth pattern was observed (gray dots), but this was more pronounced and had shorter periods under FT than in VT. In addition, the moving average (red line) suggests a tendency under both conditions to reduce the distance to the dispenser as the experiment progresses.

FIGURE 2

Figure 2. Representative within-subject results of the behavioral continuum for the initial (Session 1), intermediate (Session 10), and last session (Session 20) for one rat under FT (R1, left panels) and one rat under VT (R4, right panels). Each column depicts a session. (A) shows the routes of the rat (gray lines) and the rat's location at the moment of water delivery (black dots) in a bidimensional representation of the experimental chamber. (B) depicts the relative distance to the dispenser (Y-axis) moment to moment (X-axis). Finally, (C) shows recurrence plots. See the text for a complete description.

Finally, Figure 2C shows recurrence plots. This plot depicts the change of regions of each rat (in a matrixial configuration of 10 × 10 virtual zones) as the session progresses (see Supplementary Material). Both axes show time on a time frame of 0.2 s. If a rat was on an R_k region in a T time and T + n was in the same region, a black mark represents the recurrence in a given location. On the contrary, if on T + n the rat was on a different location, a white mark is shown. The densification and alternation of black–white checker patterns indicate high recurrence to a given region; a higher proportion of continuing black zones means higher permanence. A higher proportion of white zones means extended transitions among regions. Figure 2C shows a perspicuous difference between both conditions, higher recurrence under FT than VT, and higher permanence in zones under VT than FT.

Figure 3 shows the entropy (Figure 3A) and divergence (Figure 3B) values per session for each rat for both conditions. The entropy is helpful in the context of this experiment as a measure of the variation of locomotion patterns and the dynamic of behavior (see methodological and mathematical description in the Supplementary Material). For our subject matter, higher entropy represents high variation and dynamics of spatial behavior. In Figure 3A, the similarity between entropy plots within the condition and the difference between conditions are clear. The entropy was higher under FT than VT. A divergence index was calculated to determine the variations of spatial behavior between consecutive sessions (Figure 3B). This index was calculated by comparing the distribution of the organism's locomotion into the arena between two consecutive sessions (e.g., 1 and 2, 2 and 3, etc.). A value close to zero indicates no difference in the distribution of locomotion between sessions; a value far from zero indicates a difference in the distribution of locomotion between complete sessions (see mathematical description in Supplementary Material). Figure 3B shows that the divergence was lower and more stable under FT than VT; this implies more variation of spatial behavior between consecutive sessions under VT.

FIGURE 3

Figure 3. Entropy (A) and divergence (B) measures for subjects R1 and R2 under FT schedules (left) and subjects R3 and R4 under VT (right).

Figure 4 shows summary results of all sessions and subjects related to the spatial dimension of behavior. The traveled distance per session (Figure 4A), entropy (Figure 4B), and maximum velocity in a given frame per session (Figure 4D) were higher under FT than VT. In contrast, the divergence (Figure 4C) was significantly higher under VT than FT. On the other hand, the mean distance to the water dispenser per session (Figure 4F) was more dispersed under VT than FT and slightly higher, but the difference between both conditions was not robust. Finally, the coincidence index (Figure 4E) of the location of the organism in the dispenser zone (10 cm radius around to the dispenser allocation) at the time when water was available (3 s) was higher under FT than VT. The coincidence index is relevant because it is closely related to standard paradigms based on discrete and single response recording (e.g., entries to the dispenser).

FIGURE 4

Figure 4. Summary results of spatial-behavior measures for all sessions and subjects under FT and VT schedules. (A), traveled distance per session; (B), entropy; (C) divergence; (D), maximum velocity per session; (E), coincidence index; and (F) mean distance to the dispenser. Each box depicts the mean (dark blue vertical line), the median (yellow vertical line), the standard deviation (thin blue line), and the values between the first and the third quartile (blue highlighted area).

To identify the relevance of each dimension or variable in the emergent behavioral system, understanding it as the functional interdependent relationship between variables concerning each condition, we conducted a variable-ranking analysis based on machine learning. The variable ranking consists of ordering a set of features by the value of a scoring function (measuring the relevance of each feature) given a target as a predicting tool, in our case, the experimental condition. Variable ranking allows knowing the importance or relevance of the features that better explain a target variable (for a complete description, see Supplementary Material). Specifically, we used filter algorithms, the most used given the low computing resources used for applying them even on high-dimensional data sets. Given our subject matter, the integrative analysis of spatial dynamics of behavior with discrete responses under two different time-based schedules (FT vs. VT) and our data sets, we applied three theoretical information filter algorithms for single variable ranking, namely information gain, mean decrease impurity Gini index, and χ² (for a complete description concerning these algorithms, see Supplementary Material) for coincidence index, mean distance to the dispenser, traveled distance, maximum velocity, entropy, and divergence.

Figure 5 shows that, according to the variable-ranking procedures, the most relevant features were related to the spatial dimension of behavior. These were traveled distance, entropy, and divergence. On the other hand, the coincidence index, the most closely related variable to the measures of the standard paradigms based on discrete response recording, was less relevant than the other features related to the spatial dimension of behavior.

FIGURE 5

Figure 5. Ranking variable analysis, under information gain, mean decrease impurity Gini index, and χ² procedures for the features traveled distance per session (Trav Dist), entropy, divergence, maximum velocity per session (Max Vel), coincidence index (Coinc index), mean distance to the dispenser (Mean Dist to Disp), and session. The length of the bar ranks the features by the scoring value that measures the relevance of each feature to differentiate the behavioral system as a function of a given time-based schedule (FT vs. VT).

To have a perspicuous representation that would allow identifying if the data, given its multiple dimensions, are articulated or grouped as a function of the kind of time-based schedule employed, we conducted t-SNE. t-SNE is a machine learning algorithm for the visualization of high-dimensional data sets into a bidimensional or three-dimensional space. t-SNE performs a non-linear dimensionality reduction task for embedding data sets and obtaining low-dimension transformations as a result. Relationships between high-dimensional data, which might be impossible to observe due to a considerable amount of variables, could be distinguished after transforming them into a space with reduced dimension by t-SNE. Furthermore, the representation obtained by t-SNE is perspicuous because the data with similar values are closer to each other (in a low-dimensional space, 2-D or 3-D) than data with dissimilar values (for a complete description of t-SNE, see Supplementary Material).

Figure 6 shows a representation, using t-SNE, for the data of all experimental sessions and subjects. Each point can be seen as multidimensional data for a session, considering the distance traveled, entropy, divergence, maximum velocity, coincidence index, mean distance to the dispenser, and session number into the experiment as dimensions with the condition, FT vs. VT, as a target feature. FT data (blue points) tend to be closer to each other, and the same was observed concerning VT data (red points). Color regions are shown in the figure to facilitate the visualization of groupings. The conformation of only two predominant and well-delimitated regions is clear, and FT and VT data are separated except for a few dots inserted in the colored region of the opposite condition. Given that the multidimensional space of t-SNE could be seen as behavioral system representation as a whole, the main finding is that a well-differentiated behavioral system under each condition or schedule emerged.

FIGURE 6

Figure 6. Representation with t-SNE for the data of all experimental sessions and subjects. Each point represents multidimensional data for a session, considering the distance traveled, entropy, divergence, maximum velocity, coincidence index, mean distance to the dispenser, and session number into the experiment as dimensions with time-based schedule, FT and VT, as a target feature. The data with similar values given the multiple features or input variables taken as a whole is simply closer to each other than data with dissimilar values.

Figure 7 shows the linear projection (for a full description, see the Supplementary Material) of the multidimensional data of all sessions and subjects for six dimensions: coincidence index, divergence, mean distance to the water dispenser, entropy, traveled distance, and maximum velocity. The direction of each vector points out the direction of the increasing values for a given dimension. Colored regions related to each condition are added to facilitate the visualization of the data tendency. The prevalence of a colored region in a given dimension represents higher values for the correspondent experimental condition to such color compared with the other experimental condition. Thus, the linear projection shows higher maximum velocity, traveled distance, and entropy values under FT than VT. Although the values nearby for mean distance to the dispenser for both conditions are shown, a red shadow suggests higher values for VT.

FIGURE 7

Figure 7. Representation with linear projection by principal component analysis of multidimensional data of all sessions and subjects for six dimensions: coincidence index, divergence, mean distance to the water dispenser, entropy, traveled distance, and maximum velocity with time based-schedule, FT and VT, as a target feature. The direction of each vector points out the direction to increasing values for a given dimension.

On the other hand, the projection clearly shows higher divergence values for VT and nearby values for the coincidence index for both conditions; nevertheless, the blue shadow in this last vector points out higher values for FT. One relevant difference of this representation with other reduction dimension procedures is that it has specific representations for each relevant dimension of the data in the orthogonal space. The linear projection representation confirms the relevance of the spatial dimensions of behavior related to the programmed time-based schedules.

Example 2

Motivational Operations: Behavioral Dynamics Under Different Deprivations in Concurrent Schedules

A procedure that is conducted in most studies of the experimental analysis of behavior that uses “appetitive” stimulus is the deprivation of a given commodity (e.g., water or food) that it is later used to be delivered contingent to some response or behavioral pattern in contingent schedules (Skinner, 1938). In non-contingent schedules, the delivery of such commodity is presented in conjunction with given stimuli (e.g., Pavlovian conditioning procedures) or simply presented according to a specific time rule (e.g., time-based schedules). The deprivation procedure, in methodological terms, has the purpose of establishing the relevance or dispositional value of the delivered commodity as a stimulus (Reberg et al., 1978; Michael, 1982). This dispositional value is crucial to explaining behavioral systems, seen as articulating behavioral patterns, responses, and other stimuli. The study of the relevance or dispositional value of a given stimulus, por mor of the deprivation operation, is related to the field of motivation, under the terms of “motivational operations” (Laraway et al., 2003) and “establishment operations” (Michael, 1982, 1993), among others.

Several years ago, it was established that the “motivational” function of a given stimulus could be characterized based on direction, intensity, and variation of the spatial behavior (Duffy, 1951, 1957; De Valois, 1954; Berlyne, 1955; Schneirla, 1959; Maier and Schneirla, 1964). Nonetheless, this characterization is restricted only to the rate response of the discrete responses under the single response paradigm. Thus, the spatial dimension of the behavior and its dynamics is ignored in the contemporary study of EAB (e.g., Lewon et al., 2019).

On the other hand, generally, the effect of food or water deprivation is evaluated by removing access to them by the experimental subjects outside the experimental sessions and then presenting one or the other, either contingent or non-contingent, to a given response (Skinner, 1938; Bolles, 1975) during the session. The effect of presenting food and water concurrently when subjects are food- or water-deprived is less studied (Fallon et al., 1965, Lewon et al., 2019). Under this rationale, the objective of this study was to evaluate the effects of food- and water-deprivation conditions on the behavioral continuum in conditions in which food and water are concurrently delivered. A multidimensional and multilevel analysis and data representation was conducted using machine learning tools to integrate standard discrete responses and spatial dynamics.

Method

Subjects

Five female and one male (Subject 3) Wistar rats (5 months old) were used. According to the current phase of the experiment, rats were housed in individual home cages and placed on a water- or food-deprivation schedule for 22 h before every experimental session. All procedures were conducted according to university regulations of animal use and care and followed the official Mexican norm NOM-062-ZOO-1999 for Technical Specification for Production, Use, and Care of Laboratory Animals.

Apparatus

An experimental chamber of 92 wide × 92 long × 33 cm high was used; 2 cm above the grid floor and in the center of the opposing walls, two dispensers were located: a liquid dipper (Coulbourn E14-05) and a modified food receptacle with a pellet dispenser (Coulbourn E14-24). The dipper allowed access to 0.1 cc of water for 3 s, and the pellet dispenser delivered a 45-mg pellet with limited availability of 3 s. Entries to both dispensers were detected by head entry detectors (MED ENV-254-CB). In addition, above both dispensers, a yellow light was used as a visual stimulus (MED ENV-222M) to indicate food or water delivery (for a diagram of the apparatus, see Hernández et al., 2021).

Water and food deliveries were programmed and registered with the Software MED PC IV, and head entries were also registered using this software. Rat displacement in the experimental chamber was recorded using a video camera (Topica TP-505D/3), 1 m above the chamber. The video camera was connected to a PC with software Ethovision 2.3. With this software were obtained records of rat displacement in X, Y coordinates every 0.2 s.

Procedure

Experimental Phase

After an initial training phase to the food and water dispenser (see Hernández et al. (2021) for a complete description), subjects were exposed to two deprivation conditions: (1) water deprivation (WD) and (2) food deprivation (FD). Each deprivation consisted of 3 days with the corresponding food or water restriction and one experimental session per day. After each condition, subjects were allowed unrestricted access to both commodities for 24 h before the following deprivation condition to avoid a drastic decrease in weight and to separate the effect of each deprivation (Lewon et al., 2019). Subjects were assigned to one of two deprivation sequences to control for the potential effect of the first deprivation condition on the following condition. The specific order of deprivations for each sequence is shown in Table 1. All experimental sessions consisted of presenting a CONC FT 30 s FT 30 s schedule of food and water with limited availability of 3 s. A yellow light above both dispensers was turned on with every delivery and remained during food or water availability. All experimental sessions lasted 30 min.

TABLE 1

Table 1. Sequence of deprivation conditions for each group.

Data Analysis

The same analytical approach as in Experiment 1 was used in Experiment 2 only with appropriate settings due to the differences in methods, apparatus, and records. Specifically, measures related to water and food dispensers were added, such as entrances and derivated measures (intensity, precision to dispensers, and proportion to commodities contacted). Each measure is described in the following section.

Results

We conducted a multilevel analysis to characterize the behavioral continuum and compare the spatial dynamics of behavior under water and food deprivation. In the same way as in Example 1, first, we show representative within-subject results of the behavioral continuum within the session, for the initial, intermediate, and last sessions of the experiment for one representative subject. Second, we show summary results for measures based on spatial behavior, first and non-first order, and measures based on discrete responses in water and food dispensers. Third, we conducted a multidimensional analysis and integrative representations based on machine learning.

Figure 8 shows the continuum spatial–behavioral data for the complete initial, intermediate, and final sessions for a representative experimental subject. Figure 8A represents the routes of the subject. There was a higher variation of spatial behavior at the arena under WD than FD for the intermediate and final sessions. In addition, higher spatial behavior in the food dispenser zone (top of the plot) under FD and a distributed densification between both dispenser zones (top and down of the plot) under WD were observed. Figure 8B shows the distance to the food dispenser (red line) and water dispenser (blue line), moment to moment (each .02 s). For the intermediate and final sessions under FD, a small distance to the food dispenser and a long distance to the water dispenser was observed with only a few alternations between high- and low-distance values to both dispensers.

FIGURE 8

Figure 8. Representative within-subject results for all sessions of the behavioral continuum for one rat (R5). Each column depicts a session, and rows depict deprivation conditions (FD and WD, respectively). (A) shows the routes of the rat in a bidimensional representation of the experimental chamber; red circles represent food dispenser location, and blue circles represent water dispenser location. (B) depicts the distance to the dispensers (Y-axis) moment to moment (X-axis); the red line represents the distance to food dispenser; the blue line represents the distance to water dispenser. Finally, (C) shows recurrence plots; this changes regions of each rat (in a matrixial configuration of 10 × 10 virtual zones) as the session progresses.

In contrast, under WD, for the intermediate and final sessions, a significant alternation between high and low values of distance to the dispensers and then a clear back and forth pattern between dispensers was observed. Figure 8C depicts the recurrence plots (see Figure 2 description; Supplementary Material). High permanence (extended black zones) with only some transitions under FD and high recurrence (black–white mosaic patterns) under WD were observed in these plots. Thus, the three panels (Figures 8A–C) as a whole suggest a robust difference in the spatial dynamics by FD vs. WD under the same concurrent schedule and for the same experimental subject.

Figure 9 shows summary results for the measures based on the spatial dimension of behavior: mean distance to the food dispenser, mean distance to the water dispenser, mean distance to the center of the experimental arena, entropy index, and divergence index, under FD and WD for all sessions and experimental subjects independently of the sequence in which they were exposed. All measures were sensitive to the deprivation condition except the divergence index. The distance to the food dispenser had low values under FD and relatively high values under WD. In contrast, the opposite effect concerning the water dispenser was observed with relatively low distance values under WD and high values under FD. The data of distance to the dispensers was more spread under WD than under FD. The mean distance to the center was higher under WD than FD, and the entropy index too. All the previously mentioned findings were robust and point out a significant differential spatial dynamic of the behavior related to deprivation conditions.

FIGURE 9

Figure 9. Summary results of spatial-behavior measures for all sessions and subjects under FD and WD schedules. (A), Distance to food dispenser; (B), Distance to water dispenser; (C), Distance to center; (D), Entropy; (E), Divergence. Each box depicts the mean (dark blue vertical line), the median (yellow vertical line), the standard deviation (thin blue line), and the values between the first and the third quartile (blue highlighted area).

Figure 10 shows summary results for measures based on discrete responses, namely, (a) intensity to food dispenser index, (b) intensity to water dispenser index, (c) precision to food dispenser index, (d) precision to water dispenser index, (e) proportion to food contacted, and (f) proportion of water contacted. These measures are relevant because they are related to those employed in the standard paradigms based on single discrete responses. Figures 10A,B depict the intensity related to the dispensers; the intensity index was obtained by dividing the total number of head entries in one dispenser of each session by the maximum number of entries in any session on the whole experiment for a particular subject and dispenser. Then, this procedure was carried on for each subject and session, always using this within-subject and -dispenser comparison. The intensity to the food dispenser was high under FD and low under WD, and the intensity to the water dispenser was high, though spread, under WD and very low under FD. Figures 10C,D show the precision to the dispensers. Precision index was obtained by dividing the number of entries to each dispenser, when the commodity was available (water or food), by the total number of head entries in that session and in that dispenser. The data below percentile 5 for each subject on each dispenser were eliminated to palliate possible ceiling or floor effects in sessions. The precision related to the food dispenser was lower under FD than under WD, and there was no difference in precision to water dispenser between deprivations (FD and WD). Finally, Figures 10E,F depict the proportion to food and water contacted from the total available. The proportion to food contacted was higher under FD than WD, and water contacted was lower under FD than WD. The measures based on discrete responses as a whole show interesting findings. First, the intensity and effectiveness (i.e., proportion to food contacted) to each dispenser clearly depend on the deprivation condition; that is, high values of intensity and effectiveness to the food dispenser were observed under FD and vice versa, and high values of intensity and effectiveness to the water dispenser were observed under WD.

FIGURE 10

Figure 10. Summary results of measures based on discrete responses for all sessions and subjects under FD and WD schedules: Intensity to food dispenser index (A), intensity to water dispenser index (B), Precision to food dispenser index (C), Precision to water dispenser index (D), Proportion of food contacted (E), and Proportion of water contacted (F). Each box depicts the mean (dark blue vertical line), the median (yellow vertical line), the standard deviation (thin blue line), and the values between the first and the third quartile (blue highlighted area).

Nevertheless, the modulating effect of each deprivation condition over the behavior related to the correspondent dispenser and commodity delivery to such deprivation is not precisely the same for both deprivations. On the one hand, the modulating effect of FD for all behavioral measures based on discrete responses, is most robust for the food dispenser than the effect of WD over the same measures related to the water dispenser. On the other hand, the data under FD tends to extreme values (very low or very high) related to intensity and effectiveness, and under WD tends to intermediate and spread values. Finally, it is remarkable that the precision of behavior related to food delivery was negatively affected under FD. This effect for water delivery under WD was not observed.

Figure 11 shows the variable-ranking under three procedures, namely, information gain, impurity Gini index, and χ² for all measures, those based on continuum spatial dimension of behavior, and those based on discrete responses (for a complete description concerning these algorithms, see the Supplementary Material, and for suggested use in behavior analysis, see the description of Figure 5). The variable ranking allows identifying the relevance of each variable (i.e., behavioral measure or dimension) into the whole multidimensional system. Given our subject matter, the modulating effect of two different deprivations (FD and WD) into the multidimensional behavioral system, the variable ranking suggests that the most relevant variables were the distance to the dispensers, intensity to the dispensers, and entropy. These findings suggest that the spatial dimension of behavior was as relevant as discrete responses into the behavioral systems that emerge by modulation under FD and WD.

FIGURE 11

Figure 11. Ranking variable analysis, under information gain, mean decrease impurity Gini index, and χ² procedures for the features Distance to food dispenser (Dist FDisp), distance to water dispenser (Dist WDisp), intensity to food dispenser (Intensity FDispenser), Proportion of food contacted (Proportion FContacted), Entropy, Intensity to water dispenser (Intensity WDisp), Proportion of water contacted (Proportion WContacted), Precision to water dispenser (Precision WDisp), distance to the center of the experimental arena (Dist Center), Precision to food dispenser (Precision FDisp), Divergence, Session, Sequence.

Figure 12 shows a representation by t-SNE (for a complete explanation, see Supplementary Material and description of Figure 6) for the data of all the experimental sessions and subjects. Each point can be seen as multidimensional data for a session, considering both measures based on spatial behavior continuum and discrete responses (see Figure 11) and deprivation condition (FD and WD) as the target feature. Data tends to be closer by condition for both FD (red dots) and WD (blue dots). Two well-delimitated colored regions were formed with a clear separation between FD and WD data. Additionally, two clusters were conformed under the K-means clustering procedure (for a description, see Supplementary Material); see Figure 12A. The coincidence of each cluster data with deprivation condition data was very robust. This analysis can be taken as an explicit confirmation of the differential modulation by each deprivation condition (FD vs. WD) over the emerged multidimensional behavioral system under the same concurrent schedule and within-subject design. Finally, in Figure 12B, another complementary representation was conducted. In this panel, the circles point out deprivation sequence 1 (WD-FD), and the crosses sign out deprivation sequence 2 (FD-WD). The representation shows that the data clustering by deprivation was robust to the sequences (that is, there are well-delimited regions for each deprivation regardless of the sequence) and also suggests that the data tend to be close by sequence within deprivation. In addition, the representation suggests a contrast effect on deprivation conditions by sequence 1; that is, the FD and WD data were more distant from each other than the data for the same deprivation conditions in sequence 2.

FIGURE 12

Figure 12. Representation with t-SNE for the data of all experimental sessions and subjects. Each point represents multidimensional data for a session given all features (see Figure 11) with deprivation condition, FD (red) and WD (blue), as a target feature. The data with similar values given the multiple features or input variables taken as a whole is simply closer to each other than data with dissimilar values. (A) additionally shows clustering under the K-means procedure and two well-delimitated colored regions with a clear separation between FD and WD data and corresponding each one with a cluster (C1 circles and C2 crosses). On the other hand, (B) depicts the deprivation sequence (Sequence 1 circle and Sequence 2 crosses).

Discussion

The purpose of the present work was three-fold: (1) to propose an integrative and multidimensional approach for the analysis of behavioral systems; (2) to show novel behavioral aspects revealed under a multidimensional approach based on the integration of discrete and continuous data assisted by machine learning tools; and (3) to provide relevant and novel behavioral measures and data representations based on the integration of spatial dynamics and discrete responses for the study of behavioral systems related to relevant research areas in behavioral science, such as water-seeking behavior and motivational operations.

In the first example, concerning behavioral dynamics under FT and VT, marked differences in routes, rat's location at the moment of water delivery, distance to the dispenser, back and forth to the dispenser, and recurrence patterns were observed. These findings suggest a considerable difference in emergent spatial behavior (direction and variation) under both temporal schedules (FT vs. VT). In addition, they confirm our hypothesis that the proposed first-order measures based on spatial behavior are sensitive to EAB paradigmatic procedures.

Furthermore, the entropy, a non-first order measure, was sensitive to the programmed contingencies with higher values under FT than VT; the behavioral meaning of this finding is that the distribution of the organism location presents more variability under FT than VT. This finding is interesting because the temporal variation in water delivery is associated with lower variability of organism location, and temporal constancy or fixation is associated with higher variation of the organism location. On the other hand, the organism's location variability distribution shows a low divergence between sessions under FT and not under VT. As far as we know, the use of entropy and divergence to characterize the spatial variability of behavior is scarce. Nevertheless, our findings reveal that entropy and divergence are embedded features of spatial behavior with a higher sensitivity to the temporal schedules.

The findings together, of first and non–first order measures with data of direction of behavior, such as routes and back and forth patterns, and others about the variation of behavior, such as checker recurrence patterns and higher values of entropy under FT, can be seen as an objective measure of the idiosyncratic spatial patterns reported anecdotally in the literature as superstition behavior under FT (Skinner, 1948). Thus, in our perspective, the analysis carried out and its behavioral meaning shows the plausibility and parsimony of the CEAB approach.

On the other hand, the analysis of the different features and its ranking variable assisted by machine learning confirms our hypothesis related to the relevance of spatial features over standard discrete responses (e.g., water contacted or coincidence index) in the behavioral systems under temporal schedules. The ranking variable analysis shows that traveled distance, entropy, divergence, and maximum velocity are more sensitive to the programmed schedules than the standard feature of water contacted, measured as a coincidence index. Finally, t-SNE and linear projection were helpful to represent multidimensional behavioral systems in a perspicuous way (e.g., in bidimensional space). These representations allow confirmation that each schedule (FT and VT) gives place to well-differentiated behavioral systems based on spatial behavior and a discrete response.

In the second example, concerning behavioral dynamics in concurrent schedules under different deprivation conditions (WD vs. FD), more extended routes, back and forth patterns alternated between dispensers, and recurrence patterns were observed under WD than FD. Again, findings suggest these representations were sensitive to the deprivation condition.

Furthermore, all measures related to spatial behavior were markedly affected by deprivation conditions (e.g., distance to both dispensers, distance to the center of the arena, and entropy) except divergence. The latter indicates consistency between sessions related to the variability values of the organism's location distribution under both deprivations. As in Example 1, the findings show that entropy is a relevant feature embedded in spatial behavior that is significantly affected by a standard procedure with well-known effects. In simple words, the findings suggest spatial behavior is very sensitive to, namely, “motivational operations” under choice situations (e.g., concurrent schedules). A relevant aspect is that these features are indicators of direction and variability of the behavior that could be used as an alternative indicator to identify the motivational function of a given procedure aside from discrete responses.

On the other hand, all measures based on discrete responses were sensitive to deprivation conditions except the precision to the water dispenser. In general terms, each deprivation condition affected the direction of behavior, both spatial and discrete responses, to correspondent commodity although the effect was not exactly symmetrical. These findings are consistent with the expected under the standard paradigm and the literature. This point is crucial because it increases the validity of our findings and conclusions concerning spatial behavior (as a simile of concurrent validity in a non-statistical way).

The ranking variable analysis, assisted by machine learning, considered 11 features (five based on spatial behavior and six on discrete responses). It reveals that three are related to the spatial behavior of the five most relevant features: distance to the food dispenser, distance to the water dispenser, and entropy. These findings confirm our hypothesis related to CEAB revealing that spatial features are at least as relevant as behavioral features based on discrete responses, but now concerning other behavioral phenomena and paradigms, “motivation” and “motivational operations under concurrent schedules,” respectively. Finally, t-SNE shows that each deprivation condition gives well-differentiated behavioral systems based on spatial behavior and a discrete response under the same concurrent schedules.

As our examples show, the general proposed approach in this work helps integrate a multiple-level analysis to coalesce discrete and continuous dimensions of behavior (and derivate first and non-first order measures) as a whole system. It also proved fruitful to provide a broad characterization of the continuum of behavior in which the spatial dynamics are on the first plane. The proposed approach appears promising to characterize and integrate different behavioral features as a whole behavioral system, pointed to as relevant throughout the development of behavioral science. Among these features are direction (Schneirla, 1959; in our work, distance to the dispenser), intensity (Duffy, 1957; in our work, speed, acceleration), variation (Mowrer and Jones, 1943; Antonitis, 1951; Berlyne, 1955; Iversen, 2017; in our work, entropy), preference (Irwin, 1958; in our work, time spent in a given zone), persistence (Bolles, 1975; in our work, dispenser entries). With the proposed multidisciplinary methodological approach, the purpose of overcoming the segmented characterization of the behavioral continuum and its derived paradigms, for example, the single response paradigm (Henton and Iversen, 1978), could go beyond the theoretical level that has been maintained up to now (Kantor, 1958).

The findings of both experiments presented to exemplify our approach show that the recording and analysis of the continuum of spatial behavior of the organisms is of primary importance to account for the principles that underlie behavioral systems and suggest that (a) moment-to-moment analysis and representations of locomotion-based data across complete sessions are helpful to identify and characterize the behavioral dynamics under different stimuli schedules and deprivation conditions (see routes, distance to the dispensers, and recurrence plots); (b) the proposed non-first order variables (i.e., entropy and divergence) based on locomotion data are relevant and sensible to stimuli schedule and deprivation conditions; (c) the variables based on locomotion-data could be more sensible than variables based on discrete responses to stimuli schedules and deprivation conditions (see variable-ranking analysis based on machine learning); (d) discrete responses and the continuum of spatial behavior comprise a unitary and whole system that could apprehend and represent in a perspicuous way with machine learning tools, such as t-SNE, clustering based on K-means, and linear projection, among others.

Our examples and findings suggest that the proposed multidisciplinary approach (CEAB) allows going forward on explaining behavioral systems and reveals an integration of spatial dynamics and discrete responses hidden until now for the behavioral science. In addition, new empirical relations and insights are revealed under CEAB related to water-seeking behavior (León et al., 2020a) and motivational operations (Michael, 1982, 1993; Hernández et al., 2021).

Although the proposed approach appears to be promising, to confirm its heuristic and parsimonious value, it should be evaluated under (a) other relevant phenomena; (b) other kinds of schedules (e.g., contingent schedules); (c) different stimulating conditions (e.g., aversive stimulation); (d) different organization of behavior (e.g., behavior under stimulus control, relational behavior); (e) different species, including humans.

Finally, the proposed approach could be strengthened by integrating additional first and non-first order measures pertinent to apprehend and characterize the dynamics of relevant dimensions of behavior (such as direction, variation, and vigor, among others). On the other hand, additional artificial intelligence tools, such as predictive analysis, could be explored to extend the scope of our approach for behavioral science, specifically for the experimental analysis of behavior.

As a corollary, the fast-paced development of contemporary computational tools of fields such as artificial intelligence has rapidly changed the landscape of some fields of behavioral science in the last decades, for example, ethology (Dell et al., 2014) and neuroscience (Wiltschko et al., 2015; Datta et al., 2019; Mathis et al., 2019; Mathis and Mathis, 2020). It is time the non-mediational, systematic, parametric (Skinner, 1938; Schoenfeld and Cole, 1972; Henton and Iversen, 1978), and ecological (Timberlake, 1994; Silva and Timberlake, 1997) approaches in the experimental analysis of behavior start to profit from these tools (Turgeon and Lanovaz, 2020).

Data Availability Statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Ethics Statement

The studies were reviewed and approved by The Animal Ethics Committee at CEICAH-Universidad Veracruzana. The original studies from which the data was obtained were conducted following the Mexican Norm in agreement with the university regulations of animal use and care and followed the official Mexican norm NOM-062-ZOO-1999 for Technical Specification for Production, Use and Care of Laboratory Animals. The experimental subjects were not exposed to aversive stimulation, stressful situations, or invasive procedures. We also used data from different studies conducted in the laboratory to propose a new analytic approach, because of this, no additional ethical approval was needed from our institution to analyze the data.

Author Contributions

AL and VH: conceptualization, data curation, formal analysis, investigation, methodology, project administration, resources, software, supervision, visualization, roles/writing—original draft, and writing—review and editing. JL, PT, MA-G, CH-L, and EE: data curation and formal analysis. IG and VQ: conducted experimental sessions and data curation. All authors contributed to the article and approved the submitted version.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher's Note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary Material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fnbeh.2021.681771/full#supplementary-material

References

Antonitis, J. J. (1951). Response variability in the white rat during conditioning, extinction, and reconditioning. J. Exp. Psychol. 42, 273–281. doi: 10.1037/h0060407

PubMed Abstract | CrossRef Full Text | Google Scholar

Baum, W. M., and Rachlin, H. C. (1969). Choice as time allocation 1. J. Exp. Anal. Behav. 12, 861–874. doi: 10.1901/jeab.1969.12-861

PubMed Abstract | CrossRef Full Text | Google Scholar

Berlyne, D. E. (1955). The arousal and satiation of perceptual curiosity in the rat. J. Comp. Physiol. Psychol. 48:238. doi: 10.1037/h0042968

PubMed Abstract | CrossRef Full Text | Google Scholar

Bolles, R. C. (1975). Theory of Motivation. New York, NY: HarperCollins Publishers.

Boren, J. J., Moerschbaecher, J. M., and Whyte, A. A. (1978). Variability of response location on fixed-ratio and fixed-interval schedules of reinforcement. J. Exp. Anal. Behav. 30, 63–67. doi: 10.1901/jeab.1978.30-63

PubMed Abstract | CrossRef Full Text | Google Scholar

Datta, S. R. (2019). Q&A: understanding the composition of behavior. BMC Biol. 17, 1–7. doi: 10.1186/s12915-019-0663-3

PubMed Abstract | CrossRef Full Text | Google Scholar

Datta, S. R., Anderson, D. J., Branson, K., Perona, P., and Leifer, A. (2019). Computational neuroethology: a call to action. Neuron 104, 11–24. doi: 10.1016/j.neuron.2019.09.038

PubMed Abstract | CrossRef Full Text | Google Scholar

De Valois, R. L. (1954). The relation of different levels and kinds of motivation to variability of behavior. J. Exp. Psychol. 47:392. doi: 10.1037/h0054621

PubMed Abstract | CrossRef Full Text | Google Scholar

Dell, A. I., Bender, J. A., Branson, K., Couzin, I. D., de Polavieja, G. G., Noldus, L. P., et al. (2014). Automated image-based tracking and its application in ecology. Trends Ecol. Evol. 29, 417–428. doi: 10.1016/j.tree.2014.05.004

PubMed Abstract | CrossRef Full Text | Google Scholar

Drew, M. R., Zupan, B., Cooke, A., Couvillon, P. A., and Balsam, P. D. (2005). Temporal control of conditioned responding in goldfish. J. Exp. Psychol. 31, 31–39. doi: 10.1037/0097-7403.31.1.31

PubMed Abstract | CrossRef Full Text | Google Scholar

Duffy, E. (1951). The concept of energy mobilization. Psychol. Rev. 58:30. doi: 10.1037/h0054220

PubMed Abstract | CrossRef Full Text | Google Scholar

Duffy, E. (1957). The psychological significance of the concept of“ arousal” or“ activation.” Psychol. Rev. 64, 265–275. doi: 10.1037/h0048837

PubMed Abstract | CrossRef Full Text | Google Scholar

Elliott, M. H. (1934). The effect of hunger on variability of performance. Am. J. Psychol. 46, 107–112. doi: 10.2307/1416238

CrossRef Full Text | Google Scholar

Escobar, R. (2014). From relays to microcontrollers: the adoption of technology in operant research. Rev. Mex. Anal. Conducta 40, 127–153. doi: 10.5514/rmac.v40.i2.63673

CrossRef Full Text | Google Scholar

Fallon, D., Thompson, D. M., and Schild, M. E. (1965). Concurrent food-and water-reinforced responding under food, water, and food-plus-water deprivation. Psychol. Rep. 16, 1305–1311. doi: 10.2466/pr0.1965.16.3c.1305

CrossRef Full Text | Google Scholar

Ferster, C. B., and Skinner, B. F. (1957). Schedules of Reinforcement. New York, NY: Appleton-Century-Crofts. doi: 10.1037/10627-000

CrossRef Full Text | Google Scholar

Gibson, J. J. (1979). The Ecological Approach to Visual Perception. Boston, MA: Mifflin and Company.

Google Scholar

Henton, W. W., and Iversen, I. H. (1978). Classical and Operant Conditioning: A Response Pattern Analysis. New York, NY: Springer-Verlag New York. doi: 10.1007/978-1-4612-6310-4

CrossRef Full Text | Google Scholar

Hernández, V., León, A., and Quintero, V. (2021). Food and water deprivation modulating function on activation and direction of behavior in Wistar rats. PsyArXiv. doi: 10.31234/osf.io/234sh

CrossRef Full Text | Google Scholar

Irwin, F. W. (1958). An analysis of the concepts of discrimination and preference. Am. J. Psychol. 71, 152–163. doi: 10.2307/1419203

PubMed Abstract | CrossRef Full Text | Google Scholar

Iversen, I. H. (2017). An inexpensive method to study response variability in acquisition and extinction of operant behavior. Rev. Mex. Anal. Conducta 43, 212–241. doi: 10.5514/rmac.v43.i2.62314

CrossRef Full Text | Google Scholar

Kantor, J. R. (1958). Interbehavioral Psychology: A Sample of Scientific System Construction. Bloomington, IN: Principia Press. doi: 10.1037/13165-000

CrossRef Full Text | Google Scholar

Kantor, J. R. (1970). An analysis of the experimental analysis of behavior (TEAB). J. Exp. Anal. Behav. 13:101. doi: 10.1901/jeab.1970.13-101

PubMed Abstract | CrossRef Full Text | Google Scholar

Kuo, Z. Y. (1976). The Dynamics of Behavior Development: An Epigenetic View. New York, NY: Random House.

Google Scholar

Lachter, G. D., Cole, B. K., and Schoenfeld, W. N. (1971). Response rate under varying frequency of non-contingent reinforcement. J. Exp. Anal. Behav. 15, 233–236. doi: 10.1901/jeab.1971.15-233

PubMed Abstract | CrossRef Full Text | Google Scholar

Laraway, S., Snycerski, S., Michael, J., and Poling, A. (2003). Motivating operations and terms to describe them: some further refinements. J. Appl. Behav. Anal. 36, 407–414. doi: 10.1901/jaba.2003.36-407

PubMed Abstract | CrossRef Full Text | Google Scholar

León, A., Hernandez, A. V., Huerta, U., Hernandez, C. A., Toledo, P., Avendaño, M. L., et al. (2020a). Ecological location of a water source and spatial dynamics of behavior under temporally scheduled water deliveries in a modified open field system: an integrative approach. Front. Psychol. 11:3477. doi: 10.3389/fpsyg.2020.577903

PubMed Abstract | CrossRef Full Text | Google Scholar

León, A., Tamayo Tamayo, J., Hernández Eslava, V., Toledo Hernández, P., Avendaño Garrido, M. L., Hernández Linares, C. A., et al. (2020b). Motus: software for the behavioral analysis of displacement patterns. Mex. J. Behav. Anal. 46, 222–242. doi: 10.5514/rmac.v46.i1.76960

CrossRef Full Text

Lewon, M., Spurlock, E. D., Peters, C. M., and Hayes, L. J. (2019). Interactions between the effects of food and water motivating operations on food-and water-reinforced responding in mice. J. Exp. Anal. Behav. 111, 493–507. doi: 10.1002/jeab.522

PubMed Abstract | CrossRef Full Text | Google Scholar

Loveless, J., and Webb, B. (2021). Chaotic worms. Nat. Phys. 17, 170–171. doi: 10.1038/s41567-020-01058-2

CrossRef Full Text | Google Scholar

Maekawa, T., Ohara, K., Zhang, Y., Fukutomi, M., Matsumoto, S., Matsumura, K., et al. (2020). Deep learning-assisted comparative analysis of animal trajectories with DeepHL. Nat. Commun. 11, 1–15. doi: 10.1038/s41467-020-19105-0

PubMed Abstract | CrossRef Full Text | Google Scholar

Maier, N., and Schneirla, T. (1964). Principles of Animal Psychology. New York, NY: Dover Publications Inc.

Google Scholar

Marshall, J. D., Aldarondo, D. E., Dunn, T. W., Wang, W. L., Berman, G. J., and Ölveczky, B. P. (2020). Continuous whole-body 3D kinematic recordings across the rodent behavioral repertoire. Neuron 109, 420–437.e8. doi: 10.1016/j.neuron.2020.11.016

PubMed Abstract | CrossRef Full Text | Google Scholar

Mathis, A., Mamidanna, P., Cury, K. M., Abe, T., Murthy, V. N., Mathis, M. W., et al. (2018). DeepLabCut: markerless pose estimation of user-defined body parts with deep learning. Nat. Neurosci. 21, 1281–1289. doi: 10.1038/s41593-018-0209-y

PubMed Abstract | CrossRef Full Text | Google Scholar

Mathis, A., Pack, A. R., Maeda, R. S., and McDougle, S. D. (2019). Highlights from the 29th annual meeting of the society for the neural control of movement. J. Neurophysiol. 122, 1777–1783. doi: 10.1152/jn.00484.2019

PubMed Abstract | CrossRef Full Text | Google Scholar

Mathis, M. W., and Mathis, A. (2020). Deep learning tools for the measurement of animal behavior in neuroscience. Curr. Opin. Neurobiol. 60, 1–11. doi: 10.1016/j.conb.2019.10.008

PubMed Abstract | CrossRef Full Text | Google Scholar

Menaker, T., Zamansky, A., van der Linden, D., Kaplun, D., Sinitica, A., Karl, S., et al. (2020). “Towards a methodology for data-driven automatic analysis of animal behavioral patterns,” in Proceedings of the Seventh International Conference on Animal-Computer Interaction (New York, NY), 1–6. doi: 10.1145/3446002.3446126

CrossRef Full Text | Google Scholar

Michael, J. (1982). Distinguishing between discriminative and motivational functions of stimuli. J. Exp. Anal. Behav. 37, 149–155. doi: 10.1901/jeab.1982.37-149

PubMed Abstract | CrossRef Full Text | Google Scholar

Michael, J. (1993). Establishing operations. Behav. Anal. 16, 191–206. doi: 10.1007/BF03392623

PubMed Abstract | CrossRef Full Text | Google Scholar

Mowrer, O. H., and Jones, H. M. (1943). Extinction and behavior variability as functions of effortfulness of task. J. Exp. Psychol. 33, 369–386. doi: 10.1037/h0056182

CrossRef Full Text | Google Scholar

Pear, J. (1985). Spatiotemporal patterns of behavior produced by variable interval schedules of reinforcement. J. Exp. Anal. Behav. 44, 217–231. doi: 10.1901/jeab.1985.44-217

PubMed Abstract | CrossRef Full Text | Google Scholar

Pérez-Escudero, A., Vicente-Page, J., Hinz, R., Arganda, S., and de Polavieja, G. G. (2014). idTracker: tracking individuals in a group by automatic identification of unmarked animals. Nat. Methods 11, 743–748. doi: 10.1038/nmeth.2994

PubMed Abstract | CrossRef Full Text | Google Scholar

Reberg, D., Mann, B., and Innis, N. (1977). Superstitious behavior for food and water in the rat. Physiol. Behav. 19, 803–806. doi: 10.1016/0031-9384(77)90318-3

PubMed Abstract | CrossRef Full Text | Google Scholar

Reberg, D., Mann, B., Innis, N., and Eizenga, C. (1978). 'Superstitious' behaviour resulting from periodic response-independent presentations of food or water. Anim. Behav. 26, 507–519. doi: 10.1016/0003-3472(78)90067-2

CrossRef Full Text | Google Scholar

Reynolds, G. S. (1975). A Primer of Operant Conditioning. Glenview, IL, Scott Foresman.

Sanabria, F., Thrailkill, E. A., and Killeen, P. R. (2009). Timing with opportunity cost: concurrent schedules of reinforcement improve peak timing. Learn. Behav. 37, 217–229. doi: 10.3758/LB.37.3.217

PubMed Abstract | CrossRef Full Text | Google Scholar

Schneirla, T. C. (1959). “An evolutionary and developmental theory of biphasic processes underlying approach and withdrawal,” in Nebraska Symposium on Motivation, ed M. R. Jones (Lincoln: University of Nebraska Press), 1–42.

PubMed Abstract | Google Scholar

Schoenfeld, W. N., and Cole, B. K. (1972). Stimulus Schedules: The T-[tau (romanized Form)] Systems. New York, NY: HarperCollins Publishers.

Google Scholar

Silva, K. M., and Timberlake, W. (1997). A behavior systems view of conditioned states during long and short CS–US intervals. Learn. Motiv. 28, 465–490. doi: 10.1006/lmot.1997.0986

CrossRef Full Text | Google Scholar

Skinner, B. F. (1938). The Behavior of Organisms: An Experimental Analysis. New York, NY: Appleton-Century-Crofts.

Google Scholar

Skinner, B. F. (1966). What is the experimental analysis of behavior? J. Exp. Anal. Behav. 9, 213–218. doi: 10.1901/jeab.1966.9-213

PubMed Abstract | CrossRef Full Text | Google Scholar

Skinner, B. F. (1948). Superstition in the pigeon. J. Exp. Psychol. 38, 168–172. doi: 10.1037/h0055873

PubMed Abstract | CrossRef Full Text | Google Scholar

Spruijt, B. M., Peters, S. M., de Heer, R. C., Pothuizen, H. H., and van der Harst, J. E. (2014). Reproducibility and relevance of future behavioral sciences should benefit from a cross fertilization of past recommendations and today's technology: “back to the future”. J. Neurosci. Methods 234, 2–12. doi: 10.1016/j.jneumeth.2014.03.001

PubMed Abstract | CrossRef Full Text | Google Scholar

Timberlake, W. (1994). Behavior systems, associationism, and pavlovian conditioning. Psychon. Bull. Rev. 1, 405–420. doi: 10.3758/BF03210945

PubMed Abstract | CrossRef Full Text | Google Scholar

Torabi, R., Jenkins, S., Harker, A., Whishaw, I. Q., Gibb, R., and Luczak, A. (2020). Application of deep neural network reveals novel effects of maternal pre-conception exposure to nicotine on rat pup behavior. bioRxiv [Preprint]. doi: 10.1101/2020.07.16.206961

CrossRef Full Text | Google Scholar

Turgeon, S., and Lanovaz, M. J. (2020). Tutorial: applying machine learning in behavioral research. Perspect. Behav. Sci. 43, 697–723. doi: 10.1007/s40614-020-00270-y

PubMed Abstract | CrossRef Full Text | Google Scholar

Turvey, M. T. (2018). Lectures on Perception: An Ecological Perspective. New York, NY: Routledge. doi: 10.4324/9780429443879

CrossRef Full Text | Google Scholar

Wiltschko, A. B., Johnson, M. J., Iurilli, G., Peterson, R. E., Katon, J. M., Pashkovski, S. L., et al. (2015). Mapping sub-second structure in mouse behavior. Neuron 88, 1121–1135. doi: 10.1016/j.neuron.2015.11.031

PubMed Abstract | CrossRef Full Text | Google Scholar

Zuriff, G. E. (1970). A comparison of variable-ratio and variable-interval schedules of reinforcement. J. Exp. Anal. Behav. 13, 369–374. doi: 10.1901/jeab.1970.13-369

PubMed Abstract | CrossRef Full Text | Google Scholar