Adaptive formation learning control for cooperative AUVs under complete uncertainty

Introduction: This paper addresses the critical need for adaptive formation control in Autonomous Underwater Vehicles (AUVs) without requiring knowledge of system dynamics or environmental data. Current methods, often assuming partial knowledge like known mass matrices, limit adaptability in varied settings.

Methods: We proposed two-layer framework treats all system dynamics, including the mass matrix, as entirely unknown, achieving configuration-agnostic control applicable to multiple underwater scenarios. The first layer features a cooperative estimator for inter-agent communication independent of global data, while the second employs a decentralized deterministic learning (DDL) controller using local feedback for precise trajectory control. The framework's radial basis function neural networks (RBFNN) store dynamic information, eliminating the need for relearning after system restarts.

Results: This robust approach addresses uncertainties from unknown parametric values and unmodeled interactions internally, as well as external disturbances such as varying water currents and pressures, enhancing adaptability across diverse environments.

Discussion: Comprehensive and rigorous mathematical proofs are provided to confirm the stability of the proposed controller, while simulation results validate each agent’s control accuracy and signal boundedness, confirming the framework’s stability and resilience in complex scenarios.

1 Introduction

Robotics and autonomous systems have a wide range of applications, spanning from manufacturing and surgical procedures to exploration in challenging environments (Ghafoori et al., 2024; Jandaghi et al., 2023). However, controlling robots in such settings, especially in space and underwater, presents significant difficulties due to unpredictable dynamics. In the context of underwater exploration, AUVs have become essential tools, offering cost-effective, reliable, and versatile solutions for adapting to dynamic conditions. Effective use of AUVs is critical for unlocking the mysteries of marine environments, making advancements in their control and operation essential. As the demand for efficient underwater exploration increases and the complexity of tasks assigned to AUVs grows, there is a pressing need to enhance their operational capabilities. This includes developing sophisticated formation control strategies that allow multiple AUVs to operate in coordination, drawing inspiration from natural behaviors observed in fish schools and bird flocks (Zhou et al., 2023; Yang et al., 2021). By leveraging multi-agent systems, AUVs can work in coordinated groups, enhancing efficiency, stability, and coverage while navigating dynamic and complex underwater environments. These strategies are essential for ensuring precise operations in varied underwater tasks, ranging from pipeline inspections and seafloor mapping to environmental monitoring (Yan et al., 2023).

Despite challenges from intricate nonlinear dynamics, complex interactions among AUVs, and the uncertain dynamic nature of underwater environments, effective multi-AUV formation control is increasingly critical in modern ocean industries (Yan et al., 2018; Hou and Cheah, 2009). Historically, formation control research has predominantly utilized the behavioral approach (Balch and Arkin, 1998; Lawton, 2000), which divides the overall control design into subproblems, with each vehicle’s action determined by a weighted average of solutions, though selecting appropriate weighting parameters can be challenging. The leader-following approach (Cui et al., 2010; Rout and Subudhi, 2016) designates one vehicle as the leader while others follow, maintaining predefined geometric relationships, and controlling formation behavior by designing specific motions for the leader. Alternatively, the virtual structure approach (Millán et al., 2013).

Despite advancements in formation control and path planning for multi-AUV systems, challenges such as environmental disturbances, complex underwater dynamics, and communication limitations continue to pose difficulties (Hadi et al., 2021). To address these challenges, there is a critical need for controllers that are independent of both robot dynamics and environmental disturbances. Developing such controllers would enhance formation control by allowing for decentralized application, which increases flexibility in formation structures and improves robustness against communication constraints. Addressing these gaps is essential for advancing the capabilities and reliability of multi-AUV systems. On the other hand, communication constraints in underwater environments make decentralized control with a virtual leader-following topology ideal for AUVs, enabling coordination using local information despite communication delays or interruptions (Yan et al., 2023).

Reinforcement learning (RL) has also been extensively applied in robotic control (Christen et al., 2021; Cao et al., 2022). RL approaches, such as deep reinforcement learning (DRL), offer advantages in learning complex, non-linear control policies directly from data. However, RL methods generally lack the ability to provide mathematical stability proofs and guarantees for the controller’s behavior, making it challenging to ensure safety and reliability, especially in critical applications. Besides, while Zhang et al. (2018) developed various direct neural adaptive laws that lead to increased oscillations with higher adaptation gains, indirect neural adaptive laws using prediction error methods were proposed to mitigate this issue, though they could not guarantee parameter convergence. However, NN-based learning control methods, such as those utilizing adaptive neural networks or deterministic learning frameworks Jandaghi et al. (2024), can incorporate stability analysis and provide rigorous mathematical proofs for parameter convergence. These methods enable researchers to establish theoretical guarantees for the stability and robustness of the controller, which is essential for deploying controllers in real-world applications where safety and reliability are critical. Most recently, Tutsoy et al. (2024) proposed an optimization-based approach for path planning in Unmanned Air Vehicles (UAVs) with actuator failures using particle swarm optimization and genetic algorithms. Their method focuses on minimizing both time and distance by optimizing predefined cost functions through heuristic methods, while incorporating system constraints such as actuator limits, kinematic, and dynamic constraints, as well as parametric uncertainties.

Despite extensive literature in the field, to the best of our knowledge, existing researches assume homogeneous dynamics and certain system parameters for all AUV agents, which is unrealistic in unpredictable underwater environments. Factors such as buoyancy, drag, and varying water viscosity significantly alter system dynamics and behavior. Additionally, AUVs may change shape during tasks like underwater sampling or when equipped with robotic arms, further complicating control. Typically, designing multi-AUV formation control involves planning desired formation paths and developing tracking controllers for each AUV. However, accurately tracking these paths is challenging due to the complex nonlinear dynamics of AUVs, especially when precise models are unavailable. Implementing a fully distributed and decentralized formation control system is also difficult, as centralized control designs become exceedingly complex with larger AUV groups. To address these challenges previous work, such as Yuan et al. (2017) and Dong et al. (2019), developed adaptive learning controllers that relied on the assumption of a known mass matrix, which is not practical in real-world applications. These controllers relied on known system parameters that can fail due to varying internal forces caused by varying external environmental conditions. The solution is to develop environment-independent controllers that do not rely on any specific system dynamical parameters.

The framework’s control architecture is ingeniously divided into a first-layer Cooperative Estimator Observer and a lower-layer Decentralized Deterministic Learning (DDL) Controller. The first-layer observer is pivotal in enhancing inter-agent communication by sharing crucial system estimates, operating independently of any global information. Concurrently, the second-layer DDL controller utilizes local feedback to finely adjust each AUV’s trajectory, ensuring resilient operation under dynamic conditions heavily influenced by hydrodynamic forces and torques by considering system uncertainty completely unknown. This dual-layer setup not only facilitates acute adaptation to uncertain AUV dynamics but also leverages RBFNN for precise local learning and effective knowledge storage. Such capabilities enable AUVs to efficiently reapply previously learned dynamics after the system restarts. This tow-layer framework achieves a significant advancement by considering all system dynamics parameters as unknown, enabling a universal application across all AUVs, regardless of their operating environments. This universality is crucial for adapting to environmental variations such as water flow, which increases the AUV’s effective mass via the added mass phenomenon and affects the vehicle’s inertia. Additionally, buoyancy forces that vary with depth, along with hydrodynamic forces and torques, stemming from water flow variations, the AUV’s unique shape, its appendages, and drag forces due to water viscosity, significantly impact the damping matrix in the AUV’s dynamics. This framework not only improves operational efficiency but also significantly advances the field of autonomous underwater vehicle control by laying a robust foundation for future enhancements in distributed adaptive control systems and fostering enhanced collaborative intelligence among multi-agent networks in marine environments. Extensive simulations have underscored the effectiveness of the framework, demonstrating its potential to elevate the adaptability and resilience of AUV systems under the most demanding conditions. In summary, the contribution of this paper is as follows:

• The universal controller works in any environment and condition, such as currents or depth.

• Each AUV controller operates independently.

• The controller functions without needing information about the robot’s dynamic parameters, like mass, damping, or inertia. Each AUV can also have different dynamic parameters.

• The system learns the dynamics once and reuses the pre-trained weights, avoiding the need for retraining.

• The use of localized RBFNN reduces real-time computational demands.

• Providing rigorous stability analysis of the controller while providing mathematical proofs to ensure and guarantee the reliability of the controller.

The rest of the paper is organized as follows: Section 2 provides an initial overview of graph theory, RBFNN, and the problem statement. The design of the distributed cooperative estimator and the decentralized deterministic learning controller are discussed in Section 3. The formation adaptive control and formation control using pre-learned dynamics are explored in Section 4 and Section 5, respectively. Simulation studies are presented in Section 6, and Section 7 concludes the paper.

2 Preliminaries and problem statement

2.1 Notation and graph theory

Denoting the set of real numbers as $\mathbb{R}$, we define ${\mathbb{R}}^{m\times n}$ as the set of $m\times n$ real matrices, and ${\mathbb{R}}^n$ as the set of $n\times 1$ real vectors. The identity matrix is symbolized as $\mathbf{I}$. The vector with all elements being 1 in an $n$-dimensional space is represented as ${\mathbf{1}}_n$. The sets ${\mathbb{S}}_{+}^n$ and ${\mathbb{S}}_{-}^{n+}$ stand for real symmetric $n\times n$ and positive definite matrices, respectively. A block diagonal matrix with matrices $X_1,X_2,\dots ,X_p$ on its main diagonal is denoted by $\text{diag}\{X_1,X_2,\dots ,X_p\}$. $A\otimes B$ signifies the Kronecker product of matrices $A$ and $B$. For a matrix $A$, $\vec{A}$ is the vectorization of $A$ by stacking its columns on top of each other. For a series of column vectors $x_1,\dots ,x_n$, $\text{col}\{x_1,\dots ,x_n\}$ represents a column vector formed by stacking them together. Given two integers $k_1$ and $k_2$ with $k_1<k_2$, $\mathbf{I}[k_1,k_2]=\{k_1,k_1+1,\dots ,k_2\}$. For a vector $x\in {\mathbb{R}}^n$, its norm is defined as $|x|≔{(x^{T}x)}^{1/2}$. For a square matrix $A$, ${\lambda }_i(A)$ denotes its $i$-th eigenvalue, while ${\lambda }_{\text{min}}(A)$ and ${\lambda }_{\text{max}}(A)$ represent its minimum and maximum eigenvalues, respectively.

A directed graph $G=(V,E)$ comprises nodes in the set $V=\{\mathrm{1,2},\dots ,N\}$ and edges in $E\subseteq V\times V$. An edge from node $i$ to node $j$ is represented as $(i,j)$, with $i$ as the parent node and $j$ as the child node. Node $i$ is also termed a neighbor of node $j$. $N_i$ is considered as the subset of $V$ consisting of the neighbors of node $i$. A sequence of edges in $G$, $(i_1,i_2),(i_2,i_3),\dots ,(i_k,i_{k+1})$, is called a path from node $i_1$ to node $i_{k+1}$. Node $i_{k+1}$ is reachable from node $i_1$. A directed tree is a graph where each node, except for a root node, has exactly one parent. The root node is reachable from all other nodes. A directed graph $G$ contains a directed spanning tree if at least one node can reach all other nodes. The weighted adjacency matrix of $G$ is a non-negative matrix $A=[a_{ij}]\in {\mathbb{R}}^{N\times N}$, where $a_{ii}=0$ and $a_{ij}>0\Rightarrow (j,i)\in E$. The Laplacian of $G$ is denoted as $L=[l_{ij}]\in {\mathbb{R}}^{N\times N}$, where $l_{ii}={\sum }_{j=1}^N{a}_{ij}$ and $l_{ij}=-a_{ij}$ if $i\ne j$. It is established that $L$ has at least one eigenvalue at the origin, and all nonzero eigenvalues of $L$ have positive real parts. From Ren and Beard (2005), $L$ has one zero eigenvalue and remaining eigenvalues with positive real parts if and only if $G$ has a directed spanning tree.

2.2 Radial basis function neural networks (RBFNN)

The RBFNN Networks can be described as $f_{nn}(Z)={\sum }_{i=1}^N{w}_i{s}_i(Z)=W^{T}S(Z)$, where $Z\in {\mathrm{\Omega }}_Z\subseteq {\mathbb{R}}^q$ and $W=w_1,\dots ,w_N^T\in {\mathbb{R}}^N$ as input and weight vectors respectively (Park and Sandberg, 1991). $N$ indicates the number of NN nodes, $S(Z)={[s_1(\Vert Z-{\mu }_i\Vert ),\dots ,s_N(\Vert Z-{\mu }_i\Vert )]}^T$ with $s_i(\cdot )$ is a radial basis function, and ${\mu }_i(i=1,\dots ,N)$ is distinct points in the state space. The Gaussian function $s_i(\Vert Z-{\mu }_i\Vert )=\exp \left[-\frac{{(Z-{\mu }_i)}^T(Z-{\mu }_i)}{{\eta }_i^2}\right]$ is generally used for radial basis function, where ${\mu }_i={[{\mu }_{i1},{\mu }_{i2},\dots ,{\mu }_{iN}]}^T$ is the center and ${\eta }_i$ is the width of the receptive field. The Gaussian function categorized by localized radial basis function $s$ in the sense that $s_i(\Vert Z-{\mu }_i\Vert )\to 0$ as $\Vert Z\Vert \to \infty$. Moreover, for any bounded trajectory $Z(t)$ within the compact set ${\mathrm{\Omega }}_Z$, $f(Z)$ can be approximated using a limited number of neurons located in a local region along the trajectory $f(Z)=W_{\zeta }^{*}{S}_{\zeta }(Z)+{ϵ}_{\zeta }$. $\zeta$ denotes the indices of active RBFNN nodes where $|s_{j_i}(Z)|>\iota$, based on the state $Z(t)$. ${ϵ}_{\zeta }$ is the approximation error, with ${ϵ}_{\zeta }=O(ϵ)=O({ϵ}^{*})$, $S_{\zeta }(Z)={[s_{j_1}(Z),\dots ,s_{j_{\zeta }}(Z)]}^T\in {\mathbb{R}}^{N_{\zeta }}$, $W_{\zeta }^{*}={[w_{j_1}^{*},\dots ,w_{j_{\zeta }}^{*}]}^T\in {\mathbb{R}}^{N_{\zeta }}$, $N_{\zeta }<N_n$, and the integers $j_i=j_1,\dots ,j_{\zeta }$ are defined by $|s_{j_i}(Z_p)|>\iota$ ($\iota >0$ is a small positive constant) for some $Z_p\in Z(t)$. This holds if $\Vert Z(t)-{\xi }_{j_i}\Vert <ϵ$ for $t>0$. The following lemma regarding the persistent excitation (PE) condition for RBFNN is recalled from Wang and Hill (2018).

Lemma 1. Consider any continuous recurrent trajectory¹ $Z(t):[0,\infty )\to {\mathbb{R}}^q$. $Z(t)$ remains in a bounded compact set ${\mathrm{\Omega }}_Z\subset {\mathbb{R}}^q$. Then for an RBFNN $W^{T}S(Z)$ with centers placed on a regular lattice (large enough to cover the compact set ${\mathrm{\Omega }}_Z$), the regressor subvector $S_{\zeta }(Z)$ consisting of RBFNN with centers located in a small neighborhood of $Z(t)$ is persistently exciting.

2.3 Problem statement

A multi-agent system comprising $N$ AUVs with heterogeneous nonlinear uncertain dynamics is considered. The dynamics of each AUV can be expressed as Fossen (1999):

$\begin{array}{cc}\hfill & {\dot{\eta }}_i=J_i\left({\eta }_i\right){\nu }_i,\hfill \\ \hfill & M_i{\dot{\nu }}_i+C_i\left({\nu }_i\right){\nu }_i+D_i\left({\nu }_i\right){\nu }_i+g_i\left({\eta }_i\right)+{\mathrm{\Delta }}_i\left({\chi }_i\right)={\tau }_i.\hfill \end{array}(1)$

In this study, the subscript $i\in I[1,N]$ identifies each AUV within the multi-agent system. For every $i\in I[1,N]$, the vector ${\eta }_i={[x_i,y_i,{\psi }_i]}^T\in {\mathbb{R}}^3$ represents the $i$-th AUV’s position coordinates and heading in the global coordinate frame, while ${\nu }_i={[u_i,v_i,r_i]}^T\in {\mathbb{R}}^3$ denotes its linear velocities and angular rate of heading relative to a body-fixed frame. The positive definite inertial matrix $M_i=M_i^T\in {\mathbb{S}}_3^{+}$, Coriolis and centripetal matrix $C_i({\nu }_i)\in {\mathbb{R}}^{3\times 3}$, and damping matrix $D_i({\nu }_i)\in {\mathbb{R}}^{3\times 3}$ characterize the AUV’s dynamic response to motion. The vector $g_i({\eta }_i)\in {\mathbb{R}}^{3\times 1}$ accounts for the restoring forces and moments due to gravity and buoyancy. The term ${\mathrm{\Delta }}_i({\chi }_i)\in {\mathbb{R}}^{3\times 1}$, with ${\chi }_i≔\text{col}\{{\eta }_i,{\nu }_i\}$, describes the vector of generalized deterministic unmodeled uncertain dynamics for each AUV.

The vector ${\tau }_i\in {\mathbb{R}}^3$ represents the control inputs for each AUV. The associated rotation matrix $J_i({\eta }_i)$ is given by:

$\begin{array}{c}\hfill J_i\left({\eta }_i\right)=\left[\begin{array}{ccc}\hfill \cos \left({\psi }_i\right)\hfill & \hfill \sin \left({\psi }_i\right)\hfill & \hfill 0\hfill \\ \hfill -\sin \left({\psi }_i\right)\hfill & \hfill \cos \left({\psi }_i\right)\hfill & \hfill 0\hfill \\ \hfill 0\hfill & \hfill 0\hfill & \hfill 1\hfill \end{array}\right],\end{array}$

Unlike previous work Yuan et al. (2017), which assumed known values for the AUV’s inertia and rotation matrices, this study considers all matrix coefficients, including $C_i({\nu }_i)$, $D_i({\nu }_i)$, $g_i({\eta }_i)$, and ${\mathrm{\Delta }}_i({\chi }_i)$, as well as the inertia matrix, as completely unknown. The adaptive estimation process inherently addresses the effects of external forces and disturbances on system dynamics. This eliminates the need for explicit parameter estimation of these forces, as disturbances like water flow, varying currents, or depth variations are directly incorporated into the control input through adaptive estimation. This makes the controller universally applicable to any AUV, regardless of its design, weight, or environmental conditions by addressing both internal and external dynamic variation at the same time.

Internally, it handles unknown parameters such as mass and damping coefficients, as well as unmodeled nonlinear interactions and couplings. Externally, it accounts for unpredictable disturbances, including fluctuating water currents, depth-dependent pressures, and changes in hydrodynamic forces.

By avoiding reliance on predefined models, the proposed approach is robust and adaptable to diverse mission scenarios and unexpected environmental changes, ensuring reliable performance even in highly uncertain conditions.

In the context of leader-following formation tracking control, the following virtual leader dynamics generates the tracking reference signals:

${\dot{\chi }}_0=A_0{\chi }_0,(2)$

with “0” marking the leader node, the leader state ${\chi }_0≔col\{{\eta }_0,{\nu }_0\}$ with ${\eta }_0\in {\mathbb{R}}^3$ and ${\nu }_0\in {\mathbb{R}}^3$, $A_0\in {\mathbb{R}}^{6\times 6}$ is a constant matrix available only to the leader’s neighboring AUV agents.

Considering the system dynamics of multiple AUVs (Equation 1) along with the leader dynamics (Equation 2), we establish a non-negative matrix $A=[a_{ij}]$, where $i,j\in I[0,N]$ such that for each $i\in I[1,N]$, $a_{i0}>0$ if and only if agent $i$ has access to the reference signals ${\eta }_0$ and ${\nu }_0$. All remaining elements of $A$ are arbitrary non-negative values, such that $a_{ii}=0$ for all $i$. Correspondingly, we establish $G=(V,E)$ as a directed graph derived from $A$, where $V=\{\mathrm{0,1},\dots ,N\}$ designates node 0 as the leader, and the remaining nodes correspond to the $N$ AUV agents. We proceed under the following assumptions:

Assumption 1. All the eigenvalues of $A_0$ in the leader’s dynamics (Equation 2) are located on the imaginary axis.

Assumption 2. The directed graph $G$ contains a directed spanning tree with the node 0 as its root.

Assumption 1 is crucial for ensuring that the leader dynamics produce stable, meaningful reference trajectories for formation control. It ensures that all states of the leader, represented by ${\chi }_0=\text{col}\{{\eta }_0,{\nu }_0\}$, remain within the bounds of a compact set ${\mathrm{\Omega }}_0\subset {\mathbb{R}}^6$ for all $t\ge 0$. The trajectory of the system, starting from ${\chi }_0(0)$ and denoted by ${\varphi }_0({\chi }_0(0))$, generates periodic signal. This periodicity is essential for maintaining the Persistent Excitation (PE) condition, which is pivotal for achieving parameter convergence in Distributed Adaptive (DA) control systems. Modifications to the eigenvalue constraints on $A_0$ mentioned in Assumption 1 may be considered when focusing primarily on formation tracking control performance, as discussed later.

Additionally, Assumption 2 reveals key insights into the structure of the Laplacian matrix $L$ of the network graph $G$. Let $\mathrm{\Psi }$ be an $N\times N$ non-negative diagonal matrix where each $i$-th diagonal element is $a_{i0}$ for $i\in I[1,N]$. The Laplacian $L$ is formulated as:

$L=\left[\begin{array}{cc}\hfill {\displaystyle \sum _{j=1}^N}{a}_{0j}\hfill & \hfill -\left[a_{01},\dots ,a_{0N}\right]\hfill \\ \hfill -\mathrm{\Psi }{\mathbf{1}}_N\hfill & \hfill \mathcal{H}\hfill \end{array}\right],$

where $a_{0j}>0$ if $(j,0)\in E$ and $a_{0j}=0$ otherwise. This results in $\mathcal{H}{\mathbf{1}}_N=\mathrm{\Psi }{\mathbf{1}}_N$ since $L{\mathbf{1}}_{N+1}=0$. As cited in Su and Huang (2011), all nonzero eigenvalues of $\mathcal{H}$, if present, exhibit positive real parts, confirming $\mathcal{H}$ as nonsingular under Assumption 2.

Problem 1. In the context of a multi-AUV system (Equation 1) integrated with virtual leader dynamics (Equation 2) and operating within a directed network topology $G$, the aim is to develop a distributed NN learning control protocol that leverages only local information. The specific goals are twofold:

1) Formation Control: Each of the $N$ AUV agents will adhere to a predetermined formation pattern relative to the leader, maintaining a specified distance from the leader’s position ${\eta }_0$.

2) Decentralized Learning: The nonlinear uncertain dynamics of each AUV will be identified and learned autonomously during the formation control process. The insights gained from this learning process will be utilized to enhance the stability and performance of the formation control system.

Remark 1. The leader dynamics described in Equation 2 are designed as a neutrally stable LTI system. This design choice facilitates the generation of sinusoidal reference trajectories at various frequencies which is essential for effective formation tracking control. This approach to leader dynamics is prevalent in the literature on multiagent leader-following distributed control systems like Yuan (2017) and Jandaghi et al. (2024).

Remark 2. It is important to emphasize that the formulation assumes formation control is required only within the horizontal plane, suitable for AUVs operating at a constant depth, and that the vertical dynamics of the 6 degrees of freedom (DOF) AUV system, as detailed in Prestero (2001), are entirely decoupled from the horizontal dynamics.

As shown in Figure 1, a two-layer hierarchical design approach is proposed to address the aforementioned challenges. The first layer, the Cooperative Estimator, enables information exchange among neighboring agents. The second layer, known as the Decentralized Deterministic Learning (DDL) controller, processes only local data from each individual AUV. The development and formulation of the first layer are discussed in detail in Section 3.1, while the DDL control strategy, along with its corresponding controller design and analysis, is provided in Section 3.2.

Figure 1

Figure 1. Proposed two-layer distributed controller architecture for each AUVs.

3 Two-layer distributed controller architecture

3.1 First layer: cooperative estimator

In the context of leader-following formation control, not all AUV agents may have direct access to the leader’s information, including tracking reference signals $({\chi }_0)$ and the system matrix $(A_0)$. This necessitates collaborative interactions among the AUV agents to estimate the leader’s information effectively. Drawing on principles from multiagent consensus and graph theories Ren and Beard (2008), we propose to develop a distributed adaptive observer for the AUV systems as:

${\dot{\widehat{\chi }}}_i\left(t\right)={\widehat{A}}_i\left(t\right){\widehat{\chi }}_i\left(t\right)+{\beta }_{i1}{\displaystyle \sum _{j=1}^N}{a}_{ij}\left({\widehat{\chi }}_j\left(t\right)-{\widehat{\chi }}_i\left(t\right)\right),\forall i\in I\left[1,N\right].(3)$

The observer states for each $i$-th AUV, denoted by ${\widehat{\chi }}_i={[{\widehat{\eta }}_i,{\widehat{\nu }}_i]}^T\in {\mathbb{R}}^6$, aim to estimate the leader’s state, ${\chi }_0={[{\eta }_0,{\nu }_0]}^T\in {\mathbb{R}}^6$. As $t\to \infty$, these estimates are expected to converge, such that ${\widehat{\eta }}_i$ approaches ${\eta }_0$ and ${\widehat{\nu }}_i$ approaches ${\nu }_0$, representing the leader’s position and velocity, respectively. Equation 3 accounts for the communication graph by including the adjacency matrix information through the term $a_{ij}$. Note that ${\widehat{A}}_i(t)\in {\mathbb{R}}^{6\times 6}$ represents an estimate computed by agent i of the leader’s matrix dynamics $A_0(t)\in {\mathbb{R}}^{6\times 6}$ which is also not available to the agents. Therefore, each agent estimates such matrix using a cooperative adaptation law:

${\dot{\widehat{A}}}_i\left(t\right)={\beta }_{i2}{\displaystyle \sum _{j=1}^N}{a}_{ij}\left({\widehat{A}}_j\left(t\right)-{\widehat{A}}_i\left(t\right)\right),\forall i\in I\left[1,N\right],(4)$

which borrowed from Ren and Beard (2008) as well. The constants ${\beta }_{i1}$ and ${\beta }_{i2}$ are all positive numbers and are subject to design.

Remark 3. Each AUV agent in the group is equipped with an observer configured as specified in Equations 3, 4, comprising two state variables, ${\chi }_i$ and $A_i$. For each $i\in I[1,N]$, ${\widehat{\chi }}_i$ estimates the virtual leader’s state ${\chi }_0$, while ${\widehat{A}}_i$ estimates the leader’s system matrix $A_0$. The real-time data necessary for operating the $i$-th observer includes: (1) the estimated state ${\widehat{\chi }}_i$ and matrix $\widehat{A_i}$, obtained from the $i$-th AUV itself, and (2) the estimated states ${\widehat{\chi }}_j$ and matrices $\widehat{A_j}$ for all $j\in N_i$, obtains from the $j$-th AUV’s neighbors. Note that in Equations 3, 4, if $j\notin N_i$, then $a_{ij}=0$, indicating that the $i$-th observer does not utilize information from the $j$-th AUV agent. This configuration ensures that the proposed distributed observer can be implemented in each local AUV agent using only locally estimated data from the agent itself and its immediate neighbors, without the need for global information such as the size of the AUV group or the network interconnection topology.

To verify the convergence properties, we need to compute the error dynamics. Now we define the estimation error for the state and the system matrix for agent $i$ as ${\tilde{\chi }}_i={\widehat{\chi }}_i-{\chi }_0$ and ${\tilde{A}}_i={\widehat{A}}_i-A_0$, and then we derive the error dynamics:

$\begin{array}{ll}\hfill {\dot{\stackrel{̃}{\chi }}}_i\left(t\right)& ={\widehat{A}}_i\left(t\right){\widehat{\chi }}_i\left(t\right)-A_0{\chi }_0\left(t\right)+{\beta }_{i1}{\displaystyle \sum _{j=1}^N}{a}_{ij}\left({\widehat{\chi }}_j\left(t\right)-{\widehat{\chi }}_i\left(t\right)\right)\hfill \\ \hfill & ={\widehat{A}}_i\left(t\right){\widehat{\chi }}_i\left(t\right)-A_0{\widehat{\chi }}_i\left(t\right)+A_0{\widehat{\chi }}_i\left(t\right)-A_0{\chi }_0\left(t\right)\hfill \\ \hfill & +{\beta }_{i1}{\displaystyle \sum _{j=1}^N}{a}_{ij}\left({\widehat{\chi }}_j\left(t\right)-{\chi }_0\left(t\right)+{\chi }_0\left(t\right)-{\widehat{\chi }}_i\left(t\right)\right)\hfill \\ \hfill & =A_0{\tilde{\chi }}_i\left(t\right)+{\tilde{A}}_i\left(t\right){\tilde{\chi }}_i\left(t\right)+{\tilde{A}}_i\left(t\right){\chi }_0\left(t\right)+{\beta }_{i1}{\displaystyle \sum _{j=1}^N}{a}_{ij}\left({\tilde{\chi }}_j\left(t\right)-{\tilde{\chi }}_i\left(t\right)\right)\hfill \\ \hfill {\dot{\stackrel{̃}{A}}}_i\left(t\right)& ={\beta }_{i2}{\displaystyle \sum _{j=1}^N}{a}_{ij}\left({\tilde{A}}_j\left(t\right)-{\tilde{A}}_i\left(t\right)\right),\forall i\in I\left[1,N\right].\hfill \end{array}$

Define the collective error states and adaptation matrices: $\tilde{\chi }=\text{col}\{{\tilde{\chi }}_1,\dots ,{\tilde{\chi }}_N\}$ for the state errors, $\tilde{A}=\text{col}\{{\tilde{A}}_1,\dots ,{\tilde{A}}_N\}$ for the adaptive parameter errors, ${\tilde{A}}_b=\text{diag}\{{\tilde{A}}_1,\dots ,{\tilde{A}}_N\}$ representing the block diagonal of adaptive parameters, $B_{{\beta }_1}=\text{diag}\{{\beta }_{11},\dots ,{\beta }_{N1}\}$ and $B_{{\beta }_2}=\text{diag}\{{\beta }_{12},\dots ,{\beta }_{N2}\}$ for the diagonal matrices of design constants. With these definitions, the network-wide error dynamics can be expressed as:

$\begin{array}{ll}\hfill \dot{\stackrel{̃}{\chi }}\left(t\right)& =\left(\left(I_N\otimes A_0\right)-B_{{\beta }_1}\left(\mathcal{H}\otimes I_6\right)\right)\tilde{\chi }\left(t\right)+\left({\tilde{A}}_b\left(t\right)\otimes I_6\right)\tilde{\chi }\left(t\right)+{\tilde{A}}_b\left(t\right)\left(1_N\otimes {\chi }_0\left(t\right)\right),\hfill \\ \hfill \dot{\stackrel{̃}{A}}\left(t\right)& =-B_{{\beta }_2}\left(\mathcal{H}\otimes I_6\right)\tilde{A}\left(t\right).\hfill \end{array}(5)$

Theorem 1. Consider the error system Equation 5. Under Assumptions 1, 2, and given that ${\beta }_1,{\beta }_2>0$, it follows that for all $i\in I[1,N]$ and for any initial conditions ${\chi }_0(0),{\chi }_i(0),A_i(0)$, the error dynamics of the adaptive parameters and the states will converge to zero exponentially. Specifically, ${\lim }_{t\to \infty }{\tilde{A}}_i(t)=0$ and ${\lim }_{t\to \infty }{\tilde{\chi }}_i(t)=0$.

This convergence is facilitated by the independent adaptation of each agent’s parameters within their respective error dynamics, represented by the block diagonal structure of ${\tilde{A}}_b$ and control gains $B_{{\beta }_1}$ and $B_{{\beta }_2}$. These matrices ensure that each agent’s parameter updates are governed by local interactions and error feedback, consistent with the decentralized control framework.

Proof: We begin by examining the estimation error dynamics for $\tilde{A}$ as presented in Equation 5. This can be rewritten in the vector form:

${\dot{\stackrel{⃗}{\stackrel{̃}{A}}}}_0\left(t\right)=-{\beta }_2\left(I_6\otimes \left(\mathcal{H}\otimes I_6\right)\right){\overrightarrow{\stackrel{̃}{A}}}_0\left(t\right).(6)$

Under Assumption 2, all eigenvalues of $\mathcal{H}$ possess positive real parts according to Su and Huang (2011). Consequently, for any positive ${\beta }_2>0$, the matrix $-{\beta }_2(I_6\otimes (\mathcal{H}\otimes I_6))$ is guaranteed to be Hurwitz, Which implies the exponential stability of system (Equation 6). Hence, it follows that ${\lim }_{t\to \infty }{\overrightarrow{\stackrel{̃}{A}}}_0(t)=0$ exponentially, leading to ${\lim }_{t\to \infty }{\tilde{A}}_{i0}(t)=0$ exponentially for all $i\in I[1,N]$. Now, we analyze the error dynamics for ${\chi }_0$ in Equation 5. Based on the previous discussions, We have ${\lim }_{t\to \infty }{\tilde{A}}_b(t)=0$ exponentially, and the term ${\tilde{A}}_b(t)(1_N\otimes {\chi }_0(t))$ will similarly decay to zero exponentially. Based on Cai et al. (2015), if the system defined by

${\dot{\stackrel{̃}{\chi }}}_0\left(t\right)=\left(\left(I_N\otimes A_0\right)-{\beta }_1\left(\mathcal{H}\otimes I_6\right)\right){\tilde{\chi }}_0\left(t\right).(7)$

is exponentially stable, then ${\lim }_{t\to \infty }{\chi }_0(t)=0$ exponentially. With Assumption 1, knowing that all eigenvalues of $A_0$ have zero real parts, and since $\mathcal{H}$ as nonsingular with all eigenvalues in the right-half plane, system (Equation 7) is exponentially stable for any positive ${\beta }_1>0$. Consequently, this ensures that ${\lim }_{t\to \infty }{\chi }_0(t)=0$, i.e., ${\lim }_{t\to \infty }{\chi }_{i0}(t)=0$ exponentially for all $i\in I[1,N]$.

Now, each individual agent can accurately estimate both the state and the system matrix of the leader through cooperative observer estimation Equations 3, 4. This information will be utilized in the DDL controller design for each agent’s second layer, which will be discussed in the following subsection.

3.2 Second layer: decentralized deterministic learning controller

To fulfill the overall formation learning control objectives, in this section, we develop the DDL control law for the multi-AUV system defined in Equation 1. We use $d_i^{*}$ to denote the desired distance between the position of the $i$-th AUV agent ${\eta }_i$ and the virtual leader’s position ${\eta }_0$. Then, the formation control problem is framed as a position tracking control task, where each local AUV agent’s position ${\eta }_i$ is required to track the reference signal ${\eta }_{d,i}≔{\eta }_0+d_i^{*}$. Besides, due to the inaccessibility of the leader’s state information ${\chi }_0$ for all AUV agents, the tracking reference signal ${\widehat{\eta }}_{d,i}≔{\widehat{\eta }}_{0,i}+d_i^{*}$ is employed instead of the reference signal ${\eta }_{d,i}$. As established in Theorem 1, ${\widehat{\eta }}_{d,i}$ is autonomously generated by each local agent and will exponentially converge to ${\eta }_{d,i}$. This ensures that the DDL controller is feasible and the formation control objectives are achievable for all $i\in I[1,N]$ using ${\widehat{\eta }}_{d,i}$.

To design the DDL control law that addresses the formation tracking control and the precise learning of the AUVs’ complete nonlinear uncertain dynamics at the same time, we will integrate renowned backstepping adaptive control design method outlined in Krstic et al. (1995) along with techniques from Wang and Hill (2018) and Yuan et al. (2017) for deterministic learning using RBFNN. Specifically, for the $i$-th AUV agent described in system (Equation 1), we define the position tracking error as $z_{1,i}={\eta }_i-{\widehat{\eta }}_{d,i}$ for all $i\in I[1,N]$. Considering $J_i({\eta }_i)J_i^T({\eta }_i)=I$ for all $i\in I[1,N]$, we proceed to:

${\dot{z}}_{1,i}=J_i\left({\eta }_i\right){\nu }_i-{\dot{\widehat{\eta }}}_i,\forall i\in \mathbf{I}\left[1,N\right].(8)$

To frame the problem in a more tractable way, we assume ${\nu }_i$ as a virtual control input and ${\alpha }_i$ as a desired virtual control input in our control strategy design, and by implementing them in the above system we have:

$\begin{array}{cc}\hfill z_{2,i}& ={\nu }_i-{\alpha }_i,\hfill \\ \hfill {\alpha }_i& =J_i^T\left({\eta }_i\right)\left(-K_{1,i}{z}_{1,i}+{\dot{\widehat{\eta }}}_i\right),\forall i\in \mathbf{I}\left[1,N\right].\hfill \end{array}(9)$

A positive definite gain matrix $K_{1,i}\in {\mathbb{S}}_3^{+}$ is used for tuning the performance. Substituting ${\nu }_i=z_{2,i}+{\alpha }_i$ into Equation 8 yields:

${\dot{z}}_{1,i}=J_i\left({\eta }_i\right)z_{2,i}-K_{1,i}{z}_{1,i},\forall i\in \mathbf{I}\left[1,N\right].$

Now we derive the first derivatives of the virtual control input and the desired control input as follows:

$\begin{array}{cc}\hfill {\dot{z}}_{2,i}& ={\dot{\nu }}_i-{\dot{\alpha }}_i\hfill \\ \hfill & =M_i^{-1}\left(-C_i\left({\nu }_i\right){\nu }_i-D_i\left({\nu }_i\right){\nu }_i-g_i\left({\eta }_i\right)-{\mathrm{\Delta }}_i\left({\chi }_i\right)+{\tau }_i\right)-{\dot{\alpha }}_i,\hfill \end{array}$

${\dot{\alpha }}_i={\dot{J}}_i^T\left({\eta }_i\right)\left(-K_{1,i}{z}_{1,i}+{\dot{\widehat{\eta }}}_i\right)+J_i^T\left({\eta }_i\right)\left(K_{1,i}{\dot{\widehat{\eta }}}_i-K_{1,i}{J}_i\left({\eta }_i\right){\nu }_i+{\ddot{\widehat{\eta }}}_i\right),\forall i\in \mathbf{I}\left[1,N\right].(10)$

As previously discussed, unlike earlier research that only identified the matrix coefficients $C_i({\nu }_i)$, $D_i({\nu }_i)$, $g_i({\eta }_i)$, and ${\mathrm{\Delta }}_i({\chi }_i)$ as unknown system nonlinearities while assuming the mass matrix $M_i$ to be known, this work advances significantly by also considering $M_i$ as unknown. Consequently, all system dynamic parameters are treated as completely unknown, making the controller fully independent of the robot’s configuration such as its dimensions, mass, or any appendages and the uncertain environmental conditions it encounters, like depth, water flow, and viscosity. This independence is critical as it ensures that the controller does not rely on predefined assumptions about the dynamics, aligning with the main goal of this research. To address these challenges, we define a unique nonlinear function $F_i(Z_i)$ that encapsulates all nonlinear uncertainties as follows:

$F_i\left(Z_i\right)=M_i{\dot{\alpha }}_i+C_i\left({\nu }_i\right){\nu }_i+D_i\left({\nu }_i\right){\nu }_i+g_i\left({\eta }_i\right)+{\mathrm{\Delta }}_i\left({\chi }_i\right),(11)$

where $F_i(Z_i)={[f_{1,i}(Z_i),f_{2,i}(Z_i),f_{3,i}(Z_i)]}^T$ and $Z_i=\text{col}\{{\eta }_i,{\nu }_i\}\in {\mathrm{\Omega }}_{Z_i}\subset {\mathbb{R}}^6$, with ${\mathrm{\Omega }}_{Z_i}$ being a bounded compact set. We then employ the following RBFNN to approximate the model dynamics in (Equation 11) expressed by nonlinear functions $F_i(Z_i)$ with $f_{k,i}$ for all $i\in \mathbf{I}[1,N]$ and $k\in \mathbf{I}[\mathrm{1,3}]$ as follows:

$f_{k,i}\left(Z_i\right)=W_{k,i}^{*T}{S}_{ki}\left(Z_i\right)+{ϵ}_{k,i}\left(Z_i\right),(12)$

where $W_{k,i}^{*}$ is the ideal constant NN weights, and ${ϵ}_{k,i}(Z_i)$ is the approximation error ${ϵ}_{k,i}^{*}>0$ for all $i\in \mathbf{I}[1,N]$ and $k\in \mathbf{I}[\mathrm{1,3}]$, which satisfies $|{ϵ}_{k,i}(Z_i)|\le {ϵ}_{k,i}^{*}$. This error can be made arbitrarily small given a sufficient number of neurons in the network. A self-adaptation law is designed to estimate the unknown $W_{k,i}^{*}$ online. We aim to estimate $W_{k,i}^{*}$ with ${\widehat{W}}_{k,i}$ by constructing the DDL feedback control law as follows:

${\tau }_i=-J_i^T\left({\eta }_i\right)z_{1,i}-K_{2,i}{z}_{2,i}+{\widehat{W}}_i^T{S}_i^F\left(Z_i\right).(13)$

$K_{2,i}\in {\mathbb{S}}_3^{+}$ is a feedback gain matrix that can be tuned to achieve the desired performance. To approximate the unknown nonlinear function vector $F_i(Z_i)$ in (Equation 11) along the trajectory $Z_i$ within the compact set ${\mathrm{\Omega }}_{Z_i}$, we use:

${\widehat{W}}_i^T{S}_i^F\left(Z_i\right)=\left[\begin{array}{c}\hfill {\widehat{W}}_{1,i}^T{S}_{1,i}\left(Z_i\right)\hfill \\ \hfill {\widehat{W}}_{2,i}^T{S}_{2,i}\left(Z_i\right)\hfill \\ \hfill {\widehat{W}}_{3,i}^T{S}_{3,i}\left(Z_i\right)\hfill \end{array}\right].$

Then, from Equations 1, 13 we have:

$M_i{\dot{\nu }}_i+C_i\left({\nu }_i\right){\nu }_i+D_i\left({\nu }_i\right){\nu }_i+g_i\left({\eta }_i\right)+{\mathrm{\Delta }}_i\left({\chi }_i\right)={\tau }_i=-J_i^T\left({\eta }_i\right)z_{1,i}-K_{2,i}{z}_{2,i}+{\widehat{W}}_{k,i}^T{S}_{k,i}\left(Z_i\right).$

By subtracting $W_{k,i}^{*T}{S}_{k,i}(Z_i)+{ϵ}_{k,i}(Z_i)$ from both sides and considering Equations 9, 11, we define ${\tilde{W}}_{k,i}≔{\widehat{W}}_{k,i}-W_{k,i}^{*}$, leading to:

${\dot{z}}_{2,i}=M_i^{-1}\left(-J_i^T\left({\eta }_i\right)z_{1,i}-K_{2,i}{z}_{2,i}+{\widehat{W}}_{k,i}^T{S}_{k,i}\left(Z_i\right)-{ϵ}_{k,i}\left(Z_i\right)\right).$

For updating ${\widehat{W}}_{k,i}$ online, a robust self-adaptation law is constructed using the $\sigma$-modification technique Ioannou and Sun (1996) as follows:

${\dot{\widehat{W}}}_{k,i}=-{\mathrm{\Gamma }}_{k,i}\left(S_{k,i}\left(Z_i\right)z_{2k,i}+{\sigma }_{ki}{\widehat{W}}_{k,i}\right).(14)$

where $z_{2,i}={[z_{21,i},z_{22,i},z_{23,i}]}^T$, ${\mathrm{\Gamma }}_{k,i}={\mathrm{\Gamma }}_{k,i}^T>0$, and ${\sigma }_{k,i}>0$ are free parameters to be designed for all $i\in \mathbf{I}[1,N]$ and $k\in \mathbf{I}[\mathrm{1,3}]$. Integrating Equations 9, 13, 14 yields the following closed-loop system:

$\left\{\begin{array}{cc}\hfill {\dot{z}}_{1,i}& =-K_{1,i}{z}_{1,i}+J_i\left({\eta }_i\right)z_{2,i},\hfill \\ \hfill {\dot{z}}_{2,i}& =M_i^{-1}\left(-J_i^T\left({\eta }_i\right)z_{1,i}-K_{2,i}{z}_{2,i}+{\widehat{W}}_{ki}^T{S}_{ki}\left(Z_i\right)-{ϵ}_{k,i}\left(Z_i\right)\right),\hfill \\ \hfill {\dot{\stackrel{̃}{W}}}_{k,i}& =-{\mathrm{\Gamma }}_{k,i}\left(S_{k,i}\left(Z_i\right)z_{2,k,i}+{\sigma }_{k,i}{\widehat{W}}_{k,i}\right),\hfill \end{array}\right.(15)$

where, for all $i\in \mathbf{I}[1,N]$ and $k\in \mathbf{I}[\mathrm{1,3}]$, ${\tilde{W}}_i^T{S}_i(Z_i)={[{\tilde{W}}_{1,i}^T{S}_{1,i}(Z_i),{\tilde{W}}_{2,i}^T{S}_{2,i}(Z_i),{\tilde{W}}_{3,i}^T{S}_{3,i}(Z_i)]}^T$, and ${ϵ}_i(Z_i)={[{ϵ}_{1,i}(Z_i),{ϵ}_{2,i}(Z_i),{ϵ}_{3,i}(Z_i)]}^T$.

Remark 4. Unlike the first-layer DA observer design, the second-layer control law is fully decentralized for each local agent. It utilizes only the local agent’s information for feedback control, including ${\chi }_i$, ${\widehat{\chi }}_i$, and $W_{k,i}$, without involving any information exchange among neighboring AUVs.

The following theorem summarizes the stability and tracking control performance results of the overall system:

Theorem 2. Consider the local closed-loop system (Equation 15). For each $i\in \mathbf{I}[1,N]$, if there exists a sufficiently large compact set ${\mathrm{\Omega }}_{Z_i}$such that $Z_i\in {\mathrm{\Omega }}_{Z_i}$for all $t\ge 0$, then for any bounded initial conditions, we have: 1) All signals in the closed-loop system remain uniformly ultimately bounded (UUB). 2) The position tracking error ${\eta }_i-{\eta }_{d,i}$converges exponentially to a small neighborhood around zero in finite time $T_i>0$by choosing the design parameters with sufficiently large $\underline{\lambda }(K_{1,i})>0$and $\underline{\lambda }(K_{2,i})>2\bar{\lambda }(K_{1,i})>0$, and sufficiently small ${\sigma }_{k,i}>0$for all $i\in \mathbf{I}[1,N]$and $k\in \mathbf{I}[\mathrm{1,3}]$.

Proof: 1) Consider the following Lyapunov function candidate for the closed-loop system (Equation 15):

$V_i=\frac{1}{2}{z}_{1,i}^T{z}_{1,i}+\frac{1}{2}{z}_{2,i}^T{M}_i{z}_{2,i}+\frac{1}{2}{\displaystyle \sum _{k=1}^3}{\tilde{W}}_{k,i}^T{\mathrm{\Gamma }}_{k,i}^{-1}{\tilde{W}}_{k,i}.$

Evaluating the derivative of $V_i$ along the trajectory of Equation 15 for all $i\in \mathbf{I}[1,N]$ yields:

$\begin{array}{cc}\hfill {\dot{V}}_i& =z_{1,i}^T\left(-K_{1,i}{z}_{1,i}+J_i\left({\eta }_i\right)z_{2,i}\right)\hfill \\ \hfill & +z_{2,i}^T\left.\left(-J_i^T\left({\eta }_i\right)z_{1,i}-K_{2,i}{z}_{2,i}+{\tilde{W}}_{k,i}^T{S}_{k,i}\left(Z_i\right)-{ϵ}_{k,i}\left(Z_i\right)\right)\right)\hfill \\ \hfill & -{\displaystyle \sum _{k=1}^3}{\tilde{W}}_{k,i}^T\left(S_{k,i}\left(Z_i\right)z_{2k,i}+{\sigma }_{k,i}{\widehat{W}}_{k,i}\right)\hfill \\ \hfill & =-z_{1,i}^T{K}_{1,i}{z}_{1,i}-z_{2,i}^T{K}_{2,i}{z}_{2,i}-z_{2,i}^T{ϵ}_{k,i}\left(Z_i\right)\hfill \\ \hfill & -{\displaystyle \sum _{k=1}^3}{\sigma }_{k,i}{\tilde{W}}_{k,i}^T{\widehat{W}}_{k,i},\forall i\in \mathbf{I}\left[1,N\right].\hfill \end{array}$

Choose $K_{2,i}=K_{1,i}+K_{22,i}$ such that $K_{1,i},K_{22,i}\in {\mathbb{S}}_3^{+}$. Using the completion of squares, we have:

$\begin{array}{cc}\hfill -{\sigma }_{k,i}{\tilde{W}}_{k,i}^T{\widehat{W}}_{k,i}& \le -\frac{{\sigma }_{k,i}\Vert {\tilde{W}}_{k,i}{\Vert }^2}{2}+\frac{{\sigma }_{k,i}\Vert W_{k,i}^{*}{\Vert }^2}{2},\hfill \\ \hfill -z_{2,i}^T{K}_{22,i}{z}_{2,i}-z_{2,i}^T{ϵ}_i\left(Z_i\right)& \le \frac{{ϵ}_i^T\left(Z_i\right){ϵ}_i\left(Z_i\right)}{4\underline{\lambda }\left(K_{22,i}\right)}\le \frac{\Vert {ϵ}_i^{*}{\Vert }^2}{4\underline{\lambda }\left(K_{22,i}\right)},\hfill \end{array}$

where ${ϵ}_i^{*}={[{ϵ}_{1,i}^{*},{ϵ}_{2,i}^{*},{ϵ}_{3,i}^{*}]}^T$. Then, we obtain:

$\begin{array}{cc}\hfill {\dot{V}}_i\le & -z_{1,i}^T{K}_{1,i}{z}_{1,i}-z_{2,i}^T{K}_{1,i}{z}_{2,i}+\frac{\Vert {ϵ}_i^{*}{\Vert }^2}{4\underline{\lambda }\left(K_{22,i}\right)}\hfill \\ \hfill & +{\displaystyle \sum _{k=1}^3}\left(-\frac{{\sigma }_{k,i}\Vert {\tilde{W}}_{k,i}{\Vert }^2}{2}+\frac{{\sigma }_{k,i}\Vert W_{k,i}^{*}{\Vert }^2}{2}\right..\hfill \end{array}$

It follows that ${\dot{V}}_i$ is negative definite whenever:

$\begin{array}{cc}\hfill \Vert z_{1,i}\Vert & >\frac{\Vert {ϵ}_i^{*}\Vert }{2\sqrt{\underline{\lambda }\left(K_{1,i}\right)\underline{\lambda }\left(K_{22,i}\right)}}+{\displaystyle \sum _{k=1}^3}\left(\sqrt{\frac{{\sigma }_{k,i}}{2\underline{\lambda }\left(K_{1,i}\right)}}\Vert W_{k,i}^{*}\Vert \right),\hfill \\ \hfill \Vert z_{2,i}\Vert & >\frac{\Vert {ϵ}_i^{*}\Vert }{2\sqrt{\underline{\lambda }\left(K_{1,i}\right)\underline{\lambda }\left(K_{22,i}\right)}}+{\displaystyle \sum _{k=1}^3}\left(\sqrt{\frac{{\sigma }_{k,i}}{2\underline{\lambda }\left(K_{1,i}\right)}}\Vert W_{k,i}^{*}\Vert \right),\hfill \\ \hfill \Vert {\tilde{W}}_{k,i}\Vert & >\frac{\Vert {ϵ}_i^{*}\Vert }{2\sqrt{{\sigma }_{k,i}\underline{\lambda }\left(K_{22,i}\right)}}+{\displaystyle \sum _{k=1}^3}\Vert W_{k,i}^{*}\Vert :={\tilde{W}}_{k,i}^{*}.\hfill \end{array}$

For all $i\in \mathbf{I}[1,N]$, $\exists k\in \mathbf{I}[\mathrm{1,3}]$. This leads to the Uniformly Ultimately Bounded (UUF) behavior of the signals $z_{1,i}$, $z_{2,i}$, and ${\tilde{W}}_{k,i}$ for all $i\in \mathbf{I}[1,N]$ and $k\in \mathbf{I}[\mathrm{1,3}]$. As a result, it can be easily verified that since ${\eta }_{d}i={\eta }_i+d_i^{*}$ with ${\eta }_i$ bounded (according to Theorem 1 and Assumption 1), ${\eta }_i=z_{1,i}+{\eta }_i$ is bounded for all $i\in \mathbf{I}[1,N]$. Similarly, the boundedness of ${\nu }_i=z_{2,i}+{\alpha }_i$ can be confirmed by the fact that ${\alpha }_i$ in Equation 9 is bounded. In addition, $W_{k,i}={\tilde{W}}_{k,i}+W_{k,i}^{*}$ is also bounded for all $i\in \mathbf{I}[1,N]$ and $k\in \mathbf{I}[\mathrm{1,3}]$ because of the boundedness of ${\tilde{W}}_{k,i}$ and $W_{k,i}^{*}$. Moreover, in light of Equation 10, ${\dot{\alpha }}_i$ is bounded as all the terms on the right-hand side of Equation 10 are bounded. This leads to the boundedness of the control signal ${\tau }_i$ in Equation 13 since the Gaussian function vector $S_i^F(Z_i)$ is guaranteed to be bounded for any $Z_i$. As such, all the signals in the closed-loop system remain UUB, which completes the proof of the first part.

2) For the second part, it will be shown that ${\eta }_i$ will converge arbitrarily close to ${\eta }_{di}$ in some finite time $T_i>0$ for all $i\in \mathbf{I}[1,N]$. To this end, we consider the following Lyapunov function candidate for the dynamics of $z_{1,i}$ and $z_{2,i}$ in Equation 15:

$V_{z,i}=\frac{1}{2}{z}_{1,i}^T{z}_{1,i}+\frac{1}{2}{z}_{2,i}^T{M}_i{z}_{2,i},\forall i\in \mathbf{I}\left[1,N\right].(16)$

The derivative of $V_{z,i}$ is:

$\begin{array}{ll}\hfill {\dot{V}}_{z,i}& =z_{1,i}^T\left(-K_{1,i}{z}_{1,i}+J_i\left({\eta }_i\right)z_{2,i}\right)+z_{2,i}^T\left(-J_i^T\left({\eta }_i\right)z_{1,i}-K_{2,i}{z}_{2,i}+{\tilde{W}}_{k,i}^T{S}_{k,i}\left(Z_i\right)-{ϵ}_i\left(Z_i\right)\right)\hfill \\ \hfill & =-z_{1,i}^T{K}_{1,i}{z}_{1,i}-z_{2,i}^T{K}_{2,i}{z}_{2,i}+z_{2,i}^T{\tilde{W}}_{k,i}^T{S}_{k,i}\left(Z_i\right)-z_{2,i}^T{ϵ}_{k,i}\left(Z_i\right),\forall i\in \mathbf{I}\left[1,N\right].\hfill \end{array}$

Similar to the proof of part one, we let $K_{2,i}=K_{1,i}+2K_{22,i}$ with $K_{1,i},K_{22,i}\in {\mathbb{S}}_3^{+}$. According to Wang and Hill (2018), the Gaussian RBFNN regressor $S_i^F(Z_i)$ is bounded by $\Vert S_i^F(Z_i)\Vert \le s_i^{*}$ for any $Z_i$ and for all $i\in \mathbf{I}[1,N]$ with some positive number $s_i^{*}>0$. Through completion of squares, we have:

$\begin{array}{cc}\hfill -z_{2,i}^T{K}_{22,i}{z}_{2,i}+z_{2,i}^T{\tilde{W}}_i^T{S}_i^F\left(Z_i\right)& \le \frac{\Vert {\tilde{W}}_i^{*}{\Vert }^2{s}_i^{*2}}{4\underline{\lambda }\left(K_{22,i}\right)},\hfill \\ \hfill -z_{2,i}^T{K}_{22,i}{z}_{2,i}-z_{2,i}^T{ϵ}_i\left(Z_i\right)& \le \frac{\Vert {ϵ}_i^{*}{\Vert }^2}{4\underline{\lambda }\left(K_{22,i}\right)}.\hfill \end{array}$

Also ${\tilde{W}}_i^{*}={[{\tilde{W}}_{1,i}^{*},{\tilde{W}}_{2,i}^{*},{\tilde{W}}_{3,i}^{*}]}^T$. This leads to:

$\begin{array}{cc}\hfill {\dot{V}}_{z,i}& \le -z_{1,i}^T{K}_{1,i}{z}_{1,i}-z_{2,i}^T{K}_{1,i}{z}_{2,i}+{\delta }_i,\hfill \\ \hfill & \le -2\underline{\lambda }\left(K_{1,i}\right)\left(\frac{1}{2}{z}_{1,i}^T{z}_{1,i}+\frac{1}{2\bar{\lambda }\left(M_i\right)}{z}_{2,i}^T{M}_i{z}_{2,i}\right)+{\delta }_i,\hfill \\ \hfill & \le -{\rho }_i{V}_{z,i}+{\delta }_i,\forall i\in \mathbf{I}\left[1,N\right],\hfill \end{array}(17)$

where ${\rho }_i=\min \{2\underline{\lambda }(K_{1,i}),2\underline{\lambda }(K_{1,i})/\bar{\lambda }(M_i)\}$ and ${\delta }_i=(\Vert {\tilde{W}}_i^{*}{\Vert }^2{s}_i^{*2}/4\underline{\lambda }(K_{22,i}))+(\Vert {ϵ}_i^{*}{\Vert }^2/4\underline{\lambda }(K_{22,i}))$, $\forall i\in \mathbf{I}[1,N]$. Solving the inequality Equation 17 yields:

$0\le V_{z,i}\left(t\right)\le V_{z,i}\left(0\right)\exp \left(-{\rho }_{i}t\right)+\frac{{\delta }_i}{{\rho }_i},$

which together with Equation 16 implies that:

$\min \left\{1,\underline{\lambda }\left(M_i\right)\right\}\frac{1}{2}\left(\Vert z_{1,i}{\Vert }^2+\Vert z_{2,i}{\Vert }^2\right)\le V_{z,i}\left(0\right)\exp \left(-{\rho }_{i}t\right)+\frac{{\delta }_i}{{\rho }_i},\forall t\ge 0,i\in \mathbf{I}\left[1,N\right],$

also

$\Vert z_{1,i}{\Vert }^2+\Vert z_{2,i}{\Vert }^2\le \frac{2}{\min \left\{1,\underline{\lambda }\left(M_i\right)\right\}}{V}_{z,i}\left(0\right)\exp \left(-{\rho }_{i}t\right)+\frac{2{\delta }_i}{{\rho }_i\min \left\{1,\underline{\lambda }\left(M_i\right)\right\}}.$

Consequently, it is straightforward that given ${\bar{\delta }}_i>\sqrt{2{\delta }_i/{\rho }_i\min \{1,\underline{\lambda }(M_i)\}}$, there exists a finite time $T_i>0$ for all $i\in \mathbf{I}[1,N]$ such that for all $t\ge T_i$, both $z_{1,i}$ and $z_{2,i}$ satisfy $\Vert z_{1,i}(t)\Vert \le {\bar{\delta }}_i$ and $\Vert z_{2,i}(t)\Vert \le {\bar{\delta }}_i\forall i\in \mathbf{I}[1,N]$, where ${\bar{\delta }}_i$ can be made arbitrarily small by choosing sufficiently large $\underline{\lambda }(K_{1,i})>0$ and $\underline{\lambda }(K_{2,i})>2\bar{\lambda }(K_{1,i})>0$ for all $i\in \mathbf{I}[1,N]$. This ends the proof.

By integrating the outcomes of Theorems 1, 2, the following theorem is established, which can be presented without additional proof:

Theorem 3. By Considering the multi-AUV system (Equation 1) and the virtual leader dynamics (Equation 2) with the network communication topology $G$ and under Assumptions 1 and 2, the objective 1 of Problem 1 (i.e., ${\eta }_i$ converges to ${\eta }_0+d_i^{*}$ exponentially for all $i\in \mathbf{I}[1,N]$) can be achieved by using the cooperative observer Equations 3, 4 and the DDL control law Equations 13, 14 with all the design parameters satisfying the requirements in Theorems 1 and 2, respectively.

Remark 5. With the proposed two-layer formation learning control architecture, inter-agent information exchange occurs solely in the first-layer DA observation. Only the observer’s estimated information, and not the physical plant state information, needs to be shared among neighboring agents. Additionally, since no global information is required for the design of each local AUV control system, the proposed formation learning control protocol can be designed and implemented in a fully distributed manner.

Remark 6. It is important to note that the eigenvalue constraints on $A_0$ in Assumption 1 are not needed for cooperative observer estimation (as detailed in the Section 3 or for achieving formation tracking control performance (as discussed in this section). This indicates that formation tracking control can be attained for general reference trajectories, including both periodic paths and straight lines, provided they are bounded. However, these constraints will become necessary in the next section to ensure the accurate learning capability of the proposed method.

4 Accurate learning from formation control

It is necessary to demonstrate the convergence of the RBFNN weights in Equations 13, 14 to their optimal values for accurate learning and identification. The main result of this section is summarized in the following theorem.

Theorem 4. Consider the local closed-loop system (Equation 15) with Assumptions 1, 2. For each $i\in \mathbf{I}[1,N]$, if there exists a sufficiently large compact set ${\mathrm{\Omega }}_{Z_i}$ such that $Z_i\in {\mathrm{\Omega }}_{Z_i}$ for all $t\ge 0$, then for any bounded initial conditions and $W_{k,i}(0)=0\forall i\in \mathbf{I}[1,N],k\in \mathbf{I}[\mathrm{1,3}]$, the local estimated neural weights $W_{\zeta ,k,i}$ converge to small neighborhoods of their optimal values $W_{\zeta ,k,i}^{*}$ along the periodic reference tracking orbit ${\varphi }_{\zeta ,i}(Z_i(t)){|}_{t\ge T_i}$ (denoting the orbit of the NN input signal $Z_i(t)$ starting from time $T_i$). This leads to locally accurate approximations of the nonlinear uncertain dynamics $f_{k,i}(Z_i)\forall k\in \mathbf{I}[\mathrm{1,3}]$ in Equation 11 being obtained by $W_{k,i}^T{S}_{k,i}(Z_i)$, as well as by ${\bar{W}}_{k,i}^T{S}_{k,i}(Z_i)$, where $\forall i\in \mathbf{I}[1,N],k\in \mathbf{I}[\mathrm{1,3}]$. Also, $\zeta$ denotes the subset of neurons (or nodes) that are active when the system state $Z(t)$ is within a specific neighborhood of the state space.

${\bar{W}}_{k,i}={\mathrm{m}\mathrm{e}\mathrm{a}\mathrm{n}}_{t\in \left[t_{a,i},t_{b,i}\right]}{\widehat{W}}_{k,i}\left(t\right),(18)$

where $[t_{a,i},t_{b,i}]$ $(t_{b,i}>t_{a,i}>T_i)$ represents a time segment after the transient process.

Proof: From Theorem 3, we have shown that for all $i\in \mathbf{I}[1,N]$, ${\eta }_i$ will closely track the periodic signal ${\eta }_{d,i}={\eta }_0+d_i^{*}$ in finite time $T_i$. In addition, (Equation 9) implies that ${\nu }_i$ will also closely track the signal $J_i^T({\eta }_i){\dot{\eta }}_0^i$ since both $z_{1,i}$ and $z_{2,i}$ will converge to a small neighborhood around zero according to Theorem 2. Moreover, since ${\dot{\eta }}_0^i$ will converge to ${\dot{\eta }}_0$ according to Theorem 1, and $J_i({\eta }_i)$ is a bounded rotation matrix, ${\nu }_i$ will also be a periodic signal after finite time $T_i$, because ${\dot{\eta }}_0$ is periodic under Assumption 1. Consequently, since the RBFNN input $Z_i(t)=\mathrm{c}\mathrm{o}\mathrm{l}\{{\eta }_i,{\nu }_i\}$ becomes a periodic signal for all $t\le T_i$, the PE condition of some internal closed-loop signals, i.e., the RBFNN regression subvector $S_{\zeta ,k,i}(Z_i)$ $(\forall t\ge T_i)$, is satisfied according to Lemma 1. As mentioned in Section 2.2, $\zeta$ represents the subset of RBFNN nodes and weights that are specifically utilized along the recurrent trajectory of the system state $Z_i(t)$. This subset focuses on the active neural components required for approximating the system’s nonlinear dynamics locally, ensuring that the learning and adaptation processes are efficient and accurate within the compact region where the trajectory resides. It should be noted that the periodicity of $Z_i(t)$ leads to the PE of the regression subvector $S_{\zeta ,k,i}(Z_i)$, but not necessarily the PE of the whole regression vector $S_{k,i}(Z_i)$. Thus, we term this as a partial PE condition, and we will show the convergence of the associated local estimated neural weights $W_{\zeta ,k,i}\to W_{\zeta ,k,i}^{*}$, rather than $W_{k,i}\to W_{k,i}^{*}$.

Thus, to prove accurate convergence of local neural weights $W_{\zeta ,k,i}$ associated with the regression subvector $S_{\zeta ,k,i}(Z_i)$ under the satisfaction of the partial PE condition, we first rewrite the closed-loop dynamics of $z_{1,i}$ and $z_{2,i}$ along the periodic tracking orbit ${\varphi }_{\zeta ,i}(Z_i(t)){|}_{t\ge T_i}$ by using the localization property of the Gaussian RBFNN:

$\begin{array}{ll}\hfill {\dot{z}}_{1,i}& =-K_{1,i}{z}_{1,i}+J_i\left({\eta }_i\right)z_{2,i},\hfill \\ \hfill {\dot{z}}_{2,i}& =M_i^{-1}\left(-W_{\zeta ,i}^{*T}{S}_{\zeta ,i}^F\left(Z_i\right)-{ϵ}_{\zeta ,i}-J_i^T\left({\eta }_i\right)z_{1,i}-K_{2,i}{z}_{2,i}+{\widehat{W}}_{\zeta ,i}^T{S}_{\zeta ,i}^F\left(Z_i\right)+{\widehat{W}}_{\bar{\zeta },i}^T{S}_{\bar{\zeta },i}^F\left(Z_i\right)\right)\hfill \\ \hfill & =M_i^{-1}\left(-J_i^T\left({\eta }_i\right)z_{1,i}-K_{2,i}{z}_{2,i}+{\tilde{W}}_{\zeta ,i}^T{S}_{\zeta ,i}^F\left(Z_i\right)-{ϵ}_{\zeta ,i}^{\prime }\right).\hfill \end{array}$

where $F_i(Z_i)=W_{\zeta ,i}^{*T}{S}_{\zeta ,i}^F(Z_i)+{ϵ}_{\zeta ,i}$ with $W_{\zeta ,i}^{*T}{S}_{\zeta ,i}^F(Z_i)={[W_{\zeta ,1,i}^{*T}{S}_{\zeta ,1,i}(Z_i),W_{\zeta ,2,i}^{*T}{S}_{\zeta ,2,i}(Z_i),W_{\zeta ,3,i}^{*T}{S}_{\zeta ,3,i}(Z_i)]}^T$ and ${ϵ}_{\zeta ,i}={[{ϵ}_{\zeta ,1,i},{ϵ}_{\zeta ,2,i},{ϵ}_{\zeta ,3,i}]}^T$ being the approximation error. Additionally, $W_{\zeta ,i}^T{S}_{\zeta ,i}^F(Z_i)+W_{\bar{\zeta },i}^T{S}_{\bar{\zeta },i}^F(Z_i)=W_i^T{S}_i^F(Z_i)$ with subscripts $\zeta$ and $\bar{\zeta }$ denoting the regions close to and far away from the periodic trajectory ${\varphi }_{\zeta ,i}(Z_i(t)){|}_{t\ge T_i}$, respectively. According to Wang and Hill (2018), $\Vert W_{\bar{\zeta },i}^T{S}_{\bar{\zeta },i}^F(Z_i)\Vert$ is small, and the NN local approximation error ${ϵ}_{\zeta ,i}^{\prime }={ϵ}_{\zeta ,i}-W_{\bar{\zeta },i}^T{S}_{\bar{\zeta },i}^F(Z_i)$ with $\Vert {ϵ}_{\zeta ,i}^{\prime }\Vert =O(\Vert {ϵ}_{\zeta ,i}\Vert )$ is also a small number. Thus, the overall closed-loop adaptive learning system can be described by:

and

$\begin{array}{cc}\hfill \left[\begin{array}{c}\hfill {\dot{\stackrel{̃}{W}}}_{\bar{\zeta },i,1}\hfill \\ \hfill {\dot{\stackrel{̃}{W}}}_{\bar{\zeta },i,2}\hfill \\ \hfill {\dot{\stackrel{̃}{W}}}_{\bar{\zeta },i,3}\hfill \end{array}\right]& =\left[\begin{array}{c}\hfill -{\mathrm{\Gamma }}_{\bar{\zeta },1,i}\left(S_{\bar{\zeta },1,i}\left(Z_i\right)z_{2,i}+{\sigma }_{i,1}{\widehat{W}}_{\bar{\zeta },1,i}\right)\hfill \\ \hfill -{\mathrm{\Gamma }}_{\bar{\zeta },2,i}\left(S_{\bar{\zeta },2,i}\left(Z_i\right)z_{2,i}+{\sigma }_{i,2}{\widehat{W}}_{\bar{\zeta },2,i}\right)\hfill \\ \hfill -{\mathrm{\Gamma }}_{\bar{\zeta },3,i}\left(S_{\bar{\zeta },3,i}\left(Z_i\right)z_{2,i}+{\sigma }_{i,3}{\widehat{W}}_{\bar{\zeta },3,i}\right)\hfill \end{array}\right],\hfill \end{array}$

where

$\begin{array}{cc}\hfill {\mathrm{\Xi }}_i& =\left[\begin{array}{c}\hfill 0\hfill \\ \hfill M_i^{-1}\left[\begin{array}{ccc}\hfill S_{\zeta ,1,i}^T\left(Z_i\right)\hfill & \hfill 0\hfill & \hfill 0\hfill \\ \hfill 0\hfill & \hfill S_{\zeta ,2,i}^T\left(Z_i\right)\hfill & \hfill 0\hfill \\ \hfill 0\hfill & \hfill 0\hfill & \hfill S_{\zeta ,3,i}^T\left(Z_i\right)\hfill \end{array}\right]\hfill \end{array}\right],\hfill \end{array}$

for all $i\in \mathbf{I}[1,N]$. The exponential stability property of the nominal part of subsystem (Equation 19) has been well-studied in Wang and Hill (2018), Yuan and Wang (2011), and Yuan and Wang (2012), where it is stated that PE of $S_{\zeta ,k,i}(Z_i)$ will guarantee exponential convergence of $(z_{1,i},z_{2,i},{\tilde{W}}_{\zeta ,k,i})=0$ for all $i\in \mathbf{I}[1,N]$ and $k\in \mathbf{I}[\mathrm{1,3}]$. Based on this, since $\Vert {ϵ}_{\zeta ,i}^{\prime }\Vert =O(\Vert {ϵ}_{\zeta ,i}\Vert )=O(\Vert {ϵ}_i\Vert )$, and ${\sigma }_{k,i}{\mathrm{\Gamma }}_{\zeta ,k,i}{\widehat{W}}_{\zeta ,k,i}$ can be made small by choosing sufficiently small ${\sigma }_{k,i}$ for all $i\in \mathbf{I}[1,N]$, $k\in \mathbf{I}[\mathrm{1,3}]$, both the state error signals $(z_{1,i},z_{2,i})$ and the local parameter error signals ${\tilde{W}}_{\zeta ,k,i}$ $(\forall i\in \mathbf{I}[1,N],k\in \mathbf{I}[\mathrm{1,3}])$ will converge exponentially to small neighborhoods of zero, with the sizes of the neighborhoods determined by the RBFNN ideal approximation error ${ϵ}_i$ as in Equation 12 and ${\sigma }_{k,i}\Vert {\mathrm{\Gamma }}_{\zeta ,k,i}{\widehat{W}}_{\zeta ,k,i}\Vert$. The convergence of $W_{\zeta ,k,i}\to W_{\zeta ,k,i}^{*}$ implies that along the periodic trajectory ${\varphi }_{\zeta ,i}(Z_i(t)){|}_{t\ge T_i}$, we have

$\begin{array}{cc}\hfill f_{k,i}\left(Z_i\right)& =W_{\zeta ,k,i}^{*T}{S}_{\zeta ,k,i}\left(Z_i\right)+{ϵ}_{\zeta ,k,i}\hfill \\ \hfill & ={\widehat{W}}_{\zeta ,k,i}^T{S}_{\zeta ,k,i}\left(Z_i\right)-{\tilde{W}}_{\zeta ,k,i}^T{S}_{\zeta ,k,i}\left(Z_i\right)+{ϵ}_{\zeta ,k,i}\hfill \\ \hfill & ={\widehat{W}}_{\zeta ,k,i}^T{S}_{\zeta ,k,i}\left(Z_i\right)+{ϵ}_{{\zeta }_1,k,i}\hfill \\ \hfill & ={\bar{W}}_{\zeta ,k,i}^T{S}_{\zeta ,k,i}\left(Z_i\right)+{ϵ}_{{\zeta }_2,k,i},\hfill \end{array}$

where for all $i\in \mathbf{I}[1,N]$, $k\in \mathbf{I}[\mathrm{1,3}]$, ${ϵ}_{{\zeta }_1,k,i}={ϵ}_{\zeta ,k,i}-{\tilde{W}}_{\zeta ,k,i}^T{S}_{\zeta ,k,i}(Z_i)=O(\Vert {ϵ}_{\zeta ,i}\Vert )$ due to the convergence of ${\tilde{W}}_{\zeta ,k,i}\to 0$. The last equality is obtained according to the definition of (Equation 18) with ${\bar{W}}_{\zeta ,k,i}$ being the corresponding subvector of ${\bar{W}}_{k,i}$ along the periodic trajectory ${\varphi }_{\zeta ,i}(Z_i(t)){|}_{t\ge T_i}$, and ${ϵ}_{{\zeta }_2,k,i}$ being an approximation error using ${\bar{W}}_{\zeta ,k,i}^T{S}_{\zeta ,k,i}(Z_i)$. Apparently, after the transient process, we will have ${ϵ}_{{\zeta }_2,k,i}=O({ϵ}_{{\zeta }_1,k,i})$, $\forall i\in \mathbf{I}[1,N]$, $k\in \mathbf{I}[\mathrm{1,3}]$. Conversely, for the neurons whose centers are distant from the trajectory ${\varphi }_{\zeta ,i}(Z_i(t)){|}_{t\ge T_i}$, the values of $\Vert S_{\bar{\zeta },k,i}(Z_i)\Vert$ will be very small due to the localization property of Gaussian RBFNN. From the adaptation law (Equation 13) with $W_i^k(0)=0$, it can be observed that these small values of $S_{\bar{\zeta },k,i}(Z_i)$ will only minimally activate the adaptation of the associated neural weights $W_{\bar{\zeta },k,i}$. As a result, both $W_{\bar{\zeta },k,i}$ and $W_{\bar{\zeta },k,i}^T{S}_{\bar{\zeta },k,i}(Z_i)$, as well as ${\bar{W}}_{\bar{\zeta },k,i}$ and ${\bar{W}}_{\bar{\zeta },k,i}^T{S}_{\bar{\zeta },k,i}(Z_i)$, will remain very small for all $i\in \mathbf{I}[1,N]$, $k\in \mathbf{I}[\mathrm{1,3}]$ along the periodic trajectory ${\varphi }_{\zeta ,i}(Z_i(t)){|}_{t\ge T_i}$. This indicates that the entire RBFNN $W_{k,i}^T{S}_{k,i}(Z_i)$ and ${\bar{W}}_{k,i}^T{S}_{k,i}(Z_i)$ can be used to accurately approximate the unknown function $f_{k,i}(Z_i)$ locally along the periodic trajectory ${\varphi }_{\zeta ,i}(Z_i(t)){|}_{t\ge T_i}$, meaning that.

$f_{k,i}\left(Z_i\right)={\widehat{W}}_{\zeta ,k,i}^T{S}_{\zeta ,k,i}\left(Z_i\right)+{ϵ}_{{\zeta }_1,k,i}={\widehat{W}}_{k,i}^T{S}_{k,i}\left(Z_i\right)+{ϵ}_{1,k,i}$

$={\bar{W}}_{\zeta ,k,i}^T{S}_{\zeta ,k,i}\left(Z_i\right)+{ϵ}_{{\zeta }_2,k,i}={\bar{W}}_{k,i}^T{S}_{k,i}\left(Z_i\right)+{ϵ}_{2,k,i},$

with the approximation accuracy level of ${ϵ}_{1,k,i}={ϵ}_{{\zeta }_1,k,i}-W_{\bar{\zeta },k,i}^T{S}_{\bar{\zeta },k,i}(Z_i)=O({ϵ}_{{\zeta }_1,k,i})=O({ϵ}_{k,i})$ and ${ϵ}_{2,k,i}={ϵ}_{{\zeta }_2,k,i}-{\bar{W}}_{\bar{\zeta },k,i}^T{S}_{\bar{\zeta },k,i}(Z_i)=O({ϵ}_{{\zeta }_2,k,i})=O({ϵ}_{k,i})$ for all $i\in \mathbf{I}[1,N]$, $k\in \mathbf{I}[\mathrm{1,3}]$. This ends the proof.

Remark 7. The key idea in the proof of Theorem 4 is inspired by Wang and Hill (2018). For more detailed analysis on the learning performance, including quantitative analysis on the learning accuracy levels ${ϵ}_{1,i,k}$ and ${ϵ}_{2,i,k}$ as well as the learning speed, please refer to Yuan and Wang (2011). Furthermore, the AUV nonlinear dynamics (Equation 9) to be identified do not contain any time-varying random disturbances. This is important to ensure accurate identification/learning performance under the deterministic learning framework. To understand the effects of time-varying external disturbances on deterministic learning performance, interested readers are referred to Yuan and Wang (2012) for more details.

Remark 8. Based on Equation 18, to obtain the constant RBFNN weights ${\bar{W}}_{k,i}$ for all $i\in \mathbf{I}[1,N]$, $k\in \mathbf{I}[\mathrm{1,3}]$, one needs to implement the formation learning control law Equations 13, 14 first. Then, according to Theorem 4, after a finite-time transient process, the RBFNN weights $W_{k,i}$ will converge to constant steady-state values. Thus, one can select a time segment $[t_{a,i},t_{b,i}]$ with $t_{b,i}>t_{a,i}>T_i$ for all $i\in \mathbf{I}[1,N]$ to record and store the RBFNN weights $W_{k,i}(t)$ for $t\in [t_{a,i},t_{b,i}]$. Finally, based on these recorded data, ${\bar{W}}_{k,i}$ can be calculated off-line using Equation 18.

Remark 9. It is shown in Theorem 4 that locally accurate learning of each individual AUV’s nonlinear uncertain dynamics can be achieved using localized RBFNNs along the periodic trajectory ${\varphi }_{\zeta ,i}(Z_i(t)){|}_{t\ge T_i}$. The learned knowledge can be further represented and stored in a time-invariant fashion using constant RBFNN, i.e., ${\bar{W}}_{k,i}^T{S}_{k,i}(Z_i)$ for all $i\in \mathbf{I}[1,N]$, $k\in \mathbf{I}[\mathrm{1,3}]$. In contrast to many existing techniques (e.g., Peng et al., 2017; Peng et al., 2015), this is the first time, to the authors’ best knowledge, that locally accurate identification and knowledge representation using constant RBFNN are accomplished and rigorously analyzed for multi-AUV formation control under complete uncertain dynamics.

5 Formation control with pre-learned dynamics

In this section, we will further address objective 2 of Problem 1, which involves achieving formation control without readapting to the AUV’s nonlinear uncertain dynamics. To this end, consider the multiple AUV systems (Equation 1) and the virtual leader dynamics (Equation 2). We employ the estimator observer Equations 3, 4 to cooperatively estimate the leader’s state information. Instead of using the DDL feedback control law (Equation 13), and self-adaptation law (Equation 4), we introduce the following constant RBFNN controller, which does not require online adaptation of the NN weights:

${\tau }_i=-J_i^T\left({\eta }_i\right)z_{1,i}-K_{2,i}{z}_{2,i}+{\bar{W}}_i^T{S}_i^F\left(Z_i\right),(20)$

where ${\bar{W}}_i^T{S}_i^F(Z_i)={[{\bar{W}}_{1,i}^T{S}_{1,i}(Z_i),{\bar{W}}_{2,i}^T{S}_{2,i}(Z_i),{\bar{W}}_{3,i}^T{S}_{3,i}(Z_i)]}^T$ is obtained from Equation 18. The term ${\bar{W}}_{k,i}^T{S}_{k,i}(Z_i)$ represents the locally accurate RBFNN approximation of the nonlinear uncertain function $f_{k,i}(Z_i)$ along the trajectory ${\varphi }_{\zeta ,i}(Z_i(t)){|}_{t\ge T_i}$, and the associated constant neural weights ${\bar{W}}_{k,i}$ are obtained from the formation learning control process as discussed in Remark 8.

Theorem 5. Consider the multi-AUV system (Equation 1) and the virtual leader dynamics (Equation 3) with the network communication topology $G$. Under Assumptions 1, 2, the formation control performance (i.e., ${\eta }_i$ converges to ${\eta }_0+d_i^{*}$ exponentially with the same ${\eta }_0$ and $d_i^{*}$ defined in Theorem 3 for all $i\in \mathbf{I}[1,N]$) can be achieved by using the DA observer Equations 3, 20 and the constant RBFNN control law (Equation 4) with the constant NN weights obtained from Equation 18.

Proof: The closed-loop system for each local AUV agent can be established by integrating the controller (Equation 20) with the AUV dynamics (Equation 1).

$\begin{array}{cc}\hfill {\dot{z}}_{1,i}=& -K_{1,i}{z}_{1,i}+J_i\left({\eta }_i\right)z_{2,i},\hfill \\ \hfill {\dot{z}}_{2,i}=& M_i^{-1}\left(-J_i^T\left({\eta }_i\right)z_{1,i}-K_{2,i}{z}_{2,i}+{\bar{W}}_i^T{S}_i^F\left(Z_i\right)-F_i\left(Z_i\right)\right)\hfill \\ \hfill =& M_i^{-1}\left(-J_i^T\left({\eta }_i\right)z_{1,i}-K_{2,i}{z}_{2,i}-{ϵ}_{2,i}\right),\forall i\in \mathbf{I}\left[1,N\right],\hfill \end{array}$

where ${ϵ}_{2,i}={[{ϵ}_{21,i},{ϵ}_{22,i},{ϵ}_{23,i}]}^T$. Consider the Lyapunov function candidate $V_{z,i}=\frac{1}{2}{z}_{1,i}^T{z}_{1,i}+\frac{1}{2}{z}_{2,i}^T{M}_i{z}_{2,i}$, whose derivative along the closed-loop system described is given by:

$\begin{array}{cc}\hfill {\dot{V}}_{z,i}=& z_{1,i}^T\left(-K_{1,i}{z}_{1,i}+J_i\left({\eta }_i\right)z_{2,i}\right)+z_{2,i}^T\left(-J_i^T\left({\eta }_i\right)z_{1,i}-K_{2,i}{z}_{2,i}-{ϵ}_{2,i}\right)\hfill \\ \hfill =& -z_{1,i}^T{K}_{1,i}{z}_{1,i}-z_{2,i}^T{K}_{2,i}{z}_{2,i}-z_{2,i}^T{ϵ}_{2,i}.\hfill \end{array}$

Selecting $K_{2,i}=K_{1,i}+K_{22,i}$ where $K_{1,i},K_{22,i}\in {\mathbb{S}}_3^{+}$, we can utilize the method of completing squares to obtain:

$-z_{2,i}^T{K}_{22,i}{z}_{2,i}-z_{2,i}^T{ϵ}_{2,i}\le \left(\frac{\Vert {ϵ}_{2,i}{\Vert }^2}{4\underline{\lambda }\left(K_{22,i}\right)}\right)\le \left(\frac{\Vert {ϵ}_{2,i}^{*}{\Vert }^2}{4\underline{\lambda }\left(K_{22,i}\right)}\right),$

which implies that:

${\dot{V}}_{z,i}\le -z_{1,i}^T{K}_{1,i}{z}_{1,i}-z_{2,i}^T{K}_{1,i}{z}_{2,i}+\left(\frac{\Vert {ϵ}_{2,i}^{*}{\Vert }^2}{4\underline{\lambda }\left(K_{22,i}\right)}\right)\le -{\rho }_i{V}_{z,i}+{\delta }_i,\forall i\in \mathbf{I}\left[1,N\right].$

where ${\rho }_i=\min \{2\underline{\lambda }(K_{1,i}),(2\underline{\lambda }(K_{1,i})/\bar{\lambda }(M_i))\}$ and ${\delta }_i=(\Vert {ϵ}_{2,i}^{*}{\Vert }^2/4\underline{\lambda }(K_{22,i}))$. Using similar reasoning to that in the proof of Theorem 2, it is evident from the derived inequality that all signals within the closed-loop system remain bounded. Additionally, ${\eta }_i-{\eta }_i^d$ will converge to a small neighborhood around zero within a finite period. The magnitude of this neighborhood can be minimized by appropriately choosing large values for $\underline{\lambda }(K_{1,i})>0$ and $\underline{\lambda }(K_{2,i})>\bar{\lambda }(K_{1,i})$ across all $i\in \mathbf{I}[1,N]$. In line with Theorem 1, under Assumptions 1, 2, the implementation of the DA observer DA observer Equations 3, 4 facilitates the exponential convergence of ${\eta }_i$ towards ${\eta }_0$. This conjunction of factors assures that ${\eta }_i$ rapidly aligns with ${\eta }_{d,i}={\eta }_0+d_i^{*}$, achieving the objectives set out for formation control.

Remark 10. Building on the locally accurate learning outcomes discussed in Section 4, the newly developed distributed control protocol comprising Equations 3, 4, 20 facilitates stable formation control across a repeated formation pattern. Unlike the formation learning control approach outlined in Section 3.2, which involves Equations 3, 4 coupled with Equations 13, 14, the current method eliminates the need for online RBFNN adaptation for all AUV agents. This significantly reduces the computational demands, thereby enhancing the practicality of implementing the proposed distributed RBFNN formation control protocol. This innovation marks a significant advancement over many existing techniques in the field.

6 Simulation

We consider a multi-AUV heterogeneous system composed of 5 AUVs for the simulation. The dynamics of these AUVs are described in THE system (Equation 1). The system parameters for each AUV are specified as follows:

$\begin{array}{cc}\hfill M_i& =\left[\begin{array}{ccc}\hfill m_{11,i}\hfill & \hfill 0\hfill & \hfill 0\hfill \\ \hfill 0\hfill & \hfill m_{22,i}\hfill & \hfill m_{23,i}\hfill \\ \hfill 0\hfill & \hfill m_{23,i}\hfill & \hfill m_{33,i}\hfill \end{array}\right],\hfill \\ \hfill C_i& =\left[\begin{array}{ccc}\hfill 0\hfill & \hfill 0\hfill & \hfill -m_{22,i}{v}_i-m_{23,i}{r}_i\hfill \\ \hfill 0\hfill & \hfill 0\hfill & \hfill -m_{11,i}{u}_i\hfill \\ \hfill m_{22,i}{v}_i+m_{23,i}{r}_i\hfill & \hfill -m_{11,i}{u}_i\hfill & \hfill 0\hfill \end{array}\right],\hfill \\ \hfill D_i& =\left[\begin{array}{ccc}\hfill d_{11,i}\left({\nu }_i\right)\hfill & \hfill 0\hfill & \hfill 0\hfill \\ \hfill 0\hfill & \hfill d_{22,i}\left({\nu }_i\right)\hfill & \hfill d_{23,i}\left({\nu }_i\right)\hfill \\ \hfill 0\hfill & \hfill d_{32,i}\left({\nu }_i\right)\hfill & \hfill d_{33,i}\left({\nu }_i\right)\hfill \end{array}\right],g_i=0,\hfill \\ \hfill {\mathrm{\Delta }}_i& =\left[\begin{array}{c}\hfill {\mathrm{\Delta }}_{1,i}\left({\chi }_i\right)\hfill \\ \hfill {\mathrm{\Delta }}_{2,i}\left({\chi }_i\right)\hfill \\ \hfill {\mathrm{\Delta }}_{3,i}\left({\chi }_i\right)\hfill \end{array}\right],\forall i\in \mathbf{I}\left[\mathrm{1,5}\right],\hfill \end{array}$

where the mass and damping matrix components for each AUV $i$ are defined as:

$\begin{array}{llll}\hfill m_{11,i}& =m_i-X_{\dot{u},i},\hfill & \hfill m_{22,i}& =m_i-Y_{\dot{v},i},\hfill \\ \hfill m_{23,i}& =m_i{x}_{g,i}-Y_{\dot{r},i},\hfill & \hfill m_{33,i}& =I_{z,i}-N_{\dot{r},i},\hfill \\ \hfill d_{11,i}& =-\left(X_{u,i}+X_{uu,i}\Vert u_i\Vert \right),\hfill & \hfill d_{22,i}& =-\left(Y_{v,i}+Y_{vv,i}\Vert v_i\Vert +Y_{rv,i}\Vert r_i\Vert \right),\hfill \\ \hfill d_{23,i}& =-\left(Y_{r,i}+Y_{vr,i}\Vert v_i\Vert +Y_{rr,i}\Vert r_i\Vert \right),\hfill & \hfill d_{32,i}& =-\left(N_{v,i}+N_{vv,i}\Vert v_i\Vert +N_{rv,i}\Vert r_i\Vert \right),\hfill \\ \hfill d_{33,i}& =-\left(N_{r,i}+N_{vr,i}\Vert v_i\Vert +N_{rr,i}\Vert r_i\Vert \right).\hfill & \hfill \end{array}$

According to the notations in Prestero (2001) and Skjetne et al. (2005) the coefficients $\{X(\cdot ),Y(\cdot ),N(\cdot )\}$ are hydrodynamic parameters. For the associated system parameters are borrowed from Skjetne et al. (2005) (with slight modifications for different AUV agents) and simulation purposes and listed in Table 1. For all $i\in \mathbf{I}[\mathrm{1,5}]$, we set $x_{g,i}=0.05$ and $Y_{\dot{r},i}=Y_{rv,i}=Y_{vr,i}=Y_{rr,i}=N_{rv,i}=N_{rr,i}=N_{vv,i}=N_{vr,i}=N_{r,i}=0$. Model uncertainties are given by:

$\begin{array}{ll}\hfill {\mathrm{\Delta }}_1& =0,{\mathrm{\Delta }}_2={\left[\begin{array}{ccc}\hfill 0.2u_2^2+0.3v_2\hfill & \hfill -0.95\hfill & \hfill 0.33\Vert r_2\Vert \hfill \end{array}\right]}^T\hfill \\ \hfill {\mathrm{\Delta }}_3& ={\left[\begin{array}{ccc}\hfill -0.58+\cos \left(v_3\right)\hfill & \hfill 0.23r_3^3\hfill & \hfill 0.74u_3^2\hfill \end{array}\right]}^T\hfill \\ \hfill {\mathrm{\Delta }}_4& ={\left[\begin{array}{ccc}\hfill -0.31\hfill & \hfill 0\hfill & \hfill 0.38u_4^2+v_4^3\hfill \end{array}\right]}^T\hfill \\ \hfill {\mathrm{\Delta }}_5& ={\left[\begin{array}{ccc}\hfill \sin \left(v_5\right)\hfill & \hfill \cos \left(u_5+r_5\right)\hfill & \hfill -0.65\hfill \end{array}\right]}^T.\hfill \end{array}$

Table 1

Table 1. Parameters of AUVs.

Figure 2 illustrates the communication topology and the spanning tree where agent 0 is the virtual leader and is considered as the root, in accordance with Assumption 2. The desired formation pattern requires each AUV, ${\eta }_i$, to track a periodic signal generated by the virtual leader ${\eta }_0$. The dynamics of the leader are defined as follows:

$\begin{array}{cc}\hfill \left[\begin{array}{c}\hfill {\dot{\eta }}_0\hfill \\ \hfill {\dot{\nu }}_0\hfill \end{array}\right]=& \left[\begin{array}{cc}\hfill 0\hfill & \hfill \left[\begin{array}{ccc}\hfill 1\hfill & \hfill 0\hfill & \hfill 0\hfill \\ \hfill 0\hfill & \hfill -1\hfill & \hfill 0\hfill \\ \hfill 0\hfill & \hfill 0\hfill & \hfill 1\hfill \end{array}\right]\hfill \\ \hfill \left[\begin{array}{ccc}\hfill -1\hfill & \hfill 0\hfill & \hfill 0\hfill \\ \hfill 0\hfill & \hfill 1\hfill & \hfill 0\hfill \\ \hfill 0\hfill & \hfill 0\hfill & \hfill -1\hfill \end{array}\right]\hfill & \hfill 0\hfill \end{array}\right]\left[\begin{array}{c}\hfill {\eta }_0\hfill \\ \hfill {\nu }_0\hfill \end{array}\right],\hfill \\ \hfill \left[\begin{array}{c}\hfill {\eta }_0\left(0\right)\hfill \\ \hfill {\nu }_0\left(0\right)\hfill \end{array}\right]=& {\left[\begin{array}{cccccc}\hfill 0\hfill & \hfill 80\hfill & \hfill 0\hfill & \hfill 80\hfill & \hfill 0\hfill & \hfill 80\hfill \end{array}\right]}^T.\hfill \end{array}(21)$

Figure 2

Figure 2. The communication network topology and spanning tree of multi-AUV system of the simulation with 0 as virtual leader.

The initial conditions and system matrix are structured to ensure all eigenvalues of $A_0$ lie on the imaginary axis, thus satisfying Assumption 1. The reference trajectory for ${\eta }_0$ is defined as ${[80\sin (t),80\cos (t),80\sin (t)]}^T$. The predefined offsets $d_i^{*}$, which determine the relative positions of the AUVs to the leader, are specified as follows:

$d_1^{*}={\left[\mathrm{0,0,0}\right]}^T,d_4^{*}={\left[-\mathrm{10,10,0}\right]}^T,d_2^{*}={\left[10,-\mathrm{10,0}\right]}^T,d_5^{*}={\left[-10,-\mathrm{10,0}\right]}^T,d_3^{*}={\left[\mathrm{10,10,0}\right]}^T.$

Each AUV tracks its respective position in the formation by adjusting its location to ${\eta }_i={\eta }_0+d_i^{*}$.

6.1 DDL formation learning control simulation

The estimated virtual leader’s state, derived from the cooperative estimator in the first layer (see Equations 3, 4), is utilized to estimate each agent’s complete uncertain dynamics within the DDL controller (second layer) using Equations 13, 14. The uncertain nonlinear functions $F_i(Z_i)$ for each agent are approximated using RBFNN, as described in Equation 11. Specifically, for each agent $i\in \{1,\dots ,5\}$, the nonlinear uncertain functions $F_i(Z_i)$, dependent on ${\nu }_i$, are modeled. The input to the NN, $Z_i={[u_i,v_i,r_i]}^T$, allows the construction of Gaussian RBFNN, represented by $W_{k,i}^T{S}_{k,i}(Z_i)$, utilizing 4,096 neurons arranged in an $16\times 16\times 16$ grid. The centers of these neurons are evenly distributed over the state space $[-\mathrm{100,100}]\times [-\mathrm{100,100}]\times [-\mathrm{100,100}]$, and each has a width ${\gamma }_{k,i}=60$, ensuring bounded and structured parameter optimization for all $i\in \{1,\dots ,5\}$ and $k\in \{\mathrm{1,2,3}\}$.

The observer and controller parameters are chosen as ${\beta }_1={\beta }_2=5$, and the diagonal matrices $K_{1,i}=800\ast \text{diag}\{1.2,\mathrm{1,1}\}$ and $K_{2,i}=1200\ast \text{diag}\{1.2,\mathrm{1,1}\}$, with ${\mathrm{\Gamma }}_{k,i}=10$ and ${\sigma }_{k,i}=0.0001$ for all $i\in \{1,\dots ,5\}$ and $k\in \{\mathrm{1,2,3}\}$. The initial conditions for the agents are set as ${\eta }_1(0)={[\mathrm{30,60,0}]}^T$, ${\eta }_2(0)={[\mathrm{40,70,0}]}^T$, ${\eta }_3(0)={[\mathrm{50,80,0}]}^T$, ${\eta }_4(0)={[\mathrm{10,70,0}]}^T$, and ${\eta }_5(0)={[\mathrm{10,50,0}]}^T$. Zero initial conditions are assumed for all the distributed observer states $({\chi }_{i,0},A_{i,0})$ and the DDL controller states $W_{k,i}$ for all $i\in \{1,\dots ,5\}$ and $k\in \{\mathrm{1,2,3}\}$. Time-domain simulation is carried out using the DDL formation learning control laws as specified in Equations 13, 14, along with Equations 3, 4.

Figure 3 displays the simulation results of the cooperative estimator (first layer) for all five agents. It illustrates how each agent’s estimated states,${\widehat{\eta }}_i$, converge perfectly to the leader’s states ${\widehat{\eta }}_0$ thorough Equations 3, 4. Figure 4 presents the position tracking control responses of all agents. Figures 4A–C illustrate the tracking performance of AUVs along the x-axis, y-axis, and vehicle heading, respectively, demonstrating effective tracking of the leader’s position signal. While the first AUV exactly tracks the leader’s states, agents 2 through 5 are shown to successfully follow agent 1, maintaining prescribed distances and alignment along the x and y axes, and matching the same heading angle. These results underscore the robustness of the real-time tracking control system, which enforces a predefined formation pattern, initially depicted in Figure 2. Additionally, Figure 5 highlights the real-time control performance for all agents, showcasing the effectiveness of the tracking strategy in maintaining the formation pattern.

Figure 3

Figure 3. Simulation results of the cooperative observer (first layer) for all three states (x-axis, y-axis, and vehicle heading) of each AUV: (A) ${\widehat{x}}_i\to x_0\left(\text{m}\right)$, (B) ${\widehat{y}}_i\to y_0\left(\text{m}\right)$, (C) ${\widehat{\psi }}_i\to {\psi }_0\left(\text{deg}\right)$.

Figure 4

Figure 4. Simulation results of position tracking control performance of all agents: (A) $x_i\to x_0\left(\text{m}\right)$, (B) $y_i\to y_0\left(\text{m}\right)$, (C) ${\psi }_i\to {\psi }_0\left(\text{deg}\right)$.

Figure 5

Figure 5. Real-time control performance in simulation for all agents, demonstrating the tracking strategy’s effectiveness in maintaining the formation pattern.

The sum of the absolute values of the neural network weights in Figure 6. This convergence reflects the network’s ability to maintain consistent performance, as further adjustments to the weights become minimal. Also, updating neural network weights and their convergence throughout the learning process into their optimal valies depicted in Figure 7. This convergence of all neural network weights to their optimal values during the training process, aligns with Theorem 4 as well. This leads to achieving accurate function approximation in the second layer. Figure 8 represents the successful function approximation results for the unknown system dynamics $F_3(Z_3)$ as defined in Equation 11 for the third AUV, using RBFNN. The approximations are plotted for both $W_{k,3}^T{S}_{k,3}(Z_3)$ and ${\bar{W}}_{k,3}^T{S}_{k,3}(Z_3)$ for all $k\in I[\mathrm{1,3}]$ which defined in Theorem 4. The results confirm that locally accurate approximations of the AUV’s nonlinear dynamics were achieved. Moreover, this learned knowledge about the dynamics is effectively stored and represented using localized constant RBFNN.

Figure 6

Figure 6. Sum of the absolute values of neural network weights in simulation for the third agent, showing the network stabilized and learns a consistent tracking pattern.

Figure 7

Figure 7. Convergence of neural network weights of each state to their optimal values in simulation for the third agent: (A) ${\widehat{W}}_{\mathrm{1,3}}$, (B) ${\widehat{W}}_{\mathrm{2,3}}$, (C) ${\widehat{W}}_{\mathrm{3,3}}$. The stabilized weights demonstrate accurate learning in the second layer throughout the training process.

Figure 8

Figure 8. Simulation results of successful function approximation for all three states (k = 1, 2, 3) of the $3^{\mathrm{rd}}$ AUV: (A) $k=1$, (B) $k=2$, (C) $k=3$. Comparison of $f_{k,3}(Z_3)$, $W_{k,3}^T{S}_{k,3}(Z_3)$, and ${\bar{W}}_{k,3}^T{S}_{k,3}(Z_3)$ using stored constant NN weights $({\bar{W}}_{k,3})$.

6.2 Simulation for formation control with pre-learned dynamics

To evaluate the distributed control performance of the multi-AUV system, we implemented the pre-learned distributed formation control law. This strategy integrates the estimator observer Equations 3, 4, this time coupled with the constant RBFNN controller (Equation 21). We employed the virtual leader dynamics described in Equation 21 to generate consistent position tracking reference signals, as previously discussed in Section 6.1. To ensure a fair comparison, identical initial conditions and control gains and inputs were used across all simulations. Figure 9 illustrates the comparison of the tracking control results from Equations 13, 14 with the results using pre-trained weights $\bar{W}$ in Equation 20.

Figure 9

Figure 9. Simulation results of successful performance of position tracking control using pretrained weights $(\bar{W})$: (A) $x_i\to x_0\left(\text{m}\right)$, (B) $y_i\to y_0\left(\text{m}\right)$, (C) ${\psi }_i\to {\psi }_0\left(\text{deg}\right)$.

The control experiments and simulation results presented demonstrate that the constant RBFNN control law (Equation 4) can achieve satisfactory tracking control performance comparable to that of the adaptive control laws (Equations 13, 14), but with no computational demand. The elimination of online recalculations or readaptations of the NN weights under this control strategy significantly reduces the computational load whenever system restarts without needing to retrain again. This reduction is particularly advantageous in scenarios involving extensive neural networks with a large number of neurons, thereby conserving system energy and enhancing operational efficiency in real-time applications.

Before concluding the paper, a brief contribution of the paper is provided:

• Distributed Observer Results: Simulations showed that the distributed observer effectively estimated the leader’s state, allowing for accurate formation control without needing global information.

• Tracking Control Results: The controller demonstrated reliable tracking of reference signals, maintaining performance even under varying conditions and unknown system dynamics.

• Formation Control: The proposed controller maintained accurate formation control relative to a virtual leader in simulations, even when the system dynamics were unknown with different AUVs.

• Neural Network Weight Convergence: The simulation results demonstrated that the neural network weights converged effectively, ensuring accurate function approximation and reliable performance in controlling AUVs under uncertainties.

• Adaptability and Stability: The framework ensured stable tracking performance across various environmental conditions by relying on the RBFNN’s learning capabilities, allowing the AUVs to use prelearned information and maintain formation control without needing to relearn dynamics whenever system restarts.

• Reduction in Computational Load: The use of pre-trained neural network weights significantly reduced the computational burden during real-time operation, particularly when large neural networks were employed.

7 Conclusion

In conclusion, this paper has introduced a novel two-layer control framework designed for Autonomous Underwater Vehicles (AUVs), aimed at universal applicability across various AUV configurations and environmental conditions. This framework assumes all system dynamics to be unknown, thereby enabling the controller to operate independently of specific dynamic parameters and effectively handle any environmental challenges, including hydrodynamic forces and torques. The framework consists of a first-layer distributed observer estimator that captures the leader’s dynamics using information from adjacent agents, and a second-layer decentralized deterministic learning controller. Each AUV utilizes the estimated signals from the first layer to determine the desired trajectory, simultaneously training its own dynamics using Radial Basis Function Neural Networks (RBFNN). This innovative approach not only sustains stability and performance in dynamic and unpredictable environments but also allows AUVs to efficiently utilize previously learned dynamics after system restarts, facilitating rapid resumption of optimal operations. The robustness and versatility of this framework have been rigorously confirmed through comprehensive simulations, demonstrating its potential to significantly enhance the adaptability and resilience of AUV systems. By embracing total uncertainty in system dynamics, this framework establishes a new benchmark in autonomous underwater vehicle control and lays a solid groundwork for future developments aimed at minimizing energy use and maximizing system flexibility. We plan to expand this framework by accommodating more general leader dynamics and conducting experimental applications to validate its performance in real-world settings. Moreover, a more accurate model of some source of uncertainty could improve performance which we will address in our future research. The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Data availability statement

The original contributions presented in the study are included in the article/supplementary material, further inquiries can be directed to the corresponding author.

Author contributions

EJ: Conceptualization, Writing–original draft, Writing–review and editing. MZ: Conceptualization, Funding acquisition, Supervision, Writing–review and editing. PS: Conceptualization, Supervision, Writing–review and editing. CY: Conceptualization, Formal Analysis, Funding acquisition, Supervision, Writing–review and editing.

Funding

The author(s) declare that financial support was received for the research, authorship, and/or publication of this article. This work is supported in part by the National Science Foundation under Grant CMMI-1952862 and CMMI-2154901.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Footnotes

¹A recurrent trajectory represents a large set of periodic and periodic-like trajectories generated from linear/nonlinear dynamical systems. A detailed characterization of recurrent trajectories can be found in Wang and Hill (2018).

References

Balch, T., and Arkin, R. (1998). Behavior-based formation control for multirobot teams. IEEE Trans. Robotics Automation 14, 926–939. doi:10.1109/70.736776

CrossRef Full Text | Google Scholar

Cai, H., Lewis, F. L., Hu, G., and Huang, J. (2015). “Cooperative output regulation of linear multi-agent systems by the adaptive distributed observer,” in 2015 54th IEEE Conference on Decision and Control (CDC) (IEEE), 5432–5437.

CrossRef Full Text | Google Scholar

Cao, X., Ren, L., and Sun, C. (2022). Dynamic target tracking control of autonomous underwater vehicle based on trajectory prediction. IEEE Trans. Cybern. 53, 1968–1981. doi:10.1109/tcyb.2022.3189688

PubMed Abstract | CrossRef Full Text | Google Scholar

Christen, S., Jendele, L., Aksan, E., and Hilliges, O. (2021). Learning functionally decomposed hierarchies for continuous control tasks with path planning. IEEE Robotics Automation Lett. 6, 3623–3630. doi:10.1109/lra.2021.3060403

CrossRef Full Text | Google Scholar

Cui, R., Ge, S. S., How, B. V. E., and Choo, Y. S. (2010). Leader–follower formation control of underactuated autonomous underwater vehicles. Ocean. Eng. 37, 1491–1502. doi:10.1016/j.oceaneng.2010.07.006

CrossRef Full Text | Google Scholar

Dong, X., Yuan, C., Stegagno, P., Zeng, W., and Wang, C. (2019). Composite cooperative synchronization and decentralized learning of multi-robot manipulators with heterogeneous nonlinear uncertain dynamics. J. Frankl. Inst. 356, 5049–5072. doi:10.1016/j.jfranklin.2019.04.028

CrossRef Full Text | Google Scholar

Fossen, T. I. (1999). “Guidance and control of ocean vehicles,”. Norway: University of Trondheim. Doctors Thesis.

Google Scholar

Ghafoori, S., Rabiee, A., Cetera, A., and Abiri, R. (2024). Bispectrum analysis of noninvasive eeg signals discriminates complex and natural grasp types. arXiv Prepr. arXiv:2402.01026, 1–5. doi:10.1109/embc53108.2024.10782163

CrossRef Full Text | Google Scholar

Hadi, B., Khosravi, A., and Sarhadi, P. (2021). A review of the path planning and formation control for multiple autonomous underwater vehicles. J. Intelligent and Robotic Syst. 101, 67–26. doi:10.1007/s10846-021-01330-4

CrossRef Full Text | Google Scholar

Hou, S. P., and Cheah, C. C. (2009). “Coordinated control of multiple autonomous underwater vehicles for pipeline inspection,” in Proceedings of the 48h IEEE Conference on Decision and Control (CDC) held jointly with 2009 28th Chinese Control Conference (IEEE), 3167–3172.

Google Scholar

Ioannou, P. A., and Sun, J. (1996). Robust adaptive control, 1. Upper Saddle River, NJ: PTR Prentice-Hall.

Google Scholar

Jandaghi, E., Chen, X., and Yuan, C. (2023). “Motion dynamics modeling and fault detection of a soft trunk robot,” in 2023 IEEE/ASME International Conference on Advanced Intelligent Mechatronics (AIM) (IEEE), 1324–1329.

CrossRef Full Text | Google Scholar

Jandaghi, E., Stein, D. L., Hoburg, A., Stegagno, P., Zhou, M., and Yuan, C. (2024). “Composite distributed learning and synchronization of nonlinear multi-agent systems with complete uncertain dynamics,” in 2024 IEEE International Conference on Advanced Intelligent Mechatronics (AIM), 1367–1372. doi:10.1109/aim55361.2024.10637197

CrossRef Full Text | Google Scholar

Krstic, M., Kokotovic, P. V., and Kanellakopoulos, I. (1995). Nonlinear and adaptive control design. John Wiley and Sons, Inc.

Google Scholar

Lawton, J. R. T. (2000). A Behavior-Based Approach to Multiple Spacecraft Formation Flying (Ph.D. thesis). Brigham Young University, Provo, UT, United States.

Google Scholar

Millán, P., Orihuela, L., Jurado, I., and Rubio, F. R. (2013). Formation control of autonomous underwater vehicles subject to communication delays. IEEE Trans. Control Syst. Technol. 22, 770–777. doi:10.1109/tcst.2013.2262768

CrossRef Full Text | Google Scholar

Park, J., and Sandberg, I. W. (1991). Universal approximation using radial-basis-function networks. Neural Comput. 3, 246–257. doi:10.1162/neco.1991.3.2.246

PubMed Abstract | CrossRef Full Text | Google Scholar

Peng, Z., Wang, D., Shi, Y., Wang, H., and Wang, W. (2015). Containment control of networked autonomous underwater vehicles with model uncertainty and ocean disturbances guided by multiple leaders. Inf. Sci. 316, 163–179. doi:10.1016/j.ins.2015.04.025

CrossRef Full Text | Google Scholar

Peng, Z., Wang, J., and Wang, D. (2017). Distributed maneuvering of autonomous surface vehicles based on neurodynamic optimization and fuzzy approximation. IEEE Trans. Control Syst. Technol. 26, 1083–1090. doi:10.1109/tcst.2017.2699167

CrossRef Full Text | Google Scholar

Prestero, T. (2001). “Development of a six-degree of freedom simulation model for the remus autonomous underwater vehicle,” in MTS/IEEE Oceans 2001. An Ocean Odyssey. Conference Proceedings (IEEE Cat. No.01CH37295), Honolulu, HI, USA, 2001, pp.450–455 vol.1, doi:10.1109/OCEANS.2001.968766

PubMed Abstract | CrossRef Full Text | Google Scholar

Ren, W., and Beard, R. W. (2005). Consensus seeking in multiagent systems under dynamically changing interaction topologies. IEEE Trans. automatic control 50, 655–661. doi:10.1109/tac.2005.846556

CrossRef Full Text | Google Scholar

Ren, W., and Beard, R. W. (2008). Distributed consensus in multi-vehicle cooperative control, 27. Springer.

Google Scholar

Rout, R., and Subudhi, B. (2016). A backstepping approach for the formation control of multiple autonomous underwater vehicles using a leader–follower strategy. J. Mar. Eng. and Technol. 15, 38–46. doi:10.1080/20464177.2016.1173268

CrossRef Full Text | Google Scholar

Skjetne, R., Fossen, T. I., and Kokotović, P. V. (2005). Adaptive maneuvering, with experiments, for a model ship in a marine control laboratory. Automatica 41, 289–298. doi:10.1016/j.automatica.2004.10.006

CrossRef Full Text | Google Scholar

Su, Y., and Huang, J. (2011). Cooperative output regulation of linear multi-agent systems. IEEE Trans. Automatic Control 57, 1062–1066. doi:10.1109/TAC.2011.2169618

CrossRef Full Text | Google Scholar

Tutsoy, O., Asadi, D., Ahmadi, K., Nabavi-Chashmi, S. Y., and Iqbal, J. (2024). Minimum distance and minimum time optimal path planning with bioinspired machine learning algorithms for faulty unmanned air vehicles. IEEE Trans. Intelligent Transp. Syst. 25, 9069–9077. doi:10.1109/tits.2024.3367769

CrossRef Full Text | Google Scholar

Wang, C., and Hill, D. J. (2018). Deterministic learning theory for identification, recognition, and control. CRC Press. doi:10.1201/9781315221755

CrossRef Full Text | Google Scholar

Yan, T., Xu, Z., Yang, S. X., and Gadsden, S. A. (2023). Formation control of multiple autonomous underwater vehicles: a review. Intell. and Robotics 3, 1–22. doi:10.20517/ir.2023.01

CrossRef Full Text | Google Scholar

Yan, Z., Liu, X., Zhou, J., and Wu, D. (2018). Coordinated target tracking strategy for multiple unmanned underwater vehicles with time delays. IEEE Access 6, 10348–10357. doi:10.1109/access.2018.2793338

CrossRef Full Text | Google Scholar

Yang, Y., Xiao, Y., and Li, T. (2021). A survey of autonomous underwater vehicle formation: performance, formation control, and communication capability. IEEE Commun. Surv. and Tutorials 23, 815–841. doi:10.1109/comst.2021.3059998

CrossRef Full Text | Google Scholar

Yuan, C. (2017). Leader-following consensus of parameter-dependent networks via distributed gain-scheduling control. Int. J. Syst. Sci. 48, 2013–2022. doi:10.1080/00207721.2017.1309597

CrossRef Full Text | Google Scholar

Yuan, C., Licht, S., and He, H. (2017). Formation learning control of multiple autonomous underwater vehicles with heterogeneous nonlinear uncertain dynamics. IEEE Trans. Cybern. 48, 2920–2934. doi:10.1109/tcyb.2017.2752458

PubMed Abstract | CrossRef Full Text | Google Scholar

Yuan, C., and Wang, C. (2011). Persistency of excitation and performance of deterministic learning. Syst. and control Lett. 60, 952–959. doi:10.1016/j.sysconle.2011.08.002

CrossRef Full Text | Google Scholar

Yuan, C., and Wang, C. (2012). Performance of deterministic learning in noisy environments. Neurocomputing 78, 72–82. doi:10.1016/j.neucom.2011.05.037

CrossRef Full Text | Google Scholar

Zhang, Y., Li, S., and Liu, X. (2018). Neural network-based model-free adaptive near-optimal tracking control for a class of nonlinear systems. IEEE Trans. neural Netw. Learn. Syst. 29, 6227–6241. doi:10.1109/tnnls.2018.2828114

PubMed Abstract | CrossRef Full Text | Google Scholar

Zhou, J., Si, Y., and Chen, Y. (2023). A review of subsea auv technology. J. Mar. Sci. Eng. 11, 1119. doi:10.3390/jmse11061119

CrossRef Full Text | Google Scholar