Privacy-preserving ADP for secure tracking control of AVRs against unreliable communication

In this study, we developed an encrypted guaranteed-cost tracking control scheme for autonomous vehicles or robots (AVRs), by using the adaptive dynamic programming technique. To construct the tracking dynamics under unreliable communication, the AVR's motion is analyzed. To mitigate information leakage and unauthorized access in vehicular network systems, an encrypted guaranteed-cost policy iteration algorithm is developed, incorporating encryption and decryption schemes between the vehicle and the cloud based on the tracking dynamics. Building on a simplified single-network framework, the Hamilton-Jacobi-Bellman equation is approximately solved, avoiding the complexity of dual-network structures and reducing the computational costs. The input-constrained issue is successfully handled using a non-quadratic value function. Furthermore, the approximate optimal control is verified to stabilize the tracking system. A case study involving an AVR system validates the effectiveness and practicality of the proposed algorithm.

1 Introduction

Autonomous vehicles or robots (AVRs) have rapidly transformed from a futuristic concept to a tangible reality, driving significant advancements in automotive technology. The advancement of autonomous vehicle technology has increasingly focused on improving tracking control systems, which are crucial for effective vehicle guidance (Pan et al., 2023). However, a persistent issue is the unreliable communication between a local vehicle and a reference vehicle, leading to discrepancies in signal reception and affecting tracking precision. In addition to these developments, the emergence of connected vehicles (Li et al., 2019a; Liu et al., 2023b), which leverages cloud computing for data processing and optimization, presents both opportunities and challenges. These systems function as cyber–physical systems (He et al., 2014; Zhang et al., 2014; Mohan et al., 2020), integrating computational and physical processes to enhance real-time data exchange and improve overall traffic management (Jiang et al., 2022; Li et al., 2019b). However, during communication between the vehicle and the cloud, the network's homogeneous and civilian nature makes it, particularly, vulnerable to attacks. This vulnerability, especially in the absence of robust security protocols, exposes these systems to cyber threats, including eavesdropping.

To enhance the security of vehicular cyber-physical systems, researchers from various fields, such as communication, control systems, and information theory, have developed various strategies to address cyberattacks across different layers (Han et al., 2024; Deng and Wen, 2021; Liu et al., 2021, 2023a). Various types of attacks, including denial-of-service (DoS) attacks, false data injection (FDI) attacks, and replay attacks, have been extensively studied (Teixeira et al., 2012; Li et al., 2024; Hu et al., 2023). These types of attacks share the characteristic of being active strategies designed to disrupt system functionality or manipulate transmitted data. Although defense mechanisms have made progress in countering such threats, majority of the existing methods primarily concentrate on detecting and mitigating explicit attacks, often overlooking the fundamental challenge of ensuring communication security. In vehicular cybersecurity, one of the critical issues is the threat of eavesdropping attacks (Yang et al., 2020; Wu et al., 2022). Unlike the direct and active nature of DoS and FDI attacks, eavesdropping operates passively, enabling attackers to intercept sensitive information while remaining undetected. This makes it a significant long-term threat that can compromise communication confidentiality and can even enable more destructive attacks. Addressing this challenge requires advanced encryption and privacy-preserving techniques to ensure secure communication. Although these methods are effective, they do not ensure optimal control performance at minimal energy cost, as they do not incorporate the principles of optimal control.

Optimal tracking control has become a cornerstone of modern control theory, with adaptive dynamic programming (ADP) algorithms attracting considerable interest in recent years (Lu et al., 2020; Mu et al., 2017b). For non-linear optimal control problems, the principal challenge lies in solving the Hamilton-Jacobi-Bellman (HJB) equation—a problem that is nearly intractable through exact mathematical methods. ADP techniques have offered a promising alternative by leveraging neural networks (NNs) to approximate optimal solutions, leading to significant advancements across fields such as automatic control and artificial intelligence (Mu et al., 2017a; Guo et al., 2024). For example, El-Sousy et al. (2021) designed a three-network structure to approximate the solution of the HJB equation for permanent-magnet synchronous motor servo drives. Wang et al. (2020) proposed an dual-network to approximate local Q-functions and control policies, solving optimal consensus control for non-linear multiagent systems. Furthermore, ADP-based optimal tracking control has been widely investigated (Dong et al., 2022; Song et al., 2023), including efforts to address input-constrained systems (Yang et al., 2023; Zhang et al., 2018). However, conventional ADP approaches, particularly those employing actor-critic frameworks, are frequently hindered by significant approximation errors introduced during iterative processes and NN training, thereby restricting their practical applicability.

To address these challenges, researchers have proposed several single-network ADP methodologies designed to streamline system architectures and enhance computational efficiency in handling nonlinear systems (Xu et al., 2023; Chen et al., 2021; Zou and Zhang, 2023). Chen et al. (2021) developed an event-triggered optimal control scheme for a macro–micro stage system, using a single critic NN to solve the modified HJB equation. In Guo et al. (2024), a distributed control strategy for attitude-constrained quadrotor unmanned aerial vehicle is proposed based on a critic network. Among the core ADP algorithms, value iteration and policy iteration (PI) have been widely employed, demonstrating robust performance in numerous applications (Zhang et al., 2020; Lin et al., 2023). However, the two-stage iterative procedures inherent in these methods frequently involve information transmission, which makes them susceptible to interception by adversaries. This vulnerability necessitates additional security measures, thereby increasing computational complexity and further constraining their applicability to complex systems. Although efforts to streamline computational burdens by eliminating actor networks have yielded progress, current ADP methods still inadequately address essential issues such as input saturation and ensuring reliable system performance, leaving these critical areas as potential opportunities for future research.

Unlike the previous studies, this article proposes an encrypted guaranteed-cost tracking control scheme for input-constrained tracking system with unreliable communication, and the main contributions are summarized as follows:

1. This article introduces an encrypted guaranteed-cost tracking control scheme for AVRs under unreliable communication. Compared with existing works, this is the first attempt to integrate ADP with encryption techniques, addressing both control performance and information security challenges in vehicular networks.

2. The designed privacy-preserving control method introduces a strategy to address eavesdropping attacks in control systems. By applying consistent output masking and encryption mechanisms at both the vehicle side and the cloud side, sensitive data and critical control information are effectively protected from potential breaches. This integrated approach ensures secure data transmission while maintaining the integrity and privacy of the control system.

3. A single-network structure with enhanced computational efficiency is proposed to approximate the HJB equation. Compared to conventional dual-network designs, the single-network structure reduces computational complexity while maintaining theoretical guarantees on weight error convergence and system stability. Additionally, input saturation is explicitly addressed through the adoption of a nonlinear value function, further enhancing the robustness.

2 Preliminaries and problem formulation

Consider an AVR operating in the X-Y plane, the position and orientation of the vehicle's mass center are represented by a posture vector

$\dot{Z}:=\left[\begin{array}{c}x(t)\\ y(t)\\ \vartheta (t)\end{array}\right],$

where x(t) and y(t) denote the horizontal and vertical positions, respectively, and ϑ(t) denotes the heading direction measured counterclockwise from the X-axis. The vehicle's motion is governed by the following kinematic model:

$\begin{array}{ll}\dot{Z}=Ku(t)=\left[\begin{array}{cc}cos(\vartheta (t))& Ysin(\vartheta (t))\\ sin(\vartheta (t))& -Ycos(\vartheta (t))\\ 0& 1\end{array}\right]\left[\begin{array}{c}v(t)\\ w(t)\end{array}\right].& (1)\end{array}$

Here, v(t) and w(t) represent the vehicle's linear and rotational velocities, respectively, while $Y$ is the distance between the vehicle's mass center and its drive axle; and $K$ is the Jacobian matrix that links the control inputs to the vehicle's motion. So far, the control objectives are summarized in the following.

Control objective: For an AVR under unreliable communication, design an ADP-based robust optimal controller with secure information exchange to drive the vehicle along the target, such that the following objectives are achieved:

1) Robust tracking control objective: For an AVR, $Z$_c: = [x(t);y(t);ϑ(t)] to track the desired orbit $Z$_d: = [x_d(t);y_d(t);ϑ_d(t)] under malicious cyberattacks on the tracking process, as shown in Figure 1. Due to the occurrence of an attack, a small deviation arises between the received signal and the actual signal. This deviation, caused by malicious attacks, is defined as $Z$_a: = [x_a(t);y_a(t);ϑ_a(t)]. We assume that $Z$_a and its derivative are bounded.

Figure 1

Figure 1. Proposed scheme for tracking process of AVRs.

With the minor difference $Z$_a caused by unreliable communication, following the framework in Zhang et al. (2022), we derive the tracking error system as

$\begin{array}{ll}{\dot{Z}}_e=\left[\begin{array}{c}cos({\vartheta }_e(t))v_d(t)+y_e(t)w_d(t)-v_c(t)+{\gamma }_x(t)\\ sin({\vartheta }_e(t))v_d(t)-x_e(t)w_d(t)-w_c(t)+{\gamma }_y(t)\\ w_d(t)-w_c(t)+{\gamma }_{\vartheta }(t)\end{array}\right],& (2)\end{array}$

where $Z$_e: = [x_e(t);y_e(t);ϑ_e(t)] denotes the tracking error posture, v_d(t) and w_d(t) are the desired linear and rotational velocities, v_c(t) and w_c(t) are the control inputs of the vehicle, and [γ_x(t);γ_y(t);γ_ϑ(t)] captures the effect of cyberattacks on the received signals and given by

$\left[\begin{array}{c}{\gamma }_x(t)\\ {\gamma }_y(t)\\ {\gamma }_{\vartheta }(t)\end{array}\right]=\left[\begin{array}{c}cos({\vartheta }_c(t)){ẋ}_a+sin({\vartheta }_c(t)){ẏ}_a\\ -sin({\vartheta }_c(t)){ẋ}_a+cos({\vartheta }_c(t)){ẏ}_a\\ {\dot{\vartheta }}_a\end{array}\right].$

This model describes the dynamic behavior of the tracking error in AVR control.

To facilitate system description and control implementation, let us consider that z = [x_e(t);y_e(t);ϑ_e(t)], f(z) = [cos(ϑ_e(t))v_d(t); sin(ϑ_e(t))v_d(t);w_d(t)], g(z) = [−1, y_e(t);0, $L$−x_e(t);0, 1], and u = [v_c(t), w_c(t)]. The system (Equation 2) is rewritten as

$\begin{array}{ll}ż=f(z)+g(z)u+\gamma ,& (3)\end{array}$

where u is control input and satisfies the asymmetric constrained set 𝔒 = {u||u| ≤ ℏ}. To follow the conventional optimal tracking architecture, we can rewrite the reference trajectory as follows

$\begin{array}{ll}{ż}_d=f_d(z_d)+g_d(z_d)u_d,& (4)\end{array}$

where u_d is the steady-state control input taking the following form

$\begin{array}{ll}{u}_d=g_d^{-1}(z_d)({ż}_d-f_d(z_d)),& (5)\end{array}$

where $g_d^{-1}(z_d)g_d(z_d)=I_n$, I_n denotes an n × n identify matrix.

Assumption 1. The unreliable communication γ(t) is bounded by $\bar{\gamma }$, that is $\Vert \bar{\gamma }(t)\Vert \le \bar{\gamma }$, where $\bar{\gamma }$ is positive constant.

2) Prevent eavesdropping objective: As shown in Figure 1, the cloud handles monitoring, scheduling, optimization, and computation tasks, while the local controller is responsible for distributing control signals, albeit with limited data storage and processing capabilities. The cyberattack considered here is eavesdropping, where unauthorized interception of data during transmission allows attackers to steal sensitive system information, such as real-time control signals and operational states. To mitigate these risks, encryption and decryption mechanisms are implemented to safeguard the confidentiality and integrity of the transmitted data, ensuring secure communication and enhancing the system's overall reliability.

3) Optimal control objective: Based on the optimal control strategy, the AVR can achieve a compromise between performance and cost when running along a target, such that

$\begin{array}{ll}min& J(z)={\displaystyle {\int }_t^{\infty }}{\gamma }_1{\bar{\gamma }}^2+T(z,u)\text{d}s,\\ \text{s.t.}& ż=f(z)+g(z)u,u\in 𝔒,& (6)\end{array}$

where $T(z,u)=z^{T}Qz+\bar{U}(\mu )$, which is the utility function with feedback control μ = u − u_d, γ₁ is positive constant, $Q$ = $Q$^T > 0, and $\bar{U}(·)$ is a positive definite non-quadratic integrand function.

3 Iterative algorithm design

In this section, based on the preceding analysis, the tracking problem is reformulated into a stabilization problem for the error dynamics. To address this, a cryptography-based controller is designed, which not only mitigates the impact of unreliable communication but also ensures the security of information transmission against eavesdropping.

3.1 Encryption and decryption algorithm design

To effectively counter eavesdropping attacks on data transmitted between the vehicle side and the cloud side, privacy-preserving rules are designed for both sides. The encryption and decryption formulas (Han et al., 2024) for each iteration are provided in the following.

1) AVR to Cloud:

Encryption process: At the vehicle side, the data z to be sent are extracted from Equation 3 and encrypted using Equation 7, resulting in the encrypted data z^r. This encrypted data are then transmitted to the cloud. The encryption formula is as follows:

$\{\begin{array}{ll}{z}^s=a(t)z+A\xi (t),\hfill & \left(7\text{a}\right)\\ a(t)=e^{\left({\delta }_1{\displaystyle \sum {\Vert V^r(z)(t-1)\Vert }_2^2}\right)},\hfill & \left(7\text{b}\right)\\ \xi (t)={\rho }_1{e}^{-\left(t\mathrm{mod}{\rho }_2\right)},\hfill & \left(7\text{c}\right)\end{array}$

where a(t) and ξ(t) are encryption operators, δ₁, ρ₁, and ρ₂ are constants, and A is the channel assignment matrices. To simplify the presentation of the method, it is assumed that $V$^s(z)(t − 1) is already stored in the cloud. The value $V$(z) needs to be calculated on the cloud side. Its design is detailed in Section 3.2 and it serves as an essential component of the controller μ.

Decryption process: The cloud side receives the encrypted data z^r and decrypts it to recover the original data z. The decryption formula is as follows:

$\{\begin{array}{ll}z=\frac{z^r-A\xi (t)}{c(t)},\hfill & (8\text{a})\\ c(t)=e^{\left({\delta }_1{\displaystyle \sum {\Vert V^s(z)(t-1)\Vert }_2^2}\right)},\hfill & (8\text{b})\end{array}$

where c(t) is the counterpart of a(t). It is observed that the design forms of the encryption operators a(t) and ξ(t), and encrypted expressions are shared between the vehicle side and the cloud. Furthermore, the parameters A, δ₁, ρ₁, and ρ₂ are also shared.

2) Cloud to AVR:

Encryption process: After policy evaluation, the computed $V$(z) is encrypted using Equation 9 and sent back to the vehicle.

$\{\begin{array}{ll}{V}^s(z)=b(t)V(z)+B\zeta (t),\hfill & \left(9\text{a}\right)\\ b(t)=e^{\left({\delta }_2{\displaystyle \sum {\Vert z^r\Vert }_2^2}\right)},\hfill & \left(9\text{b}\right)\\ \zeta (t)={\varrho }_1{e}^{-\left(t\mathrm{mod}{\varrho }_2\right)},\hfill & \left(9\text{c}\right)\end{array}$

where b(t) and ζ(t) are encryption operators, δ₂, ϱ₁, and ϱ₂ are constants, and B is the channel assignment matrices.

Decryption process: At the vehicle side, the received encrypted data $V$^r(z) is decrypted using Equation 10 to recover $V$(z) for policy improvement.

$\{\begin{array}{ll}V(z)=\frac{V^r(z)-B\zeta (t)}{d(t)},\hfill & (10\text{a})\\ d(t)=e^{\left({\delta }_2{\displaystyle \sum {\Vert z^s\Vert }_2^2}\right)},\hfill & (10\text{b})\end{array}$

where d(t) is the counterpart of b(t). Similarly, the design forms of the encryption operators b(t) and ζ(t), and encrypted expressions are shared between the vehicle side and the cloud. Furthermore, the parameters B, δ₂, ϱ₁, and ϱ₂ are also shared. At this point, a complete iteration of privacy-preserving processing has been completed.

From the above encryption and decryption processes, it can be observed that the introduced masking signals ξ(t) and ζ(t) and the encryption formula designs effectively ensure privacy during data transmission between the vehicle and the cloud. Notably, the data transmitted over the network are not the raw values z and $V$(z) but their encrypted counterparts, z^s, z^r, $V$^s(z), and $V$^r(z), which effectively prevent unauthorized entities from intercepting sensitive information.

3.2 Encrypted iterative algorithm design

The objective is to stabilize Equation 3 by constructing an encrypted iterative algorithm so that minimizing the performance index function, thereby reducing control costs and enhancing system security. Recalling Equation 6, the performance index is

$\begin{array}{ll}V(z)={\displaystyle {\int }_t^{\infty }}({\gamma }_1{\bar{\gamma }}^2+z^{T}Qz+\bar{U}(\mu ))\text{d}s,& (11)\end{array}$

where

$\begin{array}{l}\bar{U}\left(\mu \right)={\displaystyle \sum _{i=1}^m}2{\theta }_1{\displaystyle {\int }_0^{u_i-u_d}}{h}^{-1}\left(\frac{\iota }{{\theta }_1}\right)r_{i}d{\iota }_i\\ =2{\theta }_1{\displaystyle {\int }_0^{u-u_d}}{h}^{-1}\left(\frac{\iota }{{\theta }_1}\right)Rd\iota ,& (12)\end{array}$

where $R$ = diag{[r₁, ..., r_m]} > 0, $\iota ={\left[{\iota }_1,...,{\iota }_m\right]}^T$. The function h(·) is assumed to be a monotonic odd function satisfying h(0) = 0. For the purposes of this article, h(·) is specifically selected as h(x) = (e^z − e^−z)/(e^z + e^−z).

According to the optimal control theory, Equation 11 is a Lyapunov function for the Equation 3 and the Hamiltonian function can be derived as

$\begin{array}{ll}H(z,\mu ,V(z))={\gamma }_1{\bar{\gamma }}^2+z^{T}Qz+\bar{U}(\mu )+\nabla V(z)(f(z)+g(z)u+\gamma ),& (13)\end{array}$

with $\nabla V(z)=\frac{\partial V(z)}{\partial z}$. On defining $V$^*(z) as the minimum value of Equation 11, based on Bellman's principle of optimality, we have

$\begin{array}{l}0=H(z,\mu ,V^{*}(z))\\ ={\gamma }_1{\bar{\gamma }}^2+z^{T}Qz+\bar{U}(\mu )+\nabla V^{*}(z)(f(z)+g(z)u^{*}+\gamma ),& (14)\end{array}$

and the optimal control u^* is obtained from $\frac{\partial H(z,\mu ,V^{*}(z))}{\partial u^{*}}=0$:

$\begin{array}{ll}{u}^{*}={\theta }_{1}tanh(-\frac{1}{2{\theta }_1}{R}^{-1}{g}^T\left(z\right)\nabla V^{*}\left(z\right))+u_d.& (15)\end{array}$

Substituting Equation 15 into Equation 12 yields

$\begin{array}{ll}\bar{U}({\mu }^{*})=\nabla V^{*T}(z)g(z)tanh(D(z))+{\theta }_1^2{\displaystyle \sum _{i=1}^m}ln(1-{tanh}^2(D_i(z))),& (16)\end{array}$

where $D(z)=\frac{1}{2{\theta }_1}{R}^{-1}{g}^T(z)\nabla V^{*}(z)$ and ${\mu }^{*}=u^{*}-u_d$. Then, the HJB equation can be derived as

$\begin{array}{l}H(z,{\mu }^{*},V^{*}(z))={\gamma }_1{\bar{\gamma }}^2+z^{T}Qz+\nabla V^{*}(z)(f(z)+\gamma )\\ +{\theta }_1^2{\displaystyle \sum _{i=1}^m}ln(1-{tanh}^2(D_i(z)))=0.& (17)\end{array}$

As highlighted in the preceding analysis, obtaining the optimal controller in Equation 15 necessitates solving the HJB Equation 17, a task well-known for its considerable computational and analytical challenges. To overcome this challenge, an iterative algorithm based on ADP is employed to obtain an approximate solution. The details of this iterative algorithm are presented in Algorithm 1.

Algorithm 1

Algorithm 1. Encrypted guaranteed cost policy iteration algorithm.

Lemma 1. By utilizing the encrypted PI process as described in Algorithm 1, which incorporates encryption and decryption steps for secure control of the tracking error dynamics in an AVR, the resulting control u_ς ensures the asymptotic stability of the system dynamics. Additionally, $V$_ς(z) will converge to the optimal value function $V$^*(z) as ς → ∞, ensuring that u_ς converges to the optimal control u^*.

Proof. Initially, without iterations, the control u₁ is considered admissible. For ∀u_ς produced during iterations, consider the Lyapunov function $V$_ς(z), which satisfies

$\begin{array}{l}{\dot{V}}_{\varsigma }(z)=\nabla V_{\varsigma }(z)ż\\ =\nabla V_{\varsigma }(z)(f(z)+g(z)u_{\varsigma }+\gamma ).& (20)\end{array}$

According to HJB Equation 17, we can drive

$\begin{array}{ll}\nabla V_{\varsigma }(z)(f(z)+g(z)u_{\varsigma }+\gamma )=-{\gamma }_1{\bar{\gamma }}^2-z^{T}Qz-\bar{U}({\mu }_{\varsigma }),& (21)\end{array}$

where μ_ς = u_ς − u_d. Then, substituting Equation 21 into Equation 22 yields

$\begin{array}{ll}{\dot{V}}_{\varsigma }(z)=-{\gamma }_1{\bar{\gamma }}^2-z^{T}Qz-\bar{U}({\mu }_{\varsigma })\le 0.& (22)\end{array}$

Therefore, the iteration process ensures that the error dynamics remain asymptotically stable. Moreover, policy improvement is achieved by minimizing the associated value function, consistent with the Kleinman method, guaranteeing convergence. As the iteration count ς → ∞, $V_{\varsigma }(z)\to V^{*}(z)$, and $u_{\varsigma }\to u^{*}$ hold. This concludes the proof.     □

Based on Lemma 1, the iterative process, enhanced with secure encryption and decryption, converges, leading to optimal control as the approximation errors diminish.

4 Critic neural network design

In this section, this study employs the fundamental update equations of PI to design a NN, utilizing the critic neural network (CNN) to approximate the solution of the HJB Equation 17 during each iteration step. Therefore, based on the universal approximation property of NNs, there exist ideal weights $W$^* such that the ideal value function can be approximated as

$\begin{array}{ll}{V}^{*}(z)=W^{*T}\phi (z)+{ϵ}_1(z),& (23)\end{array}$

where φ(z) ∈ ℝ^α denotes activation functions and α is the number of neurons. Utilizing Equation 23, HJB Equation 17 becomes

$\begin{array}{l}{\gamma }_1{\bar{\gamma }}^2+z^{T}Qz+(W^{*T}\nabla \phi (z)+\nabla {ϵ}_1^T(z))(f(z)+\gamma )\\ +{\theta }_1^2{\displaystyle \sum _{i=1}^m}ln(1-{tanh}^2(H_i(z)))=0,& (24)\end{array}$

where

$\begin{array}{l}{H}_i(z)=H_{1i}(z)+H_{2i}(z)\\ =\frac{1}{2\theta }{R}^{-1}{g}^T(z)\nabla {\phi }^T(z)W^{*}+\frac{1}{2{\theta }_1}{R}^{-1}{g}^T(z)\nabla {ϵ}_1^T(z),& (25)\end{array}$

with $\nabla \phi (z)=\frac{\partial {\phi }_1}{\partial z}$ and $\nabla {ϵ}_1(z)=\frac{\partial \phi }{\partial z}$. Therefore, by defining residual error ϵ_H, Equation 24 can be rewritten as

$\begin{array}{l}{\gamma }_1{\bar{\gamma }}^2+z^{T}Qz+W^{*T}\nabla \phi (z)(f(z)+\gamma )+{ϵ}_H\\ +{\theta }_1^2{\displaystyle \sum _{i=1}^m}ln(1-{tanh}^2(H_{1i}(z)))=0,& (26)\end{array}$

where

$\begin{array}{l}{ϵ}_H=\nabla {ϵ}_1^T(z)(f(z)+\gamma )-{\theta }_1^2{\displaystyle \sum _{i=1}^m}\frac{1}{O_{1i}(z)}tanh(O_{2i}(z))\\ (1-{tanh}^2(O_{2i}(z))),& (27)\end{array}$

with $O_{1i}(z)\in \left[1-{tanh}^2(D_i(z)),1-{tanh}^2(H_{1i}(z))\right]$, $O$_2i(z) ∈ [$D$_i(z), $H$_1i(z)]. Note that if the number of hidden layer neurons α is sufficiently large, the residual error ϵ_H will approach zero. Based on the Lipschitz assumption of the system dynamics, this ϵ_H is bounded within a compact set, that is, $\Vert {ϵ}_H\Vert \le {\overline{ϵ}}_H$. Therefore, based on Equation 23 the ideal optimal control is

$\begin{array}{ll}{u}^{*}={\theta }_{1}tanh(-\frac{1}{2{\theta }_1}{R}^{-1}{g}^T(z)\nabla {\phi }^T{W}^{*})+u_d+{ϵ}_2& (28)\end{array}$

where ${ϵ}_2=-\frac{1}{2}\sum _{i=1}^m(1-{tanh}^2({\psi }_i))R^{-1}{g}^T(z)\nabla {ϵ}_1$, ψ_i ∈ [$D$_i, $H$_1i].

Since the ideal weight is unknown, the approximated value function is

$\begin{array}{ll}\widehat{V}(z)={\widehat{W}}^T\phi (z),& (29)\end{array}$

where $\widehat{W}$ is approximated value of $W$^*. Then, we can get

$\begin{array}{ll}û=-{\theta }_{1}tanh(\frac{1}{2{\theta }_1}{R}^{-1}{g}^T(z)\nabla {\phi }^T(z)\widehat{W})+u_d.& (30)\end{array}$

Thus, approximated Hamiltonian function is

$\begin{array}{l}H(z,\widehat{\mu },\widehat{V}(z))={\gamma }_1{\bar{\gamma }}^2+z^{T}Qz+{\widehat{W}}^T\nabla \phi (z)(f(z)+\gamma )\\ +{\theta }_1^2{\displaystyle \sum _{i=1}^m}ln(1-{tanh}^2({\widehat{H}}_{1i}(z)))\\ :={\widehat{ϵ}}_H,& (31)\end{array}$

where ${\widehat{ϵ}}_H$ is the residual error due to NN approximation error.

Furthermore, let us consider $E=\frac{1}{2}{\widehat{ϵ}}_H^T{\widehat{ϵ}}_H$, and to ensure that $\widehat{W}$ converge toward the optimal weights $W$^*, the weight update formula (Zhang et al., 2018) is

$\begin{array}{l}{\dot{\widehat{W}}}_1=-\eta \frac{\tau }{{\varpi }^2}{\widehat{ϵ}}_H+\frac{\eta }{2}\kappa \nabla {\phi }_1(g(I-M({\widehat{H}}_1))g^T)\nabla V_a\\ +\eta (-{\theta }_1\nabla \phi gS\frac{{\tau }^T}{\varpi }\widehat{W}-(K_2-K_1{\tau }^T)\widehat{W}),& (32)\end{array}$

where η is learning rate, τ = ∇φ(z)(f(z)+g(z)û+γ), ϖ = τ^T τ + 1, and $K$₁ and $K$₂ are a tuning matrix. $M=\mathrm{\text{diag}}\left\{{tanh}^2({\widehat{H}}_{1i})\right\}$, $S=\mathrm{\text{sgn}}({\widehat{H}}_1)-tanh({\widehat{H}}_1)$. Based on the Lemma 2 by Zhang et al. (2018), $V$_a denotes Lyapunov function, and if ∇$V$_a(f(z)+gû + γ) > 0, then κ = 0, else κ = 1. Defining $\tilde{W}=W^{*}-\widehat{W}$, we obtain

$\begin{array}{l}\dot{\tilde{W}}=-\eta \tau {\tau }^T\tilde{W}+\eta \frac{\tau }{\varpi }({\theta }_1{\tilde{W}}^T\nabla \phi gS+{\stackrel{⌣}{ϵ}}_H)\\ -\frac{\eta }{2}\kappa \nabla {\phi }_{1}g(I-M({\widehat{H}}_1))g^T\nabla V_a\\ +\eta {\theta }_1\nabla \phi gS\frac{{\tau }^T}{\varpi }\widehat{W}+\eta (K_2-K_1{\tau }^T)\widehat{W},& (33)\end{array}$

with ${\stackrel{⌣}{ϵ}}_H={\theta }_1{W}^T\nabla \phi g(\mathrm{\text{sgn}}(H_1)-\mathrm{\text{sgn}}({\widehat{H}}_1))+2{\theta }_1^2\bar{H}-{ϵ}_H$, $\bar{H}=\sum _{i=1}^{m}ln\frac{1+\mathrm{\text{exp}}(-2H_{1i})}{1+\mathrm{\text{exp}}(-2{\widehat{H}}_{1i})}$.

Theorem 1. For the optimal control policy described in Equation 30, the weight tuning law of the CNN is determined by the update formula provided in Equation 32. Under this design, the error dynamic system z and the weight errors $\tilde{W}$ are uniformly ultimately bounded (UUB).

Proof. Define the Lyapunov function as $L$ = $L$₁ + $L$₂, where

$\begin{array}{ll}{L}_1=\frac{1}{2}{\tilde{W}}^T{\eta }^{-1}\tilde{W},L_2=V_a(z).& (34)\end{array}$

First, along Equation 33, the derivative of $L$₂ is

$\begin{array}{l}{\dot{L}}_1={\tilde{W}}^T{\eta }^{-1}{\dot{\tilde{W}}}_1\\ ={\tilde{W}}^T{\eta }^{-1}\{-\eta \tau {\tau }^T\tilde{W}+\eta \frac{\tau }{\varpi }({\theta }_1{\tilde{W}}^T\nabla \phi gS+{\stackrel{⌣}{ϵ}}_H)-\frac{\eta }{2}\kappa \nabla {\phi }_1(g(I-M({\widehat{H}}_1))g^T)\nabla V_a\\ +\eta {\theta }_1\nabla \phi gS\frac{{\tau }^T}{\varpi }\widehat{W}+\eta (K_2-K_1{\tau }^T)\widehat{W}\}\\ =-{\tilde{W}}^T\tau {\tau }^T\tilde{W}+{\theta }_1{\tilde{W}}^T\nabla \phi gS\frac{{\tau }^T}{\varpi }{\tilde{W}}^T+{\stackrel{⌣}{ϵ}}_H\frac{{\tau }^T}{\varpi }{\tilde{W}}^T-\frac{1}{2}\kappa \nabla V_a^{T}g(I-M({\widehat{H}}_1))g^T\nabla {\phi }^T\tilde{W}\\ +{\theta }_1{\tilde{W}}^T\nabla \phi gS\frac{{\tau }^T}{\varpi }{W}^{*}-{\theta }_1{\tilde{W}}^T\nabla \phi gS\frac{{\tau }^T}{\varpi }\tilde{W}+{\tilde{W}}^T(K_2-K_1{\tau }^T)\widehat{W}\\ =-{\tilde{W}}^T\tau {\tau }^T\tilde{W}+{\stackrel{⌣}{ϵ}}_H\frac{{\tau }^T}{\varpi }{\tilde{W}}^T-\frac{1}{2}\kappa \nabla V_a^{T}g(I-M({\widehat{H}}_1))g^T\nabla {\phi }^T\tilde{W}\\ +{\theta }_1{\tilde{W}}^T\nabla \phi gS\frac{{\tau }^T}{\varpi }{W}^{*}+{\tilde{W}}^T(K_2-K_1{\tau }^T)\widehat{W}\\ =-P^{T}AP+P^{T}B-\frac{1}{2}\kappa \nabla V_a^{T}g(I-M({\widehat{H}}_1))g^T\nabla {\phi }^T\tilde{W},& (35)\end{array}$

where

$\begin{array}{l}P=\left[\begin{array}{c}{\tilde{W}}^T\tau \\ {\tilde{W}}^T\end{array}\right],A=\left[\begin{array}{cc}I& -\frac{1}{2}{K}_1^T\\ -\frac{1}{2}{K}_1& K_2\end{array}\right],\end{array}$

$\begin{array}{l}B=\left[\begin{array}{c}-\frac{1}{\varpi }{\stackrel{⌣}{ϵ}}_H\\ ({\theta }_1{\tilde{W}}^T\nabla \phi gS\frac{{\tau }^T}{\varpi }+K_2-K_1{\tau }^T)W^{*}\end{array}\right].\end{array}$

Supposing that $\Vert W^{*}\Vert \le {\bar{W}}_1$, ${\bar{W}}_1>0$, and due to ${\stackrel{⌣}{ϵ}}_H$ is bound, $\Vert B\Vert \le \bar{B}$, $\bar{B}>0$. Therefore, ${\dot{L}}_1$ is

$\begin{array}{ll}{\dot{L}}_1\le -{\lambda }_{min}(A)\Vert P{\Vert }^2+\bar{B}\Vert P\Vert -\frac{1}{2}\kappa \nabla V_a^{T}g(I-M({\widehat{H}}_1))g^T\nabla {\phi }^T\tilde{W}.& (36)\end{array}$

Owing to κ of $L$₂, $\dot{L}$ is divided into two parts. For κ = 0, we have

$\begin{array}{ll}\dot{L}\le \nabla V_a^Tż-{\lambda }_{min}(A)\Vert P{\Vert }^2+\bar{B}\Vert P\Vert .& (37)\end{array}$

From a study by Rudin et al. (1964), we can know $\nabla V_a^Tż<-\Vert \nabla V_a\Vert z_m<0$, ||z|| ≤ z_m, z_m > 0, thus, $\dot{L}$ becomes

$\begin{array}{l}\dot{L}\le -\Vert \nabla V_a\Vert z_m-{\lambda }_{min}(A)(\Vert P\Vert -\frac{\bar{B}}{2{\lambda }_{min}(A)}{)}^2+\frac{{\bar{B}}^2}{4{\lambda }_{min}(A)}.\end{array}$

Moreover, $\dot{L}<0$ if

$\begin{array}{lll}\Vert \nabla V_a\Vert >& \frac{{\bar{B}}^2}{4z_m{\lambda }_{min}(A)},& (38)\end{array}$

or

$\begin{array}{ll}\Vert P\Vert >\frac{\bar{B}}{2{\lambda }_{min}(A)}.& (39)\end{array}$

According to Equation 39, we can derive

$\begin{array}{ll}\Vert \tilde{W}\Vert >\frac{2\bar{B}}{\sqrt{5}{\lambda }_{min}(A)}.& (40)\end{array}$

For κ = 1, $\dot{L}$ is

$\begin{array}{l}\dot{L}\le \nabla V_a^Tż-{\lambda }_{min}(A)\Vert P{\Vert }^2+\bar{B}\Vert P\Vert \\ -\frac{1}{2}\kappa \nabla V_a^{T}g(I-M({\widehat{H}}_1))g^T\nabla {\phi }^T\tilde{W}\\ \le \nabla V_a^T(f+gû+\gamma )-{\lambda }_{min}(A)\Vert P{\Vert }^2+\bar{B}\Vert P\Vert \\ -\frac{1}{2}\nabla V_a^{T}g(I-M({\widehat{H}}_1))g^T\nabla {\phi }^T\tilde{W}.& (41)\end{array}$

Regarding $tanh(H_1)-tanh({\widehat{H}}_1):=\stackrel{⌣}{H}$, using the Taylor series, we know

$\begin{array}{l}\stackrel{⌣}{H}=\frac{1}{2{\theta }_1}(I-M({\widehat{H}}_1))g^T\nabla {\phi }^T\tilde{W}+o({(H_1-{\widehat{H}}_1)}^2),\end{array}$

where $o({(H_1-{\widehat{H}}_1)}^2)$ is the higher order term and satisfies

$\begin{array}{c}\Vert o({(H_1-{\widehat{H}}_1)}^2)\Vert \\ \le \Vert \stackrel{⌣}{H}\Vert +\frac{1}{2{\theta }_1}(I-M({\widehat{H}}_1))g^T\nabla {\phi }^T\tilde{W}\\ \le \Vert tanh(H_1)\Vert +\Vert tanh({\widehat{H}}_1)\Vert +\frac{1}{2{\theta }_1}(I-M({\widehat{H}}_1))g^T\nabla {\phi }^T\tilde{W}\\ =({\displaystyle {\sum }_{i=1}^m}|tanh(H_1){|}^2{)}^{\frac{1}{2}}\\ +({\displaystyle {\sum }_{i=1}^m}|tanh({\widehat{H}}_1){|}^2{)}^{\frac{1}{2}}+\frac{1}{2{\theta }_1}(I-M({\widehat{H}}_1))g^T\nabla {\phi }^T\tilde{W}\\ \le 2\sqrt{m}+\frac{1}{{\theta }_1}ḡ\bar{\phi }\Vert \tilde{W}\Vert ,& (42)\end{array}$

where ||g|| ≤ ḡ, ḡ > 0 and $\Vert \nabla \phi \Vert \le \bar{\phi }$, $\bar{\phi }>0$.

Recalling Equations 28–30, the term in Equation 41 with respect to ∇$V$_ag can be written as

$\begin{array}{l}\nabla V_a^T(gû-\frac{1}{2}g(I-M({\widehat{H}}_1))g^T\nabla {\phi }^T\tilde{W})\\ =-{\theta }_1\nabla V_a^{T}gtanh(H_1)+{\theta }_1\nabla V_a^{T}go({(H_1-{\widehat{H}}_1)}^2)\\ =\nabla V_a^{T}g{u}^{*}-\nabla V_a^T{ϵ}_2+{\theta }_1\nabla V_a^{T}go({(H_1-{\widehat{H}}_1)}^2).& (43)\end{array}$

Until now, Equation 41 can be rewritten as

$\begin{array}{l}\dot{L}\le \nabla V_a^T(f+g{u}^{*}+\gamma )-{\lambda }_{min}(A)\Vert P{\Vert }^2+\bar{B}\Vert P\Vert -\nabla V_a^T{ϵ}_2+{\theta }_1\nabla V_a^{T}go({(H_1-{\widehat{H}}_1)}^2)\\ \le \Vert \nabla V_a\Vert \Vert f+g{u}^{*}\Vert +\Vert \nabla V_a\Vert \Vert \gamma \Vert -{\lambda }_{min}(A)\Vert P{\Vert }^2+\bar{B}\Vert P\Vert -\nabla V_a^T{ϵ}_2\\ +{\theta }_1\nabla V_a^{T}go({(H_1-{\widehat{H}}_1)}^2)\\ \le -{\lambda }_{min}(A)\Vert P{\Vert }^2+\bar{B}\Vert P\Vert -{\lambda }_{min}(C)\Vert \nabla V_a{\Vert }^2+{\overline{ϵ}}_2ḡ\Vert \nabla V_a\Vert +2{\theta }_1\sqrt{m}ḡ\Vert \nabla V_a\Vert \\ +\bar{\gamma }\Vert \nabla V_a\Vert +{ḡ}^2\bar{\phi }\Vert \nabla V_a\Vert \Vert \tilde{W}\Vert \\ =-{\lambda }_{min}(A)\Vert P{\Vert }^2+\bar{B}\Vert P\Vert -{\lambda }_{min}(C)\Vert \nabla V_a{\Vert }^2+\omega \Vert \nabla V_a\Vert +{ḡ}^2\bar{\phi }\Vert \nabla V_a\Vert \Vert \tilde{W}\Vert ,& (44)\end{array}$

where $\Vert {ϵ}_2\Vert \le {\overline{ϵ}}_2$, ${\overline{ϵ}}_2>0$. Let ℓ₁ and ℓ₂ satisfy 0 < ℓ₁ < 1, 0 < ℓ₂ < 1, and ℓ₁ + ℓ₂ = 1. Then, Equation 44 can be rewritten as

$\begin{array}{l}\dot{L}\le -\frac{4{\ell }_2{\lambda }_{min}(C){\lambda }_{min}(A)-{ḡ}^4{\bar{\phi }}^2}{4{\ell }_2{\lambda }_{min}(C)}(\Vert P\Vert -\frac{2{\ell }_2{\lambda }_{min}(C)\bar{B}}{4{\ell }_2{\lambda }_{min}(C){\lambda }_{min}(A)-{ḡ}^4{\bar{\phi }}^2}{)}^2\\ -{\ell }_1{\lambda }_{min}(C)(\Vert \nabla V_a\Vert -\frac{\omega }{2{\ell }_1{\lambda }_{min}(C)}{)}^2-{\ell }_1{\lambda }_{min}(C)(\Vert \nabla V_a\Vert -\frac{{ḡ}^2\bar{\phi }}{2{\ell }_1{\lambda }_{min}(C)}\Vert \tilde{W}\Vert {)}^2\\ +\frac{{\ell }_2{\lambda }_{min}(C){\bar{B}}^2}{4{\ell }_2{\lambda }_{min}(C){\lambda }_{min}(A)-{ḡ}^4{\bar{\phi }}^2}+\frac{{\omega }^2}{4{\ell }_1{\lambda }_{min}(C)}\\ =-\frac{{\omega }_1}{4{\ell }_2{\lambda }_{min}(C)}(\Vert P\Vert -\frac{2{\ell }_2{\lambda }_{min}(C)\bar{B}}{{\omega }_1}{)}^2-{\ell }_1{\lambda }_{min}(C)(\Vert \nabla V_a\Vert -\frac{\omega }{2{\ell }_1{\lambda }_{min}(C)}{)}^2\\ -{\ell }_1{\lambda }_{min}(C)(\Vert \nabla V_a\Vert -\frac{{ḡ}^2\bar{\phi }}{2{\ell }_1{\lambda }_{min}(C)}\Vert \tilde{W}\Vert {)}^2+{\omega }_2.\end{array}$

Therefore, $\dot{L}<0$ if

$\begin{array}{lll}\Vert \nabla V_a\Vert >& \frac{{ḡ}^2\bar{\phi }}{2{\ell }_1{\lambda }_{min}(C)}+\sqrt{\frac{{\omega }_1}{{\ell }_1{\lambda }_{min}(C)}},& (45)\end{array}$

or

$\begin{array}{lll}\Vert P\Vert >& \frac{2{\ell }_2{\lambda }_{min}(C)\bar{B}}{{\omega }_1}+2\sqrt{\frac{{\ell }_2{\lambda }_{min}(C){\omega }_2}{{\omega }_1}}.& (46)\end{array}$

Similar to Equation 40, we can derive

$\begin{array}{lll}\Vert \tilde{W}\Vert >& \frac{4{\ell }_2{\lambda }_{min}(C)\bar{B}}{\sqrt{5}{\omega }_1}+4\sqrt{\frac{{\ell }_2{\lambda }_{min}(C){\omega }_2}{5{\omega }_1}}.& (47)\end{array}$

By considering the two cases, κ = 0 and κ = 1, and based on the derived results as expressed in Equations 38–40 and Equations 45–47, we can conclude that the function ∇$V$_a and the error weights $\tilde{W}$ are UUB. Furthermore, knowing that $V$_a is in polynomial form, it follows that the error z is also UUB.

Remark 1. The algorithm designed in this article is depicted in Figure 2, where Algorithm 1 is implemented using a CNN. The CNN generates the estimated value function $\widehat{V}$, which is subsequently used to derive the approximated optimal control law û based on Equation 30. In contrast to the constrained optimal control designs presented in the studies by Zou and Zhang (2023); Chen et al. (2021), this work integrates privacy-preserving mechanisms during information transmission by leveraging encryption and decryption techniques. This incorporation not only safeguards data confidentiality but also enhances the overall security and reliability of the proposed algorithm.

Figure 2

Figure 2. Illustration of tracking for AVRs subject to privacy protection.

5 Simulation results

To analyze the tracking performance of the AVR, we conduct simulations based on a predefined tracking error dynamic model. The tracking error dynamics ${\dot{Z}}_e$ is modeled as

$\begin{array}{ll}\left[\begin{array}{c}{ẋ}_e\\ {ẏ}_e\\ {\dot{\vartheta }}_e\end{array}\right]=\left[\begin{array}{c}cos({\vartheta }_e)v_d\\ sin({\vartheta }_e)v_d\\ w_d\end{array}\right]+\left[\begin{array}{cc}-1& y_e\\ 0& Y-x_e\\ 0& -1\end{array}\right]u+\gamma ,& (48)\end{array}$

where $Y$ represents the distance from the vehicle's center of mass to the rear axle, set to $Y$ = −1.2m in this article. The desired reference trajectory is initialized with the state:

${\left[x_d(0),y_d(0),{\vartheta }_d(0)\right]}^T={\left[0,0,0\right]}^T,$

and the vehicle's trajectory is initialized with the state:

${\left[x_c(0),y_c(0),{\vartheta }_c(0)\right]}^T={\left[-2.5,2.5,-0.5\right]}^T.$

Consequently, the initial value of error denotes

${\left[x_e(0),y_e(0),{\vartheta }_e(0)\right]}^T={\left[2.5,-2.5,0.5\right]}^T.$

The reference trajectory's desired velocities are v_d = 0.5 and w_d = 0.04. Under the input constraints, ℏ = 1.5, meaning the constraint range is [−1.5, 1.5]. The unreliable communication γ is defined as

$\gamma (t)=\sigma \left[\begin{array}{c}sin(\sigma )x_e\\ cos(\sigma )y_e\\ sin(\sigma )x_e{y}_e\end{array}\right],$

where σ is a random variable uniformly distributed in σ ∈ [−0.1, 0.1]. For the performance evaluation, we define the cost function using the weighting matrices

$Q=\left[\begin{array}{ccc}10& 0& 0\\ 0& 10& 0\\ 0& 0& 10\end{array}\right],R=\left[\begin{array}{cc}1.5& 0\\ 0& 0.5\end{array}\right].$

The activation function vector of CNN is $\phi (z)=[z_1^4,z_2^4,z_3^4,$ $z_1^2{z}_2^2,z_2^2{z}_3^2,z_1^2{z}_3^2,z_1^2{z}_2,z_2^2{z}_3,z_1{z}_2^2,z_3^3,$ sin(z₁), sin(z₂), sin(z₃), $cos(z_1),cos(z_2),cos(z_3){]}^T$. ρ₁ = 1.1, ρ₂ = 1.03, ϱ₁ = 3.2, ϱ₂ = 1.08, ${\delta }_1=0.3\times 10^{-5}$, ${\delta }_2=0.4\times 10^{-2}$, A = 1, and B = 1.

Using the proposed method, Figure 3A illustrates the two-dimensional trajectory of the AVR. The vehicle quickly adjusts its direction and begins tracking the reference trajectory with good accuracy. After the initial phase, the vehicle follows the desired trajectory smoothly and closely. Figures 3B–G depict the tracking performance and error, demonstrating that the position error gradually reduces to zero, while the directional error also diminishes to zero, effectively ensuring precise position tracking throughout the process.

Figure 3

Figure 3. AVR driving trajectories. (A) The X-Y plot of tracking trajectories. (B–D) Tracking trajectories. (E–G) Tracking errors.

Figure 4 displays the evolution of the designed controller during the vehicle's tracking process. The dashed lines indicate the upper and lower bounds of the input constraints, which are set to [−1.5, 1.5]. The privacy-preserving characteristics of the proposed scheme are illustrated in Figure 5. It is evident that masking the vehicle-side output z effectively safeguards its privacy from potential attackers. Meanwhile, as shown in Figure 6, masking on the cloud side further prevents the leakage of critical information related to the designed control strategy. Therefore, these results ensure robust privacy protection during data transmission.

Figure 4

Figure 4. The constrained control input.

Figure 5

Figure 5. Encrypted error and decrypted error.

Figure 6

Figure 6. Encrypted value function and decrypted value function.

6 Conclusion

This study develops an encrypted guaranteed-cost tracking control scheme to address the challenges of information security and computational efficiency in AVR systems using the adaptive dynamic programming technique. By leveraging ADP and integrating encryption mechanisms between the vehicle and the cloud, the proposed method ensures stable tracking performance under unreliable communication. The input constraints are successfully managed using a nonlinear value function, while the CNN facilitates an efficient solution to the HJB equation. Simulation results from a case study confirm the stability and effectiveness of the designed algorithm, demonstrating its potential for real-world applications in AVR networks. Future work will focus on ensuring the security of cloud-based computations by processing encrypted data, further enhancing the safety and reliability of cloud operations in vehicular network systems.

Data availability statement

The original contributions presented in the study are included in the article/supplementary material, further inquiries can be directed to the corresponding author.

Author contributions

KZ: Conceptualization, Methodology, Writing – original draft. KH: Methodology, Writing – review & editing. ZH: Conceptualization, Supervision, Writing – review & editing. GT: Validation, Writing – review & editing.

Funding

The author(s) declare financial support was received for the research, authorship, and/or publication of this article. This work is supported by Beijing Nova Program (20240484516), the Fundamental Research Funds for the Central Universities (KG16314701), and Beihang World TOP University Cooperation Program.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Generative AI statement

The author(s) declare that no Gen AI was used in the creation of this manuscript.

Publisher's note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

Chen, X., Chen, X., Bai, W., and Guo, Z. (2021). Event-triggered optimal control for macro–micro composite stage system via single-network ADP method. IEEE Trans. Indust. Elect. 68, 4190–4198. doi: 10.1109/TIE.2020.2984462

Crossref Full Text | Google Scholar

Deng, C., and Wen, C. (2021). Mas-based distributed resilient control for a class of cyber-physical systems with communication delays under dos attacks. IEEE Trans. Cybern. 51, 2347–2358. doi: 10.1109/TCYB.2020.2972686

PubMed Abstract | Crossref Full Text | Google Scholar

Dong, H., Zhao, X., and Luo, B. (2022). Optimal tracking control for uncertain nonlinear systems with prescribed performance via critic-only ADP. IEEE Trans. Syst. Man, Cybernet.: Syst. 52, 561–573. doi: 10.1109/TSMC.2020.3003797

Crossref Full Text | Google Scholar

El-Sousy, F. F. M., Amin, M. M., and Al-Durra, A. (2021). Adaptive optimal tracking control via actor-critic-identifier based adaptive dynamic programming for permanent-magnet synchronous motor drive system. IEEE Trans. Ind. Appl. 57, 6577–6591. doi: 10.1109/TIA.2021.3110936

Crossref Full Text | Google Scholar

Guo, Z., Li, H., Ma, H., and Meng, W. (2024). Distributed optimal attitude synchronization control of multiple quavs via adaptive dynamic programming. IEEE Trans. Neural Netw. Learn. Syst. 35:8053–8063. doi: 10.1109/TNNLS.2022.3224029

PubMed Abstract | Crossref Full Text | Google Scholar

Han, K., Zhang, K., Wang, Z.-P., and Su, R. (2024). Resilient predictive load frequency control of multi-area interconnected power systems with privacy preserving and active detection against stealthy cyber attacks. IEEE Intern. Things J. 7, 4387–4394. doi: 10.1109/JIOT.2024.3507291

Crossref Full Text | Google Scholar

He, W., Yan, G., and Xu, L. D. (2014). Developing vehicular data cloud services in the IoT environment. IEEE Trans. Indust. Inform. 10, 1587–1595. doi: 10.1109/TII.2014.2299233

Crossref Full Text | Google Scholar

Hu, S., Ge, X., Chen, X., and Yue, D. (2023). Resilient load frequency control of islanded ac microgrids under concurrent false data injection and denial-of-service attacks. IEEE Trans. Smart Grid 14, 690–700. doi: 10.1109/TSG.2022.3190680

Crossref Full Text | Google Scholar

Jiang, M., Wu, T., Wang, Z., Gong, Y., Zhang, L., and Liu, R. P. (2022). A multi-intersection vehicular cooperative control based on end-edge-cloud computing. IEEE Trans. Vehicular Technol. 71, 2459–2471. doi: 10.1109/TVT.2022.3143828

Crossref Full Text | Google Scholar

Li, Y., Tang, C., Li, K., He, X., Peeta, S., and Wang, Y. (2019a). Consensus-based cooperative control for multi-platoon under the connected vehicles environment. IEEE Trans. Intellig. Transport. Syst. 20, 2220–2229. doi: 10.1109/TITS.2018.2865575

Crossref Full Text | Google Scholar

Li, Y., Tang, C., Peeta, S., and Wang, Y. (2019b). Nonlinear consensus-based connected vehicle platoon control incorporating car-following interactions and heterogeneous time delays. IEEE Trans. Intellig. Transport. Syst. 20, 2209–2219. doi: 10.1109/TITS.2018.2865546

Crossref Full Text | Google Scholar

Li, Z., Shi, Y., Xu, S., Xu, H., and Dong, L. (2024). Distributed model predictive consensus of mass against false data injection attacks and denial-of-service attacks. IEEE Trans. Automat. Contr. 69, 5538–5545. doi: 10.1109/TAC.2024.3371895

Crossref Full Text | Google Scholar

Lin, Z., Ma, J., Duan, J., Li, S. E., Ma, H., Cheng, B., et al. (2023). Policy iteration based approximate dynamic programming toward autonomous driving in constrained dynamic environment. IEEE Trans. Intellig. Transp. Syst. 24, 5003–5013. doi: 10.1109/TITS.2023.3237568

Crossref Full Text | Google Scholar

Liu, K., Zhang, H., Zhang, Y., and Sun, C. (2023a). False data-injection attack detection in cyber–physical systems with unknown parameters: a deep reinforcement learning approach. IEEE Trans. Cybern. 53, 7115–7125. doi: 10.1109/TCYB.2022.3225236

PubMed Abstract | Crossref Full Text | Google Scholar

Liu, R., Hao, F., and Yu, H. (2021). Optimal SINR-based dos attack scheduling for remote state estimation via adaptive dynamic programming approach. IEEE Trans. Syst. Man, Cybernet.: Syst. 51, 7622–7632. doi: 10.1109/TSMC.2020.2981478

Crossref Full Text | Google Scholar

Liu, T., Cui, L., Pang, B., and Jiang, Z.-P. (2023b). A unified framework for data-driven optimal control of connected vehicles in mixed traffic. IEEE Trans. Intellig. Vehicl. 8, 4131–4145. doi: 10.1109/TIV.2023.3287131

Crossref Full Text | Google Scholar

Lu, J., Wei, Q., and Wang, F.-Y. (2020). Parallel control for optimal tracking via adaptive dynamic programming. IEEE/CAA J. Automat. Sinica 7, 1662–1674. doi: 10.1109/JAS.2020.1003426

Crossref Full Text | Google Scholar

Mohan, A. M., Meskin, N., and Mehrjerdi, H. (2020). A comprehensive review of the cyber-attacks and cyber-security on load frequency control of power systems. Energies 13:15. doi: 10.3390/en13153860

PubMed Abstract | Crossref Full Text | Google Scholar

Mu, C., Ni, Z., Sun, C., and He, H. (2017a). Air-breathing hypersonic vehicle tracking control based on adaptive dynamic programming. IEEE Trans. Neural Netw. Learn. Syst. 28, 584–598. doi: 10.1109/TNNLS.2016.2516948

PubMed Abstract | Crossref Full Text | Google Scholar

Mu, C., Ni, Z., Sun, C., and He, H. (2017b). Data-driven tracking control with adaptive dynamic programming for a class of continuous-time nonlinear systems. IEEE Trans. Cybern. 47, 1460–1470. doi: 10.1109/TCYB.2016.2548941

PubMed Abstract | Crossref Full Text | Google Scholar

Pan, H., Zhang, C., and Sun, W. (2023). Fault-tolerant multiplayer tracking control for autonomous vehicle via model-free adaptive dynamic programming. IEEE Trans. Reliab. 72, 1395–1406. doi: 10.1109/TR.2022.3208467

Crossref Full Text | Google Scholar

Rudin, W. (1964). Principles of Mathematical Analysis, Volume 3. New York: McGraw-Hill.

Google Scholar

Song, S., Gong, D., Zhu, M., Zhao, Y., and Huang, C. (2023). Data-driven optimal tracking control for discrete-time nonlinear systems with unknown dynamics using deterministic adp. IEEE Trans. Neural Netw. Learn. Syst. 36, 1184–1198. doi: 10.1109/TNNLS.2023.3323142

PubMed Abstract | Crossref Full Text | Google Scholar

Teixeira, A., Pérez, D., Sandberg, H., and Johansson, K. H. (2012). “Attack models and scenarios for networked control systems,” in Proceedings of the 1st International Conference on High Confidence Networked Systems (New York, NY: Association for Computing Machinery), 55–64.

Google Scholar

Wang, W., Chen, X., Fu, H., and Wu, M. (2020). Model-free distributed consensus control based on actor–critic framework for discrete-time nonlinear multiagent systems. IEEE Trans. Syst. Man, Cybernet.: Syst. 50, 4123–4134. doi: 10.1109/TSMC.2018.2883801

Crossref Full Text | Google Scholar

Wu, H., Li, M., Gao, Q., Wei, Z., Zhang, N., and Tao, X. (2022). Eavesdropping and anti-eavesdropping game in uav wiretap system: A differential game approach. IEEE Trans. Wireless Commun. 21, 9906–9920. doi: 10.1109/TWC.2022.3180395

Crossref Full Text | Google Scholar

Xu, Y., Li, T., Yang, Y., Tong, S., and Chen, C. L. P. (2023). Simplified adp for event-triggered control of multiagent systems against fdi attacks. IEEE Trans. Syst. Man, Cybernet.: Syst. 53, 4672–4683. doi: 10.1109/TSMC.2023.3257031

Crossref Full Text | Google Scholar

Yang, W., Zheng, Z., Chen, G., Tang, Y., and Wang, X. (2020). Security analysis of a distributed networked system under eavesdropping attacks. IEEE Trans. Circuits Systems II: Express Briefs 67, 1254–1258. doi: 10.1109/TCSII.2019.2928558

Crossref Full Text | Google Scholar

Yang, X., Xu, M., and Wei, Q. (2023). Approximate dynamic programming for event-driven H_∞ constrained control. IEEE Trans. Syst. Man, Cybernet.: Syst. 53, 5922–5932. doi: 10.1109/TSMC.2023.3277737

Crossref Full Text | Google Scholar

Zhang, H., Qu, Q., Xiao, G., and Cui, Y. (2018). Optimal guaranteed cost sliding mode control for constrained-input nonlinear systems with matched and unmatched disturbances. IEEE Trans. Neural Netw. Learn. Syst. 29, 2112–2126. doi: 10.1109/TNNLS.2018.2791419

PubMed Abstract | Crossref Full Text | Google Scholar

Zhang, K., Liang, X., Lu, R., and Shen, X. (2014). Sybil attacks and their defenses in the internet of things. IEEE Intern. Things J. 1, 372–383. doi: 10.1109/JIOT.2014.2344013

Crossref Full Text | Google Scholar

Zhang, K., Zhang, H., Cai, Y., and Su, R. (2020). Parallel optimal tracking control schemes for mode-dependent control of coupled markov jump systems via integral rl method. IEEE Trans. Autom. Sci. Eng. 17, 1332–1342. doi: 10.1109/TASE.2019.2948431

Crossref Full Text | Google Scholar

Zhang, K., Zhang, H., Xue, W., and Zhang, R. (2022). “A robust control scheme for autonomous vehicles path tracking under unreliable communication,” in 2022 IEEE 11th Data Driven Control and Learning Systems Conference (DDCLS) (Chengdu: IEEE), 1413–1418. doi: 10.1109/DDCLS55054.2022.9858512

Crossref Full Text | Google Scholar

Zou, H., and Zhang, G. (2023). Dynamic event-triggered-based single-network adp optimal tracking control for the unknown nonlinear system with constrained input. Neurocomputing 518, 294–307. doi: 10.1016/j.neucom.2022.11.015

Crossref Full Text | Google Scholar