A 3-CS Distributed Federated Transfer Learning Framework for Intelligent Edge Optical Networks

Yang, Hui; Yao, Qiuyan; Bao, Bowen; Li, Chao; Wang, Danshi; Zhang, Jie; Cheriet, Mohamed

doi:10.3389/frcmn.2021.700912

BRIEF RESEARCH REPORT article

Front. Comms. Net., 29 July 2021

Sec. Optical Communications and Networks

Volume 2 - 2021 | https://doi.org/10.3389/frcmn.2021.700912

This article is part of the Research Topic Machine Learning Applications in Optical and Wireless Communication Systems and Networks View all 4 articles

A 3-CS Distributed Federated Transfer Learning Framework for Intelligent Edge Optical Networks

Hui Yang¹*

Qiuyan Yao¹

Bowen Bao¹

Chao Li¹

Danshi Wang¹

Jie Zhang¹

Mohamed Cheriet²

¹State Key Laboratory of Information Photonics and Optical Communication, Beijing University of Posts and Telecommunications, Beijing, China
²Department of System Engineering, University of Quebec’s École de Technologie Supérieure (ÉTS), Montreal, CA, Canada

With the rapid development of optical network and edge computing, the operation efficiency of the edge optical network has become more and more important, requiring an intelligent approach to enhance the network performance. To enhance the intelligence of the edge optical network, this article firstly provides the demand for the development of edge optical networks. Then, a cross-scene, cross-spectrum, and cross-service (3-CS) architecture for edge optical networks is presented. Finally, a federated transfer learning (FTL) framework, realizing a distributed intelligence edge optical network, is proposed. The usability of the proposed framework is verified by simulation.

Introduction

From the communication forecast by Gartner, there are no fewer than 50 billion devices and terminals in 2020 that can access the Internet. Furthermore, over 70% of data in networks need to be captured, analyzed, stored, and managed at the edge of the network by 2025 (Shi et al., 2019). Moreover, the demands for online remote telecommuting and online video conferencing have become one of the main reasons for the surge of network bandwidth requirement due to emergencies (Favale et al., 2020; Yu et al., 2021; Zhu et al., 2021), such as the worldwide COVID-19 epidemic. In line with the data statistics from the Ministry of Industry and Information Technology of the People’s Republic of China, during the COVID-19 epidemic period, the amount of Internet traffic in China has grown to approximately 50% more than that of the last year. At the same time, with the rapid development of optical communication techniques, the optical network gradually replaces the traditional electric network as the main networking infrastructure (Yang et al., 2015; Zhu et al., 2021). This constitutes the main body of the edge network, i.e., the edge optical network. Due to the above-mentioned phenomenon, we need to assess lots of network metrics, e.g., bandwidth, cost, and delay in terms of whether they satisfy the development of networking techniques.

Artificial intelligence (AI) technology, which is designed to achieve learning and decision-making, has been implemented in various scenarios of optical networks (Mata et al., 2018; Yao et al., 2019; Yang et al., 2019a). Zhu et al. (2019) argued that the demands for higher capacity and lower latency can be satisfied by future edge optical networks and further presented an AI-based control plane in the SDN-based optical network. Typically, it is an effective approach for remote clouds and edge clouds to support intelligent services by deploying AI technology (Yang et al., 2020a). To achieve that, Rahman et al. (2020) presented a privacy-preserving edge network framework that includes the common integrating MapReduce model. Nevertheless, due to the diversity of network scenarios, the dynamics of resource situation, and the variety of service connections, the edge optical network requires a resource-efficient and secure AI-based control framework for adapting to rapid networking development.

To overcome these challenges, this article expands on the previous work of Yang et al. (2020b) and focuses on three cases in edge networks including cross-scene, cross-spectrum, and cross-service, denoted 3-CS. Then, we put forward a 3-CS distributed FTL framework that is suitable for intelligent edge optical networks. In this case, a novel edge 3-CS architecture is presented. Meanwhile, to protect the privacy of all accessed terminals and devices in this network architecture, federated transfer learning is adopted as the main AI technology in the proposed distributed framework, which enhances the intelligence of the edge optical network.

Related Works

With the rapid development of AI technology, many related machine learning algorithms are applied for optical network optimization (Yang et al., 2020c; Yang et al. 2021). There are several main categories of AI applications in edge cloud optical networks, including traffic prediction, resource assignment, and failure recovery.

Specifically, Zhu et al. (2019) studied the lifecycle of optical channel provisioning operations for ROADM systems for use in edge cloud optical networks. For traffic prediction which is actually the time-series analysis (e.g., LSTM, DCRNN), the time-series traffic data are captured by the telemetry system, which forms the input of the prediction system. As the traffic volume varies and new requests arrive, this information can be used to set up the optical channels before the capacity is required. For failure recovery, the network status is monitored in telemetry and any exceptions will be sent to the SDN orchestrator for processing, and the adaptive control logic can be directly implemented. Network recovery procedures are then executed, and the network can be reconfigured by a physical layer controller. Aiming at the faulty nodes with huge traffic in the edge network, Lian et al. (2020) proposed a fault recovery algorithm. By running the dual migration and reinforcement learning-based recovery scheme, the time for failure recovery is proved to be independent of the number of failed services.

For resource allocation, Li X. et al. (2020) proposed a self-optimizing optical network (SOON) architecture by involving cloud and edge collaboration while introducing two kinds of AI, including control layer AI and on-board AI. The resource optimization process in this architecture exploits reinforcement learning (RL) to make resource assignment decisions to maximize network resource utilization. Also, Li Z. et al. (2020) presented a resource assignment algorithm based on deep RL embedded in the optical unit, which can achieve higher throughput in edge computing scenarios.

Other works have also studied the AI technology used in edge optical networks. Zhao et al. (2019) introduced on-board artificial intelligence into the edge optical networks, designed an architecture based on edge computing, and explored its potential applications. Under the proposed architecture, an alarm prediction algorithm based on on-board AI is proposed. On-board AI can assist in filtering out the operating parameters corresponding to each specific fault, and the controller can predict up to 99%.

Although the above work has improved the intelligence of the edge optical network, the efficiency issues in the intelligent process have hardly been considered, such as the training efficiency of the model. From this point, we proposed the FTL framework to carry out joint training of the model on the premise of protecting user data privacy. The challenge is how to select users with similar training needs and determine if there is a related training model that can be used directly for federated transfer.

Problem Statement

With the gradual diversification of service connection types of smart terminals and devices, bandwidth and latency demands of these connections are the main requirements of Internet traffic, especially with the increase in new emerging applications in networks (Yang et al., 2019b; Yang et al., 2020d). As for the bandwidth demand of service connection, virtual reality (VR) and augmented reality (AR) need to be serviced by 4K or Blu-ray bandwidth transmission, requiring more network resources. As for the latency demand of service connection, the low delay communication is the most needed for all live video broadcasting services (Yang et al., 2020e). To satisfy the above demands, edge computing becomes the most promising approach to support a huge bandwidth capacity and a low delay interaction via deploying a small data center in the edge network. In this case, terminals and devices in the edge network are unnecessary to share the data with the remote cloud data center of the core network for computing service, decreasing the resource consumption in the core network. Furthermore, considering the shorter communication distance with the shorter optical fiber length, which is shorter than that visiting the remote cloud data center, a lower delay may be obtained for each terminal and device at the edge of the network (Yang et al., 2019c). Thus, based on the increasing requirements of bandwidth and latency, the optical network needs to focus on the edge side of the network for better service provisioning.

Another case is that, in the edge optical network scenario, due to data privacy restrictions, the data generated by the terminals often need to be intelligently processed and analyzed locally. When the amount of local data is very small, it is easy to form data islands, burying the data value. From this aspect, the edge optical network also urgently needs to solve this problem. Furthermore, since the training time of some models is too long, it will lead to extremely low training efficiency, which is also the problem solved in this article.

3-CS Edge Optical Network Architecture Design

This section firstly presents a 3-CS architecture in the edge optical network, as shown in Figure 1. In the 3-CS architecture, cross-scene is set for new network implement scenarios in the real world, cross-spectrum is set for the available position range of the spectrum resource in the optical fiber, and cross-service is set for the emerging service connections in the edge optical network.

FIGURE 1

FIGURE 1. 3-CS edge optical network architecture.

For the sake of an intelligent edge optical network with huge bandwidth capacity, AI technology becomes the key point in networking design. With the emergency of new network applications, e.g., smart city, Industry 4.0, and remote online surgery, there are lots of different obvious features among these applications, reducing the response of the ordinary AI algorithm in the service area. With the continuous increase in new application scenarios, it is increasingly difficult for edge optical networks to support acceptable interconnection capabilities. Due to the well-known limited usable resources in networks, it is unwise to construct multiple AI models for different application scenarios, as this causes wastage of resources for additional computing and storing. Thus, an artificial intelligent architecture with security and also strong generalization capabilities is required.

Federated Transfer Learning Framework

Due to the coexistence of diverse services, scenarios, and heterogeneous resources, based on the 3-CS optical network architecture, an efficient network control plane is the main factor to enhance the bandwidth capacity of edge optical networks (Yang et al., 2016). Driven by intelligent operation, artificial intelligence technology provides a promising way for edge optical networks to achieve the functions of learning and decision-making. Besides, since the risk of data leakage exists anywhere, under the diversity of network scenarios, spectrum resources, and service connection types, the privacy protection for each accessed terminal and device has attracted more attention in the research studies of edge optical networks. In this case, it is important to implant an artificial intelligence algorithm into edge optical networks considering the security of networks when transferring the data between each terminal or device pair.

Recently, FTL is considered to be a very promising intelligent technology, which has been applied in medical health and secure image steganalysis (Chen et al. 2020; Yang et al., 2020f). To cover the above concerns, this article proposes an FTL framework, which works under the presented 3-CS architecture and is shown in Figure 2. In this framework, federated learning is used to encrypt sample training without data interaction, and transfer learning is utilized to improve the efficiency of the training model; and data transmission in the network can thus be protected by avoiding the risk of privacy leakage. Three layers are contained in the proposed FTL framework, including the data layer, the local learning (LL) layer, and the FTL layer. Specifically, the data layer works for data generation and storage among all terminals and devices in the network, where there is a situation that lots of small data exist with high application value. Note that the solution for this issue via the proposed FTL framework is one of the contributions of this article to enhance the utilization of small data. For the LL layer, each terminal or device in the network may adopt an adaptive AI algorithm that is suitable to itself, constructing the data training models for different training tasks by utilizing various types of local data. After that, all parameters of the model are encrypted and delivered to the federated transfer learning layer with the support of the federated transfer learning algorithm. It is worth noting that a service request is sent to the local learning layer from the FTL layer if transfer learning requirements between each terminal or device pair exist. If a transfer learning requirement occurs, the communication between this pair will be started after all related parameters have been already encrypted yet.

FIGURE 2

FIGURE 2. The proposed FTL framework.

Here, we present a simple example to demonstrate the entire process in the FTL framework, as shown in Figure 2. We suppose that data owner (terminal or device) 3 requires small data training. Firstly, data owner 3 may check whether the relevant model exists locally. If it exists, the local task model will be utilized to directly execute transfer learning locally; otherwise, a training demand will be delivered to the FTL layer. Then, if a related model is matched in the database of the FTL layer, the federated transfer learning process will be executed in the FTL layer; otherwise, the training demand will be recorded in the FTL layer, and a query request will be delivered to the local learning layer periodically until a related model is matched. We suppose that a suitable model for DO 3 is found in the database of the FTL layer, and then this model will be transmitted to DO 3 after finishing encryption.

Performance Evaluation and Result Analysis

In this section, we have carried out a simulation for the feasibility of the proposed framework and mainly test the efficiency of federated transfer learning. A multi-core server is set up for simulation, including 16 physical 2.20 GHz CPU cores, two GTX TITAN GPU cores, and 80 GB RAM running Ubuntu 18.04.2. Federated learning is implemented on five virtual edge nodes implemented by Docker, and Docker is used to build a simulation environment involving Tensor Flow and Python.

We assume that there is an image recognition task under the proposed 3-CS architecture. Two datasets are involved in the simulation process, including MNIST and CIFAR, and the CNN model is used (Kandel et al., 2020). We exploit federated learning first to jointly train the CNN model on the MNIST dataset. Then, we transfer the model to the CIFAR dataset for transfer learning. Note that, in order to improve the degree of protection of users’ local data, the MNIST dataset is divided into several parts and deployed to different users. When matching tasks, we set different values for different tasks and set the value of the image recognition task to 1. Since we divide the MNIST data into several parts and deployed it to different users, we will select the users whose task attribute is 1 as the federated users. In this process, the CNN model includes two convolutional layers and two fully connected layers. In transfer learning, the weight connection of the fully connected layer is removed to fine-tune the model. The learning rate is 0.01. The epoch of joint training is 150, and the epoch of transfer learning is 75.

We assume that number of federated users is indicated as N_F and simulate the training performance of the model under different numbers of federated users. The simulation results in Figure 3 show that, for the federated learning process, the training accuracy can reach 92, 94, and 93% when N_F is 10, 20, and 30, while the epoch is 150. After transfer learning, the training accuracy on the new dataset can reach 93, 94, and 94%, respectively. The simulation results further verify the feasibility of the proposed framework in edge optical networks.

FIGURE 3

FIGURE 3. Results of model transfer: (A)N_F = 10; (B)N_F = 20; (C)N_F = 30.

Conclusion

This article firstly provided the demand under the development of edge optical networks, presented a 3-CS architecture for optical networks, and finally designed an FTL framework to realize a distributed intelligence edge optical network. Note that federal transfer learning is utilized to guarantee the data privacy protection for each accessed terminal or device in edge optical networks, and the proposed framework provides basic support to fully mine the small data value. The simulation results prove the feasibility and effectiveness of the proposed framework.

Data Availability Statement

The original contributions presented in the study are included in the article/Supplementary Material, and further inquiries can be directed to the corresponding author.

Author Contributions

HY, QY, and BB designed the method. CL and DW wrote the manuscript. JZ and MC was responsible for proofreading.

Funding

This work has been supported in part by NSFC project (61871056), Beijing Natural Science Foundation (4202050), and Fund of SKL of IPOC (BUPT) (IPOC2021ZT04, IPOC2020A004).

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s Note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

Chen, Y., Qin, X., Wang, J., Yu, C., and Gao, W. (2020). Fedhealth: A Federated Transfer Learning Framework for Wearable Healthcare. IEEE Intell. Syst. 35 (4), 83–93. doi:10.1109/mis.2020.2988604