Skip to main content

ORIGINAL RESEARCH article

Front. Neurorobot.
Volume 18 - 2024 | doi: 10.3389/fnbot.2024.1507289
This article is part of the Research Topic Multi-modal Learning with Large-scale Models View all 5 articles

Privacy-Preserving Named Entity Recognition for Multi-modal Learning via Reasoning on Noisy Deep Features

Provisionally accepted
Biao Shen Biao Shen 1*Su Yan Su Yan 2Meilingzhi Chen Meilingzhi Chen 2Wei Kong Wei Kong 2
  • 1 Jiangsu Xinshun Energy Industry Group, Nanjing, China, Nanjing, China
  • 2 Nanjing Electricity Supply Industry General Corp, Nanjing, China

The final, formatted version of the article will be published soon.

    The increasing demand for privacy-preserving data processing techniques has led to significant advancements across various fields, including Named Entity Recognition (NER) within multimodal learning frameworks. This paper presents a novel approach to NER that ensures privacy by reasoning on noisy deep features across diverse data modalities, such as text, images, and audio. By introducing controlled noise into the data representation, our method preserves privacy while maintaining the effectiveness of NER across multi-modal inputs. The approach leverages advanced deep learning models, noise-adaptive attention mechanisms, and denoising autoencoders to accurately identify named entities in text and other modalities, despite the presence of noise. We evaluate our method on benchmark datasets, including CoNLL-2003, OntoNotes 5.0, and BioNLP-2004, demonstrating that it achieves competitive performance with state-of-the-art models in both unimodal and multi-modal settings. Additionally, our approach provides strong privacy guarantees, as evidenced by differential privacy metrics and resistance to data reconstruction attacks across multiple data types. Experimental results show that our method strikes a balance between privacy preservation and NER performance, offering a robust and generalizable solution for privacy-sensitive multi-modal NLP applications.

    Keywords: Differential privacy, machine learning, named entity recognition, multi-modal learning, deep learning

    Received: 07 Oct 2024; Accepted: 22 Oct 2024.

    Copyright: © 2024 Shen, Yan, Chen and Kong. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

    * Correspondence: Biao Shen, Jiangsu Xinshun Energy Industry Group, Nanjing, China, Nanjing, China

    Disclaimer: All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.