Skip to main content

ORIGINAL RESEARCH article

Front. Big Data
Sec. Big Data Networks
Volume 7 - 2024 | doi: 10.3389/fdata.2024.1448481

Cybermycelium: a Domain-Driven Distributed Reference Architecture for Big Data Systems

Provisionally accepted
Pouya Ataei Pouya Ataei *
  • Auckland University of Technology, Auckland, New Zealand

The final, formatted version of the article will be published soon.

    The ubiquity of digital devices, the infrastructure of today, and the ever increasing proliferation of digital products have dawned a new era, the era of big data (BD). This era began when the volume, variety, and velocity of data overwhelmed traditional systems that used to analyse and store that data. This precipitated a new class of software systems, namely BD systems. Whereas BD systems provide a competitive advantage to businesses, many have failed to harness the power of them. It has been estimated that only 20% of companies have successfully implemented a BD project. This is due to various challenges of adopting BD, such as organisational culture, rapid technology change, system development, data architecture, and data engineering. This paper aims to facilitate BD system development, architecture, and data engineering by introducing a domain-driven decentralised BD reference architecture (RA). This artefact was developed following the guidelines of empirically grounded RAs and has been evaluated in a real world scenario. At the end, a prototype of the artefact has been instantiated and utilised to solve a real problem in practice. Our results display a good degree of applicability and some thought-provoking architectural tradeoffs and challenges.

    Keywords: Big Data Reference Architecture, Big data architecture, Big data systems, Big data software engineering, distributed systems, Decentralised System Veracity Ver-1 Support data quality curation: classification, pre-processing, format reduction

    Received: 13 Jun 2024; Accepted: 04 Oct 2024.

    Copyright: © 2024 Ataei. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

    * Correspondence: Pouya Ataei, Auckland University of Technology, Auckland, New Zealand

    Disclaimer: All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.