AUTHOR=Ferraro Antonino , Galli Antonio , La Gatta Valerio , Postiglione Marco 

TITLE=Benchmarking open source and paid services for speech to text: an analysis of quality and input variety

JOURNAL=Frontiers in Big Data

VOLUME=Volume 6 - 2023

YEAR=2023

URL=https://www.frontiersin.org/journals/big-data/articles/10.3389/fdata.2023.1210559

DOI=10.3389/fdata.2023.1210559

ISSN=2624-909X

ABSTRACT=Speech to text (STT) technology has become increasingly popular in recent years due to the growing demand for automated transcription of spoken language. With the availability of both open source and paid services for STT, it is important to evaluate their performance and quality in order to select the most appropriate tool for a given task.In this paper, we present a benchmarking study of open source and paid STT services, with a focus on evaluating their performance in relation to the variety of input text. We consider six datasets collected from different sources, including interviews, lectures, and speeches, and used them as input for the STT tools.We evaluated the instruments using one of the standard metrics for evaluating STTs: the Word Error Rate (WER). Our analysis of the results revealed that the performance of the STT tools varied significantly depending on the input text, with some tools performing better on certain types of audio samples than others.Our study provides insights into the performance of STT tools when processing large amounts of data, as well as the challenges and opportunities presented by the multimedia nature of the data. We found that paid services generally outperformed open source alternatives in terms of accuracy and speed, but their performance varied depending on the input text.