Making dispersed data a visual event

In Desaparecido, Gatti (2022, p. 43) proposes a “fragmented, stammering” inquiry based on “emails, phone pictures, press cuttings, landscape fragments, pieces of magazines, field notes, road drawings, recordings, illustrations, napkin scribblings, newspaper clippings, and maps” to trace disappearance. Similarly, employing noisy data and official documents, witnesses' testimonies, pictures and videos, satellite data and imagery, and crowdsourced data, filmmakers, journalists, and activists are visualizing cases of human rights abuses, deprivation, and killing. Namely, the combination of information from traditional sources typical of human rights investigation with digital traces of disappearance made available by satellite data providers, governments, and people on platforms is currently being employed to weave stories of invisibility and neglect. The results are journalistic pieces, scientific investigations, court cases, and art installations.

This article reflects on previous investigations (Gutierrez, 2019, 2020a,b, 2022) and my recent experience as a jury member of the San Sebastian International Film Festival 2022, assessing seven films about invisibility. Drawing from Gatti (2022)—who favors the broadening of the concept of (forced) disappearance—, it offers insights into the possibilities of data for visual communication of social disappearance stemming from societal negligence, abuse, and cruelty and, therefore, elusive to datafication and observation.

As data activism expanded, OSINT-based forensics has provided material that has the virtue of having been collected as things happened. Open-source intelligence is a method that allows the gathering of data extracted from public sources, reclaiming and repurposing them to reconstruct different perspectives of an event. OSINT is not about a smoking gun or a witness's deposition but rather about collecting pieces of a visual, acoustic, and time riddle that, patched together, make the unseen visible. These techniques can make the lost voices of the disappeared rematerialize. People's data from selfies, and smartphone videos and pictures made available on YouTube, TikTok, or Facebook, combined with images from drone, radar, thermal, and CCTV cameras, satellite data and imagery, and footage released by authorities and news organizations can be verified, synchronized, and merged to compose a kaleidoscopic landscape of an occurrence and transform it into an event. The notion of the event is critical since, in the time of the platforms, it is not enough that a disaster, atrocity, act of war, or forced displacement affects many people, but whether it is registered as an image and exposed to publics (Pena, 2020). The transformation of data into an event depends, too, on whether investigations get promoted by platforms' recommendation algorithms.

Eight issues emerge from the reflections on the opportunities and challenges of making dispersed data a visual event.

First, digital tools' accuracy has increased journalism and activism's capacity for impact beyond their typical audiences and advocacy circuits, generating awareness and substantiation. Traditional human rights inquiries and journalistic, activist, and humanitarian practices used to depend on historical documents or the eyewitnesses' ability to remember. Not anymore. OSINT-based data and other types, such as citizen data, confer robustness to the stories of disappearance. Besides, they offer the possibility of counterweighing the challenges identified by organizations such as the Office of the High Commissioner for Human Rights related to witnesses' trauma and faulty memories (OHCHR, 2001). Digital traces found on sharing platforms fill gaps in the story and offer the opportunity to see things as they happened from the perspective of who holds the data-gathering device. Mixing the right amounts of data analysis can bestow a scientific aura to a story, resulting in impact, credibility, and reach.

Second, scientific legitimacy is insufficient to enthuse action; emotive communication is necessary. Paraphrasing ŽiŽek (2007), rationality will not mobilize people because it is not a veil of ignorance that causes our inaction but a lack of feelings. Nevertheless, striking a balance between data and emotion is hard, and this issue has generated criticism. An example is the 2020 Forensic Architecture (2020) investigation of a shipwreck in 2015 inside European waters, where at least 43 unassisted people died. Although the FA documentary explains in detail what happened, shows images, and is guided by survivors' experiences, the narrator's voice, the map occupying the screen, and the data analysis set a distance between the horrific facts and the spectator. In contrast, the documentary Purple Sea chronicling the same event, by director Alzakout (2021)—one of the survivors—, puts the viewer in the middle of the shipwreck. While FA extracts meaning and order from panic and chaos, Purple Sea progresses bluntly, layering a profoundly personal monolog. In both, there is something ethically problematic: witnessing suffering. But comparing these films, Goo (2021) argues that Purple Sea manages to propose that there are times when violence cannot be dissected, analyzed, or seen with the clarity of distance to be understood; you must feel it. Meanwhile, Ruido (Beristain, 2022)—about one of the mothers, sisters, and daughters looking for missing women in Mexico— and Los Reyes del Mundo (Mora, 2022)—narrating the journey of five outcasts in search of the promised land in Colombia— resort to dreams and delirium to talk about violence and disappearance. Ruido offers facts (i.e., there are thousands of victims of femicide), but it is most effective in transmitting horror in its dreamlike scenes and through the employment of noise and silence. In Los Reyes del Mundo, the watcher learns about the drama of displacement, but soon gets lost in otherworldly scenes and realizes that the protagonists are already dead. This is not to say that the excruciating images captured by Alzakout's camera as she half drowns are comparable to the artistic mirror games of Ruido or Los Reyes del Mundo. However, by demanding that spectators suspend their disbelief to simply feel, both the documentary and the movies effectively engage and communicate calamity. FA's films, instead, require the opposite, that spectators sharpen their wits to comprehend the intricacy of forensic research; thus, their films are caught in the middle of evidence and emotion production. This disparity raises questions about data-driven techniques as effective communication.

Third, the forum where these data-based stories are disseminated is essential; if it does not exist, they must be created. Examples are the documentaries and art exhibitions where FA shows its investigations, expanding the publics it can reach beyond courts and the usual campaigning targets. FA's videos and installations perform their presence so they can make claims. Thus, data activism broadens its reach to viewers that might not be aware of the proposed issue, multiplying the domains through which their findings can be distributed. These fora increase communication opportunities and constrain how forensic evidence can be staged, decoded, and discussed. This expansion might have inspired others. Although direct influence cannot be demonstrated, data activist methods have impacted how journalists and others operate today. Conflicts in countries such as Ukraine, Syria, or South Sudan are being increasingly investigated using OSINT techniques, verifying and chronolocating audio-visual material found on platforms, synchronizing timelines, and tracking landmarks and individuals by using visual and audio markers (e.g., BBC World Service, 2019; Triebert et al., 2019). Courts are starting to use OSINT in their prosecution of crimes too.

Four, to counteract the propaganda machinery, it is necessary to offer robust evidence, summing up the power of data and visual narratives resulting from synchronization. However, not everybody can invoke these forces. FA's director Weizman (2010) argues that only “the criminal can solve the crime”. For example, OSINT-based data activism has been criticized for using the same tools employed by human rights violators and utilizing their logic and narratives, hence, somehow legitimizing them. Accordingly, activists should be aware that applying sophisticated surveillance methods, forensics, and data analytics has drawbacks and can generate criticism.

Five, connected to the earlier point, data analysis imposes asymmetries, which should be explored. In the era of platforms, there are differences between who generates data (i.e., ordinary people) vs. who uses them (i.e., big data companies and governments). By offering data agency to ordinary citizens and organized society, data activism manages to smooth some of the asymmetries behind the data infrastructure. Nevertheless, data activist projects are not egalitarian. One example is crisis mapping during an emergency. It is not the same to be a data reporter (i.e., a citizen enduring the crisis and offering data to feed the map), a mapmaker (i.e., digital humanitarians making the map from the safety of their homes), or a humanitarian (i.e., using the map for prioritizing aid on the ground). Many of these asymmetries cannot be avoided, as they are an inevitable consequence of work division and collaboration. But the asymmetries lying behind data practices should always be investigated and, if possible, mitigated.

Six, access to data analysis also has to do with data literacy, which depends on people's ability to access the data, tools, capabilities, and opportunities to exercise their data agency. So, rephrasing Kennedy (2016), data literacy should become ordinary, so people incorporate data practices into their everyday lives, exercising their data agency independently, for data activism to thrive.

While Gatti wonders about this scrappy data-based inquiry in social science, this article considers it suitable for journalists, activists, and filmmakers. Thus, seven, these merging of practices should come with new responsibilities. Residue studies and bricolage techniques can generate bigger pictures, making visible the invisible. But what demands are made on the reliability and legitimacy of research based on mixing different data from diverse origins, analyzed in different ways, and put together for publication in a selective manner? Should journalism, film, art, and activism continue adopting science standards? To what point is the resulting meta-picture (Mitchell, 1994) an optical illusion produced by digital analysis (e.g., network analysis, heatmaps, or collaborative cartography)? These questions should be made both in theoretical research and data practices.

Finally, eight, while journalism, activism, and humanitarianism embrace these data approaches, documentary films are still to catch up. The short documentaries launched by FA, among others, are pioneering experiences (Hepp, 2016) pointing to new filmic possibilities to visualize the non-apparent. Still, their impacts seem limited.

Summarizing, data tools bestow precision and trustworthiness to filmmaking, journalism, and activism; mixed with emotion, they have the potential for reaching new publics, generating evidence, and engendering impact. However, there are shortcomings to data analysis, including embedded asymmetries and the new responsibilities it imposes. The difficulty of investigating its impact also limits the theorizing of data activism. Nonetheless, this analysis suggests that OSINT and other data methods could be part of an alternative cinematographic production that allows the piecing of “fragmented, stammering” research and is suited to the digital platforms that shape what an event is.

Author's note

This article reflects on previous investigations (Gutierrez and Milan, 2017; Gutierrez, 2019, 2020a,b, 2022), a literature review, and my recent experience as a jury member of the San Sebastian International Film Festival 2022, where I assessed seven films that talk about invisibility. Drawing from Gatti (2022)—who argues for broadening the concept of forced disappearance—, it offers insights into the possibilities of data for visual communication of social disappearance.

Author contributions

The author confirms being the sole contributor of this work and has approved it for publication.

Funding

This work was part of a project funded by the State Research Agency of the Ministry of Science and Innovation of the Government of Spain PID2020-114445RB-I00: ARES Análisis de Resistencias Antifeministas.

Conflict of interest

The author declares that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher's note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

Alzakout, A. (2021). Purple Sea. Documentary. Available online at: https://vimeo.com/ondemand/purplesea

Google Scholar

BBC World Service (2019). Sudan's Livestream Massacre. Available online at: https://www.youtube.com/watch?v=dR56qxM4kHA (accessed January 22, 2023).

Google Scholar

Beristain, N. (2022). Ruido, Netflix.

Google Scholar

Forensic Architecture (2020). Shipwreck at the Threshold of Europe. Documentary. Available online at: https://forensic-architecture.org/investigation/shipwreck-at-the-threshold-of-europe (accessed January 22, 2023).

Google Scholar

Gatti, G. (2022). Desaparecidos: Cartografías Del Abandono. Madrid: Turner.

Google Scholar

Goo, E. (2021). “Remains to be seen: the video work of forensic architecture,” in MUBI, 2021. Available online at: https://mubi.com/notebook/posts/remains-to-be-seen-the-video-work-of-forensic-architecture (accessed January 22, 2023).

Google Scholar

Gutierrez, M. (2019). “Cartographic emotions: how interactive maps mobilize people in geoactivism,” in Data Visualization in Society, Kennedy, D., and Engebretsen, N. M. Amsterdam: University of Amsterdam.

Google Scholar

Gutierrez, M. (2020a). Data Activism and Meta-Documentary in Six Films by Forensic Architecture. Studies in Documentary Film. https://doi.org/10.1080/17503280.2021.1908932 (accessed January 22, 2023).

Google Scholar

Gutierrez, M. (2020b). Data and documentaries: Methodological hybridizations in activism. Catalan J. Commun. Cult. Stud. 12, 315–331. doi: 10.1386/cjcs_00035_1

CrossRef Full Text | Google Scholar

Gutierrez, M. (2022). Documenting the invisible: how data activism fills visual gaps. Papeles Del CEIC. 2, 2. doi: 10.1387/pceic.23117

CrossRef Full Text | Google Scholar

Gutierrez, M., and Milan, S. (2017). “Technopolitics in the age of big data,” in Networks, Movements & Technopolitics in Latin America: Critical Analysis and Current Challenges, eds F. S. Caballero and T. Gravante (Palgrave Macmillan), 95-109, Available online at: https://ssrn.com/abstract=2935141

Google Scholar

Hepp, A. (2016). Hepp, A. (2016). Pioneer communities: collective actors in deep mediatisation. Media Cult. Soc. 38, 918–933. doi: 10.1177/0163443716664484

CrossRef Full Text | Google Scholar

Kennedy, H. (2016). Post, Mine, Repeat: Social Media Data Mining Becomes Ordinary. London: Palgrave Macmillan.

Google Scholar

Mitchell, W. J. H. (1994). Picture Theory: Essays on Verbal and Visual Representation. Chicago, London: The University of Chicago Press.

Google Scholar

Mora, L. (2022). Los Reyes Del Mundo, Netflix.

Google Scholar

OHCHR (2001). “Trauma and Self-Care,” in Manual on Human Rights Monitoring. Geneva: Office of the United Nations High Commissioner for Human Rights. Available online at: https://www.ohchr.org/sites/default/files/Documents/Publications/Chapter12-MHRM.pdf (accessed January 22, 2023).

Google Scholar

Pena, J. (2020). El Cine Después de Auschwitz. Barcelona, Spain: Catedra.

Google Scholar

Triebert, C., Hill, E., Browne, M., Hurst, W., Khavin, D., and Froliak, M. (2019). “How times reporters proved russia bombed syrian hospitals,”. The New York Times, (2019). Available online at: https://www.nytimes.com/2019/10/13/reader-center/russia-syria-hospitals-investigation.html (accessed January 22, 2023).

Google Scholar

Weizman, E. (2010). “Forensic architecture: only the criminal can solve the crime,” in Radical Philosophy, 2010. Available online at: https://www.radicalphilosophy.com/article/forensic-architecture (accessed January 22, 2023).

Google Scholar

ŽiŽek, S. (2007). ‘Religion between Knowledge and Jouissance'. Lacan.com. Available online at: https://www.lacan.com/zizsmokeonthewater.html (accessed January 22, 2023).

Google Scholar