Jana Pflaeging- University of Salzburg, Salzburg, Austria
This mini-review gives an overview of diachronic multimodality research, an approach that multimodality scholars have pursued only rarely so far. Acknowledging the comparably large share of case studies in this line of research, this paper sets out by surveying empirical contributions. Many of them also provide discussions of selected theoretical and methodological aspects, which are subsequently collated and complemented with concepts from a number of more extensive theoretical proposals. Identifying main developments in diachronic multimodality studies and avenues for future work, this review seeks to support its growth into a mature research strand with a solid theoretical basis, a versatile methodological toolbox, and a broad range of research objects.
Introduction
Multimodality research has so far mainly been synchronic in nature (van Leeuwen, 2005), p. 26. The development of multimodal artefacts and interactions over time has received comparably little attention (Hiippala and Tseng, 2017; Stöckl, 2017, p. 263). At the same time, scholars have stressed the potential of adopting a diachronic perspective to gain a deeper understanding of the historical situatedness of multimodal communication (van Leeuwen, 2005, p. 142; Stöckl, 2009; Waller, 2012, p. 241; Bednarek and Caple, 2014, p. 150; Bateman et al., 2014, p. 10). This mini-review surveys existing empirical research with a diachronic angle and synthesizes previous theoretical and methodological proposals. It thereby seeks to contribute to the further growth of diachronic multimodality studies into a mature research strand.
Previous empirical research
While diachronic multimodality research is generally scarce, a number of studies have appeared over the past 25 years that show a keen interest in the developments of multimodal communication and the factors that initiate and shape such processes (see contributions to special issues, e.g., Hiippala and Tseng, 2017, to collections, e.g., Hess-Lüttich et al., 1996; Holly, 1998; Schneider and Stöckl, 2011; Evangelista Allori et al., 2014; Hauser et al., 2014; Brock et al., 2019; Stöckl et al., 2019, and many stand-alone journal articles). Although points in time rarely overlap and periods differ in scope, most existing studies focus on contemporary multimodal practices and compare them to earlier related data. Perhaps not surprisingly, accounting for change (rather than stability) has been the most prominent research interest. The subsequent overview spotlights previous empirical contributions, showcases the diversity of approaches, scope of data-sets, and contributions to theory-building.
Print media
A natural consequence, perhaps, of the long history of print media, the majority of diachronic multimodality research has focused on page-based static artefacts. Even before the study of multimodality gains traction in the mid-1990s (see, e.g., Kress and van Leeuwen, 1996; Yates and Orlikowski, 1992) provide a detailed account of the business memo and describe how multimodal features change over time. A further study of corporate discourse is Deng and Feng’s (2022) work on the photographic representation of academics in 132 annual reports from six major universities in Hong Kong. Analysing materials from 1994/95–2015/16, they identify a development from context-rich group depictions to individual portraits that feature direct gaze.
Shifting the focus to mass-media journalism in the early 1990s, Bucher (1996, p. 35–41) and Bucher (1998, p. 70–94) describes an increase in the use of photographs and diagrams in German newspapers. The attested functional redistribution across modes is supported by a more modular page layout, which provides entry points to browsing readers (Bucher, 1996, p. 41–48). Durrani’s (2020) study of 840 language-image combinations in Time Asia (1981–2010) describes a similar shift towards an image-centric news discourse. This development can equally be tracked in National Geographic, where feature articles (as published in 1915, 1965, and 2015) experience a reorganisation of the page space, with an increase in spreads that contain nothing but a large photograph and a short caption (Pflaeging, 2017a, 2017b). Spurred by changes in how captions are rhetorically tied to photographs, the double-page image-caption cluster develops into a key generic pattern that offers entry points, supports narrative interpretations across a longer feature article (Pflaeging, 2017a) and, over time, splits off into a stand-alone genre (Pflaeging, 2020a, p. 110–114). Stöckl’s (2017) longitudinal study of the genre profiles, image types and language-image relations in the MIT Technology Review illustrates how changes at the genre-level are inextriably linked to higher-level developments, e.g., in genre profiles and in wider social domains (Stöckl, 2017, p. 273). Müller-Lancé’s (2016, 2019) work on 400 French and German action-sports magazines (since the 1970s) identifies economic pressures as a main driving force of change.
Bateman (2008, p. 229–248; Bateman, et al. 2004, 2007 and Bateman, 2014a, p. 246–254) offers a diachronic account of expository discourse by the example of ornithological field guides (1924, 1972, 1994, 1996). Applying his Genre-and-Multimodality (GeM) framework, he tracks a shift towards increasingly complex layout structures (Bateman, 2008, p. 230–235) and a redistribution of content elements across a broader variety of semiotic modes (Bateman, 2008, p. 235–240). Focusing equally on educational materials, Bezemer and Kress (2009, 2016) study a corpus of 240 pages from 23 school textbooks (1930s, 1980s, 2000s). Reminiscent of Bateman’s (2008) findings, they describe a trend towards a more extensive and complex use of page-based modes such as layout or typography (Bezemer and Kress, 2009, p. 256–260). The deployment of pictorial modes undergoes a shift from mainly photography (1930s, 1980s) to mostly drawings in the 2000s (Bezemer and Kress, 2009, p. 254). A recent contribution to this line of study is Keles and Yazan’s (2021) critical discourse analysis of gender representations in a series of five school textbooks (1993–2019). Results shows that, rather than mitigating gender inequities, learning materials continue to consolidate heteronormative representational practices (Keles and Yazan, 2021, p. 138).
Scholars have also turned to the development of comics media. Cohn et al. (2017, p. 19), for instance, conducted a large-scale corpus analysis of American superhero comics (1940s–2010s) with a view to multimodal interactions, attentional framing, and semantic relationships between panels. Results suggest a trend towards more complex visual narrative structures and a reduced functional load carried by verbal elements (Cohn et al., 2017, p. 30, 34). Bateman et al. (2019, p. 216) investigate a similar data-set (1,260 pages from American superhero comics, 1940s–2000s), but place a more specific focus on page composition. Combining corpus-linguistic and multimodal methods, they offer a highly systematic medium-specific description of individual layout changes and correlations between them (Bateman et al., 2019, p. 216–223).
Hiippala’s (2015) study applies the GeM-framework to a large set of tourist brochures and describes, for instance, a more elaborate use of layout after the advent of desktop-publishing. Turning to questions of genre ancestory, Molnar (2019) identifies the herbal, the trade card, and the private letter as predecessors of print-ad genres of the Enlightenment era. Further empirical work on advertisements offers theoretical remarks on why accelerated change is preferred in persuasive discourse to meet audience expectations (Cook, 2006, p. 224; Stöckl, 2010, 2014).
Digital media
One of the earliest longitudinal studies of digital media is Zhang and O'Halloran’s (2013) research on university homepages. Their work, informed by critical discourse analysis and social semiotics, reveals that education is increasingly framed as a lifestyle and that target audiences have shifted from national to global communities. Based on a corpus of 100 blogs, Schildhauer’s (2016, Ch. 6) history of the personal weblog sheds light on the genre’s development regarding layout, image types, and language-image links in response to technological advancements and emerging genre contraints. Pflaeging (2020b) and Dynel (2022) turn to viral online communication, studying soft-news items (listicles) and memes, respectively. While Dynel (2022, p. 73) finds that memes remained largely stable over 12 months, Pflaeging’s (2020a, p. 240–242) data from 2014 and 2017 suggests that generic patterns may just as well bend quickly in order to persuade users to share a listicle with their social networks. Finally, Stamenković (2022) studies the stylistic development of video-game screens from the Football Manager series. He finds that, over time, screens feature less language and more pictorial elements and show a growing diversity of sub-canvases.
Audio-Visual Media
While diachronic research on audio-visual media is comparably rare, Luginbühl (2014, 2019) provides a comprehensive, culture-contrastive account of TV news programs in the U.S. and Switzerland. Analysing 76 programs broadcast between 1949 and 2005, Luginbühl tracks changes in individual genres and the genre profile, and makes important contributions to a theory of genre development as well (Luginbühl, 2014, Ch. 9, 10; Luginbühl, 2015a; Luginbühl, 2019, p. 133–136). Brock (2019) pursues a similar interest in theory-building and describes various “genre-constitutive acts” of contemporary TV sitcoms (Brock, 2019, p. 123). Graakjær (2019), in turn, investigates a corpus of 475 McDonald’s TV commercials (2003–2018) focusing on the sonic logo. Inherent to musical practice, his study makes particularly clear that alteration and adaptation are an integral part of generic development (Graakjær, 2019, p. 580).
As suggested above, the vast majority of publications in diachronic multimodality research are empirical case studies, applying frameworks and concepts from social semiotics (e.g., Molnar 2019; Durrani, 2020; Deng and Feng, 2022), critical multimodal discourse analysis (e.g., Zhang and O’Halloran, 2013; Keles and Yazan, 2021; Dynel, 2022), empirically-oriented multimodal discourse analysis (as proposed by Bateman, Wildfeuer, and Hiippala, see e.g., Hiippala, 2015; Stamenković, 2022), and media/text linguistics (e.g., Bucher, 1996; Luginbühl, 2014; Schildhauer, 2016; Stöckl, 2017; Brock, 2019). Based on these empirical accounts, however, a number of theoretical assumptions can be deduced and synthesized with existing proposals.
Theoretical aspects
Genre as a focal point
For its strength in explaining the dynamics of stability and change of multimodal practices, many works in diachronic multimodality research have placed an emphasis on genre, usually with reference to paradigms such as “genre as social action” (Miller, 1984) or “genre as social semiotic” (Martin and Rose, 2008). Both traditions see communication as driven by social needs that arise from socio-historical contexts. As contexts change, participants explore new multimodal choices but naturally also rely on familiar patterns in text production and reception (Luginbühl, 2019, p. 132; also Eckkrammer, 2011, p. 196). Thus, while genres and patterns are “repurposed, redesigned and re-deployed” (Bateman et al., 2014, p. 10), the connection to genre ancestors is never lost (Lemke, 1999; also Molnar 2019; Sommer 2019). When genres migrate to new medial environments especially, established features are often initially retained for purposes of consolidation and orientation (Schildhauer, 2016; also Eckkrammer, 2011, p. 193–195).
Developments are stratified across various levels of abstraction (Bateman, 2008, p. 229; Luginbühl, 2014, 2019; Stöckl, 2017; Pflaeging, 2019, 2020a). At pattern-level, single (multi)modal choices (e.g., a layout pattern) may be perpetuated (pattern stability) or used more/less frequently over time (pattern change through strengthening or weakening), may be internalised through borrowing from other genres, or are novel creations (Pflaeging, 2017a, p. 259; Pflaeging, 2019, p. 77). Such individual developments were found to cluster and correlate at genre-level (see Cohn et al. 2017; Bateman et al., 2019; Deng and Feng, 2022). If the distribution of genre-typical patterns becomes increasingly untypical (Lemke, 1999), there is a strong indication of genre change (Luginbühl, 2014, p. 336), e.g., in the form of standardisation or hybridisation (Luginbühl, 2019, p. 136). When bundles of semiotic choices diverge noticeably from established use (Bhatia, 2014, p. 92), scholars speak of genre split (Lemke, 1999; Luginbühl, 2014, p. 335; Brock, 2015, p. 207; Schildhauer, 2016, p. 41; see Pflaeging, 2017a, p. 77–78). Developments like these, in turn, may result in changes at the genre profile-level (Luginbühl, 2015a, 2019, p. 133–134). By describing the genre repertoire, genre frequencies and genre networks typical of a given medium, statements can be made about a community’s multimodal genre space (Bateman, 2008, p. 225–229).
Factors that influence diachronic developments
Both empirical and theoretical works draw attention to the factors that shape the development of multimodal practices. Below, such factors will be discussed invidually, although they are tightly intertwined (Pflaeging, 2017a,b; Cohn et al. 2017, p. 32; Stöckl, 2017; Graakjær, 2019, p. 573).
Among the most central driving forces is communicative function (Brock, 2019, p. 120; also Eckkrammer, 2011, p. 196; Stöckl, 2017, p. 273), which is most apparent when genre is attributed a key role in the description of communicative practice. In advertising (Cook, 2006; Molnar, 2019) and viral online discourse especially (Pflaeging, 2020a), where texts are meant to persuade to buy or click, this may result in accelerated change. Brock (2019) makes a similar argument for humour, where textual patterns quickly fail to evoke humourous interpretations. The functional range of a genre may also shift or become more diverse over time, which equally has repercussions on the text design (Brock, 2019, p. 120).
Genre expectations are another factor of influence (Hiippala and Tseng, 2017; Pflaeging, 2017a; Molnar 2019, p. 31). As socio-cognitive entities, genres are “devices for sense-making” (Lomborg, 2014, p. 45) and serve as frames of reference for the interpretation of multimodal artefacts and interactions (see also Cohn et al. 2017, p. 33). This creates the need for perpetuating established textual patterns to some extent (Luginbühl, 2019, p. 130, in ref. to Schildhauer, 2016, p. 259–261), especially in mass-media journalism. Here, participants experience a disjunction of place and time between text production and reception (Pflaeging, 2017a, p. 257–258), audiences remain imaginary and feedback is naturally “delayed” (Bell, 1984, 1991a, 1991b). While journalistic practice is equally subject to innovation, diachronic studies have revealed tendencies towards standardisation (Pflaeging, 2017b, p. 198; Luginbühl, 2019, p. 135; Pflaeging, 2020a, p. Fn. 7), using corporate-design manuals and templates. Advertisements, viral soft-news items, comics or sitcoms, on the other hand, can be expected to exploit the potential for creativity more. Finally, developments may also be shaped by recipients’ awareness of communicative trends in the media landscape more generally, e.g., the rise of Japanese mangas on the American market (Cohn et al. 2017, p. 24), or a growing visualisation (Bucher, 1996; Caple, 2013, p. 7; also Krotz, 2015), clusterisation (Bucher, 1996), or atomization (Knox, 2007, p. 48) in journalism.
In advertising (Stöckl, 2010, p. 150), humourous discourse (Brock, 2019, p. 122), journalism (Pflaeging, 2017b, p. 201–202), and even more so in comics (Cohn et al. 2017, p. 32) and social media discourse (Dynel, 2022), change in multimodal practices may be spurred by an individual’s creative ways of breaking with conventions, which may be deliberate or entirely coincidental (Pflaeging, 2017b, p. 201–202; Brock, 2019, p. 123). Change and stability may also result from developments in participant constellations more generally. Cohn et al. (2017, p. 32), for instance, assume that a revaluation of the role of the penciler may have led to the rise of visual storytelling in comics. In a similar vein, entire production teams may undergo processes of expansion and professionalisation, e.g., at National Geographic (Pflaeging, 2017b, p. 197–198; also Stöckl, 2017, p. 263). Developments may equally be fueled by changes in audiences, which may grow, diversify (Pflaeging, 2017a, p. 197–198), or experience new degrees of involvement (see Brock, 2019, p. 114; Sommer 2019, p. 246; see also Meier and Marx 2019).
Medium-related advancements are also known to drive the development of multimodal practices, although they never determine them (Luginbühl, 2015b, p. 14; Pflaeging, 2017b, p. 192; Stöckl, 2017, p. 263; Luginbühl, 2019, p. 136; Brock, 2019, p. 120). Affordances of a given medium (and its material, technological and infrastructural qualities, Pflaeging, 2017b, p. 202; Stöckl, 2017, p. 273) may limit communicative choices, but participants are often seen to overcome supposed restrictions (Luginbühl, 2015b, p. 14–15). Shifting the focus further to the sociological dimension of media, practices may be influenced by a growing mediatization of social spheres (Androutsopoulos, 2014), incl. Journalism (Kammer, 2013) where multimodal practices have become reminiscent of a so-called media logic (Altheide and Snow, 1992; Stöckl, 2017, p. 272–273).
Such developments are closely tied to socio-economic factors as well (Eckkrammer, 2011, p. 191). Müller-Lancé (2016, p. 596) study of action-sports magazines, for instance, illustrates how tight budgets (due to changing sales figures and sponsorships) may result in an outsourcing of journalistic tasks to guest contributors and even audience members. Likewise, early journalistic practice at National Geographic was shaped by limited funds for photographic reproduction and staff. A steep rise in circulation numbers soon led to a much healthier financial situation, more editorial staff, more advanced production techniques, and, ultimately, changes at the discourse-level (Pflaeging, 2017b). Though cause-effect relations seem less immediate, communicative developments can also be impacted by the socio-political dynamics in a given cultural and historical context (Brock, 2019, p. 121; Luginbühl, 2019). Factors such as the rise of neoliberalism and the marketisation and commodification of higher-education are typically identified within critical multimodal discourse analysis, where systemic-functional-linguistic approaches allow for constructing a connection between texts and socio-cultural contexts (Zhang and O'Halloran, 2013; Deng and Feng, 2022).
Data collection and methodological aspects
Data collection and corpus compilation
Diachronic multimodality research requires data from at least two different points in time, although choosing them is far from trivial. To cover a longer phase of continuous journalistic output, Luginbühl (2014) and Pflaeging (2017b, 2019) sample from (evenly distributed) points in time, with smaller and larger gaps in between (also Stöckl, 2017; Sommer 2019; Pflaeging, 2020b; Stamenković, 2022). Their approaches differ, however, in that Luginbühl’s collection was guided by news events, whereas Pflaeging accounted for a phase of genre consolidation after moments of editorial reorientation. Stöckl (2017, p. 274) and Brock (2019), in turn, propose to focus on texts that capturemoments of innovation in a genre’s or medium’s history. Yet other studies seek to reflect more closely the amount of available data, e.g., by sampling from densely-distributed points in time (Molnar 2019; Dynel, 2022) or by selecting (nearly) all data available for a given period, medium, or genre (e.g., Hiippala, 2015; Graakjær, 2019; Keles and Yazan, 2021; Deng and Feng, 2022). Studies on web-based genres have used the Internet Archive (Schildhauer, 2016, p. 218) and web-scraping techniques for data collection (Dynel, 2022).
As results are to be related or relatable to map a development, diachronic data-sets need to show similarities in cultural contexts (e.g., Luginbühl, 2014; Sommer 2019), medium (e.g., Luginbühl, 2014; Keles and Yazan, 2021; Deng and Feng, 2022), genre (e.g., Schildhauer, 2016; Pflaeging, 2020a, 2020b; Dynel, 2022; Stamenković, 2022) or, at least, a strong similarity in textual function (Eckkrammer, 2011, p. 203), topic (Stöckl, 2017, p. 263), or mode (Graakjær, 2019; Bateman et al., 2019; Deng and Feng, 2022). Some studies have explored an ethno-categorical approach to genre in order to ensure comparability (Luginbühl, 2014; Schildhauer, 2016; Pflaeging, 2019; Sommer 2019) and to avoid circularity in sampling (Thomas, 2019, p. 86). When diachronic corpora are meant to cover extensive periods of time, their compilation can be challenging due to a limited avalability of materials or semiotically authentic, objective accounts (e.g., only a written report of a dynamic theatrical performance), or an insufficient preservation of historical materials that requires time-intensive digitalisation. Also, adding rich meta-data that is true to a given socio-historical context is not always straightforward (e.g., allocating genre categories, authorship, dates of publication) and working with historical materials generally requires a change of perspective.
Accounting for different socio-historical contexts
Pragmatics- and discourse-oriented approaches to multimodal analysis (Bateman, 2014b, Ch. 11–12) emphasise the sensitivity of discourse interpretations to contexts of use. The analysis of multimodal texts from sometimes distant points in time thus requires an awareness and careful reconstruction of socio-historical contexts (see, e.g., Bateman, 2008, p. 237; Luginbühl, 2014; Schildhauer, 2016; Pflaeging, 2017b; Molnar 2019; Brock, 2019; Cohn et al. 2017, p. 33–34). Striking a balance between detailed, context-sensitive descriptions of qualitative differences and the aim to chart general developments remains a challenge in diachronic multimodality research.
Analysing large(r) corpora and open research data
Empirical multimodality research has recently begun to turn to larger-scale data-sets (Pflaeging et al., 2021, p. 13) and research objects of considerable modal complexity (Bateman, 2021, p. 35). With diachronic corpora potentially multiplying the data under study, the issue of feasibility (Hiippala, 2017, p. 277, also in ref. to Thomas, 2009, p. 245) has become pressing. To scale-up, scholars explore new ways of annotating data using computational (Hiippala, 2016) and crowd-sourcing methods (Hiippala, 2023), and to implement statistics (Hiippala, 2015; Cohn et al. 2017; Bateman et al., 2019; also Bateman et al., 2017, Ch. 6; Bateman and Hiippala, 2020) and AI (Semedo and Magalhães, 2019).
The increasing scale of available data-sets is not least due to recent efforts in the spirit of the open-research-data paradigm (see the FAIR guidelines, Wilkinson et al., 2016 as well as an adapted version for cultural heritage objects, Koster and Woutersen-Windhouwer, 2018). Although this trend poses challenges to scholars seeking to compile historical corpora (e.g., due to time-intensive digitalisation for it to be machine-readable and a careful consideration of copyrights), such efforts are necessary to make diachronic data-sets available to various communities of researchers.
Discussion and outlook
Despite its scarcity, previous diachronic multimodality research has shown that our understanding of communicative practices today and our predictions of the future benefit from a close consideration of the past. Advances in theory-building suggest that developments in multimodal artefacts and interactions are driven by a broad range of interrelated factors and play out at various levels of the discourse. Solid theoretical frameworks are now in place (e.g., Bateman, 2008; Bateman et al., 2017; Wildfeuer et al., 2020) to systematically trace phenomena of stability and change across extensive periods of time. Future research will likely intensify the use of (semi-) automated and statistical methods to process large-scale multimodal corpora, compute correlations between changing patterns, and output significant factors of influence. The rise of digital archives and web-scraping methods opens up whole new worlds of multimodal data yet to be explored. All of these aspects make diachronic multimodality research a promising and timely endeavour: as with multimodality research in general, “there are more open questions than there are answers” (Bateman et al., 2017, p. 123) and “remember, there is always the ‘next’ paper :-)” (Bateman, 2020, pers. correspondence).
Author contributions
JP: Writing – original draft, Writing – review & editing.
Funding
The author declares financial support was received for the research, authorship, and/or publication of this article. This work was supported by the University of Salzburg Publication Fund.
Acknowledgments
I would like to thank Martin Luginbühl for his insightful comments on an earlier version of this manuscript. I am also grateful to Claudia Lehmann, Janina Wildfeuer und Tamara Drummond for their efforts in editing this special issue.
This paper is dedicated to John A. Bateman, who has been a major driving force in the development of our research field and an invaluable guide in my own exploration of multimodality.
Conflict of interest
The author declares that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Publisher’s note
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
References
Altheide, D. L., and Snow, R. (1992). Media logic and culture: a reply to Oakes. Int. J. Polit. Cult. Soc. 5, 465–472. doi: 10.1007/BF01423902
Bateman, J.A. (2008). Genre and multimodality. A Foundation for the Systematic Analysis of multimodal documents. Palgrave Macmillan: London
Bateman, J.A. (2014a). Genre in the age of multimodality: some conceptual refinements for practical analysis, Evolution in genre: Emergence, variation, multimodality, (Eds.) P. Evangelisti Allori, J.A. Bateman, and V.K. Bhatia, Bern: Peter Lang, 237–269
Bateman, J.A. (2014b). Text and image: a critical introduction to the visual/verbal divide. London/New York: Routledge
Bateman, J. A., Veloso, F. O., and Lau, Y. L. (2019). On track of visual style: a diachronic study of page composition in comics and its functional motivation. Visual Communication 20, 209–247. doi: 10.1177/1470357219839101
Bateman, J.A. (2021). Dimensions of materiality: towards an external language of description for empirical multimodality research, Empirical multimodality research, (Eds.) J. Pflaeging, J. Wildfeuer, and J.A. Bateman, Berlin/Boston: de Gruyter, 35–63
Bateman, J.A., Delin, J.L., and Henschel, R. (2004). Multimodality and empiricism: preparing for a corpus-based approach to the study of multimodal meaning-making, Perspectives on multimodality, (Eds.) E. Ventola, C. Charles, and M. Kaltenbacher, Amsterdam: Benjamins, 65–89
Bateman, J.A., Delin, J.L., and Henschel, R. (2007). Mapping the multimodal genres of traditional and electronic newspapers, New directions in the analysis of multimodal discourse, (Eds.) T.D. Royce and W.L. Bowcher, Mahwah, NJ: Lawrence Erlbaum, 147–172
Bateman, J.A., Evangelisti Allori, P., and Bhatia, V.K. (2014). Evolution in genre: emergence, variation, multimodality, Evolution in genre, (Eds.) P. Evangelisti Allori, J.A. Bateman, and V.K. Bhatia, Bern: Peter Lang, 9–16
Bateman, J. A., and Hiippala, T. (2020). Statistics for Multimodality: why, when, how - an invitation. doi: 10.31235/osf.io/7j3np
Bateman, J.A., Wildfeuer, J., and Hiippala, T. (2017). Multimodality: foundations, research and analysis – a problem-oriented introduction. Berlin/Boston: de Gruyter
Bednarek, M., and Caple, H. (2014). Why to news values matter? Towards a new methodological framework for analysing news discourse in critical discourse analysis and beyond. Discourse Soc. 25, 135–158. doi: 10.1177/0957926513516041
Bell, A. (1984). Language style as audience design. Lang. Soc. 13, 145–204. doi: 10.1017/S004740450001037X
Bell, A. (1991a). Audience accommodation in the mass media, Contexts of accommodation: developments in applied linguistics, (Eds.) H. Giles, J. Coupland, and N. Coupland, Cambridge: Cambridge University Press, 69–102
Bezemer, J., and Kress, G. (2009). Visualizing English: a social semiotic history of a school subject. Vis. Commun. 8, 247–262. doi: 10.1177/1470357209106467
Bezemer, J., and Kress, G. (2016). The textbook in a changing multimodal landscape, Handbuch Sprache im Multimodalen Kontext, (Eds.) N.-M. Klug and H. Stöckl, Berlin/Boston: de Gruyter, 476–498
Brock, A. (2015). Comedy Panel Show, Dramedy und Improv-Comedy: Zur kulturellen Ausdifferenzierung komischer Fernsehgenres in Großbritannien, Das Komische in der Kultur, (Eds.) S. Neuhaus, H. Diekmannshenke and U. Schaffers, Marburg: Tectum, 193–208
Brock, A. (2019). The emergence of contemporary British TV sitcoms, Genre emergence: developments in print, TV and digital media, (Eds.) A. Brock, J. Pflaeging, and P. Schildhauer, Frankfurt am Main: Peter Lang, 107–127
Brock, A., Pflaeging, J., and Schildhauer, P., (Eds.) (2019). Genre emergence: developments in print, TV and digital media. Frankfurt am Main: Peter Lang
Bucher, H.J. (1996). Textdesign-Zaubermittel der Verständlichkeit? Die Tageszeitung auf dem Weg zum interaktiven medium, Textstrukturen im Medienwandel, (Eds.) E.W.B. Hess-Lüttich, W. Holly, and U. Püschel, Frankfurt am Main: Peter Lang, 31–60
Bucher, H.J. (1998). Vom Textdesign zum Hypertext: Gedruckte und elektronische Zeitungen als nicht-lineare Medien, Medien im Wandel, (Ed.) W. Holly , Opladen/Wiesbaden: Westdeutscher Verlag GmbH, 63–102
Cohn, N., Taylor, R., and Pederson, K. (2017). A picture is worth more words over time: multimodality and narrative structure across eight decades of American superhero comics. Multimodal Communication. 6, 19–37. doi: 10.1515/mc-2017-0003
Deng, Y., and Feng, D. (2022). From researchers to academic entrepreneurs: a diachronic analysis of the visual representation of academics in university annual reports. Vis. Commun. 80:147035722211021. doi: 10.1177/14703572221102180
Durrani, S. (2020). Previewing news stories: how contextual cohesion contributes to the creation of news stories, Shifts toward image-centricity in contemporary multimodal practices, (Eds.) H. Stöckl, H. Caple, and J. Pflaeging, New York/London: Routledge, 123–145
Dynel, M. (2022). The life of COVID-19 mask memes: a diachronic study of the pandemic memescape. Communicar. 30, 73–85. doi: 10.3916/C72-2022-06
Eckkrammer, E.M. (2011). Diachrone Medienanalyse: Zur Analyse multimodaler Vertextungsstrategien in historischer Sicht, Medientheorien und Multimodalität: Ein TV-Werbespot – Sieben methodische Beschreibungsansätze, (Eds.) J.G. Schneider and H. Stöckl, Köln: Herbert von Halem, 190–215
Evangelisti Allori, P., Bateman, J.A., and Bhatia, V.K., (Eds.) (2014). Evolution in genre: emergence, variation, multimodality. Bern: Peter Lang
Graakjær, N. J. (2019). Sounding out i’m lovin’ it: a multimodal discourse analysis of the sonic logo in commercials for McDonald’s 2003–2018. Crit. Discourse Stud. 16, 569–582. doi: 10.1080/17405904.2019.1624184
Hauser, S., Kleinberger, U., and Roth, K.S. (Eds.) (2014). Musterwandel-Sortenwandel: aktuelle tendenzen der diachronen Text(sorten)linguistik. Bern: Peter Lang
Hess-Lüttich, E.W.B., Holly, W., and Püschel, U. (Eds.) (1996). Textstrukturen im Medienwandel. Frankfurt am Main: Peter Lang
Hiippala, T. (2015). The structure of multimodal documents: an empirical approach. London/New York: Routledge
Hiippala, T. (2016). Semi-automated annotation of page-based documents within the genre and multimodality framework, Proceedings of the 10th SIGHUM workshop on language Technology for Cultural Heritage, social sciences, and humanities, Berlin: Association for Computational Linguistics, 84–89
Hiippala, T. (2017). An overview of research within the genre and multimodality framework. Discourse, Context & Media. 20, 276–284. doi: 10.1016/j.dcm.2017.05.004
Hiippala, T. (2023). Corpus-based insights into multimodality and genre in primary school science diagrams. Vis. Commun. 29:147035722311618. doi: 10.1177/14703572231161829
Hiippala, T., and Tseng, C.-I. (2017). Editorial: media evolution and genre expectations. Discourse, Context & Media. 20, 157–159. doi: 10.1016/j.dcm.2017.11.001
Kammer, A. (2013). The mediatization of journalism. MediaKultur. 29, 141–158. doi: 10.7146/mediekultur.v29i54.17385
Keles, U., and Yazan, B. (2021). Gender in new headway: a diachronic, multimodal critical discourse analysis. Lang. Teach. 54, 132–138. doi: 10.1017/S0261444820000579
Knox, J. (2007). Visual-verbal communication on online newspaper home pages. Vis. Commun. 6, 19–53. doi: 10.1177/1470357207071464
Koster, L., and Woutersen-Windhouwer, S. (2018). FAIR principles for library, archive and museum collections: a proposal for standards for reusable collections. code{4}lib Journal. 40
Kress, G., and van Leeuwen, T. (1996). Reading images: the grammar of visual design. London/New York: Routledge
Krotz, F. (2015). Mediatisierung und die wachsende Bedeutung visueller Kultur: Zum Verhältnis zweier Kommunikationswissenschaftlicher Metaprozesse, Visualisierung-Mediatisierung: Bildliche Kommunikation und bildliches Handeln in mediatisierten Gesellschaften, (Eds.) K. Lobinger and S. Geise, Köln: Halem Verlag, 18–36
Lemke, J.L. (1999). Typology, topology, topography: Genre semantics, Available at: http://academic.brooklyn.cuny.edu/education/jlemke/papers/Genre-topology-revised.htm, (Accessed Dec 18, 2023).
Lomborg, S. (2014). Social media, social genres: making sense of the ordinary. London/ New York: Routledge
Luginbühl, M. (2014). Medienkultur und Medienlinguistik: Komparative Textsortengeschichte(n) der amerikanischen ‘CBS Evening News’ und der Schweizer ‘Tagesschau’. Bern: Peter Lang
Luginbühl, M. (2015a). Genre profiles as intermediate analytical level for cultural genre analysis, Genre studies around the globe, (Eds.) N. Artemeva and A. Freedman, Bloomington: Inkshed Publications, 251–273
Luginbühl, M. (2019). Genre emergence and change as indicator and origin of cultural change, Genre emergence: developments in print, TV and digital media, (Eds.) A. Brock, J. Pflaeging, and P. Schildhauer, Frankfurt am Main: Peter Lang, 129–159
Meier, S., and Marx, K. (2019). Doing genre in the digital media, Genre emergence: developments in print, TV and digital media, (Eds.) A. Brock, J. Pflaeging, and P. Schildhauer, Frankfurt am Main: Peter Lang, 191–212
Molnar, S. (2019). The birth of the print ad genre, Genre emergence: developments in print, TV and digital media, (Eds.) A. Brock, J. Pflaeging, and P. Schildhauer, Frankfurt am Main: Peter Lang, 29–50
Miller, C. R. (1984). Genre as social action. Q. J. Speech. 70, 151–167. doi: 10.1080/00335638409383686
Müller-Lancé, J. (2016). Trendsportmagazine in Deutschland und Frankreich: Eine medienlinguistische Analyse. Landau: Verlag Empirische Pädagogik
Müller-Lancé, J. (2019). The development of genres in German and French action sports magazines: how economic interests affect text types, Genre emergence: developments in print, TV and digital media, (Eds.) A. Brock, J. Pflaeging, and P. Schildhauer, Frankfurt am Main: Peter Lang, 51–72
Pflaeging, J. (2017a). Tracing the narrativity of National Geographic feature articles in the light of evolving media landscapes. Discourse, Context & Media. 20, 248–261. doi: 10.1016/j.dcm.2017.07.003
Pflaeging, J. (2017b). Changing potentials and their use: the case of popular science journalism, Communication forms and communicative practices: new perspectives on communication forms, affordances and what users make of them, (Eds.) A. Brock and P. Schildhauer, Frankfurt am Main: Peter Lang, 181–208
Pflaeging, J. (2019). Beyond genre names: diachronic perspectives on genre indexation in print magazines, Genre emergence: developments in print, TV and digital media, (Eds.) A. Brock, J. Pflaeging, and P. Schildhauer, Frankfurt am Main: Peter Lang, 73–104
Pflaeging, J. (2020a). On the emergence of image-centric popular science stories in National Geographic, Shifts toward image-centricity in contemporary multimodal practices, eds. H. Stöckl, H. Caple, and J. Pflaeging, London/New York: Routledge, 97–122
Pflaeging, J. (2020b). Diachronic perspectives on viral online genres: from images to words, from lists to stories, Visualizing digital discourse: interactional, institutional and ideological perspectives, (Eds.) C. Thurlow, C. Dürscheid, and F. Diémoz, Berlin/Boston: de Gruyter, 227–244
Pflaeging, J., Bateman, J.A., and Wildfeuer, J. (2021). Empirical multimodality research: the state of play, Empirical multimodality research, (Eds.) J. Pflaeging, J. Wildfeuer, and J. A. Bateman, Berlin/Boston: de Gruyter, 3–32
Schneider, J.G., and Stöckl, H., eds. (2011). Medientheorien und Multimodalität: Ein TV-Werbespot–Sieben methodische Beschreibungsansätze. Köln: Herbert von Halem
Semedo, D., and Magalhães, J. (2019). Diachronic cross-modal embeddings, Proceedings of the 27th ACM international conference on multimedia, 2061–2069
Sommer, J. M. (2019). Emergence of online comments in popular science discourse, Genre emergence: developments in print, TV and digital media, (Eds.) A. Brock, J. Pflaeging, and P. Schildhauer, Frankfurt am Main: Peter Lang, 235–259
Stamenković, D. (2022). The stylistic journey of a video game: a diachronic approach to multimodality in the football manager series, Stylistic approaches to pop culture, (Eds.) C. Schubert and V. Werner, London/New York: Routeldge, 227–246
Stöckl, H. (2009). The language-image-text: theoretical and analytical inroads into semiotic complexity. AAA – Arbeiten aus Anglistik und Amerikanistik. 34, 3–28
Stöckl, H. (2010). Textsortenentwicklung und Textverstehen als Metamorphosen: Am Beispiel der Werbung, Mediale transkodierungen: metamorphosen zwischen Sprache, Bild und Ton, (Ed.) H. Stöckl , Heidelberg: Universitätsverlag Winter, 145–172
Stöckl, H. (2014). ‘He begs to inform every person interested’: a diachronic study of address and interaction in print advertising. Anglistik Inte. J. Engl. Stud. 25, 81–106
Stöckl, H. (2017). Multimodality in a diachronic light: tracking changes in text-image relations within the genre profile of the MIT technology review. Discourse, Context & Media. 20, 262–275. doi: 10.1016/j.dcm.2017.07.001
Stöckl, H., Caple, H., and Pflaeging, J., (Eds.) (2019). Shifts toward image-centricity in contemporary multimodal practices. London/New York: Routledge
Thomas, M. (2009). Localizing pack messages: a framework for corpus-based cross-cultural multimodal analysis. PhD Thesis. Leeds: University of Leeds
Thomas, M. (2019). Making a virtue of material values: tactical and strategic benefits for scaling multimodal analysis, Multimodality: towards a new discipline, (Eds.) J. Wildfeuer, et al., Berlin/Boston: de Gruyter, 69–91
Waller, R. (2012). Graphic literacies for a digital age: the survival of layout. Inf. Soc. 28, 236–252. doi: 10.1080/01972243.2012.689609
Wildfeuer, J., Bateman, J.A., and Hiippala, T. (2020). Multimodalität: Grundlagen, Forschung und Analyse – Eine problemorientierte Einführung. Berlin/Boston: de Gruyter
Wilkinson, M. D., Dumontier, M., Aalbersberg, I. J. J., Appleton, G., Axton, M., Baak, A., et al. (2016). The FAIR guiding principles for scientific data management and stewardship. Scientific Data. 3, 1–9. doi: 10.1038/sdata.2016.18
Yates, J., and Orlikowski, W. J. (1992). Genres of organizational communication: a structurational approach to studying communication and media. Acad. Manag. Rev. 17, 299–326. doi: 10.2307/258774
Keywords: diachronic, multimodality, development, stability, change, genre, review
Citation: Pflaeging J (2024) Diachronic multimodality research – a mini-review. Front. Commun. 9:1358192. doi: 10.3389/fcomm.2024.1358192
Edited by:
Claudia Lehmann, University of Potsdam, GermanyReviewed by:
Martin Luginbühl, University of Basel, SwitzerlandCopyright © 2024 Pflaeging. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Jana Pflaeging, amFuYS5wZmxhZWdpbmdAcGx1cy5hYy5hdA==