Digital literacy involves dealing with multimodal texts in many different communicative situations while using a range of communicative resources such as images, colors, videos, sounds, and graphics. It also involves respectfully communicating with people from different cultures in addition to those with different lifestyles.

Writing and reading digital texts

As writers, we carefully select certain words and organize these ideas into logical paragraphs. Beyond these textual writing practices, however, we also need to plan the text’s graphic design. As part of planning, multimodal choices are unlimited: What font will we use? What colors should be in the background? What images might we use, and where should we place them in relation to the text? Will we add movies, animations, or hyperlinks? If so, where should we insert them, and how should they appear for the reader? Is there any benefit to organizing the information with a navigational menu?

Consequently, reading and understanding ideas designed by a multimodal writer requires attention to each of these elements in addition to how to effectively connect them to better understand why they are assembled together and what meanings they convey as a whole.

Multimodal texts: every time, everywhere

Multimodality is a basic feature of texts, according to Gunther Kress, an Australian researcher who highlights elements of multimodality in his work. However, as teachers, we may sometimes treat multimodal text features as superfluous; neglecting to call students’ attention to the important meaning(s) they convey. In Reading Images: The Grammar of Visual Design, Gunther Kress and Theo van Leeuwen remind educators of the need to consciously help students develop the ability to make sense of all of the multimodal languages blended or “orchestrated” into a text.

These efforts can more actively foster students’ critical literacy. As Jay Lemke wrote in his 2006 handbook chapter titled, Towards Critical Multimedia Literacy: Technology, Research, and Politics:

Critical literacy needs to respond to these historic changes [in digital texts]. We need a broader definition of literacy itself, one that includes all literate practices, regardless of medium. Books-on-tape are as much literate works as are printed books. Scripted films and television programs are no less products of literate culture in their performances than they were as texts. In printed advertisements, the message toward which we need to take a critical stance is conveyed not just by the textual copy, nor even by the copy and the images, but by the interaction of each with the other, so that the meaning of the words is different with the images than without them, and that of the image together with the words distinct from what it might have been alone. In the multimodal medium of the Web, the message is less the medium than it is the multiplication of meanings across media.

Hands-on multimodality

Focusing on multimodality does not require radically different ways of teaching nor does it require a computer. Even simple activities can open students’ minds to the power of images. I (Ana Elisa) have found that asking students to compose a set of directions for how to play Tic-tac-toe can produce some interesting results. Many students try to give instructions using only words, and they find it difficult to effectively communicate all of the steps. Some students ask for permission to include drawings in their directions and find they are much more successful. Taking time as a class to compare various forms of instructions creates the opportunity for rich conversations around the power of images as part of a meaning-making experience. I report on an analysis of these experiences in my book Textos Multimodais: Leitura e Produção or, in English, Multimodal Texts: Reading & Writing.

These kinds of experiences are only the tip of the iceberg! Continued experiences with multimodal text features open students’ minds to the role that multimodality plays in reading and writing.

To learn more, view a video in which Gunther Kress talks about multimodality. You might also enjoy this video in which Mary Kalantzis and Bill Cope explore the multiple modes of representing meaning in today’s media.

Carla Viana Coscarelli is a professor in the School of Language Arts at Universidade Federal de Minas Gerais (UFMG) in Brazil and coordinator of Projeto Redigir/UFMG. Ana Elisa Ribeiro is a writer and a professor in the Department of Language and Technology of Federal Center of Technological Education of Minas Gerais.