“Students engaged in learning that incorporates multimodal designs, on average, outperform students who learn using traditional approaches with single modes.” (source: Multimodal Learning Through Media: What the Research Says)

Using multiple modes to represent knowledge provides opportunities to enhance learning in new ways as technology continues to evolve. Modes include what we hear, see (i.e. read or view), taste, smell, and feel. Understanding how to best utilize various “modes” is important because we process or store information a bit differently depending on the “mode” used. For example, we process visual information differently than auditory information. Information taken in through those modes is processed and stored in different parts of the brain. When combined effectively, the likelihood of lasting learning (storing in long term memory for later retrieval) will increase. This means that if one takes in information in more than one way (for example, read it and hear it at the same time versus just read it), because the information is stored in different parts of the brain, the brain creates new memories in such a way that retrieval of the entire memory is more likely. This is especially true if the learner is provided with opportunities to think about and make sense of what they’ve learned in relation to their previous knowledge and personal world (metacognition).

The following diagram shows a visual representation of the relationship between physiologic and cognitive functions as related to types of memory.

Multi-Modal Learning - How Learning Occurs

Research shows that effective combinations of various modes takes planning and skill in order that one doesn’t overload the working memory. Inclusion of multiple modes of content delivery must be strategic and should consider the audience and what the learning goals are. For example, as described in the report Multimodal Learning Through Media: What the Research Says, the following principles for use of multimedia should be considered:

1. Multimedia Principle: Retention is improved through words and pictures rather than through words alone.
2. Spatial Contiguity Principle: Students learn better when corresponding words and pictures are presented near each other rather than far from each other on the page or screen.
3. Temporal Contiguity Principle: Students learn better when corresponding words and pictures are presented simultaneously rather than successively.
4. Coherence Principle: Students learn better when extraneous words, pictures, and sounds are excluded rather than included.
5. Modality Principle: Students learn better from animation and narration than from animation and on-screen text.
6. Redundancy Principle: Students learn better when information is not represented in more than one modality – redundancy interferes with learning.
7. Individual Differences: Principle #1: Design effects are higher for low-knowledge learners than for high-knowledge learners; Principle #2: Design effects are higher for high-spatial learners rather than for low-spatial learners.
8. Direct Manipulation Principle: As the complexity of the materials increase, the impact of direct manipulation of the learning materials (animation, pacing) on transfer also increases.

This graphic shows that students learning basic skills perform better when the multi-modal learning does not include interactive elements. It also shows that students working on higher order thinking skills who are provided with opportunities for interactivity multimodal learning perform better than those without the interactivity. 

Multi-Modal Learning: Learning Comparison Based on Interactivity Combined with Multi-modal delivery

Inclusion of multi-modal learning is also supported by the Universal Design for Learning (UDL) principles. This video, in addition to be an example of using multi-modal knowledge representation to convey information, describes that relationship and also describes how inclusion of too many modes can have a negative effect on learning. It also provides a brief explanation of the preceding image about what types of learning are supported by interactivity and multi-modal representations of information.

Additional Information about Universal Design for Learning (UDL) can be found at the National Center for Universal Deisgn for Learning

Source Quoted: Multimodal Learning Through Media: What the Research Says. Retrieved 9/1/16 from: http://www.cisco.com/c/dam/en_us/solutions/industries/docs/education/Multimodal-Learning-Through-Media.pdf