The second, thoroughly revised and expanded, edition of The SAGE Handbook of Visual Research Methods presents a wide-ranging exploration and overview of the field today. As in its first edition, the Handbook does not aim to present a consistent view or voice, but rather to exemplify diversity and contradictions in perspectives and techniques. The selection of chapters from the first edition have been fully updated to reflect current developments. New chapters to the second edition cover key topics including picture-sorting techniques, creative methods using artefacts, visual framing analysis, therapeutic uses of images, and various emerging digital technologies and online practices. At the core of all contributions are theoretical and methodological debates about the meanings and study of the visual, presented in vibrant accounts of research design, analytical techniques, fieldwork encounters and data presentation. This handbook presents a unique survey of the discipline that will be essential reading for scholars and students across the social and behavioural sciences, arts and humanities, and far beyond these disciplinary boundaries. The Handbook is organized into seven main sections: PART 1: FRAMING THE FIELD OF VISUAL RESEARCH; PART 2: VISUAL AND SPATIAL DATA PRODUCTION METHODS AND TECHNOLOGIES; PART 3: PARTICIPATORY AND SUBJECT-CENTERED APPROACHES; PART 4: ANALYTICAL FRAMEWORKS AND PERSPECTIVES; PART 5: MULTIMODAL AND MULTISENSORIAL RESEARCH; PART 6: RESEARCHING ONLINE PRACTICES; and PART 7: COMMUNICATING THE VISUAL: FORMATS AND CONCERNS.

Chapter 29: Multimodality and Multimodal Research

Multimodality and Multimodal Research

Multimodality and multimodal research
Theo van Leeuwen

Introduction

The term ‘multimodality’ dates from the 1920s. It was a technical term in the then relatively new field of the psychology of perception, denoting the effect that different sensory perceptions may have on each other. An example of this is the so-called McGurk effect: if people are shown a video of someone articulating a particular syllable, for example, /ga/, while hearing another syllable, for example, /ba/, they perceive neither /ga/, nor /ba/, but /da/ (Stork, 1997: 239). In other words, perception is multimodal. It integrates information received by different senses.

More recently the term has broadened to denote the integrated use of different communicative resources such as language, image, ...

  • Loading...
locked icon

Sign in to access this content

Get a 30 day FREE TRIAL

  • Watch videos from a variety of sources bringing classroom topics to life
  • Read modern, diverse business cases
  • Explore hundreds of books and reference titles