Editorial: The contribution of audio-visual research methods to the social sciences
By: Annick Janson, University of Auckland Business School, andFabrice Desmarais, Waikato Management School
If a picture is worth a thousand words why do we know so little about the impact visuals have on us? One of the obstacles to such a quest is the lack of robust, established research methodologies that use audio-visual material as raw data.
Research in audio-visual communication is one of the youngest within the social sciences. There are no established criteria to conduct this type of research, however, the fields of psychology, education, art history, ethnography, cultural and multimedia studies can provide some guiding principles toward building empirical knowledge. We found it useful to categorise the three stages of empirical enquiry in which audio-visual elements can make valuable contributions: data acquisition, data analysis and data reporting.
When it comes to data collection,our early memories asstudents in the social sciences includeaudio taping conversations and the labour of transcribing the dialogues into text to perform one type of content analysis or another on the resulting data. We have watched throughout the years social science researchers cling to these past-century methodologies,flying in the face of what we now know with certainty, which is thatmost of communication is non verbal! Indeed analysing audio-visual communication is not only analysing ?what is said but also ?how it is said, visually and aurally.
In this instance we also want to point out a recurrent bias in audio-visual analysis: studies of audio-visual material usually relegate voice and sound to the background and tend to focus on visual elements. In particular the nonverbal cues of audio material are consistently ignored except perhaps in afew linguistic studies. We should not forget that, as much as content, it isintonation and voice quality that convey meaning. Voice allows us to encode and decode different types of feelings or emotions and, from an audio-visual analysts viewpoint, voice and speech cues are a rich source of interpersonal impressions. Certain types of voices can be indicative of personality characteristics such as competence, extroversion, maturity, dominance, sophistication, pride, weakness, warmth, or status. Most audio-visual research ignores this important dimension of communication in the search for a better understanding of media texts. The common semiotic analysis of media material, for example, neglects the encoding and decoding of vocal signs and is never used to investigate what voice quality can ?signify and what it adds to various types of communication. This issurprising, as voice, just like visuals, is animportant feature of the communicative process. Careful listening to the aural track of any media text without visuals shows that the system of signification through sound is very developed and difficult to decode in all its complexity.
There is a pressing need to explore this virtually uncharted territory of media sound and voice, and design or refine new tools to understand it better. Fields and methods need to cross-pollinate and depart, for instance, from communication studies avoidance of including vocal parameters because of the difficulties of using phonetics. Communication studies can indeed be enriched by simple perceptive auditory study of voice without having to use a complex phonetic study of the voice. Using perceptive auditory analysis of one or several vocal environments, identifying the main prosodic features of voices and attempting to illustrate how these are signifiers of attitude, emotion or gender orientation can add richness to audio-visual analysis. We encourage researchers from all fields to try and incorporate this forgotten dimension in their work.
Take the example of a research focusing on interactions of people in given circumstances. A researcher using conventional methods of taping in audio mode and manual field-note-taking cannot possibly capture the richness and nuances of the information in real time.
Audio-visual methodologiesoffer unique advantages for data analysis: the possibility of recording rich research material in a form that is as true as possible. This is crucial in emerging research, because we know that all questions cannot be asked immediately. While this is the case inall scientific enquiry, when audio-visual material has been collected, further analysis isonly constrainedby the richness of our original material. If we have recorded material in a media-rich form, then we can re-analyse it even many years after it was recorded.
One could not possibly preserve that much of the original data for future analyses if one was relying on hand written field notes and audio recordings ? because in a field situation, researchers would have had to make choices about what information to record. By definition these are largely constrained by their current theories of action and questions (i.e. of what constitutes the right information to remember about the event under study). The researcher may not have recorded the information to answer other questions and there is no way of knowing what gets ?lost in transcription. If, on the other hand, the event under study was recorded audio-visually, then emerging questions can be answered retroactively by re-analysing audio-visual data, since it is more complete than manual field notes and audio data alone. What is captured with audio-visual methods is not only what is said, but how it was said: voice inflection for spoken contents, speakers' body language and audience reaction to the spoken words ? including micro-interactions (glances, gestures, etc).
It is true that recording and analysing audio-visual data is more complex than just transcribing an audio tape into text, however the past decades have popularised digital image recording material allowing even the technologically challenged among us to record scenes using video equipment and, in the process, record data in a non-obtrusive manner.
Lets take some specific examples: with very few exceptions, leadership scholars have shown themselves to be blind to the visual dimensions of leadership. Non-verbal (such as body language) and voice (such as intonation) cues are largely under-researched. This omission is all the more worrying given the use of images of leaders in the mass media, and their obvious impact on followers. Would you vote for a candidate if their election campaign profiled them looking confused or frowning ? Would you rather vote for a candidate whose voice projected introversion or extroversion?
In our research we examine the ways in which individuals use audio-visual media, in particular film, to present and re-present their beliefs and reflections about leadership. This research follows the recent surge in interest in aesthetically-based research within organisational studies. Since there are no established methodologies to guide this type of research, the first steps inour research projects include piloting data collection and data analysis methodologies. Some questions we explore (see for example the report on the Better by Design conference) in this respect are: How are collective leadership identities defined, constructed and represented in visual communication channels? What methodologies can we develop to make sense of this construction process? How do perceptions of leadership evolve during a leadership development programme? Some preliminary findings point to radical differences in what messagesare communicated verbally versus non-verbally, which (individual or group) leadership attributesare communicated, and how or which of the latter communication channels is preferential for say, leadership as a person or a position, an interaction between leaders and followers or a leaders personal message.
We are also able to compare with more acuity than ever before what participants express before and after leadership development courses usingevaluation measures forthe impact of the development intervention.
Our third, and last, point here is about reporting on research and dissemination of knowledge using audio-visual media. Pioneering research using audio-visual research methods comes with a responsibility ? to fight for the right to use audio-visual materials in our reports. Audio-visual reporting using audio-visual formatisvirtually impossible today given that most journals are in hard copy forms! A colleague reported he was asked to pay to have colour pictures printed as part of thearticle he had submitted to a journal. This shows that conventional hard copy journals are becoming less appropriate to audio-visual research. Since they can barely accommodate two-dimensional visual material, they will certainly not be able to accommodate dynamic material as more studies develop audio-visual research methodologies. These methodologies necessitate new dissemination channels that can support and illustrate their findings ? raising questions about which reporting media optimise this type of research.
Our research also shows that audiences engage far better withfindings when these are reported using audio-visual channels. The latest video editing software considerably shortens the cycle between production and distribution of material. As the recent boom in distribution of video material via the internet shows (popular YouTube clips can reach audiences ofhundreds of thousands at a time) it is possible to produce near-instantaneous real time reports on what viewers want to watch. Yet we are lagging behind, still developing the research methods to analyse these worldwide trends. If the effects of reporting research findings using audio-visual channels do indeed impact on the engagement levels of our audiences, how do we research such phenomena and how can we most efficiently report our findings?
Another point to consider relates to stakeholder audiences. Reporting to the very stakeholder group that has contributed to the data collection process is not a widespread practice in the social sciences. Yet, we do need to ask some uncomfortable questions, such as: What is our responsibility as social sciences researchers to the subjects we research and to society at large? Is it enough to collect data from people at work, at play or in interview situations and report in professional journals that most interviewees are unlikely to access? Should we make the effort to report in ways that are accessible to the population we studied? Would it be relevant to use for reporting the medium in which the data was collected in the first place? The organisations we work with are showing growing interest in developing ?leadership archives to record vital knowledge for their staff and customers on what unique knowledge affords them their competitive advantage. Using state-of-the-art technology, we distil interview material into relevant bits of tacit knowledge and re-distribute on intranets or DVD format to appropriate audiences.
On the academic level, we are looking forward to being able to take full advantage of the fact that PRism is online and making a unique contribution to our topic of interest. Authors are encouraged to submit audio-visual materials to support, detail or exemplify their analyses. Granted, this is a big challenge ? and we are possibly the first publication to be able to do this ? because we are unrestricted by paper medium, unlike most other online journals, which are a virtual mirror to their hard copy.
Our original call for papers raised much interest in the research community, and we know that others will be interested both to contribute and to comment on the contributions when they see some examples of the kinds of materials that can be displayed and the questions that can be raised.To this effect we would like to set an ongoing call for papersabout the contribution of research using audio-visual material in communication, public relations, management, education, and related fields. We encourage not only full-length academic contributions but also commentary pieces (up to 2500 words) and shorter opinion pieces (up to 500 words), all of which may include, of course, visual and audio-visual components.
We would like to hear your reviews ofinnovative tools to collect audio-visual information, novel ways to analyse it using original reporting channels, including this journal. These both shape and are shaped by what we notice as valid research elements and how we understand what we see. We expect contributionsrelated (but not limited) to the following fields: public relations,communication,media, journalism,education, learning, personal development, teacher development, and leadership development.
- What new research approaches can be developed to account for the specificity of audio-visual elements as opposed to textual elements?
- How can audio-visual elements be truly integrated in learning processes?
- How do different researchers use audio-visual elements to present constructs in their specific fields of interest (for example learning and teaching, personal and professional development)?
- How are collective identities defined, constructed and represented in audio-visual media? What methodologies can we develop to make sense of this social construction process?
- How do audio-visual artefacts and methods help or hinder the representation of constructs such as cognitive interpretations, understanding and insights? How may they report on interaction or impact on behaviour?
- How may individual and collective audio-visual representations complement each other in, for instance, collective capacity building?
Return to Issue 4(3)
Return to PRism Home Page