Visual Stimuli And Divergent Thinking In The Mixing Process

Introduction

Mixing is an expertly intuitive practice – that activates neural responses that transmit multimodal perceptions, especially audio-visual perception – whereby a mixing engineer is tasked to solve technical disparities surrounding an input signals frequency response, amplitude and spatial image, as well as make creative decisions that further aid in conceptualizing and realizing the artists, producers and/or their musical vision. Even highly accomplished mixing engineers deliver notably varied mixes of the same song as a result of their unique mixing process — certain engineers may use analogue and/or digital signal processors differently to others or their intended purposes — for instance, a mixing engineer may use a compressor to adjust the relative volume of an input signal as a replacement to a fader, whilst another may use it entirely differently in a real mixing environment.

Very little has been researched on the art of mixing – there have been few detailed studies addressing this phenomenon by academics like Brecht De Man et al – due to the increasing volume of audio engineering textbooks reporting elementary mixing techniques based on various instruments, genres and effects (2017). According De Man (2017), some of these mandates are not only incoherent – as they state that fashioning a mix is an exceedingly non-linear process with no set dictation on equalizer and/or compression settings, however, it is found that spectral and dynamic processing is heavily dependent on the characteristics of the input signal – but very few have been substantiated. In spite of that, academic institutions still opt to instill theoretical-based learning for their students, thus resulting in a lack of divergent thinking amongst them.

Therefore, the purpose of this research document is to further understand the multidimensional and non-linear problems faced with tailoring a mix, by investigating if audio-visual interactions of neural responses, combined with divergent thinking, have a profound effect on a mixing engineers’ aural perception and creativity.

Literature Review

Every individual shares the unique ability of creativity, however the amount of creativity is dependent on the individuals’ social constructs and potential. An individuals’ creativity is often determined by various cognitive skills and personality traits such as flexibility, eloquence, visualization, imagination, articulacy, willingness to experience and prominent schizotypal tendencies. Exceptionally creative individuals display a superior aptitude to realize products, as well as conceptualize ideas in unique, novel ways, due to their inherent ability to draw from past experiences.

Divergent thinking involves constructing unique connotations and is a fundamental cognitive component of creativity extensively studied since the 1950s. Unlike convergent thinking – which relies on the elimination of probable answers to derive at a correct answer, thus rendering it most effective for answering clearly defined problems – divergent thinking relies on flexible conceptions of multiple answers to open-ended and multidimensional problems, making it better suited for answering unconstructed and poorly defined problems – thus, affording it a platform to realize products via an unparalleled creative process. Psychologists involved in the growth of artistic development have fixated on the cognitive components affiliated with creative conceptualization, however, the main problem they face revolves around the word “creativity” and its definition. According to Webster (1990), the word “creativity,” has been exploited extensively in multiple contexts – particularly in music – thus losing its power and significance. Divergent thinking therefore aims to emphasize the process of learning, whereby individuals can generate problems and answers via a more creative process.

This kind of intuitive problem solving and thinking plays a profound role in an individuals’ creative process, yet ironically, it is the same kind of thinking that is not imparted upon academic students as factual-based education is still favored in academic institutions. According to a near-infrared spectroscopy (NIRS) study on divergent thinking, participants displayed increased bilateral frontal lobe activity when tasked to demonstrate innovative uses for everyday items, whilst participants with prominent schizotypal behavioral traits displayed increased creativity and right frontal lobe activity, stemming an empirical connection between creativity and psychosis-proneness.

Furthermore, using voxel-based morphometry, significant volume differences of gray matter had been detected amongst professional musicians, amateur musicians and non-musicians’ in the motor cortex, auditory cortex and parietal cortex. More specifically, musical prominence correlated significantly with the increased volumes of the left precentral gyrus, Heschl’s gyrus and right superior parietal cortex, thus implying that professional musicians exhibit abnormally significant cerebral superiority as compared to non-musicians, due to their innate ability to perform more cognitive operations. The precentral gyrus, or motor strip, is directly anterior to the central sulcus and is located at the posterior of the frontal lobe. The frontal lobe is the foremost lobe, and the most anterior part of the frontal lobe deals with significant cognitive functions – or executive functions – such as reasoning and judgment, as well as contains multiple cortical areas involved in the coding of voluntary actions of skeletal muscle – particularly those required for speech and swallowing. However, the motor cortex – containing the motor strip and the premotor, or supplementary motor area – is not responsible for coding motor commands for speech as these are generated in Broca’s area, which is located in the inferior third frontal gyrus, in the Frontal Lobe. The frontal lobe is separated from the temporal lobe by the lateral sulcus, or lateral fissure, placing the temporal lobe inferior to the lateral sulcus and anterior to the occipital lobe. The temporal lobe is responsible for aural perception and olfaction.

Located in the temporal lobe, Heschl’s gyrus, or the transverse temporal gyrus, is where auditory signals arrive at the brain first, making it the primary auditory area. The secondary auditory area is part of Wernicke’s area and is responsible for the comprehension of speech and its intelligibility. Wernicke’s area is located at the posterior of the transverse temporal gyrus and is separated from the occipital lobe by an illusionary line. The occipital lobe is the prior most lobe responsible for vision. The parietal lobe is directly anterior to the occipital lobe and separates it from the temporal lobe by the posterior portion on the lateral sulcus. The parietal lobe is responsible for sensation – particularly touch, localization, perception of temperature and vibration – as well as writing and reading. The primary sensory area, or sensory strip, gathers sensory inputs – similar to the motor strip – from the visual cortex, auditory cortex and sensory cortex to determine localization in a given setting. The secondary sensory area – inferior to the primary sensory area – is capable of discriminating and analyzing sensory modalities. Fibres of various compounds and axons that subserve multimodal perceptions travel through the angular gyrus located above the superior temporal lobe edge. The fibres connect the angular gyrus to Broca’s area and Wernicke’s area. According to Witten and Knudsen (2005), vision dominates sensory perception due to the statistically optimal fashion in which the brain amalgamates and processes visual stimuli over aural stimuli. When an aural stimulus clashes with a visual stimulus, the aural stimulus intermittently draws itself onto the visual stimulus, in a phenomenon referred to as “visual capture”. Scientists believe visual capture signifies an intrinsic advantage – constructed in the architecture of the brain – that favors visual perception over aural perception, as one sensory input is often subjugated by another. Theoretically, it is not entirely clear why vision would dominate sensory perception, however, one hypothesis suggests that through evolution and education, visual perception became inherently advantageous over aural perception. Intuitively, vision may dominate due to the fact that optical input signals project undistorted, high resolution images – incapable of being corrupted by an exterior setting – onto the retina topographically. As a result, real life subjects gives rise to the fibres and axons transported through the angular gyrus – to the visual cortex – for greater visual perception. Dissimilarly, aural stimuli are less precise as soundwaves – depending on the setting – distort as a result of being echoed off secondary objects in the environment.

In circuits – like the angular gyrus – that subserve multimodal perceptions, synapses transport visual stimuli impulses more effectively than aural stimuli impulses. Moreover, in circuits that subserve unimodal perceptions, visual stimuli can access and influence aural stimuli, but not vice versa. In spite of that, visual stimuli precision often relies on factors like contrast and lighting. Therefore, in low light settings, aural perception may supersede visual perception. In contrast to Witten and Knudsen’s findings, a perceptual study conducted by Hidaka and Ide (2015) found that aural stimuli induced auditory suppressive effects on visual perception, when minute bursts of white noise – 200ms with 5ms fade ins and outs at 45, 55 and 65 dBA – were transmitted through headphones in a “spatially and temporally” corresponding manner onto a subject seated away from the visual display. However, auditory suppression effects were not found when the aural stimuli were projected through speakers with the subject seated 57.3cms or closer to the display. The auditory suppression effects may have been a result of a multimodal attentional capture effect, whereby attention from the visual stimuli was severed due to the sudden projection of the aural stimuli close to the head. Furthermore, the effects were found most prominent when both visual and aural stimuli were presented simultaneously or consequently at an ipsilateral position as opposed to a contralateral one. Intriguingly, increased auditory suppression effects were found when the laterality of the aural stimuli were projected in mono as opposed to in stereo. The results detailed a 10% auditory suppression effect with the 65 dBA, mono aural stimuli and a 5% auditory suppression effect with the 45 dBA, stereo aural stimuli. At a mixing workshop in 2013, Grammy award mixing engineer Frank Fillipeti displayed two identical brain scans of a respondent exposed to aural stimuli with and without visual stimuli. The scans revealed increased brain activity in the amygdala – located in the temporal lobe and responsible for processing memory and emotional responses like fear angst and anger – when exposed to aural stimuli and no visual stimuli, thus supporting Hidaka and Ide’s study. Consequently, engineers are failing to mix and listen more critically due to the cognitive impairment a visual display causes. Unlike DAWs, analogue consoles and control surfaces have no visual aid to rely on – particularly with EQs and compressors.

Therefore, Fillipeti advices mixing engineers to mix on consoles or control surfaces that contain physical faders and EQ and compressor knobs with no visual indicators, in order to make creative decisions without the aid of a visual display. The mixing process on a modern DAW has become increasingly visual due to the perceived visual stimuli presented – particularly DSP plugins, meters and soundwaves. For instance, graphical EQs explicitly present frequency cuts and boosts, compressors reveal apparent gain reduction, meters display perceived peaking and soundwaves create anticipation of what is forthcoming, thus affecting the mixing engineer’s aural perception of the overall mix balance. According to Cochrane, the key to a more creative mixing process lies in the ability to critically listen without any visual stimuli – enabling the mixing engineer to think more creatively and audition the song like the target audience would.