Page 45 - Summer2022

Figure 2. Drawings of fantasy characters called bouba (left) and kiki (right). Courtesy of painter Mary Wünsch. bouba character and spikier shapes for the kiki character. Interestingly, even the sound of drawing a round versus a spiky shape on paper can be reliably characterized and assigned to the correct bouba and kiki figure, respectively (Margiotoudi and Pulvermüller, 2020). The effect is a robust cross-linguistic phenomenon (Ćwiek et al., 2022), which has also been shown in infants and children. There are different opinions as to why the sound-to-shape matching takes place. Some say the effect originates from articulation. In bouba, the /uː/ is a rounded sound involving lip protrusion and the /b/ is produced with the lips. In kiki, the /k/ causes the tongue to move up and down, releasing into the vowel. So, the articu- latory motions themselves seem rounder or smoother in bouba and more abrupt in kiki. Others have argued that the acoustic characteristics of the pseudowords match the visual shapes (Ćwiek et al., 2022). During /k/, the fundamental frequency is absent, leading to a period of silence followed by high-intensity, noisy spectral energy. The fundamental frequency and formants only begin at /i/. In contrast, bouba is produced with a continuous fundamental fre- quency and overall lower spectral energy, so that the changes are less extreme. A potential bias with orthography (a system of writing conventions) has also been discussed in the literature. Some researchers have wondered whether the sound- symbolic mapping is not a cross-modal correspondence between sound and shape but rather a shape-to-shape matching between the orthographic shape of letters to the depicted images. The Roman letters <b>, <o>, and <a> are obviously round, whereas <k> and <i> are spiky. However, people around the globe, using different alpha- bets, match shape and sound in a similar way relatively independent of their orthography (Ćwiek et al., 2022). For example, in Georgian, the orthographic representa- tions of bouba = ბუბა and kiki = კიკი both look round. Nevertheless, Georgian speakers match the sound of bouba and kiki to the respective round and angular shapes as reliably as speakers of other languages and orthographic systems. If orthography can evoke a bias, this bias is rather weak, whereas sound-to-shape map- ping is a robust phenomenon. Sounds Can Map to Texture Cross-modal correspondences between sound and touch are less known, but there is reason to believe that these correspondences are deeply connected in evolution. Touch is crucial in sucking, swallowing, mastication, and speech production. The major articulator, the tongue, does not move in free space but is rather in close contact with dif- ferent vocal tract boundaries. Putting hands on the front of the neck, one can feel the resonances caused by the vocal fold vibrations in sounds that include phonation. There has even been a method called “Tadoma” for deaf-blind individuals that allows them to perceive speech via touch- ing the cheeks and neck of their interlocutor (Rosenblum, 2019). Resonances can also appear on the skin of the neck while vibrating the uvula in a uvular-/ʀ/, similar to when a person is gargling. Figure 3 shows an acoustic signal Figure 3. Acoustics of a tongue tip trilled r-sound. Top: an oscillogram. Bottom: a spectrogram with the amplitude envelope superimposed (red line). Summer 2022 • Acoustics Today 45