SpeaK: A Methodology to Communicate About Sounds

SpeaK: a methodology to communicate about sounds Patrick Susini1, Olivier Houix1, Nicolas Misdariis1 and Romain Barthélémy2 1 STMS (Ircam-CNRS-SU) 1 place Igor Stravinsky, 75004 Paris 2 Ircam Amplify 7 rue de Turbigo, 75001 Paris [email protected] Abstract. The first meetings of a sound design project aim to present the request which may concern different aspects, such as pleasure and functionality of the sound of a new product, or identity of a brand. It is then crucial to elaborate recommendations for the sound designer. However, the major difficulty encountered is to specify the request in terms of sound features. The Speak methodology is based on an efficient co-design workshop to help participants to specify their request with words related to sound features. This methodology is based on a lexicon composed of 35 words often used by professionals to described sound features. Each word is related to a sound feature explained by a definition and highlighted by a corpus of sound examples, and is associated with a card used by the participants during the brief meeting. Using this methodology, participants are actively involved in the process to develop sound recommendations. Keywords: Sound Semantics, Sound Lexicon, Sound Design Toolkit 1 Introduction Sounds are not only pleasant or unpleasant, they serve many other purposes: they contribute to the brand image and the coherence of a product, elicit emotional reactions in users, and even have functional aspects in terms of information. As such, sound designers not only want to diagnose the quality of a product sound, they also want to design its timbral and temporal characteristics to address different interdependent aspects, such as pleasure, identity, and functionality, as well as taking into account the environment in which it will be heard. As an example, most people in France associate the jingle played before any vocal announcement in French railway stations with the French national railway company (SNCF). The timbral features and temporal properties of the jingle have been specifically designed to attract the attention of users and to communicate the values of the company. In addition, this sound has been designed to be enjoyable in the complex sonic environment of railway stations. Several stages of creation and testing were carried out to arrive at this proposal. Another example is the sound designed for the ZOE electric car - produced by the French car manufacturer Renault - Proc. of the 17th International Symposium on CMMR, London, UK, Nov. 3-7, 2025 259 2 P. Susini et al. to inform pedestrians of its movement on the street, and also to inform the driver about the car’s state of functioning (e.g., its speed); its sound, composed by composer Andrea Cera, is now emblematic of the car’s identity and is nicely integrated into the urban sound environment. The news sounds created in the two previous examples are referred to as intentional sounds. They are the result of a sound design approach implemented to create new sounds in order to make intentions audible in a given context of use [1]. We need to imagine and create new sounds that satisfy functional constraints (for example, detectability) as well as constraints in terms of pleasure, identity and ecology (for example, in terms of sustainability). Fortunately, practice in sound design is led by a strong creative process based on different sources of inspiration; in addition to their technical skills, sound designers are characterized by creative abilities to make sound sketches composed of different timbres, which can make all the difference in producing a successful articulation between functionality, pleasantness, and identity of a new product sound with respect to the sound environment. As it has been done for science fiction movies, sound designers have to imagine and create new intentional sounds for our everyday environments. However, brands and manufacturers have expectations or values that constrain the creative process, and it is not obvious for sound designers, at the first step of the process, during the brief, to understand and to translate those expectations by words related to sounds. For example, the French railway company wanted to have a sound logo that reflected three values - benevolent, simple, and efficient. But which sound features could be used to translate those values? Abstract values must be translated into concrete sound features. Thus, meaningful words for sounds such as “bright”, “warm”, “rough”, … related to sound features are crucial for clear communication, collaboration, and creative direction. These descriptive words should help bridge the gap between abstract auditory ideas and practical execution by focusing the proposals and exchanges on attributes of sounds during discussions between the different participants (e.g., developers, communicators, engineers, technicians, designers and project managers) in a brief. Descriptive words are vital but not sufficient because it is necessary that their meaning be shared by all participants, in order to reduce misunderstandings like a client asking for “more energy” when they mean “higher pitch”. It is therefore necessary that participants in a brief can have a shared understanding of the words with the help of clear definitions. That said, definitions can sometimes remain too abstract, leaving a certain ambiguity. One solution in this case is to offer sound examples that allow participants to grasp the meaning of the words through several examples. In addition, it is necessary to enable participants to give meaningful input without needing technical knowledge, and without restrictions due to their lack of expertise. It is therefore crucial to promote a friendly and engaging environment to let all participants expressing themselves freely and spontaneously. Words for sounds could serve as an intermediary object to foster cooperation among partners, much like in a collaborative board game [2]. In section 2, we discuss the different strategies to describe sounds and to speak about sounds. In section 3, the SpeaK methodology is introduced in order to translate design intentions into words related to sound features to provide clear creative direction for the sound designer during the brief. This methodology is based on a lexicon associated Proc. of the 17th International Symposium on CMMR, London, UK, Nov. 3-7, 2025 260 The SpeaK methodology 3 with cards used in a co-design session during the pre-design phase. Finally, we return to some of the limits, advantages and perspectives of the method in section 4. 2 How do we perceive sounds? Several experimental studies on sound perception premised that listeners are able to focus on several different aspects of sounds. The idea was initially introduced by Bill Gaver [3, 4], who proposed the distinction between musical listening (when the listener focuses on the qualities of the acoustic signal) and everyday listening (when the listener identifies the event causing the sound and its properties: type of interaction, material, shape of the objects interacting, etc.), in a widely cited discussion inspired by Gibson's ecological approach to perception [5]. Illustrations of these different modes of listening are found in different fields of research. When considering everyday listening, it appears that not only qualities of the acoustic signal are responsible for the recognition of the source (i.e. the information present within the sound): the context and the knowledge of the listener may also influence the identification. This question has been explored thoroughly in a series of studies [6, 7]. Only a few experimental studies have reported differences based on the knowledge of the listener. The influence of listener's expertise on the categorization of environmental sounds, reported in Lemaitre & al. [10], revealed that experts used acoustical similarity more often than non-experts, who used the similarity of the source of the sounds. In addition, this study reveals that identifiability of sound source - that is, to what extent the cause that produces the sound can be recognized - might influence the categorization strategies: well-identified sounds were more often grouped together because of the similarities of the cause, whereas unwell-identified were grouped together more often because of the acoustical similarities. 2.1 Different(strategies(to(describe(sounds Unfortunately, nonexperts in sound are not used to perceive, and thus to describe, sound qualities of the acoustic signal because they mainly focus on the source, and therefore when they speak about sounds, they name the source of the sounds (for example, “this is the sound of a hairdryer,” “it is a vacuum cleaner,” “this is a trumpet”) or they describe the action that produced the sounds (for example, “someone is hitting a glass,” “this is the sound of a string being pinched,” “she is pushing a switch”). This is the causal strategy, which is the most intuitive way to speak about sounds for nonexperts corresponding to the everyday listening strategy defined by Gaver. Descriptions are sometimes solely related to a specific meaning in a specific context or location: alarm sounds in intensive care units have a specific meaning only for the staff. This is the contextual strategy: verbal descriptions are not specific to a sound’s feature but are more context-dependent; for example, the knock-knock sound can indicate a neighbor is knocking at the door, or a request for silence in a court of law. Other fairly common ways of speaking about sounds involve general judgment such as "It is an annoying sound", or imitation of the sound characteristics with voice. For example, we produce the sound "Pam! Pam!" for describing an impact on wood, and Proc. of the 17th International Symposium on CMMR, London, UK, Nov. 3-7, 2025 261 4 P. Susini et al. "Tinnng! Tinnng!" for an impact on glass or metal. We can also employ onomatopoeias: "Toc! Toc!" in French or "Knock! Knock!" in English to describe a sound made by someone knocking on a door. Finally, it seems that descriptions are seldom based on the sound itself in terms of acoustic characteristics and timbre features. This is the reduced listening strategy: descriptions are directly related to the features of a sound independently of the meaning, the process that produced the sound, or its location. This is related to the musical listening strategy defined by Gaver. This distinction between the different strategies was highlighted by Pierre Schaeffer [9], and later by Michel Chion [10]. With the idea of developing a sound typology beyond tonal music, Pierre Schaeffer of the Groupe de Recherches Musicales (GRM) proposed listening experiments focused on the characteristics of sounds, notably by looping the groove of a vinyl record on itself, or by cutting some part of a sound, like the attack of a sound (eg. a piano note), which no longer allows the source at the origin of the sound to be recognized. By this trick, listeners focus their listening on the characteristics of sounds, and after several repeats, refine their listening on sounds. 2.2 Different words to speak about sounds Schaeffer went on to establish a ‘Solfège de l’Objet Sonore’ (literally music theory of sound objects), by which he aimed to describe acoustical qualities of any kind of sounds based on seven typomorphological criteria: three criteria for matter (mass, grain, harmonic timbre), two for form (dynamics, pace) and two for variation (mass profile, melodic profile). Following Schaeffer’s efforts, several attempts were made to build an exhaustive terminology to describe sounds. Electroacoustic music composer Denis Smalley focused on a particular aspect of sound morphology commonly referred to as ‘spectrum’, leading to the concept of spectro-morphology [11]. More recently, the idea of sound object was also extended to the notion of ‘semiotic object’ (or temporal semiotic units: TSU) by the Laboratoire Musique et Informatique de Marseille, MIM [12] in order to describe musical patterns and effects in electroacoustic composition. Just as Schaeffer did with his Solfège theory, the TSU theory is illustrated with various audio examples extracted from musical recordings. Although original and very detailed, the different proposals remain quite complex and very few people use them today in its fullest form. The main criticism is probably that the words, terms, or formulations require some expertise, and are not based on common words of the language easily usable by non-experts during a short brief in a sound design process, or during a training earing session. But several contributions are to be taken into account, such as different categories of criteria concerning general and temporal aspects of sounds, as well as timbre characteristics. Another important aspect is the use of sound examples to illustrate the characteristics of sounds associated with words. More recently, vocabulary employed by sound professionals was explored [13, 14, 15]. While there is some consensus on a large number of words, the meaning of words used to describe sounds can vary from person to person, even for sound professionals ranging from sound engineers to musicians [16, 17]. Proc. of the 17th International Symposium on CMMR, London, UK, Nov. 3-7, 2025 262 The SpeaK methodology 5 3 The SpeaK methodology A process in sound design includes in general three successive steps: analyzing, creating and testing. This 3-step iterative process is in line with different formats of the classic design processes that are usually proposed when designers work on a project, e.g. the brief/do/check sequence [18]. The analyzing step aims to develop specifications, it is the pre-design phase called brief. The brief is the first crucial step in a design process during which initial intentions are often expressed with terms related to a meaning, a function or an emotion, but initial intentions could be also very vague, at best defined in the form of a mood board. The challenge during the brief is to obtain welldefined intentions in terms of identity and functionality to provide recommendations for the sound creation stage. During this first step, the diversity of strategies to speak about sounds can be a serious obstacle for communication, especially when it involves participants, ranging from the project manager to the communication manager, with diverse backgrounds and levels of expertise in sounds, and even no expertise at all. But the biggest obstacle in reality is that participants often have difficulty when it comes to talk about sounds; they often have some personal and abstract sonic ideas, and do not have the necessary vocabulary to express different aspects of their ideas, and therefore, they do not dare to express themselves. On the other hand, sound designers usually need information related to sound features such as general qualities (intensity, pitch, …), timbre, and temporal properties. For example, the intention for an alarm sound in the context of a hospital could be described by a project manager by “an alerting sound but kind”. Does it fit with a long, dull, smooth, continuous, and highpitched sound, and loud enough? Unfortunately, there is no common practice for speaking about sound features; sound designers often complain about the lack of methodologies to communicate about sounds in a sound design process. In general, speaking about sound features is difficult because there is a lack of relevant words to promote communication on sounds, or to express the sensory experience with sounds, and also a lack of consensual definitions for the words. In addition, words, and their definitions, are sometimes not sufficient to describe a hearing experience associated with a sound characteristic. In that case, sound examples are a good way to understand the relationship between words and sounds, and to experience this relationship. Speaking about sounds is also difficult because there is a lack of tool for generating ideas and exchange on a target between the sound designer and stakeholders, for encouraging open dialogue to ensure all voices are heard, and for speeding up the workflow, especially when timelines are tight. Thus, the Speak methodology is proposed to fill these gaps, and it is based on: A sound lexicon. The sound lexicon words4sounds is proposed on a Web platform to present a list of words related to several sound features, with definitions and sound examples; A tangible toolkit. The tangible toolkit is composed of a set of cards related to the list of words and a game table that are used during a co-design playfull session with the different participants. Proc. of the 17th International Symposium on CMMR, London, UK, Nov. 3-7, 2025 263 6 P. Susini et al. The SpeaK methodology is used to bridge the gap between abstract ideas and a set of recommendations based on words related to sound features. 3.1 The sound lexicon words4sounds The sound lexicon words4sounds is defined by: ─ a list of words used to describe sound characteristics ─ a definition to explain the meaning of each word of the list ─ a corpus of sound examples to foster the perception of the characteristics The lexicon words4sounds was elaborated by the Sound Perception and Design group (Ircam STMS Lab) on the basis of Maxime Carron's PhD [19]. It is proposed on the SpeaK1 web platform developed in order to organize, present, and share sound lexicons combining words, definitions and sound examples. The list of words is based an academic review of a large number of works dealing with verbal descriptions of timbre for different kinds of sounds, from abstract to everyday sounds. Then, this review was combined with interviews with French speaking sound professionals from different fields (e.g., composers, sound designers, sound engineers, etc.). The lexicon is composed of 35 relevant words, frequently used by professionals, to describe the perceived characteristics of a sound was proposed (e.g., tonal/noisy, low/high, dry/resonant, dull/bright, rough, warm, round, nasal, rich, strident, dynamic, crescendo/decrescendo, ascending/descending, fast/slow attack …) as an extension of Schaeffer’s work. The lexicon is proposed in English and French. The list of words is structured in three classes of general aspects (e.g., high/low, short/long, etc.), temporal morphology (e.g., crescendo/decrescendo, continuous/discontinuous, etc.), and timbre attributes (e.g., dull/bright, nasal, warm, etc.). The definitions are based on analysis of interviews with French speaking sound professionals. As an example, the definition for Bright is: “The word Bright (brillant) is often used to describe a timbre characteristic of different sounds; the opposite word used is dull (mat). The words dull and bright refer to the amount of high-frequency energy perceived within a sound. A dull sound has a low amount of high-frequency components. The term muffled is also used. A bright sound contains a substantial amount of high-frequency components. The term sharp is also used.” Good musical examples for bright are the glockenspiel and the trumpet. The sound examples were created, recorded, and mastered under the direction of composer Roque Rivas at Ircam, except the environmental sounds which were proposed and recorded by François Hamon as part of his internship at DNSEP2 Design Sonore, ESAD TALM Le Mans. They were created to highlight the sound features related to the words for different categories of sounds. 3.2 The SpeaK session overview The first brief in a sound design project is crucial to elaborate recommendations for the sound designer. However, in the first brief which bring together different non-expert 1 The SpeaK web page is reachable here https://speak.ircam.fr/en 2 https://designsonore.esad-talm.fr/ Proc. of the 17th International Symposium on CMMR, London, UK, Nov. 3-7, 2025 264 The SpeaK methodology 7 sound practitioners ranging from project managers to designers, the major difficulty encountered is to specify the request with words related to sound features, in a quiet short time. Thus, in several projects with industrial [20] or institutional partners, the SpeaK methodology was used in order: ─ to have a shared and unique list of words to describe the expected sound features ─ to involve all the participants in a collective co-creation, and guided reflection ─ to foster the contact with the sound designer The SpeaK session unfolds in two main parts: Training in Lexicon Use and a CoDesign Workshop using a structured toolkit. Different steps can be proposed to the participants during a project using the lexicon, depending on the time available and the duration of the project. Training in Lexicon Use Phase #1. This phase is mandatory and is crucial to introduce the non-experts to the world of sounds – and related words. The session leader presents all the words of words4sounds lexicon by playing examples using the web interface. The definition of each word is presented, and participants can ask for clarification. Then each participant discovers the lexicon individually by listening to examples using headphones. This individual exploration can last more than 30 minutes. Finally, the session leader ensures that all the words have been understood and that the examples have made it possible to identify the associated sound characteristic. Phase #2. This phase is optional, but crucial to ensure a good understanding of the link between a word and the associated sound characteristic, using forced-choice listening tests with only one possible correct answer. Two levels of difficulty are tested. ─ Level#1: the first level, the simplest, consists for the participant to select the sound the most representative of the attribute involved. For example, 5 sounds are presented but only one (preselected by the session leader) is related to the word displayed. The participant is asked to select the corresponding sound. ─ Level#2: the second level is similar to the first one, but differs from it in difficulty. Indeed, the correct answer does not emerge from the corpus in an obvious way since the sound examples are constructed in order to be distinguished on smallest perceived difference along the sound feature tested. For example, 5 sounds are presented with small differences in brightness (determined from knowledge of psychoacoustics), the participant is asked to select the brightest one. Proc. of the 17th International Symposium on CMMR, London, UK, Nov. 3-7, 2025 265 8 P. Susini et al. Fig. 1. Training session in two phases During a training session, individual and collective explorations of the lexicon are alternated with the different tests. After each test, terms are discussed collectively to ensure a common understanding. This global training ensures that participants involved in the same project have a rich, varied and shared vocabulary that is adapted to describe a large number of timbre features and temporal properties appropriate for an important variety of sounds. This procedure is an alternative to sensory evaluation [21] often used to reveal a list of words specific to the timbre of a set of sounds in relation to consumer preferences. The sensory evaluation requires several steps of discussion, training, and testing with a panel of experts, a process which is often very long (several weeks) and specific to a set of sounds. Co-Design Workshop using a structured toolkit The SpeaK session is a structured co-creation session in which participants use a toolkit to describe a sonic target collaboratively. Cards, designed by Lundja Medjoub, are made available to each participant who can select or reject a word, and exchange with other participants focusing only on the sound features constrained by the cards. This codesign setup is transposed from Carron’s work and was formerly inspired by specific design approaches like Kansei. Discussions are mediated by the cards and the area of exchange is materialized with a board (figure 2 & 3), by analogy with a standard board game or role play. Supporting that, the words4sounds lexicon played the role of help to which anyone can refer during the session. Before starting, a brief warm-up is performed: a training target (e.g. an existing sound logo from another brand, a short piece of music, or a fictional brief) is used for practice. The whole process in 4 steps, presented below, and rules are explained through playful interaction lasting approximately 40 minutes. After a break, the first step of the session starts (1) (see figure 2). The group of participants collaboratively defines the creative target (e.g. a future sound identity). This target is described through brand values, objects, references, and sensory cues (mainly visual). The target can be imagined or real, but must be rich enough to inspire interpretation. Second (2), each participant describes the target using their own words related to sound or music, emotional aspects, or abstract sonic ideas. These are written on blank cards. Participants share their descriptions and justify their word choices, enabling Proc. of the 17th International Symposium on CMMR, London, UK, Nov. 3-7, 2025 266 The SpeaK methodology 9 deeper understanding of individual perspectives. Third (3), each participant now describes the same target using SpeaK lexicon cards. Each card selected must be explained, encouraging critical thinking and negotiation of meaning. This creates a bridge between intuitive descriptions and the structured vocabulary. Finally (4), a collective discussion is held to reflect on the selected semantic descriptors. Terms that are incompatible with the target are discarded by placing them in the "garbage can". This step helps refine and converge on a semantic identity for the sound. The final objective is to co-construct a sound brief in the form of a semantic portrait that is as precise as possible, and shared by all participants, using a structured participatory process. Fig. 2. Four steps of a SpeaK co-design session In 2024, IrcamAmplify3 company initiated a project to create a sound identity for Sorbonne Université, one of France's most prestigious academic institutions. The challenge was to design a sonic expression that reflects the university’s values, heritage, and forward-looking spirit, while ensuring broad acceptance across diverse internal stakeholders. To meet this challenge, sound designer Romain Barthélémy (one of the co-author) and Lundja Medjoub employed the SpeaK methodology in order to align sound design with semantic intention through structured co-creation and shared vocabulary (figure 3). 3 https://www.ircamamplify.com/ TARGET 1 The target at the center is defined by brand values, sounds, objects, references … sophisticated complex 2 Each participant describes with her/his own words related to sounds or musics using blank cards and explain his/her choice Free Verbalization TARGET Crescendo Rich 3 Each participant chooses several cards from the lexicon that best fit with the target and explain his/her choice Words from the lexicon Smooth Warm complex Free Verbalization TARGET Crescendo 4 Anti-descriptors are placed in the bin Words from the lexicon Smooth Warm complex Free Verbalization Rough Continuous TARGET sophisticated sophisticated Proc. of the 17th International Symposium on CMMR, London, UK, Nov. 3-7, 2025 267