SEMuGen: Supervenience of Emergent Musical Genres on Selforganized Sociocultural Identities

Jose Fornari Interdisciplinary Nucleus for Sound Communication – NICS State university of Campinas - UNICAMP November of 2008

Empirical evidence
With the advent of Internet media, audio compacting technologies and small devices (e.g. MP3, iPods, Smart-phones) more than ever, music is being democratically accessed by all. Individuals from all walks of life and sociocultural backgrounds are currently able to listen to music in almost any circumstance and to choose their personal repertoire, in a way that music has been, perhaps for the first time in history, broadly used as mood regulator, marketing strategy tool and therapeutic aid. As new technologies for the analysis of music features, named as MIR (Music Information Retrieval) have been also created, now it is possible to quantify musical aspects and to correlate them with cultural changes among society. This project aims to study the intercorrelations between musical genres and the sociocultural aspects of their communities. Brazil is a huge country with a heterogeneous and deeply contrasting population. It comprises almost half of South America in size and is isolated by language and historical background from its neighbouring countries. In its unusual environment, several musical genres have been spontaneously created, seeming to be intimately related to featuring characteristics of these communities. Some musical genres are world-wide famous, such as: Samba, Choro, Baião, Xote and Bossa-Nova. However there are much more genres being created, specially in the contemporary times where some new and interesting styles have appeared, such as: Sertanejo, Brega, Pagode and Baile-Funk-Carioca, among others. They impact millions and are entangled with specific sociocultural identities, (e.g. urban “tribes”, the slam towns, periferias, Brazilian rodeos, etc.). For all these reasons, Brazil is a particularly prodigal scenario to develop this study. In the year of 2007 I was fortuned to be invited to participate in the Braintuning project (, funded by the NEST (New and Emerging Science and Technology) program of the European Commission FP6-2004-NEST-PATH-028570. The Braintuning project pursues the investigation of the musical brain by combining the efforts and expertise of six research groups in Europe and Canada. My part on this project was to develop computational models able to emulate the human perception and cognition of particular musical features by extracting this information directly from digital audio (e.g. perception of complexity in music, perception of tonality, presence of musical pulse, musical event density, etc.). These algorithms reached high correlation coefficients (form 0.50 to 0.65) with ground-truth, also collected by our group. In 2008 I became a researcher at the Interdisciplinary Nucleus for Sound Communication - NICS, at the state university of Campinas UNICAMP ( I started to get in touch with the Centre of Logic and Epistemology, CLE, at Unicamp, and with others universities, such as the music cognition group at the University of Paraná - UFPR, leaded by Prof. Beatriz Ilari, where I realized the sociocultural aspects related to music perception and cognition. From these fruitful interactions I started to realize the strategic position that I was to develop this project, being in Brazil, with the expertise acquired in MIR from my PosDoc at the Braintuning project and my previous PosDoc, on Evolutionary Computation applied to Music and Sound Synthesis, sponsored by FAPESP ( ) process: 04/00499-6. This project aims to use music information retrieval methods, as further described, to study the musical aspects of musical genres in order to seek for hidden correlations with sociocultural identities.

For the series of reasons described above, the experimental part and study of cases of this project will be performed in Brazil. Nevertheless, the results will not be restricted by this scenario, as we plan to study the Brazilian case to trace global inferences of musical and sociocultural interactions. This may lead to the discovery of sociocultural factors that influence and/or are influenced by music production and consumption, therefore, bringing about a deeper understanding of music, society and cultural creation.

Epistemological definitions
It is well known that Music is an artistic expression that communicates with sounds through perceptual, cognitive and affective levels. Its uncanny ability to arouse emotional content is normally borrowed by other forms of artistic expressions to deepen the affective impact of their perceptual and cognitive discourse. Examples of these interactions are seen in the Song form (the interaction of music with poetry) and Soundtracks (music composed for and performed together with movies). In a social perspective, the amount of music produced by a community (here defined as a group of people with same sociocultural background) seems to entail to perceptive musical similarities that can be classified into categories, here defined as Musical Genres, (MG). Common sociocultural background is sometimes also bounded in time (e.g. historical era or generation) and space (e.g. a neighbourhood, city, country). This defines a set of distinguishable or unique characteristics here named as SocioCultural Identity (SCI) that characterizes its community that produces and consumes (listen to) specific MGs. It is sometimes empirically observed that the same SCI can be related to more than one MG, although it may also be consequence to the degree of scrutinity in which we want to define an MG (e.g. the distinction between MGs as rhythm styles among the escolas de sambas in the city of Rio de Janeiro, Brazil) Considering now a systemic [1] approach, SCIs and MGs could be seen as complex open systems with self-organizing [2] structuring that bring about innovation by the action and interaction of endogenous and exogenous agents, thus presenting emergent [3] properties. Our hypothesis is to consider MGs as systems that Supervenes [4] on SCI. Therefore, at some extent, social groups, such as communities, can be characterized by the description of musical aspects of emerged MGs. As it may also occur, MGs can present what is called by cybernetics as feedback [5] loops to SCI, thus influencing it as an exogenous agent. This is depicted in the diagram below.

Sociocultural Identity

feedbacks Musical Genre A Musical Genre C


Musical Genre B

Figure 1. Systemic relation between a SCI and its MGs. Normally, MGs will last as long as the correspondent SCI lasts, as described by the relation of

supervenience of MGs on the SCI. It is possible (and nowadays quite frequent) that, in this category, the correspondence of SCI and MGs to be artificialy sustained, by marketing strategies linked to economic reasons. This may create (or sustain) artificial MGs, resembling what Theodor Adorno described in its definition of culture industry [6], However, at this point, is not clear in which causality order this phenomenon occur (i.e. whether marketing interests sustain artificial MGs or MGs sustain artificial marketing values). If an MG remains existing even after its correspondent SCI disappears (due to community or historical changes) this MG turns into a closed system, thus presenting self-sustainability, what is described as Autopoiesis [7]. Consequently, this particular MG may turn to become what we named here as classical MG (e.g. Brazilian popular music genres such as Samba or Bossa-Nova). In another hand, if the same MG keeps being fed by a queue of distinct SCIs (e.g. the evolution and change of identity of a community along time), this one will remain as an open system but sustained by selforganizing properties. Thus, this particular MG may turn to be what we named here as a folkloric or just folk MG. Classical and Folk MGs can present feedback loops to SCIs. The next figure depicts the concept here explained. Time / Change line Sociocultural Sociocultural Sociocultural Identity A Identity B Identity C



“Folk” Musical Genres

Self-organizing (open system)

(closed system) (self-sustained) Musical Genres

Figure 2. Autopoietic and Self-Organizing behaviours of MGs. SCIs can be characterized by aspects described by Charles Peirce, in his pragmatic approach to semiotics, as he defines that meaning is derived from habits [8]. From Peirce´s perspective, habits are assigned to regularities, invariance, and also regularities associated to transformational process. These two cases can be described as structural and transformational invariance, which constitutes the fundamental ingredients of ecological information [9]. In the research proposed here we are inferring that habits can be assigned to an SCI and by extension they can be modulated by the information flow, derived from MGs. In that sense, the system described by SCI and MG could be related to the ecological information manner, in the context of the interaction between the perception of an organism and its environment. Therefore, it would be possible to define a qualia of information, or MG information flow from and to the SCI. In this way, habits are regulatiries. Information can be seen as the qualia of habits and it would be possible to use the concept of listening habits [10]. Musical meaning and habits will qualify the framework of this investigation. A set of habits will then modulates affordance, or all “action possibilities” in an environment, as

defined by James Gibson [11]. Thus, affordance will draw the boundaries for the set of meanings that will represent an SCI. As habits are denoted by Peirce in term of signs, it might be possible to reduce the categorization of SCIs by the measurement of a set of representing logic sings. This can be given in the form of ethnographic data, such as the ones used by Alan Lomax in his Cantometrics project [12,13], that related music features from folk song to sociological traits, as defined and organized by the non-profit organization HRAF (Human Relations Area Files) [14].

The problematic
This project conveys three main problems to be overcome. They are: 1) how to define and measure the similarity of an SCI. 2) how to define and measure the similarity of an MG. 3) How to track for the interrelations between an SCI and one or more MGs. As described above, this project plans to measure an SCI by similar means as the ethnographic data collected by the HRAF organization. This will lead to the finding of logic signs representing (Peircean) habits of this system, thus defining its characteristics and boundaries, by the principle of affordance. MGs will be described by MIR techniques of music genres classification, or music classification by genre, such as the techniques described in works such [15, 16, 17, 18], among others. The interrelation between MGs and SCI will be pursued by the measurement and statistical analysis of involuntary affective response reactions given by musical listening [19, 20, 21, 22] through the measurement of biosignals [23] of individuals belonging to the community identified by an SCI. The methodological approach to approach these problems is given below.

Methodological Approaches
Through the usage of MIR (Music Information Retrieval) techniques [24], such as audio feature extraction (acoustic descriptors), machine-learning (clustering, classification) and data-ming (similarity, search), data-banks containing music in digital audio format (i.e. MP3, WAV, AIFF, etc.) can be investigated in large scale and categorized into MGs. For this task, a group of descriptors have to be developed in order to make sure that these descriptors will describe music features in lower-level (perceptual), and contextual fashion and their measurements are not collinear. A computational model can be created to automatically classify music audio file based on a specific group of acoustic descriptors and cluster them into possible MGs MGs can be automatically classified by a specific group of descriptors, directly from channels of mass media, such as TV and radio station program. The emotional reaction of people aroused by music listening can be measured, to establish a ground-truth (GT). The appraised music emotion can be measured by voluntary signals, such as categorical models, dimensional models, the bi-dimensional, or circumplex model [25]. The affective emotional reaction can be measured by involuntary biosignals reactions, such as GSR (galvanic skin response), ECG (electro-cardiogram), EEG (electro-encephalogram), and PPG (photoplethysmography). Music is normally divided into three aspects: 1) melody; related to pitch, 2) rhythm; related to pulse and 3) harmony; related to timbre and textures. We believe that this classical taxonomy may resemble how humans discriminate musical stimuli, in a cognitive and affective way. Therefore, descriptors will be developed and organized into these three groups, in order to measure and classify each musical part and to relate them to identify and establish boundaries between MGs. The clustering and classification part of music, based on these families of descriptors, can be performed by automatic machine-learning methods. One in particular that this project plans to experiment is the Evolutionary Computation approach [26], as it is a method whose author has familiarity [27] and is being widely used as a non-supervised approach to automatically finding the best (feasible) solution of any generic problem, including the non-deterministic ones [28].

Case Studies
The empirical part of this project will be developed in Brazil because of the above mentioned reasons, such as diversity of MGs, and SCIs resulted from sociocultural and economic contrasts, language isolation (the only country in South America that speaks Portuguese), ethnological diversity and large population (estimated 190 millions, in 2008) and size (8.514.877 km²). Nevertheless, to assure that this project is feasible, it will be necessary to restrict even more this scenario. As a starting point, it is here selected three case studies. They are described below. 1) Measuring the correlation between regional speech patterns with the music rhythmic patterns found into the communities music genres. The Brazilian Portuguese language and its regional differences has been object of several studies [29]. This case study aims to select specific regions in Brazil that present deeply contrasting speech pattern, especially in terms of their pitch and rhythmic contour and measure these patterns using acoustic descriptors. In the same way, use descriptors to measure melodic and rhythmic aspects characteristic of MGs from these communities. Then, study the correlation between the communities regional accent with their musical genres. This may lead to find new cognitive relations between language and music, and correlations between sociocultural signs and musical genres aspects. 2) Music Genres and Digital Ecosystems Nowadays several means of technology allow the mass distribution of music throughout cyberspace. The organization of cyber communities and their sociocultural and economic interrelation have being defined as Digital Ecosystems [30]. Social networking and discussion sites such as (owned by Google) are widely famous in Brazil, containing a myriad of virtual communities formed around Musical activities. This case study aims to take advantage of this organization to research the sociocultural implications of in the real-world of virtual communities oriented by musical genres. 3) Comparison between individual and social music affective response This case study aims to research the influence over an individual affective response to music listening when this individual is alone and when this one is within a crowd. The mass behaviour of affective response will be analyzed with biosignals measurement tools, such as PPG, in the individual level, and specific acoustic descriptors developed to analyse audio and image mass responses. The results will be studied in order to seek for hidden correlations. An experiment that shall be conducted here is to analyse the affective response aroused by Sambas-enredos from Brazilian Escolas-de-samba in individuals and crowds of their communities (slam-towns, or favelas), as each one has its own specific SCI and MG (i.e. its specific escola-de-samba rhythmic pattern). This experiment can lead to a better understanding of the relation between objective (mass) and subjective (individual) affective musical measurements.

Expected Conclusions
This project aims to study the intercorrelations between MGs and SCIs from a systemic point of view. This may lead to a deeper understanding of several points of social musical interaction, still not clear. Among others: 1) how communities spontaneously create music, 2) how these communities relate to their musical production in a perceptual, cognitive and affective way, 3) how (and why) the communities consume their own musical styles, 4) how a community relates to “external” musical styles (the ones not produced by them), 5) how possible it is to characterize or distinguish communities by the analysis of their MGs, 6) how to characterize if a music belongs to an specific genre, 7) how folkloric music genres are created and maintained along time, 8) how classical musical genres are established and why they appeal to communities so different from the ones that created them, 9) how possible it is to predict the creation of new musical genres by analyzing the current community SCI, 10) how possible it is to predict sociocultural changes by the analysis of current MGs.

I would like to thank the helpful hints, comments and insightful discussions that I was fortunate to have. In special to Prof. Jonatas Manzolli, Prof. Adolfo Maia Jr., Prof. David Hesmondhalgh, Prof. Martin Clayton and Profa. Nikki Dibben.

[1] Charles François. Systemics and Cybernetics in a Historical Perspective. Systems Research and Behavioral Science, Vol 16, pp.203-219. 1999. [2] E. Morin. Introduction à la pensée complexe. Paris. ESF. ISBN 2710108003. 1990. [3] H. Atlan. Entre le cristal et la fumée: Essai sur l\organisation de vivant, 1979. [4] Terry Horgan. From supervenience to superdupervenience: meeting the demands of a material world. Mind. 102: 555-86. 1993. [5] R. W. Ashby. Introduction to Cybernetics. Methuen, London, UK. 1956. [6] David Hesmondhalgh. The Cultural Industries. Sage. 2002. [7] Humberto Maturana, Francisco Varela. The tree of knowledge: The biological roots of human understanding. Boston: Shambhala Publications. ISBN 978-0-87773-642-4. 1992. [8] Charles Peirce, Selected Writings: Selected Writings. Courier Dover Publications. ISBN 0486216349, 9780486216348. 1966. [9] Maria Eunice Quilici Gonzalez, Mariana Claudia Broens, Fabricio Loffredo D'Ottaviano. Abductive Reasoning, Information, and Mechanical Systems. Studies in Computational Intelligence. Springer Press. ISBN 978-3-540-71985-4. 2007. [10] L. Oliveira, W. Haselager, J. Manzolli,M. Gonzalez. Musical meaning and logical inference from the perspective of Peircean pragmatism. In: C. Tsougras & R. Parncutt (eds), Proceedings of the fourth Conference on Interdisciplinary Musicology (CIM08)'. 2008. [11] James J. Gibson. The Theory of Affordances. In Perceiving, Acting, and Knowing. Eds. Robert Shaw and John Bransford, ISBN 0-470-99014-7. 1977. [12] Alan Lomax. Folk Song Style and Culture. New Brunswick, U.S.A.: Transaction Publishers (2000). ISBN 0-87855-640-0. 1968. [13] Alan Lomax, Irmgard Bartenieff and Forrestine Paulay. Choreometrics: A Method for the Study of Cross-Cultural Pattern in Film. Sonderdruck aus Research Film 6: 515–17. 1969. [14] HRAF - Human Relations Area Files . [15] Mitali Banerjee. Music Classification by Genre: System Performance. Connexions. ( 2003. [16] McKay, C. and I. Fujinaga. “Musical genre classification: Is it worth pursuing and how can it be improved?” Proc. ISMIR, 2006. [17] Paolo Annesi, Roberto Basili, Raffaele Gitto, Alessandro Moschitti. Audio Feature Engineering for Automatic Music Genre Classification. Conference RIAO2007, Pittsburgh PA, U.S.A. 2007. [18] Perry Cook. Music, Cognition, and Computerized Sound. MIT Press. ISBN-13: 978-0-262-032568. 1999. [19] David Huron. Sweet Anticipation. MIT Press. ISBN-13: 978-0-262-08345-4. 2006. [20] David Temperley. The Cognition of Basic Musical. Structures. MIT Press, Cambridge, MA, 2001. [21] Leonard Meyer. Emotion and Meaning in Music. 1957. [22] Bob Snyder. Music and Memory. MIT press. ISBN 0262692376. 2001. [23] International Conference on Bio-inspired Systems and Signal Processing.

[24] Michael Fingerhut. Music Information Retrieval, or how to search for (and maybe find) music and do away with incipits. IAML-IASA Congress, Oslo (Norway). 2004. [25] J. Russell. A circumplex model of affect. Journal of Personality and Social Psychology, Vol. 39, pp. 1161-1178. 1980. [26] K.A. De Jong, Evolutionary computation: a unified approach. MIT Press, Cambridge MA, 2006. [27] Jose Fornari. Evolutionary Sound Synthesis. Doctoral dissertation. Faculty of Electrical Engineering. State University of Campinas - Unicamp. 2003. [28] Pedro Larrañaga, José A. Lozano. Estimation of Distribution Algorithms: A New Tool for Evolutionary Computation. Springer Press. ISBN 0792374665, 9780792374664. 2002. [29] Milton Azevedo. Vernacular Features in Educated Speech in Brazilian Portuguese. University of California. ( [30] OPAALS - Open Philosophies for Associative Autopoietic digitaL ecosystemS [31] T. Kohonen. Self-organizing maps. Springer-Verlag, Berlin, 1995. [32] M. Frühwirth, A. Rauber. Self-Organizing Maps for Content-Based Music Clustering. Department of Software Technology, Vienna University of Technology. Favoritenstr. 9 - 11 / 188, A-1040 Wien, Austria. [33] Journal of Systemics, Cybernetics and Informatics [34] Humberto Maturana, Francisco Varela. Autopoiesis and Cognition: The Realization of the Living. Springer Press. ISBN 9027710163, 9789027710161. 1980.

Project Details:
Acronym: SEMuGen Type of project: Supporting Frontier Research - ERC Starting Grant Call identifier: ERC-2009-StG_20081119 Duration: 60 months Targeted ERC review panel: SH5: Cultures and cultural production Alernative ERC review: SH4: The human mind and its complexity ERC keyword 1: SH5_9: Music and musicology, history of music 2: PE7_4: Simulation engineering and modeling 3: PE6_11: Multimedia 4: LS5_7: Cognition free-keywords: complex systems, autopoiesis, supervenience, self-organization, sociocultural identity, music genre classification, music information retrieval, music cognition, music affective response, ethnomusicology.