Composers Sample 2

COMPARATIVE STUDY OF TWO RESEARCH METHODOLOGIES FROM

PUBLISHED CASE STUDIES

PG RESEARCH METHODS ARTS | ASSESSMENT ONE

MU70168E

ELECTRONIC MUSIC COMPOSITION

Introduction
This essay will critically reflect and compare the different research methodologies and
theoretical approaches used by two case studies that present their findings on artificial
intelligence (AI) in music production. Additionally, the success of each methodology
used in relation to the outlined objectives will be evaluated.

AI is an area of study that interests me as a dynamic and constantly evolving field,

especially in music production. In the short time it has been utilised, AI has provided the
groundwork to allow technology to progress in ways that were not considered possible
before machine learning. Other multimedia fields, such as photography and film, have
embraced new technologies. However, the audio industry is far behind when compared
to these technological advancements (Moffat and Sandler, 2019). I plan to explore AI
further as part of my A2 assessment for the PG Research Methods module, my
dissertation and in-turn, my own practice.

An overview of research methodology

A research methodology is an overarching term of ‘general research strategy that

outlines the way in which research is to be undertaken’ (Howell, 2013). This is not to be
confused with research methods, which are techniques used in the acquisition of data
for analysis and evaluation to further understand or bring new information to a chosen
area of study (Kothari, 2004).

There are two principle methodological approaches when conducting research known as
qualitative and quantitative. The choice of an approach is influenced by the research
problem, the personal experiences of the researcher and the audience for whom the
researcher writes. (Creswell and Creswell, 2017)

A quantitative approach lends itself to scientific areas of study where research involves
‘measurement and assumes that the phenomena under study can be measured’
(Watson, 2015). Quantitative research is used when testing or confirming theories or
assumptions and usually involves large-scale and representative data sets (Blaxter,
Hughes and Tight, 2010). However, as an area of study, the arts generally rely on data
that is subjective or holistic in nature, therefore quantitative approaches are less likely to
be taken. Qualitative research involves collecting and analysing information in any form
that is non-numeric (Blaxter, Hughes and Tight, 2010). Commonly used in humanities
and social sciences, qualitative approaches are used to build robust understandings of
topics, relying on inductive designs that produce rich and descriptive data (Leavy, 2017).
Sample sizes are typically smaller and in-depth, but considered valid when describing a
phenomenon or area of research, making qualitative methodologies a common choice
for arts research.

Sometimes viewed as a third methodological approach, the term ‘mixed methods’ is

used to describe when a researcher collects and analyses data, presents results and
draws conclusions using both qualitative and quantitative methods in a single study,
sometimes resulting in a more comprehensive understanding of the phenomenon under
investigation (Tashakkori and Creswell, 2007; Leavy, 2017).

PAGE 1
There are a variety of methodologies available for researchers to choose from when
conducting inquiry. According to research by Creswell, the five approaches to qualitative
inquiry are narrative research, phenomenology, grounded theory, ethnography and case
studies (Creswell, 2007). However, many examples do not necessarily adhere to just
one methodology, instead, there could be a mix of methodologies used when answering
the research question. Two articles on the same topic or area could be undertaken with
differing methodologies and as a result, produce contrasting conclusions and theories.

‘Music Creation by Example’ by Frid, Gomes and Jin, 2020 and ‘Approaches in
Intelligent Music Production’ by Moffat and Sandler, 2019 are two examples of case
studies that delve into the development of AI in the production of music, but make use of
contrasting research methodologies.

Paper A: Music Creation By Example - Emma Frid, Celso Gomes and Zey Jin
(2020)

This article details Frid, Gomes and Jin’s experience whilst designing and evaluating a
user interface (UI) for an AI system. Their aim was to create a software that generates
unique music from analysed data, in this case, other compositions, that could be
adapted to the video content that creators are producing. A detailed summary was
presented, explaining current software solutions available alongside existing literature,
highlighting their pitfalls and therefore gaps in the market. It documents the process of
obtaining primary data using an iterative methodology, employing techniques found in
user studies research (UX).

According to Nielsen, when developing UIs, an iterative methodology is used as a

means to refine an interface over several stages (Nielsen, 1993). Each iteration should
be tested by users or other usability-evaluation methods which are designed to uncover
problems. UX is a popular research design when developing products and services or
gauging how people interpret and use them (Goodman, Kuniavsky and Moed, 2012).
There are a wide range of methods available when conducting UX, of which, the authors
used questionnaires, semi-structured interviews and usability tests to collect and analyse
both qualitative and quantitative data, taking a mixed method approach.

The research process, and the methods used, can be simplified into two sections. These
are preliminary studies and evaluation.

Preliminary studies

The pre-studies were conducted using a formative questionnaire with 30 participants and
an online questionnaire testing parallel prototypes with 60 participants.

The authors state that the formative questionnaire was used to understand their target
users’ needs when adding music to videos (Frid, Gomes and Jin, 2020). Formative
research is regularly used as an exploratory method during the early stages, or at the

PAGE 2
beginning of a design project which will guide the entire process (Makewa, 2015). The
research conducted helps decide and describe the target audience, understanding
factors that influence their behaviour and determine the best way to reach them (Cook
and Sutton, 2014).

After three design prototypes had been made, they were evaluated using an additional
online questionnaire. The authors specify the use of parallel prototyping to explore
possible design directions (Frid, Gomes and Jin, 2020). Parallel prototyping is used to
subject multiple product designs to user feedback simultaneously, with the intention of
finding a design that the majority of their user group prefers. Research by Dow et al.,
(2010) found that when multiple alternatives are created and evaluated in parallel, they
produce higher-quality work and experience a greater increase in self efficacy. The
authors presented three designs to 60 participants, all of whom were video creators and
musicians with a range of experience. This method proved successful, providing the
researchers with a final set of data, which would conclude the pre-study investigations
and allow them to design the UI, ready to be evaluated through user testing.

Evaluation

The first functioning prototype was tested with two rounds of usability tests, the first
being by 12 participants who met the criteria as target users. The second round of
evaluation used expert review with just 2 participants. This method of user testing
gathers information about how people use a prototype, regardless of the design process
(Goodman, Kuniavsky and Moed, 2012). The researchers encouraged participants to
think aloud while using the UI, which is a frequently used tool when conducting usability
tests. ‘Think-aloud’ is cheap and easy to do, however, the downside is that it can
sometimes provoke unnatural situations or biasing user behaviour due to the fact that
‘most people don’t sit and talk to themselves all day’ (Nielsen, 2012).

Once the participants had completed the objectives set by the researchers, semi-
structured interviews took place. The semi-structured interviews asked open-ended
questions, allowing for a discussion with the interviewee(s), providing more descriptive
and rich answers and in turn, data. This interview method is one of the most common in
qualitative research and can be applied to both individual and group settings, making it
an ideal method to use for this case study. Following on from the session, the
participants were asked to fill out a questionnaire which contained topics related to the
interview questions as well as frequently used metrics for evaluating interactive systems
(Frid, Gomes and Jin, 2020). This questionnaire used a quantitative Likert scale, which
is a method used to present multiple-choice answers that defines a range of possible
answers, including a neutral middle option (Goodman, Kuniavsky and Moed, 2012).

Sampling

PAGE 3
The researchers used purposive sampling techniques when choosing participants for
their primary research. All the methods used to gather data were given to a range of
experienced video creators and musicians, which would allow for more valid data to be
collected as participants would likely use correct terminology and have points of
reference comparing to their own regular workflows.

According to (Palinkas et al., 2013), this sampling technique is widely used in qualitative
research when selecting ‘information-rich cases related to the phenomenon of interest’.
There is a risk of unintentional bias due to the lack of random sampling that takes place
in purposive sampling. However, the researchers had a specific end user in mind whilst
developing the UI, so it made sense that their tests would be conducted on the relevant
subjects.

Analysis

The authors presented their methods and results chronologically throughout the article.
This made the process easy to follow and helps the reader understand how and why
they were using certain data collection methods to further the research. Blaxter et al.,
say that analysis is an ongoing process where earlier analysis informs later data
collection, this ties in with the authors’ use of an iterative methodology (Blaxter, Hughes
and Tight, 2010).

Thematic content analysis was specified as the technique used when analysing
qualitative data collected from open-ended answers from the questionnaire, interview
transcripts and think-aloud observations. This method of analyses is used to identify
patterns or themes within data, at the minimum organising data and describing data in
rich detail (Braun and Clarke, 2006). The researchers explain that free text answers
were encoded into categories based on keywords and answers relating to similar
keywords were grouped together (Frid, Gomes and Jin, 2020). When taking a mixed
methods approach, thematic content analysis is used to convert raw qualitative data into
quantitative frequencies by counting the number of times a certain theme may occur,
making it the appropriate choice of analysis in this case study (Boyatzis, 2009).

Paper A Conclusion

With the insights gathered from 104 video creators, the authors conclude that a
prototype has been presented which is unique in the way it produces music based on a
reference song: this addresses issues of finding music for videos. They hypothesise that
their findings could be applied to other areas of research related to human-AI interaction
and hope to encourage further exploration (Frid, Gomes and Jin, 2020). The study was
well structured and followed an iterative approach, using UX data collection and
analysation methods to reach an evaluated, functional prototype ready for market.

Paper B: Approaches in Intelligent Music Production - David Moffat and Mark B.

Sandler (2019)

PAGE 4
The second analysed article was published as part of the MDPI Arts Journal in 2019.
Approaches in Intelligent Music Production aims to provide an overview of approaches
taken in Intelligent Music production (IMP) and a number of key decisions that need to
be taken into account when producing an IMP system. The authors’ discussion is based
off the theory originally presented by Russell and Norvig, that when developing an
intelligent agent, three aspects should be considered (Russell and Norvig, 2016). These
are interpreted as ‘levels of control, knowledge representation and audio manipulation’.
Moffat and Sandler use the aspects as sub-headings, utilising a large amount of
previously published research to explore the different options that can be taken when
developing an IMP system.

Methodology

In contrast the methodology used by Frid, Gomes and Jin (Article A) collected primary
sources of data and used an iterative methodology, whereas Approaches in Intelligent
Music production (Article B) collected secondary sources of qualitative data, relying
heavily on using a literature review as their methodology. According to Adler, Clark and
Adler, a literature review is ‘the process of searching for, reading, summarising and
synthesising existing work on a topic or the resulting written summary of the search’
(Adler, Clark and Adler, 2011). Most, if not all published articles use a literature review to
summarise existing research on a topic. As demonstrated in Article A, this is usually
done as part of an article’s introduction. However, (Snyder, 2019) argues that it can be
used as a methodology, explaining there are multiple types of stand-alone literature
review.

Article B is using an ‘integrative literature review’ as their approach, which aims to

assess, critique, and synthesise representative literature on a topic allowing new
theoretical frameworks and perspectives to be generated (Torraco, 2005). Although it
can be a good method when synthesising research that shows evidence on a meta-
level, there is a possibility that literature reviews can lack thoroughness due to relying on
secondary sources, meaning there is a chance that the authors could build their
research on flawed assumptions (Snyder, 2019).

Sampling and data collection

Article A was transparent about the data collection process, providing both methods and
results chronologically, helping the reader follow the course of development. Whereas
Article B did not state their methods of data collection or evaluation. As explained above,
it is clear that a literature review was conducted, thus secondary sources of information
were utilised. However, there is no clarification as to how they searched for their sources
of research. This could indicate that there may have been some unintentional sample
bias. When conducting a literature review, researchers can be selective of the evidence
that is used to build their research, in some cases, ignoring research that points the
other way (Snyder, 2019). This could lead to problems when supporting their argument

PAGE 5
because they have not necessarily taken all the potential approaches into account. One
way this could be avoided is the use of Boolean logic when searching for literature. Once
a researcher has a list of keywords, AND/OR/NOT commands are used as a means to
open up a search to a larger number of potential sources (Oliver, 2012).

Analysis

Unlike Article A’s confirmed use of thematic analysis, Article B does not state the method
used when organising collected data. According to Whittemore and Knafl, one of the
least developed aspects of integrative literature reviews is data analysis (Whittemore
and Knafl, 2005). Additional evidence suggests that aim of data analysis in an integrative
review is to ‘critically analyse and examine the literature and the main ideas and
relationships of an issue’ (Snyder, 2019). Article B does exactly this, examining each
approach in depth whilst critically analysing by providing both pros and cons of each
approach.

Paper B Conclusion

Nonetheless, the authors’ aim of providing an overview of different approaches one

could take to each of the three aspects when developing an IMP, was synthesised in a
well-constructed and easily understood text. Although there were some questions of
sample bias, they present a vast range of previously published research to back up their
argument, providing a framework for future research and development of IMP systems to
build from.

Essay Conclusion

This essay examines the similarities and differences between two articles that explore a
similar area of study, but employ different methodologies when conducting their
research. Article A gave the reader a clear and concise overview of the processes taken
when developing a UI for an AI software. The authors chose a mixed method, iterative
approach using a variety of data collection and analysis methods found in UX. This
combination of techniques led to a successful production of an evaluated prototype. On
the other hand, Article B took a qualitative approach, using a literature review as both a
data collection method and the overarching methodology. To re-state, Moffat and
Sandler’s aim was to contribute to the field of AI by providing a theoretical framework
that could be used when developing different aspects of an IMP system. However, the
reader would have benefited from the authors providing a more detailed explanation
about how the data collection and analysis methods were conducted.

New methodologies in both qualitative and quantitative approaches have been

introduced, providing a foundation of knowledge in the various types of data collection
and analyses methods that could be used to explore or answer research questions. The

PAGE 6
importance of conducting a literature review, regardless of whether it is stand-alone, has
been highlighted. This method helps give an overview of previous research and where
they may be gaps, validating the need for inquiry. Additionally, searching for two case
studies and the difficulties that were encountered to find published and peer reviewed
research, shows there is still much to be discovered in the area of AI in music
production.

PAGE 7
References

Adler, E., Clark, R. and Adler, E., 2011. An invitation to social research. Belmont, CA:
Wadsworth Cengage Learning. p.85

Blaxter, L., Hughes, C. and Tight, M., 2010. How to research. Maidenhead, England:
McGraw-Hill/Open University Press. p. 66, p.211

Boyatzis, R., 2009. Transforming qualitative information. Thousand Oaks: Sage.

Braun, V. and Clarke, V., 2006. Using thematic analysis in psychology. Qualitative
Research in Psychology, 3(2), pp.77-101.

Cook, R. and Sutton, R., 2014. Administrators’ Assessments of Online Courses and
Student Retention in Higher Education. Handbook of Research on Transnational Higher
Education, pp.138-150.

Creswell, J., 2007. Qualitative inquiry and research design. Thousand Oaks: Sage.
p.103

Creswell, J. and Creswell, J., 2017. Research design. 5th ed. Sage Publications. p.3

Dow, S., Glassco, A., Kass, J., Schwarz, M., Schwartz, D. and Klemmer, S., 2010.
Parallel prototyping leads to better design results, more divergence, and increased self-
efficacy. ACM Transactions on Computer-Human Interaction, 17(4), pp.1-24.

Frid, E., Gomes, C. and Jin, Z., 2020. Music Creation by Example. Proceedings of the
2020 CHI Conference on Human Factors in Computing Systems.

Goodman, E., Kuniavsky, M. and Moed, A., 2012. Observing the User Experience, 2nd
Edition. p.3, p.273, p.339

Howell, K., 2013. An introduction to the philosophy of methodology. London: SAGE.

Kothari, C., 2004. Research methodology. New Delhi: New Age International (P) Ltd.,
Publishers. p.7

Leavy, P., 2017. Research design. New York, NY: The Guilford Press. p.124, p.164

PAGE 8
Makewa, L., 2015. Curriculum Development and Scientific Research. Handbook of
Research on Enhancing Teacher Education with Advanced Instructional Technologies,
pp.208-227.

Moffat, D. and Sandler, M., 2019. Approaches in Intelligent Music Production. Arts, 8(4),
p.125.

Nielsen, J., 1993. Iterative user-interface design. Computer, 26(11), pp.32-41.

Nielsen, J., 2012. Thinking Aloud: The #1 Usability Tool. [online] Nielsen Norman Group.
Available at: <https://www.nngroup.com/articles/thinking-aloud-the-1-usability-tool/>
[Accessed 20 April 2021].

Oliver, P., 2012. Succeeding with your Literature Review. Maidenhead: Open University
Press. p.42

Palinkas, L., Horwitz, S., Green, C., Wisdom, J., Duan, N. and Hoagwood, K., 2013.
Purposeful Sampling for Qualitative Data Collection and Analysis in Mixed Method
Implementation Research. Administration and Policy in Mental Health and Mental Health
Services Research, 42(5), pp.533-544.

Russell, S. and Norvig, P., 2016. Artifical Intelligence: A Modern Approach. 3rd ed.
Pearson.

Snyder, H., 2019. Literature review as a research methodology: An overview and

guidelines. Journal of Business Research, 104, pp.333-339.

Tashakkori, A. and Creswell, J., 2007. Editorial: The New Era of Mixed Methods. Journal
of Mixed Methods Research, 1(1), pp.3-7.

Torraco, R., 2005. Writing Integrative Literature Reviews: Guidelines and Examples.
Human Resource Development Review, 4(3), pp.356-367.

Watson, R., 2015. Quantitative research. Nursing Standard, 29(31), pp.44-48.

Whittemore, R. and Knafl, K., 2005. The integrative review: updated methodology.
Journal of Advanced Nursing, 52(5), pp.546-553.

PAGE 9