United States Patent
‘USOLIG17952B1
US 11,617,952 BI
a2) (10) Patent No.z
Sheng et al. (4s) Date of Patent Apr. 4, 2023
(54) EMOTION BASED MUSIC STYLE CHANGE 22019 Mas ons
USING DEEP LEARNING 300 Braeep ean
31 Tea os Soest
(71) Applicane: Eletronte Arts Ine, Redwood City, 32a Rhea Goon abt
AUS) 0a Bae fos a735
W3be%21 B33032 Guan NSE Ta st
(72) Inventors: Jie Sheng. Sumyvale, CA (US); Hulya Medea B2* $2022 Ennis ‘AF 1308
Duyat Visit, Una, IL (US), Chong (Continved)
‘hwo, Speing TX US)
OTHER PUBLICATIONS
(73) Assiance: Eletronie Art Ine, Redvood Ci aa —
ey Se tl Mii Noting Dyn ningun
ftom of Mie With Apptications to Sle Taer” 1b lnerna
: : - theterm ofthis #8! Sei fr Muse Infomation Reel Coferenee, Pais
(7) Novice: Subject io any dislimer the term of his Hel Sty M
Patent is extended or adjusted under 35 3
USC. 154(b) by 0 days. (Continued)
Primary Examiner — William H McCulloch, J
ee eed (74) Auorne, Agen, oF Firm —MeDermct Will &
(22) Fils Apr 13, 2021 Emery LLP
Gmc on ABSTRACT
AGE 1384 201401) Various aspects of the subject technology reat o systems,
AOE 1967 (201401) mothods, and machine-readable media for changing ic
AO3P 1340 (201401) ‘off video game basa on a player's emotion. The method
(2) USC includes reeiving indicators of emotion comprising in-
ce AGE 184 (201409) ABBE 13140 gape atte ofa layerina Video game. The tod so
(2014.09), A63F 14767 2014.09) inches predicting an emotion ofthe player based on the
(58). Feld of Classification Search icator of emotion from the video game. The metho also
c "AGS 13167, AGIF 13/40, AGIF 13/54 inchudes receiving original music from the video game, The
(66)
‘Se application file for complete search history
References Cited
US, PATENT DOCUMENTS,
. sates
8076565 BLY 122011 Bonet nani
.06ks57 BI* 92018 aged uot 10
method albo includes determining an original tone of the
‘original musie, The meshod also includes determining
transformed tone based atleast in part onthe emotion ofthe
player tht was predicted. The method also inchades tans-
forming the original tone of the original music 1 the
transformed tone. The method also includes generating
transformed music from the eriginal music based om the
transformed tone
“
ims, § Drawing Sheets
1 abba
| eafate
Spam al!
SN 2520)
Jo precise
Pra
Sect
Tt otUS 11,617,952 BL
Pape 2
(66) References Cited
US, PATENT DOCUMENTS,
2009082104 AL* 32009 Hepsrom AGELBI0
ra
200910082105 A1* 72000 Hepsrom ASF NID
“8
200900168632 AL* 72000 Rigopeles ..- GIOH 1008
20170357806 ALF 122017 Tein on Ht
downass ALS "S018 Mare ‘aso 160879
igure ALS Sm Me GieN S18
Soauonsspan ALS "SO03t Huang Goo 20900
SnOIOISRH ALY 39022. Kushnan Gounod
20220139376 ALY $2022. Bucs ‘GIOL LS08
“425
(OTHER PUBLICATIONS
Mali, fo al, “Noun! Trnlation of Maal Style” reeved
fiom arXiv1708038351 [es SD] Au. 11, 2017: 9 pas
* cin by examinerUS 11,617,952 BL
Sheet 1 of 5
Apr. 4, 2023
U.S. Patent
ISN POWUCJSUeL| isnyy jeuibuCQ,
IP Ald
Gzh a
saqnquny ewed-ul Jakejg~US 11,617,952 BL
Sheet 2 of 5
Apr. 4, 2023
U.S. Patent
zOld
0sz~, ove~ Ore 09%
yndjng Jepovaq - WwaieT é
th 002
ate 8
avr
Feapen mL GS) - LS
aege 7 r= Le ere 7s
erg!
wuopipalg i +
tions, and means for executing the stored instractions that
‘when executed by the means, cause the means to perform a
Iethod for changing music of a video game based on &
player's emotion. The meth includes receiving indicators
‘of emotion comprising ingame attributes of a player in a
ideo game. The method also includes prescting an emotion
‘of the player based on the indicators of emotion from the
ideo game. The method also includes receiving original
music from the video game. The method also includes
ddeermining an orignal tone of the exginal musie. The
method also ineiudes determining a transformed tone based
ft east in parton the emotion of the player that as
predicted, The method also includes transforming the origi-
hal tone of the original music to the transformed tone. The
method also includes generating tanslormes music from the
‘original musi hase onthe transformed tone
BRIEF DESCRIPTION OP THE SEVERAL,
VIEWS OF THE DRAWINGS,
‘Toss identify the discussion of any particular element
‘or act, the most significant digit or digits in a reference
humber refer tothe figure numer i which tha element is
first introduce.
FIG. 1 illasrtes am exemplary system for changing
imsie ofa video game based on payer's emotion, accond-
ing to certain aspects ofthe present diselosure.
IG. 2 usistes an exentplry’ variational autoencader
for changing musie of video game based on a player's
‘emotion, according to certain aspects of the preseat disco
TG, 3 illsates system configured for changing music
‘of video game based ona player's enotion in accordance
‘with one or more implementations.
IG. illustrates a example fw diagram for changing
muse ofa video game hosed on a player's emotion, accond-
ing to certain aspects of the prevent dislosure,
TIG. 8 isa block diagram illuseating an example com-
pute system (e,representing both client ax server wit
Which aspects of the subject technology can be imple-
mented
Tn one or more implementations, not all ofthe depicted
‘components in esch figure may be required and one or more
implementations may inelade additonal components not
shown ina figure, Variations inthe arsingement and ype ofUS 11,617,952 BI
3
the componcats may be made without departing from the
scope af the subject disclosure. Additional components