CAPSTONE PROJECT (PART I) REPORT

(Project Term August-December, 2012)

EMBEDDING SOUND EFFECTS IN MATLAB WITH THE USE OF NEURAL NETWORKS

Submitted by Name of Student1: SWATI MISHRA Registration Number: 10906494 Name of Student2: PARAS BISHT Registration Number: 10904454 Name of Student3: PALLAVI Registration Number: 10903538 Name of Student4: NEEMANI GUPTA Registration Number: 10903879 Project Group Number: KG048
Under the Guidance of Miss MADHU BALA, Lecturer

Discipline of Computer Science and Information Technology Lovely Professional University, Phagwara August to December, 2012

DECLARATION
We hereby declare that the project work entitled Embedding Sound Effects inMatlab is an authentic record of our own work carried out as requirements of Capstone Project (Part-I) for the award of degree of B.Tech in Computer Science(CSE) from Lovely Professional University, Phagwara, under the guidance of Madhu Bala, during August to December, 2012).

Name : Madhu U.ID : 13878 Designation Lecturer Signature of Faculty Mentor

Name of Student 1: SWATI MISHRA Registration Number: 10906494

Name of Student 2: PARAS BISHT Registration Number: 10904454

Name of Student 3: PALLAVI Registration Number: 10903538

Name of Student 4: NEEMANI GUPTA Registration Number: 10903879

CERTIFICATE
This is to certify that the declaration statement made by this group of students is correct to the best of my knowledge and belief. The Capstone Project Proposal based on the technology / tool learnt is fit for the submission and partial fulfillment of the conditions for the award of B.Tech in Computer Science (CSE) from Lovely Professional University, Phagwara . Name : Madhu. U.ID : 13878 Designation : Lectirer

Signature of Faculty Mentor

INDEX Profile of the Problem

Introduction Literature Review 1 4

Technique used Use Of Neural Network Functions of neural network Objective of Study Design System Flow Chart Complete Work Done Experimental Work Done Interface Expected Outcome Of Project Conclusion References

8 9 12

13 14 15 17 22 24 25

PROFILE OF THE PROBLEM

INTRODUCTION
Our project topic consists of adding sound of different emotions using matlab. Firstly we have studied about the different sound frequency of human at different emotions such as in sadness, happiness, anger etc. Communication is an important capability, not only in linguistic terms but also in emotional term. . When the recognition is based solely on voice, which is a fundamental mode of human communication. This report is organized as follows: In this section , we focus on the emotional state and the emotional group in order to explain the rationale of classification from the viewpoint of psychology. We also propose a new feature for emotion recognition called the frequency range of meaningful signal (FRMS). we compare the proposed feature with other existing features, such as energy and pitch.

Emotional state
In 2000, Hiroyasu Miwa defined the emotional space as having three levels: the activation level, the pleasant level, and the certainty level. Using the same approach, we can map emotions in the emotional space having infinite dimensions and infinite indices. Psychology of emotion is can be categorized in terms of physiological, behavioral and cognitive psychology. Each theory explains how humans feel and recognize emotions. According to these theories, humans recognize their own emotions by changes in physiology or behavior.

Fig-1- Various emotions on pleasant-activity scale 1

Although the same changes may occur, humans experience other emotions that stem from other cognitions or thoughts for given a situation. By the same approach, infinite dimensions of emotion can be categorized into three groups, and we call each component of these categories an emotional state. Behavioral and cognitive indices are not, however, recognized by using voice or facial expressions. Hence, the emotional space for emotion recognition has only physiological indices. Of the physiological indices, we propose the activity and the pleasant as the emotional states for emotion recognition. In Hiroyasu Miwas model of emotional space, we removed a certain index because it is in the cognition category thereby necessitating artificial intelligence for recognition emotion. With this model of emotional space, an infinite variety of dimensions can be compressed into two dimensions. In addition, an infinite numbers of emotions can be classified into four emotion groups namely joy, sadness, anger, and neutrality. For example, Hiroyasu Miwa defines six primary emotions into groups of joy (happiness), sadness (disgust, sadness), anger (anger, fear), and neutrality (neutrality). Hence, we dont recognize each primary emotion but rather each group of emotions that has the same emotional state.

Fig-2- Various emotions on frequency scale

FREQUENCY RANGE OF MEANINGFUL SIGNAL

What is the frequency range of meaningful signal? In general, human speech has a long frequency range. However, the important frequency range or meaningful frequency range is from 100 Hz to 5000 Hz . We have Original speech signal for four emotions.

Fig-3: Orthogonal speech signal for neutrality

Fig-4: Orthogonal speech signal for joy

Fig-5: Orthogonal speech signal for saddness

Fig-6: Orthogonal speech signal for anger

LITERATURE REVIEW
MATLAB MATLAB (matrix laboratory) is a numerical computing environment and fourth-generation programming language. Developed by MathWorks, MATLAB allows matrix manipulations, plotting of functions and data, implementation of algorithms, creation of user interfaces, and interfacing with programs written in other languages, including C, C++, Java, and Fortran. Although MATLAB is intended primarily for numerical computing, an optional toolbox uses the MuPAD symbolic engine, allowing access to symbolic computing capabilities. An additional package, Simulink, adds graphical multi-domain simulation and Model-Based Design for dynamic and embedded systems. In 2004, MATLAB had around one million users across industry and academia. MATLAB users come from various backgrounds of engineering, science, and economics. MATLAB is widely used in academic and research institutions as well as industrial enterprises.

Syntax
The MATLAB application is built around the MATLAB language, and most use of MATLAB involves typing MATLAB code into the Command Window (as an interactive mathematical shell), or executing text files containing MATLAB code and functions.

FREQUENCY OF SOUND OF MALE AND FEMALE AT DIFFERENT EMOTIONS

Table-1: Frequency of men
RECOG NEURALITY JOY 4.8 80.5 8.0 20.8 79.3% SADNESS ANGER 10.5 5.1 80.6 0.2 4.6 13.1 3.1 76.4

NEUTRALITY 80.2 JOY SADNESS ANGER OVERALL 1.7 8.4 2.7

Table-2: Frequency of women RECOG NEUTRALITY JOY 5.9 61.0 5.9 19.6 72% SADNESS 21.1 4.8 84.4 3.4 ANGER 4.8 25.9 0.8 73.8

NEUTRALITY 69.1 JOY SADNESS ANGER OVERALL 8.1 8.8 3.4

Recognizing Two Emotional States:

Table-3: Output/input of hot anger and neutral Output Input Hot anger Neutral Hot anger 128 8 Neutral 12 72

In the experiment, we attempted to distinguish two emotional types. We used all 62 features and a three-layer neural network with 20 nodes in the hidden layer to distinguish hot anger from neutral, which is considered as the easiest classification task. The testing result is shown in Table 2. 140 utterances labelled as hot anger and 80utterances labelled as neutral were tested. 128 hot anger utterances and 72 neutral utterances are classified correctly.. From the results of each test (Figure 2), we can see the classification performance is stable, and the average accuracy is 90.91%. In comparison to neutral speech, anger is produced with a lower pitch, higher intensity, more energy (500 Hz) across the vocalization, higher first formant (first sound produced) and faster attack times at voice onset (the start of speech). "Hot anger", in contrast, is produced with a higher, more varied pitch, and even greater energy (2000 Hz)

Fig-7: Graph of hot anger and neutral emotion

Aside from the recognition rate, the key advantages of the FRMS are that it is not dependent on the magnitude of the speech signal and it is robust in noisy environments. In practice, when the magnitude of a speech signal changes due to the magnitude of a speakers voice, the distance between the speaker and the microphone or the characteristics of microphone, the recognition performance deteriorates. Hence, compensation for this deterioration is a significant study theme. However, the FRMS is concerned only with the envelope of the signal and not the magnitude. Further, the FRMS focuses on the relationship between the original speech signal and the lowpass filtered speech signal. As such, the FRMS is independent of the distance or the magnitude of the voice, thereby making it powerful in practical use. To verify this independence, we performed recognition experiments using the same training data as the original magnitude but with half and double magnitude test data. As anticipated, recognition experiments of original, half and double magnitude speech data yielded almost the same results, as shown in Fig. 6.Furthermore, the results confirm that the FRMS feature has the advantage of being able to be used in practice without any pre-processing for distance compensation. Another advantage of the FRMS feature is its robustness in noisy environments. Because most noises have a frequency higher than the cut-off frequency, high frequency noises tend to disappear after low-pass filtering. In addition, if noises are weaker than the main voice, then most of these noises will not affect the envelope of the main voice. Hence, the FRMS can be robust in a noisy environment. To verify this robustness, we performed recognition experiments on voice noise and white noise speech data using the same training data as previously used. The voice noise data consisted of the main voice and two other set of emotional voice data from a different speaker with approximately half of the magnitude. The white noise
6

data consisted of the main voice and white noise at about a quarter of the magnitude of the main voice. To compare the advantages of the FRMS feature withother features (energy and pitch), and to verify our half anddouble magnitude data, as well as the voice noise data. Before starting the project we first collected all information regarding the frequencies of different voices. First we studied different sound functions in Android and Windows.

ANDROID
Android provides two main API's for playing sounds. The first one through the SoundPool class and the other one through the MediaPlayer class. SoundPool can be used for small audio clips. It can repeat sounds and play several sounds simultaneously. The sound files played with SoundPool should not exceed 1 MB. SoundPool does load the file asynchronously. As of Android API8 it is possible to check if the loading is complete via a OnLoadCompleteListener. Android supports different audio streams for different purposes. The phone volume button can be configured to control a specific audio stream, e.g. during a call the volume button allow increase decrease the caller volume. To set the button to control the sound media stream set the audio type in your application. The android.media.MediaRecorder class can be used to record audio and video. To use MediaRecorder you need to set the source device and the format.

WINDOWS
Following a trend set by Apple and Google, Microsoft has turned its formerly-device-gated app store into something you can check out from any web browser (and link to from publications that write about apps). The web welcomes a new app store today: Microsoft Windows Phone Marketplace. Scoff if you must, but until youve tried the Windows Phone 7, with all due respect, you might not know what youre talking about. Its a beautifully-designed, responsive platform, and it could make a real dent now that Nokias on board. None other than the head of Verizon thought Windows Phone 7 would beat Blackberry to become the third ecosystem behind iOS and Android, according to an erroneous InformationWeek article, since updated, which was believable enough to have been picked up by several other publications. Hey, it could happen. Again have you used it? Windows Phone Marketplace offers over 30,000 free and paid apps, as noted by Wired.com, in 16 categories with all the usual stuff: icon, price, a rating, description, screenshots and user reviews. Weve already checked it out on the Windows Zune software and on a Windows Phone 7. Now that its finally available on the web, we decided to do a little legwork to figure out where things stand music-wise on this, day one of the web-based Windows Phone Marketplace.
7

USE OF NEURAL NETWORKS IN MATLAB

The neural networks is a way to model any input to output relations based on some input output data when nothing is known about the model. This example shows you a very simple example and its modelling through neural network using MATLAB. Neural Network Toolbox provides functions and apps for modeling complex nonlinear systems that are not easily modeled with a closed-form equation. Neural Network Toolbox supports supervised learning with feedforward, radial basis, and dynamic networks. It also supports unsupervised learning with self-organizing maps and competitive layers. With the toolbox you can design, train, visualize, and simulate neural networks. You can use Neural Network Toolbox for applications such as data fitting, pattern recognition, clustering, time-series prediction, and dynamic system modeling and control. To speed up training and handle large data sets, you can distribute computations and data across multicore processors, GPUs, and computer clusters using Parallel Computing Toolbox. Characteristics of Neural Networks They exhibit some brain-like behaviors that are difficult to program directly like: Learning association Categorization generalization feature extraction optimization noise immunity. There is a wide range of neural network architectures: Multi-Layer Perceptron (Back-Prop Nets) 1974-85 Neocognitron 1978-84 Adaptive Resonance Theory (ART) 1976-86 Sef-Organizing Map 1982 Hopfield 1982 Bi-directional Associative Memory 1985 Boltzmann/Cauchy Machine 1985 Counterpropagation 1986 Radial Basis Function 1988 Probabilistic Neural Network 1988 General Regression Neural Network 1991 Support Vector Machine 1995

NEURAL NETWORKS FUNCTION

Graphical interface functions

nctool- Neural network classification or clustering tool. It opens the neural network
clustering GUI.

nftool- Neural network fitting tool. It opens the neural network fitting tool GUI. nntool- Open Data/Network Manager. It opens the network/data manager window,
which allows us to import, create, use and export neural network and data.

Cascadeforwardnet-

cascadeforwardnet(hiddenSizes,trainFcn)

Description
Cascade-forward networks are similar to feed-forward networks, but include a connection from the input and every previous layer to following layers. As with feed-forward networks, a two-or more layer cascade-network can learn any finite inputoutput relationship arbitrarily well given enough hidden neurons. cascadeforwardnet(hiddenSizes,trainFcn) takes these arguments, hiddenSizes Row vector of one or more hidden layer sizes (default = 10) trainFcn Training function (default = 'trainlm')

and returns a new cascade-forward neural network.

elmannetElman neural network

Syntax
elmannet(layerdelays,hiddenSizes,trainFcn)

Description
Elman networks are feedforward networks (feedforwardnet) with the addition of layer recurrent connections with tap delays. With the availability of full dynamic derivative calculations (fpderiv and bttderiv), the Elman network is no longer recommended except for historical and research purposes. For more accurate learning try time delay (timedelaynet), layer recurrent (layrecnet), NARX (narxnet), and NAR (narnet) neural networks. Elman networks with one or more hidden layers can learn any dynamic input-output relationship arbitrarily well, given enough neurons in the hidden layers. However, Elman networks use simplified derivative calculations (using staticderiv, which ignores delayed connections) at the expense of less reliable learning. elmannet(layerdelays,hiddenSizes,trainFcn) takes these arguments, Layerdelays Row vector of increasing 0 or positive delays (default = 1:2) HiddenSizes Row vector of one or more hidden layer sizes (default = 10) TrainFcn Training function (default = 'trainlm')

and returns an Elman neural network. feedforwardnet

Feedforward neural network

Syntax
feedforwardnet(hiddenSizes,trainFcn)

Description
Feedforward networks consist of a series of layers. The first layer has a connection from the network input. Each subsequent layer has a connection from the previous layer. The final layer produces the network's output. Feedforward networks can be used for any kind of input to output mapping. A feedforward network with one hidden layer and enough neurons in the hidden layers, can fit any finite inputoutput mapping problem.
10

Specialized versions of the feedforward network include fitting (fitnet) and pattern recognition (patternnet) networks. A variation on the feedforward network is the cascade forward network (cascadeforwardnet) which has additional connections from the input to every layer, and from each layer to all following layers.
feedforwardnet(hiddenSizes,trainFcn)

takes these arguments,

hiddenSizes Row vector of one or more hidden layer sizes (default = 10) trainFcn Training function (default = 'trainlm')

and returns a feedforward neural network.

catelements
Concatenate neural network data elements Syntax catelements(x1,x2,...,xn) [x1; x2; ... xn] Description catelements(x1,x2,...,xn) takes any number of neural network data values, and merges them along the element dimension (i.e., the matrix row dimension). If all arguments are matrices, this operation is the same as [x1; x2; ... xn]. If any argument is a cell array, then all non-cell array arguments are enclosed in cell arrays, and then the matrices in the same positions in each argument are concatenated. Syntax view(net) Description view(net) launches a window that shows your neural network (specified in net) as a graphical diagram.
11

view- View neural network

OBJECTIVE OF STUDY
Neural networks better than computer for processing of sensorial data such as signal processing, image processing, pattern recognition, robot control, non-linear modeling and prediction Survey of attractive applications of artificial neural networks. Practical approach for using artificial neural networks in various technical, organizational and economic applications Analogy with biologic neural networks is too weak to convince engineers and computer scientists about correctness. Correctness follows from mathematical analysis of non-linear functions or dynamical systems and computer simulations. Neural networks are realistic alternatives for information problems (in stead of tedious software development) Not magic, but design is based on solid mathematical methods Neural networks are interesting whenever examples are abundant, and the problem cannot be captured in simple rules. Superior for cognitive tasks and processing of sensorial data such as vision, image- and speech recognition, control, robotics, expert systems. Correct operation biologic analogy not convincing but mathematical analysis and computer simulations needed. Technical neural networks ridiculously small w.r.t. brains good suggestions from biology Fascinating developments with NN possible : specificities of the user voice-controlled apparatus, and pen-based computing. To add sound in window or linux is not advanced as everybody knew it to do, so we thought of doing something new which can be beneficial to all and learn a new thing. To also learn different sound frequencies range of different emotions of human being to be heard using Matlab.

The purpose of this work is to study the emotion recognition method and its performance. Based on this study, we plan to develop an automatic emotion recognizer, which can help people who have difficulties in understanding and identifying emotions to improve their social and interaction skills Such an assistive emotion recognition tool might help people with autism to study and practice social interactions.

12

DESIGN SYSTEM FLOW CHART

MATLAB

NEURAL NETWORK

FUNCTIONS OF GUI

nprtool nctool nftool

nmtool

view

EMOTIONS

SAD

ANGER

HAPPINESS

CRYING

13

COMPLETE WORKPLAN WITH TIMELINE

SNO 1.

MONTH

WORKPLAN

TIMELINE 20th Sept to 30th Sept 2012

SEPTEMBER We start our Capstone project from September 2012. In September month we studied about introduction to Mat-lab and introduction to Neural Network. OCTOBER

2.

In October month we 1st Oct to 30th Oct 2012 studied about Frequencies range of different emotions of sound made by human being and function of Neural Network being used. 1st Nov to 8th Nov and 15th Nov to 20th Nov 2012 7th Jan to 31st Jan 1st Feb to 28th Feb 1st March to 31st March 1st April to 30thApril

3.

NOVEMBER Coding of interface to be made in our project.

4. 5.

JANUARY FEBUARY

6.

MARCH

7.

APRIL

Implementing the neural functions in the interface Designing the code to attach interface and functions Designing the code to attach interface and functions Revising the code and finalizing the same

14

EXPERIMENTALWORKDONE
Variables
Variables are defined using the assignment operator, =. MATLAB is a weakly typed programming language. It is a weekly typed language because types are implicitly converted. It is a dynamically typed language because variables can be assigned without declaring their type, except if they are to be treated as symbolic objects, and that their type can change. Values can come from constants, from computation involving values of other variables, or from the output of a function. For example:
>> x = 17 x= 17 >> x = 'hat' x= hat >> y = x + 0 y= 104 97 116 >> x = [3*4, pi/2] x= 12.0000 1.5708 >> y = 3*sin(x) y= -1.6097 3.0000

Vectors/matrices
As suggested by its name (a contraction of "Matrix Laboratory"), MATLAB can create and manipulate arrays of 1 (vectors), 2 (matrices), or more dimensions. In the MATLAB vernacular, a vector refers to a one dimensional (1N or N1) matrix, commonly referred to as an array in other programming languages. A matrix generally refers to a 2-dimensional array, i.e. an mn array where m and n are greater than 1. Arrays with more than two dimensions are referred to as multidimensional arrays. Arrays are a fundamental type and many standard functions natively support array operations allowing work on arrays without explicit loops. A simple array is defined using the syntax: init:increment:terminator. For instance: >> array = 1:2:9 array = 13579 defines a variable named array (or assigns a new value to an existing variable with the name array) which is an array consisting of the values 1, 3, 5, 7, and 9. That is, the array starts at 1 (the init value), increments with each step from the previous value by 2 (the increment value), and stops once it reaches (or to avoid exceeding) 9 (the terminator value).

15

>> array = 1:3:9 array = 147 the increment value can actually be left out of this syntax (along with one of the colons), to use a default value of 1. >> ari = 1:5 ari = 12345 assigns to the variable named ari an array with the values 1, 2, 3, 4, and 5, since the default value of 1 is used as the incremented. Indexing is one-based, which is the usual convention for matrices in mathematics, although not for some programming languages such as C, C++, and Java. Matrices can be defined by separating the elements of a row with blank space or comma and using a semicolon to terminate each row. The list of elements should be surrounded by square brackets: []. Parentheses: () are used to access elements and sub arrays (they are also used to denote a function argument list). >> A = [16 3 2 13; 5 10 11 8; 9 6 7 12; 4 15 14 1] A= 16 3 2 13 5 10 11 8 9 6 7 12 4 15 14 1 >> A(2,3) ans = 11 Sets of indices can be specified by expressions such as "2:4", which evaluates to [2, 3, 4]. For example, a submatrix taken from rows 2 through 4 and columns 3 through 4 can be written as: >> A(2:4,3:4) ans = 11 8 7 12 14 1 A square identity matrix of size n can be generated using the function eye, and matrices of any size with zeros or ones can be generated with the functions zeros and ones, respectively. >> eye(3)
16

ans = 100 010 001 >> zeros(2,3) ans = 000 000 >> ones(2,3) ans = 111 111

PREPARING FOR INTERFACE OF DIFFERENT SOUND FREQUENCY ALONG WITH EMOTIONS

FREQUENCY RANGE OF MEANINGFUL SIGNAL A. What is the frequency range of meaningful signal? In general, human speech has a long frequency range. However, the important frequency range or meaningful frequency range is from 100 Hz to 5000 Hz . We have GRAPH for different sound frequency

Fig-8: Frequency of meaningful signal

17

INTERFACE OF PROJECT

18

Code
function varargout = cap(varargin) % CAP MATLAB code for cap.fig % CAP, by itself, creates a new CAP or raises the existing % singleton*. % % H = CAP returns the handle to a new CAP or the handle to % the existing singleton*. % % CAP('CALLBACK',hObject,eventData,handles,...) calls the local % function named CALLBACK in CAP.M with the given input arguments. % % CAP('Property','Value',...) creates a new CAP or raises the % existing singleton*. Starting from the left, property value pairs are % applied to the GUI before cap_OpeningFcn gets called. An % unrecognized property name or invalid value makes property application % stop. All inputs are passed to cap_OpeningFcn via varargin. % % *See GUI Options on GUIDE's Tools menu. Choose "GUI allows only one % instance to run (singleton)". % % See also: GUIDE, GUIDATA, GUIHANDLES % Edit the above text to modify the response to help cap % Last Modified by GUIDE v2.5 20-Nov-2012 10:42:52 % Begin initialization code - DO NOT EDIT gui_Singleton = 1; gui_State = struct('gui_Name', mfilename, ... 'gui_Singleton', gui_Singleton, ... 'gui_OpeningFcn', @cap_OpeningFcn, ... 'gui_OutputFcn', @cap_OutputFcn, ... 'gui_LayoutFcn', [] , ... 'gui_Callback', []); if nargin && ischar(varargin{1}) gui_State.gui_Callback = str2func(varargin{1}); end if nargout [varargout{1:nargout}] = gui_mainfcn(gui_State, varargin{:}); else gui_mainfcn(gui_State, varargin{:}); end % End initialization code - DO NOT EDIT
19

% --- Executes just before cap is made visible. function cap_OpeningFcn(hObject, eventdata, handles, varargin) % This function has no output args, see OutputFcn. % hObject handle to figure % eventdata reserved - to be defined in a future version of MATLAB % handles structure with handles and user data (see GUIDATA) % varargin command line arguments to cap (see VARARGIN) % Choose default command line output for cap handles.output = hObject; % Update handles structure guidata(hObject, handles); % UIWAIT makes cap wait for user response (see UIRESUME) % uiwait(handles.figure1);

% --- Outputs from this function are returned to the command line. function varargout = cap_OutputFcn(hObject, eventdata, handles) % varargout cell array for returning output args (see VARARGOUT); % hObject handle to figure % eventdata reserved - to be defined in a future version of MATLAB % handles structure with handles and user data (see GUIDATA) % Get default command line output from handles structure varargout{1} = handles.output;

% --- Executes on button press in pushbutton1. function pushbutton1_Callback(hObject, eventdata, handles) % hObject handle to pushbutton1 (see GCBO) % eventdata reserved - to be defined in a future version of MATLAB % handles structure with handles and user data (see GUIDATA)

% --- Executes on button press in pushbutton2. function pushbutton2_Callback(hObject, eventdata, handles) % hObject handle to pushbutton2 (see GCBO) % eventdata reserved - to be defined in a future version of MATLAB % handles structure with handles and user data (see GUIDATA)

% --- Executes on button press in pushbutton3.

20

function pushbutton3_Callback(hObject, eventdata, handles) % hObject handle to pushbutton3 (see GCBO) % eventdata reserved - to be defined in a future version of MATLAB % handles structure with handles and user data (see GUIDATA)

% --- Executes on button press in pushbutton4. function pushbutton4_Callback(hObject, eventdata, handles) % hObject handle to pushbutton4 (see GCBO) % eventdata reserved - to be defined in a future version of MATLAB % handles structure with handles and user data (see GUIDATA)

% --- Executes on button press in pushbutton5. function pushbutton5_Callback(hObject, eventdata, handles) % hObject handle to pushbutton5 (see GCBO) % eventdata reserved - to be defined in a future version of MATLAB % handles structure with handles and user data (see GUIDATA)

% --- Executes on button press in pushbutton6. function pushbutton6_Callback(hObject, eventdata, handles) % hObject handle to pushbutton6 (see GCBO) % eventdata reserved - to be defined in a future version of MATLAB % handles structure with handles and user data (see GUIDATA) % --- Executes on button press in pushbutton7. function pushbutton7_Callback(hObject, eventdata, handles) % hObject handle to pushbutton7 (see GCBO) % eventdata reserved - to be defined in a future version of MATLAB % handles structure with handles and user data (see GUIDATA) % --- Executes on button press in pushbutton9. function pushbutton9_Callback(hObject, eventdata, handles) % hObject handle to pushbutton9 (see GCBO) % eventdata reserved - to be defined in a future version of MATLAB % handles structure with handles and user data (see GUIDATA)

21

EXPECTED OUTCOMES OF STUDY

This project mainly aims at adding sound effects in Matlab software. So the expected outcome is that whenever we use Matlab Software we are assisted with sound while doing our work. The sound parameters are measured in nanometers and micron particles. Sound is perhaps the most sensuous element of multimedia. It is meaningful speech in any language, from a whisper to a scream. It can provide the listening pleasure of music, the startling accent of special effects, or the ambience of a mood-setting background. Some feel-good music powerfully fills the heart, generating emotions of love or otherwise elevating listeners closer to heaven. How you use the power of sound can make the difference between an ordinary multimedia presentation and a professionally spectacular one. Misuse of sound, however, can wreck your project. Try testing all 56 of your ringtones on a crowded bus: your fellow passengers will soon wreck your day. When something vibrates in the air by moving back and forth (such as the cone of a loudspeaker), it creates waves of pressure. These waves spread like the ripples from a pebble tossed into a still pool, and when they reach your eardrums, you experience the changes of pressure, or vibrations, as sound. In air, the ripples propagate at about 750 miles per hour, or Mach 1 at sea level. Sound waves vary in sound pressure level (amplitude) and in frequency or pitch. Many sound waves mixed together form an audio sea of symphonic music, speech, or just plain noise. Acoustics is the branch of physics that studies sound. Sound pressure levels (loudness or volume) are measured in decibels (dB); a decibel measurement is actually the ratio between a chosen reference point on a logarithmic scale and the level that is actually experienced. When you quadruple the sound output power, there is only a 6 dB increase; when you make the sound 100 times more intense, the increase in dB is not hundredfold, but only 20 dB. A logarithmic scale (seen below) makes sense because humans perceive sound pressure levels over an extraordinarily broad dynamic range. Sound is energy, just like the waves breaking on a sandy beach, and too much volume can permanently damage the delicate receiving mechanisms behind your eardrums, typically dulling your hearing in the 6 kHz range. In terms of volume, what you hear subjectively is not what you hear objectively. The perception of loudness is dependent upon the frequency or pitch of the sound: at low frequencies, more power is required to deliver the same perceived loudness as for a sound at the middle or higher frequency ranges. You may feel the sound more than hear it. For instance, when the ambient noise level is above 90 dB in the workplace, people are likely to make increased numbers of errors in susceptible tasksespecially when there is a highfrequency component to the noise. When the level is above 80 dB, it is quite impossible to use a telephone. Experiments by researchers in residential areas have shown that a sound generator at 45 dB produces no reaction from neighbors; at 45 to 55 dB, sporadic complaints; at 50 to 60 dB, widespread complaints; at 55 to 65 dB, threats of community action; and at more than 65 dB, vigorous community action, possibly more aggressive than when you tested your ringtones on the bus. This neighborhood research from the 1950s continues to provide helpful guidelines for practicing rock musicians and multimedia developers today. Human hearing is less able to identify the location from which lower frequencies are generated. In surround sound systems,

22

subwoofers can be placed wherever their energy is most efficiently radiated (often in a corner), but midrange speakers should be carefully placed. Impulse response components corresponding to a direct sound, an initially reflected sound, and a reverberation are separated from an original impulse response. A point N1 of which an impulse response component corresponding to a direct sound starts, a point N2 of which impulse response components corresponding to a direct sound and an initially reflected sound end, and a point N3 of which an impulse response component corresponding to a reverberation starts are represented by data associated with impulse response components. After the levels of the impulse response components corresponding to the direct sound and the initially reflected sound are adjusted, these impulse response components are combined with the impulse response corresponding to the reverberation. By performing a convolution calculation process for audio data and combined impulse response, a reverberation is generated and added to an original sound. The present invention relates to a sound effect adding apparatus such as may, for example, be used with a reverberation adding apparatus that adds a reverberation to an original audio signal. As an apparatus that adds a sound effect to an audio signal, a reverberator is known. The reverberator is used to add reverberation to an audio signal in for example a recording studio so that listeners can have a spatial impression and a deep impression. When reverberation is added to an audio signal that has been recorded in a studio or the like, a sound effect performed in a hall and a special effect can be added to the audio signal. Formerly, to add reverberation to an audio signal, sound was recorded in for example a hall where reverberation was obtained. Alternatively, a steel-plate echo apparatus or the like was used to obtain a reverberative effect. In a recent reverberator, such an effect is electrically accomplished. More recently, as digital technologies have advanced, an apparatus that digitally synthesizes reverberation is becoming common. When reverberation is added to an audio signal by a digital process, a recursive digital filter is, for example, used. With the recursive digital filter, an input digital audio signal is attenuated and recurred. Thus, reverberation is generated. The generated reverberation is mixed with the original digital audio signal. In reality, initial reflection sound is added at a position delayed by a predetermined time period against direct sound. After a predetermined time period, reverberation is added. The delay time period of the reverberation against the direct sound is referred to as predelay. By adjusting the reverberation time, adding sub-reverberation, and finely adjusting the level of reverberation, a variety of types of sound can be generated. Reverberation in a real hall has a complicated waveform because of various reflections and interferences of sound due to the shape of the hall and the position of a sound source. However, as described above, in the method of which an original digital audio signal is processed with a filter, since the original signal is simply attenuated, there is the problem that the listeners have an artificial impression about the generated sound with the resultant signal.

23

CONCLUSION
We have defined the emotional state for emotion recognition in the HRI field, and from this definition have explained the rationale of the classification of human emotions from the viewpoint of psychology. We proposed a new feature called FRMS. From the results of experiments, we show that the recognition rate is approximately 76 percent. We also show that the FRMS feature is independent of the magnitude of the speech signal, and it is robust in noisy environments. These two points of the FRMS feature are remarkable advantages that enable emotions to be easily recognized without the problem of noise or distance between the speaker and microphone. In the future, we will hope to verify the FRMS feature with human test, and we plan to compare the recognition rate of this method with that of various classification methods. A real filed test (not tested by the database) must be accomplished for the practice use of the FRMS feature. The computing world has a lot to gain fron neural networks. Their ability to learn by example makes them very flexible and powerful. Furthermore there is no need to devise an algorithm in order to perform a specific task; i.e. there is no need to understand the internal mechanisms of that task. They are also very well suited for real time systems because of their fast responseand computational times which are due to their parallel architecture. Neural networks also contribute to other areas of research such as neurology and psychology. They are regularly used to model parts of living organisms and to investigate the internal mechanisms of the brain. .

24

REFERENCES

C. M. Lee, S. Yildirim, M. Bulut, A. Kazemzadeh, C. Busso, Z. Deng, S. Lee, & S. Narayanan, Emotion recognition based on phoneme classes, Proc. ICSLP, Jeju, Korea, 2004, 889-892. MATLABSOFTWARE IEEE2005 NEURAL NETWORK RESEARCH PAPER www.mathworks.com/products/neuralnet/ www.mccormick.northwestern.edu/docs/efirst/matlab.pdf www.math.utah.edu/lab/ms/matlab/matlab.html www.users.rowan.edu/~shreek/networks1/matlabintro.html www.suraj.lums.edu.pk/~cs537s04/resources/nnet_intro.ppt www.lois.ou.edu/res/course/IE5970/IntroMatlabNN.doc

25