1

TABLE OF CONTENTS

Part-I Project Definition _________________________________________________ 3 Project Title __________________________________________________________ 3 Project Type __________________________________________________________ 3 Key Areas ___________________________________________________________ 3 Problem Definition ____________________________________________________ 3 Proposed Solution _____________________________________________________ 3 Aims and Objectives ___________________________________________________ 4 Outcomes and Deliverables ______________________________________________ 4 Research Methodology _________________________________________________ 4 Part-II Requirements ____________________________________________________ 5 Hardware and Software requirements ______________________________________ 5 Project Planning _______________________________________________________ 5 Part-III Gantt Chart_____________________________________________________ 6 Gantt Chart __________________________________________________________ 7 Part-IV Risk Management ________________________________________________ 8 Potential Risks _______________________________________________________ 8 Controling Risks ______________________________________________________ 9 Part-V Litreture Review __________________________________________________ 9 Literture Review ______________________________________________________ 9 Part-VI Project Implementation __________________________________________ 11 Project Structure______________________________________________________ 11 Part-VII Project Challenges _____________________________________________ 14 Project Challenges ____________________________________________________ 14 Part-VIII Future Plan __________________________________________________ 15 Future Plan Ideas _____________________________________________________ 15 Reference ____________________________________________________________ 16

Part-I Project Definition Project Title: Piano Player Robot

Project Type: Research and Development

Key Areas: Programming (Using NXC ( Not Exactly C) language) Robotics (Using Lego Mindstorm 2.0 robots) Automation

Problem Definition: The lack of piano players is what this project aims to solve. For instance, The American University of Iraq Sulaimani does not have a piano player. Music and piano is always good to have it everywhere. There are a lot of noises in the Cafeteria. A piano player can solve this issue, when there is music, people will enjoy and calm down. At the same time, when the music is played people can also enjoy the conversation with friends more than before. This robot can also be used in the library, too. Students always use iPods to listen to music while the study, but if there is a piano and a player, students does not even need to use the iPod.

Proposed Solution: The piano player project involves two robotic arms with a head to make it look similar to human. The robotic arms are used to create an automated piano player to be able to press the keys on the piano keyboard and produce sounds. The robot will be pre-programmed to play some nice music. This robot will have two hands. Each hand will have two to four fingers. The fingers will start playing the music when we turn on the robot through the NXT Intelligent Brick. Another functionality we can do is to pre-program the robot that can generate random music and play them on the piano. This can improve the performance of the robot. Therefore, to fix the problem that is mentioned in the problem
3

definition, a robot is required that can play music. As humans, playing piano is not easy; they have to go through some certain of practical and academic instructions to be able to play it. Therefore, a robot can solve this difficulty for those who like to have a piano player in any places they want. Also, instead of giving a very big salary for a piano player every month, the American University of Iraq Sulaimani can buy a robot once. This will also save some money for the AUIS.

Aims and Objectives: Aim: The aim is to show that not only humans can play piano but also robots. Core Objectives: Experiments with Lego NXT Mindstorms 2.0. Experiments with NXC Programing Language. Experiments with automating the Robot to play music.

Advanced Objectives: Experiments with making the robot compose music randomly. Experiments with making the robot record music and play it back.

Outcomes and Deliverables: A NXT well organized Robot that can function anywhere. Lego Mindstroms NXT 2.0 is a programmable robotics kit. This is used to build the robot. This kit is very easy to use. The robot can be shaped as how we would like to. Therefore, this makes it different and easier to use than any other kinds of robotic kits. The arms of the robot which are going to be programmed to play certain kinds of notes or melodies using NXC programming language.

Research Methodology: My methodology would be Case Study and Action Research. This is my first time to build a robot and program it. Therefore, I need to study the programming languages that I have not practiced or learned before. In this case, I need to research to find out tutorials or eBooks to teach me programming languages like NXC. The current project can solve the lack of a piano player anywhere. One of the places that can get benefit from this is the AUIS (American University of Iraq-Sulaimani) in their cafeteria or the library.

Part-II Requirements Hardware and Software requirements: Hardware: Lego Mindstorms NXT kit NXT Intelligent Brick One Piano ( This is used to build the robot ) ( It is the computer of the robot, it will be

used to input the sensors and the motors via RJ12 cables )

Software: NXC editor and compiler

Project Planning: The project consists of three parts: Submission One (It consists of the project proposal, project plan and risk management, and literature review.) Submission Two (It consists of implementing the project and developing it.) Submission Three (It consists of recording a video, presenting it, and thinking about the future plan for it.)

Part-III Gantt Chart On the next page, I have provided a detailed plan Gantt Chart for the Project Gantt Chart All the steps with details are provided in the Gantt Chart The dates on the Gantt Chart are somehow fixed. Time and the Gantt Chart Plan are two of the things that may change as the project and plan Progresses.

Part-IV Risk Management Risk Management Potential Risks

- Technical 1. When working with hardware and software, risks are always taken, therefore problems are sometimes occurring. One of the problems with the Robot is the Lego NXT Mindstorms kit. It is very easy to be broken. For example, when the piano starts playing, one or two of the fingers might get broken. 2. One other problem is with the piano. The piano is a real piano and it might be difficult for the robot to press the keys as a human does. 3. Also, the noise with the motors is another risk. Because we use the Lego kit, we might get some noise of the motors with the music when the robot plays the piano. This may annoy the listener.

- Non-Technical 1. Due to the short time that is given to the project research and development, it might cause failing to finish the project. For this kind of project two-month is probably a short time to finish everything, but still there is a possibility and it will be a big challenge to submit everything perfect and on time 2. Due to other courses I have during this semester, I do not probably have the whole time provided for the capstone. I will also need time for them when the time comes to quizzes and exams. I have to perfectly manage my time to succeed in all the classes.

 Controlling Risks:

- Technical 1. To control the first risk is to avoid spaces between the piano keys and the robot. The fingers should be in a place that the fingers are very close to the keys. - Non-Technical 1. The only solution to organize my time to this project and the other classes Will be through studying for the other courses whenever I have free time. For example, during the break time between the classes.

Part-V Literature Review In the fall 2011, when I had Dr. Hemin teaching me Advanced programming and Web systems classes, and when I got to know him more about what his specialists are, how interests he is in robotics, and how he successfully could achieve what he wanted in this field, I also became interested when he was talking about his experiences in robots. Also, when I was a kid, I thought that robot is not meant to ordinary people to have it or create it, or sometimes I thought it does not even exist. But now, the situation is different; therefore, to try building a robot, I thought about developing a piano player robot. When I have done research about this idea, I saw that there are others who tried this. Few of them used NXT to build it, some other did not. The very advanced piano player robot that has been built until now is one called Teotronico. It has 19 fingers, its ways faster than human and it can sing as well. It is made by an Italian roboticist. It took 4 whole years for him to finish it. (Teotronico, 2012)

Teotronico

When I did my research, I saw some piano player robots that have been built with NXT, but lots of them are very simple or have got only one finger that can only play some basic notes. The two most interesting once are Piano Playing Lego Mindstorm NXT Robot with iPhone and another one which is called NXT Lego Piano Player Robot. The first one is built by two roboticists on 30 April 2012. That tells it is new. As they talks about it in the objectives it is a piano player robot that can play piano using computer vision. (Frostyslug, 2010)

NXT Robot with iPhone

I watched the second one on YouTube. It has got 6 fingers and can play Beethoven with a human Piano player. The blow image is a picture of it.

The piano player robot that I am planning to build is way different than the ones I mentioned above. The first one I mentioned is very advanced, it is not built using NXT, it took a long time to make it done. The second one is built with NXT, but it is very slow, it uses iPhone,
10

and it does not look like a human. The third one might be a bit similar as mine, but still it is one single arm with six fingers, mine will be having two arms, more fingers, and a head as well to make it look exactly like a human which it has not built yet having all these functionalities together with NXT.

Part-VI Project Implementation Project Structure: I will divide the project structure into two parts, Software and Hardware. Hardware: The following hardware structure is used to implement this project: 1. Lego Mindstorms NXT kit It is used to build and design the robot from scratch. The good thing about it is that you will design according to what you have in mind and how you want you robot o look like. This is enough to build a very nice Robot to do the requirements. 2. NXT Intelligent Brick This intelligent brick is used to use it as a computer for the robot. The robot cannot do anything without this brick. This brick makes a connection between the robot and an actual computer to send the programming code and make it work as how it is programmed. 3. A computer One computer is enough to test the robot and send the data to the robot to make it work. This connection is made through with using a USB cable. 4. A Piano One piano is enough to test the music. This is used to setup the robot according to the height, width and the keys of the piano to make it play the notes.

11

 Software The only software I used to implement and program the Robot was Bricx Command Center. It is a very powerful software that I could totally control the robot like how I wished to with using a USB cable to connect it to the computer. I could turn on an off the Robot. I could tell the robot to play and stop the music whenever I wanted to, and I could compile the codding before sending it the robot to see if I have any error. The Hardware Implementation: - At the first time, when I designed the robot, I used only 2 motors and 4 fingers. The following is the screenshot:

12

Then I searched a lot to find out if there is a possibility to play music with four fingers. But I could find any, Then, I changed the design and added another motor with two more fingers. This time I could find a lot of music that anyone can play them using six fingers. Then, at the first time, I chose a song which was called what are words I programmed half of the music. Then, my second choice was Jingle Bell to program it and I did. Before the programing, I had to add two legs to the robots to make it look nicer and better attached to the piano. The following is the screenshot:

13

USBs The Brick

The Head The Motors The Legs

The Fingers The Gears

The Piano

How it works: When I finished programming, I downloaded the program to the NXT brick to make it work and play the music with making it connect it to the computer. On the brick there is an orange button that you can touch it to make the Robot on. Then, you can use the next and previous buttons to choose the software programs that is being downloaded. I named the software piano.nxc. If, anyone chooses the software program piano, the robot would start moving it is fingers to play the piano and the music that is being programmed.

The following is a screenshot of some of the codding I did with the interface of the Bricx Command Center:

14

Part-VII Project Challenges To find a suitable music was the biggest challenge I have faced. One of the reasons was that I do not how to play piano otherwise I could have written music for it. Also, the limited motors and fingers was another problem. Another problem was the piano, at the first two

weeks I was having a piano that it was difficult for the robot to press the keys, because it was not a professional piano. Therefore, I had to buy one that is easy to be pressed by the robot. Then, I had to start from the beginning to design and implement the robot according to the new piano. Another big challenge was with programming and how to find sources to learn the NXC language. I could not really find any source on the internet to teach me NXC. There is only one book called Programming LEGO NXT Robots Using NXC by Daniel Banditelli. It was my only source to finish this project. Beside the videos I have watched on YouTube about the other NXT robots that have been already built.

15

Part-VIII Future Plan Two ideas for the future 1. Play back music Write now, the project includes two music and it is being downloaded into the Robot. The future plan would be making the robot listen to music and after that plays it back on the piano. 2. Adding more fingers The robot is currently having 6 fingers. The future plan would be adding more fingers using more motors and one or two other bricks to make it play longer music.

16

References: 1. Liao, Alvin, and Ronald Orozco. Piano Playing Lego Mindstorm NXT Robot with IPhone. N.p.: n.p., 2012. WinLab. Web. <http://www.winlab.rutgers.edu>. 2. Frostyslug. "Bothoven - Lego NXT Piano Playing Robot." YouTube. YouTube, 11 Sept. 2010. Web. 03 Dec. 2012. <http://www.youtube.com/watch?v=JhkfeBfwci0>. 3. "Teotronico, the Well Dressed Robot Pianist." Teotronico, the Well Dressed Robot Pianist. N.p., n.d. Web. 03 Dec. 2012. <http://technabob.com/blog/2011/10/21/teotronico-robot-pianist/>. 4. Banditelli, Daniel. Programming LEGO NXT Robots Using NXC. Ed. John Hansen. Vol. 2.2. Italy: n.p., 2007. Web.

17