Augmented guitar: adding expression-

controller characteristics to the guitar pick

Sergio A. Majluf – sergio.majluf@nyu.edu

1 Introduction
The art of electric guitar playing has remained almost unchanged for as long as the
instrument has been available, as we know it today (Beauchamp 1937). With only a few
notable, rare and eccentric exceptions, such as artist Jimi Hendrix performing with his teeth
(Hendrix n.d.) instead of fingers, not much exploration has been done around this subject on
this particular instrument.

To play the instrument, the player's left hand is wrapped around the neck and fretboard,
applying pressure with the fingers on top of the strings along different positions through
the length of the fretboard, in a coordinated way that leads to either a sequence in notes
(melody), or simultaneous notes in certain harmonic arrangements (chords). In some cases,
the contact between the fingers and the strings can be mediated by a third party device –
for example a guitar slide - allowing a different, continuous sound to be obtained by using
such device and moving it along the fretboard on top of the strings. In coordination with
that, the right hand of the player executes either strumming - hitting several strings at the
same time in up/down patterns to convey rhythm along chord voicings - or sequentially
plucking single notes producing a melody1. Both techniques can occur bare fingered or with
the aid of a guitar pick.

A guitar pick is a small, flat, handheld tool that aids the plucking of the strings, as a
complement or replacement of fingerpicking. In general, using a plectrum or guitar pick
produces a characteristic sharp, defined, strong sound, while it’s shape, thickness and
material influence this attributes, in contrast with the more dampened sound obtained by
fingerpicking (S Carral 2008).

The vibration of the metal strings over the guitar’s electromagnetic microphones, generate
an electric signal related to the string’s frequency and amplitude, that can then be
electronically filtered, equalized, amplified, or further put through any other analog and
Digital Signal Processing (DSP) system. It is important to note that, in the realm of digital
signal processing, current software development has been able to produce good quality
audio processing for non digital instruments, like an electric guitar, therefor, deepening the

1
This description is valid for right-handed players. For left handed players, the position the guitar is hold and the
role of each hand is swapped regarding the previous description.

Page 1 of 11
need of controller devices for as many parameters as those software enable the user to
modify while playing.

A third approach to producing sound form a guitar has been created and marketed, called
The EBow and described as “a hand-held electronic bow for guitar (…) a small battery-
powered unit replaces the pick in the right hand letting the guitarist mimic strings, horns,
and woodwinds with unbelievable sensitivity” (EBow n.d.). This unit is a replacement for the
plucking device, such as it’s a different approach to make the string vibrate. However,
regarding the basic element of the musical performance – the musician in direct contact
with the instrument – there’s no much more exploration or development made in this area
to facilitate or augment the guitar player’s performance and outcome.

A thorough search in the academic literature did not provide much new information beyond
already cited, however U.S. Patents search led to the discovery of a large number of similar
artifacts, most of which focused on the shape (Leneman 2012) (Christenson 1999), material
(Jay 2012), and ergonomic (Chance and Gray 2000) properties of the pick, to cite only a few.
Other proposals were comprised of electronic devices that allowed the control of certain
electric properties (Hudak 1998) of the signal produced by the guitar’s transducer, in which
cases the performer would have to actually stop playing the instrument to interact with the
system. However three were found that suited a similar exploration to the one I went
through (Chick 2005) (Caren and P. 1999) (Ekhaus and B. 1994) regarding how we physically
interact with the instrument, through either an electric conductive guitar pick in the first
case, or pressure sensitive picks in the latter two, that can control a sound effect unit in a
remote device.

Thus, the goal of this project is to develop and document a proof of concept, up to date,
cost effective and open source product, that would register change in the pick grip pressure,
in order to augment the musical experience and/or performance, by using an electronic
signal to control a DSP host or other equivalent computerized audio/image/environment
processing system.

2 Methodology, Design and Implementation

2.1 Observation and analysis
Direct observation was made of 5 non professional guitar players subjects performing live
while using a regular guitar pick, plus visual analysis of 10 professional guitar players that
were already recorded and available on YouTube.com and other video platforms.
Performance of some of the live subjects was video recorded, and motion observed in
frame-by-frame detail to define where to attach sensors for the initial tests measurements.
No further instructions were given regarding their position, how to hold the pick, or what to
play, in order to observe in the most natural possible way how they hold and used the pick
while playing.

I expected to be able to detect, first through observation and then through sensors
measurements, consistent radial/ulnar deviation, as well as forearm pronation/supination; I

Page 2 of 11
also observed index flexion-extension and what appeared to be index/thumb pressure
among players, by interpreting how hard the players appeared to be holding the pick.
Observations were done with the goal of empirically understanding weather those
movements could be used to control parameters in external audio devices.

2.2 Sensing hardware and measurements

FORCE SENSITIVE RESISTOR

A force sensitive resistor (commonly known by the
trademark FSR™) was used to measure the force
applied while holding the pick. The electrical
resistance of the sensor decreases predictably –
within a 10% tolerance range (Adafruit 2013) – when
force is applied to its surface. Conversion of electrical
readings into other measurement units (Analog, mV, Figure 1 - Force sensitive resistor
Ohms, Conductance, Newton) was made in Arduino
software, and readings where used to define curve response and thresholds for how
responsive the FSR affixed to a guitar pick was.

FLEX SENSOR
A flex sensor was used to measure the change in
flexion/extension of the index finger. In the case of this
sensor, the resistance increases when the sensor is
flexed. This sensors are not know for being precise, and
so exact measurements, for example degrees of wrist
flexion are not expected from this part, rather that
relative flexion from an average position.
Figure 2 - Flex sensor

ARDUINO MICROCONTROLLER
An Arduino UNO microcontroller was used to connect the sensors and send the data into a
computer, where it could be recorded, analyzed, and then used to actually control the sound
of the instrument.

The hardware connection consisted of a

couple of fixed value 10KΩ resistors in
voltage divider configuration, required to
properly connect the analog sensors to
the microcontroller, which was done
following the connection scheme shown.

Two filters were applied to the readings:

signal smoothing to counteract the
Figure 3 - Arduino and sensors
irregular sensor readings, and logarithmic
mapping to match the response curve of the sensors. Data smoothing was obtained by

Page 3 of 11
calculating a running average (Arduino 2013) of 30 samples, and non-linear mapping was
done using the floating scale mapping function (Badger and Shakar 2013)

Finally the code ran on the Arduino sent analog readings to the computer’s Serial port, that
were subsequently visualized for analysis using Processing2.

2.3 Data recollection and visualization

Two different approaches were followed to store and visualize information. To capture and
save the data to disk, I used CoolTerm application, from which a text output of the Serial
buffer can be read and saved (Roberts 2013). That information was time-stamped, and
exported as a CSV file, from which offline visualizations could be built. Also, two real time
visualization were made using Processing. Both techniques served the purpose of calibrating
and defining threshold of operations, in specific, for how soft/hard the user holds the pick,
so no false triggers would be sent to the MIDI host application.

2.4 DSP Integration

MIDI
The Musical Instrument Digital Interface (MIDI 2013) was chosen to send data from the
microcontroller to a host audio processing software, given that it is a standard protocol for
controlling musical interfaces. Although it’s rather old as a protocol, it is still valid and in
broad use, and its characteristics make it play nice with the Arduino which can be used to
format and send data directly in MIDI Messages format. The following table shows the core
structure of the MIDI message sent:

Control Command Modulation Channel Data Byte

0xB0 0x01 { sensor reading }

The main Arduino code pattern is as follows:

Serial.write(controlCmd);  

Serial.write(modulationCmd01);  

Serial.write(val);  

2
All code used for this project is available in https://github.com/sergiomajluf/Augmented-Guitar

Page 4 of 11
A caveat, instead of setting up a MIDI
connection from the microcontroller, a
free software interface – Hairless3 – was
used to simplify code setup and improve
development time. Hairless takes a serial
command from an Arduino, and can send it
back and forth from the MIDI host within
the computer.
Figure 4 - Hairless MIDI Serial

AUDIO WORKSTATION
The software used to complete the set up
is Guitar Rig, a complete virtualization
package for guitar players. This software
emulates all the elements of the sound
chain for electric guitar (pre amplifier,
effects, amplifiers, speaker cabinet), with
great level of detail not only in the
aesthetics, but also in the quality of sound.
Through MIDI, control messages sent by
the force sensing pickup could be bound to
whatever parameter the user want to
Figure 5 - Guitar Rig, MIDI messages, Arduino and Hairless
control. This is commercial software,
however it is not an integral part of this
project proposal, rather than a personal
choice for DSP. Any other software that can be controlled via MIDI will work with this setup.

2.5 Test device

The test device went through a few phases, from breadboard to rapid prototype. Under the
premise that the key was to have a MVP to test the software and develop documentation.
Design and construction were always kept lean, fast and leveraging readily available
materials.

3
http://projectgus.github.io/hairless-midiserial/

Page 5 of 11
Figure 6 - Different steps ant the prototyping stage

With all the hardware in place, the software was completed

2.6 Wireless Transmission

Under the consideration that a performance augmenting
device was being developed, the ability to become wireless
was deemed relevant. For this reason, I researched
alternatives and the most cost effective was XBee radios,
from Digi. Reliability in a situation of radio spectrum noise
and low latency are critical for audio controllers o
applications.

Configuration of these radios was a challenge by itself. The set up for this project requires 2
radios, one acting like a base and the other as a remote client. The following table shows the
configuration for each radio:

AT Command Remote Base Command Description

ATID 4064 4064 Personal Area Network ID

ATMY 40 64 Define own #

ATDL 64 40 Define destination #

ATD0 2 - AD1 / DIO1 pin20 (2 = ADC)

ATD1 2 - AD1 / DIO1 pin19 (2 = ADC)

ATIR A - Sample Rate

ATIT 1 - Samples before transmit

ATP0 - 2 PWM0 Configuration (2 = PWM Output)

Page 6 of 11
ATP1 - 2 PWM1 Configuration (2 = PWM Output)

ATIU - 1 Enable I/O Output

ATIA - 40 I/O Input Address

ATWR - - Write commands

ATAC Apply Changes

Under this configuration, the radios would be Line Passing, i.e., virtually and wirelessly
connecting analog inputs D0 and D1 with output pins P0 and P1 respectively. It is important
to note that the radios will not output an analog signal, rather a PWM signal to emulate the
properties of the analog input being fed into the remote unit.

3 Results
3.1 Position and Pick usage

Figure 7 – Test subjects holding a guitar pick

After the observation phase, no conclusive insight was gained that could suggest any
transversal radial/ulnar movement, or any pronation/supination. Those movements where
dependent on the player’s technique, which varied from person to person. Although there
were big inconsistencies in how every subject used his forearm and wrist, only slight
differences in how they hold the pick were evident. What could be appreciated was that
they all followed a similar approach, whereas to hold the pick in place, pressure was
exerted between the thumb’s and index’s fingers distant phalanxes.

Page 7 of 11
With this information, next I decided to focus only on the details of the hand, i.e. index
flexion-extension, and force between index and thumb, and to use D.I.Y. measurement
equipment to evaluate further development ideas.

3.2 Smoothing and index flex sensor readings.

The range obtained form the flex sensor attached to the index finger was rather small ( >16 &
< 60, out of the 1023 possible values), due to the limited flexion extension movement that
can be made with that finger while holding the guitar pick. Because of this, the value
obtained was mapped to a larger range, comparable to that of that of the FSR, and by doing
this, a very noticeable loss in resolution was produced. Because we will be mapping these
values directly to a controller of an audio device, signals need to be smooth. Figure 8 shows
the results of the same input processed by different smoothing values, ranging from none,
to 15, to 30 samples. A very aggressive smoothing setting needs to be used to provide
usable results, however, the higher the function, the slowest the response and higher
latency is present.

Figure 8 - Three levels of value averaging, or smoothing

Although some readings could be obtained from that sensors, using the device while playing
was rather awkward, in such a way that the user would have to be extremely aware of the
device, and loose focus on the rest of the performance. The graph in Figure 5 depicts 5
different readings, this time not scaled. After the first few seconds, it’s noticeable that there
is disparity in the readings within a very constrained band, and that readings are really
irregular within this band, beyond the control of the user. This was detrimental to the
goal of augmenting the guitar playing, and thus, the exploration with this sensor was
halted.

Figure 9 - 5 users, unscaled readings from Flex sensors

3.3 Force readings and usage experience

Readings were collected in subset periods of 15 seconds for the force sensing attachment.
The collected samples were displayed as color bands, mapped to the vertical axis of the
graph, and ranging from red (no pressure excerpted) through orange, yellow to green

Page 8 of 11
(maximum pressure). Figure 10 shows the first few seconds, where the user grabbed the pick
for the first time, and accommodated, until, around 2/3 the reading, he found a comfortable
position, and could hold the pick in a consistent way.

Figure 10 - Force applied to the pick, initial interaction

Several test were made trying to play in a transparent mode, i.e. to try as much as possible to
ignore the force sensing attachment. The graph in Figure 11 shows that the force readings
remained somewhat constant and flat. This is a positive outcome, meaning that the
impact that the pick attachment has on the playability of the instrument is low to
unnoticeable, and that almost no false triggers would be made without the user’s intention.

Figure 11 - Flat response when holding the pick

When ask to consciously squeeze the pick while playing, the spike pattern appeared on
Figure 12. This is another positive and expected result. It’s seems that users can get
consistent, repeated results using the device.

Figure 12 - Conscious changes in force, while playing

Page 9 of 11
4 Discussion
This project was driven by the intention to create a new device that would allow guitar
players to experiment and augment their performances in novel ways. Research showed that
although a few of such devices existed, none were openly available. This paper describes the
methods to recreate a device for such purposes, with current open source software and
hardware technologies.

The tests that I was able to perform indicate that the concept of augmenting the
performance of an electric guitar player through a force-sensing plectrum is feasible and
yields to interesting path for sound and visual explorations. Furthermore, this project allows
any amateur and professional musician to start using the device, without significantly
altering their playability over the instrument.

While developing this prototype, physical size was not a main concern, and in this respect,
there is a definitive opportunity to continue working in this area, into miniaturizing the
technology to make it even less invasive, more portable, and in the end, more appealing as
an end product.

Finally, more formal studies regarding the impact this technology would have in the user
interaction with the instrument is still needed, to scientifically support the heuristics
observations made through this project.

5 Reference
Adafruit. Force Sensitive Resistor (FSR) Overview. 27 de 04 de 2013.
http://learn.adafruit.com/force-sensitive-resistor-fsr.

Arduino. Smoothing tutorial. 21 de 04 de 2013.

http://www.arduino.cc/en/Tutorial/Smoothing.

Badger, Paul, y Greg Shakar. Fscale. 23 de 04 de 2013.

http://playground.arduino.cc/Main/Fscale.

Beauchamp, G.D. Rickenbacker Frying Pan. USA Patente 2,089.171. 10 de August de 1937.

Caren, y Michael P. Musical effect controller and system for an electric guitar. Patente US
5864083 A. 26 de January de 1999.

Chance, Allen, y William T. Gray. Guitar pick with gripping means. Patente 6054643. 25 de
April de 2000.

Page 10 of 11
Chick, Steve. Plectrum for a string instrument, a transmitter/receiver arrangement and a
signal processing apparatus. Patente US 6946592 B1. 05 de September de 2005.

Christenson, Eric J. Guitar pick. Patente 5973243. 26 de October de 1999.

EBow. http://www.ebow.com/home.php.

Ekhaus, y Ira B. Device for controlling musical effects on a guitar. Patente US 5300730 A. 5
de April de 1994.

Hudak, William B. Proximity sensitive control circuit for electrical musical instrument.
Patente US 5731535 A. 04 de March de 1998.

Hendrix, Jimi. «Jimi Hendrix teeth solo .» Youtube.com.

http://www.youtube.com/watch?v=ExSNjdGtTBc (último acceso: Abril de 2013).

Jay, Stuart Adam. Leather plectrum. Patente US 20120247304 A1. 4 de October de 2012.

Leneman, Michael. Guitar pick. Patente US 20120260788 A1. 18 de October de 2012.

MIDI. 24 de 04 de 2013. http://en.wikipedia.org/wiki/MIDI.

S Carral, M Paset. «The influence of plectrum thickness on the radiated sound of the guitar.»
Journal of the Acoustical Society of America, 2008.

Roberts, Dustyn. Seeing Sensors: How to visualize and save Arduino-sensed data. 20 de
04 de 2013. http://www.dustynrobots.com/news/seeing-sensors-how-to-visualize-and-
save-arduino-sensed-data/.

6 Licensing
This work is licensed under the
Creative Commons Attribution-NonCommercial-ShareAlike 3.0 Unported License.
To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-sa/3.0/
or send a letter to Creative Commons, 444 Castro Street, Suite 900, Mountain View,
California, 94041, USA.

Page 11 of 11