Visual Language to Control EV3 with ROS

Yessica Rosas and Jose Herrera

Universidad Nacional de San Agustin, Arequipa, Perú,

{yrosascu, jherreraq}@unsa.edu.pe

Abstract. LEGO Mindstorms EV3 like an educational kit is a versatile

tool for teaching robotic topics, as it supports various disciplines, skills,
and it is used in a different age. However, there is much software to con-
trol and programming ev3 in educational robotics are often limited by
their operating system. In this regard, we try to be flexible and using
other packages develop by the Robot Operating System (ROS) commu-
nity to work EV3 in Linux. This paper introduces a system to control
and programming ev3 using open-source tools, currently implemented
and future use case, and experiences from our first course of robotics
with teenager peruvian girls.

Keywords: EV3, LEGO, educational robotics, computer science, Blockly,

ROS

1 Introduction

EV3 is the third generation of the Lego Mindstorms line. It is considered a

versatile and low-cost robotic platform [5]. LEGO MINDSTORMS is a set of
plastic parts and electronic components that lead to the creation of different
robotic models, this educational kit is used at different levels of education [1, 6, 7]
to develop skills such as problem-solving, computational thinking and teamwork
[2, 3, 4].
Robotics is a complex area and a multidisciplinary activity, it is in con-
stant evolution, different frameworks have been developed for the control of a
robot standing out among these ROS (Robotic Operating System) [15]. ROS is
a collection of libraries open source that simulates an operating system with an
architecture based mainly on communication by messages [13], it facilitates the
creation of applications for robots, as communication between different packages
is possible programming languages decreasing the complexity to develop new be-
haviors [17]. ROS has become a standard within the community because of the
number of companies, laboratories and developers that use them today [18].
ROS is an open-source framework that works on Linux. ROS allows extending
the functionality of a robotic system, in this way most robotic systems open field
in Linux, being an opportunity for LEGO to be supported by this operating
system. For this reason, reviewing the architecture of ROS and the packages
developed in the community is that a visual programming environment for EV3
was integrated two packages robot_blocky and the packages to control EV3
2

to obtain a visual programming language, offering the student the opportunity

to develop and understand robotic concepts in a simple way, without worrying
about syntax problems and bringing then closer real situations.

2 Programming Languages for EV3

Currently, there are many high-level programming languages, which we can cat-
egorize into text-based programming languages (TPL) and visual programming
languages (VPL). Each of these languages is applied to different levels of ab-
straction. The VPL transform the sentences of a text-based language and turn
it into a puzzle, where the pieces are joined together to carry out a specific task,
represented by a set of blocks of different sizes, shapes, and colors.
Although the LPV were born almost 3 decades ago, in Peru its use is recent as
in last years these are applied in schools as learning tools to develop programming
skills. One of the first languages introduced in the classroom for children with
didactic purposes was the programming language Scratch and Alice, where each
programming environment supports a creative space for the design of stories,
games, and animations [21, 11]. With the increase in the popularity of these
tools, the diversity of VPL was also increased. For example, MIT App Inventor1
offers the ease of creating applications for cell phones in a simpler way.
Lego Mindstorms does not out from the use of VPL. EV3 has a programming
environment, the Lego Mindstorms software which combines the construction of
robots with the programming of them. It has a VPL based on blocks that can be
customized to perform a specific action. The blocks are grouped according to the
application for the control of sensors, motors, data flow and math operations.
The interrelated blocks sequentially carry out a series of actions and, the way
in which the program behaves depends on which blocks are used and how they
are connected. Lego Mindstorms has a free version and a proprietary version. A
disadvantage of Lego Mindstorms software is the cost of the complete software
and support for some operating systems such as Windows and OS X.
Scratch was born with the aim of increasing the skills of children in devel-
opment with technology. Scratch allows programming to be understood more
easily by everyone, due to its design characteristics, for the treatment of images
and sounds, motivates the imagination [11]. With a drag and drop environment,
creating a program becomes a simple task [21]. There is an extension of this
software to work with the Lego Mindstorms EV3 kit, without having to change
the original firmware.
Open Roberta [23] is another alternative, with a web environment it is an
interoperable platform for programming several types of robots. One of its main
advantages is that in some cases it is not necessary to have hardware, as the plat-
form has a simulator implemented. Among the educational robots supported by
this platform are Lego Mindstorms and the NAO humanoids. Its programming
environment is a combination of design ideas from Scratch and the Lego Mind-
storms software.
1
MIT App Inventor
 3

Additionally, we have TPL that work with Lego Mindstorms as ev3dev and
lejOS. EV3DEV is a Debian that works from an external memory to the brick, it
has support for several programming languages, but it is mainly used to develop
in Python. LejOS, is a modified firmware based on java, table 1 summarizes the
characteristics of the programming languages supported by EV3.

Language Firmware IDE SO Open-Source

Lego Mindstorms EV3
Visual Original Yes Win /Mac No
Software LEGO
Scratch
Visual Original Yes Win Yes
For LEGO Mindstorms EV3
NEPO Yes Win/Linux/
Visual Customized Yes
VPL Open Roberta (OpenRoberta) Mac
Win/Linux/ Yes
ev3dev Python Customized No
Mac
No Win/Linux/ Yes
lejOS Java Custimized
(eclipse) Mac
Table 1. Programming languages for Lego Mindstorms EV3. Some VPL has program-
ming environment, TPL as Python and Java can work in environment known as eclipse
or text editors.

For students that begin programming a VPL is better option, the block labels
are easy to read as they are represented by a natural language hiding abstract
concepts [22, 19]. On the other hand, the blocks have a representative shape
and color, you can recognize a procedure by the color of the block and its shape
without having to memorize the syntax of a language. Then, a VPL reduces the
cognitive load of the programmers [24, 25, 22]. Additionally, if an error in the
program flow arises it is easier to remove a block than to identify exactly the line
of code that contains the error, making the flow of a program easy to compose.

3 ROS Robotic

ROS is a free code framework, based on Linux, it is used to perform a modular

control in robotic applications. Originally ROS was designed by Willow Garage
[13]and is currently supported by the Open Source Robotics Foundation 2 . ROS
is a powerful tool because it uses object-oriented programming, a programming
method organized within a modular system. ROS operation is based on sending
of messages from one node to another node through channels called topics, and
as it is a distributed and object-oriented work system, the files are organized in
packages. ROS has a many packages to perform various tasks where the only
variants are the programming language and hardware to use.

2
Open Source Robotics Fundation - https://www.osrfoundation.org/
4

3.1 EV3

EV3 has a set of ROS packages that work together 3 , they control the ev3
components. The operating system that supports the brick is based on Yocto
Linux and the ev3dev kernel and includes all ROS meta-ros packages. Requires
a configuration of multiple computers in network ROS (can not be supported
directly in the brick by his limited RAM).
In the brick the ev3_manager node is executed, it launch the ros function-
alities in the ev3. The node ev3_manager publishes the topics to control the
motors and sensors, the scripts are run on ros server or rosmaster (PC) in this
way the robot is a client.
The list of packages that make up this system are:

– Control, stores the topics,low level controllers of motors and sensors.

– Launch, it contains the files that initialize the communication nodes.
– Manager, node executed on the brick.
– Mensajes, contains the definitions of the ros messages for the lego mind-
storms ev3 kit.

3.2 ROS courses

Today, we should think about the future of robots and science that studies it, and
mainly in an interface that educates the user to control the machines. ROS as an
advanced tool is used to carry out illustrative and attractive studies on robotics
for young specialists. ROS allows the development of high quality courses in
robotics45 6 . Remote laboratories use ROS for being a flexible framework for
writing software for robots. ROS includes a large number of libraries, tools, and
conventions that simplify the task of creating complex and robust behaviors.
With the existence of educational kits, teaching robotics has become popular,
especially simple in introductory courses in programming and robotics [16], [14],
[12], [9], [10], [8]. Robotics provides a level of programming for all students to
visualize and demonstrate ideas and concepts. In a robotics course with ROS
the idea is to obtain functional scripts that are executed by the rosmaster and
its behavior is visualized in a simulator or in a real robot.

4 ROS Blockly for EV3

Robot blockly is a package of the ROS environment, developed by Erle Robotics

7
, which implemented a visual web programming environment to control the dif-
ferent robots developed by the same company. The robot blockly package has a
3
ROS for Ev3
4
Robot Ignite Academy
5
ROS Courses Wiki
6
Nanodegree Robotics Engineer
7
Erle Robotics
 5

visual environment to teleoperate its mobile robots and a programming environ-

ment based on blocks using the blockly library, two of the main characteristics
maintained in this project.

4.1 The EV3 robot

In its first version, this project works only with a mobile robot with wheels (Fig.
1) and with the four sensors of the kit (touch, ultrasonic, color, gyroscope). Only
one type of sensor can be connected to every input port (in1, in2, in3, in4). The
robot must have 2 large motors connected to the output ports B and C.

Fig. 1. Lego Mindstorms EV3 robot

4.2 Primitives

Each primitive robot behavior is encapsulated in a function call. The primitives

follow the estructure of python script for ROS and they are described below:

Robot movement Table 2 shows a list of primitives related to indoor navigation,

a core capability for service robots like the EV3 robot. The Mover Robot primi-
tive makes the robot navigate to forward and back. The Girar Robot primitive
makes the robot turn left or right with a velocity options (slow and fast) in a
certain time. Functions to control motor mantain the sctructure of python script
for ROS (Listing 1.1).
6

Name and arguments Return

Mover Robot(direction, velocity) void
Girar Robot(direction, velocity, time) void

Table 2. Input ports of brick

Listing 1.1. Python script publish in a topic

import rospy # ros for python

pub = rospy.Publisher(’topic’, String, queue_size=10) # topic that

publish information
rospy.init_node(’node’, anonymous=True)
rate = rospy.Rate(10) # 10hz
while not rospy.is_shutdown():
pub.publish("hello") # publish information in pub
rate.sleep()

Sensors Each sensor is associated with a block, each block calls one python
function that maintains the structure of a python script to subscribe to a topic
and get information from a sensor (Listing 1.2).
Listing 1.2. Python script subscriber in a topic
import rospy
from std_msgs.msg import String #message to read

def callback(msg): # function to get information

print msg.data

rospy.init_node(’topic_subscriber’)
sub = rospy.Subscriber(’topic’, String, callback)

4.3 Errors
Error handling is an important topic when designing APIs. However, we wanted
users to be able to focus on their programs, and assume that the primitives work
as described. The only primitive with which we had error was for Girar Robot
block, since it does not move the robot when the time is less than one second.

4.4 Graphical Interface

To facilitate programming for users without formal programming experience, we
used a graphical interface called Blockly. Blockly is a programming language
itself, but it is a framework for building visual programming languages too. In
Blockly, program elements such as constant values, binary operators, while loops
 7

Motors Blocks Sensors Blocks

Fig. 2. Programming blocks to control EV3

or function calls are represented as blocks shaped like puzzle pieces. These blocks
can be connected by stacking them, attaching values to inputs, or nesting blocks
inside of other blocks. Blockly allows custom blocks to be made by defining a
blocks appearance, inputs, and outputs. We designed custom blocks for each
primitive, with inputs and outputs as shown in table 2 and Fig 2. We also
provided standard programming language constructs such as loops, conditionals,
variables, functions, strings, lists, math utilities, and logical operators. A full list
of all the standard blocks can be seen on Blocklys website 8 .
The graphical interface has two tabs. The first tab, all blocks programming di-
vided into eight categories: Logic, Loops, Math, Variables, Code, Control, Motors
and Sensors. Logic blocks are about boolean operators and conditional blocks,
Loops category are while and repeat blocks, Math involves math operations and
numbers, with Variable category we can create a new math, string or bool vari-
able, Code is only one block where we can compile code python, Control has
two blocks wait and init blocks, Motors are two blocks one to go forward or
backward, last category is Sensors, it has all sensors block we can work with
EV3. In the second second tab we can see the script auto-generated in a TPL.
We use mainly Python like TPL to program the behavior of the robot. We can
see the program in textual language and the visual language to introduce users
in other language, we focus here on Python as we can see in Fig 3.
We have five buttons the user can use to control the program and create a
new program or save it: Nuevo, Ejecutar, Abrir, Guardar, Control Manual.
The Control Manual button is for teleoperate the EV3 with directional keys.

8
Blockly
8

(a)

(b)

Fig. 3. Programs developed in the system. (a) Program with blocks makes the robot
go forward ten times if distance is less 15 centimeters, (b) Python script auto-generated
correspond to blocks program.
 9

5 User Study

We evaluated our system by conducting an observational study with students,

and own experiences of program to control EV3. The goal of the study was
to understand how well each group could use our system to make real-world
programs.

5.1 Procedure

We carry out a small robotics course (Fig. 4) with high school students. The
participants were first introduced to the robot with a simple and clear descrip-
tion about EV3, introduced in Linux and open source software too. Then, they
completed a 30-minute on how to build a robot. Then, the students use the
application and made a simple program to control motors and sensor. The par-
ticipants then created five programs with our system, one program at a time.
These programs, shown in table 3, were based on the real-world use cases dis-
cussed in next section.

# Program
1 Robot goes a path using flow control sentences.
2 Robot use touch sensor to know if should go forward or stop.
3 Robot use ultrasonic sensor to know there is someone or something in front of him.
4 Robot use color sensor to identify a black line.
5 Robot use gyro sensor to follow path of square.
Table 3. The prompts describing each of the five programs to make in the user study.

5.2 Course Experiences

We evaluate the experience to introduce in linux basic and learn about open-
source software. Also talk about the system if was easy to use and whether there
is something have to better. We work with eigth students (girls).

Linux and open-source software From eight students, only two students knew
the Linux Operating System, and they had a vague idea of what open source
software meant. The other students met a new operating system for the first
time.

Lego Mindstorms EV3 All the students liked the EV3 kit, they achieved to build
simple designs with ease. Only one student knew the ev3 kit because she pre-
viously attended a robotics course. The other students had heard about LEGO
but did not know the EV3 version.
10

(a) (b)

(c) (d)

Fig. 4. Robotics course with teenager students. (a) and (b) the girls building their
robot, (c) Teacher explain how works the system and (d)Test of the robot

Motors All the students achieved to make several simple programs to move the
motors of the robot. There was no complexity to understand the input parame-
ters of each block.

Sensors In order to work with the ev3 sensors, we had to talk about variables
and data types. The students identified mainly 2 types of data (boolean and
numerical). They knew and related the functioning of each sensor with the senses
of the body.

Control Flow The students did not have programming knowledge, only one
student. Of the main flow control sentences, the most commonly used sentence
was repetir.
The system does not allow to visualize the information of the robot, this
is initially seen from the terminal that executes the ROS nodes. The students
pointed out that this was one of the characteristics that have be improved in the
system. Additionally, another improvement that they suggest is the ability to add
another sensor, such as a camera, to perform an exploration of an environment.
Also, the students point out the interface is simple and its main function is
understood, which allows creating programs to control a lego robot. Students did
not recognize the type of string data because the system does not have feedback
from the sensors’ information, so they did not use that data type.
 11

One of the most common errors reported by the students was the response
time of the robot, when the robot had to turn for a time less than a second, in
most cases, the robot did not move, and the program ended.

6 Conclusion and Future Work

We presented the design and experiences of the system using ROS and Blockly,
which allows users to rapidly experiment with new applications for a educational
robot (EV3). The system support to control a little rover with four different
sensors. The system was successfully used the introduced student in robotics
and computer science topics. Our user study showed that programmers could
use the system to make real-world programs. Our study also revealed common
mistakes users made and helped us design future iterations of the system.
As future work we will integrate the platform to work with other sensors as
ROS allows the scalability of the system. Also, we want to add the feedback to
know and see all the time the information of the sensors from the system.

Acknowledgment
This work was supported by Universidad Nacional de San Agustin, UNSA In-
vestiga, Peru, under the agreement 12-2016-UNSA.

References
[1] Vic Grout and Nigel Houlden. Taking Computer Science and Programming into
Schools: The Glyndr/BCS Turing Project. Procedia - Social and Behavioral Sci-
ences. 141, 680-685 (2014).
[2] Jrgen Lindh and Thomas Holgersson. Does lego training stimulate pupils ability to
solve logical problems? Computers and Education. 49, 1097-1111 (2007).
[3] Fabiane Barreto Vavassori Benitti. Exploring the educational potential of robotics
in schools: A systematic review. Computers and Education. 58, 978-988 (2012).
[4] Sánchez, Rafael Esteban Arango and Builes, Jovani Alberto Jiménez. LEGO
Mindstorms NXT: Juego como Herramienta de Aprendizaje de Programación.
Lámpsakos, 72-78 (2014).
[5] E. Irigoyen and E. Larzabal and R. Priego. Low-cost platforms used in Control
Education: An educational case study. IFAC Proceedings Volumes. 46, 256-261
(2013).
[6] Peter J. Bradley and Juan A. de la Puente and Juan Zamorano and Daniel Brosnan.
A Platform for Real-Time Control Education with LEGO MINDSTORMS. IFAC
Proceedings Volumes. 45, 112-117 (2012).
[7] Magnenat, Stéphane and Shin, Jiwon and Riedo, Fanny and Siegwart, Roland and
Ben-Ari, Morderchai. Teaching a Core CS Concept Through Robotics. Proceedings
of the 2014 Conference on Innovation & Technology in Computer Science
Education. 315-320 (2014).
[8] V. B. Barreto and A. L’Erario and J. A. Fabri. Teaching programming for high
school students using the Lego Mindstorms robot. 2015 10th Iberian Conference
on Information Systems and Technologies (CISTI). 1-7 (2015).
12

[9] Mataric, M. J. Robotics education for all ages. In Proc. AAAI Spring Symposium
on Accessible, Hands-on AI and Robotics Education. (2004).
[10] Ministerio de Educacin Perú. Generando Experiencias Lúdicas. [PeruEduca] http:
//www.perueduca.pe/robotica/. (2016).
[11] Romero, Juan Suarez. Library Programming with LEGO MINDSTORMS,
Scratch, and PicoCricket: Analysis of Best Practices for Public Libraries. Com-
puters in Libraries. (2010).
[12] Uvais Qidwai and Ryan Riley and Sayed El-Sayed. Attracting Students to the
Computing Disciplines: A Case Study of a Robotics Contest. Procedia - Social and
Behavioral Sciences. 520 - 531 (2013).
[13] Morgan Quigley and Ken Conley and Brian P. Gerkey and Josh Faust and Tully
Foote and Jeremy Leibs and Rob Wheeler and Andrew Y. Ng. ROS: an open-source
Robot Operating System. ICRA Workshop on Open Source Software. (2009).
[14] Khassanov Alisher and Krupenkin Alexander and Borgul Alexandr. Control of the
Mobile Robots with ROS in Robotics Courses. Procedia Engineering. 100, 1475-
1484 (2015).
[15] Magyar, G., Sink, P., & Krizsn, Z. Comparison study of robotic middleware for
robotic applications. In Emergent Trends in Robotics and Intelligent Systems.
Springer. 121-128 (2015).
[16] Michieletto, S., Ghidoni, S., Pagello, E., Moro, M., & Menegatti, E. Why teach
robotics using ROS?. Journal of Automation Mobile Robotics and Intelligent Sys-
tems, 8(1), 60-68 (2014).
[17] S. Cousins. Is ROS Good for Robotics? [ROS Topics]. IEEE Robotics Automation
Magazine. 13-14 (2012).
[18] Brian Gerkey. ROS user survey: the results are in http://www.ros.org/news/
2014/04/ros-user-survey-the-results-are-in.html (2016)
[19] Martin Erwig and Karl Smeltzer and Xiangyu Wang. What is a visual language?.
Journal of Visual Languages & Computing. 38, 9-17 (2017).
[20] Weintrop, David and Wilensky, Uri. To Block or Not to Block, That is the Ques-
tion: Students’ Perceptions of Blocks-based Programming. Proceedings of the 14th
International Conference on Interaction Design and Children. 199-208 (2015).
[21] Resnick, Mitchel and Maloney, John and Monroy-Hernández, Andrés and Rusk,
Natalie and Eastmond, Evelyn and Brennan, Karen and Millner, Amon and Rosen-
baum, Eric and Silver, Jay and Silverman, Brian and Kafai, Yasmin. Scratch: Pro-
gramming for All. Commun. ACM. 52, 60,67 (2009).
[22] Matsumura, Kazuo and Tayama, Shuichi. Visual Man-Machine Interface for Pro-
gram Design and Production. Springer US. (1990).
[23] Open Roberta Open Roberta Wiki http://lab.open-roberta.org/ (2016)
[24] Luxton-Reilly, Andrew. Learning to Program is Easy. Proceedings of the 2016
ACM Conference on Innovation and Technology in Computer Science Education.
284-289 (2016).
[25] Duncan, Caitlin and Bell, Tim and Tanimoto, Steve. Should Your 8-year-old Learn
Coding?. Proceedings of the 9th Workshop in Primary and Secondary Computing
Education. (2014)