Anderson 1

Usability Testing of Engineering Equation

Solver (EES) and TK Solver
November 7, 2016

Anderson 2

Table of Contents
Introduction

Methodology

Tasks
Test Environment

Evaluative Criteria

Results

5
Task 1: Design Layout
Task 2: Back Solving Equations

Task 3: List Solving

Task 4: Material Properties

10

Task 5: Commenting

10

Task 6: Plotting

11

Task 7: Creating Reports

12

Task 8: Unit Analysis

13

Task 9: Math Functions

14

Task 10: Troubleshooting

16

Recommendations
Appendix A: Test Metrics

17

Anderson 3
Introduction

At the University of Idaho, engineering students are oftentimes given assignments and problems
that require software to solve. Two of the primary programs used are Engineering Equation
Solver (EES), and TK Solver (TK). Both have their share of strengths and weaknesses depending
on what task the user is attempting to perform. Engineering students are often tasked with
writing computer code that contains series of equations and variables; to which the student is
attempting to solve for. In addition, plots and other visual data might be needed. Both TK and
EES are set up to solve problems of this nature.
This usability study aims to quantify the strengths and weaknesses of both programs in order to
provide users with information to help them determine which is more beneficial for certain
problems. The test was completed by an engineering student with considerable experience using
both software packages. The goal was to test usability for an engineering student using both of
these programs for the first time.

Anderson 4

Methodology
Tasks
1. Design Layout
2. Back Solving Equations
3. List Solving
4. Material Properties
5. Commenting
6. Plotting
7. Creating Reports
8. Unit Analysis
9. Math Functions
10. Troubleshooting

Test Environment
This test was completed in one the University of Idahos student computing labs. This lab
contained both pieces of software already installed on the machines, therefor eliminating the
need to use different labs for the software. The test took place on a quite Thursday evening when
the lab was mostly empty. Quiet test conditions were sought and achieved by implementing the
test at night. The test took around 2 and a half hours to complete. Various common tasks were
completed by the tester and recommendations were recorded.
The newest versions of each program were used during this test. 64-bit computers with high
quality CPUs and GPUs were used to ensure both programs would be running at the optimal
level.

Evaluative Criteria
Criteria used to evaluate the software packages are:
Effective: The task was able to be completed
Efficient: Task was able to be performed in a timely manner
Engaging: The software was able to keep user committed to achieving task
Error Tolerant: The software was able to help user troubleshoot any errors
Easy to Learn: The learning curve for this task is reasonable for a first time engineering student

Anderson 5
All five of these were used in every task to evaluate. A Likert scale was used with values of
Strongly Disagree, Somewhat Disagree, Neutral, Somewhat Agree, and Strongly Agree as the
ratings. Neutral is used when the criteria is not applicable to the task.

Results
1. Task 1:
EES: When EES is opened for the first time, it looks very neat and clean. There is a
single blank white space that dominates the screen. At first it may appear to be difficult to
navigate the software. That is until the user takes their mouse and scrolls over the various
icons in the top of the screen. As each one is highlighted, the icon will tell the user what
its function is. The design is organized very neatly.

Pictured Above: Start Screen of EES

TK: In comparison to EES, TK is much less organized. When the user opens the
application, they are greeted with a "Rules" window and a "Variables" window. Without
any prior knowledge of the software, this may be confusing to a first time user.
Additionally, as the user begins to work between the variables and rules sheet, it becomes
annoying to the user when they have to change between the two screens to work. While
the two windows might be slightly confusing, there is a ribbon on the left hand side of the
page that informs the user of several important aspects including functions, plots, and
lists.

Anderson 6

Pictured Above: Start Screen for TK Solver

Conclusion: All unfamiliar things can be challenging to grasp at first. As familiarity
grows with the software, users will find both programs are laid out well. However, the
simplicity of EES make it much more practical for new students to learn. Also, all coding is
done and completed inside the same space, which makes it much more concise.

2. Task 2: Back Solving Equations

EES: Oftentimes when solving an equation, the strategy is to simply throw equations into
a software package, specify needed values, and let the program run through the
calculations to provide the needed output. EES is absolutely wonderful at dealing with
nasty equations that are all nested together. If the system has the same number of
unknown variables as it has linearly independent equations, then EES will be able to
calculate all unknown values almost instantly. This makes life much easier for young
mechanical engineering student.

Anderson 7

Pictured Above: An equation to calculate the stress in a curved beam from a mechanics of
materials class.
TK: In many ways, TK is similar to EES in that you can put nasty equations in any order
and it will still attempt to solve them, which many other programs will not do. However,
due to some unknown design element in its source code, it won't always be able to
calculate the user's unknown values. When this happens, the user is forced to tell TK to
"Guess" on a value. The software then runs through a series of values to try and find one
that converges. Oftentimes the guesses do not actually solve the system, resulting in
many more guesses required. It is painstaking for the user to go through this.

Pictured above: The same equation imputed into EES, Only this solution requires guesses
to solve.

Anderson 8
Conclusion: When it comes to larger and more intertwined equations, EES is far superior
as it will solve them automatically and not require any further user input. One can get
better over time recognizing when and where a guess might be needed in TK, but for a
new user, the stress of figuring this out is considerable.

3. Task 3: List Solving

EES: Engineers are often required to not only solve one value with a given equation, but
several values over an input domain. Both TK and EES allow users to input a range and
thus solve for many values at once. EES is quite simple to make lists for. If your system
solves once for a given input value, then the user can be sure it will list solve. The user
can create a new parametric table (shown below), and then select from a dropdown menu
the variables they want to put into a table. The user can then select one variable and put
an initial value, final value, and number of steps in between to accomplish the task. Then
a simple click on the green run button and the values for the other variables pop up
immediately.

Pictured Above: The parametric table menu showing the variables allowed to input.
TK: TK is similar to EES, only it is less convenient. The user is forced to create a
separate list for each wanted variable. This is annoying and more time consuming for the
user. Additionally, the user must specify which variable is the input in order to get values
for other variables.
Shown Below: Picture of three lists in TK. Notice that calculation unit is required in the
two lists other than r.

Anderson 9

Conclusion: Both programs are effective in creating lists. EES takes the edge with its
simplicity and the ability to view all the values in a single table and not separate lists.

Anderson 10

4. Task 4: Material Properties

EES: This comparison might be slightly unfair as EES was developed to be used for
thermodynamic reason. However, EES software contains all kinds of thermodynamic
tables that are essential to solve problems of that nature. Material properties on thousands
of materials, from water to R-134a, EES has data on all of these. These values can be
accessed and even calculated from the command window by calling on a function with
the name of the property the user is attempting to access, and inputting the correct
conditions.
Properties are called upon using the function callout shown below
TK: No thermodynamic properties can be accessed using TK. There is no way for a user
to calculate the enthalpy of a specific fluid within the software. Instead, these values must
be looked up using another source and the values then inputted into the users code.
Conclusion: If any problem requiring thermodynamic properties is needed, EES is hands
down superior. The user will spend critical amounts of time in other areas trying to access
properties.

5. Task 5: Commenting
EES: In EES, commenting is very simple. The user only needs to put either parentheses,
double backslashes, or squiggly brackets around any statement they wish to comment out.
Alternatively, the user can highlight a selection, right click, and choose to comment or
uncomment selection to remove it from the active code.
TK: Commenting is also extremely easy in TK. Only a semicolon is needed to comment
anything behind it. Additionally, TK offers many places to insert comments. Custom
functions, tables, variables, and the rules sections all contain places set aside for a user to
insert a comment.
Conclusion: Both are very effective, but the set aside places in TK make it very organized
and easy to read. This is much more effective when you provide a report of work to
someone. They can look directly to the comments to understand what was done, instead
of looking into the main code and hunting around for a comment amidst other code.

Anderson 11

6. Task 6: Plotting
EES: Plotting and creating visuals in EES is very easy. Once a list has been created, the
user can go to plots and create a new one and simply select which parametric table from
which want to choose data. It is also simple to choose the axis. Custom coloring options
further its effectiveness. Plot overlays can also be easily created by selecting overlay plot
to put several variables in the same graph.
Shown below is the plot setup for EES

TK: Plotting is much more difficult in TK. The user has to create a new plot and select
the x axis. Then they must input the specific lists to input in a new window. It is
confusion and during this time, errors are often encountered which require the user to go
back though and find what tiny thing they were missing.
Pictured below is one of the handful of screens the user has to navigate through to
complete a plot.

Anderson 12

Conclusion: EES is once again far superior to TK when it comes to potting. Its simplicity
is desired greatly over TK.

7. Task 7: Creating Reports

EES: In EES, it is extremely easy to create a design report. Simply by going into file:
create report. There are options to include everything done under that file. Reports, plots,
code, lists. All can be used and shown in a design report.
TK: Just like EES, it is extremely easy to create reports. Just like in EES, the design
report can be found in the file tab.
Conclusion: Both pieces of software have exceptional ways to create a report. They both
allow a young engineering student the tools they need to show their professors their
work.

Anderson 13

8. Task 8: Unit Analysis

EES: In engineering, units are everything. If your answer doesn't have units then it's
wrong regardless if the number happens to be right. Luckily for a young engineering
student, EES is equipped with unit analysis software that requires users input units along
with the numbers into their code. Even more satisfying to a user, is that if the calculation
is performed and the units do not align, a warning will be given to the user, thus allowing
them a way to check their work.

Pictured to the left is an example of including units with numbers

.
TK: The units section in TK does not perform unit analysis like EES does. A user has a
place to enter units down in the variables window. Unfortunately, these units don't
actually do anything of use. They can be used as a reference or for reporting reasons.
Outside of that though they function much like a comment.

Anderson 14

Pictured above is the units section found in the variables window. Note that the space
functions much like a comment and is not actually used in the analysis.
Conclusion: In a field where units are extremely important, EES takes the cake as the
better option.

9. Task 9: Functions
EES: Occasionally a problem will require higher level math, such as integrals and
derivatives, to solve. Luckily for a user, EES has built in functions that allow a user to
perform these functions. Although it is slightly complicated to learn how to enter the
information in.

Anderson 15

Shown Above is a user using integrals to solve for drag forces on a falling sphere. Several
functions are used in this picture to perform the necessary calculations.
TK: TK also has several built in functions that users can access to perform more complex
calculations much like EES. Although there is also a way for users to input and create
their own custom functions, which can in turn be used to perform integration and other
things upon.

Anderson 16

Shown above is a picture of some of the math functions that users can load into TK to use
in their code.
Conclusions: Both pieces of software allow users access to higher order math. The ability
of TK to let users implement their own custom functions easily gives it the edge in this
category.

10. Task 10: Troubleshooting

EES: One of the most important parts of any computer code is the ability to identify
errors and correct them. Luckily in EES, the program does an excellent job of telling
the user where and why an error is occurring. When the system has more variables then
equations, the program will tell you how many more equations it requires to solve.
TK: Similar help is given in TK although the help statements are not always clean in
indicating the problem. For example, if the error us because the computer can't find a
solution, the computer won't give any indication why the computer can't find a solution.
This can be maddening to a user who doesn't understand where their mistake is taking
place.
Conclusion: EES has the upper hand in most engineering applications due to the
description of problems. For a more experienced user though, the descriptions given in
TK are sufficient to help a user understand the problem.

Anderson 17

Recommendations
For a first time engineering student, EES is hands down much easier to use and learn. The
tolerance it has to user errors make it perfect for a student who has little coding experience. It
also has unique features which make is extremely useful for certain applications, i.e.
thermodynamics. Additionally, is has a much more simple structure that users will find much
more efficient to use.

Appendix A: Test Metrics

Produc

EES

TK Solver

Anderson 18
t
Task
Efficie
nt
Effecti
ve
Engagi
ng

Design Layout

Design Layout

The design Layout was easy to Learn

Strongly
Somewhat Agree
Agree
The design layout effectively displayed the features
available
Somewhat
Disagree
Agree
The design Layout was visually appealing and engaging
to use
Agree

Error
Toleran
t

Agree

There was no errors in the design layout of the software

Strongly
Agree

Easy
to
Learn

My
notes

Produc
t
Task
Efficie
nt
Effecti
ve

Strongly Agree

The Design Layout was easy to understand

Strongly
Agree
Strongly Agree
Both Programs have an effective layout and are easy to
use.

EES
Back
Solving

TK Solver
Back Solving

The program is efficient in its back solving capabilities

Strongly
Agree
Strongly Disagree
The Program effectively back solves the equations
Strongly
Agree
Strongly Disagree

Engagi
ng
Neutral
Error
Tolera
nt

The setup is engaging to the user

Neutral
The calculations are free of errors

Strongly
Agree

Somewhat Agree

Anderson 19
Easy
to
Learn
My
notes

The coding structure is easy to learn

Strongly
Agree
Somewhat Disagree
EES is hands down superior in terms of variable solving.

Product
Task
Efficient

EES
TK Solver
List Solving
List Solving
The Program efficiently creates lists of variable
Strongly Agree

Effective

The Program is effective in creating users lists

Strongly Agree

Engaging

Somewhat Agree

The lists are engaging to the user

Somewhat Agree

Error Tolerant

Somewhat Agree

Somewhat Agree

The lists are tolerant of user errors

Strongly Agree

Strongly Agree

Easy to Learn

It is easy to learn how to list solve

Strongly Agree
Strongly Disagree

My notes

Both effective, however EES has the upper hand

Product
Task
Efficient

EES
TK Solver
Material Properties
Material Properties
The Program efficiently finds thermodynamic properties of materials
Strongly Agree

Effective

The thermodynamic properties are effective in solving problems

Strongly Agree

Engaging

Strongly Disagree

Strongly Disagree

The properties are engaging to use

Anderson 20
Neutral
Error Tolerant

The program is tolerant of errors

Strongly Agree

Easy to Learn

My notes
Product
Task
Efficient

The ability of EES to access thermodynamic properties makes it

extremely effective as a tool
EES
TK Solver
Commenting
Commenting
Comments are efficient to create and helpful for users

Easy to
Learn

Neutral

Commenting is easy to learn

Strongly Agree

Product

Strongly Agree

The Comments are tolerant of user input errors

Neutral

My notes

Strongly Agree

The comments are engaging and helpful to users

Strongly Agree

Error
Tolerant

Strongly Agree

Users can place comments in places that are effective for use
Somewhat Agree

Engaging

Strongly Disagree

Using the thermodynamic properties is easy to learn

Strongly Agree
Neutral

Strongly Agree
Effective

Neutral

Strongly Agree

TK draws the upper hand here. It has so many places to put comments.
This is very effective in aiding users.

EES

TK Solver

Anderson 21
Task
Efficient
Effective
Engaging
Error Tolerant
Easy to Learn

Plotting
Plotting
The program efficiently generates plots for the user
Strongly Agree
Somewhat Agree
The program is effective and plots what the user wants
Strongly Agree
Strongly Agree
The plots have options to create visually appealing graphs
Strongly Agree
Strongly Agree
The Program can work with user errors
Strongly Disagree
Strongly Disagree
It is easy to learn how to create a plot
Strongly Agree
Somewhat Agree

My notes

Product
Task
Efficient

EES
TK Solver
Creating Reports
Creating Reports
The program allows users to create reports in an efficient manner
Strongly Agree

Effective

The reports are effective in communicating the work that was

done
Strongly Agree

Engaging

Strongly Agree

The report design options are visually stimulating

Strongly Agree

Error Tolerant

Strongly Agree

Strongly Agree

The reports can work with user errors

Somewhat Agree

Somewhat Agree

Easy to Learn

The Design Reports are easy to create

Strongly Agree
Strongly Agree

My notes

Both Programs have excellent report options.

Product
Task
Efficient

EES
TK Solver
Unit Analysis
Unit Analysis
The unit analysis capability is easy to understand
Strongly Agree

Effective

The unit analysis capability is effective in finding a solution

Strongly Agree

Engaging

Somewhat Disagree
Somewhat Disagree

Using unit analysis is stimulating to the user

Strongly Agree

Somewhat Disagree

Anderson 22
Error Tolerant

The unit analysis is accurate

Strongly Agree

Easy to Learn

My notes

Product
Task
Efficient

It is easy to learn how to use unit analysis

Strongly Agree
Somewhat Disagree
EES is the clear cut winner when it comes to unit analysis

EES
TK Solver
Math Functions
Math Functions
The Math functions are easily accessible
Somewhat Agree

Effective

Strongly Agree

It is relatively easy to learn the use of math functions

Strongly Agree

My notes

Somewhat Agree

Using the math functions result in valid and accurate

computations

Strongly Agree

Easy to
Learn

Strongly Agree

Each math function enhances the users experience

Somewhat Agree

Error
Tolerant

Strongly Agree

Math Functions are powerful enough to solve

problems
Agree

Engaging

Somewhat Disagree

Strongly Agree

Both Programs are very effective in the list of math

functions

Anderson 23

Product
Task
Efficient

EES
TK Solver
Trouble shooting
Trouble Shooting
The program efficiently finds ways to help the user
trouble shoot
Strongly Agree

Effective

The trouble shooting tips are effective in solving

problems
Strongly Agree

Engaging

Neutral

The Program is effective in identifying user errors

Strongly Agree

Easy to
Learn

Somewhat Agree

The Program presents errors in an engaging way

Neutral

Error
Tolerant

Somewhat Agree

Somewhat Agree

It is easy to learn what the program is identifying as a

problem
Strongly Agree
Somewhat Agree

Anderson 24

My notes