Math.Comput.Sci.

https://doi.org/10.1007/s11786-019-00442-3 Mathematics in Computer Science

Analysing the “Calculator Effect” of Different Kinds

of Software for School Arithmetic and Algebra
Rein Prank

Received: 2 December 2018 / Revised: 10 April 2019 / Accepted: 16 April 2019

© Springer Nature Switzerland AG 2019

Abstract The paper investigates the possibility of getting the answer and solution of arithmetic and algebra tasks
from outside pieces of software when the intended solution environment is paper and pencil or some teacher-
controlled software system. We consider spreadsheets, lightweight drill environments, programs with step by step
dialog and “algebra calculators”. For spreadsheets the paper describes several cases of mathematically incorrect
calculations. For drill environments we ascertain that they do not create a major risk of cheating. However, “algebra
calculators” provide answers and solution steps for most of the common elementary algebra task types (usually for
a licence fee). The conclusion is that addition of detailed solutions to computer algebra systems does not change
the situation with homework cheating substantially.

Keywords Calculator effect · Spreadsheets · Drill environments · Algebra calculators

Mathematics Subject Classification 97H20 · 97H30 · 97N80

1 Introduction

Introduction of numeric calculators gave students the possibility to perform calculations with decimal numbers
automatically, asking from the device step by step execution of operations. Automatic calculation enables solving
problems that contain bulkier numeric data. But it can also undermine learning if students can get the answers from
the screen instead of doing themselves arithmetic operations. The term ‘calculator effect’ was initially introduced for
the cases where students had poor arithmetic skills because they had used calculators. Using the classical numeric
calculator, the student also gets the results of all intermediate steps. Thereby the student should know the order
of operations in an expression. However, from some level, this becomes a routine thing. Many recent calculators
enable entering a numeric expression and calculation of its value in one step.
The effect gets a new dimension with algebraic software that can solve algebraic tasks that require from the
student knowledge of the solution algorithm or invention of his/her own solution. Usually the student should here
submit (by writing on the paper or entering in computer dialog) a step by step solution. Algorithmic and creative

R. Prank (B)
University of Tartu, Kastani 25-12, 50409 Tartu, Estonia
e-mail: rein.prank@ut.ee
 R. Prank

tasks will be completely trivialized if the software provides automatically not only the answer but also the solution
steps.
The most well-known symbolic calculators are computer algebra systems (CAS). They expanded the automated
solution capacity from the tasks of arithmetic to algebra. For the basic school level, this means operations with
fractions, expanding and factorizing of polynomials, solving different equations and equation systems, etc. An
extensive analysis of CAS answers to school equations is presented in the PhD dissertation and papers of E.Tõnisson
[1]. But the CAS usually give us only the answer for such tasks. The current CAS do not output the solution steps
and often do not even have sufficiently detailed commands for getting the intermediate results by request. Therefore
they do not cause immediate trivialization of school algebra tasks.
Practically all CAS contain some programming language making it possible to design new commands. This
feature has been used for university Mathematics (exercise environments, assessment, etc.) for quite a long time.
The assessment system STACK [2,3] is probably the best-known of such applications. It covers most of the common
topics of Calculus with hundreds of questions containing step-by-step solution dialogs. There is even CAS-based
exercise software for Mathematical Logic [4], but we do not find similar applications for School Mathematics.
Secondary school teachers and software developers prefer programs that provide an easy interface for entering the
expressions. For evaluation of the current situation, we consider in this paper other existing kinds of software that
can produce answers and solutions of arithmetic and algebraic tasks:
1) Spreadsheets,
2) Lightweight drill environments for arithmetic and algebra,
3) More sophisticated solution environments that have step by step solution dialog,
4) “Algebra calculators”.

2 Spreadsheets

Spreadsheets are not designed for working with expressions containing variables. We can speak here mainly about
calculation of the values of numeric expressions that can contain elementary functions. In many countries students in
schools are taught to use spreadsheets. The Mathematics teachers should expect that students try to use spreadsheets
also in their subject at least for checking the correctness of answers but also for knowing the answer before solving
the task. It turns out that the spreadsheets can give totally unexpected and mathematically incorrect outputs.
In our practical use we think that spreadsheets do decimal calculations. But this is not completely true; spread-
sheets try to do something more. Consider first a small example in Excel 2010 or 2016 [5] (Table 1). If we enter
in a Number type cell the formula ‘=1/3’ then we see in the column C the value 0,3…300…with up to 15 digits
‘3’ (the precision can be chosen by the user). But if we add three such numbers then we get 1. This means that
Excel actually records 1/3 and then uses computer algebra. At the same time, if we compare this 1/3 with manually
entered decimal fraction 0,3…300…then they are equal (cells F3:F5)! But the display of sum in E6 depends on the
precision setting of this cell. In case of 15 digits, Excel displays 0,9…900…and this not equal to 1. If we require
less than 15 digits then the sum in E6 is 1,0…0 but the equal-looking values in C6 and E6 are not equal. Similar
results can be received by working with elementary functions. For example, we get for any value of x equality
sin 2 (x) + cos 2 (x) = 1 although the decimal values of sin(x) and cos(x) do not attest this for some values of x.
It is clear that things like equality of 1/3 with a finite decimal fraction or non-equality of sums of equal addends
are in contradiction with the content of the Mathematics syllabus. They can also cause problems when students
check answers of calculation tasks or solutions of equations.
The situation of teachers gets more complicated by the fact that the behaviour of Excel is not a standard for
spreadsheets. In LibreOffice [6] (we tested LibreOffice 5.2) the rules for the content of E6 are the same as in Excel
but here the both possible values in E6 are equal with 1,0 in C6. Students can find other spreadsheet applications
beyond Excel and LibreOffice and teachers should be ready for challenging surprises.
Consider yet the spreadsheet cells of type Fraction. Excel executes arithmetic operations with fractions, reduces
and converts the result to mixed number. For example, 1 23 + 25 = 2 15 1
. Therefore, Excel can be used by students as a
Analysing the “Calculator Effect” of Different Kinds of Software

Table 1 Treatment of 1/3 in Excel

1 B C D E F
2 Entered Received in the cell Entered Received in the cell C = E?
3 =1/3 0,3333333333333330 0,333…30 0,3333333333333330 1
4 =1/3 0,3333333333333330 0,333…30 0,3333333333333330 1
5 =1/3 0,3333333333333330 0,333…30 0,3333333333333330 1
6 =sum(C3:C5) 1,0000000000000000 =sum(E3:E5) 0,9999999999999990 0

“homework calculator” for exercises with fractions and mixed numbers. Excel 2010 and 2016 enable using fractions
with up to one, two or three decimal digits in the numerator and denominator. If the received denominator contains
more digits than allowed by the cell settings then Excel finds the nearest fraction of the selected format (fractions
with a big numerator will be converted to mixed numbers). For the school algebra it means that Excel performs
operations with less complex fractions correctly and spoils the answers of more complex ones. For example, after
entering the fraction 1234 153
5678 Excel displays 704 in the cell and corresponding decimal fraction in the formula bar.
3
Addition of quite realistic for the school fractions can give 35 + 374
= 182 251
939 instead of 1295 . It seems that the values
of intermediate results do not cause fraction-valued approximation. LibreOffice 5.2 uses only one-digit numerators
and denominators and is probably not a candidate of a fraction calculator. If a basic school student uses fraction-type
spreadsheet calculations for checking solutions of operations with fractions or linear equations then the correctness
of the results depends on the actual numeric values.
Manuals are quite selective in describing the mixture of decimal and algebraic calculations implemented in
spreadsheets and the impact of cell settings. It is also clear that a complete description would be too complex for
the school. The conclusion seems to be that we should warn teachers and students about unpredictable effects of
spreadsheet calculations. Teachers should be ready for strange answers, questions and problems.

3 Drill Environments

According to specialists of learning software, ‘drill environments’ are programs that display tasks and require from
the student entering only the answer or provide also some boxes for intermediate results that correspond to some
‘canonical’ solution algorithm fixed by the program. In the next section we consider the programs that have a step
by step solution dialog with feedback at every step and allow the student to choose the solution path and lengths
of the steps. Some drill environments allow (or require) correction of mistakes, others only detect mistakes and
calculate points. Tasks can be generated randomly or taken from a prepared database. Most drill environments do
not contain a general solution engine for solution of arbitrary tasks of their task types. If a hint/example is requested,
they display the solution of a fixed task of actual type. But some environments are able to display the solution of
the task assigned to the student.
Web-based drill environments are available for many types of exercises of basic school Arithmetic and Algebra.
Consider first the exercises on operations with fractions. In November 2018 the first page of Google search for
operations with fractions practice gave us webpages of Khan Academy [7], Varsity Tutors [8], Math-drills [9],
Math-Aids [10], IXL [11], Homeschoolmath [12] (some links pointed to passive learning materials). It is worth
mentioning that the order of the environments returned by a particular search engine is quite stable even over the
years. Table 2 presents some relevant for our interests features of fraction exercises of listed environments.
Varsity Tutors is an environment where teachers can create multiple-choice tests and the content and the properties
of the tests depend on the author. Math-drills, Math-Aids and Homeschoolmath provide only printable worksheets.
Table 3 presents information about solution of linear equations in Khan Academy [7], IXL [11], Lamar [13],
MathBitsNotebook [14], Math-only-math [15] and Softschools [16].
 R. Prank

Table 2 Demo solutions and step hints for fraction exercises

Website Topics Demo solutions Step hints Stud- Quality
entered issues
tasks

Khan Academy Fractions and mixed numbers For assigned task Yes No
Varsity tutors Arbitrary (not only Mathematics) Entered by the author of task If entered No Depend on the author
Math-drills Fractions Only answers No No
Math-aids Fractions and mixed numbers For assigned task No No
IXL Fractions and mixed numbers After the wrong answer No No
Homeschoolmath Printable worksheets Only answers No No

Table 3 Demo solutions and step hints for solving linear equations
Website Demo solutions Step hints Stud-entered tasks Quality issues

Khan Academy For assigned task Yes No

IXL For assigned tasks, explanations No No
Lamar Practice worksheets with available solutions Yes No No correctness checking
MathBitsNotebook For assigned tasks No No Multiple-choice answers
Math-only-math Solutions of fixed tasks No No Fixed worksheets
SoftSchools Fixed quizz No No Multiple choice

For automated solution of homework tasks, students need an option of entering their actual tasks. The tables show
that most popular drill environments do not enable this and therefore they cannot be used as undesired homework
calculators.

4 Step by Step Solution Environments

Development of algebra exercise environments began in nineties when the computer monitors attained sufficient
speed of display of graphical information and the programming environments developed convenient utilities for
building of two-dimensional images of expressions on the screen. In this Section we consider three quite universal
and sufficiently described in literature exercise environments MathXpert [17,18], Aplusix [19,20] and T-algebra
[21,22]. They all are already at least ten years old. There are not many new programs available. Their development
is very labour-intensive and the most interesting scientific and user interface goals were achieved with the first
projects. There are some new environments online, such as Carnegie Learning [23], MathSpace [24], Aplusix Neo
[25], but their properties do not differ very much from the older software. In addition, MathSpace and Carnegie
Learning environments are whole-class oriented commercial products, which are not individually accessible to
other students for an affordable price.
MathXpert was created in the nineties by M. Beeson (San Jose State University) for exercises in Algebra,
Precalculus and Calculus. It has a rule-based solution dialog (Fig. 1). For every solution step the student first marks
a subexpression. The program displays a popup menu with conversion rules that can be applied to the selected
subexpression. The student selects one of the rules and the program applies it. Result of the step depends slightly
on the level of the student in the student model. Explicit writing of multiplication with 77 and 55 belongs to the
beginner’s level.
Unlike drill programs, MathXpert does not enable direct production of erroneous steps by the student because
rules are applied by the program. There is a mathematical reason for choosing a Rule-based rather than an Input-
Analysing the “Calculator Effect” of Different Kinds of Software

Fig. 1 Rule selection after marking a subexpression in MathXpert

based dialog (where the student enters the result of the step and the program checks its correctness). MathXpert
accepts a very broad signature of functional symbols in expressions. Even the partial checking of equivalence of
such expressions is a hard task and the general case is algorithmically unsolvable [26]. MathXpert tries to be as
learner-friendly as possible. It has a rich spectrum of help facilities. Hint gives textual hints, for example “Multiply
the fractions to get a single fraction”. Autostep executes the next step automatically. ShowStep shows what should
be marked and what rule should be selected from the menu. AutoFinish completes the solution and adds the steps
from the current situation until the final answer.
The user of MathXpert can download a free trial version (with limited functionality) or purchase an individual
or lab licence. In the trial version the only task source is the program, the option ‘Type it in’ is disabled. For us this
means that the free version of MathXpert cannot be used as a homework calculator. And the individual licence of
MathXpert Algebra costs more than software created specifically for the school students (see the next section).
Aplusix (Nicaud et al., University of Grenoble, [20]) is an Input-based environment for elementary algebra. At
each step the student should enter the result of the step and the program checks its correctness (equivalence with
the previous row). To get a new expression/equation/equation system, the student can copy the previous row to the
next line (or start from an empty expression) and then use the expression editor. The program displays between the
two rows the sign of equality/equivalence that is crossed out when the two rows are not equal/equivalent.
Figure 2 presents a fraction calculation situation in Aplusix. The first step was performed using the ‘Perform a
7
step’ facility by help companion Chloe. At the second step the first member 14 is already the result of the step but
the rest is not yet deleted from the expression and causes inequality.
Aplusix has several modes of help. Suggest an expression proposes a subexpression to be changed. Suggestion
tells what to do with this subexpression. For example, for the first step we get “Write 2/7 with denominator 14”.
Explanation of a transformation explains the step and calculates the result in a pop-up window. Perform a step
executes the step and adds it to the solution.
The user of Aplusix can download a free trial version for 10 days, purchase a single-user, home or server licence.
The trial version of Aplusix enables typing in the new tasks (task type and initial expression/equation/…). It means
that Aplusix can be used as a homework calculator for 10 days. For some countries it is important that Aplusix does
not implement mixed numbers.
T-algebra (Prank et al., University of Tartu, [22]) is an exercise environment where the step dialog combines
Rule-based and Input-based elements. A solution step consists of three substeps. At the first two substeps the student
selects in the menu a conversion rule and marks operands for the rule. The program checks that the marking is
syntactically correct and fits with the selected rule. After that the student enters the subexpression that will replace
the marked subexpression and the program checks the equivalence and (partially) the correspondence to the applied
rule. The main goal of the authors was implementation of detailed error diagnostics for all substeps.
 R. Prank

Fig. 2 Editing the result of the step in Aplusix

Fig. 3 Solution window of T-algebra during the first two substeps. Hint for rule selection

T-algebra is able to give hints for the next substep and display the solution from current situation to the answer.
But these options can also be disabled in the task file. Figure 3 presents the solution window when the student
solves a linear equation. The operands of previous steps are highlighted with a green background. The full menu
also contains rules for fractions and simplifications. After performing two solution steps the student has requested
a hint for rule selection.
The Student program of T-algebra takes the tasks from the task file that is prepared using the Teacher program.
Thus the Student program cannot be used as a homework calculator.
Analysing the “Calculator Effect” of Different Kinds of Software

Table 4 Quality of solutions that require additional decisions from the student/program
Environment Topics, style of output Fract Lin eq Lin syst Price

Symbolab Alg+Calc mix-num incl 7:A 7: B,D 14:C,D Free. Resources for 2.49/14.99
MathPapa Elem algebra, mix-num incl 4: 8:D 9:D Free
Mathway Answer+ instructions 7:D 6:B 8:C Answ+instr free Solutions - 9.99/39.99
Free-Online-Calculator-use = Mathway
MathWareHouse =Mathway
Wyzant No eq syst 6:E 12: B,D –
eMathHelp Non-elem meth
QuickMath No mix-num 10*: D,F 11: B,D 25(add method) First attempts free, 9.99/29.99
Algebra calculators No mix-num No eq syst 6* 14: D – Free
WebMath 6:A ?: D,G ?: D,G Free
MATHCelebrity 7:B 8: B,D 8:C 9.95/

5 “Algebra Calculators”

Alongside exercise environments there are a few dozen programs designed specifically for doing students’ homework
(producing solutions with necessary explanations). A search for ‘algebra calculator’ returns several pages of web
links to such programs. Some of them are free, some have daily limitations on free use, and some are commercial
with a moderate price. There is even an Algebra Calculators Guide: “144 Calculators Separated by Skill Level”
[27]. The Guide tries to be “positive” and does not assess the quality of solutions.
If the student uses an algebra calculator for getting the solutions of practice/homework tasks then the key question
is the quality of the solutions. For some tasks it is sufficient just to follow “the textbook algorithm”. But for some
task types the textbook algorithm leaves some concrete choices to the user (for example, what unknown to isolate
first) and for some tasks it is reasonable to apply some interim simplifications between the steps of the official
algorithm.
In Table 4 we consider solutions of fraction addition task 3 48 + 56 , linear equation 3(3x − 2) − 7x = 5x + 1 and

2x + 4y = 5
equation system and try to understand how well the most popular calculators make complementary
−3x + y = 1
decisions. The numbers in the cells were meant to represent the number of steps. However, for some solvers it is
quite hard to establish what is the right number. The solution steps in the outputs are accompanied with explanations
that can contain substeps of the calculations. In the most garbled cases, it is difficult to believe that students of the
6th grade would be able to extract a reasonable solution path from the long text. The number of steps is marked by
‘*’ if the addend 48 was used instead 3 48 because the calculator does not implement mixed numbers. Letters A…G
denote most frequent drawbacks:

A Does not reduce 48 before finding the common denominator

B Collects like members in left side before moving 5x to left
C Bad choice of unknown/equation for substitution
D Too detailed elementary steps
E For addition, mixed numbers converted to improper fractions
F Unnecessary exponential presentation of numbers
G Very long explanations

The last column contains prices of licences for one month/ one year.
The best outputs seem to be given by MathPapa and Mathway. Their steps are reasonable and explanations are
clearly formulated.
 R. Prank

6 Conclusions

Consider now the situation before introduction of step by step solutions in CAS. From the point of view of software
developers, we see the following picture. There are online step by step solvers that solve most of the tasks of
School Algebra. For some task types (where textbook algorithms leave some choices to the student) the quality of
automated solutions is low. Some less common task types can be missing. Some topics are ignored (mixed numbers
or doing operations with them). However, availability of solutions in educational CAS (for example, Geogebra)
will not change the situation with possible cheating very much.
For elaboration of step by step solutions the CAS needs addition of detailed commands for school-level solution
steps. After that it is possible to implement textbook algorithms and solution methods for most important tasks
that have no textbook algorithm (for example, factorization). Mathematics traditions of some countries also require
the use of mixed numbers. In the first years, CAS solutions will most likely have the same imperfections as the
current algebra calculators. But even later the presented solutions cannot conform to all necessary combinations of
country-, textbook- or teacher-specific requirements.
If we think about the situation of teachers then, before implementing the conclusions from the content of this
paper, we should inform the teachers much better about software-provided options for getting answers and solutions
of algebra tasks. The disputable quality of answers and solutions adds different task- or program-specific issues:

1. Correctness of equality checking and fraction operations in spreadsheets.

2. Most of solvers implement textbook algorithms without any intelligence (without interim simplifications, using
data-independent choices of parameters, …). The teachers can guess the reason when a student submits a series
of strange brute-force solutions (but without any mistakes!).
3. Weaker students are not able to translate the output of CAS or algebra calculator in the required “language of
School Mathematics” and/or explain the solution steps.

References

1. Tõnisson, E.: Differences Between Expected Answers and the Answers Offered by Computer Algebra Systems to School Mathe-
matics Equations. PhD dissertation, University of Tartu. http://dspace.ut.ee/handle/10062/58398 (2017). Accessed 29 Mar 2019
2. Sangwin, C.: Computer Aided Assessment of Mathematics. Oxford University Press, Oxford (2013)
3. STACK: https://stack2.maths.ed.ac.uk/demo/ (2019). Accessed 29 Mar 2019
4. Aguilera-Venegas, G., Galán-García, J.L., Galán-García, M.Á., Rodríguez-Cielos, P.: Teaching semantic tableaux method for
propositional classical logic with a CAS. Int. J. Technol. Math. Educ. 22(2), 85–91 (2015)
5. Excel: https://products.office.com/en-us/ (2019). Accessed 29 Mar 2019
6. LibreOffice: https://www.libreoffice.org/ (2019). Accessed 29 Mar 2019
7. Khan Academy: https://www.khanacademy.org/math/arithmetic/fraction-arithmetic (2018). Accessed 25 Nov 2018
8. Varsity Tutors: https://www.varsitytutors.com/basic_arithmetic-help/fractions/operations-with-fractions (2018). Accessed 25 Nov
2018
9. Math-drills: https://www.math-drills.com/fractions.php (2018). Accessed 25 Nov 2018
10. Math-aids: http://www.math-aids.com/Mixed_Problems/Mixed_Problems_Fractions.html (2018). Accessed 25 Nov 2018
11. IXL: https://www.ixl.com/math/grade-6/add-subtract-multiply-or-divide-two-fractions (2018). Accessed 25 Nov 2018
12. Homeschoolmath: https://www.homeschoolmath.net/ (2018). Accessed 25 Nov 2018
13. Lamar: http://tutorial.math.lamar.edu/Problems/Alg/SolveLinearEqns.aspx (2018). Accessed 25 Nov 2018
14. MathBitsNotebook: https://mathbitsnotebook.com/Algebra1/LinearEquations/LEPracEquations.html (2018). Accessed visited 25
Nov 2018
15. Math-only-math: https://www.math-only-math.com/practice-test-on-linear-equations.html (2018). Accessed 25 Nov 2018
16. Softschools: http://www.softschools.com/math/algebra/linear_equations/ (2018). Accessed 25 Nov 2018
17. Beeson, M.: Design principles of mathpert: software to support education in algebra and calculus. In: Kajler, N. (ed.) Computer-
Human Interaction in Symbolic Computation, pp. 89–115. Springer, Berlin (1998)
18. MathXpert: https://www.helpwithmath.com/ (2018). Accessed 25 Nov 2018
19. Nicaud, J., Bouhineau, D., Chaachoua, H.: Mixing microworld and cas features in building computer systems that help students
learn algebra. Int. J. Comput. Math. Learn. 5(2), 169–211 (2004)
20. Aplusix: https://aplusix.com/en/ (2018). Accessed 25 Nov 2018
Analysing the “Calculator Effect” of Different Kinds of Software

21. Prank, R., Issakova, M., Lepp, D., Tõnisson, E., Vaiksaar, V.: Integrating rule-based and input-based approaches for better error
diagnosis in expression manipulation tasks. In: Li, Shangzhi, Wang, Dongming, Zhang, Jing-Zhong (eds.) Symbolic Computation
and Education, pp. 174–191. World Scientific, Singapore (2007)
22. T-algebra: http://math.ut.ee/T-algebra/ (2018). Accessed 25 Nov 2018
23. Carnegie Learning: https://www.carnegielearning.com/ (2018). Accessed 25 Nov 2018
24. MathSpace: https://mathspace.co/us (2018). Accessed 25 Nov 2018
25. Aplusix Neo: https://aplusix.org/siteTemplate.php?lang=en&page=accueil.php (2018). Accessed 25 Nov 2018
26. Richardson, D.: Some unsolvable problems involving elementary functions of a real variable. J. Symb. Log. 33, 514–520 (1968)
27. Algebra Calculators Guide: https://www.calculators.org/math/algebra.php (2018). Accessed 25 Nov 2018

Publisher’s Note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.