LaTeX Code Generation from Equation Photograph

Jim Brewer and James Sun

Department of Electrical Engineering, Stanford University

Motivation LaTeX Code Generation Pipeline

LaTeX is a powerful typesetting system that is Input Image Binarize Correct Skew
extremely useful for technical documents, in
particular mathematical equations. However, once
rendered, the output cannot be modified without
access to the underlying code. Re-coding lengthy
equations is time consuming and prone to error.
This project allows a user to take a photograph of Match Characters Segment Characters
a printed equation and produce LaTeX markup

Nearest Neighbor
Hu Moments

code to generate the equation.

Circular Topology
Final Output
Q(z)=\frac{1}{\sqrt{2\pi }}\int\limits_{z}^{\infty }e^{\frac{-x^{2}}{2}}d x

Assemble Equation
Output LaTeX Code
Future Work
• Extract equations from page or surrounding text
• Improve binarization and correct detection percentage
• Improve Skew Detection Scale # Correct % Correct Experimental Results
0.5 96 81.4%
• Recognize handwritten equations 0.75 109 92.4%
Rotate 15 clean equations in
increments of 5 degrees then
1.0 118 100.0% segment and match characters.
1.25 114 96.6%
Related Work Total: 437 92.6% Character Palette
[1] Torres-Mendez, L.A.; Ruiz-Suarez, J.C.; Sucar, L.E.; Gomez, G., "Translation, rotation, and
scale-invariant object recognition," in Systems, Man, and Cybernetics, Part C: Applications and Resize each palette
Reviews, IEEE Transactions on , vol.30, no.1, pp.125-130, Feb 2000 character by the
corresponding scale and
[2] Potocnik, B., "Assessment of Region-Based Moment Invariants for Object Recognition," in Test (# of Eq) Characters Correct % Correct
then attempt to match the
Multimedia Signal Processing and Communications, 48th International Symposium ELMAR-2006 Clean Eq. (15) 147/157 93.6%
focused on , vol., no., pp.27-32, June 2006
character with the matching
system using the original Eq. Pictures, Deskew1 (10) 91 / 123 74.0% Test matching on clean equation
[3] Kaur, Mandip, and Simpel Jindal. "An integrated skew detection and correction using fast template. Eq. Pictures, Deskew2 (10) 98/123 79.7% images and photographs of equations,
fourier transform and dct.“ in International Journal of Scientific & Technology Res, vol.2, issue 12,
Eq. Pictures, Combined (10) 106/123 86.2% tested with 2 deskewing methods.
Dec 2013