Building the Auphonic Mobile Web Application

Christoph Pojer

Bachelor Thesis Software Development and Business Management Institute for Information Systems and Computer Media Graz University of Technology

Advisor Univ.-Doz. Dipl.-Ing. Dr. techn. Martin Ebner

Abstract
Building a mobile web application is a challenging task and most applications fall short of creating a good user experience. The project is focused on designing and implementing an application for Auphonic, an automatic audio post-production service, by carefully recreating native user interface elements and by optimizing the performance to minimize the gap between web and native applications and to overcome limitations of mobile browsers. To maximize the amount of platforms the application can run on in the future, web technologies were used with only a small native wrapper that provides a mobile browser engine with special abilities on every platform. The application delivers a wide array of functionalities such as viewing, editing and managing recordings, audio productions or presets and enables to record and upload audio or videos directly to the Auphonic servers for automatic postproduction. This thesis aims to highlight the challenges that were encountered during the programming of the Auphonic application. Details about the user experience and perceived responsiveness of the application are discussed and special attention is given to design. Finally the project is evaluated and the opportunities for future work are outlined.

Zusammenfassung
Eine mobile Webanwendung zu entwickeln ist eine schwierige Aufgabe und ein Groteil der Anwendungen hat eine schlechte Bedienbarkeit (User Experience) im Vergleich mit nativen Applikationen. Es wurde eine Anwendung fr Auphonic, einem automatischen Audio-Nachbearbeitungsdienst, entworfen und entwickelt indem native Bedienelemente sorgfltig nachempfunden und die Bedienbarkeit optimiert wurde, um den Abstand von Web- und nativen Anwendungen zu minimieren und um Einschrnkungen in mobilen Webbrowsern zu umgehen. Damit die Applikation in Zukunft auf so vielen Plattformen wie mglich verffentlicht werden kann, wurden Web-Technologien verwendet und nur ein kleiner Teil ein Webbrowser mit speziellen Fhigkeiten pro Plattform wurde nativ umgesetzt. Die Anwendung hat eine groe Auswahl an Funktionen wie zum Beispiel das Anzeigen, die Bearbeitung und das Verwalten von Aufnahmen, Produktionen und Voreinstellungen. Es ist mglich, Audio- oder Videoaufnahmen direkt in der Applikation zu erstellen und sie fr die automatische Bearbeitung auf die Auphonic Webserver zu laden. Diese Bachelorarbeit zeigt die Herausforderungen die beim Entwickeln der Auphonic Anwendung aufgetreten sind. Details ber die Bedienbarkeit und Reaktionsgeschwindigkeit werden diskutiert und spezielle Aufmerksamkeit wird dem Design gewidmet. Zum Schluss wird das Projekt evaluiert und die Mglichkeiten fr zuknftige Arbeiten werden aufgezeigt.

Table of Contents
1 In tro d u c tio n ............................................................................. 5
1 . 1 A b o u t A u p h o n i c . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 1 . 2 S t a t e o f W e b A p p l i c a t i o n s o n M o b i l e P h o n e s . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

2 A p p lic a tio n F e a tu re s ................................................................. 6

2 . 1 T h e A u p h o n i c A P I . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

3 D e s ig n a n d U s e r E x p e rie n c e ...................................................... 8
3 . 1 D e s i g n E v o l u t i o n . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 1

4 L ib ra rie s a n d S p e c ific a tio n s .................................................... 1 2

4 . 1 C o m m o n J S . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 2 4 . 2 A p a c h e C o r d o v a . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 3 4 . 3 M o o T o o l s . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 3 4 . 4 S t y l u s . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 4 4 . 5 O t h e r . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 4

5 Im p le m e n ta tio n ....................................................................... 1 5
5 . 1 G e n e r a l A p p l i c a t i o n A r c h i t e c t u r e . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 5 5 . 2 S c r o l l i n g . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 7 5 . 3 R e s p o n s i v e n e s s a n d H a r d w a r e A c c e l e r a t i o n . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 8 5 . 4 A u d i o R e c o r d e r . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 0 5 . 5 A u d i o P l a y e r . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 1 5 . 6 E x e c u t a b l e A p p l i c a t i o n S t r u c t u r e . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 2 5 . 7 A u t o - U p d a t e M e c h a n i s m . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 2 5 . 8 U I I c o n s a n d I m a g e s . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 4 5 . 9 U I L o c k i n g a n d S w i p e a b l e - B e h a v i o r . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 4 5 . 1 0 P o p o v e r s a n d D e c l a r a t i v e P r o g r a m m i n g S t y l e . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 5 5 . 1 1 B u t t o n s a n d H i t - T a r g e t s . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 6 5 . 1 2 L o g g i n g . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 6

6 O p e n S o u rc e a n d C o n trib u tio n s ............................................... 2 7 7 A p p lic a tio n R e le a s e ................................................................ 2 8 8 E v a lu a tio n .............................................................................. 2 9 9 F u tu re W o rk ........................................................................... 3 1 1 0 C o n c lu s io n ............................................................................. 3 2 1 1 R e fe re n c e s ............................................................................. 3 3 1 2 L is t o f F ig u re s ........................................................................ 3 4

1 Introduction
Designing and implementing a mobile web application is a demanding task. Mobile web browsers do not provide all the functionality that is readily available to native applications. They often have random bugs or poor performance and the available debug tools are only very basic. The focus is to create an application with a user experience that matches or exceeds a native application and preserves all the advantages of a web application. This goal is measured by whether regular users of the application notice a difference in performance, user experience or other subtle behaviors. The application is built from the ground up by the author alone using HTML, CSS and JavaScript only and is released as open source software. While it is most important to support as many devices as possible, it is wise to spend time on all platforms to optimize the functionality and performance. Creating experiences targeted at specific devices will always outmatch solutions that advertise full crossbrowser support. A common result of using web user interface (UI) libraries is that an applications user experience gets lowered to match the experience on the most low-end platform that is supported by the library. It was most important to control all parts of the user experience and as a consequence all user interface elements were designed and programmed specifically for the application. The application is developed together with the Auphonic team, a young startup from Graz, Austria.

1.1 About Auphonic

Auphonic is an automatic audio post-production web service invented by Georg Holzmann. The idea behind the service is that users only need to upload their audio files or videos and they will receive an optimized result with the best possible audio quality without setting complex parameters or spending much time. Auphonic automatically removes noise, filters disturbing low frequencies, normalizes the loudness and more. Auphonic provides the ability to import and export audio files from external services. A user can import a file from Dropbox1, process it through Auphonic and export it to SoundCloud 2 from the web interface or through the Auphonic Web Application Programming Interface (API) (Rattinger 2012). The Auphonic web service can save hours of post-production work. It helps to simplify the audio production workflow and makes audio distribution on the Internet easier.

1.2 State of Web Applications on Mobile Phones

There is an ongoing struggle between application developers on whether using web technologies or developing native applications is the better choice. There are at least three problems to consider. The first argument against moving to a web based application architecture is that support for certain features is limited. While this is true, mobile browser vendors are quickly catching up and the progress of mobile browsers in the last few years has been astonishing. Quite often mobile browsers
1 2

https://www.dropbox.com accessed 4 December 2012 https://soundcloud.com accessed 4 December 2012

5

such as Google Chrome on Android almost fully match the functionality of its desktop version. The second problem is that specifications on how to access data or sensors on mobile devices need to be first discussed and prototyped before they can be implemented. Third, when building an application inside of a browser, the browser engine acts as another abstraction layer, which in turn slows down the performance by a noticeable amount and often has a negative impact on perceived responsiveness of the application. This downside is mitigated by rapidly evolving device hardware. The advantages of web applications are extensive. First of all, if done right, a web application is truly cross-platform and can basically run on any device with access to Internet and an installed web browser. The same code base can be used to serve all platforms. Developers only need to know a single development stack and adding features or resolving an issue in the code automatically improve the experience of all users. In addition web applications can be deployed at any time without requiring App Store reviews or similar that slow down the deployment process. Projects such as Cordova (see Apache Cordova) provide device APIs not available in the browser and it adds abstractions for common functionality on all platforms. The advantages far outweigh the disadvantages. Especially since mobile browsers are getting better, using web technologies is certainly the future on all platforms, desktop or mobile.

2 Application Features
The application delivers a wide array of functionalities. A user can view, edit and manage his/her recordings, audio productions or presets. A production consists of an input file from an upload or an external service, one or more audio or video output files and optional metadata such as a title, genre or artist. Additionally, chapter marks can be defined, a cover photo can be uploaded or the applied algorithms can be selected and their parameters can be adjusted. A preset allows storing common metadata and settings for future audio productions. For example a preset for a series of audio interviews could contain some metadata and a cover photo. Presets simplify the production workflow particularly because data entry on mobile phones is often unwieldy. Through the seamless integration of all Auphonic functionalities, a user can create a preset using the web service and reuse it in the application. One of the goals of the application is to provide the ability to record audio or video and upload it to the Auphonic web servers. It was made easy for users to start productions and to playback or view the results directly on their mobile phones. All mobile recordings can also be directly exported and shared to services such as YouTube or SoundCloud. During the development the audio recording feature and recording quality turned out much better than anticipated and as a consequence the feature was made more prominent throughout the application. It was aimed to recreate all features provided by the Auphonic web service that are exposed via the Auphonic Web API. The API and the Application were developed at the same time and they complement each other. As a side effect, the Auphonic application is a useful functional test of the Web API and helped to improve the API stability during its initial development.
6

2.1 The Auphonic API

The Auphonic API makes use of the Representational State Transfer (REST) pattern and always returns objects using the JavaScript Object Notation (JSON) format (see Figure 2-1). Authentication is done through the OAuth23 protocol.
GET https://auphonic.com/api/productions.json { "status_code": 200, "form_errors": {}, "error_code": null, "error_message": "", "data": [ { "metadata": { "album": "Some Album" }, }, { "output_files": [], "algorithms": { "normloudness": false, "denoise": false, "loudnesstarget": -23, }, }, { "uuid": "9KN6czHvcrVeYWex5aQz59", }, ] }
Figure 2-1 API Example Response (simplified)

The API response example shows how to retrieve a list of productions. Because of its good readability and simplicity, JSON has become a widespread format and is commonly used for web APIs. More examples on how to query the API for data and how to create and start a production can be found in the API documentation4.

3 4

http://oauth.net/2 accessed December 4 2012 https://auphonic.com/api-docs accessed December 4 2012

7

3 Design and User Experience

The design is focused on recreating the still young and evolving Corporate Design of Auphonic and mixing it with mobile operating system specific styling to form a great user experience on both Android and iOS as well as on possible future platforms. The applications design evolved over the course of the application development as it was created on a per-need-basis during the implementation and is changed multiple times. After several months of development the design of the Auphonic website was altered and the styles between the mobile application and the website started to diverge. The Auphonic team encouraged exploring new design possibilities for the mobile application.

Figure 3-1 Android Design Prototype (left), iOS Design (right)

The above screenshots (see Figure 3-1) show the application design of the production detail view for both the Android and the iOS version. The view shows details such as the cover photo, an audio player and other metadata, output files and applied algorithms. As it can be seen, both designs are similar yet they are adjusted to feel familiar to users of either platform. The font varies based on platform; on iOS Helvetica Neue and on Android Roboto (not visible in the above screenshot) are used. The screenshots highlight the most important design differences: a heavy use of gradients, shadows and rounded borders was applied on iOS as opposed to a flat style on Android. Additionally the iOS version makes use of some advanced animations, which were disabled on Android to achieve better

performance. It is important to highlight that the same behaviors and functionality for both versions were used as much as possible. In fact, during development both designs can be switched by just toggling the CSS class names android and ios on the documents body element.

Figure 3-2 Audio Recorder and Production List View

The audio recording view (see Figure 3-2) is clean and simple. It provides a red button to start and pause a recording. The level meter on the top displays the average and peak level. On the bottom the user has the ability to add, edit and remove chapter marks. The other screenshot shows the production list view and how standard lists are rendered. For every list item the cover photo, the title and additional information are shown. An arrow on the right indicates that the item can be tapped for further information. The screenshot also shows how action buttons on the right of the title element are used for further interaction such as creating a new production. In specific cases when a user needs to be notified of an event such as a finished upload or in case of an error a notification is shown on the top of the screen. The element slides in from the top and when tapped it slides out to the left. If there is more than one notification to be shown, all notices stack up and the newest notification pushes down the existing ones.
Figure 3-3 In-App Notification

The production creation view (see Figure 3-4) shows further user interface elements. The dark box represents a popover that opens when a user taps an algorithm or other elements that provide optional information. Below the Popover in the figure, checkboxes can be seen. They are programmed in JavaScript and CSS and imitate the behavior of native checkboxes on iOS. The text next to the Loudness Target item is a select box. Select boxes only have subtle styling and use the native select implementation provided by the browser to allow the user to choose a predefined value. On the top right of the content area three different icons can be seen. The cross icon allows to erase all text in the input field to which it is attached to. It focuses the input field and opens the keyboard. This is especially useful for predefined text that the user likely wants to change. The plus icon indicates that a new entry can be added to a list. In this case it is used to add an audio output format or a chapter mark to the production. The arrow allows editing a list item.
Figure 3-4 Production Creation Form

It was most important to develop a resolution-independent design. The application can be used on any mobile phone screen size without further adjustments. Both iPhone screen sizes are supported since its launch. However, the application does not yet make use of bigger screens present in tablets and desktop computers. When a link gets clicked and there is no special action associated to it, it usually results in a view transition. This means that the current view is animated to the left and a new view slides in on the right. Additionally the action buttons and the title in the header animate too. In the screenshot (see Figure 3-5) a transition to the right from the production list view to the production detail view can be seen. The back button and the title slide and fade out to the left with the new title and back button sliding in from the right. The action button on the right top only fades in and out. Transitions in iOS were carefully analyzed and imitated to achieve the best possible animation quality.
Figure 3-5 View Transition

10

Another important feature common in iOS applications is swipe-to-delete in which a user swipes to the left or right on a list item to reveal a delete button (see Figure 3-6). This feature was completely recreated with a native look and feel. Upon pressing the button, the list item fades out to the left and gets deleted. Alternatively, for Android users, the same functionality is available by using a touchholdgesture in which the user taps an element for about half a second.
Figure 3-6 Swipe-To-Delete

Null-States (see Figure 3-7) are important for user experience. When there is no content to be shown because the user has just registered a new account and hasnt created any productions or presets yet, information is shown that explains what to do next. There is always an action button on the top right and a list in the content area with useful actions. NullStates are also shown again when a user deletes all items from a list.
Figure 3-7 Preset Null-State

For mobile web applications it is often hard to recreate a native user experience because the browser implementations lack in support for common features easily or automatically available to native applications. As an example the browser on iOS and several versions of the Android Browser employ a click-delay of about 300 milliseconds5. While this is hard to see for a regular user, it has a serious impact on the perceived performance of the application, especially since the delay is not present in native applications. Circumventing issues such as these was a main focus to allow serving the best possible user experience. Some implementation details on how common user interface elements are created and how user experience problems are solved are provided in the Implementation section.

3.1 Design Evolution

As stated before, the design evolved during the applications development. Almost all of the user interface elements are changed several times before the release. Even in early designs, icons from the Noun-Project6 are used and their use is expanded over time. The most notable difference (see Figure 3-8) is that there used to be no Home-Tab with the Auphonic logo. Initially it was planned to provide a Settings section. However, there are no useful settings so it was dropped entirely. The home screen is only shown to users once, which resulted in confused reactions. It is changed to make the home screen available as a landing view with additional features. Initially the tab bar does not show the tabs title, which was also confusing
5 6

https://developers.google.com/mobile/articles/fast_buttons accessed 4 December 2012 http://thenounproject.com accessed 4 December 2012

11

to users. As it can be seen in the New Preset screenshot, the design was very different initially because most UI elements such as content areas (and their shadows) and buttons are based on the Auphonic websites design at this time. The production and preset detail view used to be chaotic and unorganized. Putting the cover photo in the background and showing less information on the top helped to improve the overall look of the application. The audio player is also expanded to show more information such as chapter marks and the duration. Buttons were changed to be subtler and the header background was changed to a simple linear gradient. The font size for list items and headlines was made smaller to display more useful content on the screen. Overall, the design evolved to be modern, organized and more stylistic but still shows similarities with the Auphonic website.

Figure 3-8 Early Design Prototypes

4 Libraries and Specifications

The application is developed using web technologies. All application logic is written in JavaScript, the design is done using Stylus and HTML templates are used to render the application. In areas where mobile web browsers do not yet provide the desired functionalities, the Cordova API is used and extended to allow for audio recordings and file uploads.

4.1 CommonJS
Since there is no module system in JavaScript and most objects are available in the global scope there is often no clear dependency between files and objects. Over time the application structure becomes incoherent and increasingly hard to understand.

12

As a countermeasure and to improve the overall code structure and quality, the CommonJS module specification7 is used in all JavaScript files. The advantage of CommonJS is that it does not introduce new syntax constructs but it works with regular JavaScript functions: every file becomes a module with an id and receives an exports object on which objects or methods can be exported. Modules also receive a require function that can be used to require other modules by their id. This method returns everything that was put on the required modules exports object. The example below (see Figure 4-1) can be executed using NodeJS8 by running node main and will print Hello Max! .
// say.js exports.hello = function(name) { console.log('Hello ' + name + '!'); }; // main.js var say = require('./say'); say.hello('Max'); // Hello Max!
Figure 4-1 CommonJS Code Example

This approach is purely a code quality improvement that aims to increase readability of the code and to make future maintenance easier.

4.2 Apache Cordova

Cordova9 (formerly named PhoneGap) is a project that provides access to native device functionality not available in the web browser such as sensors and other APIs. It embeds the application in a so-called WebView, a window spanning over the whole screen, which only contains the browser without toolbars or an address bar but with special abilities provided by Cordova. Developers can use the Cordova JavaScript library without touching the native code on the platform such as Objective-C on iOS or Java on Android. Cordova has an extendable API that allows its developers to write native plugins that can interact with the browser and JavaScript.

4.3 MooTools
MooTools10 is a general-purpose application framework for web development. The framework aims to extend JavaScript with new functionality not present in the language. MooTools adds methods to the native data types available in JavaScript such as String , Function and Object . In addition it provides a Document Object Model (DOM) abstraction to simplify interaction with HTML elements, CSS styles and event listeners. The library has been customized specifically for Auphonic to support the CommonJS specification and unnecessary legacy browser fixes were dropped to reduce the code size.
7 8

http://wiki.commonjs.org/wiki/Modules/1.1.1 accessed 4 December 2012 http://nodejs.org accessed 4 December 2012 9 http://incubator.apache.org/cordova/ accessed 4 December 2012 10 http://mootools.net accessed 4 December 2012
13

4.4 Stylus
Stylus11 is a language abstraction on top of CSS that aims to provide features not (yet) available in CSS such as variables, mixins and syntax customizations. Stylus does not have a defined style but instead allows choosing from a variety of different ways to structure code. In a scalable project it is most important to set a strict style guide. It was decided to leave out all characters that do not add any value to the code. The resulting code style is simple and more visually pleasing (see Figure 4-2).
$color = #fff a.button display block width 90% height 30px color $color background #000 border-radius 10px &.red background #800 a.button { display: block; width: 90%; height: 30px; color: #fff; background: #000; -webkit-border-radius: 10px; border-radius: 10px; } a.button.red { background: #800; }

Figure 4-2 Stylus Code Example (left) and CSS Output (right)

Stylus comes with a companion project called Nib. It provides cross browser mixins to automatically add vendor prefixes, which is necessary because some browser engines use prefixes for CSS properties that have not been finalized through the standardization process. In the CSS output example it can bee seen that -webkitborder-radius was automatically added through Nib. Stylus helps to make the application styles manageable, modular and compact.

4.5 Other
The author also used parts of a self-created project known as PowerTools!12 It provides necessary functionality required for mobile web development such as touch event abstractions (mootools-mobile), history management (mootools-history) and other extras (class-extras, custom-event, dynamic-matcher, event-stack, formautogrow, scroll-loader, router, mootilities). The modular project is split up into several repositories on GitHub and all of the used parts have been modified to support CommonJS. The HandlebarsJS 13 library is used to create HTML templates to organize and separate HTML. It provides control-flow-statements, loops and other useful functionality through its own custom template language. Another set of small libraries is used around the project for tooling support such as building and compiling Stylus and JavaScript files. NodeJS is an excellent choice for command line utilities because it exposes a JavaScript engine to run on the
11 12

http://learnboost.github.com/stylus accessed 4 December 2012 http://cpojer.net/PowerTools accessed 4 December 2012 13 http://handlebarsjs.com accessed 4 December 2012
14

command line. The application requires compiling all JavaScript modules into a single file. To achieve this, the modulr-node14 project was used, which analyzes all CommonJS modules and generates a single file. It also provides a mechanism to decrease the file size by minifying the JavaScript before it is being deployed on a server. When developing for the application a developer only needs to know JavaScript to write any code related to the project instead of also requiring knowledge of a specific language such as Ruby or Python, which are commonly used for command line utilities around web based projects.

5 Implementation
A handful of interesting problems and solutions are picked to describe in detail in this section. The full source code of the project can be viewed on GitHub, where the full repository is shared as open source software (see Open Source and Contributions).

5.1 General Application Architecture

An architecture is created to make maintaining existing code and developing new features for the application easy. Several abstractions are designed so common functionality can be accessed and re-used by following the programming standards described below.
Controller.define('/production/{uuid}', function(req) { View.getMain().showIndicator(); API.call('production/{uuid}'.substitute(req)).on({ success: function(response) { var production = response.data; prepare(production, 'production', function(production) { var object = new View.Object({ title: production.metadata.title, content: UI.render('detail', production) }); View.getMain().push(object); }); } }); });
Figure 5-1 Code to Fetch and Display a Production

In the above code example (see Figure 5-1) the most important implementation paradigms can be found. It fetches and displays data for an audio production from the API. Most methods take functions as parameters that are executed as callbacks when certain actions are completed. The application itself is, similar to traditional web sites, navigated via Unified Resource Links (URLs). This means, that every button that displays a different view when it is tapped (as opposed to simple buttons like starting or pausing a recording) updates the location of the document. Using the
14

https://github.com/tobie/modulr-node accessed 4 December 2012

15

history management support in web browsers such changes can be observed and actions can be executed. The Controller object is responsible for history management. It receives a path that executes a function when the URL is matched. To communicate with the Auphonic API, an API module that abstracts away all API functionality was created. Executing API.call(resource) requests the resource from the Auphonic API and executes a success or error callback. In the success callback the requested resource is passed in for further transformation or display. The API module also automatically caches all requests so double fetching of data does not need to happen and the user experience does not suffer. To show a loading indicator the showIndicator method can be used. However, the indicator is only displayed if the operation takes more than half a second. Most API calls complete faster than that and showing a loading indicator immediately would give the user the perception of slowness while waiting once for about 300 milliseconds to receive data from a server is often imperceptible to the user. Through the API caching, subsequent navigation to the same URL will immediately display the view without any requests to the server. In the above example, the response data is used to render a template using UI.render . This method takes as arguments the name of a HandlebarsJS template and context data to render and return HTML code ready for display. Another important implementation is View . There is one main view in the application, which manages the content of the title bar and the main content area in the center of the application. A view-object (also named just view) gets pushed on a view-stack that resides in the main view. Every content section (Home, Productions, Recordings and Presets) forms one stack. When switching the section the current stack gets discarded and a new one is created. This solution allows managing deep hierarchies of sections; it handles animations and provides the back button mechanism. New view objects usually slide in from the right and the current view object slides out to the left. When pressing the back button the animation is reversed and the previous view object gets displayed. Even though the URL is changed back, the object is not re-rendered but instead re-used. This means that all input, scroll position and other state is preserved when navigating back. Additionally, view objects provide show and hide callbacks to set up or destroy resources. A view object does not need to be pushed onto a specific stack and it can be put on the current stack instead. For example the audio recorder is accessible from all but one tab and navigating to it, pushes the recording view onto the current stack. In the beginning several view objects were pushed on new stacks and therefore forcing an automatic section change. Because the user lost control over where in the application he navigated to, this was changed later during development to improve user experience. These programming structures form an implicit plugin system. Every view (and every URL) is self-contained and does not touch or influence other parts of the application. Since every module lives in its own scope the application can grow to any scale without negative implications as long as the described paradigms are followed.

16

5.2 Scrolling
Scrolling in web applications on mobile phones is not a trivial problem. In the past it was simply not possible to provide a good scrolling experience and to still implement a native-like user interface because there was no method to create scroll areas other than the scroll area of the main document. This means that the header and footer would also scroll with the content. This behavior is confusing and different from native applications. While there are numerous scrolling solutions written in JavaScript, none of them yield a real native-like experience and they often come with problems of their own. It is generally hard and nearly impossible to guess all the animation timings for momentum and over-scrolling and because of that, mobile web applications often distinguish themselves from native applications by using custom scrolling libraries that do not feel native to the platform. In iOS 5 the webkit-overflow-scrolling CSS property was introduced. An element to which this property is applied, receives all functionalities that a native ScrollView in the iOS Software Development Kit (SDK) has, however all the configurability is lost and some behavior is different enough to have a negative impact on user experience. First of all, a WebView on iOS over-scrolls. This means, that the document is not fixed on the screen and the document can be temporarily moved up and down. Second, an element with a scroll-area inside of a document does not have the over-scroll ability when its content is scrolled to the top or bottom. Third, tapping the status bar to scroll to top gets disabled when there are two scroll-areas on the page. It is a tough problem to solve but the solution that is found is surprisingly simple and ideal (see Figure 5-2). It turns out that iOS transparently transforms elements with the webkit-overflow-scrolling CSS property into ScrollViews. Because the document also acts as a ScrollView this results in having a two-level nested scroll area and both of them are in conflict.
UIScrollView *scrollView = (UIScrollView*)[[theWebView subviews] objectAtIndex:0]; scrollView.scrollsToTop = NO; scrollView.scrollEnabled = NO;
Figure 5-2 Code Example to Disable Scrolling

While the elements ScrollView cannot be accessed either through JavaScript or Objective-C, the main WebViews ScrollView (that of the document) can be retrieved. By turning the properties scrollsToTop and scrollEnabled of the main ScrollView to NO all of the aforementioned problems can be resolved. If there is only a single scrollable area, iOS automatically decides to delegate all functionality to it. Scrolling to top by tapping the status bar works as well and as over-scrolling of the document is disabled it works properly for the content area. This solution allows an arbitrary DOM element to be the main and only scrollable element in the application. Every view object has its own scrollable element but only one of them is visible at a time. When switching between sections in the application there is no animation and the current view gets replaced by the new one immediately. A bug is introduced in iOS 6, possibly the side effect of a performance improvement that causes the view that is going to be removed to flash for a short period of time. This behavior is very noticeable and disruptive to the user experience. As a temporary solution until Apple
17

fixes this issue, the view controller detects immediate view object replacements and attempts to reuse the current scrollable element by injecting the new content directly into the element and replacing the old content which does not cause the content area to flash. Scrolling was implemented in a way that perfectly imitates native behavior. However, the performance was still not on the expected level and it was possible for elements to suddenly become invisible until after the scrolling of an area had finished. Using images and deeply nested element structures often resulted in a slow performance. The next section discusses how these problems are resolved.

5.3 Responsiveness and Hardware Acceleration

The perceived responsiveness and the amount of time it takes to execute common tasks are both connected and related to performance. Since JavaScript is singlethreaded, the user interface can lock up at any time during processing which needs to be avoided at all costs. While good performance is often expected or even demanded, users commonly only notice unresponsive behavior. Code execution during active interactions with the application should be limited and heavily optimized. Another problem area is animations. CSS Transitions 15 and CSS Animations 16 , both currently W3C working drafts, were used exclusively for all animations in the application. Non-optimized animations can lead to frame drops and result in a bad user experience. By using a number of tricks, several layers (usually a collection of DOM elements) can be rendered using the Graphical Processing Unit (GPU). Animations will then be calculated by directly using the hardware instead of a software based abstraction. This can be triggered by adding transform: translate3d(0, 0, 0) to the element via CSS, which essentially tells the rendering engine to draw the element in a 3D-space. The translate3d value allows setting the elements position. By using three zero values the element is left at its original position. Because of its many side effects it is important to use this method only when absolutely necessary. The memory of the GPU on older devices can fill up quickly and result in the opposite effect of worse performance, frame drops and flickering. It is also not documented nor specified which CSS property and attribute combinations can trigger hardware acceleration and operating system updates need to be closely observed for changes. In the Auphonic application, this method is only used when it is found absolutely necessary. Since the main development mobile phone is an iPhone 4 a device with an outdated processor but with a high-density screen (High Dots Per Inch; HDPI) it was possible to monitor any negative side effects and countermeasures were taken to optimize performance to achieve the best possible experience. It also needs to be noted that possible flickering through the use of hardware acceleration does not in fact cause the flashing issue during the replacement of scroll areas that was mentioned previously as for testing purposes all hardware acceleration was turned off.

15 16

http://www.w3.org/TR/css3-transitions/ accessed 4 December 2012 http://www.w3.org/TR/css3-animations/ accessed 4 December 2012

18

If an element gets injected into the document, an animation can only be started after the element has been rendered. By delaying the start of the animation by one millisecond, the animation will be correctly applied. However another interesting discovery was that on older devices the insertion of elements with a lot of content such as text, links or images, could randomly take longer than expected. Thorough testing yielded that using a 50-millisecond delay on older devices shows the best results and usually does not drop any animation frames. This number is completely arbitrary but was chosen because of its limited impact on perceived responsiveness and frame drops can only be observed in very infrequent situations where the operating system is occupied with the processing of other tasks. As mentioned before, iOS and some versions of Android employ a 300-millisecond delay before executing a click event, which makes user interactions feel slow. To overcome this feature, touch events such as touchstart , touchmove and touchend are used in JavaScript to emulate click events. The abstraction is completely invisible and does not require code changes click event listeners are still attached directly to an element. This however leads to problems with scrollable areas. Because it is impossible to detect if scrolling is currently in process, clicks are not correctly canceled in very rare occasions when scrolling is interrupted or after scrolling has stopped. The :hover CSS selector works differently on mobile browsers and if certain CSS styles are changed on hover, the element requires two taps to execute its click function instead of just one. This was made so regular web sites where a menu would be shown on hover will also work in mobile browsers. Applying styles through CSS on hover is completely circumvented by a module called ActiveState . The modules responsibility is to detect the currently tapped element and to highlight it by adding and later removing the active CSS class on the element. This CSS class effectively replaces the hover pseudo class. Another responsiveness issue is presented with all input fields and the keyboard on iOS. When an input element gets tapped there is a delay (also about 300 milliseconds) until the keyboard is revealed. This also pointed out another problem: The keyboard on iOS cannot be shown without an explicit click on an element. As it can be seen in the code example (see Figure 5-3), this only works by using a legacy click event listener and by directly assigning the onclick attribute of the element. The code example assigns the listeners to a container element, which contains the input element and a label to describe the field. The trigger to reveal the keyboard is to focus the input element by calling its focus method.
var onClick = function(event) { event.preventDefault(); var input = this.getElement('input, select, textarea'); if (input) input.focus(); }; element.addEvent('click', onClick); element.onclick = onClick;
Figure 5-3 Code Example to Reveal the Keyboard

In iOS 6, the ability is added to show the keyboard without requiring user interaction. In the code example this can be seen by the addEvent(click, ) function call, which, as described before, automatically uses touch events to

19

overcome the click delay. It is found that sometimes the keyboard would close itself immediately for no reason between the touchend and actual click event. To solve this problem and to also support older iOS versions both event listeners were kept. As shown previously, some input elements that often contain pre-filled text, have a cross-icon attached to them. This icon is shown only when the associated input element contains a value and when tapping on it, it clears out the value of the field, focuses the element and shows the keyboard. The same idea to open the keyboard by applying two event listeners is used to achieve this effect. Because input elements can be cleared out by a single tap, this clearable behavior allows prefilling of input elements with no additional time-cost to the user if she wishes to replace the value. Through a handful of different measures and tricks the maximum responsiveness and the best possible performance are achieved on newer devices as well as older ones. User interactions respond quickly and do not lock up the application. Several small modules attach useful functionality to provide a rich user interface.

5.4 Audio Recorder

One of the main reasons on why it was necessary to use Cordova is to access device APIs not otherwise available in a mobile browser engine. There is currently no way to retrieve audio signals from a microphone through the browser. Even though work is done to implement the Media Capture and Streams API17, it is not yet deployed to any mobile operating systems web browser. Cordova provides a Media object responsible for playing back and recording audio. Additionally it allows access to the local file system to store and retrieve files via the Cordova File API. The audio recording and playback feature in Cordova is very basic and not well supported. There were a number of obstacles in implementing a high quality audio recording feature. First of all there are issues with the media-recording feature on older iPhones in which the recorder could not be initialized but the error was only thrown after recording has finished. An error was also thrown when the recording is interrupted by a phone call. It was unacceptable to release the application without being able to guarantee that a successful recording can be made. Second, there was no possibility to pause a recording - a basic feature of an audio recording hardand software. Third, it was only possible to record using the Waveform Audio File Format (WAV), which is uncompressed and results in very large file sizes that can take very long to upload on mobile network connections. Fourth, there was no way to show level meter information. All of these issues are resolved and implemented directly in the Cordova Media implementation in both native code and the JavaScript library. Extra steps are taken to throw errors as early as possible so they can be handled appropriately. A method for pausing a recording is added and the default audio recording file format is changed to Advanced Audio Coding (AAC), which is much smaller than WAV. The level meter information is exposed to the JavaScript library of Cordova.

17

http://www.w3.org/TR/mediacapture-streams accessed 4 December 2012

20

The audio recorder implementation in JavaScript consists of two parts. The AudioRecorder user interface object that sets up all user interface elements that are displayed in the recording interface. The CordovaAudioRecorder is an interchangeable object that implements the actual recording feature. It uses the Cordova JavaScript library to interface with the native implementation. In future, an implementation using the Media Capture API in the browser can be provided and selected automatically based on browser support. Through the user interface a recording can be started, paused or stopped. During an active recording all navigation buttons such as the footer, the back button and the action button in the title bar are hidden and a set of new user interface elements slide in from the bottom. They allow creating, editing and removing chapter marks and show the duration of the recording. The level meter is shown on top and it shows the peak and average loudness similar to how it would appear on audio recording hardware. There is also a video recording feature that uses a standard video recording interface, which was added for completeness and its use is currently being evaluated. A stable audio recording feature is implemented by extending Cordova. The changes were pushed back to the Cordova project and are adopted or are pending review (see Open Source and Contributions). It is planned to further extend the audio recording feature, for example by automatically setting the audio recording level or by giving users the ability to select their preferred audio format.

5.5 Audio Player

The implementation of the audio player is similar to the audio recorder. It has a user interface object AudioPlayer responsible for displaying the play and pause button, the waveform and the details such as duration and current chapter mark. It also provides the ability to seek within the audio file. The user interface is backed by two different implementations called WebAudioService and CordovaAudioService , which are responsible for playing back audio and seeking within a file. Initially only the Cordova Media object is used which posed the main drawback of requiring to download the full media file before starting playback. It locked up the user interface and does not provide the ability to cancel. This is clearly unacceptable and rendered the implementation almost useless. It is tried to use the web audio API available in most modern browsers. While the API is simple and works seamlessly it is a challenge to make correct use of it in Mobile Safari because the web audio implementation behaves unexpectedly and illogically. For example the seeking functionality requires a combination of unrelated event listeners, pausing and playing and try-catch behaviors to work correctly. It was decided to use the WebAudioService for remote files and CordovaAudioService for local files. The user will not notice any difference in behavior.

21

During playback the current chapter mark is shown on the bottom (see Figure 5-4). When a chapter changes, it slides to the left or right similar to the title in the header during a view transition. The waveform is seek-able via touch gestures and a popover shows the time information.
Figure 5-4 Audio Player

The audio player is implemented using two different audio playback providers. To overcome an issue with pre-loading audio files through the Cordova Media object, a web audio implementation is used for remote files. The audio player features advanced functionality such as the display of chapter marks and seeking. After creating the design for the Android version the audio players style is also used for iOS because it fits better into the application.

5.6 Executable Application Structure

The executable that is downloaded to the device through the operating systems distribution mechanism (i.e. Apple App Store or Google Play Store) only consists of a small set of application specific files. Among the Cordova implementation and the main Cordova JavaScript file only a small HTML file, the Splash image and Application icon, a CSS file with the base styles and images are included. The HTML file has a loading mechanism, which downloads the full application code and style from the Auphonic servers when the user opens the application. There are only three compiled files to be downloaded from the web server. The update mechanism is described in the next section.

5.7 Auto-Update Mechanism

When deploying software the update process often requires the user to manually download and install the application. This process is often lengthy and inconvenient. Users are faced with notifications or application badges and they are able to choose not to update specific applications. When developing software it is desirable to update all users as fast as possible and to eliminate older versions that may contain critical bugs. Ideally the update process is automatic and does not require user interaction. Programs such as Google Chrome provide an update mechanism that is invisible to the user. Unfortunately it is not possible to auto-update applications on iOS or Android yet and deploying a new application version can take several weeks on the Apple App Store. Web sites have the ideal update mechanism as they always automatically serve the latest version that is stored on the web server. However, this implicit feature comes at the cost of requiring a reload of the entire web site, which has unacceptable user experience implications on slow Internet connections. The Cordova project encourages developers to bundle their application resources with the executable and therefore losing all of these advantages. This approach was not followed. It is decided to only bundle a small set of files with the executable (described in the previous section) and store all application files on the Auphonic servers. The executable has a loading mechanism embedded which works as follows: 1. The user opens the application.

22

2. The application always downloads a file called Version.js . This file only contains the version as a Git commit hash, a SHA-1 hash, which is a string of 40 characters (version ID). 3. With this information the application downloads two more files called App{hash}.js and App-{hash}.css using the version ID obtained in the previous step. These two files contain all code and styles required to run the application. If the version ID has not changed since the previous start of the application, the files will be loaded immediately from the browser cache. If the version ID has changed or the user opens the app for the first time the files get downloaded and cached. 4. Upon loading these files successfully, the global entry point of the application is called via JavaScript. This entry point is named window.__BOOTAPP and its reference is removed after a successful start. The web server automatically serves all files using gzip-compression. Version 1.0.4 (third update to the Application; a week after the release) consists only of 11 KB compressed CSS and 64 KB compressed JavaScript. The described mechanism always downloads the latest application version or loads from the browser cache. Due to the small size of the application resources, starting or updating the application usually do not take more than four seconds, depending on the network connection. When loading from the browser cache the application starts within 1.5 seconds. There are however several things that cannot be updated from the server only. While, for example, the audio recording interface can be completely redesigned and the use of the device camera can be implemented purely by a server-side push, there are limitations. Such limitations include all changes to the native code on the platform. It is planned for a future version to allow the user to select her preferred recording settings such as audio encoding and sampling rate, which will require changes to the Cordova Media plugin and therefore will require a push to the App Stores. These changes are intentionally kept minimal and it is planned to not push executable updates more than about four times per year.

Figure 5-5 Version Numbers in Logs from November 1927

The chart above (see Figure 5-5) shows all log entries with an attached version number over time. As it can be seen only a single log entry from a single user did not receive the first version upgrade immediately, likely because the application hasnt been closed in the time the update was issued. The first two updates are made on 20 November (Version 1.0.2) and 22 November (Version 1.0.3) with critical bug fixes. These dates exactly reflect the color changes in the chart. The Auphonic application almost completely eliminated the dependency on App Stores for updates. Application updates are silent, fast and automatic for all users. Critical bug fixes and new features can be deployed at any time. Old application versions can be deprecated immediately after deploying a new version.

23

5.8 UI Icons and Images

Images, particularly those used for user interface elements, have the potential to increase a web applications size and to dramatically increase the load time. When referenced in CSS, images will only be downloaded once the CSS selector, which contains the definition of the image, is used by an element in the document. This requires another round-trip to the server and can result in missing images or flickering behavior for a short period of time. This is unacceptable for user experience. A number of measures were taken to reduce the applications size and the use of images. First of all, it was tried to always use a CSS solution instead of an image when possible. Common examples are gradients, shadows and rounded borders features available in CSS that do not need images to render. As a second step it was decided to only include images at twice the resolution, targeted at HDPI screens. By using the background-size CSS property images are scaled to their containing element. An image with a resolution of 64x64 pixels is scaled down to 32x32 pixels. On normal screens the image and the containing element are both of equal size (32x32 pixels). On HDPI screens the element and the image are drawn at 64x64 pixels but appear at half the size. While this approach does not result in a pixel-perfect design on regular screens the difference is negligible and most users are not likely to notice it at all. The advantage is that only half the images need to be drawn and maintained. A huge performance improvement is to embed all images into a single CSS file by using Base64 encoding. This means that when an element with a background image becomes visible in the application, the image does not need to be loaded from the server and can be displayed immediately. Lastly all user interface icons and images are bundled with the application binary. This way, all images are already pre-installed on the device and it was possible to drop 60 % of the application code that is served from the Auphonic server. If an image needs to be changed, it can be served through the server by overwriting the same CSS selector that was used by the previous image bundled with the binary. With this solution all user-interface elements can be rendered without a delay. The overall size and load time of all images was reduced to a minimum by omitting unnecessary images entirely and by bundling them with the binary. Preloading of images is implicit to this solution.

5.9 UI Locking and Swipeable-Behavior

Even though an application should never lock up completely and always respond to user input, there are several cases where user interaction should not be possible, particularly during animations. One such example is a transition between views. If a user touches anything inside of the application during an ongoing animation, the interaction should be discarded unless the animation has a cancelable behavior defined. There are numerous solutions to this problem but most of them require a lot of manual setup. The solution that was discovered automatically disables all event listeners on the document and its children when an animation is started. This works

24

through adding a pointer-events: none CSS style to the document. Since most CSS properties inherit their values from its parent element, applying this style to the document also applies the style to all of its children that do not have the property set explicitly. The pointer-events property allows disabling an element from receiving events even though the element might have event listeners attached to it. The use of this property is to make events fall-through to the element that is positioned behind another element and not specifically a parent of the overlaying element. This is especially useful to make areas that consist of large user interface elements such as shadows invisible to the event manager of the browser.
.disable-events * pointer-events none .enable-events pointer-events all
Figure 5-6 Code Example to Disable UI Elements

Through the functions UI.disable and UI.enable the CSS class disable-events from the example (see Figure 5-6) gets added or removed. In some cases it is necessary to disable large parts of the user interface but still enable a single button to receive events. Such an example is the swipeablebehavior. By using a swipe gesture on a list item, a button to delete the item is revealed. Once this button is shown, it can either be clicked or canceled by tapping outside of the button. When the button is visible, no other element in the document should receive events. The exceptions those elements that should still receive events can be passed as a second parameter to the UI.disable function. By adding the CSS class enable-events to an element inside of a container with disable-events applied, the pointer-events CSS property is overwritten and therefore the element preserves its ability to receive events. This solution is elegant and simple. It does not require manually written code inside of event handlers to check whether parts of the user interface are currently locked.

5.10 Popovers and Declarative Programming Style

Popovers are an interface element to show optional additional information or actions through little popups over the content area. They are shown when specific interface elements are tapped and animated using a cubic-bezier timing function that is not bounded between (0,1;0,1) to achieve a bouncing effect when they are opened and closed. Popovers can be added to a view by simply putting them into the HTML templates and they do not require additional JavaScript code (see Figure 5-7).
<label class="show-popover"> Show Favorite Quote <div class="hidden popover top" data-position="top"> <i>"If I had more time I would have written you a shorter letter"</i> -- Blaise Pascal </div> </label>
Figure 5-7 Popover HTML Example

25

This paradigm is a type of declarative programming. When the documents content changes, for example when a new view gets pushed onto the view stack, the new content is being searched for elements specified by CSS selectors such as div.popover . The popover object is then being instantiated and event listeners for opening and closing it are automatically attached and managed. Simple optional parameters can be provided by the use of data-attributes directly in HTML. This approach is practical for behavior that does not need complex configuration for every occurrence of the pattern. Behavior of certain HTML elements is applied automatically and used almost throughout the full application. Every link, form element and the audio player have behaviors attached to them and using these user interface elements does not require additional JavaScript code.

5.11 Buttons and Hit-Targets

An interesting mechanism to improve user experience is to make clickable areas such as buttons bigger than they appear to be. Because small user interface elements can be hard to tap, they can be given a larger invisible area surrounding them to make it easier to dispatch events. Good examples are buttons in the header. They are expanded to the edges of the screen as well as they overlay parts of the title text. When tapped (see Figure 5-8), they even expand their area, so when the user moves her finger outside of the button the tap is still registered in an expanded area. Native applications on iOS use a similar behavior for buttons.
Figure 5-8 Example of a Hit-Target

5.12 Logging
Before the release a logging system is introduced to log user behavior, devices and program errors. The API provides a special endpoint exclusively for the Auphonic application to store logging information. The JavaScript method to log data is API.log(data) which expects a JavaScript Object with arbitrary logging information. The following information is being logged: User logins JavaScript errors and API request errors Successful recordings and recording errors Successful uploads and upload errors All logging information is anonymised and regularly deleted. The purpose of logging is to detect programming errors and to understand how the application is being used. Through error logging several previously unknown issues are discovered and have been fixed and deployed since.

26

6 Open Source and Contributions

It is decided to release the project as open source and allow an insight into the development of the application from the beginning. GitHub was chosen to store the repository due to its popularity and simplicity. The repository18 is available through the Auphonic organization on GitHub. It is planned to use the Pull Request feature once more contributors start working on the application. Another positive side effect of making the applications source code available is for other developers to see how to implement a full API client for Auphonic in JavaScript and also for mobile web application developers to see and understand how to work around common browser issues. To help interested developers install, run and work on the application a readme file is provided in the repository. During development changes and fixes were made to other open source projects and most changes are contributed back to the main repositories or the effort is still ongoing at the time of this writing. Contributions to Cordova Fix Media.startRecord() failure19. Fix seek to position 020. Add ability to pause a recording21 (pending review). Expose level meter information22 (pending review). Encode audio files using AAC23 (work in progress). Given the multi-platform nature of the Cordova project, contributions are only likely to be accepted if they are made for multiple platforms. Auphonic is committed to continue pushing audio playback and recorder fixes so all developers who use Cordova can provide their users with the improvements made by the Auphonic team. Currently the audio recording feature for Android is being worked on to increase the likelihood of adoption by the Cordova team. Contributions to Stylus and Nib Ignore the file size limit and always inline images in the CSS output24. Fix for excluding Internet Explorer specific CSS25. Contributions to modulr-node26 and other changes Minify module identifiers to be as small as possible27.
18 19

https://github.com/auphonic/auphonic-mobile accessed 4 December 2012 https://github.com/apache/incubator-cordova-ios/pull/60 accessed 4 December 2012 20 http://git.io/IgzBNA accessed 4 December 2012 21 http://git.io/BxXCVA, http://git.io/Jn7k_g and http://git.io/MieeuA accessed 4 December 2012 22 http://git.io/aDrHrQ and http://git.io/2tOZbw accessed 4 December 2012 23 http://git.io/-Xy2LA accessed 4 December 2012 24 https://github.com/LearnBoost/stylus/pull/789 accessed 4 December 2012 25 https://github.com/visionmedia/nib/pull/136 accessed 4 December 2012 26 https://github.com/cpojer/modulr-node/commits/master accessed 4 December 2012 27 https://github.com/tobie/modulr-node/pull/28 accessed 4 December 2012
27

 Change the context of modules from global to module scope (change only). Preserve newlines in the compiled output files (change only). It needs to be noted that minifying the module identifiers from a 32-character string to about 2-character strings has resulted in a 5 KB reduction of gzip-compressed file size for the application. Depending on the amount of modules this effect can be even bigger and now every project, which uses modulr-node for building compiled JavaScript can benefit from this modification. Changes to MooTools Core28 CommonJS Compatibility Removal of unnecessary functionality such as legacy browser fixes Performance improvements Contributions to HandlebarsJS Add CommonJS export option for template compilation29.

7 Application Release
The application was launched free of charge for iOS 5 and higher on 19th November 2012 in the Apple App Store30 and an announcement was made on the Auphonic Blog (Pojer 2012). Facebook and Twitter were used as distribution channels to inform interested users of the launch.

Figure 7-1 Selection of Feedback on Twitter

28 29

https://github.com/cpojer/mootools-core/commits/1.5cjs accessed 4 December 2012 https://github.com/wycats/handlebars.js/pull/371 accessed 4 December 2012 30 https://itunes.apple.com/app/auphonic/id575204274?mt=8 accessed 4 December 2012
28

Figure 7-2 German App Store Reviews

User reviews on Twitter (see Figure 7-1) were continuously positive and the application received a 5-star rating on the German version of the App Store (see Figure 7-2). In addition several German podcasters have made video tutorials or mentioned the application in their podcasts.

8 Evaluation
The application was well received and downloaded more than 200 times in the first 48 hours. Half of the users have tested the recording feature and more than half of the recordings were uploaded and processed through the Auphonic web service. It was apparent that initially most users were only testing the recording feature. In the first week of its release only about 25 users have used the application for recordings longer than ten minutes. To increase the number of uploads the upload button was made more prominent in an update. The main takeaway is to wait and analyze the application usage over a longer period of time. It is believed that users are unlikely to change their recording habits but more and more professional equipment and adapters are becoming available for mobile devices and the usage of mobile phones for professional recordings will definitely increase. The Auphonic team will focus on further improving the recording feature and pushing users to use the application for all kinds of recordings. The opportunity is particularly big when it comes to engaging new podcasters because they are not likely to have professional equipment. All project goals are reached or exceeded. There are zero user complaints about the design, functionality or behavior, which indicates that almost no users noticed that the application was built as web application. This highlights that it was the right approach to focus on user experience and to recreate useful behaviors native to the supported platforms. It was certainly the right approach for Auphonic to build a mobile web application instead of a native application. It turned out positive to only focus on the most recent and previous version of iOS. Only four users were using iOS 5 and only seven users were not on the latest

29

version of iOS 6. Efforts to optimize the experience for older devices and older versions of iOS would have been a bad time investment. The commit graph (see Figure 8-1) shows when the most changes were made to the code. There were more than 500 commits since April 2012 31 . The amount of commits peaked prior to the release in November. It was estimated about one third to half of the development time could have been reduced by using an existing (user interface) library. This would have likely resulted in a much bigger code size, less programming freedom and a worse user experience through standardized web components instead of native-like interface elements. Since time was not a constraint, this was a non-issue.

Figure 8-1 Git Commits over Time (Graph from GitHub)

Because the application is primarily about its user interface there were no tests written during development. It was deemed too time expensive to create and modify functional tests during development so all testing was done manually. It was decided to start writing tests once the design is unlikely to receive big updates. Finally the application gave more visibility to Auphonic, introduced new users to the service and yielded new use cases such as voice recordings or interviews.

31

https://github.com/auphonic/auphonic-mobile/graphs/contributors accessed 4 December

2012

30

9 Future Work
There are several major feature updates planned. As a first step the other members of the Auphonic team will be involved more heavily in the applications development, especially focusing on the audio recording feature. A main goal is to implement all necessary features in order to replace other applications or recording equipment through Auphonic alone. There is a large amount of ideas that aim at improving the design, user experience, functionality as well as other experimental features such as resumable file uploads or a complete redesign of the application. An important change to be made is to allow people to use and try out the application without registration, which was not implemented due to a time constraint. The next step for the application is to launch for Android on the Google Play Store. During development it was focused on launching on the first platform as fast as possible. Time resources are shifted to the iOS version as implementing and testing proceeded quicker and the Android version seemed to require more work. At the time of this writing the Android version is in the final testing stages and will be launched with additional features not available in the iOS version such as file uploads from other applications. It is important to note, that it is one of the goals to release the application only once a certain quality standard is met instead of publishing the application unfinished and unpolished. Another important feature that is already in development is a full audio sequence editor where a user can edit her recordings by cutting mistakes or adding sound effects. Naturally it is planned to create a design for bigger screens such as tablets. It is mainly a design problem to release the application for the iPad or similar devices. Once a suitable design has been prototyped it will be focused on making it deviceindependent. This opens the door to a downloadable application release for desktop computers. In addition it will be continuously evaluated which platforms it is worth deploying to. The most important one to be monitored at the moment is Windows Phone 8 and whether it gains a significant market share in the near future. It is also intended to continue contributing code to Cordova, so other web developers can benefit from improved recording features. Several problem solutions that are discovered are likely of interest to the mobile web developer community. It is planned to write articles on specific solutions to educate and support other developers. A target for the distant future is to share the applications code with the Auphonic web service to unify code bases, be able to iterate more quickly and to deploy code for both the application and the web service simultaneously.

31

10 Conclusion
It is shown that engineering a modern, fast and scalable mobile web application with a native-like user experience and design is feasible and the right approach given the limited available resources. Development was driven by design, fully controlled and specifically created for the application. Solutions were found for all problems that would have had a negative impact on user experience. The release for iOS is successful and well received. Through the automatic update mechanism new features and bug fixes can be deployed without requiring user action and at the time of this writing four updates have already been pushed to users. Several people are already using the Auphonic application for their podcasts and audio interviews. A strong effort will be put into making the application available to an even bigger audience. While the project itself was released as open source software, a number of contributions were also made to other open source projects and changes to Cordova will continue to be pushed. The applications release for Android is imminent. Future developments will include a better audio recorder, a full audio sequence editor and release on tablets and other mobile operating systems. Web technologies are certainly the future on all platforms, desktop or mobile, and web browsers are quickly catching up with new features. For now, the Cordova project is a helpful tool for all mobile web developers as it circumvents limitations of mobile browsers and gives more control over the behavior of a program. It is advised to use Cordova for mobile web development until web browsers are equally rich in features.

32

11 References
[1] Pojer, Christoph. Auphonic Mobile App for iOS. 11 19, 2012. https://auphonic.com/blog/2012/11/19/auphonic-mobile-app-ios/ (accessed 11 21, 2012). [2] Rattinger, Andr. Implementing the Auphonic Web Application Programming Interface. December 2012.

33

12 List of Figures
F ig u re 2 -1 A P I E x a m p le R e s p o n s e (s im p lifie d ) .................................. 7 F ig u re 3 -1 A n d ro id D e s ig n P ro to ty p e (le ft), iO S D e s ig n (rig h t) ........... 8 F ig u re 3 -2 A u d io R e c o rd e r a n d P ro d u c tio n L is t V ie w ........................ 9 F ig u re 3 -3 In -A p p N o tific a tio n ........................................................ 9 F ig u re 3 -4 P ro d u c tio n C re a tio n F o rm ............................................ 1 0 F ig u re 3 -5 V ie w T ra n s itio n ............................................................ 1 0 F ig u re 3 -6 S w ip e -T o -D e le te .......................................................... 1 1 F ig u re 3 -7 P re s e t N u ll-S ta te ......................................................... 1 1 F ig u re 3 -8 E a rly D e s ig n P ro to ty p e s ............................................... 1 2 F ig u re 4 -1 C o m m o n J S C o d e E x a m p le ............................................ 1 3 F ig u re 4 -3 S ty lu s C o d e E x a m p le (le ft) a n d C S S O u tp u t (rig h t) .......... 1 4 F ig u re 5 -1 C o d e to F e tc h a n d D is p la y a P ro d u c tio n ........................ 1 5 F ig u re 5 -2 C o d e E x a m p le to D is a b le S c ro llin g ................................ 1 7 F ig u re 5 -3 C o d e E x a m p le to R e v e a l th e K e y b o a rd .......................... 1 9 F ig u re 5 -4 A u d io P la y e r ............................................................... 2 2 F ig u re 5 -5 V e rs io n N u m b e rs in L o g s fro m N o v e m b e r 1 9 2 7 ............. 2 3 F ig u re 5 -6 C o d e E x a m p le to D is a b le U I E le m e n ts ........................... 2 5 F ig u re 5 -7 P o p o v e r H T M L E x a m p le ............................................... 2 5 F ig u re 5 -8 E x a m p le o f a H it-T a rg e t ............................................... 2 6 F ig u re 7 -1 S e le c tio n o f F e e d b a c k o n T w itte r .................................. 2 8 F ig u re 7 -2 G e rm a n A p p S to re R e v ie w s .......................................... 2 9 F ig u re 8 -1 G it C o m m its o v e r T im e (G ra p h fro m G itH u b ) .................. 3 0

34