Development of a cross-platform Code

Coverage calculation tool for automated and

interaction based testing using Playwright,
Cypress and Electron

A Colloquium report based on Summer Internship

IPG-MTech

by

Abhishek Garain(2017IMT-002)

ABV INDIAN INSTITUTE OF INFORMATION

TECHNOLOGY AND MANAGEMENT
GWALIOR-474015

2021
 1

CANDIDATES DECLARATION

I hereby certify that the work, which is being presented in the report, entitled Develop-
ment of a cross-platform Code Coverage calculation tool for automated and inter-
action based testing using Playwright, Cypress and Electron, submitted to the insti-
tution is an authentic record of my own work carried out during the period 22 February
2021 to 30 June 2021. I have also cited the reference about the text(s)/figure(s)/table(s)
from where they have been taken.

Date: 13th September 2021 Signatures of the Candidate

Date:August 08 , 2021

Employee Name: Abhishek Garain

Employee ID: 300054132

Dear Abhishek Garain,

This is in furtherance to your resignation dated June 30 , 2021, wherein you had requested to be relieved from the services.

Kindly note we have completed all your Full & Final Settlement formalities and you are relieved from the services of “Rakuten India Enterprise Private
Limited” with effect from June 30 , 2021.

We thank you for your association with Rakuten India. We wish you great success in your future endeavours.

Your service record with the company is as follows:

Date of Joining Rakuten India Enterprise Private Limited: February 22 , 2021

Last Working Day at Rakuten India Enterprise Private Limited: June 30 , 2021
Last Designation: Intern

For Rakuten India Enterprise Private Limited

*Nisha Verma

Manager - Human Resources

nisha.verma@rakuten.com

*This is a system generated letter and does not require any signature
 3

ABSTRACT

Code coverage becomes an important aspect in real life software development. It

is not only considered as a good practice but also becomes a necessity to generate and
maintain a threshold of code coverage, below which, the team of developers needs to
work on the testcases to make sure that every part of the code is tested and there is
absence of dead code in the project.
The internship project mainly consisted to two tasks, the first one being, building an
automated code coverage calculation tool that can track the code coverage of a project
in real-time or based on automated testcases, the second task being, integrating the code
coverage tool with Rakuten’s existing software testing platform, Atssa2.0.
Keywords: Code Coverage, Testcases, Testsuite, Atssa2.0, Cypress, JSON
 4

ACKNOWLEDGEMENTS

I take this opportunity to express my profound gratitude and deep regards to all my men-
tors for their exemplary guidance, monitoring and constant encouragement throughout
the course of this internship. The blessing, help and guidance given by them time to
time shall carry me a long way in the journey of life on which I am about to embark. I
also take this opportunity to express a deep sense of gratitude to my internship mentors
Mr. Madhusudan Ganda(Senior Engineering Manager, GATD Team, Rakuten India),
Mr. Rakhsit Arora(QA Lead, GATD Team, Rakuten India), Mrs. Sona Ojus(Software
Engineer, GATD Team, Rakuten India) for their cordial support, valuable information
and guidance, which helped me in completing the tasks through various stages.

(Abhishek Garain)
TABLE OF CONTENTS

ABSTRACT 3

LIST OF TABLES 6

LIST OF FIGURES 7

1 Company Profile 10
1.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
1.2 About the GATD team . . . . . . . . . . . . . . . . . . . . . . . . . . 11
1.2.1 About Atssa2.0 . . . . . . . . . . . . . . . . . . . . . . . . . . 12
1.3 Mission and Vision . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
1.3.1 Always Improve, Always Advance . . . . . . . . . . . . . . . . 12
1.3.2 Passionately Professional . . . . . . . . . . . . . . . . . . . . . 12
1.3.3 Hypothesize → Practice → Validate → Shikumika . . . . . . . 12
1.3.4 Maximize Customer Satisfaction . . . . . . . . . . . . . . . . . 13
1.3.5 Speed!! Speed!! Speed!! . . . . . . . . . . . . . . . . . . . . . 13
1.4 How Rakuten is transforming India . . . . . . . . . . . . . . . . . . . . 13

2 Problem Statement and Objectives 14

2.1 Problem Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
2.2 Objectives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

3 Literature Review 16
3.1 Code Coverage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
3.2 Importance of Code Coverage for a project . . . . . . . . . . . . . . . . 16
3.2.1 Examples of why code coverage is important . . . . . . . . . . 17
3.3 Types of Code Coverage metrics used in a project . . . . . . . . . . . . 17
3.4 Existing tools and platforms . . . . . . . . . . . . . . . . . . . . . . . 18
3.4.1 BrowserStack: . . . . . . . . . . . . . . . . . . . . . . . . . . 18
3.4.2 Cypress: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
3.4.3 Zalenium: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
3.4.4 Disadvantages of existing platforms: . . . . . . . . . . . . . . . 19

5
TABLE OF CONTENTS 6

3.5 Why Desktop app? . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

3.6 Testcases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
3.7 Testsuites . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
3.8 Types of testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
3.8.1 Automation Testing . . . . . . . . . . . . . . . . . . . . . . . . 20
3.8.2 Manual Testing . . . . . . . . . . . . . . . . . . . . . . . . . . 20
3.8.3 Unit Testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
3.8.4 Integration Testing . . . . . . . . . . . . . . . . . . . . . . . . 21
3.8.5 Regression Testing . . . . . . . . . . . . . . . . . . . . . . . . 22
3.8.6 Performance Testing . . . . . . . . . . . . . . . . . . . . . . . 22
3.8.7 Load Testing . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

4 Methodology 23
4.1 Tools and Techniques . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
4.1.1 Cypress . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
4.1.2 Instrumentation of code . . . . . . . . . . . . . . . . . . . . . 24
4.1.3 IstanbulJS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
4.1.4 nyc . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
4.1.5 Playwright . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
4.1.6 XtermJS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
4.1.7 Jenkins . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
4.1.8 ElectronJS . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
4.1.9 ReactJS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
4.1.10 NodeJS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
4.1.11 ngrok . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
4.1.12 Docker . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
4.2 System Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
4.2.1 Steps to achieve the final code coverage reports . . . . . . . . . 28
4.2.1.1 Instrumenting customer’s code . . . . . . . . . . . . 28
4.2.1.2 Choice of testsuite: Manual testcase execution . . . . 28
4.2.1.3 Choice of testsuite: Automated testcase execution . . 30
4.2.1.4 Generation of code coverage reports . . . . . . . . . 30
4.2.2 Sending the reports back to the cloud platform . . . . . . . . . 33

5 Results and Conclusion 34

5.1 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
5.2 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34

REFERENCES 35
LIST OF TABLES

1.1 Facts at a glance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

7
LIST OF FIGURES

1.1 Rakuten’s Logo . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

1.2 Rakuten x FC Barcelona . . . . . . . . . . . . . . . . . . . . . . . . . 11

2.1 Problem Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

3.1 Knight Capital Group’s stocks went down within minutes . . . . . . . . 17

3.2 The JSON coverage variable . . . . . . . . . . . . . . . . . . . . . . . 19
3.3 Unit Tests in Cypress test suite . . . . . . . . . . . . . . . . . . . . . . 21
3.4 Integration Tests in Cypress test suite . . . . . . . . . . . . . . . . . . . 22

4.1 Cypress’s dashboard . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

4.2 Playwright . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
4.3 Jenkins pipeline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
4.4 Tunneling localhost:3000 with ngrok . . . . . . . . . . . . . . . . . . . 26
4.5 Flowchart of the system . . . . . . . . . . . . . . . . . . . . . . . . . . 27
4.6 The code coverage app . . . . . . . . . . . . . . . . . . . . . . . . . . 28
4.7 Script to instrument code . . . . . . . . . . . . . . . . . . . . . . . . . 28
4.8 The out.json file . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
4.9 Command to generate the reports . . . . . . . . . . . . . . . . . . . . . 31
4.10 Code coverage report in HTML format having 50% code coverage . . . 32
4.11 Code coverage report in HTML format having 100% code coverage . . 32
4.12 Code coverage report in text format . . . . . . . . . . . . . . . . . . . 33
4.13 Structure of the folder . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

8
LIST OF FIGURES 9

ABBREVIATIONS

PoC Proof of Concept

PM Product Manager
EM Engineering Manager
KT Knowledge Transfer
LOC Lines of Code
GATD Global AdTech Team
CHAPTER 1

Company Profile

Figure 1.1: Rakuten’s Logo

This chapter includes the profile of Rakuten, Rakuten India and the Global AdTech
Team at Rakuten India of which I was a part of, during my four months internship.

1.1 Introduction
Rakuten Group, Inc. is a Japanese electronic commerce and online retailing company
based in Tokyo, founded by Hiroshi Mikitani in 1997. Centered around Rakuten Ichiba,
its a business-to-many E-Commerce platform and one of the largest e-commerce sites
in Japan, its businesses include financial services utilizing fintech, as well as digital
content and communication services such as messaging app Viber, e-book distribu-
tor Kobo, and Japan’s fourth mobile carrier Rakuten Mobile. Rakuten has 18,000+
employees worldwide, operating in 29 countries and regions, and its revenues totaled
US$7.2 billion with operating profits of about US$347.9 million as of 2016. It is often
referred to as "the Amazon of Japan".
Its major investments include Buy.com (now Rakuten.com in the US), Priceminister,
Ikeda, Tradoria, Play.com, Wuaki.tv, Ebates, Viki, The Grommet. The company also
has stakes in Ozon.ru, AHA Life, Lyft, Cabify, Careem, Carousell and Acorns. [1]

10
CHAPTER 1. COMPANY PROFILE 11

Founded 7 February 1997

Type Public
E-Commerce, Cellular Networks, TV and Entertainment,
Services
AI, Contextual Ad platform, E-books
Rakuten Ichiba, Rakuten Networks, Rakuten Viki,
Products
Rakuten Advertising, Rakuten Kobo, Rakuten TV and many more
Headquarters Setagaya, Tokyo, Japan
Indian Office Crimson House Bangalore, Karnataka, India
Table 1.1: Facts at a glance

Figure 1.2: Rakuten x FC Barcelona

1.2 About the GATD team

The Global AdTech team mostly consists of SDEs and QA members working on the
displaying and managing contextual Ads on Rakuten Ichiba, the E-commerce website
of Rakuten. Along with working on the Ads platform, they are also working on some
projects that are yet to be released in the market. One such project is Atssa2.0 that I
was working on during the internship.
CHAPTER 1. COMPANY PROFILE 12

1.2.1 About Atssa2.0

Atssa is codeless software automation testing platform built with keeping Dev and QA
engineers requirement at the core. It enables them to perform functional testing, perfor-
mance testing and model-based testing throughout the life cycle of the product/project.
The testing can be done across multiple operating systems and browsers. Atssa comes
fully packed with features and can perform multiple tests efficiently and effectively on
a single-click.
Atssa is a powerful platform for automation because the users can create and execute
hundreds of tests on hundreds of machines simultaneously. The platform also has auto
tag suggestion feature that can help create tests fast and efficiently. Atssa also makes
it easier to manage tests, while also providing feature rich test reports with screenshots
and detailed test data about passing and failing of tests.

1.3 Mission and Vision

Rakuten Group has an unswerving commitment to building an egalitarian society by
empowering individuals and companies to be successful in business and in life. The
Five Principles for Success are designed to keep each of us on track, to ensure we make
continuous progress toward our ultimate goals. [2]

1.3.1 Always Improve, Always Advance

There are only two kinds of people in the world: Best Effort people are satisfied with
the status quo; when they fail, they console themselves by saying "I did my best." Get
Things Done people are absolutely committed to reaching their goals. With enough
determination and effort by being a Get Things Done person, you can achieve anything.

1.3.2 Passionately Professional

Rakuten is an organization composed of true professionals. Real professionals in any
field whether in sports, the arts, science, business or anything else, only reach the top
because they have the self-discipline to develop and improve themselves continuously.
Only by consistently working many times harder than our competitors can we consis-
tently win.

1.3.3 Hypothesize → Practice → Validate → Shikumika

Success in business depends on making and executing concrete specific action plans.
CHAPTER 1. COMPANY PROFILE 13

1.3.4 Maximize Customer Satisfaction

Rakuten is first and foremost a service company. We’re proud to serve our customers
and we ceaselessly seek to improve their experience.

1.3.5 Speed!! Speed!! Speed!!

The internet makes the window of opportunity for every small business. Accomplish
in one month what it takes other companies a year to do, because we can only win by
being many times faster than our competitors.

1.4 How Rakuten is transforming India

Rakuten India is the development centre and key technology hub of the Rakuten Group,
Inc. They enable businesses with Rakuten Inc’s depth of knowledge in multiple streams
of technology such as Mobile and Web Development, Web Analytics, Platform Devel-
opment, Backend Engineering, Data Science, Machine Learning, Artificial Intelligence
and much more. Their unique 24/7 support center ensures reliability and sustenance of
the Rakuten Ecosystem. With dedicated centres of excellence for Mobile Application
Development, Data Analytics, Engineering, DevOps and Information Security, they en-
sure the success of multiple units of Rakuten Group, Inc. With 1000+ employees and
growing, Rakuten India is housed in Crimson House Bangalore in the heart of the city.
[3]
CHAPTER 2

Problem Statement and Objectives

2.1 Problem Statement

Figure 2.1: Problem Statement

This chapter deals with the problem statement that acted pivotal for the internship
project and the objectives set forth and accomplished during the internship period. The
internship mainly focused on one problem to solve: There are no available platforms
that can find code coverage based on real-time software testing.

The project majorly consists of two goals that we had to achieve.

• To create a codeless automated code coverage calculation tool that can measure
code coverage for any particular project based on manual testing or automation
testing.

• To integrate the code coverage tool with the existing Atssa2.0 platform for pre-
serving the code coverage reports for all the testcases present in the portal.

14
CHAPTER 2. PROBLEM STATEMENT AND OBJECTIVES 15

2.2 Objectives
• To develop a method to fulfill the requirements.

• To choose the tech stack to develop the project based on high level research and
PoC.

• To choose the platform on which the tool needs to be developed based on discus-
sion with PMs and stakeholders.

• To design the high level system architecture of the whole project and get it ap-
proved from senior EMs and leads.

• Prepare documentation and KT sessions for team-mates.

CHAPTER 3

Literature Review

The sole purpose of doing an extemsive literature review of all the available technolo-
gies was needed to create a market viable product that can be shipped easily to the
customers without any hassle. Following are the topics that were explored in-depth
during the course of the internship.

3.1 Code Coverage

Code coverage is a metric that helps us to understand how much of the codebase was
tested and which part of the codebase is not used by any feature. It’s an important
feature to help us assess the quality of the testcases and quality of the code. [4]

3.2 Importance of Code Coverage for a project

Before building any project based on an important software engineering metric, we
need to understand the importance of that particular metric and why it is widely used
in the industry.
The importance of code coverage for any project are: [5]
• Find out which parts of the codebase is covered by tests and what is not: This
will help up to write testcases for uncovered code.

• Find out code execution paths that are missed.

• High code coverage points to a well-written and testable code.

• This will also enforce a culture of writing unit tests using code coverage rules.

• A high code coverage leads to confidence in code written by developers.

• A high code coverage is crucial to investors to pitch them that we are not wasting
lot of capital in maintaining big piece of code.

16
CHAPTER 3. LITERATURE REVIEW 17

• A high code coverage matters to some potential customers and brings confidence
in them.

• A low code coverage could lead to code refactoring making it more optimal.

• Code coverage reports also signifies whether tests are executed or not.

3.2.1 Examples of why code coverage is important

• A famous quote by Edsger Djikstra, the person behind the famous Djikstra’s
shortest path algorithm says that, Testing shows the presence, not the absence
of bugs, so we can understand that writing unit tests is not enough to find pres-
ence of bugs.

• Knight Capital Group, a HFT firm lost $460m in 45 minutes on 01 August, 2012
due to deployment of a new algorithm. While the algorithm was perfectly tested
to check for the presence of bugs, what they forgot was, removing old pieces
of code, and that interfered with the newly added algorithm and that brought
down the servers during the peak business hours. The RCA report stated that
their mistake was to not have any code coverage calculation tool that could have
reported them the correct usage of the code before deployment of the algorithm.
[6]

Figure 3.1: Knight Capital Group’s stocks went down within minutes

3.3 Types of Code Coverage metrics used in a project

1. Statement Coverage: Statement Coverage helps to find the coverage related
to the smallest possible units available in the code such as assignments, return
statements, assertions, loops and conditionals.
CHAPTER 3. LITERATURE REVIEW 18

2. Branch Coverage: Branch Coverage evaluates the of logical branches covered

in the code such as, if-else block.

3. Function Coverage: Function Coverage refers to the number of functions present

in the code that were tested.

4. Line Coverage: Line Coverage refers to the number of lines of code evaluated
by the tests. [7]

3.4 Existing tools and platforms

While creating a new platform to achieve a goal and release it for the end-users, we did
a lot of market research to find out the potential opponents and shortcomings of their
platforms that we were going to address in our platform.

3.4.1 BrowserStack:
BrowserStack is an Indian cloud web and mobile testing platform providing develop-
ers the power to test their websites and mobile applications across different types of
browsers in different platform in real-time. BrowserStack is a leading platform for
testcase execution but it doesn’t provide any feature to calculate code coverage for the
testcases. [8]

3.4.2 Cypress:
Cypress is a newly launched front-end testing tool built for the modern web. They
address the key difficulties developers and QA engineers face when testing modern web
applications. They make it possible to set up tests, write tests, run tests, debug tests.
It is often compared to Selenium and other browser automation framework, however
Cypress is both fundamentally and architecturally different. [9]
Cypress provides this library called @cypress/instrument-cra that helps to instrument
code written in ReactJS.

3.4.3 Zalenium:
Zalenium is a flexible and scalable container based Selenium Grid with video recording,
live preview, basic auth & dashboard. Zalenium is a very powerful platform to execute
testcases but at the same time, to generate code coverage reports, we need to write
scripts manually. [10]
CHAPTER 3. LITERATURE REVIEW 19

3.4.4 Disadvantages of existing platforms:

The already available platforms are good enough to execute already available testcases
in a testsuite and find the code coverage of the project after the testcase execution has
completed.
But the only disadvantage of these platforms are, they can’t help to find the code cov-
erage of a project based on manual testcase execution and find code coverage based on
real-time testcase execution.
To address this particular issue, we had to create our desktop app that can track the code
coverage JSON object in real-time and generate the code coverage reports as and when
the user interacts with the project.

3.5 Why Desktop app?

As mentioned previously, we did an intensive PoC before coming to the point that we
need to use a desktop app to achieve the code coverage reports, some of the reasons to
choose a desktop app as a development platform rather than a website were:

• The JSON coverage variable resides in the devtools of the browser but normal
browsers can’t provide a programmatic interface to access those variables.

Figure 3.2: The JSON coverage variable

CHAPTER 3. LITERATURE REVIEW 20

• To achieve this feature, we need to trigger a new browser programatically, such

that we get complete access of the devtools and browser variables.

• So, we need a platform to trigger our own browser, and that’s where the desktop
app comes into picture.

• Also, we had to execute some scripts in the user’s syatem, and this couldn’t be
done from a browser as we get access to a virtualized environment in a browser
but we can easily get access to user’s system through a desktop app.

3.6 Testcases
A test case is a set of steps that are performed on a project to determine if it satisfies
software requirements and functions correctly. The purpose of a test case is to deter-
mine if various features within a system are performing as desired and to confirm that
the system satisfies all related standards, guidelines and stakeholder requirements. The
process of writing a test case can also help reveal errors or defects within the system.

3.7 Testsuites
Testsuites are softwares that are used to run automated testcases, it helps to track which
particular testcases are currently running, the testcases that have already completed
execution and the testcases that are yet to be executed.

3.8 Types of testing

There are various types of software testing methodologies that have different purpose
to solve in the software engineering world. They are:

3.8.1 Automation Testing

The use of software tools to control the execution of tests, compare their outcomes, and
report functions to avoid manual activities is called automation testing.
So, the tests will be written by developers or the QA engineers and some sort of testsuite
will be used to execute the testcases.

3.8.2 Manual Testing

Manual testing is a software testing process where, testcases are manually executed
without the need of any automated tool. All testcases are manually executed by the
CHAPTER 3. LITERATURE REVIEW 21

tester according to the end user’s perspective. It ensures whether the application is
working as mentioned in the requirements document or Jira tickets.
So, the tests will be written by developers or the QA engineers and it will be the re-
sponsibility of the QA engineers or the developers to execute the testcases manually to
check for performance degradation or presence/absence of bugs in the project.

3.8.3 Unit Testing

Unit Testing is a type of software testing where individual units or nuclear components
of a software are tested. The purpose is to confirm that each unit of the software code
performs as expected. Unit Testing is done during the development of an application
by the developers.

Figure 3.3: Unit Tests in Cypress test suite

3.8.4 Integration Testing

Integration Testing is defined as a type of testing where individual software modules are
integrated logically and tested in one go. A typical software project consists of multiple
software modules, coded by different programmers. The purpose of this level of testing
is to find out if features are breaking after individual components are merged.
CHAPTER 3. LITERATURE REVIEW 22

Figure 3.4: Integration Tests in Cypress test suite

3.8.5 Regression Testing

Regression Testing is defined as a type of software testing to confirm that a recent
program or code change has not adversely affected existing features.

3.8.6 Performance Testing

Performance Testing is a software testing process to test the speed, response time, sta-
bility, reliability, scalability and resource usage of a software application under partic-
ular workload.
The main purpose of performance testing is to check if the project is complying to the
standards that are mentioned in the documentation, features like accessibility etc.

3.8.7 Load Testing

Load Testing is a non-functional software testing process in which the performance of
software application is tested under a specific expected load. It determines how the
software application behaves while being accessed by multiple users simultaneously.
The main purpose of performance testing is to check if the project is able to handle
huge amount of traffic when it goes into production.
CHAPTER 4

Methodology

4.1 Tools and Techniques

4.1.1 Cypress
Cypress is a newly launched front-end testing tool built for the modern web. They
address the key difficulties developers and QA engineers face when testing modern web
applications. They make it possible to set up tests, write tests, run tests, debug tests.
It is often compared to Selenium and other browser automation framework, however
Cypress is both fundamentally and architecturally different. [9]
Cypress provides this library called @cypress/instrument-cra that helps to instrument
code written in ReactJS.

Figure 4.1: Cypress’s dashboard

23
CHAPTER 4. METHODOLOGY 24

4.1.2 Instrumentation of code

To find out which part of code has been executed and which part of code has not been
executed, we have several options, the first one being, use large scale logging com-
mands like console.log() in JavaScript or using debuggers like binding,pry in Ruby on
Rails etc. But these approaches are clearly not feasible when we are working with
large piece of codes having 1000+ LOC. So, to deal with these big codebases, we can
place counters after each line of code, and after execution of a testcase, we can retrieve
the value of these counters and we can easily comment if a particular line of code is
touched or not, also, we can comment about the frequency of execution of each line
of code based on the values of these counters. This task is known as instrumentation,
where the library adds counters to all lines of codes and compares two JSON objects
containing values of these counters before and after execution of a testcase.

4.1.3 IstanbulJS
IstanbulJS helps to instrument ES5 and ES2015+ JavaScript code with line counters,
so that we can track the quality of our unit-tests exercise the given codebase. IstanbulJS
comes as a NPM package that needs to be installed in any ES5+ projects and it needs
to be run with react-scripts to instrument the codebase.

4.1.4 nyc
nyc is a command-line-client for Istanbul that works well with most JavaScript testing
frameworks: tap, mocha, AVA, etc. In my project, nyc was used to generate code
coverage reports from binary data containing counters of each line of code.

4.1.5 Playwright
Playwright is a NodeJS library to automate Chromium, Firefox and WebKit with a
single API. Playwright is built to enable cross-browser web automation that is ever-
green, capable, reliable and fast. Headless execution is supported for all the browsers
on all platforms.

Figure 4.2: Playwright

CHAPTER 4. METHODOLOGY 25

4.1.6 XtermJS
XtermJS is a front-end component written in TypeScript that lets applications bring
fully-featured terminals to their users in the browser. XtermJS was used to integrate
a terminal inside the desktop app such that users can execute any command inside the
desktop app that will help to ease the lives of users. Also, all the commands that are
run by the desktop app will be executed in these terminals directly and not behind the
scenes such that user has clarity that what all commands are executed in their system.

4.1.7 Jenkins
Jenkins is the leading open-source automation server, it provides hundreds of plugins
to support building, deploying and automating any project. Jenkins is a self-contained,
open source automation server which can be used to automate all sorts of tasks related
to building, testing, and delivering or deploying software.

Figure 4.3: Jenkins pipeline

4.1.8 ElectronJS
Electron is a free and open-source software framework developed and maintained by
GitHub. It helps to develop desktop GUI applications using web technologies: it com-
bines the Chromium rendering engine and the NodeJS runtime.
The code coverage desktop app was built on top of ElectronJS, it was chosen on the
basis of the literature review and PoC that our team did in the initial sprint.
CHAPTER 4. METHODOLOGY 26

4.1.9 ReactJS
React is a free and open-source front-end JavaScript library for building user interfaces
or UI components. It is maintained by Facebook and a community of individual devel-
opers and companies. React can be used as a base in the development of single-page or
mobile applications
ReactJS was used to create the UI for the code coverage desktop app. MaterialUI
was the UI library used to design the components.

4.1.10 NodeJS
NodeJS is an open-source, cross-platform, back-end JavaScript runtime environment
that runs on the V8 engine and executes JavaScript code outside a web browser.
NodeJS was used as the runtime to power the desktop app and connect it with the
frontend packages and to render the UI.

4.1.11 ngrok
ngrok provides a real-time web UI that can introspect all HTTP traffic running over
tunnels. It is a cross-platform application that enables developers to expose a local
development server to the internet with minimal effort.
ngrok was used in the project to tunnel the user traffic to execute testcases on a local
version of the project in our cloud based testcase execution tool i.e. Atssa2.0.

Figure 4.4: Tunneling localhost:3000 with ngrok

4.1.12 Docker
Docker is a set of platform as a service products that use OS-level virtualization to
deliver software in packages called containers. Containers are isolated from one another
CHAPTER 4. METHODOLOGY 27

and bundle their own software, libraries and configuration files; they can communicate
with each other through well-defined channels.
Docker images of Windows and Ubuntu were used to generate an executable file
for the ElectronJS project as I only had access to a MacOS system. Docker was also
used to dockerize the React project into a docker image such that it can run anyone’s
system without much hassle.

4.2 System Design

Figure 4.5: Flowchart of the system

CHAPTER 4. METHODOLOGY 28

Figure 4.6: The code coverage app

4.2.1 Steps to achieve the final code coverage reports

4.2.1.1 Instrumenting customer’s code

The first step before choosing any testsuite is to instrument the user’s code such that,
once the code is hosted and the project is used, JSON coverage variable can be tracked
and the reports can be generated. To instrument the code, we are using cypress/instrument-
cra library that is used to spawn the react-scripts process to instrument the code and host
the code at the same time.

Figure 4.7: Script to instrument code

4.2.1.2 Choice of testsuite: Manual testcase execution

In manual testcase execution, the users can easily test a particular scenario by manually
testing the project and finding the code coverage simultaneously.
CHAPTER 4. METHODOLOGY 29

1. To facilitate this task, we will need to record all the movements of the user in the
browser to find which features are used, and similarly, we can generate the JSON
object for the coverage data.

2. As we know that we cannot extract devtools variables from a normal browser,

we would requrie a browser automation framework like Selenium or Puppeteer
or Playwright. After an intensive PoC, we came to the decision that Playwright
can be used to solve our needs because it’s the modern open source browser
automation framework powered by Microsoft that can easily be integrated with
NodeJS, as the bindings for Selenium were only working seamlessly for Java.

3. Once the choice of browser automation framework had been chosen, we inte-
grated it with the desktop app.

4. The user needs to enter two things to find the code coverage:

(a) The localhost port where the project is hosted

(b) The folder in which the project is located

5. Once the user fills in all the details and starts the process, the instrumentation of
the code begins.

6. Once the instrumentation process ends, user will be provided with a choice of
browser where they want to test the scenario. They are:

(a) Chromium: Chromium is an open-source browser project aiming to make

Internet surfing much more safer, faster, and more stable for the users.
(b) Firefox Nightly: Nightly is an unstable testing and development platform
for the Mozilla Firefox browser.
(c) Webkit: WebKit is the web browser engine that powers all the Internet re-
lated apps of MacOS like Safari, Mail, App Store etc.

All of these browsers serves as an engine that powers Google Chrome, Mozilla
Firefox and Safari browser respectively and is mostly used to develop softwares
that needs programmatic access to the browser internals.

7. The users can perform manual testing in these newly opened browsers and can
manually test their project and automatically the code coverage reports will be
updated.
CHAPTER 4. METHODOLOGY 30

4.2.1.3 Choice of testsuite: Automated testcase execution

In automated testcase execution, the users can use a testsuite like Atssa2.0 to execute a
testcase in the cloud and after the testcase is executed, the JSON object can be extracted
from the browser and that can be used to calculate the code coverage reports.

1. Facilitating this task is easier as comparison to the manual testcase execution

as the testcases are being run in the cloud browser automation framework and
we don’t need to track the coverage variable in real-time. The coverage JSON
variable will be extracted from the browser after the testcase has been executed.

2. To execute the testcases and find the coverage variable, we need a tunneled URL
of user’s localhost URL. To achieve this, we are using ngrok to tunnel the local-
host URL and Atssa2.0 will use this URL to run the testcases.

3. Once the testcases are executed, the browser automation library i.e. Selenium
will extract the coverage object and it will save the response in the MongoDB
database.

4. Once the user wants to explore the results of the executed testcases in Atssa2.0
dashboard, the code coverage variable will be sent as an API response to the
dashboard.

5. The user needs to enter two things to find the code coverage:

(a) The user needs to copy the coverage object from the dashboard and paste it
in the desktop app.
(b) The folder that contains the project.

6. Once the user fill in all the details and starts the process, the code coverage reports
are generated.

4.2.1.4 Generation of code coverage reports

Once the JSON coverage data has been extracted from the browser automation frame-
work and accessible to the desktop app, the data will be written in the file ‘./.nyc_output/out.json‘,
after that, the nyc tool will be used to use the ‘out.json‘ file to generate code coverage
reports in HTML form.
CHAPTER 4. METHODOLOGY 31

Figure 4.8: The out.json file

Figure 4.9: Command to generate the reports

CHAPTER 4. METHODOLOGY 32

Figure 4.10: Code coverage report in HTML format having 50% code coverage

Figure 4.11: Code coverage report in HTML format having 100% code coverage
CHAPTER 4. METHODOLOGY 33

Figure 4.12: Code coverage report in text format

4.2.2 Sending the reports back to the cloud platform

The reports that are generated in the user’s system can be sent back to the cloud for
future purpose. To solve this feature, we provided a feature to select the folder from
user’s system and the HTML files can be uploaded to Amazon S3 storage, once the files
are stored, it can be retrieved from Atssa2.0 dashboard, this also solves the purpose of
referring to the reports in future to check if the code coverage has increased or decreased
in comparison to current reports. Based on the verdict, we can comment that the tests
are robust enough to check all the features and utils implemented in a project.

Figure 4.13: Structure of the folder

CHAPTER 5

Results and Conclusion

5.1 Results
• All the proposed objectives were successfully implemented.

• The code coverage desktop app was packaged and shipped to the customers in
the form of executable files for Windows, Linux and MacOS platforms.

• The reports storage mechanism was successfully integrated with the cloud infras-
tructure.

• Enterprise users can download the desktop app to manually calculate the code
coverage, else normal users can track the JSON code coverage variable through
the dashboard of Atssa2.0.

5.2 Conclusion
The four months internship at Rakuten was a complete success in terms of understand-
ing a multi national corporation from a close point of view and in terms of learning
new technologies and experimenting new approaches to build new stuffs with a tinge
of business logic to make a product successful. One of the biggest takeaway was how
big teams manage their work and get work done with regular meetings, weekly sprints
and how agile methodology of making a software works. Several concepts which were
learned theoretically like agile methodology was used practically during the internship.
Several non-technical concepts like growth mindset, importance of diversity and inclu-
sion, necessity to pursue other passions etc. were also emphasised by the organization,
especially the senior management. This gave valuable life lessons. I also got the op-
portunity to learn several new technologies like ElectronJS, Code Coverage, System
Design etc. My manager was extremely happy with my progress right from the first
week and was impressed by the ramp-up speed and this motivated me further. My

34
CHAPTER 5. RESULTS AND CONCLUSION 35

internship was a great learning experience which was both productive and enjoyable.
REFERENCES

[1] Rakuten Group Inc. Rakuten group inc. https://global.rakuten.com/corp/.

[2] Rakuten Group Inc. Rakuten shugi | rakuten group inc.

https://global.rakuten.com/corp/about/philosophy/principle/ethic.

[3] RIEPL. Rakuten india enterprise private limited. https://corp.rakuten.co.in/.

[4] Atlassian. An introduction to code coverage.

https://www.atlassian.com/continuous-delivery/software-testing/code-coverage.

[5] Rafiullah Hamedy. 10 reasons why code coverage matters, 2020 (accessed
Sept 11, 2021). https://codeburst.io/10-reasons-why-code-coverage-matters-
9a6272f224ae.

[6] Bishr Tabbaa. The rise and fall of knight capital — buy high, sell low. rinse and
repeat., 2018 (accessed Sept 11, 2021). https://medium.com/dataseries/the-rise-
and-fall-of-knight-capital-buy-high-sell-low-rinse-and-repeat-ae17fae780f6.

[7] Thomas Hamilton. Code coverage tutorial: Branch, statement, decision, fsm,
2021 (accessed Sept 11, 2021). https://www.guru99.com/code-coverage.html.

[8] BrowserStack. Browserstack. https://www.browserstack.com/.

[9] Cypress. Cypress. https://www.cypress.io/.

[10] Zalenium. Zalenium. https://opensource.zalando.com/zalenium/.

36