WeTest Testing Solution

Value for Users, Tech for Good

Content

l Tencent & WeTest Introduction

l Product Features & Advantages

l Successful Cases
About WeTest
About Us

WeTest was established in 2014, as a Quality Open Platform is the

official one-stop testing service platform for game developers. We are
a dedicated team of experts with more than ten years of experience
in quality management. We are committed to the highest quality
standards of application development and product quality and tested
over 450,000 applications.

WeTest integrates cutting-edge tools such as automated testing,

compatibility testing, functionality testing, remote device,
performance testing and security testing, covering all testing stages of
games throughout their entire life cycle.

5 significant dimensions, 41 indicators, comprehensive product

quality protection.

For more details, please visit WeTest Official Website

 About WeTest
Mission & Vision Core Value
Focus on the Global Market, lead the way with quality assurance Original technological innovation

Industry leader, Embrace Innovation Wetest High Quality Standard

People-Oriented, Shape the Future Wetest Gold Q&A Team

Tech for Good, Eco partner prosperity Leading the industry standard

Certifications: Advanced Hardware Infrastructure

ISO/IEC 27001 ISO 9001

Information Security Management Systems, ISMS Quality Management System Certification
ISO/IEC 20000
Information Security Management Systems, ISMS DevOps Members
Content

l Tencent & WeTest Introduction

l Product Features & Advantages

l Successful Cases
 WeTest Infrastructure

Product Live Testing Automation Proprietary Cloud

Live Testing Automation Analysis

Mobile
Remote
Performance Web-UI REST API Log
Testing Debugging
Feature SaaS
ADB Command Power CI/CD Parallel Testing Video

Platform Project File Dashboard Subscription Enterprise Screenshot

Device Clean Up Docker Image Pre-installed Issue Track Uninstall

Management
Platform Permission Storage Environment Redis MySQL Log Service PaaS
Back-end
service Task Scheduling Device Scheduling Case Execution Video Stream

Real Devices Intelligent

Hardware
(iOS/Android)
Cloud Service Network Mobile Data Center
Hardware
Cabinet IaaS
Product Overview
Automated
Live Testing
• Automated Testing
• Real Device
Perform mobile automation
tests in popular frameworks
on real Android and iOS
devices.
• Automated Testing on
Compatibility & Functionality
(Subscription Based)
• Real Device (Time
Consumption Based)
Expert Testing
• Manual Compatibility
Testing
• Manual Functional Testing
Cover all test cases in an
efficient way on up to 300
devices in specific regions
by Tencent expert testers
• Expert Compatibility Testing
• Expert Functionality Testing
WeTest
Performance
Security Testing Localization & Gameplay
Management
• Client Performance
• Crowd Testing–Real
• Server Performance • Penetration Testing
Players
Tools and SDKs that help you • LQA Service
Conduct penetration testing in
manage the performance of
two ways(black box and gray
your game by reporting & Identify potential issues and
box) to find all vulnerabilities
analyzing data in real time improve game quality using
and design flaws of your project
feedbacks from different
types of real users.
Products
• Perfdog & Perfdog Service - • Expert Penetration Testing • Crowd Testing - Gameplay
Client Performance Testing Tool
• Localization Quality Assurance
• CrashSight - Crash Monitoring
SDK-
• PerfSight - Performance
Monitoring SDK
Product

l Automated & Manual Testing

l Live Testing
WeTest Automated Testing
Increase quality and velocity with the cloud-based digital quality platform.
Perform automated testing in popular frameworks against a wide range of real Android and iOS devices.

!Maximum Device Coverage !Integrate with dev cycle

• 1000+ different device models • Web-UI

• Android/iOS Real Device & Virtual Device • REST-API（API Code Generator）
• Public & Private device farm with 24/7 instant access • CI/CD Plugin

!Any platform or test environment !All test analysis in one view

• Multi-platform support (iOS/Android) • Statistics of test cases on each device

• Popular mobile testing frameworks (Appium Client- • App performance statistics
side/Cloud-side, Espresso, XCTest/XCUITest…) • log/screenshot/video
• Custom framework and test environment • Remote debugging on real devices
• .aab, .apk , .ipa extension !Ease of use & Customer support
!Speed up your test runs
• 30-day free trial
• Improve your testing speed. Reduce release times by more • Demo apps & scripts in Github Repository
than 10x through parallel test execution on WeTest cloud. • Customer technical support in Slack
Private Device Cloud
A fully customizable app-testing infrastructure
Dedicated devices in a dedicated environment

Private Device Cloud, 24/7

Available 24/7 with unlimited access, usage and users, do not have to share devices
with others. Lower device management and infrastructure maintenance cost.

Maximum Device Coverage

Flexible choosing dedicated devices including iPhone, Huawei, Xiaomi, Samsung
Galaxy, Pixel with different screen size, OS versions.

Custom Testing Environments and Device Settings

Support your own testing environments by using containers. Configure device
cleanup procedures, reset device connections, monitor the performance of your
environment and more device settings.

Enterprise-grade Security
The real devices and network infrastructure are hosted and maintained by the
WeTest team in one of highly secured data centers. The service is encrypted and all
uploaded content as well as all test results are visible only to private cloud users.
Test Automation: Mobile Testing Framework And Submit

Popular Framework Multiple Options to submit your test

ü iOS ü Web-UI
• Appium • Choose App
• XCTest • Choose Test Suite
• XCUITest • Choose Device
ü Android ü CI/CD
• Appium • Jenkins Plugin
• Espresso • Jenkins Pipeline
• GameLoop ü REST-API
……. • REST API Doc
ü Custom Environment • Code generator

Scripting languages Java - Python …

Test Automation: Data Center & Parallel Testing

Maximum real device market coverage Execute tests in parallel

Real Device Cloud Cut Down Testing Cost

- 15+ Brands
- 50+ OS versions
- 1000+ iOS/Android devices
- Up to run 15 devices in parallel
- Up to submit 50 devices in a test
User coverage of each
dimension of TOP100 models:

Global Data Center

Mobile phone User coverage
dimension rate %
- 24/7 real device stability
Android version 98.35
keep 99.9%
Screen 98.5
- Intelligent operation and Resolution
maintenance system Memory 99.1
manufacturer 99.96
CPU clock speed 99.9
CPU architecture 92.65
Quick Start

Prepare Test Create Test Submit Test Test Execution Test Analysis
Test Automation: Successful Cases in Four Main Scenario
1. Popular Framework
Using CI/CD plugin or Web-UI to
perform automation tests in popular
frameworks against real Android and
iOS devices
3. Monkey Test
No test suite required, only uploading
App on WeTest platform and executing
WeTest intelligent Monkey test to
automatically find compatibility issues
on multiple Android devices.
2. Custom Framework
Using PaaS API to perform automation
tests against real Android and iOS
devices in your own framework and
environments.
4. Private Cloud
Using ADB shell, users have access to
WeTest private device cloud and
perform automation test easily.
 Live Testing: Why Live Testing

1 2 3

Huge purchase cost and device New devices constantly updated Hard to find and fix online bugs
management expenses. on customer using device
 Live Testing: Features

1. Maximum real device market coverage

Test on thousands of Android and iOS real devices including
iPhone, Huawei, Xiaomi, Samsung Galaxy, Oppo, Vivo, Pixel
with different screens and OS versions.
2. Test Dev and Published Apps
Upload and test on your dev APK/AAB/IPA files. Also be
allowed to install production apps from Play Store/App Store.
3. Real-time Debugging
Debug your app instantly, view device logs, inspect UI element,
and use stacktrace to find and fix bugs easily.
4. ADB Command Debugging
Easily access Android cloud devices directly from adb shell
which could help you ship quality apps even faster.
Real Testing: Why choose WeTest Live Testing

Maximum real device market coverage Real-world user experience

Real Device Cloud

- 50+ OS versions
- 1000+ iOS/Android devices

Global Data Center

- 24/7 real device stability
kept at 99.9% High
Low Latency
- Intelligent operation and
Definition < 150ms
720P+60fps
maintenance system
Multi-Touch Gesture
Replicate real-world user condition
Manual Testing: Expert Compatibility & Functional Testing

Delivery time:48H Manual testing

Installation Test
Log in Test • Flexible Testing Plans:
Start Test • 20min/50min per device
Operational Test
UI Test • Core Loop:
Functional Test • Install, start, beginner's guide, core plots,
APP Crash uninstall
Scope of Testing Process exits • Professional case design
delivery No Response • nearly 100% coverage of main scene
Exception
• Provide multi-dimensional info on bugs
Security keyboard
Photograph • logs, snapshots,etc.
Face Recognition
Voice • 2000+ real devices including regional
3D Touch specific models across different GPUs,
Gravity Sensor CPUs,RAM,OS,etc
 Manual Testing: Expert Compatibility & Functional Testing Test Analysis

Dashboard

App Info
Package name, File name, API level…

Test Result
Duration, Success Rate, Test ID…

Device Analytics
Multiple dimensions of Device
statistics(Brand、Screen、OS versions)

Device List
Detail info of test run on each device
 Manual Testing: Expert Compatibility & Functional Testing Test Analysis

Device Session

Device Info
Brand, OS, CPU, GPU, RAM…

Issue
Details of each issue found during
running this test (ANR, Crash…)

Cases
Case result and details of the device

Screenshot/Log/Video
Debug your Apps instantly using text
logs, video recordings and screenshots
of the test run

Performance
Client-side performance data on the
device(Memory, FPS, CPU Usage…)
Product

l Performance Solutions

l Security Testing
Serve Performance Testing: Server Performance Solution-LoadMaster

LoadMaster—A SaaS server performance testing Importance of Server Performance

solution for modern DevOps teams

Stress Test
Project Team Operation&
Maintenance
Concurrency
Test Traffic volume?
Testing
dimensions Optimization
Capacity before release
Test Performance under
huge traffic?
User experience&
Stability Test Operation Index
 Server Performance Testing: Server Performance Solution-LoadMaster

WeTest provides use case design, How to find out server

configuration, execution, report performance issues?
analysis, expert on-site
interpretation of reports, problem
positioning, and optimization advice.
• Supports distributed pressure sources
• Supports TCP/UDP/WebSocket with binary
data flow like ProtoBuf and other customized Benchmarking Verification test Capacity test
key-value protocols • Locate existing level • Acceptance of system • Detecting
• Supports HTTP&HTTPS protocols, etc. E- of system performance, accurate performance
performance& detection of site bottlenecks in the
commerce, government, life services and other relevant Indicators capabilities system
apps and website pressure testing • Verify business goals & • To provide directions
optimization effects for business
optimization.
 Client Performance Testing: PerfDog

• Various ROOT/jailbreak methods tried

• Performance test& analysis tool for full mobile
platform(iOS & Android)
• All types of applications(games, apps, browsers,
mini programs, mini-games, H5, background • Multiple tools needed —>limitations
system processes, etc.)
• Quickly locating the performance issues without
ROOT/Jailbreak • Various XCode source code
• Plug-and-play, no need of extra modification on compilation required—> tool
hardware or apps
vacancy
• Retest needed—> inaccuracy
Client Performance Testing: PerfDog Performance Data Collecting

Android
iOS
p Screenshot
p FPS- Avg(FPS),VAR(FPS),Drop(FPS)
p Screenshot
p FPS- Avg(FPS),VAR(FPS),Drop(FPS) p Jank- BigJank(1s), Jank(10min),
p Jank BigJank(10min)
p Ftime- Avg ( FTime),Delta(FTime) p Ftime- Avg ( FTime),Delta(FTime)
p CPU Usage-Total,App p CPU Usage-Total, App
p CPU Clock
p Memory(Footprint)
p XCode Memory p Memory(PSS Memory)
p Real Memory p Swap Memory
p Wakeups p Memory Detail(Native PSS, GFX, GL,
p CSwitch Unknown)
p GPU Usage(Qualcomm chip only)
p GPU Utilization- Render, Tilter, Device
p Network(Recv/Send) p GPU Frequency(Qualcomm chip only)
p Battery Power-Current, Voltage, p Network(Recv/Send)
Power p CTemp
p Log p Battery Power-Current, Voltage,
Power
p Log
 Client Performance Testing: PerfDog Features & Advantages

Full Mobile Platform Data Accuracy

• Various systems (Android/iOS) • Performance impact on CPU<1%

• Various devices: mobile phone, Pad, • No performance impact on FPS
Watch, Android simulator • Screenshot
• PerfDog Windows& Mac • FPS, Jank, Ftime
• No ROOT for Android
• CPU, GPU, Memory
• No Jailbreak for iOS
• Battery, Network, Ctemp

Application Universality Cloud-based Dashboard

• All kinds of apps • Store, visualize, edit, manage,
download data
• Games
• Whenever & wherever possible
• Mini programs, mini games Professional, efficient,
comprehensive and scalable • Set up tasks
• H5
one-stop performance
infrastructure services (Precipitating • Invite members to join the task
• Websites
expert testing and analysis capabilities into the
Service lowers user competency standards. • Share & review the progress of the task
• No installation And support for custom development and
automation)
• Plug-and play
Client Performance Testing: PerfDog Service Integration Overall Structure

(Windows、Mac、Linux)

Perf
Automation

GRPC
(Windows, Mac, Linux)
UI GRPC
Provide interfaces(local/remote) PerfDog
iOS (C#,C++, Java, Python, PHP, Go)

PerfDogSDK PerfDog Service Perf***

(Windows, Mac, Linux) (Windows,Mac,Linux)
Android
***
GRPC/TCP

Performance
Remote Perf*** tools developed
(Windows,Mac,Linux)
by users
 PerfSight: Client Performance Monitoring SDK

PerfSight can monitor the whole lifecycle of application

from R&D to operation phase.
PerfSight collects real user performance data, including
FPS, PSS, CPU usage, network latency, and supports
1 · Serve for mobile 2
Silicon Valley Cloud games China IDC statistical display by time, versions and scenarios.

3 Performance 4
Singapore Cloud
monitoring closed 1 · Serve for mobile 2 · Thorough indicators&
loop games dimensions

Performance 4 · Full cycle of game

Collects real user performance data Displays after analyzing & computing in the cloud
3 monitoring closed loop development covered
PerfSight All-round Monitoring of Multidimensional Counting x 7*24 Monitoring

All-round Monitoring Multi-dimensional Counting

Fluency Indicators Network and Hardware Indicators Device Type Statistics Statistics of Game Features

p FPS mean p Disconnection(custom) p brand p version->scenario-

p Operating system >device type

p FPS jitter p Packet loss(custom)
p Scenario Loading Time
p FPS jank p Latency(custom) p RAM
p Simulator Determination
p FPS low frame p PSS peak p GPU model
p Front and back stage
p Pull back(custom) p CPU mean p OpenGL ES version determination of app
p Temperature p Picture quality statistics
p Core

7*24 | 5 mins delay of single user data|1 h delay of statistics data

PerfSight Performance Monitoring Closed Loop for Application

p Overview p Performance daily flow

（FPS、PSS Ring ratio ） p PerfSight daily mail
p Version compare
Compare Monitoring
p Hot models
（model granularity、FPS、PSS） p Hardware information

Monitoring
closed loop
p Version analysis
p PerfSight Alarm Mail
p Scenario analysis Locate Alarm
（ adjustable threshold, impact
p Device analysis
surface ）
p Single user backtracing(30
p PerfSight Daily baseline
days, scenario granularity)

http://apm.wetest.net to check demo.

CrashSight:Client Exception&Crash Monitoring SDK

Full Platform Exception Reporting System

SDK, unified web data platform, customizable features
Exception Monitoring and Chinese Version International Version
Diagnosis For Global Games

Covers 3 Types of Gaming Platforms

Mobile
Deep Capabilities
Ability to capture on the game engine side
has been enhanced:
ü supports capturing the Mono engine's
managed stack,
ü with 3new stacks on the Unity engine's
side, including the Java, C++, and C#
stacks;
ü supports capturing UE engine crashes.
ü Enterprise Email Login General Capabilities
ü New Advanced Configuration Module
ü Issue Display Optimization
ü New system time configuration function
ü New hourly crash comparison feature
ü Reporting Robot
PC Console
ü Data Reporting: Report the
exception information of the PC
end-game client. ü Supports XBOX, PS4, Switch and
other mainstream console
ü Accurate Location: Accurately
monitoring, add advanced
locate the crash code location, error
configuration module.
type and other information.
ü Optimization on the presentation of
ü Professional Analysis: Direct
assistance in analyzing the causes existing hosts
of crashes and solving crash
problems.

Strong Exception Smart Analyzing Full Chain of Full Platform Privatized

Reporting Capabilities Capabilities Data Access Supporting Deployment
Security Testing- Penetration Test

• Game communication protocol Testing Methods

• Client function security
• Script logic
• Memory security
• Static resource security
• …. Black-box
Gray-box Testing
Expert Experience
Testing
ü Tencent IEG mobile game security testing team
ü Game category specified vulnerability sample No file required
Requires the
database game protocol
ü Served for 700+ product, exclusive penetration testing structure file
solution Exploit
vulnerabilities in
Star Standard the game from an
ü Tencent 6-star game QS, access to the top level attacker's point of
testing-service-quality as same as "King Glory", Exploit the view
"Crossing the Fire“,etc. vulnerabilities of
the game from a
Cost Reduction with Convenience& Safe tester's point of Unity 3D & UE4
ü No need to access SDK view. engines supported
ü no impact on the game itself
ü highly private, no detail of vulnerabilities is disclosed
Product

l Crowd Testing-Gameplay Testing

l LQA
Crowd Testing-Gameplay Testing

Crowd Testing Service is about testing new player experience, core gameplay,
numerical system design, sensory experience, game balance, commercialization at different
phases of a project, including development, testing and operation. It identifies potential
issues and helps improving game quality using feedbacks from different types of real users.

ü Supported genres:

FPS MOBA MMO Open World SLG

ARPG Sandbox Autochess TCG Interactive Story

SIM RAC Music ……

Crowd Testing-Gameplay Testing
Crowd Testing Service is about testing new player experience, core gameplay, numerical
system design, sensory experience, game balance, commercialization at different phases of a
project, including development, testing and operation. It identifies potential issues and helps
improving game quality using feedbacks from different types of real users.

ü Service Capabilities

1. Recruit Target Users 2. User Management 3. Questionnaire Design 4. Core User Group

•Find and recruit target users •Confidentiality •Customized questionnaire •Hardcore players, core users
accurately •Timely and effective design •More sensitive to product
•Recruit over 100K users feedback •Accurately locate issues and and experience
•Traceable and trackable collect feedback

Other 7. Questionnaire 8. Competitor

5. User Interview 6. Data Analysis 9. Art Quality
Research Comparison
Capabilities
Crowd Testing Service Supported Products
Special Review (In-depth) 1000-User
1000-User
Game Genre Gameplay New Player Commercialization Content Sensory Experience Art Quality Test
Core Gameplay Balance Social Experience Quality Test
Experience and Growth Experience Experience Review
Classic
FPS Side-scrolling Shooter ☑ ☑ ☑ ☑ ☑ ☐ ☑ ☑ ☑ ☑
Real-time
MOBA Multiplayer Versus ☑ ☑ ☐ ☑ ☑ ☐ ☐ ☑ ☑ ☐

Classic
MMO Real-time ☑ ☑ ☐ ☐ ☑ ☑ ☐ ☑ ☑ ☐
Open World

Turn-based
SLG Infinite Borders-like
☑ ☑ ☑ ☑ ☑ ☑ ☑ ☑ ☑ ☐
Real-time
Idle Real-time ☑ ☑ ☐ ☐ ☑ ☑ ☐ ☑ ☐ ☐
Autochess
Wargame ☑ ☑ ☐ ☑ ☑ ☐ ☑ ☑ ☐ ☐
Shooter
SIM Japanese Farm Sim ☑ ☑ ☐ ☐ ☑ ☐ ☑ ☑ ☑ ☐
Sandbox Island Survival/Creation ☑ ☑ ☐ ☐ ☑ ☑ ☐ ☑ ☑ ☐
Open world
ARPG ☑ ☑ ☐ ☑ ☑ ☑ ☑ ☑ ☐ ☐
Classic
TCG Turn-based ☑ ☑ ☑ ☐ ☑ ☐ ☐ ☑ ☐ ☐
Music/dress-up Falling notes ☑ ☑ ☐ ☑ ☑ ☑ ☑ ☑ ☐ ☐

Football
Sports Basketball ☑ ☑ ☐ ☐ ☑ ☐ ☐ ☑ ☐ ☐
Racing
Strategy Classic ☐ ☑ ☐ ☑ ☑ ☑ ☑ ☑ ☐ ☐
Apocalyptic
SOC ☐ ☑ ☐ ☐ ☑ ☐ ☑ ☑ ☐ ☐
Survival
Casual Match three ☑ ☑ ☐ ☐ ☑ ☐ ☐ ☑ ☐ ☐

Ø We have experience in providing services for projects with the above game genres and gameplay.
Ø Besides these genres, we also provide services for all game genres, including special review (in-depth), 1000-user experience review and
1000-user experience review.
 Overseas operation pain point: text translation alone is not enough to achieve good localization results

When working with multiple languages, the translation is often performed out of context, which means that the translation
may be incompatible with local culture/user interface, causing risks for the product.

Ø Lack of Context for Translation

• The feature is still under development
• The source text is out of context

Layout & Missing Translation Language & Semantics Culture & Compliance Multimedia Content
Ø Demand for Translation Speed • Hard-to-read text caused by • Inconsistency between dubbing
Glitched UI due to a lack of space • Cultural taboos that offends players
• grammatical errors and subtitle can mislead the
• Tight project schedule and short • Sensitive issues that may cause PR
for translated text • Using words out of context players
turnaround time crisis
• Confusion caused by missing • Non-idiomatic, awkward • Inconsistency between dubbing
• The industry is word-count-based translation • Violating local laws and regulation
translation and character animation
and risk of getting banned
• Truncated text • Misleading the player with • Missing dubbing can affect player
Mistranslation experience
Ø Localizing a Domestic Game
• The elements in Chinese UI is too
small for other languages
• Culture-specific words. E.g. (Koi fish,
which means a lucky person in
Chinese)
Affecting Player Experience Affecting Market Performance Causing PR Crisis
 LQA – localization quality test after implementing the translation in the version

• Localization Quality Assurance (LQA) is a localization testing service provided by YFSail, supporting different formats including text, UI, images,
audio and video. It can detect many issues related to character limits and missing translation, language and semantics, culture and customs,
mistranslations, UI layout, laws and regulations, dubbing, consistency, etc. It is a comprehensive service that covers every step of the localization
process and provides efficient iteration management, improving the user experience and product quality in all languages!

Translation review – checking the quality of the text

Standpoint: Developers
Independent
Translate translation/review
key Source Text Translation-1 Translation-2
• Only checks the translation quality, not the final
XXXXXXXXX XXXXXXXXX XXXXXXXXX XXXXXXXXX
XXXXXXXXX XXXXXXXXX XXXXXXXXX XXXXXXXXX result
XXXXXXXXX XXXXXXXXX XXXXXXXXX XXXXXXXXX
XXXXXXXXX XXXXXXXXX XXXXXXXXX XXXXXXXXX • Out of context review
XXXXXXXXX XXXXXXXXX XXXXXXXXX XXXXXXXXX
XXXXXXXXX XXXXXXXXX XXXXXXXXX XXXXXXXXX
• Only checks the text and cannot check UI, images,
XXXXXXXXX XXXXXXXXX XXXXXXXXX XXXXXXXXX audio and video content
Implement the
Implement translation file into
the game LQA – checking the interface of the product
Standpoint: Users
ü Check the translation quality and the final result
in the corresponding scenario and interface
ü Check the appropriateness of translation in
Check localization
specific scenarios and context
LQA quality under game
context ü Comprehensive examination of UI, text, images,
audio and video
 Content

l Tencent & WeTest Introduction

l Product Features & Advantages

l Successful Cases
Global Partner and Clients
Successful cases

Retail

Entertainment

Finance

Government

Game