This action might not be possible to undo. Are you sure you want to continue?
Session for Test Management CoP
Date – 22 Sep 2011
The contents of this deck and the session conducted for TMCoP leverages • iGATE Patni’s past experiences in this space • Research references from Industry Research firms • Individual opinion and research conducted on this subject over a period of time This is not a snap-shot of iGATE Patni’s capability in Mobile Testing For any comments and queries regarding this presentation please contact email@example.com
Proprietary and Confidential September 27, 2011
8 Nokia. 453 Apple. 2011 -3- Samsung. 116. 280. 47.7 Proprietary and Confidential September 27.2 .The Nielsen Company Worldwide 2011 Q2 Smartphone sales by Operating System . 48.8 LG.5 RIM.Gartner Market Share Handset Makers ( mn units 2010) : Strategy Analytics 2011 Others. 413.Mobile Telephony Trends More than 60 % of world’s population today has access to a Mobile Phone! Ratio of Feature phone to SmartPhone users was 80:20 ( Q4 2009 ) Projected to become 70:30 by Q4 2011 world-wide .
CDPD. UWB. PSTN. CDMA. DSL. Bluetooth. Satellite OS/ Platforms Mobile Devices PDA Smartphones Mobile phones Rugged Terminal POS/ Tablets Laptop RS232. WLL. 2011 -4- .Mobility Landscape Enterprise Systems Legacy HIS Supply Chain Financial Applications SFA/FFA Distribution/Retail Management Dashboard Patient Monitoring Credit Card/Claims Middleware/ Platforms Sybase –Afaria/ Unwired IBM DB2 Everyplace Rhomobile Broadbeam/ Mobileaware Blackberry Enterprise Server Sierra/ Motorola Jasper/ Telenor LAN. GSM. WiFi. GPRS. Zigbee. USB . IrDA Peripherals RFID reader Barcode Scanner Camera Printer Sensors GPS Receiver Reference – iGP Mobility Deck Proprietary and Confidential September 27. HomeRF. UMTS.
Authentication. location based. Authorization • Technology Convergence • Provisioning and device management • Scalability • Services – Messaging.Mobility Landscape and Challenges Enterprise Systems Applications Mobility Challenges • Real-time. Near-real-time • Security. 2011 -5- . Off-line. notification Complexity of Middleware • Applications • Range of devices • Connectivity options • Technology options OS • Vendors and • Platforms Application Challenges • Commercial grade apps • Content Aggregation • Customizations and personalization • Multi-device support • Multi-lingual support • Application categories Mobile Devices Peripherals Device Challenges • Device capabilities • Programming models • No dominant standard • Wide range of target environments Reference – iGP Mobility Deck Proprietary and Confidential September 27.
Twitter Proprietary and Confidential September 27.Mobile Testing Landscape Mobile Handset Testing / Device Qualification Mobile Handset App / Mobile WAP Testing Handset / Mobile Device Test Handset from OEM Test Branded PreReleased Handsets supplied by Telecom Operators Native Applications Test Apps that reside on the device and may or may not require network connectivity Web Based Applications Test Applications that are accessible through mobile browsers Third Party Applications Test Third party Apps that are specifically installed on the mobile handset Examples New / Upgrade of Handset from OEM directly Co-branded Handset with Telecom Operator Examples Messaging. Multimedia. PIM etc Examples Mobile Banking Payment solutions Examples Facebook. 2011 -6- . Games.
2011 -7- . Nokia Please note – Types of testing highlighted in Italics are not offered as a service in this space by iGATEPatni today Proprietary and Confidential September 27.g.Mobile Handset Testing • • • • • • • User Interface Branding Mobile Phone Controls Mobile Handset Applications Partner Applications Multimedia and Graphics Integration with other systems Types of Testing • Functional • Regression • Integration • Automation • Multi-lingual • Usability • Field / Coverage • Quality of Service • Performance Testing • Security Testing Focus Areas The customer in this case could be a Telecom Operator or an OEM for e.
6. 4. are as per the specifications given in the UI specification document. themes (both preloaded and downloaded) appear properly. Partner applications : This test is to verify whether all the 3rd party applications present on the handset work properly an example of this could be verifying that the 3rd party application like MSN messenger is launching properly and is working as needed.Mobile Handset Testing – Focus Areas 1. navigations and the soft keys etc. wallpapers .e. Multimedia and graphics : This test is to verify whether the graphics on the handset i. 3. 5. Integration with other systems : if required Proprietary and Confidential September 27. Mobile phone controls : Verify that all the controls or buttons on the handset are mapped correctly for instance pressing a dedicated camera key should launch the camera application only and not any other application. screen savers. User Interface and Branding : Verifying menus. 2. verifying that the alarm on the handset can be set and that it goes off at the given time.g. 2011 -8- . Mobile Handset application : Verifying the functional correctness of all the applications on the handset for e.
voice memo. 2011 .stopwatch.msn.) SETTTINGS CAMERA PRE LOADED CONTENTS GAMES HELP & INFORMATION CONTENT MEMORY CALLS MEDIA PLAYER (audio and video) LANGUAGE UI (in cases of another language being present like Spanish) -9- Proprietary and Confidential September 27. • RECEIVING VARIOUS ALERTS (low balance.etc.Network and Non-Network Testing Mobile Handset Testing Requires Network DATA • • • • • WAP | SMS/MMS | DOWNLOAD CONTENTS DOWNLOAD GAMES WAP deck and 3rd party apps INSTANT MESSAGING (like yahoo.) BLUETOOTH CONTACTS Doesn’t Require Network TOOLS (calculator.google) EMAIL(3rd part email apps) • • • • • • • • • • • • MAKE/RECEIVE CALLS | VOICE MAIL | SPECIAL NUMBERS • RECENT CALL LISTS | INTER CARRIER SWITCH • ALARMS AND SCHEDULER APPS. promo alerts etc.
10 - .Sample Handset Specification Reference – Collateral from past engagement Proprietary and Confidential September 27. 2011 .
11 - . 2011 .Mobile App Testing Types of Testing • Functional • Compatibility • Automation • Regression • Performance • Security • Usability Mobile App Testing The customer in this case could be an Enterprise OR a content Provider or an App/WAP provider Please note – Types of testing highlighted in Italics are not offered as a service in this space by iGATEPatni today Proprietary and Confidential September 27.
This by no means is a list of most recommended tools by iGATEPatni in this space Proprietary and Confidential September 27.12 - . 2011 .Popular Testing Tools used in this space Commercial • Device Anywhere • Perfecto Mobile • Test Quest • Squish • Jamo Solutions • Eggplant Open Source • Robotium (Android Apps) • FoneMonkey (iphone Apps) • TestMaker • T-Plan Robot Please note – This is only a sample list of tools.
BREW.g. Win 7.Feature Set comparison of Popular Tools Parameters Device and Platform Support Device Anywhere All Leading Handset OEM s Android. tools Licensing Model Yes Yes Yes No No Yes HP QTP and QC Subscription based SaaS* Yes Yes Yes No No No No Opensource * Each package is a collection of devices with some common characteristics such as network (e. WebOS Drag drop functionality Ability to script in Java Yes Yes No Built-in Test Case Manager Yes HP QTP and QC IBM Rational Quality Subscription based SaaS* [ 12 mth commitment ] TestQuest Android. Windows Blackberry Yes Yes Yes No No Yes No Cost Robotium Android Scripting Functional Testing Automation Testing Performance Testing Test Management Remote Connectivity Integration with industry std. iphone. iOS. operating system (Android package) etc. Blackberry. iOS. 2011 . Blackberry. Symbian. Nokia. Blackberry. Eggplant Android. Reference – Collateral from Individual Websites of respective toolset vendors | These are some of the key features . UTS. Symbian. Windows. however one needs to do a thorough POC to arrive at a recommendation Proprietary and Confidential September 27. Windows No Scripting Yes Yes Yes Built-in Test Mgr Yes Not available Cost Perfecto Mobile Android.13 - . Verizon Package). manufacturer (Samsung Package).
2011 .What is Remote Device Testing? 100 s of “real” devices / phones connected to live network are “installed” in their Data centre available for use based on a subscription package (SaaS) Through the DeviceAnywhere Test Centre a tester can remotely connect to these devices / phones via an https web based connection in real time and can remotely perform any action. Some key benefits of device testing tools No need not buy all the devices/phones ( including SIM card ) Test multiple combinations of devices / operating systems Leverage feature of the Testing tool to automate and test manage the whole testing process Conduct this testing from a remote location / offshore – thus making it further attractive Proprietary and Confidential September 27.14 - ..
Automated Document Closure Report Document Test Strategy Setup Test Environment NonFunctional Present Findings Tool Selection and POC * Create Test Data Defect Management Go Live! using automated tools OR Emulators using actual devices Please note – this is a technical opinion of the author.Approach for Testing Test Planning Test Design Test Execution Test Closure Analyze Requirements Design Test Cases Functional. Proprietary and Confidential September 27. 2011 .15 - .
2. 7. iPhone etc . Do not leave any loose ends – for e.Key considerations for Drafting a Strategy 1.g. else you will run out of time during re-tests! Testing report should highlight the features that are working / not working across devices Conduct automation post a thorough ROI calculation only Some tools do not integrate with traditional Test Management and OR defect management tools – therefore account for the overhead ( effort and cost ) of managing test cases / defects in a different toolset Defect Management – Prioritize defects upfront with the Dev/Vendors and Business In third party testing scenario ( OEM ) account for additional rounds of discussions for defect triage Please note – this is a technical opinion of the author. 9. 6. Budget for Testing Number of devices / operating systems / browsers ( including versions of each ) to be tested Duration of the programme Results of Technical Proof of Concept If certifying a common functionality / WAP / APP across devices account for Regression testing of defects across the devices/operating systems/browsers . Automation etc Agree the number and types of devices that need to be tested for example Blackberry Bold. Clearly establish the scope of Testing – Functional. 2011 . 3. Proprietary and Confidential September 27.16 - . 8. “ Test this on Blackberry” – which models?? Identify the testing toolset of choice – based on the following key criteria • • • • 4. 5. Blackberry Storm.
Case Studies Proprietary and Confidential September 27.17 - . 2011 .
2011 .Virgin Mobile USA – Handset Testing • First MVNO in US and UK • Customer base ~ 5 M • “Pay as you go” mobile services and handsets • Uses Sprint’s network • Target segment 15-30 yr old • Partnered with iGATEPatni for QA and UAT of the IT stack • Handset. CRM. WebPortal.18 - Virgin Mobile USA Customers Internet Security VMU Web Portal Sales Handset Plans Promos Contact Center/ IVR Top Ups Register CC Adhoc Top Up Paypal Top Up Auto Top Up 3rd Handset “Virgin Xtras” Ringtones Wallpapers Mobile Web Messaging Games Party Content Activations Port In/Out Change MIN/ESN Promos Help/Self Care Services Phones VirginXtras Gateway CRM Distribution Center Billing Credit Card Transactions CDR Telcordia . IVR. DW and Billing platforms • 5 year relationship • 52 person strong team in onshore / off Proprietary and Confidential September 27.
2011 .19 - .Areas of Testing Handset Functionality Handset Testing User Interface Branding Mobile Phone controls Partner Applications Multimedia and Graphics WAP WAP Enabled Application Third Party Applications Third Party Content Integration Integration with Mediation Managers Billing system Customer Care system Functional Testing | Regression Testing | End to End Testing | Compatibility Testing Proprietary and Confidential September 27.
20 - . 2011 .HRD VMU PATNI FLASHING NEW BUILD UNSTABLE END CUTOMER STABLE BUG REPORTING TESTING Proprietary and Confidential September 27.Testing Process and Flow VMU REQUIREMENTS NOKIA KYOCERA UT STARCOMM HANDSET UNDERSTANDING DOCS TEST CASE PREP UI SPEC & HANDSET HANDSET.UI SPEC .
21 - Proprietary and Confidential September 27.Handset Testing WDIG HANDSET PORTFOLIO Offshore GATEWAY P100 P205 Telcordia DATA Platform LG200 SCH110 STIC LAB Onsite Voice Contacts SMS MMS Email Browsing Settings Calendar Alarm Camera functionalities Hands free Calling Content Download Family Monitor Family Alerts Family Locator Photo Upload and Download . 2011 .
Proprietary and Confidential September 27. 2011 .22 - .
This action might not be possible to undo. Are you sure you want to continue?
We've moved you to where you read on your other device.
Get the full title to continue listening from where you left off, or restart the preview.