Executing JMeter Tests in an Azure Pipeline 14 Feb 2020 on testing, build Visual ti Load Testg ols hve boon ceprecsted lng wth Cloud Loed Testing. In ths pol investiga how o use JMeter a lad tesing ams. 1. The Solution 2. Test Plan Considerations sh 2. testish 3, WSL Gotcha 3, Executing from a Pipeline Executable Atrbutes” 4, Conclusion Microsoft have deprecated Load Testing in Visual Studio. Along with ths, they have also deprecated the cloud load tetingcapabilty in Azure/Azure DevOps. On the oficial altematives document, several altematve load testing tools and ATTN ATE MENTOTET, METI JMeter. What isnot clear ‘rom this page s how exIENY FOUTS SUPPOSED to integrate JMeter into your pipelines. — | nave a demo that shows how you can use Applicaton Insights to provide business telemetry. In the dem, | update a website (PartsUnlmited) and ‘thon use traffic routing to route 20% offic to a canary slot. To simulate WME TTUTA CUM oad test. Unfortunatly, I won tbe able to use that for much longer since the cots BSCTESTTNETOTaM WIT BNEG’ fe soon! | set about figuring out how fo run ths test using JMeter. -IMeter tests can be run ona platform called SlazeMeter BlazeMeter has integration with Azure DevOps. However, | wanted to see i could get a ‘olution that git require @ subscription sarrce——~ ease The Solution Meter isa Java-based application. 'm nota big fan of Java - even though | authored a lot ofthe Java Hands on Labs for Azure DevOps! | had to instal the JRE on my machine inorder to open the JMeter GLI so that| could author my test. HoWEVET-THUMT WaMTTE NETIC META Ta or Meter ‘on the bud agent ~ 50 of course |Tooked to Docker. And it tums out that you can run JMeter tests in a Docker container prety easily! ‘Te summarize the solution 1. Create your JMeter test plans (and supporting les lke CSV fles) and put them into a folder in your repo 2 Create a “un.h fle that launches te Docker image ane runs the tests 3. Create a test sh fle for each JMetar fost plan — this ust calls “un.sh" passing in the test plan and any parameters 4. Publish the reports” directory for post-run analysis. Ive ereated a pothat has the source code for this example, Test Plan Considerations | won't go over recording and creating a JMeter test in ths post ~ | assume that you have a JMeter test ready to go. Ido, however, want to dscuss parameters and data fs. It's common to have some parameters for your test plans. For example, you may want to run the same test plan against mulple sites - DEV or STAGING for example. In tis case you can speciy'a parameter ealed host that you can epecily when you run be tet. To accose parameters in “Meter, you have to use the parameter function," P". JMeter dstingulshes between parameters and variables, so you can have both a variable and {a parameter ofthe same name. Inthe figure below, Ihave a test plan calles CartTestjmx where | specify a User Defined Variable (UDV) called “host. I use the parameter function to road te parametor value it exists, or default to "edpartsun2-prod azurewobsites.net'I the parameter dovs not exist‘The value ofthe host UDV is “S{__P(host.cdparsun2-prod.azurewebsites.ne)". Ofcourse you can use the _P function wherever you need it- not lust for UDV. In my test plan, | also have a CSV for test data | set the path of his fle asa relative path tothe JMX tle: Now that have tne tes plan and supporting data files, | am ready to script test executon. Bofore we ge to running the lest in a container, lot's see how | ean run the test rom the command lin. | mply execute this command from win te folder containing Une JM fle jse=er =~ Notes + -ntells JMeter to run in non-GUI mode + -tspectes the path tothe test plan + specifies the path to output results to + -Jename>= is how | pass in parameters: there may be multiple of these + specifies the path fo te lg fle + -© species that JMeter shoule produce a report + -0 species the report flder location \We now have all the pieces fo script this into pipeline! Lets encapsulate some of this logic into wo scripts “run sh" which wil launch @ Docker container and execute a test plan, and "esta that fs 2 wrapper for execuing the CartTest mx fle run.sh | based this sxpt ofthis Githiud repaby Just van den Broeck+ 1 un Meter Docker mage wih opens Naves'inetenes IMAGE stbamatritst po SROOTPATHE Fray nn ‘ockar sop INAME 8g Movil 28gt8an ‘ockarrm SNAME Bg evil 28GLr9.1 ‘eckarrn-name $NAME +o” SROOTPAT st eat SIVAGE $12) Notes: + The NAME variable is the name ofthe container instance ‘+ Tho IMAGE is tho container image to launch — inthis caso ‘justb4/jmotoratost’~ this containor includes Java and JMot thal launches a JMeter test + ROOTPATH isthe first arg tothe script and i the path that contains the JMeter test plan and data les +The script stops any running instance of the container, and then deletes it ‘+ The fina ine of the script rns a new instance ofthe container, mapping a volume from ‘ROOTPATH" on the host machine toa folder in the container called “ies” and then passes in remaining parameters (skpoing ROOTPATH) as arguments tothe enlypont of the scrip. These the JMeter test arguments. 18 well as an enrypoint test.sh [Now we have 8 gener way to launch the container, map the files and run the tests. Lets wrap this call nto script for executing the CartTest/mx plan: st ‘ootPanest twaten$2 roe=$3| sch Roel ath: ScatP ate 00 Tot es Stee" ‘sro Hort Shoe 101 1 Reporting stat ash ROIS Oreo ro SOIR gt oul tart mks 2 5R_OR ‘-4ST_DiRtest plant ST_o1Rymeterog ht evn 28gtsane:t Jransh SrootPth Dlg fv etor=DEBUG | >RoaactFie 4 $T_DIRtat plan ST_oiRsettig 2038 DIR ¢S7_Diapratelog 51S7_DIatastan shone HTML Test Ropar. '2%0"See HIML stop in $4_OIRInsox nent Notes:+ Lines 3-5: We need 3 args: the roatPath onthe host containing the test plan, the name ofthe test plan (the jx le) and @ host parameter, which is ‘specific to ts test plan + Line 9: set the T_DIR tothe currant directory Lines 14-16: Create a report directory, clearing it iit exists already + Line 18: Clear previous result fles + Lines 20-23: Call un.sh, passing in the rootPath and all the other JMeter args and parameters we need to lwoke the est + Lines 22:28: Echo the location ofthe log, raw report and HTML reports ‘As long as we have Docker, we can run the script and we don't need to insiall Java or JMeter ‘We can execute the est from bash tke so: sass SPH CesTest x edpatsur2devazutensbstan. nel WSL Gotcha ‘One caveat for Windows Subsystem for Linux (WSL): $PWD wil net work forthe volume mapping. This s because Docker for Windows is running on ‘Windows, while the WSL paths are mounted in the Linux subsystem, In my case, the folder in WSL is “imntclrepos!Om/parsunlimitedimeter, while ‘he folder in Windows is “e"vapos\Omiparisunlimtedjmator. I iook me a whe fo figure this out~ the volume mapping works, bul the volume is always empty. To work around this, ust pass in the Windaws path instead: ;us ‘ep Omlgeeuniadeer Carat rx cpeeurddevazutewebse al Executing from a Pipeline \We've done most of the haré work —now we can put the scrptinto a pipeline. We need to execute the test serpt withthe correct arguments and Upload the test results ard we'e done! Hare's the pipaline: aoe 0: rear task Basha dispayane: segura SW Corot jn Sis? tone Publ Piping tat ‘ieplyNane:Punian Mtr Report ‘This is very simple — and we don't even have to wary about installing Java or Mater — the only prerequisite we have is that the agent is able to un Docker containers! The fst stop executes the fest sh sorpt, passing in he arguments just Ike we aid from in the console. The second task publishes ‘the ropert folder so that wa can analyze be 1. Hore's a snippet of the log while the tests executing: we can s0e the download of the Docker image and the boot up — now we just wat forthe test to complete.eee) pe Executable Attributes ‘One quick note inally when | committe the erots fo the repo, they didnt have the executable attribute set — this caused the build to fall because the seripis were not executable. To sol he executable albu, | ran the following carsmand inthe folder canlaning the sh files: 31 update=tndes. ‘Once the build completes, we can download the rpert fle and analyze the lest runPee et Ec Ty Conclusion ‘Once you have a JMeter est, is fairly simple fo ru the tin a Docker container as part of your bulé (or release) process, Of course this dovsn test load fears multiple locations and is limited tothe amount of threads the agent can spin up, But for quick performance metics i's a clean and easy way to execute load tests. Ald to that the powerful GUI authoring capabilies of JMeter and you have a good performance testing platform. Happy load testing!