oe a pl ag at ae ieee taeta04 THeeseute tooteor, ‘TP3: PDI Creating basic task flows: Jobs 1. Basics about jobs Now you will create a Job similar to the one you created at the end of the frst tutorial (TP1, a simple job), but with the adction of ome extra tasks, The new version of the lob ‘with 2 name of your choice. fan error occurs the fact willbe reported tothe POI log, ACTION: 3. Open the Job you created before and save it under a diferent name 2+ Double-click the Create folder entry and replace the name of the folder with Siinternol.Entry.Current.Directoryl/S(MY_FOLDER) 3. From the Utility category of steps, drag to the canvas a Write To Log entry. We want to run this last entr TY only when the creation of the folder fails. We do it as follows: 4. Create a new hop from the Create folder entn Y to this new entry. The hop will be green, & Glckon the litle ion ofthis hop (once or twice) until it becomes red, Double-click on the Write to log entry and fil tas shown in the figure 1: 1 Wty Figure : Configuring the Wite To Log entry ¥. ELvounouss! 1/8 2021-2022 année Cycle dingéieurs~ Gi Business intelligence / Data Lam Prenton anos Warehouse =3 année Cycle d’ingéieurs ~ GI ob Alp dal apall “ae Business Intelligence / Data teCeOt | x0 1estOCl ton tennton cre Warehouse = sete 7. Close the window, Your Job will look lke this (figure 2): creneaiteraer Figure 2: Simple Job with eror handing 8. Save the job. For specifying the name of the new folder, you didn't type a fixed name but a Kettle user- defined variable named ${MY_FOLDER). Let's run the Job so you can see how you set this variable: 1. Click F9 to run the Job. In the Run Options window, select the tab named Variables. In the grid, you should see your variable. You should give it a value before proceeding. You do it by typing a name under the Value column, as shown in the figure 3. Run the Job. A folder with the name you just typed should have been created. If you look at the work area and the result window, you should see that the Create folder entry succeeded; this is marked visually with a green tick on the Create folder icon. This is the reason why the Write To Log entry didn't run, Run the Job again. As the new folder, type the same name as before or ignore it, (don’t type it). This time, you should see that Create folder failed; this is marked with 2 red tick on the Create folder icon, Because of this failure, the Write to log entry runs and sends a message that you can see in the log. ¥. ELYOUNoUSS! 2/8 2021-2022lab hdl pl éulog! dana 3** année Cycle d'ingéieurs ~ GI ith tece0¥ 1 HeeoeaI0 Hesikecl Business Intelligence / Data _— enews Warehouse 2 Run Options Pentaho lca D Options tn emote top leve [Deed Berio beter nneing Cltnte tte mode Chater petermance metic Stan ob | tenant Curent. Dretory ie PEsENSATEFENSAe Since 201-20192018-202 | Internal ob Filename Directory Hle//TESENSATe/ENSATe-Since-2018-2019/2919-202.. | ner Flame ame simplejbeerndingo. | | | neat Name simple job ero, handing | | | meenasb epertenDiecony ers snc 2018209720120. | ae sm teeter amar show dig onrun Ore Figute 3 Setting variables while running «Job 2. Sending emails When you work with ETL processes, Qn, you wil sending emails is a very common requirement. In this learn how to send emails with PDI by enriching your previous work. In this case, the email will replace the log message, For trying this exercise, you need access to an SMTP server. You also need at least one valid account to play with, The example uses Gmail's SMTP server (smtp.gmail.com), but you can use any email server, as long as you have access to the information required by Pol. Y. ELyounouss! 3/8 2021-20223” année Cycle dingéieurs — icc sca exo woo Business Inligence/ Data eatcneenamnes ieee Warehouse Rees Before starting to work with Spoon, let's introduce the kettle.properties file. In this file, you define variables with the JVM scope. We will use it for storing the email settings. ACTION: 1, Open the kettle.properties file located in a folder named .kettle inside your home directory. If you work under Windows, that folder could be (C-\Users\your_user_name\. If you work under Linux (or similar) or Mac OS, the folder will most probably be /home//. 2, Add the following lines, replacing the values between < > with valid information, that is, with your email and password. You may also change the values for the SMTP server as well, Then save the file 2 follows: SMTP SERVERSsntp.gnailcon SMIP_PORT=465 AUTH_USER= AUTH_PASSHORD= Now we will modify the Job: 3. Restart Spoon so it can recognize the changes in the kettle.properties file. 4. Open the Job created in the previous exercise, and save it with a new name. 5. Delete the Write to log entry and replace it with a Mail entry from the Mail category. This is how your Job wil look (figure 4) Sia Creataaifeiger Figure 4: simple job that send emallfo error handling 6. Double-click the Mail entry. Under the Addresses tab, fill the Destination address textbox with a valid email address. in addition, you need to complete the Sender address and the Sender name textboxes. ¥. EL YOUNOUSSI 4/8 2021-20223 année Cycle dingéieurs ~ GI Business intelligence / Oat Warehouse N.B: You can specify more than one destination account, using @ space as 0 separator in the destination address of the Mail Job entry 7. Complete the Server tab as shown in the following screenshot (figure 5): eS gestae | ase We nshemtesin? athens (SETTRTSTAL =a) eset ntbetcton? Secetconmtan type Over Figure 5: Configuing the server ina mal lb entry 8. Complete the Email Message tab. 9, Save the job and run it. N.B: if it doesn’t work, you should probably turn on «allow less secure apps » Gmi option. 3, Running Transformations from a Job We have already mentioned that jobs are typically composed of a sequence of a several transformations. Then, usually, you don't run a PDI Transformation isolated; you embed itina job. Y.ELYOUNOUSSI 5/8 2021-20223#* année Cycle dingéieurs —G a tant Bonne mies Oat ieee er ert viornoe ne In this section you will learn how to run a transformation from a job using a special Job entry, the transformation entry. ACTION: 1. Createa new Job. 2. Expand the General category of entries and drag a START and a Transformation entry to the canvas, 3. Create a hop from the START entry to the Transformation entry. 4, Save the Job. Before continuing, look for the last Transformation of TP2. Open it and save it under a different name, Add a Microsoft Excel Output step after the last step of the transformation and configure it by adding the Excel file name (in the File tab) and the fields metadata (by clicking on the Get Fields button). Save the transformation and close it 5. Now, double-click on the Transformation entry. 6. Fill in the Transformation field by browsing and selecting the transformation file on OK. you've just created. 7. Save the job and run it. 4, Changing the flow of execution based on conditions The execution of any Job entry either succeeds or fails. For example: ‘+ AMail entry succeeds if PDI can send the message, but it falls if the specified SMTP. host is unknown, * Create a folder succeeds if PDI can create the folder, but fails ifthe folder existed or there were no permissions for creating it. So, when you create a Job, you not only arrange the entries and hops according to the expected order of execution, but also specify under which condition each Job entry runs. The conditions are defined in the hops. You can identify the different hops either by looking at the color of the arrow or at the small icon on it. The following table lists all kind of hop conditions: Option Icon Y. El Younouss! 6/8 2021-2022SST Peer re unde ce og = ‘stun Incond > | Thenest ob entry wil be executed regardtess of Unconditional the result of the oxiginating Job entry Follow when “The nent job entry willbe executed only when the result is true oar result ofthe originating job entry is true. Follow when “The next job entry will only be executed when the result is false -9—>— | result of the originating job entry was false To change the hop behavior (color), do any of the following: ‘= Right-click on the hop, select Evaluation, and then the condition. * Click on the small icon on the hop. Each time you click on it, the type of hop changes. Finally, a hop can also look light grey. This means that the hop has been disabled. if you don't want an entry to be executed (mainly for testing purposes), you can disable the incoming hop or hops. ACTION: 1. Open the last job and save it under a different name. 2. Change it in order to display a success message if the transformation would successfully run and a fall message if the transformation would fail. To display the message, use the Display msgbox info entry. 3. Save the job and runit, 5. Changing the execution to be synchronous ly set that action. Todo that, Ifyou intend to run several entries in parallel, you can expl right-click on the entry from which the parallel task will run. Then select Launch next entries in parallel, ACTION: Y, EL oUNoUss! 78 2021-2022gt année Cycle d'ingéi cor a a scat Business Intelligence /C Werehouse fees sa ta NB: | open the first job of this tutorial which creates 2 folder using 2 defined user variable. Save it under a different name. Modify it to look like the job in the figure 6 bellow Créatefle3 ‘arti Figure 6: Running Job entiles in parale! Dummy entry is a special entry that does nothing, Abort entry forces the job to abort after executing Write to log task. ¥.& L YOUNOUSSI 3/8 2021-2022