P. 1
Autosys

Autosys

|Views: 1,591|Likes:
Published by Joshi Medidi

More info:

Published by: Joshi Medidi on Apr 05, 2011
Copyright:Attribution Non-commercial

Availability:

Read on Scribd mobile: iPhone, iPad and Android.
download as PPT, PDF, TXT or read online from Scribd
See more
See less

11/06/2012

pdf

text

original

Concept of AutoSys - Apr 2010

Syed Mohani

INVESTMENT BANK TECHNOLOGY

Agenda / Topics to be Covered

Topics:------------------------------------ What is autosys how it works --- How to check Autosys is running. --- Jil (Job Information Language) --- How to setup a new/delete/upgrade Jobs --- Autosys vs Cron jobs --- Common commands used (Operate Prospective) --- Automatic conversion from Cron jobs to Autosys jobs
INVESTMENT BANK TECHNOLOGY

--- Changes from Solaris to Linux Environment AutoSys jobs --- Automation on Monitoring --- Demo --- Q & A

2

What is Autosys

AutoSys is an automated job management system that lets operate schedule, monitor and report.

This presentation is intended for Operate who will be responsible for defining, monitoring and managing jobs run by AutoSys.

Using Autosys utilities, we can define a job by assigning it a name and specifying the arrributes that describe its associated behavior.

--- The AutoSys job information Language (JIL)
INVESTMENT BANK TECHNOLOGY

3

Event Server (Autosys Database) --.Remote Agent INVESTMENT BANK TECHNOLOGY 4 .Software Components From a software perspective.Event Processor --. the AutoSys environment is composed these main components:--.

Jobs Types --.File Watcher Jobs --.Box Jobs INVESTMENT BANK TECHNOLOGY 5 .Command Jobs --.

Box Jobs A Box is a container of jobs with like starting conditions. either Date/time conditions or job dependency conditions. INVESTMENT BANK TECHNOLOGY 6 .Jobs Types --.

autostatus (to check connectivity with the database) ( -x returns ver info -D check the database info (to check the job status) (Status of the job) INVESTMENT BANK TECHNOLOGY 7 . ---.chk_auto_up ---. configure.autoping ---.AutoSys Commands and Monitors There are several commands unique to AutoSys that can be used to control.autorep ---. and report on the system behavior.

update_job --.delete_job --. --.AutoSys Job Definition There are four major JIL subcommands.delete_box Allows to add a new job to AutoSys Allows to change an existing job Allows to remove an existing job Removes a box job and all of its subordinate jobs INVESTMENT BANK TECHNOLOGY 8 .insert_job --.

Job dependency conditions --.success(job_name) INVESTMENT BANK TECHNOLOGY --.notrunning(job_name) --.done(job_name) 9 .Data and time condition --.days_of_week --.run_window --.AutoSys Job Definition A job performs a particular action based on a set of instructions called a job definition.failure(job_name) --. Starting Conditions: --.start_times --.

has never been run) (Exit code equal) (Job exited with an exit code) (Job terminated while in the RUNNING state) (Job was unable to start (or failed) and schedule to restart) (Job is itself has not yet started. (Job is on hold and will not be run until it receives the JOB_OFF_HOLD. On the other hand. but is still defined.ON_ICE --.INACTIVE --. if an "on ice" job is taken "off ice. even if its starting conditions are already satisfied.Job Status At every point in processing. --.ON_HOLD The difference between "on hold" and "on ice" is that when an "on hold" job is taken off hold.RESTARTED --.) (Job is removed from all conditions and logic.SUCCESS --.STARTING --. 10 .RUNNING --.FAILURE ST RU IN SU FA (The job is in the process of ‘coming up”) (The job is running) (Job has not been processed. a job has what is called its current status or “state”.ACTIVATED INVESTMENT BANK TECHNOLOGY RE AC OI OH --. and it will run. if its starting conditions are already satisfied." it will not start. This job will not run until its starting conditions reoccur.TERMINATED TE --. it will be scheduled to run. --.

Common Commands # autorep -j %lmc% Job Name uat_lmc_live_rate_Dly Syntax Last Start Last End ST Run Pri/Xit 04/14/2010 14:26:03 ----- RU 1868/1 autorep {-J job_name -M machine_name -G global_name} [-s -d -q -o over_num] [-r run_num] INVESTMENT BANK TECHNOLOGY 11 .

0 SP2 46a989d1 snjcqacriba01 INVESTMENT BANK TECHNOLOGY To Check AutoSys is up and running:# chk_auto_up Primary Scheduler is RUNNING on machine: <Machine-name> # ps –ef |grep autosys # autoping -m <Machine-name> 12 .:# chase -E -A Display all jobs scheduled to run between these two dates: job_depends -t -J ALL -F "04/01/2010 00:00" -T "12/26/2010 00:00" Get version information:# autoflags -a INC2 LINUX ANY 11.e test-for-autosys-job-creation Checking on Jobs in the STARTING state.Common Commands # autostatus -j <job-name> SUCCESS i.

autosyslog -e --.archive_events --.autoping -m ALL -D --.autoflags -a --.chk_auto_up Definuing Calendars --.autocal --.autosys_secure (Maintain Autosys Edit and Exec super user owner) 13 .chase –A –E --.dbstatistics Managing Security --.clean_files –d days --.autocal_asc Maintaining Databases INVESTMENT BANK TECHNOLOGY (info about autosys and system configuration) (Verifies versions AutoSys comm are correctly) (Display the Event Processor log files) (Verifies autosys database are running) (Inspects the DB /environment variables) (Adds. delete. and print custom calendar) (Removes old info from autosys database) (Removed Remote Agent Log Files) --.Common Commands Checking system Status --.

sendevent Backup of All jobs --.autorep --.autorep –J ALL –q > dump_file INVESTMENT BANK TECHNOLOGY (Reports current status of the job) 14 .autostatus –j <job-name> AutoSys job status send --.Common Commands Reporting Job Dependencies and Conditions --.job_depends Reporting Job status --.

Common Commands To check the jobs schedule between two Dates:# job_depends -t -j ALL -F "04/01/2010 00:00" -T "04/30/2010 00:00“ Job Forecast Report From: 04/01/2010 00:00:00 To: 04/30/2010 00:00:00 Job Name ______________________________ qa_cri_batch02_queue_Health_c_min qa_cri_batch03_queue_Health_c_min INVESTMENT BANK TECHNOLOGY Next Start Atomic Start Conditions _______________ 04/21/2010 18:00:00 ------04/21/2010 18:00:00 ------- 15 .

Common Commands To specify that the jobs can be run on any day except those days specified in the <nameholidays> calendar. exclude_calendar : <name-hoilidays> Sendevent:- sendevent -J <job-name> sendevent -J <job-name> sendevent -J <job-name> sendevent -J <job-name> INVESTMENT BANK TECHNOLOGY -E CHANGE_STATUS -s TERMINATED -E STARTJOB -E FORCE_STARTJOB -E JOB_ON_ICE sendevent -E DELETEJOB -J <job-name> 16 .

Common Commands Logs:# autosyslog -e -J <job-name> INVESTMENT BANK TECHNOLOGY 17 .

SendEvents Options To start or force start a job manually using sendevent : sendevent –E FORCE_STARTJOB -J <job-name> sendevent -E STARTJOB -J <job-name> To put jobs on OFF ICE or ON ICE : sendevent -E OFF_ICE -J <job-name> sendevent -E ON_ICE -J <job-name> INVESTMENT BANK TECHNOLOGY 18 .

This option is required. sendevent is normally used with "-E" & -J option -J job_name : Specifies the name of the job to which the specified event should be sent. Any one of the following events may be specified as: STARTJOB KILLJOB DELETEJOB FORCE_STARTJOB JOB_ON_ICE JOB_OFF_ICE JOB_ON_HOLD JOB_OFF_HOLD CHANGE_STATUS STOP_DEMON CHANGE_PRIORITY COMMENT ALARM SET_GLOBAL SEND_SIGNAL 19 INVESTMENT BANK TECHNOLOGY . This command is also used to set AutoSys global variables or cancel a scheduled event. stopping the Event processor. and putting a job on hold. including starting or stopping AutoSys jobs. -E event :Specifies the event to be sent.SendEvents Options Sendevents to AutoSys for a variety of purposes.

Command Attributes AUTOREP USAGE: autorep [-J JobName] [-d | -s | -q | -o OverRide # | -a | -w] [-B GroupName] [-I ApplicationName] [-t] [-X ExternalInstanceName] [-Y JobType] [-R Run_num] [-L PrintLevel] [-N Retry] [-x] [-z GlobalBlobName] [-f BlobDownloadDirectory] [-G GlobalName] [-M MachineName] [-D DataSource] [-?] INVESTMENT BANK TECHNOLOGY 20 .

jil..bak -rwxr-xr-x 1 citwl ugcrmcit 423 Apr 14 11:42 uat_lmc_live_rate_Dly.Job Physical loaction # pwd /cri_staging/syed/jil/lmc *** 2193 smccuatcriwa06 citwl:ugcrmcit /cri_staging/syed/jil/lmc # ls -altr total 20 drwxr-xr-x 3 citwl ugcrmcit 4096 Apr 13 17:27 .20100414 INVESTMENT BANK TECHNOLOGY -rwxrwxrwx 1 citwl ugcrmcit 395 Apr 14 12:07 uat_lmc_live_rate_Dly.jil 21 . -rwxr-xr-x 1 citwl ugcrmcit 423 Apr 13 18:31 uat_lmc_live_rate_Dly.

---.Jobs Streams A job stream (or Flow) refers to the logic of how related AutoSys jobs are run. It will run only on the successful completions of job_backup. INVESTMENT BANK TECHNOLOGY 22 .Job dependency conditions lob_backup Command job has date and time condition (Runs every day of the week at 01:01 am Job_clean Command job has a job dependency condition specified for its starting parameters.Date and Time conditions ---.

When condition is met. Example job is to be run every day of the week at 2:00 pm.Jobs Streams File Watcher Jobs:A file watcher job is used to watch for an incoming file. INVESTMENT BANK TECHNOLOGY 23 . Job has date and time conditions specified For its starting condition. The job_Fwatch job will check every 15 second for the creation of the incoming File. job_Fwatch will notify the server and complete with a SUCCESS status. It watches For a file arriving.

JIL to create jobs definitions.update_job --.Machine Name --. we. fr) (condition: success(my_job) (watch_file: /tmp/<file-name> 24 .insert_job --. This job definition must be submitted to the AutoSys database before the job can be run.Job Type (c f and b) --.Defining and Running jobs using Job Information Language (JIL) We can use AutoSys proprietary scripting language --.Job Name --. JIL Sub-Commands:--.delete_job Jil Attributes Types:INVESTMENT BANK TECHNOLOGY --.File to Watch For (insert_job: <job-Name>) (job_type: c) (machine: <machine-name>) (days_of_week: mo.Starting Conditions --.

out std_out_file: /cri_staging/syed/jil/demo/test-output.jil> # JIL Script # This script will re-direct a message to the file /tmp/test-output.jil 25 .JIL Syntax Writing a JIL script.out Saving a job: Jil < test-for-autosys-job-creation.mx command: /bin/echo "AutoSys KT demo" > /cri_staging/syed/jil/demo/test-output. follow the syntax rules:1 – sub_commnad: job_name (insert_job: <user-define-job-name> 2 – attribute_keyword: value Example:Creating a job: vi <test-for-autosys-job-creation.wx.out update_job: test-for-autosys-job-creation job_type: c INVESTMENT BANK TECHNOLOGY machine: cri_batch_job_queue permission: gx.

sh machine: cri_web_job_queue owner: citwl@smccuatcriwa06 permission: gx.th.we.stdout std_err_file: /local/ss/cit/userprojects/log/lmc/lmc_liverate.uat_lmc_live_rate_Dly ----------------.*/ insert_job: uat_lmc_live_rate_Dly job_type: c command: /apps/ss/userprojects/lmc/live_rate.stderr alarm_if_fail: 1 26 .Jobs Details # autorep -j uat_lmc_live_rate_Dly -q /* ----------------.tu.mx date_conditions: 1 days_of_week: mo.fr INVESTMENT BANK TECHNOLOGY start_times: "22:00" description: "Live Rate Job" std_out_file: /local/ss/cit/userprojects/log/lmc/lmc_liverate.wx.

JIL Syntax Delete a Job: At the unix Command Prompt # jil <enter-key> At the jil prompt jil>>1> delete_job: <job-name> exit INVESTMENT BANK TECHNOLOGY 27 .

sh` machine: snjprdcria1.com #owner: capbg@snjprdcria1.received the outpuy from save Attempting to Delete Job: prd_cit_cap_CPPfile_fwatch Jobs which are dependent on the DELETED Job: prd_cit_cap_CPPfile_fwatch prd_cit_cap_CPPmonthRun Delete was successful! Exit Code = 0 28 .Save the jil file into Database.ge.*/ delete_job: prd_cit_cap_CPPmonthRun job_type: c command: `cd /home/capbg/scripts. .Job Delete thru jil:Delete the Job:Step 1 --.jpmchase./CPP_monthly.Vi the jil file and replace command with delete_job: /* ----------------.jpmchase.wx condition: success(prd_cit_cap_CPPfile_fwatch) std_out_file: /home/capbg/scripts/CPP_month. INVESTMENT BANK TECHNOLOGY # jil < [Jil-file-name] Step 3 --.com permission: gx.stderr alarm_if_fail: 1 Step 2 --.stdout std_err_file: /home/capbg/scripts/CPP_month.prd_cit_cap_CPPmonthRun ----------------.

jil or *.convert files names i. cp_[123] run_calendar: <callendar-name> autocal_asc -s test_demo -E test_export.input as crontab file ---. respectively.cal files are generated.txt INVESTMENT BANK TECHNOLOGY Export the calendar entries into file autocal_asc -lc List the cycle calendar Note: cron2jil does not generate JIL files for jobs that are defined in crontab to start every minute After the *. 29 .Translates crontab files into JIL format cron2jil -f crontab_file [-d output_directory] [-i include_file] [-m machine] [-p prefix] ---. you must submit them to the database using the jil and the autocal_asc commands.Auto Transfer Cronjobs into Autosys jobs Commands:cron2jil .# cron2jil –f file-name ---.e.

each of which must be delimited with the calendar: Calendar_name attributes. These test files may contain multiple calendars. Calendars contained in ASCII test files can be imported Into AutoSys database. We can import calendar text files. For expample.Auto Transfer Cronjobs into Autosys jobs Combing Calendars:Calendars can be combined in a number of ways. calendar: Q1paydays 01/01/2010 01/15/2010 02/01/2010 INVESTMENT BANK TECHNOLOGY 02/15/2010 03/01/2010 06/15/2010 calendar: Q1holidays 01/01/2010 30 . we can create a calendar that includes all the dates that are in either one calendar or another.

This can be overridden manually by way of the Date Range option from the options menu. ! 1 = prior year plus current year. ! 2 = thru next year.Auto Transfer Cronjobs into Autosys jobs Date Range:The following resource sets the number of years in the data range of the calendar. ! 10 = thru tenth year Autocal. 5 = thru fifth year.dateRange: INVESTMENT BANK TECHNOLOGY 31 . ! 4 = thru forth year. as a default at start up. 3 = thru third year.

mx                <-----.net citssbatchapp[1234]-qa.us.jpmchase.us.us.New Setup for Linux Environment Reporting Job Dependencies and Conditions Virtual Machine crrt_job_queue crm_crws_job_queue crm_rad_job_queue cri_batch_job_queue cri_web_job_queue Physical Machine citcrrtapp[12]-qa.jpmchase.net citportalapp[123]-qa.jpmchase.   32 .net machine: cri_web_job_queue    <-----.wx.Job Permissions add “mx” INVESTMENT BANK TECHNOLOGY To convert existing Jobs definitions Jil out the current job definition from existing environment into a text file Update machine name with virtual machine name (refer next slide ) update existing permission to add additional permission “mx” Jil in the job definition to new environment.us.us.net citportalapp[12]-qa.jpmchase.net citsswebapp[1234]-qa.Virtual Machine Name permission: gx.jpmchase.

Env-App-JobName XX-XXXXX-XXXXXXXXXXXXXXXXXXX XX = XXXXX = INVESTMENT BANK TECHNOLOGY [DR/ST/QA/UT/PR] [CMART/CAPLO/BASEL/IBFTP/……] XXXXXXXXXXXXXXXXXXXX = JOB Name 33 .AutoSys Jobs naming convention Get the AutoSys jobs status thru web tool. While apply the naming convention we can write a cgi script and use apache to get status on web.

Priority: 1 Set the job priority for run. INVESTMENT BANK TECHNOLOGY Run_window: <time-time> Before starting the job autosys will check the window time.AutoSys vs CronJob max_run_alarm: <mins> Specifies the max run time that a job should require to finish normally. Such as the applications stuck in a loop. watch_file: <path> Attributes used as Watch file min file size and watch interval 34 . run_calender: <calendar-name> Running job on the last business day of the every month. n_retrys: 5 To set the job to be automatically restarted up to five times after an application Failure. This test can catch an error.

INVESTMENT BANK TECHNOLOGY Q&A 35 .

You're Reading a Free Preview

Download
scribd
/*********** DO NOT ALTER ANYTHING BELOW THIS LINE ! ************/ var s_code=s.t();if(s_code)document.write(s_code)//-->