You are on page 1of 53

Groovy Business Process Automation

Olaf David
Colorado State University Dept. of Civil Engineering, Dept. of Computer Science US Department of Agriculture

Objective

Automate and integrate software project management practices using CB with respect to
◦ Trackers ◦ Forums ◦ Builds ◦ Reports ◦ Documents ◦ Users by using an efficient scripting approach.

Olaf David, CSU/USDA

2

A First Example
import cbscript.CB import static cbscript.CB.* jf = CB.login("http://localhost:8080", “bond", “pass") proj = jf.projects.find{it.name == "MyProj"} needsBuild = proj.trackers.find{it.name ==“Bugs"}.items.any{ bug -> bug.status == "Fixed" && isThisMonth(bug.submitted) && bug.priority > LOW && bug.hasCommits } if (needsBuild) { println "starting build ....“ log = proj.builds.find{it.name == “BuildApp"}.invoke() println " Build done: ${log.status} “ println " Output: ${log.stdOut} " } jf.logout();

Olaf David, CSU/USDA

3

Principal Architecture
Groovy API -High level, object centric, and productivity oriented -For process automation

cbscript

cb-api

CB Server

Java API -Low level, session centric and stateless -For a wide range of applications
Docs DB SCM

Olaf David, CSU/USDA

4

Why Scripting?

Semantic density and flexibility of scripts
◦ Usability/Efficiency aspects of scripts vs. plain Java vs. Web UI

Remote Management of (multiple) CB server  Leveraging general scripting productivity (Groovy)

◦ Closures, Properties, Duck typing ◦ Builder (DSL)

CB web UI does not allow complex queries  No automation using the CB Web UI

Olaf David, CSU/USDA

5

Prerequisites
Java 1.6+  Groovy 1.5+  cbscript-5.3.1.jar

◦ Contains cb-api.jar of the same CB version

CB 5.3.x
◦ Network access to context
‘<cburl>/cb/remote-api’

◦ Project user access (account)

Olaf David, CSU/USDA

6

Login/Logout
import cbscript.CB import static cbscript.CB.* jf = CB.login("http://localhost:8080", “bond", "007") // or jf = CB.login( url: "http://10.177.19.63:8080", login: "bond", password:“pass") // do some work. jf.logout();

API respects CB licenses, access permissions

Olaf David, CSU/USDA

7

Session information
assert jf.login == "bond" assert jf.url == "http://localhost:8080" assert jf.user.realName == "Default System Administrator"

meta information can be obtained about the session.  read/only

Olaf David, CSU/USDA

8

Getting Projects
// list all projects jf.projects.each { println " id: ${it.id} name : ${it.name}" } // find project(s) assert jf.projects.find{it.name == "Moose"}.id == 2374 assert jf.projects.find{it.id == 2374}.name == "Moose“ jf.projects.findAll{it.createdBy.name == "bond"}.each{ println it.name }

‘projects’ returns a list of projects  Using closures to iterate, filter, search

Olaf David, CSU/USDA

9

Managing Tracker
// get all tracker of the ‘oms’ project. jf.projects.find{it.name == "oms"}.trackers.each{ tracker -> print tracker.name print tracker.description }

‘trackers’ returns a list of tracker  Use closures to iterate, filter, search

Olaf David, CSU/USDA

10

Tracker Items query
// Find items that are not Closed def bugTracker = jf.projects.find{ it.name == "oms“ }.trackers.find {it.name == "Bugs"} bugTracker.items.findAll{ it.status != 'Closed' }.each{ println it.name }

Closures help traversing resources  Obtain the Bug tracker in the oms project. Then find all the ‘Closed’ items

Olaf David, CSU/USDA

11

Tracker Items query (cont.)
bugTracker.items.findAll{ it.assignedTo == jf.user && it.status == 'New' }.each { println it.name }

Find all items assigned to me and submitted last week

Olaf David, CSU/USDA

12

Tracker Items query (cont.)
bugTracker.items.findAll{ it.summary.contains('big problem') }.each{ println it.name }

Find a tracker item that has the string 'big' in summary

Olaf David, CSU/USDA

13

Tracker Items query (cont.)
bugTracker.items.findAll{ it.status == "Resolved" && it.assignedTo.name == "bond" && thisMonth(it.submittedAt) }.each{ println it.name }

All 'Resolved' items assigned to user 'bond' that were submitted this month

Olaf David, CSU/USDA

14

Tracker Items query (cont.)
bugTracker.items.findAll{ return it.submitter == jf.user && !it.hasCommits }.each{ println it.name }

All tracker items submitted by me and do not have an attached SCM commit.

Olaf David, CSU/USDA

15

Tracker Items query (cont.)
bugTracker.items.findAll{ bug -> bug.priority == HIGH && bug.status != “Closed” && bug.assigned == null }.each{ println it.name }

All tracker items with high priority that are still open and are not assigned yet to a developer.

Olaf David, CSU/USDA

16

Submit a new task
// get the bug tracker def bugs = jf.projects.find{ it.name == “oms"}.trackers.find {it.name == "Bugs"} bugs.submit ( new Task(name:"My Bug", text:"This is a new Bug", priority:NORMAL) )

Use ‘Task’ properties to configure  Priority: LOWEST, LOW, NORMAL, HIGH, HIGHEST

Olaf David, CSU/USDA

17

Forums
def newsForum= jf.projects.find{ it.name == "oms"}.forums.find {it.name == "News"} newsForum.findAll{ post -> isToday(post.submitted) }.each{ print it.text }

‘forums’ contains all project forums  Check for new posts today.

Olaf David, CSU/USDA

18

Post a new Forum message
news= jf.projects.find{it.name == "oms"}.forums.find {it.name == "News"} news.post( new Message(subject:"New message", text:"Content here ...")

‘forums’ contains all project forums  Check for new posts today.

Olaf David, CSU/USDA

19

Check for Posts
news.findAll{ post -> post.submitter.realName == “Joe Poster” && thisWeek(post.submitted) }.each{ print it.text }

Check for all new posts from “Joe Poster” submitted this week and print out the message text.

Olaf David, CSU/USDA

20

Documents
prj = jf.projects.find{it.name == "cbtest"} doc = prj.artifact(‘download/EC2TW.txt’) assert doc.bytes.size() == 709 assert doc.name == ‘EC2TW.txt’ assert doc.directory == false

Use ‘artifact’ to reference a document or folder in the project ‘Documents’  Always absolute to ‘Documents’ root.

Olaf David, CSU/USDA

21

Documents (cont.)
prj = jf.projects.find{it.name == "cbtest"} doc = prj.artifact("download/EC2TW.txt") new File('c:/tmp/test1').withOutputStream { it.write doc.bytes }

Copy a file from a CB server to the local FS

Olaf David, CSU/USDA

22

Builds
build = prj.builds.find{it.name == "build-lib"} log = build.invoke();

‘builds’ contains all the builds of a project.  Call ‘invoke’ to execute the build on a CB server. (Invoke is a synchronous call)

Olaf David, CSU/USDA

23

Builds (cont.)
build = prj.builds.find{it.name == "build-lib"} log = build.invoke(); assert assert assert assert assert log.output != null log.error == null log.status == “Successful” log.successful == true log.startedBy == jf.user

Build ‘log’ captures the result of the build.

Olaf David, CSU/USDA

24

Associations
bug = bugTracker.submit{ new Task(name:"My Bug", text:"This is a new Bug", priority:HIGH) } def lib = prj.artifact("builds/ngmf.jar") bug.relate(lib, RELATED)

Create associations by calling ‘relate’ on tasks, artifacts, documents, etc.  Pass in the ‘other’ task, artifact, document and the kind of relationship (RELATED, CHILD, PARENT, DEPENDS)

Olaf David, CSU/USDA 25

Associations (cont.)
readyToBuild = prj.trackers.find { it.name == "Bug“ }.items.findAll { it.status == "Fixed" && thisMonth(it.submitted) } readyToBuild.each { bug -> milestone.relate(bug, DEPENDS) }

Create a Association to many items in one ‘bulk’ operation (use closure)  ‘milestone’ is a tracker item

Olaf David, CSU/USDA

26

InterWiki links
build = prj.builds.find{it.name == "build-lib"} result = build.invoke(); String wikitext = “””!!Milestone info * Output Log Reference: ${CB.link(result)} “”” // resolves to: // “ Output Log Reference : [BUILDLOG:23456]”

Use ‘CB.link()’ to create interwiki links for task messages, forum posts, artifact descriptions to reference other resources
Olaf David, CSU/USDA 27

Examples* Examples*
Problem elevation SCRUM support SCM commit control Auditable Build Management

* The examples demonstrate the support of some aspects of the topics above

Olaf David, CSU/USDA

28

Simple Problem Elevation
bugs.findAll{ bug -> bug.status == “Verified" && bug.assignedTo.name == “Joe” && bug.spendHours == 0 && !bug.hasCommits && bug.submittedAt < new Date(2009,4,1) }.each{ it.priority++ }

Increase the priority of all verified bugs submitted before April that are assigned to Joe that he did not fixed… so far.  Call this above every day (e.g. cron)

Olaf David, CSU/USDA

29

SCRUM Support
jf.projects.find{it.name == "oms"}.trackers.find{ it.name == "Features"}.items.findAll { task -> task.status == "Closed" && withinDays(task.modifiedAt, 30) }.each { println "${it.summary} - ${it.description}" }

Prepare a Sprint review meeting
◦ Check which tasks in 'Feature' requests in the 'oms' project were finished over the last 30 days.

Olaf David, CSU/USDA

30

SCRUM support (cont.)
jf.projects.find{it.name == "oms"}.trackers.find {it.name == "Bugs"}.items.findAll{ it.status != "Closed" && it.submittedAt > new Date(2008, 1, 4) }.each { println "${it.summary} - ${it.description}" }

Prepare a Sprint review meeting
◦ Get all the bugs that are still not closed and were submitted since April 1st 2008 (last sprint meeting)

Olaf David, CSU/USDA

31

SCRUM Support (cont.)
def fitems = jf.projects.find{it.name=="oms"}.trackers.find{ it.name == "Features"}.items.findAll { it.category == ‘UI’ } (-7..0).each { day -> map[day] = fitems.findAll { it.status == "Closed" && daydiff(it.modifiedAt, day) }.size() } println map // [-7:10, -6:3, -5:12, -4:4, -3:10, -2:5, -1:1, 0:2]

Creating a sprint backlog for all closed tasks in the UI category of new features.
key: day diff to today (-1 is yesterday) value: the number of closed tasks
Olaf David, CSU/USDA

◦ The map contains the last seven days,
32

SCRUM Support (cont.)
hours = 0 def fitems = jf.projects.find{it.name=="oms"}.trackers.find{ it.name == "Features“ }.items.findAll { it.status == “InDevelopment” }.each { hours += it.estimatedHours – it.spentHours } f = new File(“burndown.dat”) today = new Date() f.append("${today} , ${hours}")

Creating daily burndown chart data
◦ Call this script every day to track the hours being ‘burned’ in developing new features . ◦ Create the graph based on this file.
Olaf David, CSU/USDA 33

SCM Commit triggers status change jf.projects.find{it.name == "OMSProject"}.trackers.find {
it.name == "Bugs“ }.items.findAll{ it.hasCommits&& it.status != "Closed“ }.each { task -> task.commits.each { if (it.message ==~ /.*\(status=(.*)\).*/ && today(it.commitedAt)) { task.status = str.replaceAll(/.*\(status=(.*)\).*/, { all, status -> return status }) } } }

A SVN commit message such as

svn commit –m “#1234 (status=Fixed) Changed loop order”

will trigger a status change on a task to fixed; no separate UI operation needed. (Put this in cron and execute frequently)

Olaf David, CSU/USDA

34

Auditable Build Management

Requirement

◦ Flexible build process triggering by tracker content information ◦ Capture all information that relate to a build
Bugs <-> Build Features <-> Build Build process log <-> Build Build product <-> Build

◦ Developer independent build control ◦ Flexible build notification ◦ Automated build product download

Olaf David, CSU/USDA

35

Pick up Edit Comment Status change

Bugs Feature Requests

A project

Developer changes task status to.. Fixed

DE V

Implemente d Commit assoc.

Commi t Update Push Pull

Repo

SC M

The developer’s view

Olaf David, CSU/USDA

36

Auditable Build Management (1)
import cbscript.* import static cbscript.CB.* // login to CB jf = CB.login("http://localhost:8080", "bond", “pass") // find the project of interest prj = jf.projects.find{it.name == "OMSProject"} ...

Login and get the project reference

Olaf David, CSU/USDA

37

Bugs Feature Requests Milestones Build-lib

A project
Identif y

L ocal

DE V

(2 )

MG R

SC M Docume nts

Olaf David, CSU/USDA

38

Auditable Build Management (2) ...
// normal and higher priority bug fixed this month readyBugs = prj.trackers.find {it.name == "Bug"}.items.findAll { it.status == "Fixed" && isThisMonth(it.submitted) && it.priority > LOW } // A feature is implemented and there is a commit readyFeatures = prj.trackers.find { it.name == “Features"}.items.findAll { it.status == "Implemented" && it.hasCommits } // combine all features and bugs into one single list readyList = readyBugs + readyFeatures ...

 

Find ‘candidates’ in ‘Bugs’ and ‘Features’ that should lead to a new build Combine the lists in to one.
Olaf David, CSU/USDA 39

Bugs Feature Requests

A project

L ocal

DE V

Issue build

Milestones Build-lib
3.2 build 3.3 log 3.1 checkout/update

(3 )

MG R

SC M

3.4 <cbrelease

Docume nts
Olaf David, CSU/USDA 40

Auditable Build Management (3) ...
// build check if (!readyList.empty) { // invoke the build. build = prj.builds.find{it.name == "build-lib"} result = build.invoke(); ...

If the list is not empty, we need to build  Find the build and invoke it.  The build will create the file ‘builds/ngmf.jar’ in ‘Documents’

Olaf David, CSU/USDA

41

Bugs Feature Requests Milestones Build-lib

A project

L ocal

DE V

New milestone build

(4 )

MG R

SC M

Docume nts
Olaf David, CSU/USDA 42

Auditable Build Management (4) ...
String wikiinfo = "!!Milestone info \n" + "!Milestone Build:\n" + "* Build Status: ${result.status}\n" + "* Output Log Reference: ${CB.ref(result)}\n" // create a new milestone in the milestone tracker mt = prj.trackers.find {it.name == "Milestones"} milestone = mt.submit( new Task(name:"MS-1.2", text:wikiinfo, priority:LOW))

...

  

Create a WIKI description with the result info (status and link to the Buildlog) Submit a new milestone task with the WIKI content. Notification will be sent on submission.
Olaf David, CSU/USDA 43

A project Bugs Feature Requests Milestones Build-lib
5.2 relate

L ocal

DE V

(5 )
5.1 relate / link

MG R

SC M

Docume nts
Olaf David, CSU/USDA 44

...

Auditable Build Management (5)
// the build product def lib = prj.artifact("builds/ngmf.jar") // relate the milestone to all // bugs/features/and the build product milestone.relate(lib, RELATED) readyList.each{ task -> milestone.relate(task, DEPENDS) }

...

Relate the new milestone to all bugs and features in the readyList.  Also relate it to the build product (ngmf.jar)

Olaf David, CSU/USDA

45

A project Bugs Feature Requests Milestones Build-lib
6.2 copy 6.1 Task Status change .. Fixed -> ReadyForTesting Implemented -> ReadyForTesting

L ocal

DE V

(6 )

MG R

SC M
ngmf.jar

Docume nts
Olaf David, CSU/USDA 46

...

Auditable Build Management (6)
// copy the new lib from CB to my local FS (optional) new File('c:/tmp/ngmf.jar').withOutputStream { it.write lib.bytes } // change the status in all items in readylistto // "ReadyforTesting" readyList.each{ task -> task.status = "ReadyForTesting" }

}

Copy the build product to the local FS for testing to skip manual download. Change the status for all bugs and features to “ReadyForTesting” to indicate their processing
Olaf David, CSU/USDA 47

Before invocation and …

Olaf David, CSU/USDA

48

… After …

Olaf David, CSU/USDA

49

.. and After ..

Olaf David, CSU/USDA

50

.. and After ..

Olaf David, CSU/USDA

51

.. and After ..

Olaf David, CSU/USDA

52

References

http://cbscript.javaforge.com
◦ CB script examples, download cbscript.jar

https://codebeamer.com/cb/wiki/18830
◦ Codebeamer API

http://groovy.codehaus.org
◦ All about Groovy

 https://codebeamer.com

◦ Codebeamer

Olaf David, CSU/USDA

53