@...

&,
@ 0 • ( Slide show I •) lltJ Share
>ols Help

© View only

Background
Cu rre nt Situation:

C2 LKG
- Ju kw aa LKG
tion.
76 repos in github un de r houzz organiza
svc : ov er 10 0 sub mo du les , for diff ere nt language. java, python, go
Within C2

Containerized services (in k8s)

t~ production: 66

Staging: 69
 CIC;D concepts

Notif icati on

l clean
J Faile d
'I R~e wI
I I
setu p
Succ eed Stag ing
f conm its~ Style chec k Publ ish

I I
I

I
lcod e B a s e p - ~ - - - _
..o I Build I prod uctio n
I

Unit Test
l 1
othe r relat ed
I Integ ratio n Test ]
code Base
CD PIPELINE
Cl PIPELINE
I
I
 .
.

Ho w to ad ap t ou r ne w CIC D flow \\

1. crea te ci-c onfi g.js on file into your ·rep o or mod ule
get hoz exec utab le from aws s3
aws ·s3 cp s3://h ouzz~ packa ges-s tagin g/won derho uzz/h oz .

und er you r targ eted folde r

run "hot init"

Mak e chan ges _to ci-config.json

,.

exam gle
che ck-i n the ci-co nfig file into your repo and notify productivity team
2. Wai tinQ for noti fica tion from hoz-pipeline in yo'ur specific slack chan
nel.

'1'1 D
(\ (\ Hoz-Pipeline APP 3:01 PM
Hoz check passed in wonderhouzz master branch wonderhouz_z after commit fbf2878d

Jenkins Job link

Cl Work flow
1. Checkin code
2. Github trigg~r pipeHne check HOZ pipelines
3. HOZ parse changeset rhodule
4. if ci-config.json presents, parse Cl commands
5. Execute each cqnlmand which we supports
6. Parse result·
7. N,otification I publish
Ab,oiJt •Wo'nderhouzz

~~sbck
kubernetes

CLI - hoit ,
. ,_,
Other services
Pangolia~
typhoon.
Tsdb.
- - - • . . t i Wonder ,:•. Jenkins
Houzz . scripts 1 Emails

l,wvice , I
aws
-~..,
Jenldna
'

..

'I
ID Home - Houzz ~, Git 0 test pipelines M lnbox (348) - yash ... 0 Groovy pipelines .{3 Dashboarc

Standardize CICD Flow··

Project Description
To increase the productivity of development, test and release, we are providing a standardized continuous
integration flow at Ho~z for each service. That means, each product must follow the same development
flow from compile, stylecheck, build, test, and release. We aim to provide a centralized solution to
support different products with different languages, a solution that can be applied to different type of
project, whether it is a library, executables, or deployable services.

Overview-
WhatisCICD
Continuous Integration and Continuous Deployment. Any commit(s) to a repo will go through a series of
checks before publishing. any code passed_ the CI steps will automatically _be deployed to staging
environment for further testing

A general CI flow is described as the belpw figure.

Clean publish
Setup
StyleCheck
Build test result
Post Commit trigger 1---~ analyzer ,____.i

arser
Unit Test
Integration
notification

For each service, each commit or batch of commits to its repo triggers the standardized flow to verify the
changes in the commits not introducing issues, bugs, hence to generate a package for next stage -
deployment.
 project or submodule that want to adapt the CICD flow only need to ptovide a config file with the
following entry. (ref: How to enable CICD)

Example: https://github. cotn/Houzz/c2svc/blob/master/src/iava/mpacampai ./ci-config.j son

• h /blob/ma

{
"nam.e": "product-name",
''group": ''jukwaa@houzz.com or slack channel",
"author"~ [''kane", ·,
"Roberto"
"Karthik"
'
J
"description": "",
"main": "index.js",
"dockerURL": ''url to docker address",
''CI": { • • .
"clean": "gulp clean", •
"setup": "npm install --ignore-scripts"',·
"build": '"', .
"stylecheck ": "eslint ./src --quiet --cache --ext=.js,.jsx,.json",
"unit-test'': "jest --coverage -json --outputFile=.jest-test-results.json",
"integration--test0 : "jest --coverage --j son --o~utFile=.jest-test-results.json",
h
"Publish": [{
"auto_on": true,
"target:": "stghouzz",
"repository":, u ECR", .
"script": "$GI1 _ROOT/build/src/staging/mpa/build-c~.sh"
J

},
{
"auto_on": false,,
"target:'': "houµ2'', 1/
"repository'': "!ECR",
"script": "$GIT_~OO'l'~uil~src/staging/mpa/build-ci.shtt
}],
"deploy": [ {
''auto_on": true,
"target": "stghouzz".
"type": "k8s". .
"scriptu: ''push_config.sh" . . . .. . 11
1gns. yaml
"config_file": "$GIT_ ROOT /build/src/stagtng/mpa/rol>acampa1gns/mpa-campa
},
{
"auto"'Ton"; false,
"target": '"hou,.z2",
'I "type"; ''kijs", . "
"confjg_fileH :"SGlT_ROOT/build/src/houzz2/mpa/mpacampaigns/mpa-campaigns. yam I
L
 System Archit ecture

., / CLI- hoz
I Other
services:
Pangolia,
-
Wonder Jenkins typhoon,
Houzz scri ts_ . Tsdb,
Emails

-
=
/se~ I
Q··
slack ----- -
Jenkins

Hoz tool
We created HOZ tool which picks up ci-config.json and execute all necessary steps. This is the essential
tool for out CICD flow.
It has 2 prerequisites:

Had hoz. pex installed local.Iy

Specify ci-config.json file within project
 ...
,, clean,llnt,build
Hozcommand " '

D
... Unit.int lest
ci-config .json

l analyze !1--- -..),,, publish/deploy

-~

L-,_--_ _ :.....=...__r_e_po_rt_(_s,_a_ck_or_e_m_a_i_l)_ _ ___,/

Refer ences
docum entati on

Code Location
Pipeline flows, services, and command are in the repo:
https://github.com/HouzzJwonderhouzz.git

Projects under HOZ

I. Jukwaa products
Jukwaa
Jukwaa-core
Jukwaa-shared:-components
Jukwaa-ui

A snapshot:
 w ...
"C\ Hoz-Pipellne Ntf) 1:57 PM
'V~ Cl
Hoz check passed in jukwaa Release branch after commit Oe799226
Jenkins Job link
Cl Hoz check passed in jukwaa master branch after commit 200889e6
Jenkins Job link
Hoz-Plpellne APP 2:03 PM
! @xutian Hoz check failed in jukwaa mast~r branch after commit 7053~ 9a3
int-test:
link:https://jenkins.stghouzz.com/job/PostCommit_Jukwaa_master/559 /artifact/master/jukwaa/./ci_
result/int-test.log/*view• /
errors:
{ Error. SyntaxError:
/home/jenkins/workspace/PostCommit_Jukwaa_master/master/jukwaa/apps/notifications/pages/n
· otifications/proSolutionsNotifications.ctx.js: Unterminated string constant (178:3)
at process.fallback
(/home/jenkins/workspace/PostCommit_Jukwaa_master/master/jukwaa/node_modules/async-
listener/index.js:565:15) 'error@context': 0} 'Error: SyntaxError:
/home/jenkins/workspace/PostCommit_Jukwaa_master/master/jukwaa/apps/notifications/pages/n
otiftcations/proSolutionsNotiftcations.ctx.js: Unterminated string constant (178:3)\n at _loop

2. C2SVC modules:
mpacampaigns
mpaserver
Insight
searchmixer

These modules under c2svc mostly been automatically deployed to an k8s instance at staging for
each commits, but of course project owners can always control what their projects to do by
editing the ci-config.json entry.

* :.1
I eo • t. ---..... • -• • I J _,, .......... , ___1 1 0 , . , - .,_::::,• • - - ~ - " - - - - .... , ....... - ·• • - · . , _ _ -- -· O I O ..... : , , --- ll~o,

Hoz-Pipellne .~PP 3:01 PM . & '. @ 1 ¢ I

fJ Hoz check passed in c2svc master branch src/java/mpacampai.gns after commit Sb0d2a4~
Jenkins Job link -

l'I src/java/mpacampaigns published to ECR sucessfully for c2svc master branch

I

fJ src/java/mpacampaigns deployed on stghouzz successfully for cZsvc master branch

Hoz-Pipellne APP 3:12 PM

rJ Hoz check passed in c2sv<; master branch src/java/mpaserver after commit c388a0d4
Jenkins Job link
~..,;;;.;.~~L,.:---_-_-....- A . - r , - . - . - . - . - - - - - - _ _ _ : _ - - ~ - - , - - . , . . _ ~ : _ _ - - - - - - - - - - - - - new messages