Install BuildBot with Perforce on Windows

Patrick Tsai http://baby.homeip.net/patrick/

BuildBot
• The BuildBot is a system to automate the compile/test cycle required by most software projects to validate code changes. • buildbot-0.7.5 (http://buildbot.sourceforge.net)

Two Options
• Win32
– Python win32 (http://www.python.org/) – Pywin32 (http://www.python.org/) – Twisted 2.5 Win32 Installer for Python2.5 (http://twistedmatrix.com)

• Cygwin
– Cygwin Installation (http://www.cygwin.com) – Python 2.5 – Twisted 2.5 tarball

Win32 Installation
• Run the installers of Python, pywin32, Twisted in order. • Add c:\python25;c:\python25\scripts to %PATH. • buildbot-0.7.5> python setup.py install

Cygwin Installation
• Install python and gcc in Cygwin installer. • Install Twisted
– Twisted-2.5.0/zope.interface-3.3.0$ python setup.py install – Twisted-2.5.0$ python setup.py install

• buildbot-0.7.5$ python setup.py install

BuildBot Setup
• Create one master and one slave (check BuildBot README for details):
– buildbot create-master <basedir> – buildbot create-slave <basedir> <master> <name> <passwd>

• Edit master.cfg

Master.cfg #1: Version Control System
• I am using Perforce (http://www.perforce.com) as version control system.
– Create a P4 user for buildbot. – Add the P4 common settings to the beginning of master.cfg:
p4env = { 'port': 'localhost:1666', 'user': 'buildbot', 'passwd': ‘your password here', }

Master.cfg #2: P4 Source
• Remember to specify p4bin otherwise the master will fail to start in Native Win32 installation. • You don’t need p4bin for Cygwin installation. Just ensure p4.exe is in the PATH.
from buildbot.changes.p4poller import P4Source c['sources'].append( P4Source( p4base='//depot/', p4port = p4env['port'], p4user = p4env['user], p4passwd = p4env['passwd'], pollinterval = 60, p4bin = 'd:/p4d/p4.exe', split_file = lambda branchfile: branchfile.split('/', 1)))

Master.cfg #3: Schedulers
• I’ve three folders: main, development, and release under Perforce depot. • Therefore, I created three schedulers for them.
from buildbot.scheduler import Scheduler c['schedulers'] = [] c['schedulers'].append( Scheduler( name = "main", branch = 'main', treeStableTimer = 2*60, builderNames=["main"])) c['schedulers'].append( Scheduler( name = "development", branch = 'development', treeStableTimer = 2*60, builderNames=["development"])) c['schedulers'].append( Scheduler( name = "release", branch = 'release', treeStableTimer = 2*60, builderNames=["release"]))

Master.cfg #4: Builders
• Create three builders for each schedulers. • Create one P4 client for each builder.
f1 = factory.BuildFactory() f1.addStep( P4, p4port = p4env['port'], p4user = p4env['user'], p4passwd = p4env['passwd'], p4client = 'buildbot-main', p4base = '//depot/', mode = "copy") b1 = { 'name': "main", 'slavename': "bot-blackbox", 'builddir': "main", 'factory': f1 } … c['builders'] = [b1, b2, b3]

Final Step?
• Run build master:
– buildbot start <master_dir>

• Run build slave
– buildbot start <slave_dir>

• If you are as unlucky as me, you may encounter the following problems…

Problem #1
• Win32: “buildbot start …” throws exception • Remedy:

Problem #2
• Cygwin & Win32: the slave cannot run because of some exception related to ‘LOGNAME’. • Remedy: patch buildbot\slave\commands.py
– Native Win32: replace LOGNAME to USERNAME – Cygwin: replace LOGNAME to USER

Problem #3
• Win32: The web page shows garbage characters in UTF-8 encoding • Remedy: patch buildbot\status\html.py

Problem #4
• Cygwin: P4poller cannot detect P4 changes • Remedy: patch twisted\internet\posixbase.py (It works but I am not sure whether it’s correct)

Problem #5
• Cygwin: the slave will hang when executing ‘p4 … client –i’ • Remedy: replace the action with dummy one. However, you need to configure the root for each P4 client manually.

Finally BuildBot Can Run

Conclusion
• Both installations cannot run perfectly with patches. • Cygwin installation is more problematic than Win32 one • BuildBot is good and still worths the effort.