An Introduction to Puppet, installation and basic features

Before we begin to delve into introduction and uses of puppet, it is important to understand the
category of tools of which puppet is a part of. Puppet unlike the conventional scripting languages
belongs to a special category of tools come into play not very long back, known as DevOps. Now the
question arises what is DevOps, what is the need of such tools and also what are the other tools that
falls into this category.

Let’s begin this interesting journey into the world of puppet, which reduces the time of configuring
multiple machines, across different platforms from weeks or maybe months into days or maybe
minutes. But first of all let’s just try and understand what exactly DevOps is.

1. Introduction to DevOps
DevOps as the name suggests comes from the words Development and Operations, two building blocks
of any organization. Now the next question arises is why DevOps, why some new tools when we already
have a plethora of scripting languages like python, TCL or shell and for configuration management
activities we have tools like Arch, clearcase etc. In order to understand the requirement of DevOps and
answer the above question we need to understand some basic points. Generally developers and
operations team are unaware of the tools used by each other, the main concern of development team is
concerned about making the required changes into their piece of code to keep the software up-to-date,
operations team is more concerned about stability and reliability which means maintain the consistency
across the systems and usually these teams are unaware of the tools used by the other.

DevOps provides the solution by enabling IT alignment by aligning development and operations role and
process in the context of shared business objective. Thus it can be said that the important theme of
DevOps is that the entire development-to-operations lifecycle must be viewed as one end-to-end
process. DevOps helps in standardizing the development environment and it also helps in making
automation frameworks more flexible and reduces the human intervention to the the minimum level.
The usage of DevOps has considerably reduced the time to configure multiple servers across different
platforms, it also reduces the risk of errors which may occur due to inconsistency of the systems
deployed.

Some of the very famous DevOps which have been used widely are puppet, ansible, salt, chef etc. In the
course of this document we will focus more on puppet but would give a brief overview in the end of
other available DevOp tools.

2. Introduction to Puppet
Puppet was first introduced in 2005 by puppetlabs as opensource under general public license. Puppet
enterprise is the commercial version which deals with the IT automation challenges; also it integrates
with several other opensource software products to create a full, comprehensive platform for IT
automation.
Puppet can broadly be defined as the configuration management tool, which allows the user to define
the correct state of the servers. These machines can be physical as well as virtual machines. Puppet is
used to automate every step of software delivery process which includes provisioning of physical or
virtual machines to orchestration as well as report generation. It can also be used for code development,
testing, production releases and updating as well as maintenance of resources.

Puppet is written in ruby on rails with an extensive library of functions in the background while uses a
Domain Specific Language(DSL), which makes it platform independent in true sense of word as the same
code can be executed across different operating systems such as Linux, Unix, Debian, Windows and even
MAC OS to name a few.

3. Puppet Modes
Puppet can operate both in standalone as well as in client-server mode. In standalone mode there is
only one server mode which acts as both puppet master and client.

While in client server architecture there are multiple client machines which are connected to one central
server known as puppet master. All the manifests are stored on puppet master which sends them to
client machine at regular interval (say 30 min).

All the communication between client and server takes place through XMLRPC requests which works
over HTTPS, i.e. over secure connection. All the communication between master and client take place
through CURL requests. However before transfer of manifests between client and server there should be
an SSL transfer between master and all the client nodes, wherein all the nodes receive a secure SSL key
from the master and all the future communication between master and client takes place through that
secure SSL key only.
 4. Puppet Workflow

Puppet master requests the facts from the client machine before sending the manifests, facts include
the details about the client machines such as their operating systems, Ethernet ports, memory etc.
There is a tool called facter which is being used to gather information about the client machine. This gets
installed along with the puppet itself.

After gathering the facts about the client machine, puppet master prepares a catalog according to the
specifications received which contains the information that how a node should be configured.

After sending the catalog, the node repots back to the puppet master about whether the configuration
has been done successfully or not, if not it returns the error. The status of node is visible in dashboard as
well as in the database.

As a plugin, a report generation tool can also be configured along with the puppet; this plugin if
configured can be used to send the report back to the third party software.

5. Installation and Configuration of Puppet

In client-server mode, it is required to install and configure the puppet on both client and server nodes
separately. It is necessary that the master and client nodes should be accessible to each other via their
hostnames. In /etc/hosts of master puppet node, the entry of all the client nodes should be present
while on the client nodes, in /etc/hosts the entry of IP of master node along with the hostname should
be there. After changing /etc/hosts of master and nodes ping them via their hostnames and they should
be accessible.

Master and client should be accessible to each other via hostnames because when manifests are
transferred to the client nodes they are transferred via their hostnames. In the manifest the codes for
each node can be separated using their hostnames.

5.1 Installation on master node:

1. Install the puppet master on the master machine:

apt-get install puppet puppetmaster or yum install puppet puppetmaster

/etc/init.d/puppetmaster start

2. In the file /etc/puppet/puppet.conf

[master]

dns_alt_names = puppet

certname = puppet

5.2 Installation on client nodes:

1. Install the puppet on the client machines:

apt-get install puppet or yum install puppet

/etc/init.d/puppet start

2. In the file /etc/puppet/puppet.conf

[agent]

server = puppet

After installing the puppet on master and clients, now generate the SSL certificates for all the client
nodes from the master. All the future communication between master and client takes place using this
secure SSL key. To generate the new SSL key, remove the SSL directory already present.

rm –rf /var/lib/puppet/ssl

After removing the ssl directory, send the request from the clients to the master

puppet agent –server <hostname of puppet master> --waitforcert <configurable time before timeout>

On master
To view the list of certificate present on the master for signing, execute the following command:

puppet cert –list

To sign the certificate for the requests present on the master, execute the following command:

puppet cert –sign <name of node for which request is present>

6. Resources
Puppet can be used to manage a variety of resources ranging from packages, users, commands etc, it
can install packages, create users, copy files, execute commands for system level configurations.

6.1 Packages
In order to manage packages it is only required to specify the name of package and the action which is
to be done on that particular package such as installation, uninstall and upgradation of package.

package {“ntp”:

Ensure => “present”

6.2 File
Puppet can also be used to manage the file resource on the client nodes either by copying the files
directly to the destination location using the puppet feature of filebucket which is used to copy the files
from a particular location to the specified location on the client machine or the file can be manipulated
using the template feature of puppet in which a template of the file is created and the values are
replaced by the variable names. These variable names can be specified at the beginning of any script and
while copying the files these variables are replaced by the values specified in the script. We can also
manipulate the file permissions, user and group of any file. Using this feature we can also copy scripts at
the client machines.

file { '/tmp/hello':

content => "Hello, world\n",

This example will create a file called hello with the contents “Hello, world” at the client node.

file { '/usr/local/bin/papply':

source => 'puppet:///modules/puppet/papply.sh',

mode => '0755',

}

This example will copy the file from the path specified in the source field at the path specified after the
file keyword on the destination machine. We can copy the source file as it is or we can keep a common
source file for multiple destinations while using variables names into it i.e. a template is created of the
source file.

6.3 Users
Puppet can also be used to create users, their home directories and permissions can be assigned to
them. In the similar fashion groups can also be created.

User {‘test_user’:

ensure => ‘present’,

home => ‘/usr/test_usr’

6.4 exec
It is also possible to execute some shell level command or some script using puppet by specifying the
path at which the command is present.

exec {'update-rsyncd.conf':

command => '/bin/cat /etc/rsyncd.d/*.conf > /etc/rsyncd. conf',

refreshonly => true,

This example will execute the “cat” command.

6.5 Services
With puppet we can also manage the state of a service that whether it should be running or stopped.

Service{“ntp”:

Ensure => running

Differences between Different DevOps

Puppet 3.0 Chef 11.4 Ansible 1.3 Salt 0.17
 Modules are written Mainly uses Ruby Ansible uses the standard Modules are written
mainly in Domain for writing modules. YAML format and the either in python or
Specific Language of simple jinja2 templating special DSL called PyDSL.
Puppet. language.

Standard master/node Chef has a Ansible is only needed to Standard master/node

architecture, where a complicated be installed on the master architecture, where a
client is needed to be master/node setup. node. No separate client is needed to be
installed on all the It requires a installation on children installed on all the
nodes. workstation to nodes are required. nodes.
control the master
node.

Directives do not run in Directives run in the Directives run in the exact Directives do not run in
any specific order. If exact order they are order they are written exact order of code. In
any specific ordering is written order to run the
required keywords like directives in some
before, after and particular order,
require can be used dependencies are
needed to be exclusively
defined.

Issues and their debugging steps

Error #1​: Could not retrieve catalog from remote server: Error 400 on SERVER: Could not parse for
environment production: Syntax error at 'include'; expected '}' at /etc/puppet/manifests/site.pp:14 on
node agent

Solution​: init.pp should be properly indented.

Error #2:​ Service puppet already running …skipping reading catalog

Solution:​ puppetd --enable on agent machine.

Error #3:​ Agents are failing with a “hostname was not match with the server certificate” error

Solution:​ sudo puppet master --configprint certname

Delete and regenerate certificate
find $(puppet master --configprint ssldir) -name "$(puppet master --configprint certname).pem" –delete
stop and restart puppet service
Error #4​ puppet agent --server puppet master --waitforcert 60 --test
err: Could not request certificate: The certificate retrieved from the master does not match the agent's
private key.
Certificate fingerprint: B3:91:20:7F:81:3F:D6:DA:18:60:D1:27:45:EE:39:9B

Solution:​ To fix this, remove the certificate from both the master and the agent and then start a puppet
run, which will automatically regenerate a certificate.
On the master:
puppet cert clean agent
On the agent:
rm -f /var/lib/puppet/ssl/certs/agent.pem

Error #5​ puppet agent -t

err: Could not retrieve catalog from remote server: getaddrinfo: Name or service not known
warning: Not using cache on failed catalog
err: Could not retrieve catalog; skipping run
err: Could not send report: getaddrinfo: Name or service not known

Solution:​ To remove this error on agent machine​, ​in /etc/puppet/puppet.conf under [agent] give
Server = <hostname of puppet master which the agent is able to ping>

Error #6:​ err: Could not retrieve catalog from remote server: Error 400 on SERVER: Could not find class
ntp for agent at /etc/puppet/manifests/site.pp:1 on node agent
Solution:​ Every class created inside module should be named as init.pp and no other name is successful.

Error #7:​ err: Could not retrieve catalog from remote server: Error 400 on SERVER: Puppet:
:Parser::AST::Resource failed with error ArgumentError: Invalid resource type pa
file at /etc/puppet/manifests/site.pp:6 on node agent

Solution:​ Enable plugin sync in puppet.conf on both master and agent nodes

Error #8:​ Puppet agent was sending request to sign certificates to master but master was unable to
receive cert request.
Solution:​ From the agent
sudo rm -rf /var/lib/puppet/ssl
From the puppet master
sudo rm -rf /var/lib/puppet/ssl/certificate_requests/*
After this resend the request

Error #9​: err: Could not send report: SSL_connect returned=1 errno=0 state=SSLv3 read server certificate
B: certificate verify failed. This is often because the time is out of sync on the server or client
Solution​: date and time should be in sync on master and client

Error #10:​ err: /Stage[main]//File[/tmp/ruchi​.​txt]: Failed to generate additional resources

using 'eval_generate: Error 403 on SERVER: Forbidden request: compute2(172.18.7
.26) access to /file_metadata/etc/puppet/manifest/files/ruchi.txt [search] authe
nticated at line 100
Solution:​ Mount point should be specified in fileserver.conf as given below
[test]
path /etc/puppet/modules/test
allow *
Mount point can be checked with the command
puppet config print modulepath

Error #11: Could not evaluate: Could not retrieve information from environment production
source(s) puppet:///modules/test/ruchi at /etc/puppet/manifests/site.pp:6
Solution:​ Source path should be properly defined while copying files.

Error #12:​ Error 400 on SERVER: Puppet::Parser::AST::Resource failed with error ArgumentError: Invalid
resource type pafile at /etc/puppet/manifests/site.pp:6 on node compute2
Solution:​ The line before closing brace should not have a comma (,)

Links and Refernces

1. http://dev2ops.org/2010/02/what-is-devops/

2. http://en.wikipedia.org/wiki/DevOps/

3. Puppet 3 Cookbook- John Arundel

4. Puppet 3 Beginner’s Guide –John Arundel