Pipe and Filter

Software Architecture
Agenda
• History

• Pipe and Filter

• Example

• Terminology

• Properties

• Working

• Problems

• Advantage and Disadvantage

• Summary
History
• The popularity of the architecture is mainly due to the Unix operating system. It
has become popular because Ken Thomson (who created Unix, together with
Dennis Ritchie) decided to limit the architecture to a linear pipeline. Using the
architecture at all was an idea of Doug McIlroy, their manager at Bell Labs at the
time (1972).
• Unix programs. The output of one program can be linked to the input of another
program.
Pipe

• The pipe is the connector that passes data from one filter to the next. It is a
directional stream of data, that is usually implemented by a data buffer to store
all data, until the next filter has time to process it.
• Pipes act as unidirectional
• Pipes change none of the data content. In the pure pipe-and-filter style, filters
interact only through pipes.
Filter

• The filter transforms or filters the data it receives via the pipes with which it is
connected. A filter can have any number of input pipes and any number of output
pipes.
• Filters can act asynchronously.
• Filters interact only through pipes.
Pipe and Filter Architecture

• Pipe and Filter is a simple architectural style, which connects a number of

components that process a stream of data, each connected to the next
component in the processing pipeline via a Pipe.
• A pipe and consists of a chain of processing elements arranged so that the
output of each element is the input of the next.
• Works on “FIFO”.
Continue..

• The information that flows in these pipelines is often a stream of records, bytes
or bits, and the elements of a pipeline.
• Use the Pipes and Filters architectural style to divide a larger processing task into
a sequence of smaller, independent processing steps (filters) that are connected
by channels (pipes).”
• Pipe-and-filter systems have the nice property that the overall computation can
be treated as the functional composition of the compositions of the filters.
Example

• Compiler (Lexical analysis, parsing, code generation )

• Functional programming (algebraic operations)
• Unix programs. The output of one program can be linked to the input of another
program.
Terminology

• Pump - Component that feeding a pipeline with data .

• Filter - Component that transforming data.
• Sink - Component that collects data and terminates a pipeline.
Pipe Properties

• Connections
• Streams
• Channels
Filter Properties

• Transform
• Component
• Concurrent
• Process
• Independent
How It Works?
• The application links together all inputs and outputs of the filters by pipes, then
spawns separate threads for each filter to run in.
Here's an idea of the relationships that can be created between the different
filter processes, through pipes.
• All filters are processes that run (virtually) at the same time. That means, they can
run as different threads, be located on different machines entirely. Every pipe
connected to a filter has its own role in the function of the filter. So if you
connect a pipe, you also need to specify the role it plays in the filter process. The
filters should be made so robust that pipes can be added and removed at
runtime. Every time the filter performs a step, it reads from its input pipes,
performs its function on this data, and places the result on all output pipes. If
there is insufficient data in the input pipes, the filter simply waits.
When should you use it?

• This architecture is great if you have a lot of transformations to perform and you
need to be very flexible in using them, yet you want them to be robust.
Problems

• If a filter needs to wait until it has received all data (e.g. a sort filter), its data
buffer may overflow, or it may deadlock.
• If the pipes only allow for a single data type (a character or byte) the filters will
need to do some parsing. This complicates things and slows them down. If you
create different pipes for different data types, you cannot link any pipe to any
filter.
Pipe and Filter Style Advantages

• Simplicity – Allows designer to understand overall input/output behavior of a

system in terms of individual filters.
• Maintenance and reuse.
• Concurrent Execution –Each filter can be implemented as a separate task and be
executed in parallel with other filters.
Pipe and Filter Style disadvantages

• Interactive transformations are difficult.

• No filter cooperation.
• Performance – may force a lowest common denominator on data transmission.
Summary of the pipe-and-filter style

Elements • Component types: filter. Filter ports must

be either input or output ports.
• Connector types: pipe. Pipes have data-in
and data-out roles.

Relations Attachment relation associates filter output

ports with data-in roles of a pipe and filter
input ports with data-out roles of pipes and
determine the graph of interacting filters.
Continue..

Topology Pipes connect filter output ports to filter

input ports. Specializations of the style may
restrict the association of components to an
acyclic graph or a linear sequence.

Computational model • Filters are data transformers that read

streams of data from their input ports and
write streams of data to their output
ports.
• Pipes convey streams of data from one
filter to another.