Building Socket Applications

in the Internet page of the Component palette. The Borland components,

TcpClient and TcpServer, were probably developed to replace the
ClientSocket and ServerSocket components available in past versions of
Delphi. However, now that the ClientSocket and ServerSocket components
have been declared obsolete (although they are still available), Borland
suggests using the corresponding Indy components instead.

In this chapter I'll focus on using Indy during my discussion of low-level

socket programming, not only when I cover support for high-level Internet
protocols. To learn more about the Indy project, refer to the sidebar "Internet
Direct (Indy) Open Source Components"; keep reading to see how you can use
these components for low-level socket programming.

Before I present an example of a low-level socket-based communication, let's

take a tour of the core concepts of TCP/IP so you understand the foundations
of the most widespread networking technology.

Internet Direct (Indy) Open Source Components

Delphi ships with a collection of open-source Internet components called
Internet Direct (Indy). The Indy components, previously called WinShoes (a
pun on the term WinSock, the name of the Windows socket library), are built
by a group of developers led by Chad Hower and are also available in Kylix.
You can find more information and the most recent versions of the
components at www.nevrona.com/indy.

Delphi 7 ships with Indy 9, but you should check the website for updated
versions. The components are free and are complemented by many examples
and a reasonable help file. Indy 9 includes many more components than the
previous version (Indy 8, available in Delphi 6), and it has two new pages on
the component palette (Indy Intercepts and Indy I/O Handlers).

With more than 100 components installed on Delphi's palette, Indy has an
enormous number of features, ranging from the development of client and
server TCP/IP applications for various protocols to encoding and security. You
can recognize Indy components from the Id prefix. Rather than list the various
components here, I'll touch on a few of them throughout this chapter.

Blocking and Non-Blocking Connections

When you're working with sockets in Windows, reading data from a socket or
writing to it can happen asynchronously, so that it does not block the
execution of other code in your network application. This is called a non-
blocking connection. The Windows socket support sends a message when
data is available. An alternative approach is the use of blocking
connections, where your application waits for the reading or writing to be
completed before executing the next line of code. When you're using a
blocking connection, you must use a thread on the server, and you'll generally
also use a thread on the client.
The Indy components use blocking connections exclusively. So, any client
socket operation that might be lengthy should be performed within a thread or
by using Indy's IdAntiFreeze component as a simpler but limited alternative.
Using blocking connections to implement a protocol has the advantage of
simplifying the program logic, because you don't have to use the state-
machine approach of non-blocking connections.

All the Indy servers use a multithreaded architecture that you can control
with the IdThreadMgrDefault and IdThreadMgrPool components. The first is
used by default; the second supports thread pooling and should account for
faster connections.

Foundations of Socket Programming

To understand the behavior of the socket components, you need to be
confident with several terms related to the Internet in general and with
sockets in particular. The heart of the Internet is the Transmission Control
Protocol/Internet Protocol (TCP/IP), a combination of two separate protocols
that work together to provide connections over the Internet (and that can also
provide connection over a private intranet). In brief, IP is responsible for
defining and routing the datagrams (Internet transmission units) and
specifying the addressing scheme. TCP is responsible for higher-level
transport services.

Configuring a Local Network: IP Addresses

If you have a local network available, you'll be able to test the following
programs on it; otherwise, you can use the same computer as both client and
server. In this case, as I've done in the examples, use the address 127.0.0.1
(or localhost), which is invariably the address of the current computer. If your
network is complex, ask your network administrator to set up proper IP
addresses for you. If you want to set up a simple network with a couple of
spare computers, you can set up the IP address yourself; it's a 32-bit number
usually represented with each of its four components (called octets)
separated by dots. These numbers have a complex logic underlying them, and
the first octet indicates the class of the address.

Specific IP addresses are reserved for unregistered internal networks.

Internet routers ignore these address ranges, so you can freely do your tests
without interfering with an actual network. For example, the "free" IP address
range 192.168.0.0 through 192.168.0.255 can be used for experiments on a
network of fewer than 255 machines.

Local Domain Names

How does the IP address map to a name? On the Internet, the client program
looks up the values on a domain name server. But it is also possible to have a
local hosts file, which is a text file you can easily edit to provide local
mappings. Look at the HOSTS.SAM file (installed in a subdirectory of
the Windows directory, depending on the version of Windows you have) to
see a sample; you can eventually rename the file HOSTS, without the
extension, to activate local host mapping.

You may wonder whether to use an IP or a hostname in your programs.

Hostnames are easier to remember and won't require a change if the IP
address changes (for whatever reason). On the other hand, IP addresses don't
require any resolution, whereas hostnames must be resolved (a time-
consuming operation if the lookup takes place on the web).

TCP Ports
Each TCP connection takes place though a port, which is represented by a
16-bit number. The IP address and the TCP port together specify an Internet
connection, or a socket. Different processes running on the same machine
cannot use the same socket (the same port).

Some TCP ports have a standard usage for specific high-level protocols and
services. In other words, you should use those port numbers when
implementing those services and stay away from them in any other case.
Here is a short list:

Protocol Port
HTTP (Hypertext Transfer Protocol) 80
FTP (File Transfer Protocol) 21
SMTP (Simple Mail Transfer Protocol) 25
POP3 (Post Office Protocol, version 3) 110
Telnet 23
The Services file (another text file similar to the Hosts file) lists the
standard ports used by services. You can add your own entry to the list,
giving your service a name of your own choosing. Client sockets always
specify the port number or the service name of the server socket to which
they want to connect.

High-Level Protocols
I've used the term protocol many times now. A protocol is a set of rules the
client and server agree on to determine the communication flow. The low-
level Internet protocols, such as TCP/IP, are usually implemented by an
operating system. But the term protocol is also used for high-level Internet
standard protocols (such as HTTP, FTP, or SMTP). These protocols are
defined in standard documents available on the Internet Engineering Task
Force website (www.ietf.org).

If you want to implement a custom communication, you can define your own
(possibly simple) protocol, a set of rules determining which request the client
can send to the server and how the server can respond to the various possible
requests. You'll see an example of a custom protocol later. Transfer protocols
are at a higher level than transmission protocols, because they abstract from
the transport mechanism provided by TCP/IP. This makes the protocols
independent not only from the operating system and the hardware but also
from the physical network.

Socket Connections
To begin communication through a socket, the server program starts running
first; but it simply waits for a request from a client. The client program
requests a connection indicating the server it wishes to connect to. When the
client sends the request, the server can accept the connection, starting a
specific server-side socket, which connects to the client-side socket.

To support this model, there are three types of socket connections:

 Client connections are initiated by the client and connect a local client
socket with a remote server socket. Client sockets must describe the
server they want to connect to, by providing its hostname (or its IP
address) and its port.
 Listening connections are passive server sockets waiting for a client.
Once a client makes a new request, the server spawns a new socket
devoted to that specific connection and then gets back to listening.
Listening server sockets must indicate the port that represents the
service they provide. (The client will connect through that port.)
 Server connections are activated by servers; they accept a request
from a client.

These different types of connections are important only for establishing the
link from the client to the server. Once the link is established, both sides are
free to make requests and to send data to the other side.

Using Indy's TCP Components

To let two programs communicate over a socket (either on a local area
network or over the Internet), you can use the IdTCPClient and IdTCPServer
components. Place one of them on a program form and the other on another
form in a different program; then, make them use the same port, and let the
client program refer to the host of the server program, and you'll be able to
open a connection between the two applications. For example, in the
IndySock1 project group, I've used the two components with these settings:

// server program
object IdTCPServer1: TIdTCPServer
DefaultPort = 1050
end
// client program
object IdTCPClient1: TIdTCPClient
Host = 'localhost'
Port = 1050
end

Note The Indy server sockets allow binding to multiple IP addresses and/or ports, using
the Bindings collection.
As this point, in the client program you can connect to the server by
executing

IdTCPClient1.Connect;

The server program has a list box used to log information. When a client
connects or disconnects, the program lists the IP of that client along with the
operation, as in the following OnConnect event handler:

procedure TFormServer.IdTCPServer1Connect(AThread: TIdPeerThread);

begin
lbLog.Items.Add ('Connected from: ' +
AThread.Connection.Socket.Binding.PeerIP);
end;

Now that you have set up a connection, you need to make the two programs
communicate. Both the client and server sockets have read and write
methods you can use to send data, but writing a multithreaded server that
can receive many different commands (usually based on strings) and operate
differently on each of them is far from trivial.

However, Indy simplifies the development of a server by means of its

command architecture. In a server, you can define a number of commands,
which are stored in the CommandHandlers collection of the IdTCPServer. In
the IndySock1 example the server has three handlers, all implemented
differently to show you some of the possible alternatives.

The first server command, called test, is the simplest one, because it is fully
defined in its properties. I've set the command string, a numeric code, and a
string result in the ReplyNormal property of the command handler:

object IdTCPServer1: TIdTCPServer

CommandHandlers = <
item
Command = 'test'
Name = 'TIdCommandHandler0'
ParseParams = False
ReplyNormal.NumericCode = 100
ReplyNormal.Text.Strings = (
'Hello from your Indy Server')
 ReplyNormal.TextCode = '100'
end

The client code used to execute the command and show its response is as
follows:

procedure TFormClient.btnTestClick(Sender: TObject);

begin
IdTCPClient1.SendCmd ('test');
ShowMessage (IdTCPClient1.LastCmdResult.TextCode + ' : ' +
IdTCPClient1.LastCmdResult.Text.Text);
end;

For more complex cases, you should execute code on the server and read and
write directly over the socket connection. This approach is shown in the
second command of the trivial protocol I've come up with for this example.
The server's second command is called execute; and it has no special
property set (only the command name), but has the
following OnCommand event handler:

procedure TFormServer.IdTCPServer1TIdCommandHandler1Command(
ASender: TIdCommand);
begin
ASender.Thread.Connection.Writeln ('This is a dynamic response');
end;

The corresponding client code writes the command name to the socket
connection and then reads a single-line response, using different methods
than the first one:

procedure TFormClient.btnExecuteClick(Sender: TObject);

begin
IdTCPClient1.WriteLn('execute');
ShowMessage (IdTCPClient1.ReadLn);
end;

The effect is similar to the previous example, but because it uses a lower-
level approach, it should be easier to customize it to your needs. One such
extension is provided by the third and last command in the example, which
allows the client program to request a bitmap file from the server (in a sort of
file-sharing architecture). The server command has parameters (the filename)
and is defined as follows:
object IdTCPServer1: TIdTCPServer
CommandHandlers = <
item
CmdDelimiter = ' '
Command = 'getfile'
Name = 'TIdCommandHandler2'
OnCommand = IdTCPServer1TIdCommandHandler2Command
ParamDelimiter = ' '
ReplyExceptionCode = 0
ReplyNormal.NumericCode = 0
Tag = 0
end>

The code uses the first parameter as filename and returns it in a stream. In
case of error, it raises an exception, which will be intercepted by the server
component, which in turn will terminate the connection (not a very realistic
solution, but a safe approach and a simple one to implement):

procedure TFormServer.IdTCPServer1TIdCommandHandler2Command(
ASender: TIdCommand);
var
filename: string;
fstream: TFileStream;
begin
if Assigned (ASender.Params) then
filename := HttpDecode (ASender.Params [0]);
if not FileExists (filename) then
begin
ASender.Response.Text := 'File not found';
lbLog.Items.Add ('File not found: ' + filename);
raise EIdTCPServerError.Create ('File not found: ' + filename);
end
else
begin
fstream := TFileStream.Create (filename, fmOpenRead);
try
ASender.Thread.Connection.WriteStream(fstream, True, True);
 lbLog.Items.Add ('File returned: ' + filename +
' (' + IntToStr (fStream.Size) + ')');
finally
fstream.Free;
end;
end;
end;

The call to the HttpDecode utility function on the parameter is required to

encode a pathname that includes spaces as a single parameter, at the
reverse the client program calls HttpEncode. As you can see, the server also
logs the files returned and their sizes, or an error message. The client
program reads the stream and copies it into an Image component, to show it
directly (see Figure 19.1):

procedure TFormClient.btnGetFileClick(Sender: TObject);

var
stream: TStream;
begin
IdTCPClient1.WriteLn('getfile ' + HttpEncode (edFileName.Text));
stream := TMemoryStream.Create;
try
IdTCPClient1.ReadStream(stream);
stream.Position := 0;
Image1.Picture.Bitmap.LoadFromStream (stream);
finally
stream.Free;
end;
end;
 Figure 19.1: The client program of the
IndySock1 example

Sending Database Data over a Socket Connection

Using the techniques you've seen so far, you can write an application that
moves database records over a socket. The idea is to write a front end for
data input and a back end for data storage. The client application will have a
simple data-entry form and use a database table with string fields for
Company, Address, State, Country, Email, and Contact, and a floating-point
field for the company ID (called CompID).

Note Moving database records over a socket is exactly what you can do with DataSnap and a
socket connection component (as covered in Chapter 16, "Multitier DataSnap
Applications") or with SOAP support (discussed in Chapter 23, "Web Services and
SOAP").
The client program I've come up with works on a ClientDataSet with this
structure saved in the current directory. (You can see the related code in
the OnCreate event handler.) The core method on the client side is the
handler of the Send All button's OnClick event, which sends all the new
records to the server. A new record is determined by looking to see whether
the record has a valid value for the CompID field. This field is not set up by
the user but is determined by the server application when the data is sent.

For all the new records, the client program packages the field information in a
string list, using the structure FieldName=FieldValue. The string
corresponding to the entire list, which is a record, is then sent to the server.
At this point, the program waits for the server to send back the company ID,
which is then saved in the current record. All this code takes place within a
thread, to avoid blocking the user interface during the lengthy operation. By
clicking the Send button, a user starts a new thread:

procedure TForm1.btnSendClick(Sender: TObject);

var
SendThread: TSendThread;
begin
 SendThread := TSendThread.Create(cds);
SendThread.OnLog := OnLog;
SendThread.ServerAddress := EditServer.Text;
SendThread.Resume;
end;

The thread has a few parameters: the dataset passed in the constructor, the
address of the server saved in the ServerAddress property, and a logging
event to write to the main form (within a safe Synchronize call). The thread
code creates and opens a connection and keeps sending records until it's
finished:

procedure TSendThread.Execute;
var
I: Integer;
Data: TStringList;
Buf: String;
begin
try
Data := TStringList.Create;
fIdTcpClient := TIdTcpClient.Create (nil);
try
fIdTcpClient.Host := ServerAddress;
fIdTcpClient.Port := 1051;
fIdTcpClient.Connect;
fDataSet.First;
while not fDataSet.Eof do
begin
// if the record is still not logged
if fDataSet.FieldByName('CompID').IsNull or
(fDataSet.FieldByName('CompID').AsInteger = 0) then
begin
FLogMsg := 'Sending ' + fDataSet.FieldByName('Company').AsString;
Synchronize(DoLog);
Data.Clear;
// create strings with structure "FieldName=Value"
for I := 0 to fDataSet.FieldCount - 1 do
Data.Values [fDataSet.Fields[I].FieldName] :=
 fDataSet.Fields [I].AsString;
// send the record
fIdTcpClient.Writeln ('senddata');
fIdTcpClient.WriteStrings (Data, True);
// wait for reponse
Buf := fIdTcpClient.ReadLn;
fDataSet.Edit;
fDataSet.FieldByName('CompID').AsString := Buf;
fDataSet.Post;
FLogMsg := fDataSet.FieldByName('Company').AsString +
' logged as ' + fDataSet.FieldByName('CompID').AsString;
Synchronize(DoLog);
end;
fDataSet.Next;
end;
finally
fIdTcpClient.Disconnect;
fIdTcpClient.Free;
Data.Free;
end;
except
// trap exceptions in case of dataset errors
// (concurrent editing and so on)
end;
end;

Now let's look at the server. This program has a database table, again stored
in the local directory, with two more fields than the client application's table:
LoggedBy, a string field; and LoggedOn, a data field. The values of the two
extra fields are determined automatically by the server as it receives data,
along with the value of the CompID field. All these operations are done in the
handler of the senddata command:

procedure TForm1.IdTCPServer1TIdCommandHandler0Command(
ASender: TIdCommand);
var
Data: TStrings;
I: Integer;
begin
Data := TStringList.Create;
try
ASender.Thread.Connection.ReadStrings(Data);
cds.Insert;
// set the fields using the strings
for I := 0 to cds.FieldCount - 1 do
cds.Fields [I].AsString :=
Data.Values [cds.Fields[I].FieldName];
// complete with ID, sender, and date
Inc(ID);
cdsCompID.AsInteger := ID;
cdsLoggedBy.AsString := ASender.Thread.Connection.Socket.Binding.PeerIP;
cdsLoggedOn.AsDateTime := Date;
cds.Post;
// return the ID
ASender.Thread.Connection.WriteLn(cdsCompID.AsString);
finally
Data.Free;
end;
end;

Except for the fact that some data might be lost, there is no problem when
fields have a different order and if they do not match, because the data is
stored in the FieldName=FieldValue structure. After receiving all the data
and posting it to the local table, the server sends back the company ID to the
client. When receiving feedback, the client program saves the company ID,
which marks the record as sent. If the user modifies the record, there is no
way to send an update to the server. To accomplish this, you might add a
modified field to the client database table and make the server check to see if
it is receiving a new field or a modified field. With a modified field, the server
should not add a new record but update the existing one.

As shown in Figure 19.2, the server program has two pages: one with a log
and the other with a DBGrid showing the current data in the server database
table. The client program is a form-based data entry, with extra buttons to
send the data and delete records already sent (and for which an ID was
received back).
Figure 19.2: The client and server programs of the data-base socket example
(IndyDbSock)