Professional Documents
Culture Documents
Publishing Dynamic
Information
Graham Butcher
Graham Butcher
Graham Butcher
Graham Butcher
Graham Butcher
Graham Butcher
Graham Butcher
Contents
Chapter 1 : Introduction and Objectives....................................................................1
1.1
1.2
1.3
1.4
1.5
Graham Butcher
5.1 Requirements............................................................................................... 43
5.2 Background..................................................................................................43
5.3 Problem analysis........................................................................................... 43
5.4 Functional analysis of web-access to DBs.....................................................47
5.5 Practical advice............................................................................................47
5.6 Heuristics...................................................................................................... 47
5.7 High level development................................................................................47
5.8 Scalability..................................................................................................... 47
5.9 Ease of maintenance....................................................................................47
5.10 Design and Development of Solution..........................................................47
Chapter 6: Details of third example, Centres Documentation Management System
50
6.1
6.2
6.3
6.4
Requirements............................................................................................... 50
Background..................................................................................................50
Problem analysis........................................................................................... 50
Design and Development of Solution............................................................53
General Recommendations...........................................................................58
Recommendations from Solution 1..............................................................58
Recommendations from Solution 2...............................................................58
Recommendations from Solution 3...............................................................59
Chapter 9: Conclusions............................................................................................ 61
9.1
9.2
9.3
9.4
ii
Graham Butcher
Figure 1 - Example of an Application that uses CGI and Perl, Autosave Router
Configurations........................................................................................................... 5
Figure 2 - An example of WODA.................................................................................6
Figure 3 - Web interface for MySQL management and administration.......................8
Figure 4 - An example of ASP 3.0...............................................................................9
Figure 5 - MacroMedia's own description of ColdFusion plus the Price.....................11
Figure 6 - An Example of CFML.................................................................................14
Figure 7 - CFML page............................................................................................... 16
Figure 8 - CFML with error........................................................................................16
Figure 9 - Microsoft's pricing for Visual Studio.Net...................................................18
Figure 10 - Web Matrix, design window....................................................................20
Figure 11 - SQL Server control panel........................................................................20
Figure 12 - Details of CodeCharge Studio's pricing structure ..................................22
Figure 13 - CodeCharge Studio IDE..........................................................................23
Figure 14 - PHP - ASP.Net comparison table from ref [5]..........................................27
Figure 15 - Author's comparison table.....................................................................28
Figure 17 - View of Assets Table...............................................................................30
Figure 18 - Login, obtain password page..................................................................30
Figure 19 - Simple search page of UKIP Inventory....................................................31
Figure 20 - Power search of UKIP Inventory..............................................................32
Figure 21 - Documentation System, database chooser............................................33
Figure 22 - Documentation System, search for "Peakflow dos"................................34
Figure 23 - Show versions of "Peakflow DOS upgrade ...".........................................34
Figure 24 - Requested document retrieved to users web browser...........................35
Figure 25 - Starting point, existing NVT database - MS access.................................37
Figure 26 - Existing database, view of Assets..........................................................37
Figure 27 - Flowchart showing choices made for first solution.................................39
Figure 28 Features provided with Easily.co.uk Web hosting...................................40
iii
Graham Butcher
iv
Graham Butcher
Copyright
British Telecommunications plc, 2004. All rights reserved.
BT maintains that all reasonable care and skill has been used in the compilation of
this publication. However, BT shall not be under any liability for loss or damage
(including consequential loss) whatsoever or howsoever arising as a result of the
use of this publication by the reader, his servants, agents or any third party.
All third-party trademarks are hereby acknowledged.
Document history
Revision
Author
Date
Notes
0.A
Graham
Butcher
21st
October
2003
0.B
Graham
Butcher
31st May
2004
Second Draft
0.C
Graham
Butcher
22nd
August
2004
Third Draft
1.0
Graham
Butcher
2nd
Septemb
er 2004
Issued version
Distribution
Distribution will be controlled by:
Graham Butcher
Orion 3rd Floor, Adastral Park, Ipswich IP5 3 RE
graham.butcher@bt.com
tel: +44(0) 1473 649059
Graham Butcher
Abstract
The skills required to be able to publish dynamic information on the Internet or an
Intranet are as important today (2004) as using a word processor or business
software was in the late 1980s. Those individuals and organisations that had (and
have) these skills command a distinct advantage over their competitors in the
business world.
The purpose of this project is to investigate technology that can publish, this kind of
information, using a common interface such as a Web Browser. The main objective
of the project however, is to provide tools and skills that can benefit the Authors
work area - the Data and IP Integration and Test Centre which is part of BT Exact at
Adastral Park.
Within the Test Centre there are many teams doing a variety of activities these
include providing the test environment, testing itself, managing resources and
dealing and communicating with end customers. Normally information relating to
Inventories, Booking Systems, Configurations, Documentation, Ownership and other
related topics is held in databases. If this information were permanently and
instantly available, everyone would benefit.
The aim of this project is therefore to look at different methods of implementing the
display of non static information held in databases by using dynamic web pages.
Three examples were produced for the project, adding value to the Authors
workplace.
vi
Graham Butcher
Declaration
Signed:
Date:
________________________________
__________________________
vii
Graham Butcher
Co-ordinate infrastructure for Test Teams in the UKIP test area improve
documentation and flow of information across the area.
1 of 88
Graham Butcher
Web site managers who wish to enhance their web pages with more
information ie database based and searchable.
1.5 Stakeholders of the report, Motivation for writing and Business Case
Stakeholders
2 of 88
Graham Butcher
3 of 88
Graham Butcher
Simple Web Server side coding methods. These involve writing and
running programs on a web server that, when called, go off and perform
tasks like connecting to databases, querying them and outputting the results
in HTML to the users browser. In the old days this was done by CGI scripts
but nowadays dedicated scripting languages such as PHP, ASP and CFML are
frequently used. A knowledge of one of the PHP, ASP or CFML languages is
required for this type of solution or some knowledge of how CGI works.
There are of course grey areas and overlaps between these artificial segregations of
categories.
First we will look at simple Server Side Scripting Solutions.
2.2 Web Server Side Coding Methods
2.2.1 CGI Type Solutions
In the early days before dedicated web scripting languages such as ASP and PHP
were developed programmers and developers had to produce their own code,
native to the operating System hosting the web service. They used existing
languages supported on the web server. These specialised pieces of code were
usually placed in a directory called the CGI bin the Common Gateway Interface,
4 of 88
Graham Butcher
A second example of using CGI and Perl to produce Database Web Pages
One of the first examples of a complete Web based database System is WODA
(Web Oriented Database), which is claimed by the Designer, Prof Ziga Turk, to have
originated around 1995. The System is completely written in Perl. Turk also
comments if in 1994 concepts like active server pages and database Systems
5 of 88
Graham Butcher
built right into the HTTPD servers were available, as easily as today, the
evolution of WODA would perhaps be different ref [6]. Turk claims that WODA
is still used internally by NASA, Siemens and Amazon.com.
Dynamic HTML
Proxy services
6 of 88
User authentication
Image maps
Java servlets
Graham Butcher
Its fast. Because it is embedded in HTML code, the response time is short
Its secure the user does not see the PHP code
PHP works by embedding scripting code in normal HTML using extensions to the
HTML tag method. A piece of PHP code will start with the <?PHP tag and finish with
the ?> tag. As a result all HTML statements are sent to the browser as is but PHP
statements are pre-processed and outputted in HTML before sending to the browser.
Eg
<?PHP echo <p>Hello World; ?>
Will be processed to
<p>Hello World
which is HTML Hello World is displayed in the users browser.
The consequence of this is that the end user never sees the PHP code - making it
transparent and secure.
MySQL is a freely available full RDBMS (Relational Database Management System)
similar to Oracle. It is available at www.mysql.com. MySQL is developed, marketed,
and supported by a Swedish company MySQL AB. There are two types of licence:
I. Open Source Software available via the GNU General Public Licence
(GPL). Anyone who meets the GNU requirements can use the product
for free, even if they are making money from it. The disadvantage is
that developments have to be made public.
7 of 88
Graham Butcher
Putting the three products Apache, PHP and MySQL - together makes an
attractive, open, low budget solution for implementing web based database
Systems.
8 of 88
Graham Butcher
ASP will run automatically on all Microsoft based web servers - IIS and PWS but
will not work on Apache or other web servers. This is because there is no MS install
program available. ASP is built into MS web servers. To get round this problem
however, a third party company called Chilisoft produced an open version of ASP,
called Chilisoft ASP. This product is fully compatible with non-Microsoft platforms
and servers.
Chilisoft ASP used to be freely available at www.chilisoft.com When that URL is
now accessed though, you are re-directed to Sun Java System, Active Server Pages
4.0 so it looks as though Sun have bought out the company and distributes it as
part of its web server products. Chilisoft ASP is basically an open version of
Microsofts language designed to run on all platforms. Chilisoft ASP is particularly
useful for porting legacy Systems from a MS/ASP environment into an open one.
Microsofts main Web Server product is called IIS Internet Information Service.
The product comes free with various versions of Windows usually known as
professional eg Windows XP, Professional. With Windows 98 and 2000 there is a
free optional installable product called PWS Personal Web Server. This is a light
weight Web Server, useful for testing out ASP code and Web pages on your PC.
Microsofts two database offerings are Access and SQL Server. The former is
more aimed at the single user customer with the latter aimed at the multi-user
heavy weight end of the market. Many web based database solutions however, run
quite happily using Access as their database.
A simple example of ASP (written in VB script Visual Basic Script) code would be:
<html>
<body>
<center>
<% Dim ftsize, ftcolor%>
<% For ftsize = 1 to 7 %>
<font size = <%=ftsize%>>
Hello and Welcome!<br>
<% Next %>
</center>
</body>
</html>
NB This is actually a mixture of HTML and ASP. The ASP code, embedded in the
HTML and highlighted starts with <% and finishes with %>. In this way it works like
PHP and is not visible to the end user. The result of calling this short bit of code
would be to display the words Hello and Welcome! in 7 different font sizes.
9 of 88
Graham Butcher
By using, for example, IIS, ASP and Access, a quick and simple web based
database System can be built. A knowledge of the ASP language will however be
required. Alternatively, if additional tools are used like Microsofts Front Page
their web page and site generator - then wizards and short cuts can also be used for
simple solutions without any knowledge of ASP.
Also, a dedicated code generation tool such as CodeCharge Studio (see section
2.3.3) can be used which removes the need to learn any languages.
10 of 88
Graham Butcher
ColdFusion server
ColdFusion Administrator
11 of 88
Graham Butcher
Variables and functions that are always surrounded by hash signs (#).
If the ColdFusion server finds any HTML or plain text in the page, the ColdFusion
server returns it to the web server untouched.
The ColdFusion server processes all the ColdFusion instructions found, and returns
any remaining results to the web server. The web server then sends the entire
output to the browser.
The ColdFusion Markup Language
ColdFusion Markup Language (CFML) is a tag-based language similar to HTML that
uses special tags and functions. With CFML you can enhance your standard HTML
files with database commands, conditional operators, and high-level formatting
functions, and rapidly produce easy-to-maintain web applications.
CFML looks similar to HTML: it includes start and end tags, and each tag is enclosed
in angle brackets. All ending tags are preceded with a forward slash (/) and all tag
names are preceded with cf; for example:
<cfstarttagname></cfendtagname>
12 of 88
Graham Butcher
Debugging output
Server settings
Application security
Creating a ColdFusion page
Creating a ColdFusion page involves using tags and functions. The best way to
understand this process is to create a ColdFusion page.
In the following procedure, you will create a simple ColdFusion page by using HTML
tags, one ColdFusion tag, and two ColdFusion functions. The following table briefly
explains the ColdFusion tags and functions:
Element
Description.
Now()
A function supported in CFML that you can use to retrieve information from your
System.
You will use the Now ( ) function in the following procedure to return the current
date that is retrieved from your System.
DateFormat() A function that instructs ColdFusion to format the date returned by the Now()
function.
cfoutput
A ColdFusion tag that you use to return dynamic data (data retrieved from a
database) to a web page.
You will use the cfoutput tag in the following procedure to display the current date
retrieved from your System.
Note: ColdFusion tags and functions are considered primary elements of CFML.
13 of 88
Graham Butcher
</body>
</html>
14 of 88
Graham Butcher
Most often the end tag encloses the tag name in brackets and includes a slash (/),
like this:
</tagname>
The information processed by ColdFusion is placed between the start and end tag,
like this:
<tagname>
info to be processed ...
</tagname>
ColdFusion tags, for the most part, share these common characteristics:
Use of attributes (like html tags), and most attributes have values.
Some ColdFusion tags, such as cfset, omit the closing tag. This type of tag uses one
set of angle brackets and places all the required information between the left (<)
and right (>) angle brackets, like this:
<cfsetname="bob">
Tag attributes
Tag attributes instruct the ColdFusion server about the details of an operation. For
example, to update a database table, the server needs to know specifics about the
database, such as the database name and the table name. The code required to
write this type of statement might look like this:
<cfupdatedatasource="mydb"tablename="mytable">
where datasource and tablename are attributes of the cfupdate tag and "mydb" and
"mytable" are attribute values.
Functions
Typically, a function acts on data. It can generate a value or a set of values, usually
from some input. You can perform the following operations (actions) with functions:
Usually, a function performs an operation on a value, and the value can include the
value of a variable. For example: to format the value of a variable containing a
value in dollars, the code to write this statement might look like this:
#DollarFormat(price)#
15 of 88
Graham Butcher
The DollarFormat function returns a value as a string and formats that value with
two decimal places, thousand separator, and dollar sign. The hash signs (#) around
the function instruct ColdFusion to evaluate the content between the hash signs and
display the value.
Functions and parentheses
All functions have parentheses, regardless of whether the function acts on data.
Consider the following function:
#Now()#
If you put anything inside the parentheses of the Now() function, an error would
occur. The Now() function returns an unformatted date and time. However, you can
format the results of this function with other functions, such as the DateFormat() or
TimeFormat() functions.
Nesting functions
Functions can generate data as well as act on data. Consider the following example:
#DateFormat(Now(),"mm/dd/yyyy")#
In this example, the Now() function generates the date, and then the DateFormat
function formats the date.
Functions and hash signs
You use hash signs (#) with functions to display the results of a function on the
page. Hash signs tell the ColdFusion server to evaluate the content between the
hash signs and display the value, for example:
<cfoutput>
Helloworld,<br>
Today'sdateis#DateFormat(Now(),"mm/dd/yyyy")#
</cfoutput>
The following figure shows the output of this example:
16 of 88
Graham Butcher
If you did not include the hash signs around the DateFormat(Now(),"mm/dd/yyyy")
function, ColdFusion would not evaluate the function and the previous example
would display your source code, as follows:
Variables
Variables let you store data in memory on the server - they always have a name and
a value. You can assign a value to a variable, or you can instruct ColdFusion to
assign variable values based on data that it retrieves from a data source, such as a
database table.
Summary
ColdFusion was probably the first robust integrated development environment
introducing features such as server side scripting with CFML (ColdFusion Markup
Language) using embedded declarative tags. It is now losing popularity a little to
cheaper more versatile solutions. CFML requires a ColdFusion Web Server to host
the site, applications and CFML web pages. Yue and Ding ref [2] writing from
Design and Evolution of an Undergraduate Course on Web Application say The
initial inclusion of ColdFusion in 1998 and its exclusion in 2000 may serve of an
example
ColdFusion would suit the developer that works exclusively with Macromedia tools
and has a ColdFusion Web Server to publish on.
17 of 88
Graham Butcher
Visual Basic.Net
Visual C#
Visual FoxPro
Visual C++.Net
Easy ASP.Net programming ie WYSIWYG (what you see is what you get) and
Code Aware Editors
The .Net initiative started to take off around the middle of 2000. It came about
because of the increasingly important role of the Internet, more collaboration
between developers and the gradual shift from desktop computing to distributed
computing.
Its method of communicating between applications, environments and platforms is
XML (eXstensible Mark-up Language).
18 of 88
Graham Butcher
As people have some knowledge of the older MS languages of Visual Basic and
Visual C++ this discussion will just focus on the new elements of the package
ASP.Net and C# languages.
ASP.Net is Microsofts next version of (ASP) Active Server Pages the earlier
version, ASP 3.0 is mentioned in section 2.2.3 of this report. The main difference
between ASP 3.0 and ASP.Net is that the latter is object oriented and compiled. It
can be produced within Visual Studio.Net IDE using any of the languages mentioned
earlier. The advantages of ASP.Net over ASP 3.0 are:
Unlike ASP 3.0, ASP.Net is compiled therefore applications will run faster. Just
In Time (JIT) binding is possible.
State management is handled better. ASP.Net offers three types of states for
Web applications: application, session and user.
Better updating. With 3.0 its necessary to shutdown the Web server to
update the ASP with ASP.Net its not.
Multiple audience targeting. The same page can deliver text only for a
mobile phones or rich-dynamic content for IE 6.0. This is a built-in function
that detects the browser being used.
19 of 88
Graham Butcher
20 of 88
Graham Butcher
Summary
Using .Net technology is an investment for the future as the older Microsoft ASP 3.0
is likely to be phased out and unsupported some time in the future. The visual
Studio.Net environment also has the advantage that it can develop for the Web, for
Windows and for Mobile devices such as PDAs. This means that applications
written for one platform can be ported across relatively easily to another. The
prohibitive cost of the product Visual Studio.Net and lack of free trial period
prevented an in-depth study in this project.
21 of 88
Graham Butcher
This type of solution has created quite a buzz in the industry as companies like Yes
Software are now challenging the big names Microsoft and Macromedia in the
field of Rapid Application Development of dynamic database driven web pages.
At the start of the project this type of solution wasnt considered important but as
the project progressed, this particular tool became more and more important. A
simple internet search revealed CodeCharge Studio as being a method of
developing code for database web applications. The full blown product is available,
free of charge, for one month and consists of a 25 MB download. After 3 months of
trials (on 3 separate computers) the Author was so impressed that the product was
purchased at the full price of $180. Subsequently the price has risen to $499 see
above.
CodeCharges own sales information is quoted as saying CodeCharge Studio is a
visual application builder and code generator that provides a feature-rich
environment for rapid application development for the web. It is the most
productive way available today to create powerful, scalable, and secure web
applications quickly. Based on a sophisticated code generation engine, CodeCharge
Studio opens up new possibilities for web developers by automating the creation of
virtually all web application components and by generating robust, professional-
22 of 88
Graham Butcher
level server code in any of the following programming languages: ASP.NET (C# and
VB), ASP 3.0, PHP 4.0, Java Servlets 2.2, JSP 1.1, ColdFusion 4.01, and PERL 5.0 ref
[4].
How CodeCharge works
CodeCharge Studio is functionally and visually similar to other RAD tools, such as
Microsoft Studio.Net and Macromedia ColdFusion with the ability to maximize
developer productivity while keeping the cost of development to a minimum. While
visual programming tools are traditionally used for developing desktop and client
applications, CodeCharge Studio is ideal for developing sophisticated web
applications.
23 of 88
Graham Butcher
24 of 88
Graham Butcher
Summary
CodeCharge Studio offers an attractive and flexible way of rapidly producing
Dynamic Database Web pages. It would be the solution that the Author would
recommend that people, new to the arena, look at first.
25 of 88
Graham Butcher
Normally before accessing an ODBC database, you must install on your System the
appropriate ODBC driver for that database (if its not installed already).
A Data Source Name (DSN) is what ODBC uses to allow you to associate a database
with a driver. You use the ODBC Administrator (in Control Panel on PCs) to configure
and give the DSN a unique name and then associate it with both a database and a
driver.
When you open a connection to ODBC, you specify what DSN you want to work with.
ODBC takes care of all the little details involved in opening that database.
Before you can access ODBC databases, you must configure the ODBC data source
names, the ODBC drivers, and the configuration values used in ODBC.INI. You also
should understand the structure of an ODBC driver and the ODBC API as well as
some ODBC-related terminology.
In order for any application to access data in a database, it must establish a
connection to that database through its corresponding ODBC driver. To do this, the
application must request a connection from the ODBC Driver Administrator,
specifying the data source desired.
Although ODBC is the oldest method it is by no means the only way of connecting to
databases. Other methods are JDBC, ADO (Active Data Objects) and Jet (MS Access
database engine). A tool like CodeCharge Studio will allow you to choose the type
and sort out the necessary code for connecting to databases automatically.
2.5.3 Whats Oracles contributions to the discussion?
Oracle is the worlds largest database company. Along with Oracle v7 and possibly
v8, there used to be a free Web Server and Wizards that allowed you to develop
dynamic web pages. Although this appears to be supported with the latest version,
Oracles support seems to be withdrawing and they generally seem to be
recommending other proven methods such as PHP and ASP.Net. See Hull, ref [5].
2.5.4 Security
Web and Database security are massive subjects in their own right. The discussion
in this report is limited to a few mistakes that can be made when designing
Database Web pages and sites:
1. Access database (or other single file database) can be downloaded. This
actually happened where a holiday company made available all their
customers credit card and other details stored in a single access database.
The hacker types something like:
http://www.holiday-company.com/customers.mdb
and the browser asks if the file should be opened or saved. Even if the
hacker doesnt know the name of the database file, it can be found by a
trial and error program. The way to protect against this is to obviously,
locate all data files in an area that can be reached by the code but not
browsers ie do not use the root directory for any data.
2. Permissions on all files on the Web Server have to be correct. On some
web sites, typing an incorrect URL gives you a listing of all files held their.
There is then the opportunity to download files that were not meant to be
downloaded. Sometimes database usernames and passwords have been
known to be stored in text files accessed by the code.
3. All Web servers have a default file or default filenames that are activated
even though not specified. The most common is index.html or index.htm.
26 of 88
Graham Butcher
The developer needs to be aware of all the Web Server settings when
designing the code to be used.
2.6 A comparison of different solutions
This is one of the hardest sections to write as there are so many possible
combinations of different products and technologies. When making choices the
starting point is usually what is available to you at the start. This may be anything
from a Web Server, knowledge of a certain language or use of a certain
development tool. In the case of this project, the Author started with virtually
nothing so using an external hosting company with many built in features appeared
to be an attractive choice. More information is given in details, assessments and
conclusions of the three solutions in this report particularly in figures 27, 31 and
33.
The following table however, does try to compare two elements and focuses on
comparing PHP with ASP.Net. It is from an article by Sean Hull, PHP and ASP.Net go
head to head, Ref [5]
PHP 4
PHP 5
ASP.NET
Software price
free
free
free
Platform price
free
free
$$
Speed
strong
strong
weak
Efficiency
strong
strong
weak
Security
strong
strong
strong
Platform
strong
strong
Source
available
yes
yes
no
Exceptions
no
yes
yes
OOP
weak
strong
strong
Hulls conclusion is that PHP 5 is best overall but with the caveat that its not so
good at Object Orientation (as .Net).
27 of 88
Graham Butcher
The following is a summary of the Authors findings regarding some of the products
and technologies discussed so far:
PHP
4
PHP
5
ASP.Net
ASP
3.0
CGI
MySQL
Access/
SQL
Server
Apche
IIS
CFusn
Web
Server
CFML
CCharge
Studio
Software
price
free
free
free
free
free
free
$$
free
$$
$$$$
free
$$
Platform
price
free
free
$$
$$
free
free
$$
free
$$
$$$$
$$$$
$$
Speed
stron
g
stron
g
weak
stron
g
strong
strong
strong
strong
stron
g
strong
stron
g
strong
Efficiency
stron
g
stron
g
weak
stron
g
strong
strong
strong
strong
stron
g
strong
stron
g
strong
Security
stron
g
stron
g
strong
stron
g
strong
strong
strong
strong
stron
g
strong
stron
g
strong
Platform
stron
g
stron
g
weak
(IIS
only)
Weak
(IIS
only)
strong
strong
Weak
(IIS
only)
strong
weak
strong
stron
g
Weak
(PC only)
Source
available
yes
yes
no
no
yes
no
no
no
no
no
no
no
Exceptns
no
yes
yes
no
possible
no
no
no
no
no
no
no
OOP
weak
stron
g
strong
weak
possible
weak
strong
weak
stron
g
strong
stron
g
strong
28 of 88
Graham Butcher
Access was controlled by session-based passwords and all usage of the System,
logged in the database itself.
The following screen shots are from the solution:
29 of 88
Graham Butcher
30 of 88
Graham Butcher
31 of 88
Graham Butcher
32 of 88
Graham Butcher
Multiple databases
33 of 88
Graham Butcher
34 of 88
Graham Butcher
35 of 88
Graham Butcher
The System should be used through the Web using a Web browser.
There isnt a high demand for changing and updating the data. It is
presently carried out by one person the Author.
The generated HTML or any other code (Java, JavaScript) should be portable
and run on as many browsers as possible. Minimal requirement are second
generation browsers with tables support.
Data types common to the Web should be supported, such as text areas, radio
buttons, URLs, email addresses...
Speed and scalability. Users on the Web are expected to wait a second or two
for a reply from a server. The System should respond within that time frame.
Security. The System will not be used to handle financial transactions but may
contain details of equipment and locations.
4.2 Background.
Concert was a joint venture company formed from its two parents BT and AT&T. It
lasted for 3-4 years before it was divided up and given back to the original founding
companies. One of the main businesses of Concert was to supply and manage
Global Bandwidth. The Authors role in the company was to test, point to point
superhighways carrying SDH and SONET traffic, sometimes supplied by outside
36 of 88
Graham Butcher
37 of 88
Graham Butcher
38 of 88
Graham Butcher
Application Choices
Choices made for solution 1
Start Here
Have you a
Web Server
available?
Do you want to
choose an
external host?
Yes
No
No
You must
build
it yourself
IIS
Choose
your
database
SQL
Server
Access
Choose
your
language
ASP
v3.0
Choose
your
database
Other
ODBC eg
Oracle
MySQL
Choose
your
language
Choose your
development
environment
Visual
Studio.Net
Any ODBC
or JDBC
database
Choose
your
language
PHP
ASP.Net
Choose your
development
environment
ColdFusi
on
Apache
Choose
your
database
Try
easily.co.uk
NB Not all
languages are
included here
What type is
it?
Yes
Choose
your
language
CFML
Choose your
development
environment
CodeCharge
Studio
Choose your
development
environment
ColdFusion
Finished Application
39 of 88
Graham Butcher
Product
Included
100 MB Diskspace
FTP
Php 4
MySQL
FrontPage 2000 Extensions
Sun Chilisoft ASP
Own CGI-bin
Mod Perl
Graphical Website Statistics
Quick Use Templates
Extra Diskspace Available
HTML, Graphics & Javascript
Flash
Dreamweaver 3 Compatible
HotMetal Pro Compatible
Helpdesk Support
Linux & Apache Based
Free Software Downloads
Free Links to Development Sites
"Pay As You Go" Data Transfer
Internet Connection via 100 MB line
No Hidden Extra Costs
40 of 88
Graham Butcher
41 of 88
Graham Butcher
42 of 88
Graham Butcher
The new System should be used through the Web using a Web browser no
specialised software required.
The new System should be read only the database is administered by the
infrastructure team
The new System must not force the infrastructure team to work in a different
way.
The generated HTML or any other code (ASP, Java, JavaScript) should be
portable and run on as many browsers as possible. Minimal requirement are
second generation browsers with tables support.
The search syntax must be simple for novices but advanced users must be
able to define complex search expressions. If the results are sortable by all
columns, that would be an advantage
Data types common to the Web should be supported, such as text areas, radio
buttons, URLs, ordering by columns with a single click etc.
Speed and scalability. Users on the Web are expected to wait a second or two
for a reply from a server. The System should respond within that time frame.
Security. The System will not be used to handle financial transactions. It will
hold information about equipment and locations and should be shown outside
BT.
5.2 Background.
The UKIP Test area of the Authors Centre consists of several teams sharing a range
of equipment used for simulating, testing and integrating new products and
features. These will be used in different areas of the live BT IP Networks. The
current Access database is held on a shared drive and is accessible by using the
interface shown in figure 26 below.
43 of 88
Graham Butcher
This interface did not give away many clues about the internals of the database.
The designer of the database was consulted to find out more details.
The next task was to analyse in detail, the structure of the database by looking at
the tables and their relations. MS Access itself provides a useful tool for doing this
under Tools/relationships from the pull-down menus. The analysis showed:
44 of 88
Graham Butcher
This analysis revealed there were 7 tables in total and surprisingly, that there were
no links or relationships in place between the tables. All the Asset data was in one
table. This made the design of the solution considerably easier as the task could
be reduced to displaying one table the 1 Network Devices table.
The main remaining problems to be overcome were:
1. A choice of technologies/languages and interfaces/connections had to be
chosen
2. A suitable platform had to be found to host the service available to all in
UKIP area of the Test Centre.
3. The existing data had to be incorporated.
4. Very importantly the new System should not interfere with the mission
critical, existing System managed by the Infrastructure Team. It should not
even be seen to slow down or change the ways of working of the existing
methods.
5. The new presentation of the data the web pages should be Read Only.
New data should be added and modified using the existing Access interface
which has full auditing built-in.
6. Some new skills had to learned to implement the solution
At this point in time the CodeCharge Studio Development Environment and
automatic code generation System had been discovered and it was decided to put
it to use using an IIS Webserver with ASP 3.0 to interface to the existing Access
database.
45 of 88
Graham Butcher
Application Choices
Choices made for solutions 2 & 3
Start Here
Have you a
Web Server
available?
Do you want to
choose an
external host?
Yes
No
No
You must
build
it yourself
IIS
Choose
your
database
SQL
Server
Access
Choose
your
language
ASP
v3.0
Choose
your
database
Other
ODBC eg
Oracle
MySQL
Choose
your
language
Choose your
development
environment
Visual
Studio.Net
Any ODBC
or JDBC
database
Choose
your
language
PHP
ASP.Net
Choose your
development
environment
ColdFusi
on
Apache
Choose
your
database
Try
easily.co.uk
NB Not all
languages are
included here
What type is
it?
Yes
Choose
your
language
CFML
Choose your
development
environment
CodeCharge
Studio
Choose your
development
environment
ColdFusion
Finished Application
46 of 88
Graham Butcher
Realising the importance of the existing System and building round that
System with a low risk enhancement.
Choosing the interface. In contrast to the original interface, see figure 29,
the web based interface is extremely simple. For the simple search a
single text box is used to search for any record containing any supplied
string see figure 19. For example if just the letter e was entered the
result would be a list of all records. If the word Peakflow is entered, 4
records are displayed all containing Peakflow in at least one field. The
power search allows more specific information. In figure 20 all records that
satisfy building=b48, room=9, owner=james, status=in use are displayed.
5.8 Scalability
With a maximum of 40 or 50 people working in the UKIP area, the chances of
numerous simultaneous queries occurring are low so using Access as a database
should not be a problem. That the current size of the database file is 10 MB, having
grown from 6 MB two years ago, means that there shouldnt be a problem with disk
space on the server. The System implemented should therefore be adequate for
hosting present and future needs.
5.9 Ease of maintenance
All database management and data content is handled by the infrastructure team.
The IIS Webserver is also managed by another team. The only check that has to be
made is that the files are being copied across to the web server as expected. If this
fails, the database web pages will still work but the data wont be completely up to
date. To summarise, the designed System needs very little maintenance.
47 of 88
Graham Butcher
48 of 88
Graham Butcher
This was also tested on a few key users before being released along with the links
between the two pages holding the searches.
5. The final stage was to make the URL to find the web pages more friendly.
This was done by adding a DNS entry on the BT intranet so that URL used
was:
http://xgh1.nat.bt.com/asset register
The development time for this solution was significantly shorter than the time taken
for the first, Concert example. The reasons are:
More experience.
Better Methodology.
49 of 88
Graham Butcher
The System should allow web based access to all the Centres registered
documents1.
The System should remove the need for the existing dedicated software to
view and download documents.
The System should be read only, in that it should not be used for registering
and storing documents
The System should work with specified browsers. As the browser has the
dual purpose of locating the documents and also displaying them, the
specified browser will be MS IE5 or later. This is because MS Office is used
for most documentation within the Centre. If a different type of browser is
used it will be able to locate documents but not display them unless they
are TEXT, JPEG, HTML or other web supported formats.
6.2 Background
XGH1, the Authors work Centre has had its own documentation System for a
number of years. The System which is MS Access based was developed by Paul
Wigens, a work colleague. There have been a number of releases of this dedicated
software. The System is used for storing and retrieving documents and managing
their change control. Within XGH1 there are a number of work areas, each has its
own database of stored documents, the documents themselves being stored on
various different file servers. Most people in XGH1 only have access to two or three
areas, their own work area and the Centres common area. The common area holds
general information like templates used by the Centre.
This System is also seen as mission critical as, if for example, documents (or
details about documents) were lost, it could have a catastrophic economic affect on
the Centres finances. For this reason the Author again developed a safe System
that would not risk the integrity of the existing data and files. Copies are made, 4
times daily, of all database files to the web server, with the live FileStore files left
untouched. The new System is Read Only so does not risk any database corruption.
1
Not everyone has access to all documents in all areas. The new System should
allow the same people the same access that they would have had through the
existing System.
50 of 88
Graham Butcher
The problem therefore resolved to producing two types of search on two different
web pages the Document oriented search and the Author oriented search.
The next page documents, with the help of a flowchart, the platform choices made.
51 of 88
Graham Butcher
Application Choices
Choices made for solutions 2 & 3
Start Here
Have you a
Web Server
available?
Do you want to
choose an
external host?
Yes
No
No
You must
build
it yourself
IIS
Choose
your
database
SQL
Server
Access
Choose
your
language
ASP
v3.0
Choose
your
database
Other
ODBC eg
Oracle
MySQL
Choose
your
language
Choose your
development
environment
Visual
Studio.Net
Any ODBC
or JDBC
database
Choose
your
language
PHP
ASP.Net
Choose your
development
environment
ColdFusi
on
Apache
Choose
your
database
Try
easily.co.uk
NB Not all
languages are
included here
What type is
it?
Yes
Choose
your
language
CFML
Choose your
development
environment
CodeCharge
Studio
Choose your
development
environment
ColdFusion
Finished Application
52 of 88
Graham Butcher
53 of 88
Graham Butcher
54 of 88
Graham Butcher
55 of 88
Graham Butcher
Summary
The resulting product produced by the tool CodeCharge Studio was significantly
quicker in the making and had a more professional look and feel and interface to
it. After having used the product it would be difficult to go back to producing by
hand coding and building from first principles. This finding was put into practice
with the third example as almost the same choices were made as far as database,
language, server and development tool were concerned. As a result the third
example was produced even quicker.
7.3 Evaluation of solution 3
Although solution 3 applies to a completely different area than 2, technically it is
quite similar. A lot of the comments applied to 2 also apply to 3.
One of the main achievements of this solution was to not only locate the document
as a URL, but return the URL back to the browser so that the required document
could be displayed (in the browser).
One of the biggest challenges was to link several tables into a query that would
display the required data. The CodeCharge Studio query builder certainly helped
with this task.
The solution works particularly well when used over a 56k dial up link. The
previous, Access program based solution forces you to down load the whole
FileStore (back end) database before it can supply any information. This can take
over an hour to download over 56k, telephone link. The new web based version
retrieves information in a matter of seconds and documents, depending on size, in a
matter of moments.
Another advantage of the System is that documents can now be described by
their URL. The URL can be sent or distributed rather than the document itself. For
example if a large document is to be distributed to a group of people, it can first be
stored in FileStore and then the URL can be cut and pasted into an email to the
distribution list. This has two advantages:
The recipients dont get their email Inboxes clogged up with large
attachments
If the version of the document changes or the document is updated, the URL
is still valid and the latest version is found.
56 of 88
Graham Butcher
Surprisingly this facility wasnt picked up within the Centre. FileStore seemed to
keep its perceived function as a repository for documents rather than a
collaboration tool or an email un-clogger.
The final advantage is that the web based front end allows users to switch
between databases very easily. This is achieved by using the database chooser
page. The old Access based System forced a download of a complete database to
the users PCs before displaying any information.
There are of course problems with retrieving documents directly in to ones browser
as this technique is totally dependent on the state of the network. Sometimes the
web browser would time out before the document arrived because the local LAN
was overloaded. Using the refresh button seems to help because the first half of
the file transfer gets cached and refresh usually completes the transfer. Even
cutting and pasting the URL into a new browser window has been known to work.
As mentioned earlier MS Project files did not appear to be viewable in MS browsers
even when the application MS Project - was loaded on the recipients PC. The
contents of Zip files were viewable but required the application to be installed.
During the course of the development of this third solution the Author was asked to
design a web based uploading System. The reasons for not doing this were fairly
simple:
It would take too long and extend the project beyond its recommended timespan and effort.
It had the risk of not working properly, corrupting the databases and losing,
mission critical information.
The FileStore System was due to phased out in a matter of years anyway
because of the universal adoption of the bought in solution called LiveLink.
LiveLink is a commercial product rolled out to the whole of BT with a few
additional features. Documents can be uploaded and downloaded through
its web interface, permissions can be controlled more tightly, communities
and group ownerships can be created and it covers the whole of the
Company. More information at
http://www.opentext.com/products/livelink/enterprise-server/
Users werent ready for a new way of working and didnt have the skills to
use the new interface. An example of a comment received was you cant
browse certain areas or categories of documents in a hierarchical way that
you could with the old software. This is not in fact true as the orderable
columns allow you to separate categories and types. It needs practice,
though.
57 of 88
Graham Butcher
One place the new software is used is in the test labs where testers are away from
their desktop PCs and require documents quickly. Being able to retrieve these
without the usual dedicated program is a definite advantage.
There was initially some concern that the whole of BT would be able to see all
documents in FileStore but the fileservers that hold the documents prevent this. All
that people can retrieve is the location, version and a few other details.
58 of 88
Graham Butcher
Chapter 8 : Recommendations
General recommendations are given before specific ones for each solution.
8.1 General Recommendations
1. Consider the Security of the solution first. This can affect the location of
files and permissions and is harder to modify later in the development.
See the section on security 2.5.4 Security.
2. Consider the risks involved with the data. Who should see it, who
shouldnt? When building a new System from scratch, what is the worst
case scenario that can happen? The same applies to enhancing an existing
System but probably more so. Design accordingly.
8.2 Recommendations from Solution 1
The following recommendations are:
1. Choose a web server or ISP where applications such as PHP and MySQL are
already installed.
2. Search the Web for existing code that can be adapted for your needs instead
of writing everything from scratch. Get involved with communities that
discuss and help each other overcome problems.
3. Dont expect Microsoft Database files (eg *.mdb) to be readable (or
openable) on a UNIX, Apache Server.
4. Use a development environment and software tools as much as possible as
in solutions 2 and 3.
Future Improvements and Enhancements
If this solution were to be continued the following improvements are recommended:
1. Writing a web page to enter data. As mentioned earlier, the current
System uses the built-in Administration tool to add data. A web page (or
pages) dedicated to updating the data would be a more efficient way of
doing things. There are two ways, or levels of difficulty, of doing this:
a. The first is fairly simple and straightforward one table is updated at
a time using a simple and separate web form for each table.
b. The second type of web page would be one that takes account of
links between tables. For example if the Assets table were being
updated a pull-down list of locations, extracted from the locations
table could be displayed. If a suitable location didnt exist it could be
added at that point without going to a form specific for adding
locations.
2. A second enhancement is something that would have to be done before
b) above. That is, establish links between the tables by writing the
necessary SQL. This would allow deployment of the database as it was
originally designed in its Access format ie with linked tables. The links
were lost when the data was ported from Access to MySQL.
3. The third improvement or enhancement recommended would be to
present the data in a more attractive and compact way. See Figure 17 View of Assets Table for the current method of presentation.
8.3 Recommendations from Solution 2
The recommendation and lesson learnt from this solution is use development
tools as much as possible. This has been mentioned several times in the report.
59 of 88
Graham Butcher
60 of 88
Graham Butcher
One methods allows you to perform a script based upload and the other method
allows you to perform an ADO.Stream based file upload.
In both cases it appears that the component requires your server to have a
temporary directory into which the file is initially loaded and a 'permanent'
directory into which the file is subsequently moved if it passes validation.
However, there are times when the application will require that the Upload
component must upload the file directly into a field in a database table. The
reasons can range from the need for additional security or simply because the ISP
that hosts your site is unwilling to allow sufficient permissions to the necessary
directories.
Furthermore, if you do have sufficient permissions on these directories, you may be
concerned that the directories could be discovered and in one way or another
compromised.
As a result of this concern, you have probably adjusted your component to disallow
the uploading of certain types of files, especially .exe, .com, .bat files and so forth.
And you would be correct to have done so.
Still, there will be times when a System requires a higher level of security or the
ability to upload the executable type files. And thus make it harder for a hacker to
compromise your web sites security measures.
Uploading the file directly to the database and downloading the file directly from the
database may by your only choice.
Since CodeCharge Studio comes with sufficient information about their upload
component, our application is going to use a different one.
The component we're going to use is a free Upload component called
PureASPUpload. This component is apparently capable of uploading files up to 2 GIG
in size.
Normally however, this limitation is no limitation at all.
This component can be downloaded from
http://www.ormacdigital.com/pureaspup load/help/default, htm
They also sell another product called HUGE-ASP Upload that allows you to upload files
larger then 2 GIG.
Note: Downloading documents, images and other binary files from your database
can cause a strain on both the database and your bandwidth. Unfortunately, reality
often dictates things and we have no choice but to perform uploads directly to a
table.
Since this component allows files to be uploaded directly to a database field and is
free, it's a likely choice to use until the CodeCharge Studio component provides
similar functionality.
61 of 88
Graham Butcher
Chapter 9 : Conclusions
This section covers the conclusions from the 3 examples and also a few overall
conclusions.
9.1 Conclusions from Solution 1
This type of solution using Apache, PHP and MySQL must be the cheapest. It also
gives considerable scope for expansion and Scalability. The main thing that went
against it was the overhead of having to learn PHP. PHP is not a difficult language
but as, like most languages, it has a few quirks that need getting used to.
After having developed the second and third solutions using the CodeCharge Studio
development tool, this solution was re-visited with a view of improving the code and
adding updating pages. It was found that a live development environment was not
easy to setup with the remote server managed by a third party. As a result the
CodeCharge Studio tool could not connect directly to the deployed database and
test during development. There was no local copy of MySQL to work with so
development by this method was abandoned.
9.2 Conclusions from Solution 2
Several unexpected findings came out of this solution:
One reaction of to the solution was what are the printing capabilities of the
solution? This demonstrated that people are not always ready for new
ways of working.
People dont always want their data publicised. This might be for several
reasons it might not be as up-to-date as they would like, it may result in
more empowerment to outside groups or they simply dont like changes to
the status quo.
The main surprise from this solution was, that after some initial teething problems
with the Development Tool, how easy it was to produce a simple working System
that met the design requirements.
9.3 Conclusions from Solution 3
The main conclusions drawn from producing solution number three were:
It would have been nice to have also produced pages for uploading
documents instead of just the Read Only solution produced. The Author
believes however, that the correct decision was made in not doing this, as
the risks were too high.
The solution is still in use but will have a limited life time as FileStore is being
phased out to make way for LiveLink. This will happen in the next 4-6
months during which time the enhanced FileStore can be used for finding
and porting over documents and files to the new System.
9.4 Overall Conclusions
62 of 88
Graham Butcher
63 of 88
Graham Butcher
Chapter 10 : Acknowledgements
Special thanks are given to my two supervisors, Stefan Poslad, Queen Mary College
and Alan Wheeler, BT Exact.
Also thanks to Paul Wigens and Bob Reason for their help and encouragement with
FileStore development.
Thanks to John Williams, Alex Workman and Mike Carter with the document review
process.
Chapter 11 : References
Dr Phill
Edwards
2003
Kwok-Bun
Yue and Wei
Ding
ITICSE, Leeds, UK
Not
Applicable
www.macromedia.com
Not Applicable
Not
Applicable
Not
Applicable
www.codecharge.com
Not Applicable
Not
Applicable
Sean Hull
www.oracle.com/technology/pu
b/articles/hull_asp.html
August
2004
Prof Ziga
Turk
July 1999
Not
Applicable
www.ASP.Net
Not Applicable
Not
Applicable
Janet Valade
2002
Author
Unknown
Building a Document
Management System
August
2004
10
Bill Hatfield
1999
11
Nitin Pandey,
Yesh Singhal
and Mridula
Parihar
2002
12
Ben Forta
and Nate
Weiss
2003
64 of 88
Graham Butcher
ADO
Apache
API
ASP
Active Server Page (Language) - not to be confused with Application Server Platform
CFML
CGI
ColdFusion
CSV
DB
Database
GNU
"Not Unix"
GPL
HTML
IDE
IIS
Java
JavaScript
An interpreted, scripting language, similar to Java - originates from Sun Computers - runs
within user's browser - is browser dependent
JDBC
JScript
LAN
Linux
MS
Microsoft
NCSA
NVT
PERL
PHP
RAD
RDBMS
SDH
65 of 88
Graham Butcher
Solaris
SQL
UKIP
The UK IP part of BT
WODA
Web Oriented Database (System Prof Ziga Turk, see ref [6])
WOSA
XGH1
XML
66 of 88
Graham Butcher
67 of 88
Graham Butcher
68 of 88
Graham Butcher
Extensibility
CodeCharge Studio has an open architecture and is fully extensible. Most of the functionality is implemented as HTML,
JavaScript, XML, and XSL. Users can customize almost any part of the built-in functionality or develop their own
components, such as Builders, Components, Actions, and Themes. In the future, users will be able to obtain the
CodeCharge SDK and develop their own code generation templates.
69 of 88
Graham Butcher
70 of 88
Graham Butcher
Applications generated with CodeCharge Studio can be easily extended by adding custom code or by using any of the
predefined Actions. Actions are user-definable code components that are inserted into events. Many actions are
provided with the product, while additional actions can be created by users and shared with others. Internally, actions
consist of XML and XSL code that can be easily customized.
71 of 88
Graham Butcher
Unlike manually edited code that cannot always be recovered without the use of a versioning System or a backup,
actions can be regenerated at any time.
On the server, the generated code is fully separated from the HTML. During execution, the code files load
corresponding HTML files into memory, replace static content with database values and output the final HTML to the
browser. This method of separating the code from the HTML design content allows designers to make future changes to
the site without having to modify the code.
72 of 88
Graham Butcher
Sample Databases
Several sample databases and applications are provided with the distribution to help you get started with CodeCharge
Studio.
73 of 88
Graham Butcher
74 of 88