Advanced Topics

Advanced Topics

HTTP Servers
CGI
Fill-out Forms
Processing Form Data
Imagemaps
Creating Dynamic Documents

Now we discuss some of the more sophisticated aspects of the World Wide Web. We reserve this for now because some of it requires you to have access to certain programs and files on a computer system which is running the HTTP server.

First, we will talk about the HTTP server software, the heart of WWW operations. Then we will go onto CGI (Common Gateway Interface) which allows browser to interact with programs as well as simple HTML files. Finally we will cover imagemaps.

HTTP Servers

A HTTP Server is simply a program which runs on a computer and handles requests for information from other computers on the Internet. At time of printing, there are dozens of public domain and commercial Web server software packages available. While Unix is the operating system platform most often used for HTTP servers, there is server software solutions for a variety of other systems as well.

While it is beyond the scope of this book to cover the intricate details of the installation, configuration and maintenance of a HTTP server, we do provide information about where the software is available and how to find such information yourself.

This section will cover some of the more popular servers: NCSA, CERN, and Netscape's Netsite.

NCSA

NCSA's server software is, by far, the most common HTTP server. There is a secure NCSA server available as well.

The NCSA HTTP Server software distribution is available via anonymous FTP at ftp.ncsa.uiuc.edu in the directory /Web/httpd/Unix/ncsa_httpd/.

Information about the server, its installation, and configuration is available at the URL http://hoohoo.ncsa.uiuc.edu/.

CERN

Because the World Wide Web began at the CERN particle physics laboratory, this server software has been around longer than any other. This presents some advantages in terms of reliability and stability. However, the CERN server software is not known for its ease of installation and configuration.

Information about downloading the public domain CERN server software and how to install, configure, and maintain it is available at the URL http://www.w3.org/hypertext/WWW/Daemon/Status.html .

Netscape Netsite

Netscape's commercial servers are very popular for those people who are interested in paying money for Web server software.

At time of printing, Netscape offers two server packages: Netsite Communications Server and Netsite Commerce Server. They are both fully functional HTTP servers which Netscape claims to be very efficient and easy to install. The Netsite Commerce Server is different than the Netsite Communications Server because it adds the Netscape Secure Mode for encrypted data transfer capability for secure financial transactions.

For information about these products see the URL http://www.netscape.com/MCOM/products_docs/server.html or contact Netscape Communications via telephone at (415)528-2555.

CGI

CGI, the Common Gateway Interface, adds a whole new level of functionality to the World Wide Web. CGI allows you to use programs to generate Web pages or to process input information from Web pages, with fill-out forms.

This opens a door to a whole new realm of possibilities for the Web.

Unfortunately, it is beyond the scope of this text to go into great detail about the CGI programs. Our goal is to introduce you to the functionality of CGI so that, when presented with the need to incorporate it into your Web designs, you will understand how it is done.

Most CGI programs are written in C, Perl, or Unix shell script. But most any programming language can be used.

Because of the security considerations involved with letting any anonymous user on the Internet run programs on your computer system, CGI programs are usually kept in one directory which is often only accessible by the system administrator. Unfortunately, this often makes installation and maintenance of CGI programs difficult for the normal user.

There are solutions to this problem, however. One is a CGI program called cgiwrap, which resides in the systems HTTPD cgi-bin directory. It allows normal users on a Unix system to create a directory called cgi-bin in their HTML directory (i.e. public_html/) for the purpose of storing their own CGI programs. With cgiwrap, a user can refer to a script or program in their personal cgi-bin directory by referencing a URL like this:

http://www.netfx.com/cgi-bin/cgiwrap?user=jblow&script=myinfo.pl

What Can I Do with CGI?

While many people can gain an understanding of what CGI is and how it works, it is often unclear what it can be used for.

As stated before, with CGI we can use programs on the WWW server computer to either produce HTML or to process information from fill-out forms, or do both. First we'll look at how we can use programs to generate Web pages for us.

Here is a Unix shell script which tells us the time and date according to the WWW server:

#!/bin/sh
echo "Content-type: text/html\n"
echo "<html><head><title>DATE AND TIME</title></head>"
echo "<body>The date and time right now:<pre>"
date
echo "</pre></body></html>"

This script, when called as a CGI program through the WWW server will produce this output:

There really is no limit to what you can do with CGI. Examples of applications of CGI programs which only generate HTML output:

Print access statistics about your WWW pages
Create a list of random URLs to select from
Check to see if the page author is online
Countdown timer (i.e. Minutes until I'm married)

These are just a small sampling of the possibilities.

When producing output for an HTML browser, it is essential that you include a MIME content type specification as the first line in your output and a blank line after it. For HTML information this should be Content-type: text/html. If you are sending plain ASCII text, this should be Content-type: text/plain.

Fill-out Forms

Fill out forms are the current epitome of interaction on the Web. They allow you to input information into you browser, which is then fed to a CGI program. The program may generate some sort of output for you to see, or it may do something entirely different (i.e. send a mail message to someone.)

An example of the use of fill-out forms is any of the search engines on the Web- including those we cover in the book (see Chapter 3). Let's look at the Yahoo! When using an input field which is of the type TEXTor PASSWORD, you can use the SIZE argument to adjust how wide the input field will appear in the form. The size is measured in number of characters.

Let's look at Yahoo's search page again.

You will notice, the Yahoo form has several new page elements which have not been discussed yet: text input fields, buttons, radio buttons, and selection menus.

Now lets look at some of the HTML used to generate the Yahoo search form above.

<h2>Yahoo Search</h2>
<form method=GET 
action="http://search.yahoo.com/bin/search">
Find all matches containing the <i>keys</i> 
(separated by space)<br>
<input size=30 name=p> <input type=submit 
value=Search> <input type=reset value=Clear> <br>
<dl>
<dt>Find matches in
<input type="checkbox" name=t checked>Title
<input type="checkbox" name=u checked>URL
<input type="checkbox" name=c checked>Comments<br>
<dt><input type="checkbox" name=i > Case 
sensitive matching<br>
<dt>Find matches that contain
<dd><input type="radio" name=s value="o">At least one 
of the <i>keys</i> (boolean <b>or</b>)
<dd><input type="radio" name=s value="a" 
checked>All <i>keys</i> (boolean <b>and</b>)
<dd><input type="radio" name=s value="s">All 
<i>keys</i> as a single string
<dt>Consider <i>keys</i> to be
<dd><input type="radio" name=w 
value="s" checked>Substrings
<dd><input type="radio" name=w value="w">Complete words
<dt>Limit the number of matches to 
<select name=l><option selected>100<option> 200<option> 
300<option> unlimited</select>
</dl>
</form>

<FORM>
As you can see in the HTML shown, all the fill-out form elements are place between the <FORM> and the </FORM> tags.
The <FORM> tag has some arguments which indicate what is to happen when you attempt to submit the form to a server.
The first argument to <FORM>, METHOD, can be set to either POST or GET. The difference between the two of these is how the information is passed to the CGI program. When using GET, the data from the form is sent to the program in an environmental variable called QUERY_STRING. When you use POST, the data is sent to the program as standard input to the program. For general purposes, it is recommended that you use POST.
The next argument shown, ACTION, indicates the URL which identifies the location of the CGI program which will handle the data processing, in this case, http://search.yahoo.com/bin/search.
<INPUT>
The <INPUT> tag is used for collecting a variety of different kinds of input.
The TYPE argument to the tag indicates what kind of input we are collecting with the tag. Here is a list of possible values for the TYPE argument:

TEXT Normal text.

PASSWORD Same as normal text except letters appear as asterisks.

RADIO A single button, which toggles, to set a value on or off.

CHECKBOX An open box which can be checked on or off.

The NAME Argument
Each input tag in a form is usually given a name with the NAME argument to the <INPUT> tag. This name becomes some sort of identifier variable to the CGI program which handles the input. For example, here we create a text entry field called fullname:

<input type=text name=fullname>

Let's assume that in the form, we input Joey Jo Jo Jr. Shabadoo in this input field. When the CGI program receives the form data, it will create some sort of relationship which establishes that the information represented by fullname contains "Joey Jo Jo Jr. Shabadoo".
The VALUE Argument
This is used to assign a default value to a field. For example, you could make a radio button already selected. You could set up a text field to already have text in it when the form is loaded. When working with a check-box, use the CHECKED argument instead, to indicate the check-box should already be checked.

Check here if you want pizza: <input type="checkbox" name="pizza" value="yes">

This would create a default value for the name pizza to be yes.
Other Arguments
When using an input field which is of the type TEXTor PASSWORD, you can use the SIZE argument to adjust how wide the input field will appear in the form. The size is measured in number of characters. This does not limit the form user to how many characters can be entered in, just how many characters will be displayed at one time. You can use the MAXLENGTH argument to specify the maximum number of characters that can be entered in a TEXT or PASSWORD field.
The SUBMIT and RESET Types
These are two special TYPE values for the <INPUT> tag which are used to create special push buttons.
A SUBMIT button is used to send the form data to the CGI program. Selecting this button is typically the last thing a person would do in working with a fill-out form.
The RESET button clears all the values in the form and sets them back to the default values (as specified by the HTML code).
You can use the VALUE argument to create a label for the button:

<input type="submit" value="Send the information">

<SELECT>
The <SELECT> tag allows you to let users choose from a list of options. The list usually appears as a pull down menu. This tag is used in Yahoo!'s search form to let the user specify the number of matches to return.

Unlike the <INPUT> tag, there is a matching </SELECT> tag which is used to mark the end of the options list information.
You use the NAME argument with <SELECT> to give the selection an associated name to be used by the CGI program.
Each option is listed between the <SELECT> and </SELECT> tags and enclosed in <OPTION> and </OPTION> tags:

<SELECT name="icecream"> <OPTION>Vanilla</OPTION> <OPTION>Chocolate</OPTION> <OPTION>Strawberry</OPTION> </SELECT>

By default, the browser will usually show the first option as being the default selection. If you wish to change the default selection, include the argument SELECTED within the <OPTION> tag:

<OPTION SELECTED>Chocolate</OPTION>

You can change the <SELECT> element from a pull-down menu into a scrolled list with the SIZE argument to the <SELECT> tag. If this argument is set to the value of 1, it will produce a pull-down menu (this is the default). If it is set to 2 or more, a scrolling menu will be shown with the value specifying how many selections will be shown at one time.

The default behavior of the <SELECT> tag is to allow one item from the options list to be selected. However, if you use the MULTIPLE argument to the <SELECT> tag, multiple items in the options list can be selected.
<TEXTAREA>
To gather multiple lines of text input, you can use the <TEXTAREA> tag. Like the <SELECT>tag, you must also use a </TEXTAREA> tag to mark the end of the element.
You can use ROWS and COLS arguments with <TEXTAREA> to define the dimensions of the multi-line text input field.
Unlike with <INPUT> tags, where you can use the VALUE argument to preset the contents of the form element, with the <TEXTAREA> element, all you need to do is put the data you want to default between the <TEXTAREA> and </TEXTAREA> tags.
Remember, also, to use the NAME argument to assign the input data a symbolic name.

Enter your comments below: <br> <textarea name="comments" rows=5 col=60></textarea>

Using Hidden Fields
There are times when you may want to pass information to a CGI program from a form, but not make the information available to the user to modify. You can use the HIDDEN value for the TYPE argument to the <INPUT> tag to include hidden data in your form:
<input type="hidden" name="access" value="restricted">

Processing Form Data

The method used for processing data submitted by a fill-out form depends what METHOD is used to submit it. Therefore, we divide this section up for POST and GET methods.
It goes without saying that you will need some programming experience to tackle the task of processing form data. If you lack this knowledge, you may want to consider a book on Perl.
The data sent from a form arrives at the CGI program in the following form, regardless of what method is used:
name1=value1&name2= value2&name3=value3
The first thing the CGI program needs to do is to extract the name-value pairs from this string.
Processing POST Data
The POST method sends the data to the CGI program as standard input. Most form data processing programs are written in Perl and C rather than Unix shell because the data needs to be stripped down to each individual name and its corresponding value.
Below is an excerpt of Perl code which handles POST submitted data and creates an array for handling by the program:

# Get the input read(STDIN, $buffer, $ENV{'CONTENT_LENGTH'}); # Split the name-value pairs @pairs = split(/&/, $buffer); foreach $pair (@pairs) { ($name, $value) = split(/=/, $pair); # Un-Webify plus signs and %-encoding $value =~ tr/+/ /; $value =~ s/%([a-fA-F0-9][a-fA-F0-9])/pack("C", hex($1))/eg; $FORM{$name} = $value; }

This code takes the data and puts it into individual array slices in the array $FORM. Once you have the data separated like this, you can do conditional tests on the values sent to the CGI program or send the data on to another program, like a mailer.
Processing GET Data
Form data sent by the GET method arrives as a string assigned the environmental variable QUERY_STRING. So, to extract the name-value pairs from that variable, we can use the following Perl routine. It is nearly identical to the routine we use for POST data.

# Get the input $buffer = $ENV('QUERY_STRING'); # Split the name-value pairs @pairs = split(/&/, $buffer); foreach $pair (@pairs) { ($name, $value) = split(/=/, $pair); # Un-Webify plus signs and %-encoding $value =~ tr/+/ /; $value =~ s/%([a-fA-F0-9][a-fA-F0-9])/pack("C", hex($1))/eg; $FORM{$name} = $value; }

Imagemaps

An imagemap is similar to a scratch and sniff book you might have had as a child. For the most part it is a regular image file (i.e. GIF or JPEG), except that if and when you select the image with your mouse pointer, your graphical browser will report information about where in the image your pointer was to the WWW server. The following figure illustrates how an image map works:

This gives you the ability to put up a picture or graphic which lists options either in the form of graphical buttons, or with objects like flags, signs, or in the case of the above example, common objects in the image. Then, when someone selects the image, a different link loads depending on what area of the image was selected.
Next is a real image which is used as an imagemap. This is a somewhat complex imagemap and is used to show all the WWW services in the Salt Lake City/Ogden/Provo area in Utah. Clicking on a site name will load their WWW pages.

We can divide the discussion on imagemaps into two sides: the HTML- what you as an author needs to do to produce a clickable imagemap- and the WWW Server- what needs to be done to configure the HTTP software to process your imagemap.
The HTML Side
Specifying that an image is a clickable imagemap in your HTML is very simple. All you need to do is include the ISMAP argument inside your <IMG> tag:

<img src="housemap1.gif" ismap>

The ISMAP argument tells you browser to send pointer position information if the image is selected.
The image still needs to be anchored, or assigned to a link, so that a program is called to process the information about the pointer location.
This is done by calling a program in the server's cgi-bin directory called imagemap and including the name of the corresponding entry in the server's imagemap.conf file (see next section). Here is an example:

<a href="/cgi-bin/imagemap/housemap"> <img src="housemap1.gif" ismap></a>

This probably looks weird because housemap looks like a file in a directory called imagemap. However, that is how it works and it's only the beginning of the seemingly awkward methods of doing imagemaps.
The WWW Server Side
As shown above, in order for the server to process imagemap information, there needs to be information in the imagemap.conf file about the image. This file is usually stored in the conf/ directory of the server software installation.
The syntax of this contents of this file is as follows:
symbolic-name:path-to-actual-map-file

This is meant to be misleading, so if you feel as if you're being mislead, do not fret. Because the URL in the HTML file calling the imagemap program doesn't contain the name of the file where the actual map file (the file which contains the link information for each zone of the image) resides, there is no way for someone to find out unless they have direct access to the imagemap.conf file on the server. An example entry in this file which would correspond to the HTML example in the previous example might look like this:
housemap: /home/fozz/public_html/maps/house1.map

So to step back and go through all this again: Let's assume someone clicks on our clickable imagemap (housemap1.gif). The browser sends the information to the URL /cgi-bin/imagemap/housemap. The imagemap program consults the imagemap.conf file looking for an entry called housemap. It finds it and then discoveres the map file is in the file /home/fozz/public_html/maps/house1.map. The imagemap program then processes that file, which contains the information about what URL should be loaded based on the pointer location on the image, and sends that information back to the browser, which then loads the new URL. So, now we just need to know how to create one of these map files.
Some server implementations allow normal users on the system to house their own map files. In this case, the map file can exist in the user's public_html directory and the URL in HTML would look something like this:
<a href="http://www.someplace.com/cgi-bin/imagemap/~jdoe/maps/map1.map">
<img src="map1.gif"></a>

Creating a Map File
The recommended way to create a map file, is with a program which lets you graphically specify the areas in your image that you want to be made clickable and then what URL should be assigned to each.
Here is an example of what a map file looks like:

default http://www.webcom.com/empty.html rect http://www.webcom.com/overview.html 8,5 81,34 rect http://www.webcom.com/news.html 86,7 139,34 rect http://www.webcom.com/products.html 145,6 215,35 rect http://www.webcom.com/contacts.html 221,8 291,37 rect http://www.webcom.com/other.html 299,6 437,35

The URLs in this file need to be explicit URLs.
There are programs available for Windows, Macintosh and Unix platforms. Here are some WWW sites to check:
Windows v3.1/Windows NT/Windows 95

MapThis http://galadriel.ecaetc.ohio-state.edu/tc/mt/

MapEdit http://sunsite.unc.edu/boutell/mapedit/mapedit.html

HotSpots http://www.hooked.net/users/1auto/hotspots.html

Macintosh

Mac-Imagemap http://weyl.zib-berlin.de/imagemap/Mac-ImageMap.html

WebMap http://www.city.net/cnx/software/webmap.html

Unix

MapMarker http://www.dl.ac.uk/CBMT/mapmarker/HOME.html

MapEdit http://sunsite.unc.edu/boutell/mapedit/mapedit.html

Creating Dynamic Documents

The concept of a Web page we have described so far defines a Web page which we can load, view, select links from it to load other pages, input data into a form, and submit form data through a URL. Up to this point, the pages have never had any control of themselves. That is going to change.
Client-pull and server-push are ways you can make your Web pages more dynamic. For example, you can have a page load and then two seconds later, a new page will load. You can produce a WWW slideshow. You could also have a graphic continue to reload giving the effect of animation or live video, or have up to date stocks or weather information constantly reloading.
At time of printing, the Netscape Navigator is the only Web browser which supports these operations.
Client Pull
With client pull, all the tricks are in the HTML code itself. The secret to making this work is including a <META> tag in your HTML header (between the <HEAD> and </HEAD> pages). Using the <META> tag, we can trick the browser into thinking a function has been called. Here is an example:

<meta http-equiv="Refresh" content="4">

If you included this in your HTML header, the page would reload every 4 seconds... indefinitely (or until you clicked on the Stop button). This could be very interesting if you had a daemon, or background process, changing your inline graphics on a regular basis as well.
We can also use this tag to trick the browser into thinking a new URL has been referenced and to load a specified page:

<meta http-equiv="Refresh" content="6; http://www.someplace.com/~jdoe/nextpage.html">

With this code in the HTML header, the browser will display the current page for 6 seconds and then proceed to load the page at URL http://www.someplace.com/~jdoe/nextpage.html .
You can produce some pretty interesting effects by linking dynamic pages like this together- each waiting and then loading another page. This is how a slideshow could be created.
Server Push
Unlike client pull, the tricks in server push are done with some kind of CGI program. The CGI program is then used to send data to the browser on an open connection that never closes unless the CGI program or the browser forces it closed.
The trick to creating a connection like this is by using a MIME code which tells the browser the informaton is mixed and to keep loading over the old data. The MIME Content-type code for this is multipart/x-mixed-replace.
One popular use of server push is to generate a simplistic animation using several iterations of image data sent to the browser one at a time, each placed over the last one drawn.
For example, here is a Perl script which would continually sends a new image, or animation frame, to the browser:

#!/usr/local/bin/perl $num = 1;#setting variable at start $bound = "\n--boundarystring\n";#display at boundaries $end = "\n--boundarystring--\n";#display at end print "Content-type: multipart/x-mixed-replace;boundary=boundarystring\n\n"; while (1 == 1) { if ($num < 50) { $pic = `cat ../images/push$num.gif`; print $bound; print "Content-type: image/gif\n\n"; print $pic; # send the GIF file $num = $num + 1; sleep 2 } else { $num = 1; } } print $end; #it is likely we will never get here.

Essentially, here is what is happening here:
1. We send Content-type: multipart/x-mixed-replace;boundary=boundarystring to introduce open our server push connection so we can keep sending data.
The boundary=boundarystring tells the browser that the string "boundarystring" should be used to mark the beginning and end of each data block (in this case, each GIF image) sent to the browser.
2. We output the boundary marker (print $bound).
3. We send the MIME code for the data (Content-type: image/gif).
4. We send the data (the GIF file) with the print $pic command.
5. We increment the $num variable which ultimately gives us a new GIF file to look at.
6. Then we go back to Step 2

TEXT	Normal text.
PASSWORD	Same as normal text except letters appear as asterisks.
RADIO	A single button, which toggles, to set a value on or off.
CHECKBOX	An open box which can be checked on or off.

MapThis	http://galadriel.ecaetc.ohio-state.edu/tc/mt/
MapEdit	http://sunsite.unc.edu/boutell/mapedit/mapedit.html
HotSpots	http://www.hooked.net/users/1auto/hotspots.html

Mac-Imagemap	http://weyl.zib-berlin.de/imagemap/Mac-ImageMap.html
WebMap	http://www.city.net/cnx/software/webmap.html

MapMarker	http://www.dl.ac.uk/CBMT/mapmarker/HOME.html
MapEdit	http://sunsite.unc.edu/boutell/mapedit/mapedit.html