Using RPG-CGI for Web Enablement

On most web servers other than the AS/400 system, Common Gateway Interface (CGI) programming implies working with scripts written in the PERL language. Although there is an unsupported version of PERL available for the AS/400, most AS/400 CGI programs are created using ILE RPG, ILE COBOL or ILE C. Because AS/400 CGI programs are compiled, they typically will perform better than interpreted CGI programs such as Net.Data or PERL scripts.

All compiled AS/400 CGI programs must be invoked from the AS/400 library file system. CGI programs are created with traditional AS/400 commands using the AS/400 command line.

Why use CGI programming

The primary reason to use CGI programming on the AS/400 system is that you or your staff may already be very familiar with one of the AS/400 system programming languages. For example, if you already know RPG, it will be relatively simple to learn how to incorporate CGI processing techniques into an RPG program, compared with learning Net.Data or Java servlets.

When you write a CGI program, you have access to all of the AS/400 system programming tools and constructs that you are used to working with. For example, you can use native database operations in your CGI programs. You can also use string handling operations in the language to create the exact HTML statements that you need. Finally, you can use the same debugging tools that you work with for other types of application programs to help you quickly put your CGI program into production.

CGI Processing

To work with CGI programs effectively, you must understand the following:

· How a CGI program is invoked in response to a request in the browser

· How form data is sent from the browser to the CGI program

· The IBM HTTP Server for AS/400

· How form data is made available to the CGI program while it is running

· How HTML generated in the CGI program is returned to the browser

Assuming that your IBM HTTP Server for AS/400 is properly configured with the required MAP, PASS and EXEC directives, here is the process that occurs when you invoke a CGI program. The step numbers correspond to the steps shown in Figure 1.

Figure 1: CGI processing flow.

Step 1 – A CGI program is requested on an incoming URL.

You start a CGI program by entering a URL containing the request in your browser’s address entry area or you click on a link or a button on a web page that contains the request. The request is sent to the IBM HTTP Server for AS/400 along with any form data that was entered on the web page.

Step 2 – The CGI program is invoked.

The IBM HTTP Server for AS/400 uses the path on the incoming URL to locate the CGI program. The program is located based on a MAP, PASS or EXEC directive that matches the /cgibin part of the URL. You need at least one EXEC directive in your HTTP Server configuration file to enable CGI programs to be invoked. The EXEC directive may contain the replacement URL string that points to the library where the CGI program is located, or a MAP or PASS directive may contain the replacement URL.

For example, the following directive in the HTTP Server configuration file might be used to allow the TESTCGI program to be invoked:

Exec /cgibin/* /QSYS.LIB/CGIBIN.LIB/*.pgm

Steps 3, 4– The CGI program gets requested data.

Now that the CGI program is invoked, it runs like any other AS/400 program. It can open files, work with the DB2/400 database, run SQL statements, or call other AS/400 system services such as other programs, commands or APIs. At this point, the CGI program is conceptually similar to an AS/400 system workstation program, in that it has received input from the browser form and is preparing a response to be sent back to the browser.

Step 5 – Resulting HTML is returned to the IBM HTTP Server for AS/400.

After constructing the response HTML, the CGI program uses API calls to send the resulting HTML to the IBM HTTP Server for AS/400.

Step 6 – The resulting HTML is sent back to the browser.

The IBM HTTP Server for AS/400 sends the completed HTML page back to the browser. At this point, the process is complete. The user can request another CGI program which starts the process over again.

How a CGI program performs I/O to the browser

When your CGI program is launched by a Web server job that is capable of handling CGI requests, the Web server job provides two stream files that are used to communicate with the browser, as shown in Figure 2:

STDIN (Standard Input) – this stream file contains a stream of data sent from the browser. The data in the stream is HTTP encoded.

STDOUT (Standard Output) – this stream file is used to return HTTP server headers and content to the browser. In most cases, you will let IBM HTTP Server for AS/400 generate the required HTTP server headers, and provide only the content.

Figure 2: The Web server job provides the STDIN and STDOUT files to the CGI program.

Because RPG and COBOL do not directly support access to the STDIN and STDOUT stream files, you need to use system APIs to work with those files.

HTTP Server Directives

Before your AS/400 can accept and process CGI programs in the AS/400 HTTP Server, you need to enable CGI processing in the server configuration file. The following lines show a sample web server configuration that will allow CGI programs in library CGIBIN to be invoked:

# HTTP CONFIGURATION FILE

Port 8099

Exec /cgibin/* /qsys.lib/cgibin.lib/*.pgm %%EBCDIC%%

Pass /HTML/*

BindSpecific Off

UserID %%SERVER%%

DNS-Lookup Off

Enable GET

Enable HEAD

Enable POST

Enable OPTIONS

Enable TRACE

Disable CONNECT

The Exec directive indicates the path name that you can use in the HTML source code. The path name cgibin is redirected to the AS/400 library system, since that is the only place on the AS/400 that supports executable programs.

You need to use the Enable directives for either or both of the GET and POST methods. If you don’t have Enable directives, the browser will not be able to start CGI programs on the AS/400.

The difference between the GET and POST methods is this:

· HTML forms that use GET send input to the server in the QUERY_STRING environment variable, which has a length limitation of 1024 bytes.

· HTML forms that use POST send input to the server as stream data that is read in the CGI program in input file STDIN. There is no length limit to the stream in STDIN.

This manual shows an example using the POST method.

CGI – First Test

If you have never written a CGI program, you may want to try a simple test before trying to develop a Web-based form. The purpose of the test is to see if your AS/400 HTTP Server is configured correctly for CGI processing.

This example uses a simple HTML form, shown in Figure 3:

CGI03

Figure 3: Sample test CGI form, sends a message to System Operator.

Step 1: Verify that configuration file changes are made and the AS/400 HTTP Server is active

Verify that the changes have been made to the configuration file as described in the HTTP Server Directives section of this manual.

Start or restart the AS/400 HTTP Server.

Step 2: Put the HTML file into IFS directory /HTML

The HTTP server configuration assumes that you will be serving web pages from Integrated File System (IFS) directory /HTML. If you don’t already have that directory, you can use the AS/400 Operations Navigator to create it.

Enter the following HTML source code or upload the code as file CGICL.HTML

<!-------------------------------------------------------------

File CGICL.HTML

Test form for CL-CGI script. Send message to QSYSOPR

to verify configuration for CGI processing.

Craig Pelkie

Bits & Bytes Programming, Inc.

craig@web400.com http://www.web400.com

-------------------------------------------------------------->

<html>

<head>

<title>Test CGICL program</title>

</head>

<body>

<h1 align="center">Tester for CL CGI</h1>

<form action="/cgibin/testcl"

method="Post">

Click the button below, then check the System Operator message queue.

<br>

If the HTTP server is set up correctly, you should have a message there.

<br>

<input type="submit"

value="Test call to CL program">

</form>

</body>

</html>

Step 3: Compile test CL program CGIBIN/TESTCL

This is the test CL program that will be started in response to the browser form

TESTCL: PGM

             SNDMSG     MSG('This is from the CGI-BIN program') +
                          TOUSR(*SYSOPR)
             ENDPGM

Step 4: Open the test HTML file in the browser

Open file CGICL in your browser (it doesn’t matter which browser you use). You should see the simple form shown in Figure 3.

Click the button to test the AS/400 CGI processing.

Step 5: Verify the message in System Operator message queue

If the AS/400 HTTP Server is configured for CGI processing, you should get a message in the System Operator message queue. If you do not see the message, verify that all of the required steps have been completed successfully.

CGI – RPG Form Processor

Now that you’ve verified that CGI processing is enabled on your AS/400, you can move on to the RPG CGI program. This program is more involved than the simple test, since it accepts input from the browser and uses the data for AS/400 processing.

In this test, you will fill in name and address data as shown in Figure 4. When you click the Submit button, the RPG CGI program receives the data and formats the response page shown in Figure 5.

CGI01

Figure 4: The TESTCGI form, with name and address data filled in.

CGI02

Figure 5: The TESTCGI program produces this response page.

The HTML Form

There is a close interaction between the HTML form and the RPG CGI program. To understand how the RPG CGI program processes the data from the HTML form, start by examining the HTML form. You will need to enter the code for the form or upload the sample code to the /HTML directory in the AS/400 IFS.

More about the <FORM> tag and the <INPUT> tags

The <FORM> tag is used to indicate how the form is to be processed when the Submit command button is clicked. The ACTION attribute indicates the name of the directory and the name of the CGI program that is to be executed to process the form. When the Submit button is clicked, the browser sends the form contents to the AS/400 HTTP Server. The HTTP Server looks at the Exec directive in the server configuration file to determine the location of the program. The name of the CGI program is TESTCGI.

Because the form uses the Post method, data entered on the form is passed to the CGI program in the STDIN stream file. The data is formatted with the name of the field (from the INPUT field NAME attributes), followed by an equal sign, followed by the value entered for the field.

<!-------------------------------------------------------------

File TESTCGI.HTML

Test form for RPG-CGI program. Accept input, pass to

AS/400 using POST, call for dynamic response.

Craig Pelkie

Bits & Bytes Programming, Inc.

craig@web400.com http://www.web400.com

-------------------------------------------------------------->

<html>

<head>

<title>Test program TESTCGI - Dynamic Response</title>

</head>

<h1 align="center">Test program TESTCGI - Dynamic Response</h1>

<!-------------------------------------------------------------

FORM tag identifies the name of the CGI program

-------------------------------------------------------------->

<form action="/cgibin/testcgi"

method="Post">

<table border=1

cellpadding=1

bgcolor=wheat>

<!---------------------------------------------------

Table header

----------------------------------------------------

<tr>

<td colspan=2

align=center

bgcolor=navy>

<font size=+1 color=white>Enter Name and Address</font>

</td>

</tr>

<!---------------------------------------------------

Name row

---------------------------------------------------->

<tr>

<td>

Name

</td>

<td>

<input type="text"

size="40"

maxlength="40"

name="DBNAME">

</td>

</tr>

<!---------------------------------------------------

Company row

---------------------------------------------------->

<tr>

<td>

Company

</td>

<td>

<input type="text"

size="40"

maxlength="40"

name="DBCOMP">

</td>

</tr>

<!---------------------------------------------------

Address row

---------------------------------------------------->

<tr>

<td>

Address

</td>

<td>

<input type="text"

size="40"

maxlength="40"

name="DBADDR">

</td>

</tr>

<!---------------------------------------------------

City / State / Zip code row

---------------------------------------------------->

<tr>

<td>

City

</td>

<td>

<input type="text"

size="20"

maxlength="20"

name="DBCITY">

State

<input type="text"

size="2"

maxlength="2"

name="DBSTAT">

ZIP Code

<input type="text"

size="5"

maxlength="5"

name="DBZIP">

</td>

</tr>

<!---------------------------------------------------

Phone row

---------------------------------------------------->

<tr>

<td>

Phone

</td>

<td>

<input type="text"

size="15"

maxlength="15"

name="DBPHON">

</td>

</tr>

<!---------------------------------------------------

Button row

---------------------------------------------------->

<tr>

<td colspan=2

align=center>

<input type="submit"

value="Submit to AS/400">

<input type="reset"

value="Clear">

</td>

</tr>

</table>

</center>

</form>

</body>

</html>

Output from the Browser Form

The browser sends field name / value pairs to the server. The CGI program can receive the data in the STDIN input file. The “raw format” of data sent from the browser to the server is shown in Figure 6.

CGI04

Figure 6: View of the "raw" STDIN data sent from the browser to the server.

To use the data from the browser, the CGI program must parse the string from STDIN and break out the field names and values. Also, the CGI program must translate the “escaped” ASCII characters (the character strings %xx) into equivalent characters, as entered.

Punctuation and other special characters that you might have entered are passed in the format %xx, where xx is two hexadecimal digits that represent the ASCII value of the character. Spaces are replaced with plus signs (+).

For example, using the sample HTML form shown in Figure 4, the first part of the string passed to the CGI program in STDIN is:

DBNAME=Craig+Pelkie&DBCOMP=Bits+%50+Bytes+Programming%6B+Inc.&DBADDR=

The string is made up of these elements

DBNAME – the name of the first field on the form. This is where you type in the Name field. The value DBNAME is from the NAME attribute of the INPUT tag.

Craig+Pelkie – the value entered for the field. Note that the blank between the first and last name is replaced with the + sign.

& -- used to separate the value of the first field from the identifier of the second field.

DBCOMP – the name of the second field on the form. This is where you enter the Company field.

Bits+%50+Bytes+Programming%6B+Inc. – the value entered for the company field. Blanks are again replaced with + signs. The ampersand (&) that was actually typed in the field is replaced with the string %50, which is the hexadecimal value of the ASCII ampersand character. The ampersand needs to be replaced when entered in data, since it is used as the separator character for fields passed in STDIN.

AS/400 APIs for CGI

There are several APIs you can use for CGI programming on the AS/400. These APIs are in Service Program QHTTPSVR/QZHBCGI.

APIs to read STDIN

There are two options you can use to work with the STDIN stream file:

· Use an obsolete set of three APIs (QtmhGetEnv, QtmhRdStin, QtmhCvtDb). When you use those APIs, you need to create a physical file that contains field definitions that correlate to fields used on your HTML form.

· Use the QzhbCgiParse API.

The APIs are:

QtmhGetEnv – get the CONTENT_LENGTH environment variable, which is used to determine the length of the STDIN input stream.

QtmhRdStin – read from STDIN.

QtmhCvtDb – convert data from STDIN using a database file format. This API uses a database file format as a “template” to match with the HTML Input fields.

QzhbCgiParse –read from STDIN and the parse fields into field name / value pairs.

API to write to STDOUT

To return output to the browser, you must write to the STDOUT stream file:

QtmhWrStout – write to STDOUT.

All of these APIs are described in the IBM manual Web Programming Guide V4Rx (GC41-5435). The manual is available for V4R3 and later releases of OS/400.

The QzhbCgiParse API

Starting with OS/400 V4R3, IBM introduced the QzhbCgiParse API for use in CGI programming. This API can be used to take the place of the QtmhGetEnv, QtmhRdStin and QtmhCvtDb APIs in your CGI programs.

The primary advantage of this API is that it has no additional dependencies on external objects, as does the QtmhCvtDb API. For example, if you change your HTML form to include additional form fields, you need to change the database file format used in the QtmhCvtDb API and recompile the program so that the new form fields will be recognized. With the QzhbCgiParse API, you do not need to recompile the program to have access to the additional form fields.

A sample of the QzhbCgiParse API is shown below:

****************************************************************

* Variables for QzhbCgiParse call

* zhbCmd - command string to execute

* zhbFormat - output format; CGII0200=use CGI form var format

* zhbBuffer - target buffer

* zhbBufLen - length of target buffer

* zhbRspLen - length of response

****************************************************************

D zhbCmd s 6

D zhbFormat s 8 inz('CGII0200')

D zhbBuffer s 512

D zhbBufLen s 9b 0 inz(%len(zhbBuffer))

D zhbRspLen s 9b 0 inz(0)

D/COPY QSYSINC/QRPGLESRC,QUSEC

****************************************************************

* Call QzhbCgiParse API, return input from browser

* to field zhbBuffer

****************************************************************

C eval zhbCmd = '-POST' + x’00’

C callb (e) 'QzhbCgiParse'

C parm zhbCmd

C parm zhbFormat

C parm zhbBuffer

C parm zhbBufLen

C parm zhbRspLen

C parm QUSEC

The QzhbCgiParse API uses six parameters:

zhbCmd

A field containing a command to indicate how the API is to parse browser input. The –POST flag indicates that data was supplied from the browser using the METHOD=POST technique. This must be a null-terminated string.

zhbFormat

The format used to parse data from the browser. The value CGII0200 indicates that data is supplied in the standard field name / field value form.

zhbBuffer

A buffer that contains the parsed data from the browser.

zhbBufLen

The length of the buffer that you allocated in your program. If there is more data in STDIN then will fit in the buffer, you can call the API again to continue reading from STDIN.

zhbRspLen

The number of bytes actually used in the buffer.

QUSEC

The standard API error structure. This structure is described in the System API Reference. If an error occurs while using the API, the OS/400 message and additional message data can be retrieved from the error structure.

The QtmhWrStout API

After your RPG CGI program has retrieved the STDIN string and parsed it out, you need to return a response to the browser. Until you return a response, the browser is locked. The response you return can be a dynamically generated web page or a direction to a location where another web page is located.

A sample of the QtmhWrStout API is shown below:

****************************************************************

* Parameter definitions for QtmhWrStout

****************************************************************

D StdOutData s 256

D StdOutLen s 9B 0

****************************************************************

* Call QtmhWrStout to write response HTML to StdOut

****************************************************************