CGI script for dynamic web forms using flat file databases, server-side only
The data structure handled by this script is as follows. A list of unique names, each with optional attributes (text fields), is the backbone of the entire structure, called the index or base. Each of these index names may have exactly one entry in each of a number of pages, while all entries in a given page have the same structure of additional fields.
(In principle, the same functionality can be provided by one single big list with exactly one entry per index name, but it would be rather impractical to handle and process.)
An example of this structure may be an index of user names (or personnel numbers), with attributes like full name, address, telephone numbers and e-mail address, and a collection of lists (pages) with exactly one entry per person (or none at all) in each list. The lists might contain data like access rights to various equipment, subscription data to mailing lists, number of hours worked, etc.
Another example would be an index of computers with attributes like system responsibles and location or use, and lists with patch information, available periphery/accessories, or running costs.
The script must be installed in a directory where CGI scripts can
be executed. It must be called (e.g, by some HTML link) with at
least the variable
db set, like
Calling it without any
db value will generate a fatal error unless
the default configuration file
defcfg.cfg exists in the path of
the script. Of course this can be modified in the script, if the
database should be hardcoded.
To work properly, at least the configuration file (
the above example) and the pages referred by it must exist and be
readable and writable for the script. A minimal installation
therefore consists in the script itself, a configuration file
*.cfg , an index/base file, and one page file.
There is one base or index file, containing one unique index number/name per line, with one or more descriptive fields. This index is used as reference in additional files, which again contain a field for the index, and an arbitrary number of record fields. Each of these files are rendered as HTML tables by the script.
All lines start with a flag character, which is one of the set
#+-* (possibly more in future versions). All fields are separated
TAB characters which therefore are forbidden as content of any
#(comments) or any unrecognized character are completely ignored, as well as empty lines.
*is the table header; any additional line starting with
-are database entries, and their fields after
-contain the index.
Index fields must be unique, and this is enforced by the script: any duplicate entry may be overwritten and only one retained.
Normally, entry lines start with the
+ flag character. This
renders them as available or "shown."
Index entries with leading
- are not available for edition or
creation of page entries, i.e, any entry with such an index name
cannot be modified and is "hidden." When an index entry is to be saved, the
show/hide operation can also be applied to all pages, i.e the
corresponding page records are simultaneously shown/hidden.
Header fields with the structure
list=listname will result in the
corresponding field being a selection field, with options coming from
all lines of the file
listname beginning with '+' (only the part after
TAB is used). This allows to predefine a limited number of possible
entries for certain fields.
Header fields with
xlist=... instead of
list=... allow for
evaluation of selection fields, which may be a security risk.
(This feature is deactivated by default, and must be activated by
setting the appropriate flag at the beginning of the source code.)
See the example further below for the syntax of list file entries!
Header fields with the structure
will result in the corresponding field being a selection field, with
options empty, old value, or current time (in minute resolution, for
now= header) or day (for
day= header). In case of
the time will be followed by
NNNN is the epoch time
(in general since Jan.1st 1970 UTC) in minutes; this can be used for
logbook applications to calculate time differences.
now= fields may also be of the form
1,2,3 set additional options with "delta times", i.e the current time
plus 1, 2 or 3 minutes, e.g
now,30,60=end defines a selection field
with additional options "current time plus 30 min" and "current time
plus 60 min".
Page entries with leading
- normally are not displayed when the
page is rendered, but they can be "un-hidden," and also be edited.
Please note that any existing entry (index or page data) may be overwritten without warning, if the user has appropriate permissions.
Please note the TAB characters always separating fields!
# names and email addresses * PersNr Family name Name E-Mail list=color + 1001 Family01 One firstname.lastname@example.org red + 1003 Family03 Three email@example.com green + 1005 Family05 Four firstname.lastname@example.org blue + 1008 Family08 Eight email@example.com black + 1006 Family06 Six firstname.lastname@example.org yellow + 1004 Family04 Four email@example.com lilac + 1007 Family07 Seven firstname.lastname@example.org amber - 2001 Family9000 HAL email@example.com gold + 1002 Family02 TwoTwo firstname.lastname@example.org cyan
# entrance and leave database * PersNr day=Arrival date day=Departure date list=divis + 1008 sooner later HR + 1006 2015-01-22 2015-12-1 IT + 1007 May 1984 present IT - 1002 2015-01-22 2015-12-19 mgmt + 1003 June 2014 November 2015 finances + 1004 June 2008 August 2015 production + 1005 beginning end production + 1001 April 1948 2001 HR
In this case, the index entry 2001 would not be available in the record tables, and the entry for index 1002 would not be displayed, when the record file is rendered.
# favourites * color + green + blue + cyan + gold + black 5 + lilac + amber
black is only allowed for up to 5 times in a page.
The other selections have no limits.
# evaluate * varia + blue + system = $DEFCOLOR
system will be displayed as the content of the variable
$DEFCOLOR which might be defined in the environment.
Effectively, if the third column (not counting the '+') of a selection entry is not empty, it will be evaluated by the webforms script, and in this case, the first column is an irrelevant placeholder. This allows for dynamic content, e.g a list with entries
+ -- + now = $field + now = $nowstring=$nowminutes
will have the same functionality as a column with a
$nowminutes are internals of the script),
and the entry
+ host = `hostname`
will evaluate to a selection of the current hostname (if this command is available to the shell).
WARNING: Be careful with this powerful type of list, there might be side effects affecting the script or environment! Any user who can control such entries in the list file can execute arbitrary shell commands through the script!
Each collection of webforms is defined by a configuration file with
.cfg , which is indicated to the CGI script by a GET
variable and a hardcoded directory, or completely hardcoded for
This file contains the name of the base/index file (indicated by
base field), and the names of the (one or more) page
files (indicated by
page/upag/ulog). Each file name must be prepended with
the page name, and may be followed by a description, which will be
included in the rendered page headers.
The page name for the index file must be set to
file for correct syntax.
Pages with type
upag instead of
page allow for user-dependant entries:
Unless the user has editor permission or higher (see below for permissions),
they can only access page entries with an index field corresponding to
their own user name. This can be used to let users submit entries from a
given number of options, or handle their self-supplied data.
Pages with type
ulog instead of
page allow for user-dependant
entries with timestamps: Indices will be generated as username followed
_ and a number-only timestamp, and non-admin users can
only generate entries with their username as index prefix.
This can be used for logbook entries of different users.
List files used for populating selection fields are indicated by
xlist for selections allowing dynamic evaluation)
followed by their reference name, file name, and optionally description.
File names can be absolute or relative; the starting directory is the working directory of the CGI script.
By setting the entry
nopageindex to something else than
displaying of the column for the index/base string can be suppressed.
This column normally is shown as the first one in page views,
but it may be of little use in case of numeric or random-like values.
With the field
emptywarn a string can be defined which will be displayed
in place of empty fields. This can be any valid HTML code (see example).
showindex can be used to indicate additional fields from the
index/base file to be displayed in pages. The field positions correspond
to the header fields of the base file, and any
showindex field other than
- will display the corresponding base entry field with its name as header,
c.f example configuration.
The value after
maxfieldlength will limit form entry fields to the given
character length; however, any length will displayed when database contents
fieldchars the permitted characters in entry fields can be listed.
By default, all printable ASCII characters are allowed.
TAB is always
excluded from all fields, though.
Permission levels for user names (passed via
REMOTE_USER from the webserver)
can be set with field names
visitor line is present, then only users explicitly listed
on any of the permission lines are allowed to access the script.
Otherwise, all users not listed as
editor are allowed
visitor access (i.e, read-only access to all pages).
editor lines contain the wildcard
* then any user will get
the corresponding permissions. The highest level available will be applied.
E.g, an entry of
admin * will grant admin permissions to all users,
even if they are listed in
User names are sanitized: only characters from the set '0-9A-Za-z.-' are allowed, and entries in the configuration file must conform to this.
Logging can be switched on by setting the field
log with page name 'file'
and the file name. If the file is not writable, no logging will occur, without
# test config file # file definitions # base file relative/path/to/basefile.txt basefile description base file test.base People page dates test.dates Dates # in 'phone' page, users below admin level only see their own entries upag phone test.phone Phone numbers list color test.color favourite color xlist varia test.eval various dynamic values list divis test.div company division # uncomment to suppress displaying index/base field in page view #nopageindex true # user permissions: admin chief nobody editor deputy # if visitor line is defined, only allow explicitly listed users visitor dings # display definitions: # additional index field names to be shown in pageviews: # skip fields with a single dash '-', never use '|' in these names! # here, the 3rd and 4th index/base entry fields are used, and the # 1st and 2nd are skipped showindex - - Email Fav.color # how to warn about empty fields (can be undefined) emptywarn <font color="red"><b>#EMPTY#</b></font> # reduce the maximum size of text fields in record input form maxfieldlength 80 # logfile (no logging if unwritable!) log file test.log # pattern of allowed characters in fields, # default = all ASCII characters from SPC to ~ (tilde) #fieldchars ' -~'
The script renders various pages, based on CGI environment variables:
vwview (display selection)
faflag for hiding/showing entries
fNfield number N
This variable value selects the view or command.
vw=listpageslist all available pages
vw=listindexlist all indices
vw=descindexdetails of index entry
vw=editindexedit index entry
vw=saveindexsave index entry
vw=editentryedit page entry/record
vw=saveentrysave page entry/record
By default, version control is done in a very simple way: For each database
file, an old version with the suffix
.old is saved, and the result from
diff -e $new $old is appended to a file with the suffix
(For this to work, the script of course must be able to write to these files.)
In principle, from this any old version can be reconstructed, but currently
there is no automatic way provided.
Version control can also be done with RCS or Git.
However, for this to work, the function
must be modified; please read the source!
If defined, the logfile contains information about all runs of the script, including user and remote host information — be careful about privacy issues!
vw=saveentrybut not for the corresponding edit commands.
vw=editindexthe script tries to generate a new and unique index number, if numerical index values are encountered.
2019-5-1 // Y.Bonetti