Chapter 8. The Record Model
The Zebra system is designed to support a wide range of data management applications. The system can be configured to handle virtually any kind of structured data. Each record in the system is associated with a record schema which lends context to the data elements of the record. Any number of record schemas can coexist in the system. Although it may be wise to use only a single schema within one database, the system poses no such restrictions.
The record model described in this chapter applies to the fundamental, structured record type grs, introduced in the Section called Record Types in Chapter 5.
Records pass through three different states during processing in the system.
When records are accessed by the system, they are represented in their local, or native format. This might be SGML or HTML files, News or Mail archives, MARC records. If the system doesn't already know how to read the type of data you need to store, you can set up an input filter by preparing conversion rules based on regular expressions and possibly augmented by a flexible scripting language (Tcl). The input filter produces as output an internal representation, a tree structure.
When records are processed by the system, they are represented in a tree-structure, constructed by tagged data elements hanging off a root node. The tagged elements may contain data or yet more tagged elements in a recursive structure. The system performs various actions on this tree structure (indexing, element selection, schema mapping, etc.),
Before transmitting records to the client, they are first converted from the internal structure to a form suitable for exchange over the network - according to the Z39.50 standard.
Local Representation
Four basic subtypes to the grs type are currently available:
Canonical Input Format
To use the canonical format specify grs.sgml as the record type.
<Distributor>
<Name> USGS/WRD </Name>
<Organization> USGS/WRD </Organization>
<Street-Address>
U.S. GEOLOGICAL SURVEY, 505 MARQUETTE, NW
</Street-Address>
<City> ALBUQUERQUE </City>
<State> NM </State>
<Zip-Code> 87102 </Zip-Code>
<Country> USA </Country>
<Telephone> (505) 766-5560 </Telephone>
</Distributor>
|
Record Root
The first tag in a record describes the root node of the tree that makes up the total record. In the canonical input format, the root tag should contain the name of the schema that lends context to the elements of the record (see the Section called Internal Representation). The following is a GILS record that contains only a single element (strictly speaking, that makes it an illegal GILS record, since the GILS profile includes several mandatory elements - Zebra does not validate the contents of a record against the Z39.50 profile, however - it merely attempts to match up elements of a local representation with the given schema):
<gils>
<title>Zen and the Art of Motorcycle Maintenance</title>
</gils>
|
Variants
The following is an example of a title element which occurs in two different languages.
<title>
<var lang lang "eng">
Zen and the Art of Motorcycle Maintenance</>
<var lang lang "dan">
Zen og Kunsten at Vedligeholde en Motorcykel</>
</title>
|
The syntax of the variant element is <var class type value>. The available values for the class and type fields are given by the variant set that is associated with the current schema (see the Section called The Variant Set (.var) Files).
Variant elements are terminated by the general end-tag </>, by the variant end-tag </var>, by the appearance of another variant tag with the same class and value settings, or by the appearance of another, normal tag. In other words, the end-tags for the variants used in the example above could have been omitted.
Variant elements can be nested. The element
<title>
<var lang lang "eng"><var body iana "text/plain">
Zen and the Art of Motorcycle Maintenance
</title>
|
Associates two variant components to the variant list for the title element.
Given the nesting rules described above, we could write
<title>
<var body iana "text/plain>
<var lang lang "eng">
Zen and the Art of Motorcycle Maintenance
<var lang lang "dan">
Zen og Kunsten at Vedligeholde en Motorcykel
</title>
|
The title element above comes in two variants. Both have the IANA body type "text/plain", but one is in English, and the other in Danish. The client, using the element selection mechanism of Z39.50, can retrieve information about the available variant forms of data elements, or it can select specific variants based on the requirements of the end-user.
Input Filters
An expression can be either of the following:
An action is surrounded by curly braces ({...}), and consists of a sequence of statements. Statements may be separated by newlines or semicolons (;). Within actions, the strings that matched the expressions immediately preceding the action can be referred to as $0, $1, $2, etc.
The available statements are:
- begin type [parameter ... ]
Begin a new data element. The type is one of the following:
- record
Begin a new record. The following parameter should be the name of the schema that describes the structure of the record, eg. gils or wais (see below). The begin record call should precede any other use of the begin statement.
- element
Begin a new tagged element. The parameter is the name of the tag. If the tag is not matched anywhere in the tagsets referenced by the current schema, it is treated as a local string tag.
- variant
Begin a new node in a variant tree. The parameters are class type value.
- data
Create a data element. The concatenated arguments make up the value of the data element. The option -text signals that the layout (whitespace) of the data should be retained for transmission. The option -element tag wraps the data up in the tag. The use of the -element option is equivalent to preceding the command with a begin element command, and following it with the end command.
- end [type]
Close a tagged element. If no parameter is given, the last element on the stack is terminated. The first parameter, if any, is a type name, similar to the begin statement. For the element type, a tag name can be provided to terminate a specific tag.
The following input filter reads a Usenet news file, producing a record in the WAIS schema. Note that the body of a news posting is separated from the list of headers by a blank line (or rather a sequence of two newline characters.
BEGIN { begin record wais }
/^From:/ BODY /$/ { data -element name $1 }
/^Subject:/ BODY /$/ { data -element title $1 }
/^Date:/ BODY /$/ { data -element lastModified $1 }
/\n\n/ BODY END {
begin element bodyOfDisplay
begin variant body iana "text/plain"
data -text $1
end record
}
|
If Zebra is compiled with support for Tcl (Tool Command Language) enabled, the statements described above are supplemented with a complete scripting environment, including control structures (conditional expressions and loop constructs), and powerful string manipulation mechanisms for modifying the elements of a record. Tcl is a popular scripting environment, with several tutorials available both online and in hardcopy.