A Gentle Introduction to SGML

The encoding scheme defined by these Guidelines is formulated as an application of a system known as the Standard Generalized Markup Language (SGML). International Organization for Standardization, ISO 8879: Information processing---Text and office systems---Standard Generalized Markup Language (SGML), ([Geneva]: ISO, 1986). SGML is an international standard for the definition of device-independent, system-independent methods of representing texts in electronic form. This chapter presents a brief tutorial guide to its main features, for those readers who have not encountered it before. For a more technical account of TEI practice in using the SGML standard, see chapter ; for a more technical description of the subset of SGML used by the TEI encoding scheme, see chapter .

SGML is an international standard for the description of marked-up electronic text. More exactly, SGML is a metalanguage, that is, a means of formally describing a language, in this case, a markup language. Before going any further we should define these terms.

Historically, the word markup has been used to describe annotation or other marks within a text intended to instruct a compositor or typist how a particular passage should be printed or laid out. Examples include wavy underlining to indicate boldface, special symbols for passages to be omitted or printed in a particular font and so forth. As the formatting and printing of texts was automated, the term was extended to cover all sorts of special markup codes inserted into electronic texts to govern formatting, printing, or other processing.

Generalizing from that sense, we define markup, or (synonymously) encoding, as any means of making explicit an interpretation of a text. At a banal level, all printed texts are encoded in this sense: punctuation marks, use of capitalization, disposition of letters around the page, even the spaces between words, might be regarded as a kind of markup, the function of which is to help the human reader determine where one word ends and another begins, or how to identify gross structural features such as headings or simple syntactic units such as dependent clauses or sentences. Encoding a text for computer processing is in principle, like transcribing a manuscript from scriptio continua, a process of making explicit what is conjectural or implicit, a process of directing the user as to how the content of the text should be interpreted.

By markup language we mean a set of markup conventions used together for encoding texts. A markup language must specify what markup is allowed, what markup is required, how markup is to be distinguished from text, and what the markup means. SGML provides the means for doing the first three; documentation such as these Guidelines is required for the last.

The present chapter attempts to give an informal introduction---much less formal than the standard itself---to those parts of SGML of which a proper understanding is necessary to make best use of these Guidelines.

What's Special about SGML?

There are three characteristics of SGML which distinguish it from other markup languages: its emphasis on descriptive rather than procedural markup; its document type concept; and its independence of any one system for representing the script in which a text is written. These three aspects are discussed briefly below, and then in more depth in sections and .

Descriptive Markup

A descriptive markup system uses markup codes which simply provide names to categorize parts of a document. Markup codes such as para or \end{list} simply identify a portion of a document and assert of it that the following item is a paragraph, or this is the end of the most recently begun list, etc. By contrast, a procedural markup system defines what processing is to be carried out at particular points in a document: call procedure PARA with parameters 1, b and x here or move the left margin 2 quads left, move the right margin 2 quads right, skip down one line, and go to the new left margin, etc. In SGML, the instructions needed to process a document for some particular purpose (for example, to format it) are sharply distinguished from the descriptive markup which occurs within the document. Usually, they are collected outside the document in separate procedures or programs.

With descriptive instead of procedural markup the same document can readily be processed by many different pieces of software, each of which can apply different processing instructions to those parts of it which are considered relevant. For example, a content analysis program might disregard entirely the footnotes embedded in an annotated text, while a formatting program might extract and collect them all together for printing at the end of each chapter. Different sorts of processing instructions can be associated with the same parts of the file. For example, one program might extract names of persons and places from a document to create an index or database, while another, operating on the same text, might print names of persons and places in a distinctive typeface.

 Types of Document

Secondly, SGML introduces the notion of a document type, and hence a document type definition (DTD). Documents are regarded as having types, just as other objects processed by computers do. The type of a document is formally defined by its constituent parts and their structure. The definition of a report, for example, might be that it consisted of a title and possibly an author, followed by an abstract and a sequence of one or more paragraphs. Anything lacking a title, according to this formal definition, would not formally be a report, and neither would a sequence of paragraphs followed by an abstract, whatever other report-like characteristics these might have for the human reader.

If documents are of known types, a special purpose program (called a parser) can be used to process a document claiming to be of a particular type and check that all the elements required for that document type are indeed present and correctly ordered. More significantly, different documents of the same type can be processed in a uniform way. Programs can be written which take advantage of the knowledge encapsulated in the document structure information, and which can thus behave in a more intelligent fashion.

 Data Independence

A basic design goal of SGML was to ensure that documents encoded according to its provisions should be transportable from one hardware and software environment to another without loss of information. The two features discussed so far both address this requirement at an abstract level; the third feature addresses it at the level of the strings of bytes (characters) of which documents are composed. SGML provides a general purpose mechanism for string substitution, that is, a simple machine-independent way of stating that a particular string of characters in the document should be replaced by some other string when the document is processed. One obvious application for this mechanism is to ensure consistency of nomenclature; another, more significant one, is to counter the notorious inability of different computer systems to understand each other's character sets, or of any one system to provide all the graphic characters needed for a particular application, by providing descriptive mappings for non-portable characters. The strings defined by this string-substitution mechanism are called entities and they are discussed below in section .

 Textual Structure

A text is not an undifferentiated sequence of words, much less of bytes. For different purposes, it may be divided into many different units, of different types or sizes. A prose text such as this one might be divided into sections, chapters, paragraphs, and sentences. A verse text might be divided into cantos, stanzas, and lines. Once printed, sequences of prose and verse might be divided into volumes, gatherings, and pages.

Structural units of this kind are most often used to identify specific locations or reference points within a text (the third sentence of the second paragraph in chapter ten; canto 10, line 1234; page 412, etc.) but they may also be used to subdivide a text into meaningful fragments for analytic purposes (is the average sentence length of section 2 different from that of section 5? how many paragraphs separate each occurrence of the word nature? how many pages?). Other structural units are more clearly analytic, in that they characterize a section of a text. A dramatic text might regard each speech by a different character as a unit of one kind, and stage directions or pieces of action as units of another kind. Such an analysis is less useful for locating parts of the text (the 93rd speech by Horatio in Act 2) than for facilitating comparisons between the words used by one character and those of another, or those used by the same character at different points of the play.

In a prose text one might similarly wish to regard as units of different types passages in direct or indirect speech, passages employing different stylistic registers (narrative, polemic, commentary, argument, etc.), passages of different authorship and so forth. And for certain types of analysis (most notably textual criticism) the physical appearance of one particular printed or manuscript source may be of importance: paradoxically, one may wish to use descriptive markup to describe presentational features such as typeface, line breaks, use of white space and so forth.

These textual structures overlap with each other in complex and unpredictable ways. Particularly when dealing with texts as instantiated by paper technology, the reader needs to be aware of both the physical organization of the book and the logical structure of the work it contains. Many great works (Sterne's Tristram Shandy for example) cannot be fully appreciated without an awareness of the interplay between narrative units (such as chapters or paragraphs) and page divisions. For many types of research, it is the interplay between different levels of analysis which is crucial: the extent to which syntactic structure and narrative structure mesh, or fail to mesh, for example, or the extent to which phonological structures reflect morphology.

 SGML Structures

This section describes the simple and consistent mechanism for the markup or identification of structural textual units which is provided by SGML. It also describes the methods SGML provides for the expression of rules defining how combinations of such units can meaningfully occur in any text.

Elements

The technical term used in the SGML standard for a textual unit, viewed as a structural component, is element. Different types of elements are given different names, but SGML provides no way of expressing the meaning of a particular type of element, other than its relationship to other element types. That is, all one can say about an element called (for instance) blort is that instances of it may (or may not) occur within elements of type farble, and that it may (or may not) be decomposed into elements of type blortette. It should be stressed that the SGML standard is entirely unconcerned with the semantics of textual elements: these are application dependent.Work is currently going on in the standards community to create (using SGML syntax) a definition of a standard document style semantics and specification language or DSSSL. It is up to the creators of SGML conformant tag sets (such as these Guidelines) to choose intelligible names for the elements they identify and to document their proper use in text markup. That is one purpose of this document. From the need to choose element names indicative of function comes the technical term for the name of an element type, which is generic identifier, or GI.

Within a marked up text (a document instance), each element must be explicitly marked or tagged in some way. The standard provides for a variety of different ways of doing this, the most commonly used being to insert a tag at the beginning of the element (a start-tag) and another at its end (an end-tag). The start- and end-tag pair are used to bracket off the element occurrences within the running text, in rather the same way as different types of parentheses or quotation marks are used in conventional punctuation. For example, a quotation element in a text might be tagged as follows: This is the silliest stuff that ere I heard of! clearly indicate ... ]]> As this example shows, a start-tag takes the form name, where the opening angle bracket indicates the start of the start-tag, name is the generic identifier of the element which is being delimited, and the closing angle bracket indicates the end of a tag. An end-tag takes an identical form, except that the opening angle bracket is followed by a solidus (slash) character, so that the corresponding end-tag would be /name. The actual characters used for the delimiting characters (the angle brackets, exclamation mark and solidus) may be redefined, but it is conventional to use the characters used in this description.

Content Models: An Example

An element may be empty, that is, it may have no content at all, or it may contain simple text. More usually, however, elements of one type will be embedded (contained entirely) within elements of a different type.

To illustrate this, we will consider a very simple structural model. Let us assume that we wish to identify within an anthology only poems, their titles, and the stanzas and lines of which they are composed. In SGML terms, our document type is the anthology, and it consists of a series of poems. Each poem has embedded within it one element, a title, and several occurrences of another, a stanza, each stanza having embedded within it a number of line elements. Fully marked up, a text conforming to this model might appear as follows: The example is taken from William Blake's Songs of innocence and experience (1794). The markup is designed for illustrative purposes and is not TEI-conformant. The SICK ROSE O Rose thou art sick. The invisible worm, That flies in the night In the howling storm: Has found out thy bed Of crimson joy: And his dark secret love Does thy life destroy. ]]>

It should be stressed that this example does not use the same names as are proposed for corresponding elements elsewhere in these Guidelines: the above is not a valid TEI document. It will however serve as an introduction to the basic notions of SGML. White space and line breaks have been added to the example for the sake of visual clarity only; they have no particular significance in the SGML encoding itself. Also, the line ]]> is an SGML comment and is not treated as part of the text.

This example makes no assumptions about the rules governing, for example, whether or not a title can appear in places other than preceding the first stanza, or whether lines can appear which are not included in a stanza: that is why its markup appears so verbose. In such cases, the beginning and end of every element must be explicitly marked, because there are no identifiable rules about which elements can appear where. In practice, however, rules can usually be formulated to reduce the need for so much tagging. For example, considering our greatly over-simplified model of a poem, we could state the following rules: An anthology contains a number of poems and nothing else. A poem always has a single title element which precedes the first stanza and contains no other elements. Apart from the title, a poem consists only of stanzas. Stanzas consist only of lines and every line is contained by a stanza. Nothing can follow a stanza except another stanza or the end of a poem. Nothing can follow a line except another line or the start of a new stanza.

From these rules, it may be inferred that we do not need to mark the ends of stanzas or lines explicitly. From rule 2 it follows that we do not need to mark the end of the title---it is implied by the start of the first stanza. Similarly, from rules 3 and 1 it follows that we need not mark the end of the poem: since poems cannot occur within poems but must occur within anthologies, the end of a poem is implied by the start of the next poem, or by the end of the anthology. Applying these simplifications, we could mark up the same poem as follows: The SICK ROSE <stanza> <line>O Rose thou art sick. <line>The invisible worm, <line>That flies in the night <line>In the howling storm: <stanza> <line>Has found out thy bed <line>Of crimson joy: <line>And his dark secret love <line>Does thy life destroy. <poem> <!-- more poems go here --> </anthology> ]]> </eg></p> <p>The ability to use rules stating which elements can be nested within others to simplify markup is a very important characteristic of SGML. Before considering these rules <!-- further, you may like to consider how text marked up in the --> further, you may wish to consider how text marked up in the form above could be processed by a computer for very many different purposes. A simple indexing program could extract only the relevant text elements in order to make a list of titles, or of words used in the poem text; a simple formatting program could insert blank lines between stanzas, perhaps indenting the first line of each, or inserting a stanza number. Different parts of each poem could be typeset in different ways. A more ambitious analytic program could <!-- determine how many stanzas or lines begin with lower-case --> <!-- letters and thus (perhaps) in mid-sentence. --> relate the use of punctuation marks to stanzaic and metrical divisions. <note> Note that this simple example has not addressed the problem of marking elements such as sentences explicitly; the implications of this are discussed below in section <ptr target="SG152">. </note> Scholars wishing to see the implications of changing the stanza or line divisions chosen by the editor of this poem can do so simply by altering the position of the tags. And of course, the text as presented above can be transported from one computer to another and processed by any program (or person) capable of making sense of the tags embedded within it with no need for the sort of transformations and translations needed to move word processor files around.</p></div3></div2> <div2 id="SG14"><head>Defining SGML Document Structures: The DTD</head> <p>Rules such as those described above are the first stage in the creation of a formal specification for the structure of an SGML document, or <term>document type definition</term>, usually abbreviated to <term>DTD</term>. In creating a DTD, the document designer may be as lax or as restrictive as the occasion warrants. A balance must be struck between the convenience of following simple rules and the complexity of handling real texts. This is particularly the case when the rules being defined relate to texts which already exist: the designer may have only the haziest of notions as to an ancient text's original purpose or meaning and hence find it very difficult to specify consistent rules about its structure. On the other hand, where a new text is being prepared to an exact specification, for example for entry into a textual database of some kind, the more precisely stated the rules, the better they can be enforced. Even in the case where an existing text is being marked up, it may be beneficial to define a restrictive set of rules relating to one particular view or hypothesis about the text---if only as a means of testing the usefulness of that view or hypothesis. It is important to remember that every document type definition is an interpretation of a text. There is no single DTD which encompasses any kind of absolute truth about a text, although it may be convenient to privilege some DTDs above others for particular types of analysis.</p> <p>At present, SGML is most widely used in environments where uniformity of document structure is a major desideratum. In the production of technical documentation, for example, it is of major importance that sections and subsections should be properly nested, that cross references should be properly resolved and so forth. In such situations, documents are seen as raw material to match against pre-defined sets of rules. As discussed above, however, the use of simple rules can also greatly simplify the task of tagging accurately elements of less rigidly constrained texts. By making these rules explicit, the scholar reduces his or her own burdens in marking up and verifying the electronic text, while also being forced to make explicit an interpretation of the structure and significant particularities of the text being encoded.</p> <div3 id="SG141"><head>An Example DTD</head> <p>A DTD is expressed in SGML as a set of declarative statements, using a simple syntax defined in the standard. For our simple model of a poem, the following declarations would be appropriate: <eg> <![ CDATA [ <!ELEMENT anthology - - (poem+)> <!ELEMENT poem - - (title?, stanza+)> <!ELEMENT title - O (#PCDATA) > <!ELEMENT stanza - O (line+) > <!ELEMENT line O O (#PCDATA) > ]]> </eg> These five lines are examples of formal SGML element declarations. A declaration, like an element, is delimited by angle brackets; the first character following the opening bracket must be an exclamation mark, followed immediately by one of a small set of SGML-defined keywords, specifying the kind of object being declared. The five declarations above are all of the same type: each begins with an <code type="kw">ELEMENT</code> keyword, indicating that it declares an element, in the technical sense defined above. Each consists of three parts: a name or group of names, two characters specifying <term>minimization rules</term>, and a <term>content model.</term> Each of these parts is discussed further below. Components of the declaration are separated by white space, that is one or more blanks, tabs or newlines.</p> <p>The first part of each declaration above gives the generic identifier of the element which is being declared, for example <mentioned>poem</mentioned>, <mentioned>title</mentioned>, etc. It is possible to declare several elements in one statement, as discussed below.</p></div3> <div3 id="SG142"><head>Minimization Rules</head> <p>The second part of the declaration specifies what are called <term>minimization rules</term> for the element concerned. These rules determine whether or not start- and end-tags must be present in every occurrence of the element concerned. They take the form of a pair of characters, separated by white space, the first of which relates to the start-tag, and the second to the end-tag. In either case, either a hyphen or a letter O (for <q>omissible</q> or <q>optional</q>) must be given; the hyphen indicating that the tag must be present, and the letter O that it may be omitted. Thus, in this example, every element except <gi tei="no">line</gi> must have a start-tag. Only the <gi tei="no">poem</gi> and <gi tei="no">anthology</gi> elements must have end-tags as well. <!-- If no minimization rules are given for an element, --> <!-- then neither start nor end-tags may be omitted. --></p></div3> <div3 id="SG143"><head>Content Model</head> <p>The third part of each declaration, enclosed in parentheses, is called the <term>content model</term> of the element, because it specifies what element occurrences may legitimately contain. Contents are specified either in terms of other elements or using special reserved words. There are several such reserved words, of which by far the most commonly encountered is #PCDATA, as in this example. This is an abbreviation for <gloss>parsed character data,</gloss> and it means that the element being defined may contain any valid character data. If an SGML declaration is thought of as a structure like a family tree, with a single ancestor at the top (in our case, this would be <tag tei="no">anthology</tag>), then almost always, following the branches of the tree downwards (for example, from <gi tei="no">anthology</gi> to <gi tei="no">poem</gi> to <gi tei="no">stanza</gi> to <gi tei="no">line</gi> and <gi tei="no">title</gi>) will lead eventually to #PCDATA. In our example, <gi tei="no">title</gi> and <gi tei="no">line</gi> are so defined. Since their content models say #PCDATA only and name no embedded elements, they may not contain any embedded elements.</p></div3> <div3 id="SG144"><head>Occurrence Indicators</head> <p>The declaration for <gi tei="no">stanza</gi> in the example above states that a stanza consists of one or more lines. It uses an <term>occurrence indicator</term> (the plus sign) to indicate how many times the element named in its content model may occur. There are three occurrence indicators in the SGML syntax, conventionally represented by the plus sign, the question mark, and the asterisk or star. <note> <!-- Added footnote. -msm --> Like the delimiters, these are assigned formal names by the standard and may be redefined with an appropriate SGML declaration. </note> The plus sign means that there may be one or more occurrences of the element concerned; the question mark means that there may be at most one and possibly no occurrence; the star means that the element concerned may either be absent or appear one or more times. Thus, if the content model for <gi tei="no">stanza</gi> were <code type="frag">(LINE*)</code>, stanzas with no lines would be possible as well as those with more than one line. If it were <code type="frag">(LINE?)</code>, again empty stanzas would be countenanced, but no stanza could have more than a single line. The declaration for <gi tei="no">poem</gi> in the example above thus states that a <gi tei="no">poem</gi> cannot have more than one title, but may have none, and that it must have at least one <gi tei="no">stanza</gi> and may have several. <!-- It uses an --> <!-- occurrence indicator (the plus sign) to --> <!-- indicate how many times the element named in its --> <!-- content model may occur. There are three occurrence --> <!-- indicators, known in the standard as --> <!-- X plus , opt and --> <!-- m>rep rm>. Plus, which is usually represented --> <!-- by a plus sign, means that there may be one or more --> <!-- occurrences of the element concerned; opt, usually --> <!-- represented by a question mark, means that there may --> <!-- be at most one and possibly no occurrence; rep, --> <!-- usually represented by a star, means that the element --> <!-- concerned may either be absent or appear one or more --> <!-- times. Thus, if the content model for STANZA were --> <!-- (LINE*), stanzas with no lines would be possible as --> <!-- well as those with more than one line. If it were --> <!-- (LINE?), again empty stanzas would be countenanced, --> <!-- but no stanza could have more than a single line. --></p></div3> <div3 id="SG145"><head>Group Connectors</head> <p>The content model <code type="frag">(TITLE?, STANZA+)</code> contains more than one component, and thus needs additionally to specify the order in which these elements (<gi tei="no">title</gi> and <gi tei="no">stanza</gi>) may appear. This ordering is determined by the <term>group connector</term> (the comma) used between its components. There are three possible group connectors, conventionally represented by comma, vertical bar, and ampersand. <note> <!-- Added footnote. -msm --> What are here called <q>group connectors</q> are referred to by the SGML standard simply as <q>connectors</q>; the longer term is preferred here to stress the fact that these connectors are used only in SGML model groups and name groups. Like the delimiters and the occurrence indicators, group connectors are assigned formal names by the standard and may be redefined with an appropriate SGML declaration. </note> The comma means that the components it connects must both appear in the order specified by the content model. The ampersand indicates that the components it connects must both appear but may appear in any order. The vertical bar indicates that only one of the components it connects may appear. If the comma in this example were replaced by an ampersand, a title could appear either before the stanzas of a <gi tei="no">poem</gi> or at the end (but not between stanzas). If it were replaced by a vertical bar, then a <gi tei="no">poem</gi> would consist of either a title or just stanzas---but not both!</p></div3> <div3 id="SG146"><head>Model Groups</head> <p>In our example so far, the components of each content model have been either single elements or #PCDATA. It is quite permissible however to define content models in which the components are lists of elements, combined by group connectors. Such lists, known as <term>model groups</term>, may also be modified by occurrence indicators and themselves combined by group connectors. To demonstrate these facilities, let us now expand our example to include non-stanzaic types of verse. For the sake of demonstration, we will categorize poems as one of <term>stanzaic</term>, <term>couplets</term>, or <term>blank</term> (or <term>stichic</term>). A blank-verse poem consists simply of lines (we ignore the possibility of verse paragraphs for the moment) <note> It will not have escaped the astute reader that the fact that verse paragraphs need not start on a line boundary seriously complicates the issue; see further section <ptr target="SG152">. </note> so no additional elements need be defined for it. A couplet is defined as a <gi tei="no">line1</gi> followed by a <gi tei="no">line2</gi>. <eg><![ CDATA [ <!ELEMENT couplet O O (line1, line2) > ]]> </eg> <!-- Can't believe it's a good idea to use as an example --> <!-- something we would never do ourselves. --> <!-- Why not define SONNET, OCTET, QUATRAIN, SESTET? --></p> <p>The elements <gi tei="no">line1</gi> and <gi tei="no">line2</gi> (which are distinguished to enable studies of rhyme scheme, for example) have exactly the same content model as the existing <gi tei="no">line</gi> element. They can therefore share the same declaration. In this situation, it is convenient to supply a <term>name group</term> as the first component of a single element declaration, rather than give a series of declarations differing only in the names used. A name group is a list of GIs connected by any group connector and enclosed in parentheses, as follows: <eg> <![ CDATA [ <!ELEMENT (line | line1 | line2) O O (#PCDATA) > ]]> </eg> The declaration for the <gi tei="no">poem</gi> element can now be changed to include all three possibilities: <eg> <![ CDATA [ <!ELEMENT poem - O (title?, (stanza+ | couplet+ | line+) ) > ]]> </eg> That is, a poem consists of an optional title, followed by one or several stanzas, or one or several couplets, or one or several lines. Note the difference between this definition and the following: <eg> <![ CDATA [ <!ELEMENT poem - O (title?, (stanza | couplet | line)+ ) > ]]> </eg> The second version, by applying the occurrence indicator to the group rather than to each element within it, would allow for a single poem to contain a mixture of stanzas, couplets or blank verse.</p> <p>Quite complex models can easily be built up in this way, to match the structural complexity of many types of text. As a further example, consider the case of stanzaic verse in which a refrain or chorus appears. A refrain may be composed of repetitions of the line element, or it may simply be text, not divided into verse lines. A refrain can appear at the start of a poem only, or as an optional addition following each stanza. This could be expressed by a content model such as the following: <eg> <![ CDATA [ <!ELEMENT refrain - - (#PCDATA | line+)> <!ELEMENT poem - O (title?, ( (line+) | (refrain?, (stanza, refrain?)+ ) )) > ]]> </eg> That is, a poem consists of an optional title, followed by either a sequence of lines, or an un-named group, which starts with an optional refrain, followed by one of more occurrences of another group, each member of which is composed of a stanza followed by an optional refrain. A sequence such as <mentioned>refrain - stanza - stanza - refrain</mentioned> follows this pattern, as does the sequence <mentioned>stanza - refrain - stanza - refrain</mentioned>. The sequence <mentioned>refrain - refrain - stanza - stanza</mentioned> does not, however, and neither does the sequence <q>stanza - refrain - refrain - stanza.</q> Among other conditions made explicit by this content model are the requirements that at least one stanza must appear in a poem, if it is not composed simply of lines, and that if there is both a title and a stanza they must appear in that order.</p></div3></div2> <div2 id="SG15"><head>Complicating the Issue: More on Element Declarations</head> <p>In the simple cases described so far, it has been assumed that one can identify the immediate constituents of every element defined in a textual structure. A poem consists of stanzas, and an anthology consists of poems. Stanzas do not float around unattached to poems or combined into some other unrelated element; a poem cannot contain an anthology. All the elements of a given document type may be arranged into a hierarchic structure, arranged like a family tree with a single ancestor at the top and many children (mostly the elements containing #PCDATA) at the bottom. This gross simplification turns out to be surprisingly effective for a large number of purposes. It is not however adequate for the full complexity of real textual structures. In particular, it does not cater for the case of more or less freely floating elements that can appear at almost any hierarchic level in the structure, and it does not cater for the case where different elements overlap or several different trees may be identified in the same document. To deal with the first case, SGML provides the <term>exception</term> mechanism; to deal with the second, SGML permits the definition of <socalled>concurrent</socalled> document structures.</p> <div3 id="SG151"><head>Exceptions to the Content Model</head> <p>In most documents, there will be some elements that can occur at any level of its structure. Annotations, for example, might be attached to the whole of a poem, to a stanza, to a line of a stanza or to a single word within it. In a textual critical edition, the same might be true of variant readings. In this simple case, the complexity of adding an annotation element as an optional component of every content model is not particularly onerous; in a more realistically complex model perhaps containing some ten or twenty levels such an approach can become much more difficult. <!-- It would not make much sense to include optional --> <!-- annotation elements explicitly as a component of every --> <!-- element in even a moderately complex DTD. --> <!-- Suppressed above on grounds that since the TEI does --> <!-- it, we probably shouldn't say it makes no sense. --> <!-- (msm) --></p> <p>To cope with this, SGML allows for any content model to be further modified by means of an <term>exception</term> list. There are two types of exception: <term>inclusions</term>, that is, additional elements that can be included at any point in the model group or any of its constituent elements; and <term>exclusions</term>, that is, elements that cannot be included within the current model.</p> <p>To extend our declarations further to allow for annotations and variant readings, which we will assume can appear anywhere within the text of a poem, we first need to add declarations for these two elements: <eg> <![ CDATA [ <!ELEMENT (note | variant) - - (#PCDATA)> ]]> </eg> The note and variant elements must have both start- and end-tags, since they can appear anywhere. Rather than add them to the content model for each type of poem, we can add them in the form of an inclusion list to the poem element, which now reads: <eg> <![ CDATA [ <!ELEMENT poem - O (title?, (stanza+ | couplet+ | line+) ) +(note | variant) > ]]> </eg> The plus sign at the start of the (NOTE | VARIANT) name list indicates that this is an inclusion exception. With this addition, notes or variants can appear at any point in the content of a poem element---even those (such as <gi tei="no">title</gi>) for which we have defined a content model of #PCDATA. They can thus also appear within notes or variants!</p> <p>If we wanted for some reason to prevent notes or variants appearing within titles, we could add an exclusion exception to the declaration for <gi tei="no">title</gi> above: <eg><![ CDATA [ <!ELEMENT title - O (#PCDATA) -(note | variant) > ]]> </eg> The minus sign at the start of the (NOTE | VARIANT) name list indicates that this is an exclusion exception. With this addition, notes and variants will be prohibited from appearing within titles, notwithstanding their potential inclusion implied by the previous addition to the content model for <gi tei="no">poem</gi>.</p> <p>In the same way, we could prevent notes and variants from nesting within notes and variants by modifying the definition above to read <eg><![ CDATA [ <!ELEMENT (note | variant) - - (#PCDATA) -(note | variant) > ]]> </eg> The meticulous reader will note that this precludes both variants within notes and notes within variants. Inclusion and exclusion exceptions should be used with care as their ramifications may not be immediately apparent.</p></div3> <div3 id="SG152"><head>Concurrent Structures <!-- this section should probably be killed --></head> <p>All the structures we have so far discussed have been simply hierarchic: that is, at every level of the tree, each node is entirely contained by a parent node. The figure below represents the structure of a document conforming to the simple DTD we have so far defined as a tree (drawn on its side through exigencies of space). We have already seen how Blake's poem can be divided into a title and two stanzas, each of four lines. In this diagram, we add a second poem, consisting of one stanza and a title, to make up an instance of an anthology: <eg> <![ CDATA [ |-------------------title | | |----line1 | |----line2 |------POEM1---|----stanza1---|----line3 | | |----line4 | | | | |----line5 | |----stanza2---|----line6 | |----line7 | |----line8 anthology-| | | |-------------------title | | | | |----line1 | | |----line2 |------POEM2---|----stanza1---|----line3 |----line4 |----line5 ]]> </eg></p> <p>Clearly, there are many such trees that might be drawn to describe the structure of this or other anthologies. Some of them might be representable as further subdivisions of this tree: for example, we might subdivide the lines into individual words, since no word crosses a line boundary. But equally clearly there are many other trees that might be drawn which do <emph>not</emph> fit within this tree. We might, for example, be interested in syntactic structures --- which rarely respect the formal boundaries of verse. Or, to take a simpler example, we might want to represent the pagination of different editions of the same text.</p> <p>One way of doing this would be to group the lines and titles of our current model into pages. A declaration for such an element is simple enough: <eg><![ CDATA [ <!ELEMENT page - - ((title?, line+)+) > ]]> </eg> That is, a page consists of one or more unnamed groups, each of which contains an optional title, followed by a sequence of lines. (Note, incidentally, that this model prohibits a title appearing on its own at the foot of a page). However, simply inserting the element <tag tei="no">page</tag> into the hierarchy already defined is not as easy as it might seem. Some poems are longer than a single page, and other pages contain more than one poem. We cannot therefore insert the element <gi tei="no">page</gi> between <gi tei="no">anthology</gi> and <gi tei="no">poem</gi> in the hierarchy, nor can it go between <gi tei="no">poem</gi> and <gi tei="no">stanza</gi>, nor yet in both places at once! What is needed is the ability to create a separate hierarchy, with the same elements at the bottom (the stanzas, lines and titles), but combined into a different superstructure. This is the ability which the CONCUR feature of SGML gives.</p> <p>A separate document type definition must be created for each hierarchic tree into which the text is to be structured. The definition we have so far built up for the anthology looks, in full, like this: <eg> <![ CDATA [ <!DOCTYPE anthology [ <!ELEMENT anthology - - (poem+) > <!ELEMENT poem - - (title?, stanza+) > <!ELEMENT stanza - O (line+) > <!ELEMENT (title | line) - O (#PCDATA) > ]> ]]> </eg> As this example shows, the name of a document type must always be the same as the name of the largest element in it, that is the element at the top of the hierarchy. The syntax used is discussed further below (see section <ptr target="SG182">). Let us now add to this declaration a second definition for a concurrent document type, which we will call a paged anthology, or <tag tei="no">p.anth</tag> for short: <eg><![ CDATA [ <!DOCTYPE p.anth [ <!ELEMENT p.anth - - (page+) > <!ELEMENT page - - ((title?, line+)+) > <!ELEMENT (title|line) - O (#PCDATA) > ]> ]]> </eg></p> <p>We have now defined two different ways of looking at the same basic text---the PCDATA components grouped by both these document type definitions into lines or titles. In one view, the lines are grouped into stanzas and poems; in the other they are grouped into pages only. Notice that it is exactly the same text which is visible in both views: the two hierarchies simply allow us to arrange it in two different ways.</p> <p>To mark up the two views, it will be necessary to indicate which hierarchy each element belongs to. This is done by including the name of the document type (the view) within parentheses immediately before the identifier concerned, inside both start- and end-tags. Thus, pages (which are only visible in the <gi tei="no">p.anth</gi> document type) must be tagged with a <tag tei="no">(p.anth)page</tag> tag at their start and a <tag tei="no">/(p.anth)page</tag> at their end. In the same way, as poems and stanzas appear only in the <gi tei="no">anthology</gi> document type, they must now be tagged using <tag tei="no">(anthology)poem</tag> and <tag tei="no">(anthology)stanza</tag> tags respectively. For the line and title elements, however, which appear in both hierarchies, no document type specification need be given: any tag containing only a name is assumed to mark an element present in every active document type.</p> <p>As a simple example, let us assume that Blake's poem appears in some paged anthology, with the page break occurring half way through the first stanza. The poem might then be marked up as follows: <eg><![ CDATA [ <(anthology)anthology> <(p.anth)p.anth> <(p.anth)page> <!-- other titles and lines on this page here --> <(anthology)poem><title>The SICK ROSE <(anthology)stanza> <line>O Rose thou art sick. <line>The invisible worm, </(p.anth)page> <(p.anth)page> <line>That flies in the night <line>In the howling storm: <(anthology)stanza> <line>Has found out thy bed <line>Of crimson joy: <line>And his dark secret love <line>Does thy life destroy. </(anthology)poem> <!-- rest of material on this page here --> </(p.anth)page> </(p.anth)p.anth) </(anthology)anthology> ]]> </eg></p> <p>It is now possible to select only the elements concerned with a particular view from the text, even though both are represented in the tagging. A processor concerned only with the pagination will see only those elements whose tags include the P.ANTH specification, or which have no specification at all. A processor concerned only with the ANTHOLOGY view of things will not see the page breaks. And a processor concerned to inter-relate the two views can do so unambiguously.</p> <p>A note of caution is appropriate: CONCUR is an optional feature of SGML, and not all available SGML software systems support it, while those which do, do not always do so according to the letter of the standard. For that reason, if for no other, wherever these Guidelines have identified a potential application of CONCUR, they also invariably suggest alternative methods as well. For fuller discussion of these issues, see chapter <ptr target="NH">. <!-- For an alternative, and possibly --> <!-- simpler, way of dealing with the problems of marking page --> <!-- boundaries and similar concurrent hierarchies, see section --> <!-- xref target=>. --></p> <p>Note also that we cannot introduce a new element, a page number for example, into the <gi tei="no">p.anth</gi> document type, since there is no existing data in the <gi tei="no">anthology</gi> document type which could be fitted into it. One way of adding that extra information is discussed in the next section.</p></div3></div2> <div2 id="SG16"><head>Attributes</head> <p>In the SGML context, the word <mentioned>attribute</mentioned>, like some other words, has a specific technical sense. It is used to describe information which is in some sense descriptive of a specific element occurrence but not regarded as part of its content. For example, you might wish to add a <att tei="no">status</att> attribute to occurrences of some elements in a document to indicate their degree of reliability, or to add an <att tei="no">identifier</att> attribute so that you could refer to particular element occurrences from elsewhere within a document. Attributes are useful in precisely such circumstances.</p> <p>Although different elements may have attributes with the same name, (for example, in the TEI scheme, every element is defined as having an <att>id</att> attribute), they are always regarded as different, and may have different values assigned to them. If an element has been defined as having attributes, the attribute values are supplied in the document instance as <term>attribute-value pairs</term> inside the start-tag for the element occurrence. An end-tag may not contain an attribute-value specification, since it would be redundant.</p> <p>For example <eg><![ CDATA[ <poem id=P1 status="draft"> ... </poem> ]]> </eg> The <tag tei="no">poem</tag> element has been defined as having two attributes: <att>id</att> and <att>status.</att> For the instance of a <tag tei="no">poem</tag> in this example, represented here by an ellipsis, the <att>id</att> attribute has the value <code type="val">P1</code> and the <att>status</att> attribute has the value <code type="val">draft.</code> An SGML processor can use the values of the attributes in any way it chooses; for example, a formatter might print a poem element which has the status attribute set to <code type="val">draft</code> in a different way from one with the same attribute set to <code type="val">revised</code>; another processor might use the same attribute to determine whether or not poem elements are to be processed at all. The <att>id</att> attribute is a slightly special case in that, by convention, it is always used to supply a unique value to identify a particular element occurrence, <!-- added next line. -msm --> which can be used for cross reference purposes, as discussed further below. <!-- N.B. ID is allowed to have other functions. -msm --></p> <p>Like elements, attributes are declared in the SGML document type declaration, using rather similar syntax. As well as specifying its name and the element to which it is to be attached, it is possible to specify (within limits) what kind of value is acceptable for an attribute and a default value.</p> <p>The following declarations could be used to define the two attributes we have specified above for the <gi tei="no">poem</gi> element: <eg> <![ CDATA [ <!ATTLIST poem id ID #IMPLIED status (draft | revised | published) draft > ]]> </eg></p> <p>The declaration begins with the symbol <code type="kw">ATTLIST</code>, which introduces an <term>attribute list specification</term>. The first part of this specifies the element (or elements) concerned. In our example, attributes have been declared only for the <gi tei="no">poem</gi> element. If several elements share the same attributes, they may all be defined in a single declaration; just as with element declarations, several names may be given in a parenthesized list. Following this name (or list of names), is a series of rows, one for each attribute being declared, each containing three parts. These specify the name of the attribute, the type of value it takes, and a default value respectively.</p> <p>Attribute names (<att tei="no">id</att> and <att tei="no">status</att> in this example) are subject to the same restrictions as other names in SGML; they need not be unique across the whole DTD, however, but only within the list of attributes for a given element.</p> <p>The second part of an attribute specification can take one of two forms, both illustrated above. The first case uses one of a number of special keywords to declare what kind of value an attribute may take. In the example above, the special keyword <code type="kw">ID</code> is used to indicate that the attribute <att tei="no">ID</att> will be used to supply a unique identifying value for each poem instance (see further the discussion below). Among other possible SGML keywords are <list type="gloss"> <label>CDATA</label> <item>The attribute value may contain any valid character data; tags may be included in the value, but they will not be recognized by the SGML parser, and will not be processed as tags normally are</item> <label>IDREF</label> <item>The attribute value must contain a pointer to some other element (see further the discussion of ID below)</item> <label>NMTOKEN</label> <item>The attribute value is a <term>name token</term>, that is, (more or less) any string of alphanumeric characters</item> <label>NUMBER</label> <item>The attribute value is composed only of numerals</item> </list> <!-- changed def of NUMBER above from math to graphics. -msm --> <!-- replaced NMTOKEN with NAME -msm --> <!-- restored NMTOKEN since I was wrong -msm --></p> <p>In the example above, a list of the possible values for the <att tei="no">status</att> attribute has been supplied. This means that a parser can check that no <gi tei="no">poem</gi> is defined for which the <att>status</att> attribute does not have one of <code type="val">draft</code>, <code type="val">revised</code>, or <code type="val">published</code> as its value. Alternatively, if the declared value had been either CDATA or NAME, a parser would have accepted almost any string of characters (<code type="frag">status=awful</code> or <code type="frag">status=12345678</code> if it had been a <code type="kw">NMTOKEN</code>; <code type="frag">status="anything goes"</code> or <code type="frag">status = "well, ALMOST anything"</code> if it were <code type="kw">CDATA</code>). Sometimes, of course, the set of possible values cannot be pre-defined. Where it can, as in this case, it is generally better to do so. <!-- <code type=frag>status = "well, <emph>almost</emph> --> <!-- anything"</code> --> <!-- DB objects to having the EMPH tag inside a literal. --> <!-- So does RC, who thinks it will mislead novices. --> <!-- Joke doesn't seem so funny to me anymore either, so --> <!-- have deleted it. (msm) --></p> <p>The last piece of each information in each attribute definition specifies how a parser should interpret the absence of the attribute concerned. This can be done by supplying one of the special keywords listed below, or (as in this case) by supplying a specific value which is then regarded as the value for every element which does not supply a value for the attribute concerned. Using the example above, if a poem is simply tagged <tag tei="no">poem</tag>, the parser will treat it exactly as if it were tagged <tag tei="no">poem status=draft</tag>. Alternatively, one of the following keywords may be used to specify a default value for an attribute: <list type="gloss"> <label>#REQUIRED</label> <item>A value must be specified.</item> <label>#IMPLIED</label> <item>A value need not be supplied (as in the case of <att>ID</att> above).</item> <label>#CURRENT</label> <item>If no value is supplied in this element occurrence, the last specified value should be used.</item> </list></p> <p>For example, if the attribute definition above were rewritten as <eg><![ CDATA [ <!ATTLIST poem id ID #IMPLIED status (draft | revised | published) #CURRENT > ]]> </eg> then poems which appear in the anthology simply tagged <tag tei="no">poem</tag> would be treated as if they had the same status as the preceding poem. If the keyword were #REQUIRED rather than #CURRENT, the parser would report such poems as erroneously tagged, as it would if any value other than <code type="val">draft,</code> <code type="val">published,</code> or <code type="val">revised</code> were supplied. The use of #CURRENT implies that whatever value is specified for this attribute on the first poem will apply to all subsequent poems, until altered by a new value. Only the status of the first poem need therefore be supplied, if all are the same.</p> <p>It is sometimes necessary to refer to an occurrence of one textual element from within another, an obvious example being phrases such as <q>see note 6</q> or <q>as discussed in chapter 5.</q> When a text is being produced the actual numbers associated with the notes or chapters may not be certain. If we are using descriptive markup, such things as page or chapter numbers, being entirely matters of presentation, will not in any case be present in the marked up text: they will be assigned by whatever processor is operating on the text (and may indeed differ in different applications). SGML therefore provides a special mechanism by which any element occurrence may be given a special identifier, a kind of label, which may be used to refer to it from anywhere else within the same text. The cross-reference itself is regarded as an element occurrence of a specific kind, which must also be declared in the DTD. In each case, the identifying label (which may be arbitrary) is supplied as the value of a special attribute.</p> <p>Suppose, for example, we wish to include a reference within the notes on one poem that refers to another poem. We will first need to provide some way of attaching a label to each poem: this is done by defining an attribute for the <gi tei="no">poem</gi> element, as suggested above. <eg><![ CDATA [ <!ATTLIST poem id ID #IMPLIED > ]]> </eg></p> <p>Here we define an attribute <att>id</att>, the value of which must be of type <code type="kw">ID</code>. It is not required that any attribute of type <code type="kw">ID</code> have the name <att tei="no">id</att> as well; it is however a useful convention almost universally observed. Note that not every poem need carry an <att>id</att> attribute and the parser may safely ignore the lack of one in those which do not. Only poems to which we intend to refer need use this attribute; for each such poem we should now include in its start-tag some unique identifier, for example: <eg><![CDATA[ <POEM ID=Rose> Text of poem with identifier 'ROSE' </POEM> <POEM ID=P40> Text of poem with identifier 'P40' </POEM> <POEM> This poem has no identifier </POEM> ]]> </eg></p> <p>Next we need to define a new element for the cross reference itself. This will not have any content---it is only a pointer---but it has an attribute, the value of which will be the identifier of the element pointed at. This is achieved by the following declarations: <eg><![CDATA[ <!ELEMENT poemref - O EMPTY > <!ATTLIST poemref target IDREF #REQUIRED > ]]> </eg></p> <p>The <gi tei="no">poemref</gi> element needs no end-tag because it has no content. It has a single attribute called <att>target</att>. The value of this attribute must be of type IDREF (the keyword used for cross reference pointers of this type) and it must be supplied.</p> <p>With these declarations in force, we can now encode a reference to the poem with id <code type="val">Rose</code> as follows: <eg> <![CDATA[ Blake's poem on the sick rose <POEMREF TARGET=Rose> ... ]]> </eg></p> <p>When an SGML parser encounters this empty element it will simply check that an element exists with the identifier <code type="val">Rose</code>. Different SGML processors could take any number of additional actions: a formatter might construct an exact page and line reference for the location of the poem in the current document and insert it, or just quote the poem's title or first lines. A hypertext style processor might use this element as a signal to activate a link to the poem being referred to. The purpose of the SGML markup is simply to indicate that a cross reference exists: it does not determine what the processor is to do with it.</p></div2> <div2 id="SG17"><head>SGML Entities</head> <p>The aspects of SGML discussed so far are all concerned with the markup of structural elements within a document. SGML also provides a simple and flexible method of encoding and naming arbitrary parts of the actual content of a document in a portable way. In SGML the word <term>entity</term> has a special sense: it means a named part of a marked up document, irrespective of any structural considerations. An entity might be a string of characters or a whole file of text. To include it in a document, we use a construction known as an <term>entity reference</term>. For example, the following declaration <eg> <![CDATA[ <!ENTITY tei "Text Encoding Initiative"> ]]> </eg> defines an entity whose name is <ident rend="noindex" type="ge">tei</ident> and whose value is the string <q>Text Encoding Initiative.</q> <note>By convention case is significant in entity names, unlike element names.</note> This is an instance of an <term>entity declaration</term>, which declares an <term>internal entity</term>. The following declaration, by contrast, declares an <term>external entity</term> (sometimes called, loosely, a <term>system entity</term>): <eg> <![CDATA[ <!ENTITY ChapTwo SYSTEM "sgmlmkup.txt"> ]]> </eg> This defines an external entity whose name is <ident rend="noindex" type="ge">ChapTwo</ident> and whose value is the text associated with the system identifier --- in this case, the system identifier is the name of an operating system file and the replacement text of the entity is the contents of the file.<note>Strictly speaking, SGML does not require system identifiers to be operating-system file names, nor need external entities be files at all; they can in principle be any data source available to the SGML processor: files, results of database queries, results of calls to system functions, Web pages --- anything at all. System identifiers can be any method of naming the entity which the SGML parser's interface to its operating environment can use to elicit the data from the environment. It is simpler, however, when first learning SGML, to think of external entities as referring to files, and this discussion therefore ignores the other possibilities. All existing SGML processors do support the use of external entities to refer to files; fewer support the other possible uses of external entities.</note></p> <p>System identifiers are, by their nature, system dependent; in the interests of data portability, therefore, SGML provides another way of naming external entities: <eg> <![CDATA[ <!ENTITY p3.sg PUBLIC "-//TEI//TEXT Guidelines Chapter on SGML//EN" > ]]> </eg> The <code type="kw">SYSTEM</code> has been replaced by <code type="kw">PUBLIC</code>, and the system identifier has been replaced by a <term>public identifier</term>. Public identifiers can (in principle) take virtually any form; it is usual to use the form shown above (also known as a <term>formal public identifier</term>), in which the delimiters <mentioned>//</mentioned> divide the identifier into the following parts: <list> <label>TEI</label> <item>indicates the owner of this public identifier (often but not necessarily the owner of the data in question); the preceding <mentioned>-</mentioned> signals that this particular owner identifier is not registered with ISO (a <mentioned>+</mentioned> would imply that one could find out the full name and address of the owner from the official registry of owner identifiers)</item> <label>TEXT</label> <item>is a keyword indicating the nature of the entity: other legal values are <code>DOCUMENT</code> (for full SGML documents), <code>DTD</code> (for document type definitions), <code>ELEMENTS</code> (for sets of element declarations), <code>ENTITIES</code> (for sets of entity declarations), and a number of others which are less frequently needed and will not be discussed here.</item> <label>Guidelines Chapter on SGML</label> <item>gives a descriptive name to the entity.</item> <label>EN</label> <item>is the ISO language code for the language in which the entity is written.</item> </list> Public identifiers do help make SGML documents less dependent on particular computer systems, but they do require SGML systems to provide mechanisms for mapping from the public identifiers to file identifiers or other system identifiers. (The most widely used such mechanism at this time is the SGML Open Catalog format described briefly below in section <ptr target='SGpatanc'>.</p> <p>Once an entity has been declared, it may be referenced anywhere within a document. This is done by supplying its name prefixed with the ampersand character and followed by the semicolon. The semicolon may be omitted if the entity reference is followed by a space or record end.</p> <p>When an SGML parser encounters such an <term>entity reference,</term> it immediately substitutes the value declared for the entity name. Thus, the passage <q>The work of the &tei has only just begun</q> will be interpreted by an SGML processor exactly as if it read <q>The work of the Text Encoding Initiative has only just begun</q>. In the case of an external entity, it is, of course, the contents of the operating system file which are substituted, so that the passage <q>The following text has been suppressed: &ChapTwo;</q> will be expanded to include the whole of whatever the system finds in the file <ident rend="noindex" type="file">sgmlmkup.txt</ident>.</p> <p>This obviously saves typing, and simplifies the task of maintaining consistency in a set of documents. If the printing of a complex document is to be done at many sites, the document body itself might use an entity reference, such as <code type="frag">&site;</code>, wherever the name of the site is required. Different entity declarations could then be added at different sites to supply the appropriate string to be substituted for this name, with no need to change the text of the document itself.</p> <p>This <term>string substitution</term> mechanism has many other applications. It can be used to circumvent the notorious inadequacies of many computer systems for representing the full range of graphic characters needed for the display of modern English (let alone the requirements of other modern scripts or of ancient languages). So-called <socalled>special characters</socalled> not directly accessible from the keyboard (or if accessible not correctly translated when transmitted) may be represented by an entity reference.</p> <p>Suppose, for example, that we wish to encode the use of ligatures in early printed texts. The ligatured form of <mentioned>ct</mentioned> might be distinguished from the non-ligatured form by encoding it as <code type="frag">&ctlig;</code> rather than <code type="frag">ct.</code> Other special typographic features such as leafstops or rules could equally well be represented by mnemonic entity references in the text. When processing such texts, an entity declaration would be added giving the desired representation for such textual elements. If, for example, ligatured letters are of no interest, we would simply add a declaration such as <eg> <![CDATA[ <!ENTITY ctlig "ct" > ]]> </eg> and the distinction present in the source document would be removed. If, on the other hand, a formatting program capable of representing ligatured characters is to be used, we might replace the entity declaration to give whatever sequence of characters such a program requires as the expansion.</p> <p>A list of entity declarations is known as an <term>entity set</term>. Standard entity sets are provided for use with most SGML processors, in which the names used will normally be taken from the lists of such names published as an annex to the SGML standard and elsewhere, as mentioned above.</p> <p>The replacement values given in an entity declaration are, of course, highly system dependent. If the characters to be used in them cannot be typed in directly, SGML provides a mechanism to specify characters by their numeric values, known as <term>character references</term>. A character reference is distinguished from other characters in the replacement string by the fact that it begins with a special symbol, conventionally the sequence <mentioned>&#</mentioned>, and ends with the normal semicolon. For example, if the formatter to be used represents the ligatured form of ct by the characters c and t prefixed by the character with decimal value 102, the entity declaration would read: <eg><![CDATA[ <!ENTITY ctlig "fct" > ]]> </eg> Note that character references will generally not make sense if transferred to another hardware or software environment: for this reason, their use is only recommended in situations like this.</p> <p>Useful though the entity reference mechanism is for dealing with occasional departures from the expected character set, no one would consider using it to encode extended passages, such as quotations in Greek or Russian in an English text. In such situations, different mechanisms are appropriate. These are discussed elsewhere in these Guidelines (see chapter <ptr target="CH">). <!-- expand on following to show how ST works in simple terms --></p> <p>A special form of entities, <term>parameter entities</term>, may be used within SGML markup declarations; these differ from the entities discussed above (which technically are known as <term>general entities</term>) in two ways: <list type="bullets"> <item>Parameter entities are used <emph>only</emph> within SGML markup declarations; with some special exceptions which will not be discussed here, they will normally not be found within the document itself.</item> <item>Parameter entities are delimited by percent sign and semicolon, rather than by ampersand and semicolon.</item> </list> Declarations for parameter entities take the same form as those for general entities, but insert a percent sign between the keyword <code type="kw">ENTITY</code> and the name of the entity itself. White space (blanks, tabs, or line breaks) must occur on both sides of the percent sign. An internal parameter entity named <ident type="pe">TEI.prose</ident>, with an expansion of <code>INCLUDE</code>, and an external parameter entity named <ident type="pe">TEI.extensions.dtd</ident>, which refers to the system file <ident type="file">mystuff.dtd</ident>, could be declared thus:<note>Such entity declarations might be used in extending the TEI base tag set for prose using the declarations found in <ident type="file">mystuff.dtd</ident>.</note> <eg> <![ CDATA [ <!ENTITY % TEI.prose 'INCLUDE'> <!ENTITY % TEI.extensions.dtd SYSTEM 'mystuff.dtd'> ]]> </eg></p> <p>The TEI document type definition makes extensive use of parameter entities to control the selection of different tag sets and to make it easier to modify the TEI DTD. Numerous examples of their use may thus be found in chapter <ptr target="ST">.</p></div2> <div2 id="SG17bis"><head>Marked Sections</head> <p>It is occasionally convenient to mark some portion of a text for special treatment by the SGML parser. Certain portions of legal boilerplate, for example, might need to be included or omitted systematically, depending on the state or country in which the document was intended to be valid. (Thus the statement <q>Liability is limited to $50,000.</q> might need to be included in Delaware, but excluded in Maryland.) Technical manuals for related products might share a great deal of information but differ in some details; it might be convenient to maintain all the information for the entire set of related products in a single document, selecting at display or print time only those portions relevant to one specific product. (Thus, a discussion of how to change the oil in a car might use the same text for most steps, but offer different advice on removing the carburetor, depending on the specific engine model in question.)</p> <p>SGML provides the <term>marked section</term> construct to handle such practical requirements of document production. In general, as the examples above are intended to suggest, it is more obviously useful in the production of new texts than in the encoding of pre-existing texts. Most users of the TEI encoding scheme will never need to use marked sections, and may wish to skip the remainder of this discussion. The TEI DTD makes extensive use of marked sections, however, and this section should be read and understood carefully by anyone wishing to follow in detail the discussions in chapter <ptr target="ST">.</p> <p>The <socalled>special processing</socalled> offered for marked sections in SGML can be of several types, each associated with one of the following keywords: <list type="gloss"> <label>INCLUDE</label> <item>The marked section should be included in the document and processed normally.</item> <label>IGNORE</label> <item>The marked section should be ignored entirely; if the SGML application program produces output from the document, the marked section will be excluded from the document.</item> <label>CDATA</label> <item>The marked section may contain strings of characters which look like SGML tags or entity references, but which should not be recognized as such by the SGML parser. (These Guidelines use such CDATA marked sections to enclose the examples of SGML tagging.)</item> <label>RCDATA</label> <item>The marked section may contain strings of characters which look like SGML tags, but which should not be recognized as such by the SGML parser; entity references, on the other hand, may be present and should be recognized and expanded as normal.</item> <label>TEMP</label> <item>The passage included in the marked section is a temporary part of the document; the marked section is used primarily to indicate its location, so that it can be removed or revised conveniently later.</item> </list></p> <p>When a marked section occurs in the text, it is preceded by a <term>marked-section start</term> string, which contains one or more keywords from the list above; its end is marked by a <term>marked-section close</term> string. The second and last lines of the following example are the start and close of a marked section to be ignored: <eg> <![ RCDATA [ In such cases, the bank will reimburse the customer for all losses. <![ IGNORE [ Liability is limited to $50,000. ]&nil]> ]]> </eg> Of the marked section keywords, the most important for understanding the TEI DTD are <code type="kw">INCLUDE</code> and <code type="kw">IGNORE</code>; these can be used to include and exclude portions of a document --- or a DTD --- selectively, so as to adjust it to relevant circumstances (e.g. to allow a user to select portions of the DTD relevant to the document in question).</p> <p>The literal keywords <code type="kw">INCLUDE</code> and <code type="kw">IGNORE</code>, however, are not much use in adjusting a DTD or a document to a user's requirements. (To change the text above to include the excluded sentence, for example, a user would have to edit the text manually and change <code type="kw">IGNORE</code> to <code type="kw">INCLUDE</code>. It might be thought just as easy to add and delete the sentence manually.) But the keywords need not be given as literal values; they can be represented by a parameter entity reference. In a document with many sentences which should be included only in Maryland, for example, each such sentence can be included in a marked section whose keyword is represented by a reference to a parameter entity named <ident type="pe">Maryland</ident>. The earlier example would then be: <eg> <![ RCDATA [ In such cases, the bank will reimburse the customer for all losses. <![ %Maryland; [ Liability is limited to $50,000. ]&nil]> ]]> </eg> When the entity <ident type="pe">Maryland</ident> is defined as <code type="fragment">IGNORE</code>, the marked sections so marked will all be excluded. If the definition is changed to the following, the marked sections will be included in the document: <eg> <![ CDATA [ <!ENTITY % Maryland 'INCLUDE'> ]]> </eg> When parameter entities are used in this way to control marked sections in a DTD, the external DTD file normally contains a default declaration. If the user wishes to override the default (as by including the Maryland sections), adding an appropriate declaration to the DTD subset suffices to override the default.<note>This is so because the declarations in the DTD subset are read before those in the external DTD file, and the first declaration of a given entity is the one which counts. This was described briefly in section <ptr target="SG17">.</note></p> <p>The examples of parameter entity declarations at the end of the preceding section can now be better understood. The declarations <eg> <![ CDATA [ <!ENTITY % TEI.prose 'INCLUDE'> <!ENTITY % TEI.extensions.dtd SYSTEM 'mystuff.dtd'> ]]> </eg> have the effect of <emph>including</emph> in the DTD all the sections marked as relevant to prose, since in the external DTD files such sections are all included in marked sections controlled by the parameter entity <ident type="pe">TEI.prose</ident>. They also override the default declaration of <ident type="pe">TEI.extensions.dtd</ident> (which declares this entity as an empty string), so as to include the file <ident type="file">mystuff.dtd</ident> in the DTD.</p></div2> <div2 id="SG18"><head>Putting It All Together</head> <p>An SGML conformant document has a number of parts, not all of which have been discussed in this chapter, and many of which the user of these Guidelines may safely ignore. For completeness, the following summary of how the parts are inter-related may however be found useful.</p> <p>An SGML document consists of an SGML <term>prolog</term> and a <term>document instance</term>. The prolog contains an <term>SGML declaration</term> (described below) and a <term>document type definition</term>, which contains element and entity declarations such as those described above. Different software systems may provide different ways of associating the document instance with the prolog; in some cases, for example, the prolog may be <socalled>hard-wired</socalled> into the software used, so that it is completely invisible to the user.</p> <div3 id="SG181"><head>The SGML Declaration</head> <p>The SGML declaration specifies basic facts about the dialect of SGML being used such as the character set, the codes used for SGML delimiters, the length of identifiers, etc. Its content for TEI-conformant document types is discussed further in chapters <ptr target="GR"> and <ptr target="CF">. Normally the SGML declaration will be held in the form of compiled tables by the SGML processor and will thus be invisible to the user.</p></div3> <div3 id="SG182"><head>The DTD</head> <p>The document type definition specifies the document type definition against which the document instance is to be validated. Like the SGML declaration it may be held in the form of compiled tables within the SGML processor, or associated with it in some way which is invisible to the user, or requires only that the name of the document type be specified before the document is validated.</p> <p>At its simplest the document type definition consists simply of a base document type definition (possibly also one or more concurrent document type definitions) which is prefixed to the document instance. For example: <eg> <![ CDATA[ <!DOCTYPE my.dtd [ <!-- all declarations for MY.DTD go here --> ... ]> <my.dtd> This is an instance of a MY.DTD type document </my.dtd> ]]> </eg></p> <p>More usually, the document type definition will be held in a separate file and invoked by reference, as follows: <eg> <![ CDATA [ <!DOCTYPE tei.2 PUBLIC "-//TEI P3//DTD Main Document Type//EN"> <tei.2> This is an instance of an unmodified TEI type document </tei.2> ]]> </eg> Here, the text of the TEI.2 document type definition is not given explicitly, but the SGML processor is told that it may be read from the file with the system identifier given in quotation marks. The square brackets may still be supplied, as in this example, even though they enclose nothing.</p> <p>The part enclosed by square brackets is known as the <term>document type declaration subset</term> or <socalled>DTD subset</socalled>. Its purpose is to specify any modification to be made to the DTD being invoked, thus: <eg><![ CDATA[ <!DOCTYPE tei.2 PUBLIC "-//TEI P3//DTD Main Document Type//EN" [ <!ENTITY tla "Three Letter Acronym"> <!ELEMENT my.tag - - (#PCDATA)> <!-- any other special-purpose declarations or re-definitions go in here --> ]> <tei.2> This is an instance of a modified TEI.2 type document, which may contain <my.tag>my special tags</my.tag> and references to my usual entities such as &tla;. </tei.2> ]]> </eg> <!-- this whole chunk needs rewriting: it sucks --> In this case, the document type definition in force includes first the contents of the DTD subset, and then the contents of the file specified after the keyword <code type="kw">SYSTEM</code>. The order is important, because in SGML only the first declaration of an entity counts. In the above example, therefore, the declaration of the entity <ident type="ent">tla</ident> in the DTD subset would take precedence over any declaration of the same entity in the file <ident type="file">tei2.dtd</ident>. It is perfectly legal SGML for entities to be declared twice; this is the usual method for allowing user modification of SGML DTDs. (Elements, by contrast, may not be declared more than once; if a declaration for <gi tei="no">my.tag</gi> were contained in file <ident type="file">tei.dtd</ident>, the SGML parser would signal an error.) Combining and extending the TEI document type definitions is discussed further in chapter <ptr target="ST">.</p></div3> <div3 id="SG183"><head>The Document Instance</head> <p>The document instance is the content of the document itself. It contains only text, markup and general entity references, and thus may not contain any new declarations. <!-- < note > --> <!-- Except for SUBDOC entities; see example in < xref --> <!-- target=z69 > . --> <!-- < /note > --> A convenient way of building up large documents in a modular fashion might be to use the DTD subset to declare entities for the individual pieces or modules, thus: <eg> <![ CDATA [ <!DOCTYPE tei.2 [ <!ENTITY chap1 system "chap1.txt"> <!ENTITY chap2 system "chap2.txt"> <!ENTITY chap3 "-- not yet written --"> ]> <tei.2> <teiHeader> ... </teiHeader> <text> <front> ... </front> <body> &chap1; &chap2; &chap3; ... </body> </text> </tei.2> ]]> </eg></p> <p>In this example, the DTD contained in file <ident type="file">tei2.dtd</ident> has been extended by entity declarations for each chapter of the work. The first two are external entities referring to the file in which the text of particular chapters is to be found; the third a dummy, indicating that the text does not yet exist (alternatively, an entity with a null value could be used). In the document instance, the entity references &chap1; etc. will be resolved by the parser to give the required contents. The chapter files themselves will not, of course, contain any element, attribute list, or entity declarations---just tagged text.</p></div3> <div3 id="SGpatanc"> <head>Ancillary Files</head> <p>In practice, working SGML systems use a number of ancillary files to hold configuration information. Style sheets (now still commonly using software-specific formats, though standard formats should become more widespread with time), files with specialized processing instructions, files with setup information for specific programs, will all be needed in practical use of SGML. Many programs also use environment variables for some parts of their setup (e.g. to tell the program where to find its configuration files). The details of the files needed, and how they affect the actions of an SGML system, vary with the system and cannot be described here.</p> <p>Most recent SGML software does use a standard format called the <term>SGML Open Catalog</term> format<note>The SGML Open catalog format described here is documented in <bibl>SGML Open Technical Resolution 9401:1997, <title>Entity Management, which is available from the SGML Open Web site at http://www.sgmlopen.org. for the catalog files they use to help the SGML parser locate the files (or network resources) corresponding to public and system identifiers used in declarations of entities (and of other SGML constructs). A catalog file using that format may contain entries for each public identifier in a DTD in a form like the following: Each entry has three parts: the keyword PUBLIC, the public identifier, and the system identifier under which SGML parsers can find the entity named by the public identifier.

An SGML Open Catalog file with entries for each part of the TEI DTD is available from the TEI. See chapter for more information. Using SGML

A variety of software is available to assist in the tasks of creating, validating and processing SGML documents. Only a few basic types can be described here. At the heart of most such software is an SGML parser: that is, a piece of software which can take a document type definition and generate from it a software system capable of validating any document invoking that DTD. Output from a parser, at its simplest, is just yes (the document instance is valid) or no (it is not). Most parsers will however also produce a new version of the document instance in canonical form (typically with all end-tags supplied and entity references resolved) or formatted according to user specifications. This form can then be used by other pieces of software (loosely or tightly coupled with the parser) to provide additional functions, such as structured editing, formatting and database management.

A structured editor is a kind of intelligent word-processor. It can use information extracted from a processed DTD to prompt the user with information about which elements are required at different points in a document as the document is being created. It can also greatly simplify the task of preparing a document, for example by inserting tags automatically.

A formatter operates on a tagged document instance to produce a printed form of it. Many typographic distinctions, such as the use of particular typefaces or sizes, are intimately related to structural distinctions, and formatters can thus usefully take advantage of descriptive markup. It is also possible to define the tagging structure expected by a formatting program in SGML terms, as a concurrent document structure.

Text-oriented database management systems typically use inverted file indexes to point into documents, or subdivisions of them. A search can be made for an occurrence of some word or word pattern within a document or within a subdivision of one. Meaningful subdivisions of input documents will of course be closely related to the subdivisions specified using descriptive markup. It is thus simple for textual database systems to take advantage of SGML-tagged documents. Much research work is also currently going into ways of extending the capabilities of existing (non-text) database systems to take advantage of the structuring information made explicit by SGML markup.

Hypertext systems improve on other methods of handling text by supporting associative links within and across documents. Again, the basic building block needed for such systems is also a basic building block of SGML markup: the ability to identify and to link together individual document elements comes free as a part of the SGML way of doing things. By tagging links explicitly, rather than using proprietary software, developers of hypertexts can be sure that the resources they create will continue to be useful. To load an SGML document into a hypertext system requires only a processor which can correctly interpret SGML tags such as those discussed in chapter .