One way to understand what XML is and why it is important is to think about it as a way of structuring or organizing information.

Structure is what makes digital information useful to us; it helps us find things, it helps us identify them and understand what they are, and it helps us communicate about them to other people. We can think of the difference between structured and unstructured information as being similar to the difference between different approaches to organizing your tools (or anything else you organize).

In the first image, the only structure is the outer container; essentially, that toolbox only lets us say, "it's in there somewhere..." However, it doesn't give us a way of predicting exactly where something will be, or grouping similar or related things together.

In the middle image, there is a bit more structure; the toolbox has compartments to group things together. But the groupings here are not very rigorous, and might not be the most efficient way of finding what you need.

On the right we have the logical extreme. Every tool has a very specific location, and similar tools are grouped together. You can instantly see if something is missing. If you had a team of workers who all used this same toolbox, they could trade boxes and still be able to find things with their eyes closed.

It's important to note that we're not claiming here that it's always useful to have as much structure as possible. Sometimes the first or second toolbox is exactly what you want: just a big container to hold things together. The question we're going to explore in this tutorial is, how much structure is useful? and how do we represent it?

XML provides a very specific way of structuring information and of looking at the world. It asks us to identify pieces of information that we are interested in, and it asks us to group them together. We can represent these groupings in various different visual ways, all of which represent the same thing—XML's structure. The first useful metaphor is that of a set of nested boxes, like in the example on the left. These boxes always nest perfectly within each other, and do not overlap. You can also think about the structure of XML as being like a tree, as in the middle example. Each XML document is a complete tree, and each XML element is like a branch of that tree. The example on the right shows XML as it is written, using a system of tags that mark the boundaries of the boxes and the structural groupings that make up the informational system. Each box or tree node (or element in XML parlance) is represented by a pair of tags that mark its beginning and its end, and these elements can be nested inside one another.

So what exactly is the relationship between XML and the TEI?

The TEI is an encoding language, specifically a language meant to describe documents. The TEI provides us with terms like paragraph and heading, and defines what these terms mean. Taking the example from the slide, we can think of the TEI as being the vocabulary we use to label the different tools in our toolbox. We can think of TEI as being analogous to a specific toolbox: one that groups together all of the different chisels and names them in a specific way, and separates the chisels from the hammers.

There are lots of encoding languages, such as HTML (hypertext markup language), MML (music markup language), or MathML. We'll use HTML as our example, since it is the most commonly-used. Like TEI, HTML also gives us terms for labeling things, although it has a much smaller vocabulary than the TEI. We could think of HTML as being analogous to a simpler toolbox, one with fewer, more generic slots; instead of having different size slots for each different kind of chisel, it just has a bunch of more generic bins.

So if TEI and HTML are encoding languages, what, then, is XML? XML is our mechanism for defining encoding languages. In the context of our toolbox metaphor, XML makes up the materials that allow us to construct toolboxes and create labels. Think of it as the boxes and markers and stickers that allow you to create an organizational system. It doesn't determine how many little slots your toolbox has, or what you call them, but it says you must have a box, and any little slots inside it must have labels. It provides a general mechanism for representing data structures.

Another way to describe this relationship is to say, where TEI and HTML are languages, XML is a metalanguage. A language has a vocabulary and grammar; the vocabulary tells you what words exist in the language, and the grammar tells you how they may be used. As we learn the TEI, we're going to learn a lot of its vocabulary and grammar rules; these are all specific to the encoding language.

XML, as a metalanguage, provides the mechanisms for defining the grammatical structures of all encoding languages, including TEI. Also important for this tutorial is that XML provides the notation (syntax) for writing in text encoding languages. XML's notation allows us to distinguish between the content we encode and the markup itself.

We've already discussed how XML is a metalanguage that allows us to define the grammar and vocabulary of the TEI, but now we're going to discuss in more detail how these pieces relate to each other.

In text encoding, all of the concepts that underlie our modelling come from us: our disciplines, theoretical frameworks, culture, etc.

XML defines a syntax for text encoding. It is a method for distinguishing our markup from the content that we are marking up.

The TEI provides the specifics of the actual encoding language. It translates our concepts into XML, and provides a controlled vocabulary that we use in marking up the text.

Let's take a minute to step back and look at the big picture.

What we're trying to do is structure our data and represent information structures that matter to us. There are many different ways that we could do this, so what is so distinctive about XML?

First of all, XML is an international standard, which makes it easy to share your data. It is also non-proprietary, which means that no one owns this standard. Another benefit of XML is that it is expressed in plain text, which makes it easily human-readable. Finally, it is hardware and software independent, so it can be processed by a variety of tools on any hardware platform. This means that it is not tied to one specific program or application. This is especially important to the longevity of your data (basically, it will be useful even if the specific application you use becomes outdated).

An XML element is like a slot in a toolbox, or a box in your attic, or field in your database. It's a thing that identifies and labels a piece of information that you want to know about. In the toolbox, we mark the boundaries of the things we're trying to organize, so that everything stays in place.

In XML, we mark the boundaries of elements with things called tags. These also serve as a label for the element. We should also note that tags in XML are not just important for us as readers (serving as human-readable labels for the features we choose to mark). XML tags are also important for the processor. The brackets and the forward-slashes serve to delimit the element, and allow the processor to differentiate elements from the content. Take a minute to look over the way that we notate start tags, end tags, and empty elements.

Since we've said that what we're looking at here is a language, we can also think of elements as being like the nouns in our language: the things we want to name and talk about.

Take a minute to look over these sample passages, and acquaint yourself with what elements look like, and how they are used. If it's useful for you, try to figure out: how many elements are in each example? how many tags? This will help to acquaint you with XML's syntax. Additionally, this will help you understand the difference between an element and a tag.

An XML attribute is like an adjective to the element's noun, in that it modifies an element. However, attributes are a bit more detailed than adjectives. They tell you what aspect of the element they're describing, like its type, or its size, or its color (the name of the attribute) and the specific descriptor within that category of information (the attribute value).

Once again, the rules of XML are at play here. XML specifies that the attribute name must be separated by whitespace from the element name (or other attributes), and it must be followed by an equal sign, and the attribute value must be in quotes.

It is also important to note that you can have more than one attribute on a given element (we often use more than one adjective for a noun!), but we cannot repeat a given attribute more than once (so you can't have two type attributes on name). The order of the attributes does not matter.

To practice the notation of attributes, try creating a person element for yourself that describes things like, your occupation, your alma mater, your hairstyle, your gender, or your place of birth.

For example, you may put Paul Simon]]>

Take a minute here to look over the anatomy of an element as a whole. The vocabulary here is pretty important, and it's useful to get the hang of it early on. Also, feel free to bookmark this slide and come back to it later, if you need a refresher!

Now that you're coming to understand elements and how they function, lets look at a more detailed example of an XML document. The previous slides didn't state this explicitly, but by now, you've probably figured out that XML representation has order: an XML document is an ordered sequence of elements. XML elements can also nest inside each other, as in the document here.

Take a minute and write down (or just think about) the different type of information that the XML is telling us in this case. What are the different elements and attributes conveying?

You'll notice that in this example, we are taking certain concepts about literature and making them explicit through markup. Unless we had very good natural language processing, our computers would probably not be able to recognize that the work above is a sonnet (more specifically a Shakespearean sonnet), so we need to use our markup to make these statements. So, as we do literary analysis, we note that the poem has 14 lines, so we label the lines with the n attribute.* We further know that this group of lines has a rhyme scheme of ababcdcdefefgg, so we state that using the rhyme attribute. When doing literary analysis, we can come to the conclusion that a 14-line poem with this rhyme scheme is a sonnet. If we wanted to, we could also further state what the meter of the lines in the poem (iambic pentameter), or that the poem is broken down into 4 line groups: 3 quatrains, 1 couplet.

The markup also states that there is a name, and that this name is an epithet. This example also shows that you can do phrase-level literary analysis using markup.

*This step isn't entirely necessary, since computers can count pretty well...

Here is another example that we can walk through.

Here, we have another poem that is represented by the lg element (which stands for line group). We can see from its attributes that it is a limerick with the rhyme scheme aabba. Presumably it is the third in some sequence of poems, as we can see from the n attribute being 3. It has a heading and five lines. The rhyme element is telling us which words rhyme with each other. There is also a note that is glossing the term relative and pointing us to the Wikipedia page on the Theory of Relativity.

This slide provides another way of representing the limerick we saw on the previous page: this time in a tree structure. This is a simplified tree because it does not contain the information provided by the attributes and omits the text nodes. However, it does show us how the XML is nested, and additionally provides us with the linking and pointing information between term and note, and ptr and the Wikipedia page.

The last thing you need to know about XML documents is the rules that they follow!

These rules come from the W3C standard for XML, so all XML documents need to follow these three rules. These are specifically called the rules of well-formedness, and every XML language—from TEI to HTML to MathML—must follow these rules.

The first is that all elements must be properly delimited, i.e. they must contain all necessary characters (such as pointy brackets, quotation marks, forward slashes) in the appropriate places.

The second rule is that all documents must have a single root element, or there must be one and only one outermost wrapper element that encloses the whole document. One way to think about this rule is by thinking about XML as a tree. Usually trees have one trunk off of which everything branches. Without the singular root element in XML you wouldn't have a tree; you would have a forest!

Finally, XML cannot overlap. Basically this means that elements must be contained completely within each other. You cannot have a said element that begins in the middle of one p element and ends in the next. As you can imagine, this rule of XML may not always do the best job of describing our documents. Sometimes documents have overlapping structures. We will discuss how to deal with the problem of XML overlap later, but for now, you just need to remember that it isn't allowed.

Take a minute to go through these examples and determine which snippets of XML are well-formed and which ones aren't. This will help to solidify your understanding of XML syntax. As a plus: it will help you with troubleshooting when you begin to encode your own TEI documents!

In addition to well-formedness (conforming to the rules of XML as a metalanguage), there is also validity which requires conforming to the rules of a specific language, such as TEI or HTML.

These rules are very similar to the rules for speaking in a human language: am I using the right words? and am I using them in the correct manner?

When we check an XML document for validity, we are making sure that the document obeys the rules of a specific language.

Validity is determined by a schema, which defines the specific rules for an encoding language. At the bottom of this document, you can see a very simple sample schema. Above it, is the schema translated into more easily-readable terms. Take a minute to look over the rules in this schema, and then we can test our knowledge of validity (you don't need to memorize the rules of the schema, as they are reiterated on the next page).

Take a minute to look through these examples and determine if they are valid, according to the rules of our encoding schema. As you go through, you will probably realize that our schema is not the best at describing the data we have (many of the examples are invalid according to our schema, but are perfectly acceptable as letters). In addition to teaching you about validity, this exercise will show you the importance of having a schema that fits well with your data!