Subject: version 1.5 discussion
From: Brian Tingle <[log in to unmask]>
To: [log in to unmask]
Content-Type: text/plain
Content-Transfer-Encoding: 7bit
Message-Id: <[log in to unmask]>
Mime-Version: 1.0
X-Mailer: Ximian Evolution 1.4.6 
Date: Fri, 29 Jul 2005 14:02:17 -0700

Forwarded for Jerome McDonough


The editorial board is considering several potential revisions to the 
primary METS schema. However, some of these revisions would not be 
backwards-compatible with existing METS documents, and we would like 
some input from the community before we decide to implement 
these. Roughly summarized, the areas where we are considering changes 
to the schema include 1. Adding support for nested files; 2. Bringing 
the <structLink> 'to' and 'from' attributes into conformance with the 
Xlink standard; and 3. Loosening the processContents specification for 
xsd:any elements within METS.


One issue that has come up repeatedly is a need to record within METS 
information about files that may be embedded inside of other 
files. Typical examples of this would be *.zip or *.tar.gz files. Some 
archives are using these and other formats to package together sets of 
files, and they would like a mechanism to indicate that a file contains 
subsidiary files, and to allow the association of descriptive and 
administrative metadata with those subsidiary files. This also implies 
a need for recording some type of information regarding how to 'unpack' 
a packaging file. An issue related to the embedded file problem has 
arisen with respect to multimedia file formats that support the 
multiplexing of audio and video streams together in a single file; some 
libraries/archives would like the ability to record the existence of 
separate streams within a file, as well as linking to separate 
technical metadata records for different streams.

A relatively straightforward way to achieve these goals might be to do 
the following:

1.      Make the <file> element recursive, so that a <file> element
contain other <file> elements;
2.      Add a subsidiary <stream> element beneath <file>, to record the 
existence of separate data streams within a particular file;
3.      Add a new <transformFile> element beneath <file>, to provide an 
indication of the 'unpacking' steps that need to be taken to gain 
access to subsidiary <file> elements. This would need to be repeatable 
to allow for specifying a series of steps to unpack a file, and might 
provide a link to a <behavior> in the <behaviorSec> for performing the 

So, for example, if you had a *tar.gz file with two embedded files, one 
of which had two embedded streams, you'd have a METS fileSec including 
something like this:

    <Flocat />            <!-- location of *tar.gz file -->
    <transformFile /> <!-- Instructions on reversing gzip -->
    <transformFile /> <! Instructions on reversing tar -->
    <file />                 <!-- first embedded file -->
    <file >                  <!-- second embedded file -->
         <stream />      <!-- first embedded stream -->
         <stream />      <!-- second embedded stream -->

These changes could be made to the METS schema without breaking 
backwards compatibility; METS documents created under the current 
schema would also validate against this schema. However, if we allow a 
<transformFile> element to link to a behavior in the <behaviorSec>, we 
would be breaking with past practice, which specified that you should 
be able to remove the <behaviorSec> of any METS document and still have 
a valid METS document. This could be avoided by placing the 
<transformFile> element in the behavioral metadata section and having 
it link to the appropriate <file> element. While this would remove the 
information regarding necessary transformations from proximity with the 
<file> they are paired with, it might also have the advantage of 
allowing a single description of a set of transformations (like the 
ungzip/untar combo) above to point to a variety of files, eliminating 
duplication of information in those cases where a METS document refers 
to multiple *.tar.gz files. 

Our questions for the METS community on these changes would be:

1.      Does making <file> recursive and adding a <stream> element
the needed functionality for addressing compound/embedded files? Would 
people prefer another approach?
2.      Would people rather have a <transformFile> element be located
the <file> element (which would be easier to read) or have it located 
within the <behaviorSec> somewhere (which might eliminate duplication 
of information regarding transformations)?


In the current METS schema, the 'to' and 'from' attributes on the 
smLink element in the structLink section are currently in the METS 
namespace, and are defined as type IDREF. This makes it very easy for 
the attributes to point to <div> elements in the structMap and to 
validate that the attributes are linking to real elements. However, 
this is somewhat problematic inasmuch as the 'to' and 'from' attributes 
are really drawn from the XLink standard, and compatability with other 
schema and tools which use XLink would be enhanced if these attributes 
were placed within the XLink namespace. Additionally, having 'to' and 
'from' defined as being of type IDREF limits them to serving as a link 
between <div> elements in a single METS document. Users wishing to 
decompose extremely complicated objects into a series of METS documents 
cannot link between <div> elements in separate METS documents using the 
current schema. To rectify these problems, it has been proposed that 
'to' and 'from' be placed within the XLink namespace, and that they 
draw their definitions from the current XLink schema available at the 
METS website; this will have the practical upshot of renaming them 
xlink:to and xlink:from and defining both of them as type 
"string". Presumably most users would use either an Xpath or a URI to 
specify links in these attributes, but this would not be enforced by 
the schema.

This change would not be backwards compatible. Existing METS documents 
which use the smlink 'to' and 'from' attributes would not validate 
against an updated schema. It might be possible to avoid this, but only 
by leaving in the existing 'to' and 'from' attributes and also adding 
new xlink:to and xlink:from attributes. Having the two sets of 
attributes seems guaranteed to cause confusion.

The question for the METS community at this point is: would you rather 
1. Leave the 'to' and 'from' attributes as they are, even though this 
means that METS uses a variant of xlink not conformant with the 
standard; 2. Change the 'to' and 'from' attributes to xlink:to and 
xlink:from and redefine them as strings, so that METS' use of Xlink 
conforms to the standard (although this breaks backwards compatibility; 
or 3. Keep the existing 'to' and 'from' attributes, add xlink:to and 
xlink:from attributes to smLink, and indicate that the use of 'to' and 
'from' is now deprecated and may eventually be discontinued?


A final change to the METS schema that has been suggested seems 
relatively unproblematic to the editorial board: changing the 
definition of all METS elements which are of type xsd:any (mdWrap and 
FContent) so that they have a processContents attribute value of 
'lax'. This will allow METS documents that contain XML metadata records 
in a format not defined by a schema placed within an mdWrap element to 
validate; currently they produce an error. XML metadata that does 
conform to a schema will still validate if a schema is available.

Any and all comments on these topics are welcomed, and should be 
addressed to the METS mailing list. Thank you!



Jerome McDonough