Hi all,
[Hedzer]
> prox:
> The special-case prox relation is implemented using the WHEN
> and WHEN NOT booleans in Adlib. Unit 0 selects WHEN, unit -1
> will select WHEN NOT. In Adlib, the WHEN (NOT) booleans on
> two distinct indexes will first search records using the
> left operand index. For each record the right operand o
> perator and value will be checked in the same occurrence. The
> Adlib-specific property of occurrences comes down to the
> following: each field can have 0 or more values (unless
> explicitly stated as non-repeatable), whereas conventional
> RDBMS's can only hold 0 or 1 values in a field. Indexes can
> be specified to index either the first or all occurrences.
> The WHEN operator explicitly checks matching occurrences. See
> section 6.3.10 in the Adlib User Guide (available from
> http://www.adlibsoft.com/) for more information.
[Rob]
> I have no clear understanding as to what is happening in the below
description, but it sounds wrong :)
> I think what you want is:
> a prox/unit=element/distace=0 b
> for WHEN
> (which is to say, a and b within the same element)
> An example of this is used when searching ZeeRex databases:
> <name set="cql">anywhere</name>
> Could be matched as:
> zrx.set = (cql's URI) prox/unit=element/distance=0 zrx.index =
anywhere
> Further docs/examples:
> http://srw.cheshire3.org/contextSets/ZeeRex/
> I don't think WHEN NOT can be expressed without additional
> constructions. It could be done with a negation modifier on prox
however.
Hmm, I guess you're right. I knew I was abusing the prox boolean
anyway.. I'll add adlib.when and adlib.whennot
modifiers to the 'and' boolean, that's a better solution I guess.
>> I'll create an adlib.allIndexes index for the current behaviour, and
>> program cql.anywhere like it should be.
>So adlib.allIndexes will search all indexes in the adlib context-set,
yes?
Yep, the adlib context set being all indexes defined for the configured
database.
>> So it's perfectly possible to have a cql.word indexed title, and no
>> cql.string index available for the same field.
>Right. It'd be very strange to have a string indexed dc.description,
for example.
>> but then I'd have to throw an error message on every query
>> that makes me imply cql.string where the index is actually word, and
vice versa..
>That's exactly what it should do.
This seems a bit strange from an Adlib standpoint, which has no 'exact'
operator or word/string modifiers. It interprets search terms based on
the index type. If CQL works differently, I'll just have to get used to
that :-)
> You can include a configInfo section within the index, and give the
relations and/or relation modifiers
> that are available. This lets you say that you support within for
dc.date, but only cql.word for dc.description.
Hey that's new to me! I found one (1) paragraph mentioning this in the
DTD explanation document after re-reading it. Maybe you could add at
least an example making this feature a bit more clear?
>> Then there's still one option question about CQL context set
modifiers: how do I type check number & isoDate
>> searches?
> In my opinion you can determine the structure of the term as you wish,
unless the client has specifically told
> you the structure to use with a relation modifier.
Okay, then I'll use the index type when no modifier is supplied.
Attached is an updated version of the Adlib Base Profile. If I receive
no further comments I'll implement the
features described there, starting January 3rd.
Merry Christmas & happy holidays everyone!
Best regards,
Hedzer Westra, Systems Developer
Adlib | Information Systems
Reactorweg 291
3542 AD Utrecht
Postbus 1436
3600 BK Maarssen
tel: +31-30-241 1885
www: http://www.adlibsoft.com
The Adlib Base Profile version 1.0
and
The Adlib Context Set version 1.0
Beta-3
Initiated: Monday December 13th, 2004
Last updated: Friday December 23rd, 2004
Hedzer Westra, [log in to unmask]
Adlib Information Systems
http://www.adlibsoft.com
Contents
Introduction
URIs
SRW, SRU and CQL
SRW Requirements
Optional and server-dependent features
Configuration
Implemented SRW/CQL features
Unimplemented optional features
Adlib server-specific implementations
Indexes
Relations
Terms
Modifiers
Sorting
The Adlib Context set
CQL to Adlib bridging
-------------
Introduction
This document describes the Adlib Base Profile 1.0 of the SRU server that is implemented by Adlib Information
Systems in its Internet Server software. It is only a partial profile; it only describes the software
capabilities. For each installation and configuration of this software a full profile document should be
defined.
URIs
The URI for the Adlib Base Profile is "info:srw/profile/6/1.0".
The Adlib context set is identified by the URI "info:srw/cql-context-set/6/1.0". The preferred identifier
for this URI is 'adlib'.
These URIs are defined using the SRW 'info URI' convention, using the Adlib authority string "6", which
was assigned by the ZING working group.
SRW, SRU and CQL
The SRW and SRU protocols, and the CQL language, CQL context set and SRW Base Profile are all described on
http://www.loc.gov/z3950/agency/zing/index.html. Please refer there for information about this search
& retrieval protocol.
SRW Requirements
The Adlib Internet Server implements all features required by the SRW Base Profile,
refer to http://www.loc.gov/z3950/agency/zing/srw/base-profile.html.
Optional and server-dependent features
SRW/U and CQL are quite broad protocols which allow for many optional and server-dependent features.
This document defines which optional and server-dependent features are implemented, and how these function.
Configuration
Adlib supplies a number of applications to its customers, who can make changes and additions to their
applications. Therefore there is no single 'Adlib' context set or profile. Each customer can configure their
Internet Server to produce the SRU context sets and profiles that are required. For any such configuration a
context set and profile document should be created - or existing ones should be configured. This document
describes the Adlib Base Profile and the Adlib Context Set, which form the basis of such a customer-defined
document. Therefore, only meta-indexes are defined here. Metadata formats (like Dublin Core or MarcXML) are
not defined here.
Implemented SRW/CQL features
- protocol version 1.1
- the SRU protocol, i.e., HTTP GET/POST CGI requests
This is a slight extension of SRU, since the current documentation does not mention SRU POST.
- explain operation
- searchRetrieve operation
- CQL 1.1 parsing
- CQL 1.1 handling
- CQL context set as far as the Base Profile requests it, plus some extras:
+ and, or, not booleans
+ =, >, <, <=, >=, <> relations
+ exact, all, any, scr relations
+ within relation
+ cql.anywhere, adlib.allIndexes and adlib.record meta-indexes
+ cql.serverChoice surrogate index
- sorting
- surrogate & non-surrogate diagnostics generation
- recordSchemas
- request echoing, xSortKeys and XCQL
Unimplemented optional features
- recordXPath handling
- result sets
- full proximity searches, i.e. the 'prox' boolean
- word anchoring (^)
- matching on a single character using '?'
- scan operation
- SRW (SOAP) as communication layer
Adlib server-specific implementations
Indexes
- The adlib.allIndexes meta-index searches all indexes defined in the Adlib database at once. This is
different from cql.anywhere, which searches all indexes in all context sets.
- The adlib.record meta-index searches the whole record, so all of the fields in each record. This includes
data that is not indexed (and possibly not even displayed in any record schema) and therefore not
searchable using CQL indexes. The relation must be '='.
Note: In future versions, CQL might support a cql.record meta-index with the same semantics.
Relations
- cql.scr is always handled as '='
- The 'within' relation is implemented using range searching. Exactly two words must be supplied, separated
by a single space. The range search type can be selected by using an adlib.range modifier, with the
following values:
leftexclusive
rightexclusive
exclusive
inclusive (default)
Example: 'date within/adlib.range=leftexclusive "2000 2004"'
Note: the CQL context set always uses inclusive range searching; there is no range modifier.
Terms
- empty term searches are not supported
- * for pattern matching is only usable at the beginning and/or end of a search term
Modifiers
- the 'and' boolean accepts one of the modifiers 'adlib.when' and 'adlib.whennot'. See below for more
information.
- Thesaurus-enabled searches can be executed by issuing an adlib.thesaurus modifier with one of the
following values:
generic
broader
narrower
related
topterm
parents
These only work correctly on indexes with thesaurus links defined. Otherwise, they fall back on
normal searching. The modifier is supported only for the 'exact' and '=' relations.
- there are two types of CQL context set modifiers: data type modifiers and pattern modifiers.
The accepted data type modifiers are:
cql.string
cql.word
cql.isoDate
cql.number
Note: cql.uri is invalid in the Adlib Base Profile.
The accepted pattern modifiers are:
cql.masked
cql.unmasked (not yet defined in CQL context set)
Of each modifier type only one can be active for each search clause. The combinations
/cql.masked/cql.number and /cql.masked/cql.isoDate are invalid.
- Adlib interprets terms in the following manner:
If modifiers are sent in the query, those are used. If they are not, modifiers are implied according to
the following rules:
1. pattern modifer: cql.masked is always assumed, unless cql.number or cql.isoDate are supplied or
implied.
2. data type modifier, implied using the relation and sometimes the Adlib index type:
+ 'exact': implied modifier is cql.string
+ '=': implied modifier is cql.word, cql.number or cql.isoDate, depending on the index type
+ 'any' and 'all': implied modifier is cql.word. Words are combined using OR (for 'any') or AND
(for 'all')
+ 'within': there is no implied data type modifier; there must be two words separated by a single space
Modifier cql.unmasked means:
The CQL pattern match character * has no special meaning; pattern matching is not possible using
this modifier. The characters ^, ? and * do not have to be escaped.
Modifier cql.masked means:
Pattern matching on * is possible. The characters ^, ?, * and \ must be escaped with \.
Modifier cql.word means:
Words are split and then re-combined using the Adlib separators and concatenators rule.
Word adjacency is not used when searching. An error will be returned when searching with cql.word
on a string index.
Separator characters are: [];,!@()|{}<>? carriagereturn newline space tab
Concatenator characters are: `-=\./~#$%^&_+:"'*
Note: the CQL context set says nothing about how words are to be split but instead leaves that up
to implementations to be specified.
Modifier cql.string means:
Terms are not inspected for separators or concatenators. An error will be returned when searching with
cql.string on a word index.
Modifier cql.isoDate means:
Term is interpreted as a single ISO 8601 date. If the Adlib index type is European or US, the term will
be translated before the actual search takes place.
Modifier cql.number means:
Term is interpreted as a single 32-bit signed integer
The difference(s) between the CQL context set is/are the following:
- word adjacency, which should be used with the cql.word data type modifier, is a feature that the Adlib
database engine currently does not implement.
Sorting
- sorting is only supported for hard-wired (case sensitive) paths, not for full XPaths. The customer can
define a path for each CQL index.
The Adlib Context set
The Adlib context set version 1.0 defines:
(meta-)indexes:
- adlib.record (whether this will be added to the CQL context set is still unclear, so the prefix is
adlib, not cql)
- adlib.allIndexes
modifiers:
- adlib.thesaurus its six accepted values (generic, broader, etc.)
- adlib.range for the within relation
- adlib.when and adlib.whennot for the and boolean
- cql.unmasked (it is assumed this modifier will eventually be added to the CQL context set, hence the
cql prefix)
CQL to Adlib bridging considerations
and boolean modifiers:
The and relation modifiers are implemented using the WHEN and WHEN NOT booleans in Adlib. In Adlib, the
WHEN (NOT) booleans on two distinct indexes will first search records using the left operand index.
For each record the right operand operator and value will be checked in the same occurrence. The
Adlib-specific property of occurrences comes down to the following: each field can have 0 or more values
(unless explicitly stated as non-repeatable), whereas conventional RDBMS's can only hold 0 or 1 values in
a field. Indexes can be specified to index either the first or all occurrences. The WHEN operator
explicitly checks matching occurrences. See section 6.3.10 in the Adlib User Guide (available from
http://www.adlibsoft.com/) for more information.
within:
This relation is implemented using range searching, i.e. the Adlib WHEN boolean used on two identical
indexes. Note that this is a very different operation from the WHEN boolean used on two distinct indexes.
Performance:
- The adlib.allIndexes and cql.anywhere meta-indexes might have slow search responses if there are a lot of
indexes.
- The adlib.record meta-index cannot use any index and will always be slow.
- Searching on *...* is done using the Adlib 'contains' operator, which is slow since no index can be used.
|
