Hi all,

I received some 'complaints' and suggestions about the Adlib Base
Profile from Mike & Rob. Here are my replies. I would like the Base
Profile to be final before I start reprogramming, so here are my answers
to their remarks. The new Base Profile (Beta-2) has been attached.

Many thanks for your help so far!

>> - The meta-index cql.anywhere searches all indexes defined in the
>>   Adlib database at once. It does not search all indexes in all
>>   context sets, as the CQL context set suggests.
>It is at best inadvisable, and probably just wrong, to _re_define the
meaning of an existing index like 
>this -- especially such a core one. If cql.anywhere doesn't meet your
needs, it would be better to define 
>your own index that does (or ask to have it added to the CQL set if you
think it's of general interest).
>I think that's exactly what cql.anywhere means.  Search all indexes
that you know about, but you can't be 
>expected to search indexes that you don't know about.
>Which is (thankfully, for once) exactly how it's defined:
>This means "search all indexes from all context sets you know".
Okay, okay, you're right. I was just being lazy; iterating *all* indexes
is easier than iterating over all
CQL-linked indexes. I'll create an adlib.allIndexes index for the
current behaviour, and program cql.anywhere
like it should be.

>> - The adlib.record meta-index searches the whole record. The operator
doesn't matter.
>Perhaps we should consider adding cql.record for whole-record searching
(where supported).
You can if you wish. I'll implement it for 1.2 if it's added then.

>You can't really say "the operator doesn't matter" as this is
overriding established semantics of CQL and 
>the CQL context set.  It would be much better to say "the operator must
be '=': all others will be rejected".
Will do, good suggestion.

>I'm not sure what exactly happens here. Is it:
>(a)  You send some terms and it searches the entire record.  (And how
is this different to cql.anywhere in 
>     practice?)
Not all fields are indexed.

>(b)  You send the entire record as a term, and it returns the entire
record back to you if it finds it. 
>The former.  No-one wants (b).

> it is conventional in CQL to talk of "relations" rather then
"operators".  Unless you have 
> a compelling reason, you should probably stick to this convention.
I'll change that.

>> - The Adlib thesaurus operators 'adlib.generic', 'adlib.broader',
>>   'adlib.narrower', 'adlib.related', 'adlib.topterm' and
>>   'adlib.parents' do thesaurus-enabled searches. These only work
>>   correctly on indexes with thesaurus links defined. Otherwise, they
fall back on '=' searching.
> Are these relations or relation modifiers?  If you don't already have
this nailed down, I would recommend 
> the latter, as they are all refinements on the general relation of
Hmm, good suggestion. They're relations now, because 1. they're
operators in the Adlib Expert Search 
language and 2. I was too lazy to implement modifier handling :-)

I'll change it to: 'term =/adlib.thesaurus=generic myTerm' etc. if that
suits everyone.

>There really should be a thesaurus-use context set defined outside of
Adlib, for use in this and other 
>profiles (or the relevant elements should be added to the existing
Zthes context set).  
generic, topterm and parents are rather esoteric operators so I guess
bt, nt and rt are just fine. 
Adlib does use/uf substitution automatically, and has no concept of le,
so those three aren't supported as 
operators. (FYI I'm using 'operator' here because I'm referring to Adlib
expert search.)

>We actually started this process a month or two back, but got
sidetracks -- or maybe mired in excess 
Are there any concrete results from that discussion, or was it just
that: a discussion?

>Depending on the urgency of your Adlib work, you might try to restart
that process and use the resulting 
>"official" thesaurus-expansion support. 
I just heard yesterday from our customer that Q1 2005 will be a testing
phase, and Q2/Q3 will be the 
implementation phase so there's enough time to reach an agreement. To
top that, the customer probably won't
need explicit thesaurus support, I just added it to offer to SRU/CQL as
much functionality as the Adlib 
query language has.

>> - The 'encloses' and 'within' operators are implemented using the
>>   Adlib WHEN operator. Some examples:
>>   'term encloses "2000 2004"' translates to 'term >= 2000 WHEN term
<= 2004'
>Nope -- "encloses" is the converse of "within", so
>        term encloses "2000 2004"
>translates to
>        term <= 2000 WHEN term >= 2004
Erhm that would always be the empty set.. WHEN is just like AND, only a
bit different. Read the Adlib manual for details :-)

>>   'term within "2001 2005"' translates to 'term > 2001 WHEN term <
>My reading of the CQL context set indicates that this relation is
inclusive of endpoints, so you should 
>translate to term >= 2001 WHEN term <= 2005
> within "2000 2004"
> -> Does the record contain a date between (inclusive) 2000 and 2004
I'll add a modifier: within/adlib.range=exclusive (or inclusive -
default, rightexclusive, leftexclusive).
Makes everybody happy, right? ;-)

> foo.rangeOfDates encloses 2002
> -> Does 2002 fall within (inclusive) the date range in the record.
> <record>
>  <dateRange>2000 2004</dateRange>
> </record>
> would match the encloses query.
> (Hopefully this is how your internal queries work)
No they don't. I'm a bit confused now about encloses works. In the
manual they're not really well defined, 
if I may say so. If encloses specifically needs a two-dimensional index
and a single search term, Adlib 
can't support it. But Mike's example (the one which results in the empty
set) seems to suggest otherwise..

>> - there are two types of modifiers: data type modifiers and pattern
>>   modifiers [...]
>This whole section belongs in the CQL context-set document.
I know, but since it isn't there I thought I'd just make it explicit in
Adlib's profile, so that it's at 
least defined somewhere..

>>   The pattern modifiers are:
>>    cql.masked
>>    cql.unmasked (not defined in CQL context set)
>We should fix that!
Something for a 1.2 proposal?

>>   Note that the CQL context set is not required by the SRW Base
>That's not really true, as the CQL context set provides some of the key
elements used in pretty much CQL 
>queries, e.g. the meaning of all the relations.  Probably the base
profile should make this explicit.
Fine by me, I just wanted to make clear that Adlib's implementation
violates some of the CQL context set 
descriptions. Pending this discussion, I'll clear up all or most of
these violations.

>>   The modifiers cql.word and cql.string can not relate directly to
>>   Adlib term or word matching because this is defined per index by
>>   the user; in Adlib each index can be either word or term
>>   indexed. If required, a field can be indexed by term as well as by
>[I don't understand this fully, I think because it assumes you know
something about Adlib.  Is Adlib's 
>"term" searching similar to what we mean by "string"?]
Yes, it is. I'll try to use CQL terminology in the next version of the

>>   These two indexes can be be reflected using two separate CQL
>>   indexes. It is not possible to use modifiers to switch from one to
the other.
> Why not?  It seems an eminently sensible way of expressing the
Well if you create an index on a field in Adlib, you explicitly have to
tell what type it is:
 int, date, word, string. 
So it's perfectly possible to have a cql.word indexed title, and no
cql.string index available for the same
field. In fact, in the standard app.'s we deliver it's very rare to have
a field indexed on both word and 
string. I could reprogram my implementation to use proper CQL semantics,
but then I'd have to throw an 
error message on every query that makes me imply cql.string where the
index is actually word, and vice 
versa.. And there's no way to report the index type in a ZeeRex record.
I already asked our customer whether 
this is a problem for them but didn't hear back from them yet.

>>   Adlib interprets terms in the following manner:
>>   + operator 'exact': implied modifiers are cql.unmasked [...]
>No, we all agreed that "exact" does _not_ imply unmasked.
Okay, I'll change my implementation. I'll have to add modifier handling
for masked/unmasked selection, then.
I chose these semantics because exact and = do not dictate string resp.
word currently, but it seems
I will have to change that..

>> + operator '=': implied modifiers are cql.masked and either cql.word
>>     or cql.string, depending on the index type. This cannot be seen
>>     in the explain information but must be described in a profile.
>This is _not_ what "=" means in CQL.  It means that the term is
word-structured, irrespective of the index 
>being searched, unless overridden by a relation modifier.
See above: that's not how Adlib functions natively, bridging from
correct CQL to Adlib can be done but has 
some nasty consequences.

>I disagree, Mike.  It means, currently, exact equality, but does not
say how equality is to be determined. 
>Unless the term is a list (which may be a singleton) of words.
>Au contraire.  This is precisely the difference between "=" and
I guess Mike is right, although I don't like it :-)

>>   + operators 'any' and 'all': implied operators are cql.word and
>>     The words are combined using OR (for 'any') or AND (for 'all').
>Yup.  This was always the intent and should probably be explicit in the
CQL context-set document.
The CQL (1.0) tutorial says so, copying it from there to the CQL context
set might be nice indeed.

>>   + adlib.record meta-index: implied operators are cql.string and
>There is nothing in CQL that allows you to infer different
term-structure and masking semantics from an 
>index name.
>So you have to treat = "2004-12-25" as a word? or a string?
Sure you can do this.
>Hmm.  Your example is strangely provoking.
Another fine point to discuss, just like the URI/word splitting thingy.
Anyway, I'll change adlib.record to 
only accept '=', with the correct CQL modifier checks.

>>   This implied behaviour will remain intact in future versions, even
>>   if modifiers will be supported then.
>Aha!  Finally, I spot a tiny, tiny error in the English :-)  That
sentence should say "... even if modifiers 
>ARE supported then".
Ah, the fine concept of 'tenses'. Quite hard to grasp for the Dutch,
since Dutch is far more liberal in that 

>>   - operator 'exact' does not imply cql.string, since cql.string or
>>     cql.word is index dependent on Adlib.
>The correct way to handle this is to have a single CQL index be mapped
to either one of two different 
>underlying Adlib indexes, dependent on whether string or word structure
is used.
Yep, I understand that. See discussion above.
I've extended the profile with a new concept: a prox implementation, as
far as Adlib can implement it. I'm very curious if any of you can accept
me defining it the way I did :-) Adlib can't support any of the prox
functionality as described in the CQL context set..

Then there's still one option question about CQL context set modifiers:
how do I type check number & isoDate searches? I can either infer those
modifiers from the index (which isn't allowed in CQL according to Mike)
or from
the term (which is not foolproof, e.g. string indexes can store numbers
as well). If I want to check all index
types I should check that the modifier is cql.number for an Adlib
integer index, but then users are required to
send =/cql.number for number indexes always. Probably not a nice thing
to do... 

Best regards,

Hedzer Westra, Systems Developer

Adlib | Information Systems
Reactorweg 291
3542 AD Utrecht
Postbus 1436
3600 BK Maarssen
tel: +31-30-241 1885