LISTSERV mailing list manager LISTSERV 16.0

Help for ZNG Archives


ZNG Archives

ZNG Archives


[email protected]


View:

Message:

[

First

|

Previous

|

Next

|

Last

]

By Topic:

[

First

|

Previous

|

Next

|

Last

]

By Author:

[

First

|

Previous

|

Next

|

Last

]

Font:

Proportional Font

LISTSERV Archives

LISTSERV Archives

ZNG Home

ZNG Home

ZNG  May 2002

ZNG May 2002

Subject:

Re: cql index definitions

From:

Alan Kent <[log in to unmask]>

Reply-To:

Z39.50 Next-Generation Initiative

Date:

Mon, 20 May 2002 10:34:27 +1000

Content-Type:

text/plain

Parts/Attachments:

Parts/Attachments

text/plain (65 lines)

On Fri, May 17, 2002 at 02:11:30PM +0200, Janifer Gatenby wrote:
> Tasmania?  - right truncation
> Tasmania? tiger? - right truncation
> Tasmania? tiger - truncation 104
> Tasmanian tiger? - truncation 104

Overall, I agree with the basic semantics brought up in this thread.
But we hit a few issues in the past that I thought I would raise.
Its only important *if* we decide that we should (as a part of the
CQL spec) mandate how CQL should be turned into RPN. If we leave it
up to an implementor, then there is nothing to be done (my preference).

In our CCL->RPN parser we certainly try to be smart about the above
sorts of things. But it can be tricky to get right. For example,
we allow TITLE to be indexed as words and as a single complete string
(that is, the full title as one term). Do do Bath first-in-field
right truncation character based, we use the TITLE as a single term
and do right truncation on it. Piece of cake. When searching the TITLE
as words, you can right truncate each word.

Now this separation of 'terms' from 'words' may be our implemenation
choice, and not part of Z39.50 as such. But to us, a term in an index
can be any sequence of bytes. The database designer chooses how to
turn content into a set of terms to be indexed.

The problem is how does the CQL parser know if it sees

    Tasmanian Tiger

whether its a single term or multiple terms? (Our system allows spaces
in terms.) How can the CQL parser safely break the input text into terms
in the same way as the database engine?

While you might say "well, if the attribute indicates 'words' then
use spaces to separate terms", but I don't think this is good enough
because different systems treat punctuation differently. Is '-' part
of a word, or a word separator? How about '.'? How about '/'? etc.
I think its *valid* for different systems to have different word parsing
rules. As such, how can a CQL parser work out how many terms are in
the string

    a/b?c.d#e!f%g^h(i)j

This is important in order to get all the truncation examples above
correct.

I raise all this just to point out sometimes life is not as easy as
you expect. We could just ignore the problem, but this problem is
one of the reasons that type 104 trunction is so important to us.
We try our best to use right truncation etc, but at times its just
too hard (unless you assume index terms are always separated by
spaces and nothing else).

Alan

ps: I am not proposing anything in this mail - just raising some issues.
I guess it depends on how fine grained we want to get when describing
how to turn CQL into RPN.
--
Alan Kent (mailto:[log in to unmask], http://www.mds.rmit.edu.au/~ajk/)
Project: TeraText Technical Director, InQuirion Pty Ltd (www.inquirion.com)
Postal: Multimedia Database Systems, RMIT, GPO Box 2476V, Melbourne 3001.
Where: RMIT MDS, Bld 91, Level 3, 110 Victoria St, Carlton 3053, VIC Australia.
Phone: +61 3 9925 4114  Reception: +61 3 9925 4099  Fax: +61 3 9925 4098

Top of Message | Previous Page | Permalink

Advanced Options


Options

Log In

Log In

Get Password

Get Password


Search Archives

Search Archives


Subscribe or Unsubscribe

Subscribe or Unsubscribe


Archives

July 2017
October 2016
July 2016
August 2014
February 2014
December 2013
November 2013
October 2013
February 2013
January 2013
October 2012
August 2012
April 2012
January 2012
October 2011
May 2011
April 2011
November 2010
October 2010
September 2010
July 2010
June 2010
May 2010
April 2010
March 2010
February 2010
January 2010
December 2009
October 2009
September 2009
August 2009
July 2009
May 2009
April 2009
March 2009
February 2009
December 2008
November 2008
October 2008
September 2008
August 2008
July 2008
June 2008
May 2008
April 2008
March 2008
February 2008
January 2008
December 2007
November 2007
October 2007
September 2007
August 2007
July 2007
June 2007
May 2007
April 2007
March 2007
January 2007
December 2006
November 2006
October 2006
September 2006
August 2006
July 2006
June 2006
May 2006
April 2006
March 2006
February 2006
January 2006
December 2005
November 2005
October 2005
September 2005
August 2005
July 2005
June 2005
May 2005
April 2005
March 2005
February 2005
January 2005
December 2004
November 2004
October 2004
September 2004
August 2004
July 2004
June 2004
May 2004
April 2004
March 2004
February 2004
January 2004
December 2003
November 2003
October 2003
September 2003
August 2003
July 2003
June 2003
May 2003
April 2003
March 2003
February 2003
January 2003
December 2002
November 2002
October 2002
September 2002
August 2002
July 2002
June 2002
May 2002
April 2002
March 2002
February 2002
January 2002
December 2001
November 2001
October 2001
September 2001
August 2001
July 2001

ATOM RSS1 RSS2



LISTSERV.LOC.GOV

CataList Email List Search Powered by the LISTSERV Email List Manager