Comments from another lurker.

Metasearch engines generally do search over diverse sources, so any
commonality (from Z39.50 or SRU/W, etc.) is only a bonus; it cannot be
considered the norm. Consequently the engines will have to undertake all the
heavy lifting for sites which do their own thing anyway.

Marc is correct that there is still the issue of what to do with the
results. It is bad enough at the moment. Adding a set of results of unknown
type (from a set of say 5) would mean a lot more conversion logic in the
engine.

Supporting this functionality at the search engine (target) side will mean
quite major extensions to the processing for those systems to choose the
"best" action to take. Unfortunately this may also be client dependant. So
things could be moving away from a decent result to something very much less
desirable.

One of the perennial big complaints from the content providers is that the
metasearch engines are "dumbing down" both their search capabilities and the
quality of the data returned. Unfortunately this could have a bad effect in
both of these areas. So I think the content providers would not be too
excited about it. As a metasearch engine provider, it will not really help
us as we have to take account of the non-standard guys as well.

I also can't quite see how any of this could be considered as a computing
grid arrangement, but that seems like another thread.

I will be in NC so there will be plenty of us to discuss this in person.

Peter Noerr
MuseGlobal

-----Original Message-----
From: Z39.50 Next-Generation Initiative [mailto:[log in to unmask]]On Behalf Of
Marc Cromme
Sent: Tuesday, April 20, 2004 8:04 AM
To: [log in to unmask]
Subject: Re: metasearch


Interesting proposal
I'll comment inbetween your lines:

Theo van Veen wrote:

>Currently it is not easy with SRU/W to broadcast the same query to many
>SRU/W servers because one has to take into account all the differences
>between different servers.
>
Definitely true - this due to the fact that SRW is a client-server
protocol, whereas
metasearch broadcasting as you are describing is more a grid computing task.

Currently, a client has to have good knowledge of the multiple servers
asked, and one has to program the metasearch client logic accordingly to
the capabilities of the SRW servers used.

The real solution might be a SRW like grid computing protocol, not an
extention to a client-server protocol.

>Especially in metasearching I think it would
>be convenient when there was a possibility to send a query saying "give
>me what is closest to this query" and allow different servers to respond
>with a servers choice according to one or more predefined responses. The
>responses could a.o. be:
>1) searchRetrieveResponse
>2) scanResponse
>3) results of a fuzzy match
>4) number of hits for different access points
>5) etc.
>
>Without having to find out how to translate a query for different
>targets such an "give me the best you can" request returns one or more
>response blocks and the client can use the ones that it understands to
>generate guidance to the user to improve his search.
>
I think a give-me-the-best-you-can answer does not resolve the problem,
since the client still must merge result sets from multiple servers, and
does not even know if the server considered the hit set to be "the real
thing" or "just the best I can". The problem is still the logic inside
the client - how to know what and how to merge??

>It is not the same
>as the "x-scanOnSearchFail" parameter, because it can also apply to
>other situations. For example when there are thousands of hits a server
>could provide a response block in which it gives the number of hits for
>different indexes. The client can use this to propose new searches, even
>with indexes that it would not have offered otherwise.
>
>
>
This means spreading the indexes from server to client - IMHO it smells
like distributed hash tables or distributed inverted indexes in grid and
peer-to-peer networks. If you do not want to write considerable amount
of logic for each client, it might be better to throw out the
client-server philisophy entirely and let the network merge and keep
track of indexes. But this will not possible by a server-client centric
protocol like SRW/SRU.

>I remember having proposed something like this earlier and we will
>implement this as a private extension. However, in the context of the
>NISO metaseach meeting there may be more support for this concept.
>
>1) Who would support a proposal for extending SRU/SRW with such an
>operation?
>2) Should this be done via a new x-parameter or via a new operation?
>
>BTW Who of this group is attending the NISO metasearch meeting?
>
>Theo
>
>
>
This said, I would like to see more specific how you'd plan to do your
extentions. Probably I understand then better your motivation and ideas.

I see forward to see more about your ideas.

Marc Cromme, Index Data