> Date: Tue, 29 Jun 2004 10:36:08 +0100
> From: "Matthew J. Dovey" <[log in to unmask]>
>
> > > The XML spec guys really did exclude most chars in the 0-0x01f
> > > range. I wonder why.
>
> Well, here is someone claiming that it is a good thing:
> http://www.awprofessional.com/articles/article.asp?p=169537&seqNum=2

This article is totally irrelevant to us, as it starts out by saying:

        Since XML is text, it does not include binary
        characters such as NULL (#x00), BEL (#x07), DC1 (#x11)
        through DC4 (#x14), and so forth.

What he means is that _he_ uses XML for representing text. He should
know that it is also used for many other purposes. His point
basically seems to be "if you echo data with these characters to a
terminal then something funny might happen" which is just about the
weakest thing I've ever heard.

What _is_ interesting about this article is the strong case it makes
that XML 1.1 is a waste of time. (Although it allows ESC to be
included in data, it still doesn't allow NUL, for example). Which
makes me somewhat less than sanguine that it will be widely, or indeed
at all, adopted.

> > Regarding Rob's actual question, the correct answer is and
> > must be that XML is just a broken transport.
>
> Which is probably why it has been fixed in XML 1.1 (remember how we
> fixed some stuff when we moved from SRW 1.0 to 1.1?)

Yes. That is nice for the maybe four or five XML 1.1 applications out
there.

> > HOWEVER, we clearly also need to be able to send
> > base64-encoding CQL queries,
>
> Well no - we just need a relational modifier to indicate that the
> *term* is base64 encoded (which we need anyway for sending binary
> data e.g. my mimeEncoded modifier at
> http://www.ceridwen.com/srw/music-contextset.html) rather than
> encode the whole query.

Yes, this is a better approach one as it solves the problem for other
CQL applications as well as SRW. I have only two comments: first,
that the relation modifier's name "mimeEncoded" is ambiguous as it
could refer either to Base64 or Quoted-Printable, and second that this
is useful and general enough to belong in the base CQL context set
rather than an application-specific one.

> > Holy moley. All that to fix a bug that was deliberately written
> > into the XML specification. Unbelievable. Unbelievable.
>
> Do we really want to go to that extreme - these are somewhat
> sweeping changes to SRW and will break a lot of the
> type-checking/validation we currently have in the schemas.

You're probably right that the pragmatics in this case dictate that we
just mumble under our breath and live with the brokenness. But we
shouldn't let that diminish our sense of outrage. (Question: should
that outrage be towards the people who designed XML as a text
language? Probably not: it should be towards those who generated and
propagated the doctrine that XML is suitable as a generic structured
data language. And towards ourselves for not spotting the lie.)

> A) XML 1.1. already fixes this - XML 1.1 is currently in a final
> draft so it may be 6 months or so before it becomes part of the Web
> Services stack but any fix we make now will be rendered obsolete
> when XML 1.1 is supported.

Give it six _years_ and you may be right.

> B) having control characters in the scan seems to me to be such a
> pathological (albeit real) case, that taking such a radical knife to
> SRW seems out of proportion.

I don't think we should be quick to dismiss real cases just because
they don't seem like the kind of thing _we_ would want to do. We
should fix SRW's use of XML encoding to allow binary scan terms to get
through -- probably (I agree with you about the standard-wide
upheaval) by making a change only to scan, such as a
thisTermIsBase64Encoded attribute on the term element.

> Until XML 1.1 et al stabilises we could adopt the practice of the
> SQLX guys (ISO/IEC 9075-14:2007) and use _x001b_ to represent this
> character?

Ouch. Then let's hope no-one ever wants to use SRW to search a
database of ISO/IEC standards and wants to find _x001b_. No,
seriously, we ought to avoid this kind of information-losing hack.

 _/|_ _______________________________________________________________
/o ) \/ Mike Taylor <[log in to unmask]> http://www.miketaylor.org.uk
)_v__/\ "The purpose of computers is human freedom" -- Ted Nelson.

--
Listen to free demos of soundtrack music for film, TV and radio
        http://www.pipedreaming.org.uk/soundtrack/