Print

Print


Excellent description of EDAC.  My apologies to Robin for not explaining
what EDAC is. I agree with Jerry that EDAC fixes without warnings is not
good and that EDAC is no substitute for backup.


-----Original Message-----
From: Jerome McDonough [mailto:[log in to unmask]]
Sent: Sunday, March 17, 2002 6:27 PM
To: [log in to unmask]
Subject: Re: [METS] Checksum


EDAC: Error Detection And Correction.
EDAC techniques are used to ensure that
data is correct and has not been corrupted
by hardware failures or errors in transmission.
The Reed-Solomon algoithm mentioned is usually
used to correct errors in peripheral devices
(like tape/disk storage).  Use of EDAC for disk
subsystems is actually pretty standard in larger
mainframe style computer systems.  You'll more
typically see discussion of EDAC in PCs around
RAM.

Note that EDAC is *not* a substitute for a backup.
EDAC can detect and correct minor errors in a file,
but if a file gets really and truly corrupted,
EDAC can't bring it back to life.  Also, obviously,
EDAC corrects errors in a file, which buys you nothing
if somebody does something stupid like delete it.

One of the downsides of EDAC, from my perspective
(at least in its common form, where it's tightly
integrated into the hardware), is that by compensating
for minor errors without providing warning that
something's not kosher with a file, you tend not
to find out that something's wrong with a disk
subsystem until something more disastrous happens.
I suppose I'm not really objecting to it fixing
problems so much as the fact that it doesn't warn
you.  The reason why I've been planning on using
file checksums/file integrity checking systems to
monitor files is that they're *designed* to warn
you when there's a problem.  Most EDAC systems
I've seen aren't.


----- Original Message -----
From: Robin Wendler <[log in to unmask]>
Date: Saturday, March 16, 2002 11:24 pm
Subject: Re: [METS] Checksum

> For the technically challenged, can someone explain what this means
> (EDAC?)  and what the implications of it are for the METS checksum
> attribute?
>
> --Robin
>
>
> On Fri, 15 Mar 2002, Marshall, Neil wrote:
>
> > Carl,
> >
> > In my opinion, checksums at the essence file level should be a
> secondary> tool for file integrity in a digital repository.  I
> believe the primary tool
> > for file integrity should be some form of EDAC applied to the
> AIP or its
> > components.  Checksums tell you something is broke but they
> can't fix it.
> > EDAC such as Reed-Solomon not only tells you when something is
> amiss but can
> > also fix single bit and burst errors as well.
> >
> > Have a great weekend,
> > Neil
> >
> > -----Original Message-----
> > From: Carl Fleischhauer [mailto:[log in to unmask]]
> > Sent: Friday, March 15, 2002 8:38 AM
> > To: [log in to unmask]
> > Subject: Re: [METS] Checksum
> >
> >
> > Thanks for the dialog, MacKenzie.  In our theorizing here at LC,
> we have
> > wished for a file integrity monitoring tool, and pictured a
> system that
> > checks and rechecks a file/object over time.  We seek
> reassurance about
> > the integrity of our objects over the long haul.
> >
> > Now: how does that relate to METS?  Ummm.  If METS was the
> metadata for an
> > OAIS AIP, then there might be an argument that the "original"
> checksum (or
> > equivalent) is parked there, with the object.  The job of the
> monitoring> system would be to run comparisons and alert the owner
> when a change is
> > noticed.  In addition, you would probably want the system to
> keep a log of
> > when the comparisons were made.  Now, in such a system, is it
> useful or
> > needful to know the date when the original checksum (or
> equivalent) was
> > created?  I'd be curious to hear your thoughts.
> >
> > Carl Fleischhauer
> > Library of Congress
> >
> > On Thu, 14 Mar 2002, MacKenzie Smith wrote:
> >
> > > In the better late than never category, I've tried to think of any
> > problems
> > > with this
> > > proposal and can't, except the general caveat that it *is*
> possible to
> > take
> > > generalization to an absurd extreme, however in this case it
> makes sense
> > to me
> > > to go for a general solution over a specific one since I agree
> that there
> > > will be
> > > other checksum algorithms and we shouldn't make invalid
> presumptions.> >
> > > Could we, pehaps, have an optional attribute of checksumtype,
> and if that
> > > attribute is missing, but there is a checksum, assume it's an MD5?
> > >
> > > And what on earth would the benefit of a checksum create date be?
> > >
> > > MacKenzie/
> > >
> > > At 09:57 AM 3/5/2002 -0500, Robin Wendler wrote:
> > > >One of the general questions that came out of the recent
> > > >(and soon to be documented, I swear) meeting on technical
> metadata for
> > > >audio was about the METS <checksum> attribute of the <file>
> element.> > >Right now, this is defined explicitly as MD5, but
> there are now and
> > > >undoubtedly will continue to be other checksum algorithms in
> use. We were
> > > >wondering whether it would be better/possible to generalize
> this in METS,
> > > >providing for the checksum type, value, and create date. It
> seems better
> > > >to raise this now, rather before we hit the big Version 1.0.
> > > >
> > > >What do others think about this?
> > > >
> > > >-- Robin
> > > >
> > > >Robin Wendler  ........................     work  (617) 495-3724
> > > >Office for Information Systems  .......     fax   (617) 495-0491
> > > >Harvard University Library  ...........     [log in to unmask]
> > > >Cambridge, MA, USA 02138  .............
> > >
> > > MacKenzie Smith
> > > Associate Director for Technology
> > > MIT Libraries
> > > Building 14S-208
> > > 77 Massachusetts Avenue
> > > Cambridge, MA  02139
> > > (617)253-8184
> > > [log in to unmask]
> > >
> >
>
> --
> Robin Wendler  ........................     work  (617) 495-3724
> Office for Information Systems  .......     fax   (617) 495-0491
> Harvard University Library  ...........     [log in to unmask]
> Cambridge, MA, USA 02138  .............
>