[From my blog post]

What needs to be done before we go much further is to begin to state  
our specific goals and the criteria we will use to determine if we  
have met those goals. Some goals we will discover in the course of  
developing the new environment, so this should be considered a growing  
list. I think it is important that every goal have measurements  
associated with it, to the extent possible. It makes no sense to make  
changes if we cannot know what those changes have achieved. Here are  
some examples of the kinds of things I am thinking of in terms of  
goals; these may not be the actual goals of the project, they are just  
illustrations that I have invented.

  - goal: it should be less expensive to create the bibliographic data  
during the cataloging process
    measurement: using time studies, compare cataloging in MARC and in  
the new format
  - goal: it should be less expensive to maintain the format
    measurement: compare the total time required for a typical MARBI  
proposal to the time required for the new format
  - goal: it should be less expensive for vendors to make required  
changes or additions
    measurement: compare the number of programmer hours needed to make  
a change in the MARC environment and the new environment

  - goal: collaboration on data creation with a wider group of communities
    measurement: count the number of non-library communities that we  
are sharing data with before and after
  - goal: greater participation of small libraries in shared data
    measurement: count number of libraries that were sharing before  
and after the change
  - goal: make library data available for use by other information communities
    measurement: count use of library data in non-library web  
environments before and after

  - goal: library technology staff should be able to implement "apps"  
for their libraries faster and easier than they can today.
    measurement: either number of apps created, or a time measure to  
implement (this one may be hard to compare)
  - goal: library systems vendors can develop new services more  
quickly and more cheaply than before
    measurement: number of changes made in the course of a year, or  
number of staff dedicated to those changes. Another measurement would  
be what libraries are charged and how many libraries make the change  
within some stated time frame

As you can tell from this list, most of the measurements require  
system implementation, not just the development of a new format. But  
the new format cannot be an end in itself; the goal has to be the  
implementation of systems and services using that format. The first  
MARC format that was developed was tested in the LC workflow to see if  
it met the needs of the Library. This required the creation of a  
system (called the "MARC Pilot Project"[1]) and a test period of one  
year. The testing that took place for RDA is probably comparable and  
could serve as a model. Some of the measurements will not be available  
before full implementation, such as the inclusion of more small  
libraries. Continued measurement will be needed.



Karen Coyle
[log in to unmask]
ph: 1-510-540-7596
m: 1-510-435-8234
skype: kcoylenet