[-empyre-] Selective and comprehensive archives

["Margaret Phillips" <mphillips@nla.gov.au>]

Hello Everyone

This is my opening statement to the discussion on preserving our online
heritage.  Thank you to Melinda for setting up this forum for us.  The
discussion is already vigorous and stimulating.

I'd like to add to Paul Koerbin's comments on the difference between a
selective archive like PANDORA and the Internet Archive.  As he notes,
there are advantages and disadvantages to both approaches.  

During the mid to late 1990s, a small number of national libraries
established web archives to fullfil their responsibilities for
collecting their nation's published output - Sweden, Norway, Finland,
Denmark, Canada and Australia, being ones that readily come to mind.
Sweden, Finland and Norway took the comprehensive  approach by
undertaking periodic domain harvests (similar to the Internet Archive
approach, only concentrating on the web domain of their own countries).
Canada, Denmark and Australia took a selective approach.  After a few
years of archiving, however, while still committed to their chosen
approach for the perceived advantages, the disadvantages were also
becoming plain, and a number of those countries are now supplementing,
or planning to supplement, their principal approach with a little of the
other.  For instance, the National Library of Sweden, while firmly
committed to continuing their periodic domain harvests, is also
undertaking some targetted selective harvesting.  The National Library
of Denmark started off collecting only static web publications deposited
by publishers, but now, with upgraded legal deposit legislation, will
embark on whole domain harvests.  

The National Library of Australia also is planning a whole domain
harvest to supplement its selective archive.  Funds have been allocated
for this in 2005-06.  This decision has been taken in recognition of the
fact that, no matter how skilled and dedicated librarians are, building
the collection that is a selective archive is based on certain
assumptions about what will be useful for research in the future.  For
many future research needs these assumptions, which are evolving from
and are still quite close to those that operated in the print world,
will remain valid.  But we can anticipate that other researchers will
want to take advantage of the possibilities opened up by the Web and
conduct research in vary different ways.  

Anticipating the question, why do this when the Internet Archive has
already done it, please let me explain that in fact we would be
harvesting the Australian domain in greater depth than the Internet
Archive can possibly do.  We would be drilling down and capturing
material that the Internet Archive, in its huge world wide sweep, can
only skim over.  That's the theory.  We think it needs testing and will
do just that.  

Though the cost implications of any one collecting agency supporting
both selective and comprehensive approaches to archiving are scary, it
seems that a number are starting to do just that, in order to provide
their constituents with the advantages of a combined approach.

Best wishes to all.  I'm looking forward to some great discussion.

Margaret Phillips

Director Digital Archiving
National Library of Australia
email: mphillips@nla.gov.au
phone: + 61 2 6262 1140
fax: + 61 2 6273 4322