12/12/2011

Data Conversion Questions

I recieved this email from a NH library director and thought it is very likely a topic that others may be wondering about as well, so I am sharing my reply with the nhaisLOCAL community here.


Good morning Mary--
I am just getting up to speed on ILS and hope you can answer one of my biggest worries. It concerns the conversion of our records:
1. What does the conversion process involve?
2. Where are our records uploaded from--NHUPAC, NHAIS or directly
from us? If from NHUPAC or NHAIS, how do we keep our barcode
numbers, etc. intact?
3. Does our cataloging have to be "perfect" for conversion?
4. Is the conversion process different from vendor to vendor, or is it a standard process?
This set of questions is one of the major sticking points for this project as resolving the various issues around data conversions and data loads is a key issue to bringing down the price of a library system change.

 There are vendors who will do "data conversions" for a new system and typically the prices start at several thousand dollars and go up from there. However, what is actually being done to "convert" the records is very different depending on what the records you start with look like. Assuming all the vendors in question are reputable vendors, the process doesn't really vary vendor to vendor, but is hugely different library to library because it is driven by what your records look like when they start the process. Based on my experience with the files of records libraries send to NHAIS Services for loading to NHU-PAC some of the records in local systems contain no more data than you would find on the price sticker at Barnes & Noble. Others have very good, complete records (often downloaded from NHU-PAC). The estimate you get from a vendor to do the conversions is (in my opinion) unlikely to be accurate unless the vendor asks to see a sample of your data.

NHU-PAC records (which are the only NHAIS records there are, so I will stick with the term "NHU-PAC records") are clean, full-MARC21 records that require no conversions (which really means cleaning up the weird unreadable junk in your records and in some cases putting the data that was dropped any-old-place in the record into the tags it belongs in) and can be loaded directly into a new system. Some vendors will want to charge us for a per-bib loading fee, others may be willing to let us (meaning NHAIS Services) load the data ourselves for nothing, or for a small programming fee to set up the load parameters. This is one of the places where the nhaisLOCAL group may be able to offer savings to libraries over going direct to a vendor to set up your opensource system.

There are a couple of issues with just loading NHU-PAC records into local systems:
- There are almost no NHAIS libraries that have kept up their holdings in NHU-PAC consistently enough that what is in NHU-PAC exactly matches their actual collection. If you have, your library will have fewer issues re loading than others will.
- NHU-PAC item records don't include barcodes, they could, but they never have. For libraries that have never barcoded this is not an issue, but I think there are only a handful of these. A two step load may resolve this -- we may be able to take records from NHU-PAC and load them to the new system for a given library (so they have full MARC records in the new system), then load data extracted from whatever system they use now which should match to the MARC records already loaded and create the items with barcodes, call #s, etc. Will this cost extra? Can NHAIS Services staff do this (time, access, training)? Can a given library exract their current item data? Will the local records have enough detail to match on load to the full MARC? (If the library has used NHU-PAC records in their system the answer to the last question is yes, if they created home-made quick records or took records from a vendor or from whatever free z39.50 source they found online then the answer is maybe).
- I'm not sure what a "perfect" catalog would be. Conversions are a per-record cost, so if your catalog is full of junk you either don't really have or should have discarded years ago then you are wasting money and time by "converting" it.

One of the things the nhaisLOCAL project will account for is the dataloading and making it as inexpensive and straitforward as possible for everyone, but there are a lot of variables and issues to sort out with this.

For anyone thinking of implementing nhaisLOCAL in their library I would STRONGLY advise AGAINST contracting with anyone to do any kind of conversion on their database at this point. At implementation of nhaisLOCAL this data work will be bundled into the load/set up cost. Because it will be a group project with group pricing, you won't get a break for having done it before because what you have done may not exactly match the parameters that are needed for the loading to nhaisLOCAL and the price of that work will be based on volume of the whole group. If this cooperative price structure doesn't sit well with you then nhaisLOCAL may not be a viable solution for your library.

If your NHU-PAC holdings are out of date (ie. You haven't kept up with deleting discards from NHU-PAC) working on cleaning them up -- add holdings for what you have, delete holdings for what you don't have, and including local call numbers in the records -- would be a good step to take to prepare for an automated system. If you go with the nhaisLOCAL project or if you decide to implement a system on your own you can get a file of marc records for your collection from NHU-PAC that can be used to load into your new system (the cost to your library for this data file is $0)

12/09/2011

Pricing info delayed

I know everyone is anxious for pricing information, but it is going to be a few more days. Today, December 9th was my announced goal for the announcement, but I didn't get everything I needed to finish first done. I am hoping to have everything sorted out and everyone I need to talk to talked to before Christmas.