Data exchange
This section includes a summary of the various enhancements to the import and export processes that are included in OLIB Service Pack 12.
WorldCat data sync enhancements
When using the WorldCat data sync process, “response” files are produced that contain information about the results of the data sync. Improvements have now been made to cater for large response files of several thousand records. A new field has also been added to make it easier to view the files.
.
The new attribute - Discovery System Report - in the Titles domain allows you to view the response details.
This shows each line of the report from data sync of the current Title record you are viewing:
This list attribute shows the date and time of the processing of the response, together with the OCLC Number, the severity and message information.
The new list attribute needs to be added to the Titles layout using Layout Manager. It is suggested that the attribute is placed close to the OPAC Status information.
Here it is on the OPAC sheet of the new RDA layout:
To view the WorldCat responses
- Go to Titles and retrieve the relevant record.
- In the sheet containing your OPAC information, review the responses in the Discovery System Report field.
- Make changes to the Title record if required, depending on the severity / message.
- When you tick the box to select a response, you can use the Delete link for that line. This allows you to remove each line once you have corrected it in the catalogue record.
Bibliographic Exception Detail file
This contains the local system control number and the OCLC control number of records that OCLC was unable to match to records in WorldCat
Format of file name: metacoll.[user-symbol].datasync.[Collection ID].D[date processed].T[time procssed].bibdetailexcpt.1.mrc.txt
Example file
name: metacoll.OCWMS.datasync.1002106.D20160729.T082400.bibdetailexcpt.1.mrc.txt
The bibdetailexcpt file includes Severe (logical), Critical (structural) and Minor responses. Titles that have either Severe or Critical responses will be marked Rejected.
During the processing, titles may be marked as Rejected for the following additional reasons:
- More than one OCLC Number has been supplied for a single title*
- The same OCLC Number has been supplied for multiple titles
- The OCLC Number supplied is already assigned to another title
- The title has been archived
In order to prevent the incorrect update of a record to a new status when processing a bibdetailexcpt response file several days after the file has been generated, the status will not be updated if it was changed more than 2 days** after the generation of the bibdetailexcpt file, unless the update is simply to “Sent to WorldCat, Holdings Pending”. This will ensure that records which have been corrected, re-sent (and possibly received, holdings updated and processing completed) are not rejected due to an error that has been remedied.
- Processing and validation in this respect will occur only within blocks of 1,000 lines of the response file. If a subsequent OCLC Number for the same title appears much later in the bibdetailexcpt file, this will not be reported as an error.
** 2 days is interpreted as follows:
- The date in the response file includes no time element and is therefore treated as midnight (00:00 hours)
- Daystart may have sent the file at, for example, 01:30 BST, and the response file may be generated, for example, 90 minutes later at 03:00 BST
- The data sync processing will therefore report this as 22:00 EDT the previous day, without the time element
- The date and time of the current status of the title (the date and time it was sent by daystart) is compared with midnight 2 days after 00:00 on the date in the bibdetailexcpt file in order to determine whether or not to update the status.
Author Matching during import
During OLSTF and MARC imports, OLIB attempts to match on the author records.
The author matching process helps to achieve accurate author matching during imports and to prevent duplicate author records if possible for libraries who do not use the initials field in the Authors table. The process is the same for OLSTF and MARC imports.
During normal usage, you do not normally need to visit or maintain this information. However, it could be useful to ascertain why an imported record has mismatched an existing author. You can then amend the match value to avoid future mismatches.
Author initials generation
During the import, Author initials are now not generated if:
- your OLIB system does not include the initials field on any authors layout
- the OLIB Defaults, under the Cataloguing heading set the Prevent Generation of Author Initials field to Yes.
Match values
During imports, OLIB generates and maintains a list of match values against each author. You can view these on the Names layout introduced in OLIB 9 Service Pack 12:
Import Matching fields:
During the import
OLIB's author matching follows this process during imports:
- The author matching process first checks to see if ORCid, VIAF or ISNI are present on the incoming record. If one of these match an existing author, any further processing is ignored
- If not present, the import will generate a Match Value from the incoming data and attempt to match this with a Match Value for an existing author.
- If no match is found by Match Value, the import will attempt to match on the surname, forename, initials etc... in the author record itself
- If a match is found, the process adds Match Values to the author it has matched with
- If no match is found, the process creates a new author record in OLIB, with the generated Match Value
- If this causes a duplicate record, you can edit the Match Values in the Authority Control Details above for your preferred author record.
View a match value
From the Import Matching field, you can view any existing match values - if there has been a previous import since OLIB 9 Service Pack 12.
- Go to Cataloguing> Names and search for the author.
- Go to Authority Control Details> Import Matching field. For the current author, this will show information such as:
For any future imports, the Match Value is the value that will be used in preference for matching. The import processes will first attempt to match exactly and then attempt a case insensitive match on just the letters, numbers and spaces in the field.
You can see who the new value was generated by and when. In the Source field where an OCLC#, Import RCN or Batch number is shown, this match value was generated by an import.
Value generated by existing author record
If the import author matching does not find a successful match for an author, then OLIB matches on a value generated from the existing author record. To view what this would be:
- Go to Cataloguing> Names and search for the author.
- On the Authority Control Details sheet > Import Matching click New. OLIB generates a match value based on the author record:
- You can manually edit the match value in the list box, or it can be removed if you check the box and click the Delete link below the list. You can click New to create an additional match value. (The import process will try to match with the first value and if not an exact match, then try the next value).
Note
For MARC imports, the match value will be generated from all but the Relator Term and Relator Code subfields. The subfield codes other than $a will be included in the match value.
For OLSTF imports, the match value will be generated as Forenames Initials Surname for personal names, as seen when generated by New above. In addition, there is an OLSTF import setting that permits the dates associated with the author to be ignored as part of the matching. If this is then enabled, the search will be carried out as a secondary match after the generation of the match value with dates, and a match value including dates will be added to the new or existing author record.
WorldCat data sync communications
The WorldCat metadata API process now records its communications to aid diagnosis of problems.
This is recorded in the worldcat_exchange_diagnostics table. Note this is for OCLC staff Support usage only.
To improve diagnostics, the following additional changes were made:
- a flag on the Output Format allows the one-off resending during the next daystart of records that have previously been rejected – Export Generation sheet:
- a refinement option for OCLC symbol has been added to the Locations domain. This refinement is not available in the list of standard refinements by default but can be added by setting the Enable Filter flag on the attribute record.
“WorldCat Batchload” changed to “WorldCat data sync”
References to WorldCat Batchload have been corrected so that they now refer to WorldCat data sync.
For example –
- Data Exchange Reference Data> Output Format Details (Type dropdown: WorldCat data sync and metadata API)
- Data Exchange>Import Batch record – Batch Type: WorldCat data sync CrossRef:
MARC Export handling of invalid characters
The MARC export no longer exports a record if the MARC record it generates includes critical invalid characters.
This was required because WorldCat data sync was rejecting the whole file if the first record (and only the first record) had invalid characters in Leader/06 and Leader/07.
The following data elements are now validated to ensure that they only contain a valid character:
- Leader/05 – Record Status
- Leader/06 – Type of Record
- Leader/07 – Bibliographic Level
- Leader/08 – Type of Control
- Leader/09 – Character encoding scheme
- Leader/10 and 11 – Indicator and Subfield counts
- Leader/17 – Encoding Level
- Leader/18 – Descriptive Cataloguing form
- Leader/19 – Multipart resource record level
- Leader/20 and 21 – Lengths of directory entry elements
- Leader/22 – Length of the implementation-defined portion
- Leader/23 – Undefined
In addition, if a title record does not have a language of cataloguing, the default cataloguing language specified in Admin Client Defaults will be exported as the language of cataloguing.
New OLSTF Tags
Several new tags have been added for Export.
Tag |
Data table |
Description |
---|---|---|
PCM |
Titles non-repeatable block |
The title’s primary classmark |
CP_GPTI CP_GPSUBTI |
Copies export table |
The title record's grandparent title and subtitle |
CP_GEO_COLL CP_GEO_COLLD |
Copies export table |
Geographic Collection code and description (copies.geo_collection) |
A Titles OLSTF import tag has been added for the Collection Code in the copy record. The OLSTF export process has also been enhanced to support this tag.
Tag |
Data table |
Description |
---|---|---|
COLLN |
(Copies repeatable block in the Titles import) |
A Collection code. This OLSTF tag is for import and export |
The COLLN Collection code field is available in the Copies domain via Layout Manager. The attribute is
Collection (for layouts)co_collection