Data exchange

New OLSTF Tags

Several new OLSTF tags have been added.

Configure CSV Mapping for OLSTF Import

This information is about further optional configuration that can be used with an OLSTF import. If your source file for importing into OLIB is in CSV format, it is possible to create a mapping between the source CSV file and an appropriate OLSTF tag. The overall procedure for processing the import is the same as for a standard OLSTF Import Process.

1. Go to Data Exchange> Import Batch.

2. Create an Import Batch record as normal. Select Text File CSV from the Batch Type drop down, the screen re-draws to display the relevant fields:

Field	Description
Batch Type	Defaults to MARC Exchange format. Change this to Text file - CSV. The screen will change to the CSV Import Batch screen.
Domain	In the drop down, select Users or Titles from all the OLSTF record import types.
Description	Enter some meaningful text as this will allow you to subsequently retrieve the Import Batch for processing.
Match Flag	Select from the drop down – Users Either OLSTF Users: Match on Barcode value or OLSTF Users: Match on Secondary Code value depending on which value you are going to match records on for subsequent uploads. Using the Secondary Code value (SEC tag) with the primary key from the source system will allow changes to the Barcode value without complicating transaction, messaging or fine histories. The first tag in each record must be SEC, if matching on Secondary Code, and BAR, if matching on Barcode. Titles RN: Import Record Number OCLC: OCLC number IS: ISBN (Importing with a 10- or 13-digit value will match against the 10- or 13- digit equivalent) TNUPD: Title Number (potentially useful when the data came from this system, was massaged and is now being re-imported / supplemented) XC: Control Number (e.g. Other System Control Number)
Field Names In Row1?	In a CSV file, column headers are sometimes included as the first line, and each subsequent line is a row of data. In this case set to Yes. If the data commences in the first row of the file set to No.
Last Mand. Field	This setting is pertinent only if the incoming data contains line-breaks within the data. This is primarily relevant to Title data for Abstracts and not for User data.

Field Delimiter	Enter the character used in the file to separate the fields. For a CSV file this is normally a comma (,).
Text Qualifier	Normally for a CSV file you should enter double quotes (“). This prevents the comma in an actual field value from being interpreted as a field separator.
Qualifier Escape	Normally set to double quotes (“) for CSV. Double quotes are used as above for the Text Qualifier, therefore you would need to escape this field wrapper itself.
Character Set	Select Unicode 3.0 UTF-8, unless advised otherwise by the supplier of the file.
Record Source	Optionally select from drop down. This serves purely to help find the Import Batch.
Fields	Click New to obtain the Import Field Details window as below.

3. Use Import Field Details to create a record to map each incoming field from your source CSV file to the equivalent OLSTF tag.

Note: if any special handling of data is required, please contact OCLC Support.

Field	Description
Import To	From the drop down select the OLSTF tag that this field will map to, e.g. BAR, SEC..
Description	OLIB populates from your above choice, e.g. Identifier from Source system (import).
Source Tag	For CSV do not use.
Start Char.Pos./Field No.	For CSV enter the column number from your source file.
Field Length	This is for fixed width files only.

4. Save the details and repeat for all the fields that are required (in the batch record again, click New in Fields, enter the details above and save).

On completion, the CSV Import Batch record displays the mapping in the Fields box:

For User address information see below. Otherwise, continue to load the import CSV file as for any other OLSTF import.

User Addresses

Additional processing is required for the address information in a Users import file.

The User Import requires that the address information is presented in a separate OLSTF record. If the address information is present in the same line as the other user data, then a record separator is required before the address tags are mapped. Starting a new record also means adding the SEC or BAR identifier, too.

In order to add the separator, you must map a field that is always populated, for example the identifier:

To process this as a separator, rather than a field, select “INSROWAFTER …” from the drop- down on the Process Using line and enter “* *” in the field below. Before Saving this field mapping, click on the “>>” button to add the process to the process steps:

Immediately after this must be the chosen record matching field (in the example here it is Barcode) so that the import can identify which user this address applies to. The address tags can then be mapped, giving a field list such as:

Note that the fields are processed in the order listed on the Import Batch and not the field order on the incoming file. Note also that a record terminator is automatically added after each of the incoming lines has been processed, so a final terminator is not required to be specified here.

Tip: you can add more new fields, if you later find that a field should be added. To do this:

1. In modify mode, check the box to select a record in the list, above which the new field should be positioned.

-This is critical if you are adding a new user field mapping (such as LOC for Location) so that the field is not mapped in the address record.

2. Click New and complete the above fields as normal. In Start Char. Pos / Field No, enter the appropriate column number for this new field.
3. Save and Close – OLIB now displays the new field above the one you had selected.

Note, you can also change the numbers if required.

On completion, Save and Continue. Next, you attach the file to import and load the data as normal.

Whilst processing, in the background OLIB will populate the records for import in the Import Record Fields:

This displays a row for each record showing the data to be imported and the original data. Hover over a record to view more details.

Import / Don’t Import

You can toggle between these options by checking the box to select a record and clicking

Toggle Import/Don’t Import in the Actions drop down. The Import? column displays Yes/No accordingly.

Save the Batch record to proceed with the import. The Imported? and Imported On columns will be populated with Yes and today’s date/time when the data is loaded.

Create Import Batch for source database table

Libraries can carry out a data mapping from a column in a source database Table, to the appropriate OLSTF tag.

Note: the examples below assume that the data being imported is a User import. Using OLIB Web, create an Import Batch record as follows:

1. In the pin menu go to Data Exchange> Import Batch.

2. Click New Record and complete the following fields:

Import Batch Fields

Field	Description
Batch Type	Defaults to MARC Exchange format. Change this to In Database Transform. The screen will change to the In Database Transform Batch screen.
Description	Enter some meaningful text as this will allow you to subsequently retrieve the Import Batch for processing.
Domain	Select Users from the dropdown.
Match Flag	Select from the dropdown – either OLSTF Users: Match on Barcode value OLSTF Users: Match on Secondary Code value depending on which value you are going to match records on for subsequent uploads. Using the Secondary Code value (SEC tag) with the primary key from the source system will allow changes to the Barcode value without complicating transaction, messaging or fine histories. The first tag in each record must be SEC, if matching on Secondary Code, and BAR, if matching on Barcode.

Record Source	Optionally select from dropdown.
Source Tables	In this text box, enter a comma separated list of OWNER.table_names. E.g. olib900sp12.marc_batch, olib900sp12.data_ingest
Fields	Click New. To obtain the Import Field Details window.

3. Import Field Details: use this window to create a record to map each incoming field from your source database Table to the equivalent OLSTF Users tag.

(Note: if any special handling of data is required, please contact OCLC Support.)

Import Field Details

Field	Description
Import To	From the dropdown, select the required OLSTF tag that this field will map to.
Description	OLIB populates from your above choice, e.g. Identifier from Source system (import)
Source Tag	Not required for data ingest Table
Start Char.Pos./Field No.	Not required for data ingest Table
Source Table	This dropdown is populated from the tables you listed in the Import Batch screen. Select the table in which this column resides. The screen re-draws as OLIB populates the following Column Name dropdown .
Column Name	This is populated from the columns in your chosen table. Select a Column Name which will map onto the chosen OLSTF tag.
Process Using	Input not required. If any data does not map to a tag, contact OCLC Support for advice.

Data Setup Mapping

4. For the initial configuration, there must be a Data Setup mapping which identifies the records to retrieve. For usage of the system supplied data_ingest table, this normally involves adding several processing steps such as:

These were created as follows:

1. Select “INSERT Add data to the beginning”
   • Specify “ di_sourcesystem = 'TST' AND di_recordtype = 'USER' AND di_date_of_data >= to_date(' ”
     • Where ‘TST’ and ‘USER’ are values supplied in the table by the source system
     • Note the single quote at the end of the value, this is required
   • Click “>>”
2. Select “REPLACECHARS1 Replace all occurrences of this=>that”
   • Specify “ [!=>', ' “
   • Click “>>”
3. Select “REPLACECHARS1 Replace all occurrences of this=>that”
   • Specify “ !]=> ') “
   • Click “>>”
4. Select “SAVE Save the data for use later in this record”
   • Specify “DATA_INGEST-DataFilter”
     • This is case sensitive and will be used internally to select which records from the DATA_INGEST table to import, using the Date Last Ingested (from the Column Name field) as the date from which to select records
   • Click “>>”
5. Select “MAPLIKE Map Data Values using “like””
   • Specify “%=di_date_of_data”
   • Click “>>”
6. Select “SAVE Save the data for use later in this record”
   • Specify “DATA_INGEST-DataSort”
     • This is case sensitive and will be used internally to determine the order in which the records are processed from the from the DATA_INGEST table.
   • Click “>>”

 

5. Save your field mapping record. OLIB returns you to the Batch record. Click New and repeat to map all the fields that are required.

Address Information

The User Import requires that the address information is presented in a separate OLSTF record. If the address information is present in the same line as the other user data, then a record separator is required before the address tags are mapped. Starting a new record also means adding the SEC or BAR identifier, too.

6. In order to add the separator, you must map a field that is always populated, for example the identifier.

To process this as a separator, rather than a field, select “INSROWAFTER …” from the drop- down on the Process Using line and enter “* *” in the field below. Before Saving this field mapping, click on the “>>” button to add the process to the process steps:

Immediately after this must be the chosen record matching field (in the example here it is Barcode) so that the import can identify which user this address applies to. The address tags can then be mapped, giving a field list such as:

Note that the fields are processed in the order listed on the Import Batch and not the field order on the source table. Note also that a record terminator is required at the end of the processing.

7. When all the fields are completed click Process Data:

This populates the Records to Import and the Import Record Fields box.

It displays a row for each record showing the data to be imported and the original data:

(Hover over a record to view more details.)

Edit Field Mappings

8. If the data is not going to map to the correct OLIB field, go back to Fields and simply click the link to edit the mapping in Import Field Details.

When you click Process Data again, the Import Record Fields will show which OLIB tag the original source data will map to.

9. When the original fields are all set to map to the desired OLIB tag, you can use Validate Ref. Data (there must first be data in the Import Record Fields list) This populates Reference Data Mappings with a list of the source Tag and the domain and reference data it will map to:

To change a reference data mapping, click on the link to edit the value it will map to:

Import / Don’t Import

Optionally in Import Record Fields you can also choose to include or not include a field - select the box next to the field and in Actions choose Toggle Import/Don’t Import

The Import? column displays Yes/No accordingly.

10. When ready, press the Load Data button, OLIB responds:

The data load has been scheduled for immediate processing. (Job No: nnnnn)

11. Click OK and save the Batch record to proceed with the import. The Imported? and Imported On columns will be populated with Yes and today’s date/time when the data is loaded.