User Customization of the Validation tables

MARC Report version 25x supports user customization of the main validation tables used by Batch mode and in an Edit session.

If the instructions on this page are followed carefully, the user will be able to replace the validation tables distributed by the program with modifications that include new MARC definitions (as published by LC). Theoretically, the user may also add their own 'definitions' to the table.

Prerequisites

MARC Report must be installed, registered, and run at least once, and then closed.

If custom validation is still in the beta period (ends Dec. 31, 2021), then the beta activation option code must be set.

These instructions require a very good knowledge of MARC21, and at least an average knowledge of Windows and a program like Excel. One should not attempt to modify the tables without this knowledge, as a failed customization will prevent the validation module from loading at all.

Overview

1. Create folders
2. Copy the TMQ validation table
3. Make changes to the table
4. Compile the table
5. Enable custom validation in MARC Report
6. Test the customization

Two new folders will be required to support customization of the validation tables. One folder is required by MARC Report; the other folder is where you will save your changes.

First, create a folder named validation in your “Documents\MarcReport” folder. If you are not sure where your “MarcReport” folder is located, follow the steps in this wiki FAQ to find it.

For example, if your username is 'jsmith', this “MarcReport” folder would probably be named:

c:\users\jsmith\Documents\MarcReport

The folder to be created is thus:

c:\users\jsmith\Documents\MarcReport\validation

Do not add any files to this folder. This folder must be used only by the utility descibed in Step 4 below.

Next, create a folder for your validation customizations. You will use this folder for all of your customization work (as described in Step 3 below).

You can name this folder whatever you wish. For example (continuing with jsmith), you might name this folder:

c:\users\jsmith\Documents\validation-work

Do not put this folder in “Documents\MarcReport” as above, but do make sure that it is part of your daily backup routine.

Beginning with version 25x, the raw data used by our validation tables will be distributed in the MARC Report installation folder. The default location for the installation is:

C:\Program Files (x86)\TMQ\MARC Report

If you cannot find your installation folder, follow the steps in this wiki FAQ.

Three validation tables are distributed in the subdirectory '\defaults\validation' (eg. C:\Program Files (x86)\TMQ\MARC Report\defaults\validation)

The tables are dubbed: 'bib' (bibliographic format), 'auth' (authorities format), and 'hold' (holdings format). Each file is available in two formats: tab-delimited text format ('.txt'), and an older Excel format ('.xls') ¹⁾.

The naming convention for these tables includes the date that the data was produced on (201202 in the examples); thus, the distributed validation files might be named like:

_bib201202.txt    
_bib201202.xls    9178 rows
_auth201202.txt   
_auth201202.xls   3556 rows
_hold201202.txt
_hold201202.xls   1974 rows

The number of rows in each table listed above are as of MARC Report version 256.

Copy the validation table(s) that you want to customize to your validation work folder. For example, if you want to customize the bib table, copy these files to your work folder:

_bib201202.txt    
_bib201202.xls

Note: Do not move the tables. MARC Report may not work if they are not present in the distribution folder.

Open a validation table in excel, make your changes to it, and then save the table.

It sounds simple, but this will be the most difficult step in customizing the validataion tables.

The documentation for actually editing the table appears on a separate page.
Visit Validation Table Format for a detailed description of each column in the table and how to enter data for it, including examples.

After opening the validation table in excel, make sure gridlines are enabled, and that column headings remain visible while scrolling.

When saving the changes you make to the table:

1. Be sure to change the filename
2. The output file must be saved as type: Text (Tab-delimited) (*.txt)

When changing the filename, do not change the beginning tags: '_bib', '_auth', '_hold'.

Any format other than 'Text (Tab-delimited)' will fail to compile in the next step. (Note: if you are using a program like OpenOffice, an extra step might be needed to create a tab-delimited text file).

However, it does make sense to additionally save a copy of the modified table in Excel format so that you do not have to re-import it from text whenever you want to change it.

The Library of Congress typically publishes a one-page summary whenever a new MARC21 version is released. We have always found this summary to be a convenient worksheet to use when updating the validation tables. You can find these update summaries here: https://www.loc.gov/marc/status.html. You can also keep abreast of the future changes that might be coming your way by following the MARC Proposals and MARC Discussion Papers (https://www.loc.gov/marc/mac/index.html). Fortunately, the MARC21 updates that require updates to the validation tables are typically limited to a few entries for each biannual release.

IMPORTANT

Our validation tables are maintained in Microsoft® Excel®, and any changes to them should also be made in Excel or in an Excel-like program.

Take care if you import the tab-delimited '.txt' version of the table into Excel (or into an Excel-like program). When one drags and drops a text table onto a worksheet, Excel will seem to import the column and row structure perfectly–however, all leading zeroes are truncated (so that '001' becomes just '1'). This results in a corrupt table as far as MARC Report is concerned. The workaround is to use the File|Open menu in Excel to drop into the Text Import Wizard (an illustration of such an import may be found on this page).

On the other hand, if you work with the distributed .xls table instead, you won't have to deal with this problem, as all of the columns have already been defined as 'Text'.

Distributed with this version of MARC Report is a utility named valmaker.exe, which you should find in the program installation folder (described in Step 2, above). This utility must be run to compile to the customized validation table(s) created in step 3 above.

Locate this utility and make a shortcut to it. You can drag the shortcut to your validation work folder if you wish.

Note: do not copy the utility itself–valmaker must be launched from a shortcut that points to the MARC Report installation folder.

Double-click on the shortcut to launch valmaker.

In general, you fill out the form that appears, and then press the “Compile” button. For specific details about using valmaker.exe, see the Help in the utility itself.

valmaker performs three functions. First, it applies a matrix of known MARC data elements to your customized table. The purpose of this is to check that the customized validation data makes some sort of sense–a validation of the validation table, so to speak.

Second, valmaker assigns a version number to the table based on the current timestamp. This version number is added as the first row in every validation table–it may help you keep track of different validation files over time.

The third function is to compile the table into the pseudo-code recognized by MARC Report. This step both compresses (by a ratio of about 100:1) and encodes the validation table.²⁾

If compilation succeeds, a file called 'valdata.xxx' will be created–where the '.xxx' file extension represents the type of table, as follows:

Input    Compiled Output     Type of table
_bib     valdata.loc         bibliographic format
_auth    valdata.aut         authorities format
_hold    valdata.hld         holdings format

If there are no errors when running valmaker, all of the files necessary to initialize the MARC Report validation module are moved from your work folder to your MARC Report validation folder.

The last step is to start MARC Report, go to the main options menu, and open the Validation tab.

On the top right, in the box labelled “Validation Tables”, select the option labelled Custom validation ….

The program will then check to see if a validation folder is present, and if so, test whatever validation tables are found in the folder.

If everything checks out, you will be greeted with a pop-up listing the filename and version number of each validation table that was found.

If there any errors, the program will uncheck the Custom validation box and pop-up a message about the problem it found. Files that trigger error messages should be removed from the validation folder.

If any errors are reported, MARC Report will fallback to using the distributed tables–i.e., it will ignore all of the files in the validation folder, even if some are OK.

Notes

Not every validation table is available for customization. For example, the OCLC Bibliographic table ³⁾, and the MARC21 Community Information table⁴⁾, are not customizable.

Also, the “Local Validation Data” option (on the right, outside of the screenshot) is incompatible with “Custom validation”–only one of these two options may be enabled at a time. A workaround might be to add any local validation requirements to one of the customizable validation tables.

If customized validation is successfully activated, the next step is to make a test MARC record that contains all of the content designators that were added to, modified, or deleted from, the customized table. Load that record into MARC Report and test that your changes are working as designed.

For example, if you have added a new tag (as in the “Example” wiki page), test that

1. content designators you defined do not trigger an error when they are used, and that
2. content designators not defined do trigger an error when they are used
3. MARC Help is working as desired by putting the cursor on your new tag and pressing F1

Also, don't forget to test for Tag and Subfield repeatability if applicable.

If any of the testing fails, go back to Step 3 and make the necessary changes to your customized validation table, then repeat the remaining steps described above.

If, at any time, you end up with a broken validation module, go back to Step 5 and uncheck the Custom validation box. Then restart the program. MARC Report should then revert to using the default validation tables.

If you are not able to get into the program options at all: make sure the program isnt running, open the marcreport.ini file in a text editor, and find the line that reads:

PREFERUSERVALIDATIONTABLES=Y

Delete that line, save the .ini file, and restart the program. MARC Report will then revert to the last distributed validation tables, and the validation module should start working as before.