Home → Cataloging in NC Cardinal → Appendices → Appendix P: "Needs Humans" Manual Deduplication Process

10.16. Appendix P: "Needs Humans" Manual Deduplication Process

Last Updated 06/03/2024

What are "Needs Humans" Lists?

As described in the Annual Deduplication Process Knowledge Book page, "Needs Humans" lists are lists of MARC records that look as if they might be duplicate records but are nevertheless deemed just different enough by the automated process to prevent it from automatically merging the records itself.  Instead, these matches are added to a "Needs Humans" list for catalogers to review and decide whether or not the matched records should, in fact, be merged. 

In an effort to simplify the process of reviewing whether these potential duplicates are in fact duplicates that need to be merged, we have created record buckets for each member library with bib records that have your items attached along with the other potentially matching bib record/s.  The lists were split up in this manner so that catalogers would primarily be responsible for records most relevant to them.  If, in reviewing a record (or records), they found that they needed to refer back to their physical copy, they would be able to do so, as the records in their buckets are based on those with their items attached.

Additionally, merge/overlay profiles were created for different item formats that catalogers can use when merging records so that the necessary fields from the sub bib are copied over to the lead bib.  Detailed instructions on how these profiles work and how to use them will follow below.

Merge/Overlay Profiles

The following merge/overlay profiles have been created specifically for this process, and their respective fields that they copy over are listed with them:

• Cardinal Book Dedup
  • =020
  • =035
• Cardinal LP Book Dedup
  • =020
  • =035
  • =250 (important for edition statement like "Large print edition")
• Cardinal Audiobook Dedup
  • =020
  • =024 (audiobooks often have a UPC)
  • =028 (audiobooks often have a publisher/distributor number)
  • =035
  • =250 (important for edition statements like "Abridged" versus "Unabridged")
• Cardinal Video Dedup
  • =020
  • =024 (videos often have a UPC)
  • =028 (videos often have a publisher/distributor number)
  • =035
  • =250 (important for edition statements like "Widescreen" versus "Full screen") 

Each merge/overlay profile has been created to bring over relevant fields from the sub bib to the lead bib upon merging.  It should be noted that there is no way to only bring over non-duplicate fields, so you may end up with multiple fields with identical information on the lead bib.  In such instances, you will want to go in and delete these duplicate fields.  This will be covered below.

Accessing Your Record Bucket

NC Cardinal staff will email a record bucket ID to all library staff with the necessary cataloging permissions.  You may want to write down this number for easy access later.

To access your shared record bucket:

1. Select "Record Buckets" from the "Cataloging" dropdown menu in the staff OPAC:
   
2. Under the "Bucket View" tab, select "Shared Bucket" from the "Buckets" dropdown menu:
   
3. In the popup window that appears, enter your bucket ID in the "Bucket ID" field and click the blue "Load Bucket" button:
   

Reviewing Your Record Bucket

To review your shared bucket:

1. Click on the "Deleted?" column to sort the records according to whether or not they have already been deleted (records in your bucket may already be deleted by the time you review them because someone else was looking at the same record/s in their bucket and already merged the matching records in question, thus deleting the sub bib/s):
   
2. You will want to click this column until the records that have already been deleted are listed first:
   
   
   
3. If any records are deleted, select them by checking the box next to the title:
   
   
   
4. Then, by either right clicking on the selected records or clicking the "Actions" dropdown menu, select "Remove Selected Records from Bucket" from the list of options:
   
   
   
5. Once you have removed the deleted records from your bucket, you will then want to click the "Fingerprint" column to sort the records according to that criteria:
   
   
   
6. This will organize duplicate records so that they appear in the list one after the other (though it should be noted that there may be some outliers, in which case you may still find other matching records elsewhere in the bucket):
   
   
   
7. Now you are ready to begin comparing and merging records.

Comparing and Merging Records

The Cataloging Knowledge Book provides instructions for comparing MARC records and how to merge them, and you will want to follow these guidelines as you compare the records in your record bucket.  You can find this information here and here.

To compare records in your record bucket:

1. Select the matching records and then, by either right clicking on the selected records or clicking the "Actions" dropdown menu, select "Merge Selected Records" from the list of options:
   
   
   
2. This will bring up the "Merge records?" window:
   
   
   
3. Select the appropriate merge/overlay profile from the "Choose merge profile" dropdown list.  You will want to make sure you do this first before selecting a lead bib, because once you select a lead bib, the MARC fields from the sub bib will automatically be added to the lead bib.  You do not want to have the incorrect merge profile selected when you select your lead bib.  In the example below, two large print book records are being compared, so the "Cardinal LP Book Dedup" merge/overlay profile is selected:
   
   
   
4. Compare the records using the criteria and guidelines linked above at the beginning of this section and, if they are a good match, select a lead bib by clicking the "Use as lead record" button:
   
   
   
5. After selecting the lead bib, you will notice that the relevant fields from the sub bib have been added to the lead bib:
   
   
   
6. You will notice the duplicate "=250 \\$aLarge print edition." field in the example above.  You will want to remove this duplicate field before merging, and you can do this by clicking the "Edit" button above the record:
   
   
   
7. This will open the flat text editor like when you are editing a MARC record normally:
   
   
   
8. From here you will delete the duplicate field and click the blue "Save" button.
   • Important Note:  If you edit the lead bib and save, but do not merge the records for whatever reason, the edits you made to the lead bib will still be saved and will now be reflected in the bib record in the catalog, just as if you had edited it normally.  Before you make and save any edits to the lead bib, make sure you are going to merge the records after.
9. Now you can merge the records by clicking the blue "Merge" button at the bottom of the window:
   
   
   
10. Once the records have been merged, you may remove them from the bucket by selecting the records and then, by either right clicking on the selected records or clicking the "Actions" dropdown menu, selecting "Remove Selected Records from Bucket" from the list of options:
    
    
    
11. If, however, after comparing the records you have determined that they are not a good match and should not be merged, you should still remove them from the bucket.  That way, if multiple people are working on the same bucket, they will know that those records do not need to be reviewed again.