Taming the NSRL Hash DB Beast Background Ive been trying to extend Sleuthkit in a variety of ways, such as by adding a table to the Sleuthit databese to capture file/mime types and improve strings extraction over that of the traditional blkls | strings > file variety. I came to the realization that elimination of files through hashing could speed these processes. This is not a new concept to me or anyone else in data forensics, but I do believe hash elimination is under used. I think this is due in large part to the unwieldy size of the National Software Reference Library (NSRL) hash database. The Problem As of January 2011, The NSRL data is distributed in 4 .iso files and contains a whopping 62,294,036 file entries. The NSRL tries to help you manage the data volume by directing you to concatenate the NRSLFile.txt files (each containing over 4.7 million rows) found in each .iso and then export only rows with unique hash values. Such ...