NSRL Hash Set
Latest update (as of December 2017) of a subset of the NSRL data available here.
I have taken two data sets from the NIST NSRL website and massaged the data files to produce two output files.
The original data set is from the link from "Current RDS Hash Set".
Modern RDS (link is external) (microcomputer applications from 2000 to present).
These 112 million items were reduced and de-duped on SHA which resulted in approximately
38,312,509 unique SHA values. These SHA along with their MD5 values were output to a
single fixed length record file. The format of the output file is a comma delimieted
fixed length file containing three fields: SHA, MD5, CRC
"0000002D9D62AEBE1E0E9DB6C4C4C7C16A163D2C","1D6EBB5A789ABD108FF578263E1F40F3","FFFFFFFF" "00000142988AFA836117B1B572FAE4713F200567","9B3702B0E788C6D62996392FE3C9786A","05E566DF"This should be easily massaged for input to most any program capable of handling the values. (Although i must say, I have complained for some time, that import to X-Ways is difficult if the first record doesn't contain the word SHA or MD5. ) I Leave it to the user to figure out how to get the first record adjusted accordingly.
In an effort to make life easy for others, I have then taken only the MD5 values and placed them in a data file which looks like
00000040F69913A27FF7401B8BF3CFD1 000001EEB4F15B705A97758C91F2C73ANotice again, NO MD5 header. You are a forensic expert, figure how to add the header. :-).
There is also a subset of approximately 23,598,567 items which is a unique set of SHA values from the "Modern RDS (link is external) (unique) " found on this page.
This sub-set is currently not on the website. But can be made available.
The two files mentioned above, the complete 33million SHA's, or the 33 Million MD5's can be downloaded here.SHA/MD5 file. MD5 file.
They are .7z compression because of the large data sizes. But in order for these to be downloade, I had to add the zip extension. So, if you download don't forget to rename (remove the .zip) and work with the .7z file. Remember to drop the .zip when expandind.