OPEN_INVITE

============================================================
Interested in the forensic processes used to bring digital evidence to court?
Is you forensic process and software defensible?
And, in your cyber security role, do you know how reliable your software is? Will you be able to defend its operation.

KSU: Kennesaw State University has gratiously allowed me to set up this tool-testing process as an in-person lab.

The "in-person" hands-on lab will be hosted
* Date: Saturday, Nov. 9, 2024;
* Time: 08:30 - 1:00PM
* Room: Room 462;
* Location: KSU Center, 1000 Chastain Rd NW (off Busbee Drive/behind Cracker Barrell), Kennesaw, GA 30144. This is not at the main campus.

This hands-on event will run from 8:30 AM to 1:00 PM (Check-in will begin at 8:00 AM)

For those wishing to attend, read about the class here on the KSU website, or contact chollin2@kennesaw.edu directly to register.
If you wish to to receive more detailed administrative information, or to register for the session,
email carole at: chollin2@kennesaw.edu
and for more technical information email me at dm@dmares.com

Session discussions will be conducted relating to developing test data and basic testing of the software tools you use in your forensic analysis of seized computers, and the tools you may need to use in your role as cyber security expert.
Discussions will include the testing of software you may use to accomplish the basic steps of: catalog/list, hash, copy and zip files for evidence. Whether it be for administrative, forensic/legal processes, or in your role as cyber security expert in the organization. Why you should verifying the forensic reliability and accuracy of the software used and tested.

HOPEFULLY, You will test yours and other software to see if it meets simple evidentiary requirements that might be challenged in court or civil processes.

An alternative to this live, in-person session is a challenge you can take at your own pace, in your own location. See this link for details.

For those who wish to go where no-one has gone before. Visit this page and test the software yourself. DAH!!!.

   sweet/suite talk          about executing these tests using a suite
   programs tested          contains a list of the programs you may want to test.
   download test files     this section lists test files and test cases to download
   the test environment     setting up the test environment on your computer.
   testing section.            contains actual test requirements and steps of the tests.
   summary                    of the requirements you need to test for.

I have set up this testing process so you can practice your forensic skills and determine if your stand-alone forensic catalog/listing, copy, hash, zip/unzip/restore software provides true full catalog/list, copies and hashes of all the files and does NOT alter in any way the original files content or meta data. (dates, times, sizes).

If you think about it, (I know thats hard to do), regardless of what type of investigation you are performing, and what type of digital evidence is involved, the ultimate evidence integrity, and ability to show, maintain and prove its integrity is important. And I think (there I go again, thinking), that the items or processes described here could be an important part of almost any evidentiary preservation, integrity and evidentiary defense issue. So no matter what your investigation, the ability to properly and completely catalog the files within the evidence tree, the validation (hashing) of the evidence files, the forensic copy of the evidence for analysis, and saving (zip/retention) of the digital evidence is of concern. So get to it.

SUITE TALK

DON'T BOTHER USING A 4-STAR (****) MULTI-MEGA-DOLLAR SUITE ON A FULL BIT IMAGE, AS MOST SUITES ARE DESIGNED TO PROCESS FULL BIT STREAM PHYSICAL "IMAGES", AND WHEN PROCESSING FULL PHYSICAL IMAGES WILL GENERALLY PASS THE TESTS.

SUITES, AGAIN GENERALLY HAVE THE CAPABILITY OF PROCESSING DATA IN THE FOLLOWING MANNER:
A. FULL BIT IMAGE CAPABILITY
B. PROCESS A LOGICAL DRIVE LETTER (IE: D:)
C. PROCESS A TREE/FOLDER LEVEL (IE: D:\SUSPECT_FOLDER)
D. PROCESS A SINGLE FILE (IE: D:\SUSPECT_FOLDER\PORN_PHOTO)

HOWEVER, TO KEEP THE TESTING COMPATABLE WITH MOST STAND ALONE PROCESSES, CONSIDER THAT YOU ARE
1. AT A SUSPECT LOCATION, SUCH AS A CORPORATION OR SUSPECTS HOME, AND
2. THE COMPANY OR SEARCH WARRANT ADVISES YOU CAN ONLY CAPTURE/PROCESS THE SPECIFIC FOLDER FOR WHICH THE EVIDENCE RESIDE.

THIS SITUATION IS THE ONE WHICH WOULD BE IDENTIFIED AS C: OR D: ABOVE, AND THUS WOULD NOT ALLOW FOR A FULL BIT IMAGE CAPTURE OF MORE DATA THAN THE SPECIFIC FOLDER THE EVIDENCE RESIDES IN.

THE TESTS ARE DESIGNED TO TEST THE SOFTWARE CAPABILITY AT THE FOLDER/FILE LEVEL, AND AS SUCH ASSUME YOU ARE PERFORMING THESE TESTS ON LIVE SUSPECT MACHINES WHICH NORMALLY WILL NOT ALLOW INSTALLATION OF SUITES, SOFTWARE OR WRITE BLOCKERS. SO YOU MUST RUN YOUR SOFTWARE FROM A STAND ALONE LOCATION.

IF YOU WISH TO TEST THAT PART OR SECTION OF A SUITE WHICH HANDLES THE INDIVIDUAL TREE OR FILE, FEEL FREE. THE SUITE PROCESS TO TEST IS TO SEE IF THE PROGRAM CAN PERFORM THE TESTS AND PRODUCE TRUE FORENSIC RESULTS. VARIOUS FORENSIC SUITES POSESS THE SAME CAPABILITY. (ONE EXAMPLE: FTK-IMAGER and MAGNET-ACQUIRE: which are "stand alone" programs can capture/copy the original file structure to its own internal format, and then restore it to your work location. Isn't this technically a copy process?)

IF YOU CHOOSE TO TEST A SUITE (OR OTHER PRE-INSTALLED SOFTWARE) FROM YOUR MACHINE, DO THIS. HAVE THE TEST DATA ON AN EXTERNAL THUMB DRIVE. THEN SEE IF THE SUITE PROCESS TO CATALOG FILES, COPY, HASH, ZIP/UNZIP/RESTORE CAN PROCESS THE ORIGINAL EVIDENCE LOCATED ON THE EXTERNAL THUMB WITHOUT CORRUPTING THIS ORIGINAL EVIDENCE WHILE PERFORMING ITS DESIGNATED EVIDENTIARY TASK.

Also, regarding the processing of the test files provided by suites. Remember, we are testing the ability to provide complete and accurate list of files within the evidence tree, complete and accurate hash, and forensic copy, zip/unzip/restore of all the files located and identified within the evidence location. Most suites will be able to perform these tests. For instance: If you think about the suite process of finding a suspect file on a suspect drive (the evidentiary thumb drive), copying it or saving it to a forensic drive for transport from the original source to your analysis location, then restoring it at your analysis location, isn't that technically copying the suspect file from point A (suspect) to point B (analysis/evidence location). So suites do copy, and some, also perform a source zip, and destination unzip. They may not call it that, (usually its called "ACQUIRE") but the process and effect is the same. So use a suite if you wish, just operate ONLY at the folder/file level. No physical bit/sector analysis allowed for these tests.

Also, and this is a big one for the suite people. These testing requirements DO NOT expect that you will be looking at or testing your product for items NOT talked about in the tests. Meaning, the tests do not want you to consider whether the program displays what sector the file begins at, access rights, other master file table contents, or what low level meta data the operating system is maintaining for that file. That is information that is too low level and possibly too technical to consider and be part of these tests. Remember, the test are only to see if you can catalog/list, hash, copy, zip/unzip the files in a forensic manner that will not alter items which a "normal" defense attorney might target.

JUMP directly to the section explaining the steps involved, and to the download link found in step #3 (C. PROCESS A TREE/FOLDER LEVEL) of the steps to follow for this test. Then go to the actual testing section.

The file structure of the evidence provided is designed to stress test any of the programs you may be testing, and will possibly point out some forensic and thus evidentiary shortcoming of some of the software. For this reason you should conduct these tests on an NTFS file system. Many of the tests make use of the capabilities of files located on NTFS file systems.

You are encouraged to use as many of the more popular and regularly recommended programs you hear about which "supposedly" produce true and accurate forensic results. I guarantee that if you test more than about 5 programs in each group, you will find that over 90% fail one or more of the tests I have set out. As an added incentive, one of the files contains an encrypted version of the spreadsheets I used, fully filled out with program names, and the failures I detected. If your forensic software can find this spreadsheet, feel free to examine it, and let me know if your results are different than mine. If you can't find it, then you have missed important evidence in your case.

Just to induce some curiosity, here is a sample (NOT INCLUSIVE) of some of the "forensic" catalog, hash, copy and "zipping process" programs that I have seen recommended over the years, and I have tested some, but not all. Some are segments of full suites which I isolated the "list", "hashing", "copy", or "zip/compress" process/segment, others require payment, and I may not have bought that one(s) (I'm cheap). But it is up to you and your curiosity to get them, and see which pass my tests. As I have said, over 90% of them fail one or more of the tests outlined below. This is NOT a complete list, and the zip process was also considered a "copy". Again, not all of these were tested because some required payment. I'm not about to pay for those. If you see your software listed, be sure to at least test your version.

Excuse Note:
Most versions of these programs were tested in the 2020-2021 time frame. I suspect that if the authors of these programs performed the same tests as I outline here they might find where to improve their product. For this reason, if you test current versions of the programs listed below, you may find they now pass all the tests.

Below is a list of executable files which are associated with the categories of the testing sections/requirements listed below. You MUST download these files and have them ready on your computer (either on an NTFS thumb drive, or a subdirectory of your main drive). These files contain all the test data and additional software which you will test on your WINdows machine. If you come with a MAC or *IX, your on your own.

During the class session, only one password at a time will be provided for the first three items. This is to ensure you complete each step before going to the next. If you wish the passwords for the software sections listed below, (catalog, hash, copy) let me know. But it is probably easier for you to download seperate programs in each catagory and test each as you download the current version.

The tests/executable contents will be processed during the session in the order listed here. In other words, process the WAREHOUSE12 data set first, then the CATALOG_EVIDENCE and finally the main tests are for the DEMO_FILES set. Each one hopefully will cause you to get more involved in the testing process.

   _WAREHOUSE12.exe CASE     Sample small case to work on to see if you are up to the challenge. This is step 1.
   _CATALOG_EVIDENCE.exe      Sample evidence and file/directory cataloging software to test. This is step 2.
   _DEMO_FILES.exe                  Final and complete. Tool testing, Main "Evidence" files for the hash, copy and zip testing.

The following contain sample programs I have found and am providing them for you to test in the appropriate category. However, it would be more beneficial if you were to find your own particular set of files in the catagory and test them yourself. This way you will have a personal stake in the testing, and have downloaded the most recent version of the programs in the catagory which interest you the most.

   _SOFTWARE_CATALOG.exe Programs to test the file catalog/listing process.
   _SOFTWARE_HASH.exe Programs to test in the hash tool tests
   _SOFTWARE_COPY.exe Programs to test in the copy tool tests.

Follow these steps to set up your testing environment, then proceed to the actual testing section.

Since we are talking about testing your software, wouldn't it be nice to have a process and/or other software that will in fact determine and test if the software we are testing did what it was supposed to do. What I'm trying to say, is that before you test any of your software, you must find other software, and develop a process which you feel comfortable with that will use your known software and process to test and confirm that the software you are testing is doing its job. A circle roundtable.

So, actually the first test is to find software that can count and catalog or list the number of files which are in the test environment. Otherwise known as the catalog test. This is actually the first test, TEST 1. You must find and develop a process that will determine if the number of files the software you are testing did in fact find X number of files. You must find software which can accurately determine the meta data that is being tested (ie: file dates) and accurately report when your tested software didn't perform correctly. So the first thing is to find programs and develop a process which confirm the process you are testing. (CATALOG_EVIDENCE section) Chicken and egg???

return to top of page

These tests do not attempt to govern how/why you perform such processes or tests, or how you defend those processes or programs in court. That is for you, your organization, and your attorneys to decide. I only want you to see how different (recommended) programs may operate in your forensic and evidentiary process.

Before doing any of the tests, you must set up your environment which will allow you to see the problems which are encountered when conducting the tests. This environment setup is designed to stress test the programs ability to process the evidence in an NTFS file system. Your actual investiation environment might be different. But which will the defense challenge you on?

First: Set up a thumb drive to receive the test files. The file system MUST be the NTFS file system because some of the tests take advantage of the NTFS capabilities and software failures in that arena. The total size of the data is only about 3 meg in size, so it will not take up too much room. However, if have downloaded the SOFTWARE groups then an 8G thumb is recommended to hold all the software provided. (A clean directory can be setup on your hard drive, but the provided batch files for testing and confirming will only work from the root of the tree.)

Second: Make sure that your computer has last access update turned on. Some evidentiary tests rely on this setting/capability of the NTFS file system. You never know when the last access date update may become an integral piece of evidence. So your suspect/corporation may default the last access to on. To do this, make sure the registry keys are set as shown here: set it to 0. (WIN10 has other options, but the 0 is what we need)

Third: Depending on which stage of the tests you are in, download the appropriate file from the list shown above. The order of download should be:
   Prepare yourself: A WAREHOUSE12 case        Sample small case to work on to see if you are up to the challenge.
   www.dmares.com/pub/nt_32/_CATALOG_EVIDENCE.exe    Sample evidence and file/directory cataloging software to test.
   www.dmares.com/pub/nt_32/_DEMO_FILES.exe for all the tests except the cataloging test.

Be sure to read the __README files.
If the file is not there, please call, as I am constantly updating them, and may have forgotten to upload to the site.

Fourth: Once downloaded the encrypted executables, run the following command to extract the files. (Assume you running from a clean thumb drive F:). The command has to be EXACT, else the files will not extract correctly. (P.S. Some more robust virus checkers will find small signatures in the executable and claim it is a virus. Ignore that, as the findings are only bits associated with some esoteric virus')

_WAREHOUSE12 -s2 -ts+ -tp+ -os -pcall_for_password
_CATALOG_EVIDENCE -s2 -ts+ -tp+ -os -pcall_for_password
_DEMO_FILES -s2 -ts+ -tp+ -os -pcall_for_password

The _SOFTWARE_HASH and _SOFTWARE_COPY files mentioned above contain various free/shareware software I have downloaded and tested. I have made them available so you don't have to download each one as you need to test them.

There is a more intensive test and data set described in this document here: My forensic software test challenge.
It basically tests the same criteria, but with some test software provided. Also, if you wish to use this version, contact me at dan at dmares.com for the specific password.

Fifth: Depending on the version of the test data, and the section you are currently testing after the cataloging session, either in the PRIVATE directory, or within the HASH and COPY software sections, are two (empty) spreadsheet shells which I have used for my own testing. When seeing the categories tested, you will know exactly how to proceed with the spreadsheet.

TEST_REPORT_COPY_empty.xlsx
TEST_REPORT_HASH_empty.xlsx

Feel free to use them as templates and guides to be filled in when you test each piece of software. As you can see by the number of blank lines in the spreadsheets, I have tested over 20 stand-alone packages in each category. And I hate to say that over 90% fail one or more of the appropriate columns which in this test environment might hold evidence.

The tests (catalog, hash, copy, zip/unzip/restore) which you perform cover areas which are routinely used in computer forensics, evidence presentation and preservation.
But first, check out these articles as a background:

Now that you have read the articles, have set up your system as NTFS, with last access date update, and have discussed the uses of the processes and files, you need to begin testing the various programs which claim to perform these "forensic" tasks. The order for which to do this is:

The requirements/tests in this section reference the sample folders of D1-D4. They are there for you to play with and practice. After you are comfortable with the setup and requirements, you may proceed to process the evidence in CATALOG_EVIDENCE\FILES and in the SOURCE1 and SOURCE2 directories as if they were original evidence, not to be corrupted. HA HA.

The folders/files you extracted, except for the PRIVATE directory should now all reside in the root of the thumb drive. In the D1, D2 (play) folders are just over 80+- files (if I gave you the exact number that would spoil all the fun). The tree structure is comprised of "normal" files, and some alternate data streams, (one) encrypted containing my original test results, no hidden, no deleted, or other weird stuff. Just live files with data streams which you might encounter in day to day investigations.

One of the files included contained a hash inventory of the files in the Dx tree. However, since the Dx tree is constantly changing, this list may become outdated by the time you read this. Because I made the hash inventory prior to building the data structure, I think the inventory may be lacking one or two items. But they/it is an administrative file, and of no consequence. Take it upon yourself, at some point, identify the file, and see if your data matches its contents. Or more better, create your own inventory of the evidence. What a novel idea.

Now: Mr Phelps, should you accept this assignment, here is what you should do.

Assume at all times, this is live data you are working with. The owner/company/court will not allow you make an image, and you must perform these initial processes on their live server system. You will hash, copy and restore your copies as if the files are all original evidence. Because of this requirement of working on a live suspect system, you CANNOT use a write blocker on the thumb drive. Remember, this thumb drive is source evidence at the suspects computer in a private firm which you don't have permission to alter.

Since it is a small amount of data, your software process should be able to handle the situation. HA HA. You can use the Dx series of directories as initial test areas. Their content is less, and allows you to play in that sandbox before running your software against the SOURCEx folders which are the "original" evidence. Alter, change, miss any of the evidence, and the defense argument is won because you missed either incriminating, or exculpatory evidence. After you have built up your confidence, proceed to process the evidence in SOURCE1 and 2.

This www.dmares.com/pub/nt_32/_WAREHOUSE12.exe executable contains less than 100 files with a simulated case from a warehouse. Thus the case name WAREHOUSE12. When it expands onto your NTFS file system you will see two directories in the root directory called WAREHOUSE12. One directory is called ADMIN which contains a file: REQUIREMENTS.docx. This is a summary of the case, and questions you will need to find the answers to. The 2nd file in the ADMIN is called: WAREHOUSE12_TEST.EXE. This is an exe which you will run from within a CMD window. You will answer the questions (similar to the requirements found in the docx) and take the test after finding the evidence described in the document.

The other folder is named EVIDENCE. Guess what, this is the evidence of the case. This EVIDENCE folder contains the case evidence which will hold the answers to the questions asked in the document. If you fail the test, then you probably aren't ready to proceed with the rest of the testing.

First Forensic Testing Requirement: CATALOG/LIST files within the evidence tree

As stated before, the capability to properly catalog or list all the files within the evidence tree should be a major consideration. After all, if you don't know how many files, or where they are located, how can you testify to the integrity of the evidence.

This first test used the files within the www.dmares.com/pub/nt_32/_CATALOG_EVIDENCE.exe. When this exe is expanded, you end up with CATALOG_EVIDENCE/FILES directories to process. You should find a program that can properly identify, catalog, list ALL the files within that tree. Then once you have satisfied yourself that you can in fact catalog all the files, there is an exam in the CATALOG\ADMIN folder which you can take and see if you have in fact identified all pertinent evidence within the CATALOG_EVIDENCE/FILES tree. So here is what you need to do:

The final forensic tests come after you download _DEMO_FILES.exe which contains all the remaining evidence directories which you will use to test your hash, copy, and zip software.

Any requirements/processes described here are referencing the test directories of D1-D4. Once confident, re-do the processes using SOURCE1, as your safety net, and SOURCE2 as the original evidence. The safety net is there to restore SOURCE2 when you corrupt the original evidence in SOURCE2.

Now: first things first: Practice with the Dx folders. Hash all the files within the D2 directory and confirm your hashing software finds and processes all the files (60+-). This will also by default, produce a sort of catalog of all the files in the suspect directory. Wouldn't it be nice to have a hash inventory/catalog list of all the suspect data?

These following items are what your hashing should accomplish, or NOT.

Failure to properly complete A thru C of the above might be a valid evidentiary challenge. At the least, you may have missed valuable evidence.

Don't forget, if your program(s) alter and do not reset the last access date, additional testing will be corrupted. AND: you may have to testify why you altered evidence. So you must figure a way to reset the original evidence before the next test.

Once you have played with and tested the Dx folders, its time to put your copy process to the real test. Now you will use the SOURCEx folders. Copy SOURCE2 original evidentiary location to a working directory (SOURCE3) of your choosing. This copy will serve ultimately as your working data back at the office. Any alteration of the original evidence in the SOURCE2 folder, or incomplete copy to your working destination is unacceptable.

Failure to properly complete A thru D of the above might be a valid evidentiary challenge. At the least, you may have missed valuable evidence.

Don't forget, if your program(s) alter and do not reset the last access date, additional testing will be corrupted. So you must figure a way to reset the original evidence before the next test.

Finally, zip the (hopefully still) pristine evidence in SOURCE2 and restore/unzip it to an empty SOURCE4 folder, or any emtpy folder of your choice.

Then see if the zipping process properly found, captured, and restored/unzipped all the files while maintaining all original and destination file dates, and tree structure.

This zipping/unzipping process could be considered the step at which you have identified appropriate evidence, and wish to zip it for distribution to the reviewer (ie: prosecutor, manager), OR you are preserving it for retention in the evidence safe, for which it will be preserved for posterity and "hopefully" unzippped properly at a later date with all meta-data in-tact..

Quick FYI: during the zip/unzip process, I tested three of the more popular zip/unzip software. And found only one passed the tests.

You will be surprised at the results of the "forensic" utilities if you conduct your tests properly. Remember; fill in the results in the spreadsheets found in the PRIVATE folder.

Reminder: All three tests above assume:
SOURCE1 is a safe copy of the evidence to restore in case you mess up the evidence.
SOURCE2 is to be considered your original evidence to work with.
SOURCE3 and 4 should start out as empty work folders to place the copies and unzipped evidence to.
Initially, SOURC3 and 4 may have dummy place holders in them so the initial extraction creates these folders. Delete those place holders before dropping files into these areas.

In summary: After you have finished the catalog, hash, copy, and zip test. These are the four items that you will test for.
The two spreadsheets will give you an idea on what I was really looking to discover and/or prove in my tests.
I tested for these failures, and found many software programs (over 90%) will fail in one or more.

1. Last Access altered:
For all the tests, see if the process, (hash, copy, zip) alters the original last access date. If any of these actions alter and do not reset the last access date of the source, it could be argued by the defense that you altered evidence. Or at a later time, could not show the original last access date. Obviously, the owner of the data has last access update must be turned on in the registry. If you corrupt the original dates, find some way to reset the dates. Except for a few modified during the setup, which was in june 2020, Most dates on test files have been set to: 2019... or 2020-01-01:12:34:56:789 GMT

2. Long Filenames properly process > 255
In all the tests (again, hash, copy, zip) see that all the programs can find and process files with long filenames > 255 characters. In my tests, some programs saw them but could not access them. Another point to consider when "restoring" the zip contents is that some programs I tested turn Long Filenames into 8.3 filename/paths. Just try to explain that to the defense. A defense attorney's wish.

3. Restore/maintain both Source and Destination MAC dates.
In the hash tests, make sure the original MAC date, especially the last access is not altered. In the copy and zip/restore, make sure the source and destination/restored file dates/times were reset to the original. Again, if you are producing reviewable evidence, you want to maintain as much meta data as possible to eliminate defense arguments of alteration.

4. Alternate Data Streams. (ADS)
And, finally: with all the hash, copy, zip/restore processes, makes sure the programs can see and hash, copy, zip/restore any alternate data streams associated with the evidence. For instance, did you know that Firefox creates alternate data streams showing the source URL of a graphic when it is saved to disk. This might be important in a pornography case, or virus case. Alternate data streams can contain a lot of hidden evidence, including passwords that may be missed if they are not copied or retained.

Any, or all of the above factors could be ammunition for a defense attorney. And should be considered a minimal requirement where possible.