Digital preservation practices and processes

The stability and lifespan of a digital record depends on many variables, including the software and hardware required to render a file.

If you are planning to transfer your records to the Libraries, preferred formats are file types that are ideal for long-term preservation according to best practice.

Acceptable formats are those that can be transferred to the Libraries, but where preservation services may be more limited. Some acceptable formats may be migrated to preservation formats, while others may be monitored over time until they become obsolete.

Formats and hardware not included in the tables below may be accepted on a case-by-case basis.

Through our Forensic Recovery of Evidence Device, and other supporting hardware, the Libraries can read and image optical discs, removable magnetic storage media, SATA, IDE, SAS, USB 3, FireWire, and PCIe based SSD storage devices. Other types of media may be accepted on a case-by-case basis.

To ensure that your digital records remain usable over time and that they are in good condition when they are transferred to, or deposited with, the Libraries, it is recommended that those creating and/or managing the records review our Guide to self-preservation for advice on extending the life of your digital records.

Forensic recovery of evidence device

The Libraries use a Forensics Recovery of Evidence Device (FRED) to review digital records transferred to the Libraries for long-term preservation and access. This system is used to perform digital forensics processes on digital records and allow the Libraries to:

• Securely access digital records so that they are not altered during the review process
• Create an exact copy of the files for processing purposes 
• Recover deleted files or retrieve passwords where explicitly permitted by the creator of the content
• Review files more efficiently by generating reports about the files
• Flag duplicate files to reduce redundancies and use storage space more efficiently
• Identify personally identifiable or sensitive information that should be removed or redacted

Disk imaging and file recovery

To securely extract files without altering the original records, a disk image may be created as a working copy when media is deposited with, transferred, or donated to the Libraries for digital preservation purposes.

A disk image is a bit-for-bit copy of the original source. As such, a disk image also includes files that were intentionally or unintentionally deleted on the original source media provided for processing. Unless explicitly requested by the creator of the original files, the Libraries do not review, extract, or preserve any deleted content from the materials it processes.

Those considering transferring media to the Libraries for preservation purposes are encouraged to bring up any concerns they have about how digital forensics will be used on their files ahead of transferring their records so that preservation processes can be altered to address their concerns.

Who can use FRED?

FRED is only used by a limited number of Libraries staff who are trained in digital forensics in order to process digital records either within the Libraries’ holdings, or created by the University of Manitoba community.

Archivematica

The Libraries use Archivematica as their digital preservation system for normalizing files to more stable formats as needed, and packaging the files and related metadata into Archival Information Packages (AIPs) for long-term preservation. 

Once processed through Archivematica, AIPs are monitored to minimize the risk of data loss or file corruption, and may be further migrated to new formats over time to address technological changes. Dissemination Information Packages (DIPs) may also be generated for access and may be made accessible to users unless restrictions on access and/or use apply.

Other software

In addition to these tools, the Libraries further integrates the following software into their preservation processing work:

• AllDup
• BitCurator
• DROID
• Exactly
• FileZilla
• ForensicToolKit
• FTK Imager
• TeraCopy

Using the tools referenced in the above section, the following steps are applied to records throughout the preservation process:

• Virus scanning
• Checksum generation and validation to monitor changes in the files overtime
• File format identification
• Identifying and removing or restricting sensitive information
• Sanitizing file names to remove problematic characters
• Normalizing file formats (for example: Generating a tiff from a png file)

In addition, the following actions may be applied depending on the content:

• Re-arrangement of records to facilitate research and secondary use
• Renaming of files to improve discoverability
• Deletion of empty file directories, and/or duplicated, transitory, or low-value content
• Generation of folder directory lists

Digital preservation processes rely on a significant level of infrastructure and technological support. Within the University of Manitoba, this work is supported by the Libraries and IST. The Libraries further engage with both formal and informal networks of experts in various fields related to digital preservation, including:

• Archive-it
• Archivematica user group
• BitCurator user group
• Canadian Association of Research Libraries
• Canadian Data Curation Network
• Canadian Digital Forensics Community
• Council of Prairie and Pacific University Libraries
• New Digital Research Infrastructure Organization
• Internet Archive
• Portage Network