Choosing Appropriate Data Storage

The Feinberg School of Medicine (FSM) data storage policy uses different language to describe research data categories and makes specific recommendations about where Feinberg researchers should store their data. Contact fsmhelp@northwestern.edu with questions about this policy and data storage in general.

Many factors go into choosing appropriate data storage for your data. The best place to store your research data depends on what you will do with it.

Consider the following questions when choosing a location to store your research data.

What Policies and Regulations Apply to My Data?

Not all data storage services meet the minimum requirements for all research data. All Northwestern research data is subject to Northwestern University’s Research Data policy and Data Classification policy, which describes categories of data (levels 1 to 4) referenced in the table below. Other policies and regulations may apply.

Northwestern University Data Classification Policy Categories

* Refers only to the technical controls required to store this data type. Level 3 data and above requires other policies and procedures to be fully compliant with contractual or legal requirements they are subject to.

** Cloud services can usually be configured to satisfy most data storage requirements

Note: Using personal accounts on unsupported storage services like Dropbox is not allowed for storing Northwestern research data. Google Drive, including those provided by Northwestern (e.g., u.northwestern.edu) is also not permitted for Level 2 and above data. Please contact researchdata@northwestern.edu with questions and for advice about alternatives.

How Much Storage Do You Need? How Much Will It Cost?

Another factor to consider when evaluating storage options is the amount and type of data files that will be generated or used as part of your project. Tips for considering capacity when choosing data storage, include:

• Estimate on the high end when planning. If you have a mix of very large and very small files, doing the estimate for very large files should be sufficient.
• Use multiple storage services to their strengths. For example, SharePoint is excellent for collaborating with your research group, but it cannot accommodate files larger than 250 GB. Store collaborative documents like manuscripts and notes in SharePoint, and find another platform to store your files larger than 250 GB.
• The location of your data will change through the course of your project. For example, your data could be produced in a core facility and stored on RDSS, then moved to Quest for analysis. Results can be integrated into a manuscript in SharePoint.
• Archival storage is often less expensive. Think about putting infrequently accessed data in archival storage, such as Amazon S3 Glacier Deep Archive.

Any associated costs to storing your data can be included in your grant budgets as data management costs.

Cost and Capacity of Northwestern Storage Services

* Microsoft is currently reviewing its policy on charging for storage.  We are tracking any changes here that may impact the community.

** Feinberg School of Medicine Only. Contact fsmhelp@northwestern.edu with questions about your quota.

Is the Data Backed Up?

Backing up your data prevents data loss due to hardware failure, disasters, file corruption, and human error. While many data storage services do this automatically, understanding what you are protected against when using a storage service is important.

Different types of “backup” strategies protect your data from different risks.

Data Protection Features on Northwestern University Storage Service

Replication involves creating a completely separate copy of your data on another server, typically in a distinct geographic location. Replication is critical for disaster recovery or bringing back an entire system from scratch if it goes down due to natural disaster or cyberattack. RDSS combines versioning with replication by copying its snapshots between the Evanston campus and the Chicago campus.

Who Needs Access to Your Data?

Different storage services have different rules about what is possible with regard to sharing your data with collaborators or the public.

Each system has different default permissions for new files and folders.

• Locations that only you have access to by default are great for storing files that no one else should see. Keep in mind that files that are associated with an individual user account will disappear if they leave Northwestern.
• Locations that grant access to a specific group of people by default are great for collaboration and are not tied to a single user account.

These services may have a method to share outside of this default access group. The following table also outlines who has access by default on each service and how to share outside the default access group.

* Sharing here is defined as granting access to people outside the users who have access by default.

** Feinberg School of Medicine Only

Is My Data Accessible to Compute Sources

Much like real estate, data storage is all about location. Can I access my data from where I need to analyze it? Common compute sources include Quest, your computer, or virtual machines run by Northwestern or public cloud providers.

Northwestern-run storage services can be directly mounted or synchronized to compute sources. Others require you to transfer your data to the compute source for use. To facilitate transfer among storage services, Northwestern University subscribes to Globus, a tool that facilitates large data transfers. Review the following table to see which storage services are accessible to Globus. Also, please see our documentation on Globus.

*Data on RDSS can currently only be transferred to Quest. The resfilesaudit zone is not accessible to Globus.

Making Your Decisions

Choosing where to store your data so that it is safe, compliant, and easy to work is harder than it seems. If you need help deciding where to store your data, email researchdata@northwestern.edu, and our data management consultants will set a time to talk about your workflow and discuss options.