All of Us workspaces have two dedicated storage locations – the workspace bucket and your storage disk. With your storage disk, you can select between either a standard disk or a reattachable persistent disk. A standard disk is created and deleted with your cloud environment, while a reattachable disk can be saved even when your compute environment is deleted.
A Persistent Disk (PD) is a reliable, high-performance block storage for virtual machine instances. Like a USB drive, the persistent disk can be detached from the Virtual Machine before deleting or recreating the Cloud Environment and attached to a new one. The PD lets you keep the packages your notebook code is built upon, input files necessary for your analysis, and outputs you’ve generated - without having to move anything to the workspace bucket for permanent storage.
You can learn more about persistent disks here: https://cloud.google.com/compute/docs/disks#pdspecs
How to add a Reattachable Persistent Disk to your Cloud Environment
To add a reattachable persistent disk to your cloud environment's compute configuration, navigate to a workspace and click on the "Cloud analysis environment" button in the right navigation bar. Scroll to “Storage Disk options” section to configure your standard disk or reattachable persistent disk:
If you select a reattachable persistent disk, you can select either a standard persistent disk or a solid state-drive (SSD) persistent disk. You can learn more about the disk types here: https://cloud.google.com/compute/docs/disks#disk-types
Managing your Reattachable Persistent Disk
Your persistent disk will be mounted on the Jupyter server $HOME directory (/home/jupyter). Persistent disks, like your cloud analysis environment, are specific to each researcher. The data stored in your persistent disk is not shareable, unlike the workspace bucket which is shared between workspace researchers.
Note: if you install Python and R packages, they will live under `$HOME/packages/` (/home/jupyter/packages).
Once you have a reattachable persistent disk attached to an active environment, you can change the persistent disk type. However, some changes may require deletion and re-creation of your persistent disk and cloud environment to take effect. This will delete all files on the disk. If you want to save some files permanently, such as input data, analysis outputs, or installed packages, copy them to the workspace bucket. Note: Jupyter notebooks are autosaved to the workspace bucket, and deleting your disk will not delete your notebooks.
Delete your environment with a reattachable Persistent Disk
If you have attached a persistent disk and want to delete your environment, you will have two options:
If you save your persistent disk for later and do not have an active environment, you can delete your persistent disk by navigating to the cloud analysis environment panel. At the bottom, you will see an option to ‘Delete Persistent Disk’:
If you decide to delete your persistent disk, all files on the disk will be deleted. If you want to permanently save some files from the disk before deleting it, you will need to create a new cloud environment to access it.
Cost of a reattachable Persistent Disk
A cost per hour is associated with maintaining the disk even when the cloud compute is paused or deleted. A standard persistent disk costs $.04/GB/month. [Disks and images pricing | Compute Engine: Virtual Machines (VMs)]
Reattachable Persistent Disk Limitations
If you want to use a dataproc cluster instead of a Standard VM, you can only use standard disk as a storage option. Reattachable persistent disks are not supported with Dataproc.