Difference between revisions of "Storage and Backup"
(→Storage) |
|||
| (4 intermediate revisions by 2 users not shown) | |||
| Line 1: | Line 1: | ||
| − | The storage servers and importand hosts (like the login server) are connected with infiniband. Therefor prefer this host for doing larger file I/O | + | The storage servers and importand hosts (like the login server) are connected with infiniband. Therefor prefer this host for doing larger file I/O as it will be much faster. |
== Storage == | == Storage == | ||
| − | There are different file systems with different | + | There are different file systems with different lifetimes and redundancies which you can use to store your data. |
;your home directory | ;your home directory | ||
| Line 9: | Line 9: | ||
;/tmp | ;/tmp | ||
| − | :This is the counterpart to your home. It is located on the local disk and should be used for temporary data. Unused files are | + | :This is the counterpart to your home. It is located on the local disk and should be used for temporary data. Unused files are deleted after 14 days. |
| − | ;/mnt/ceph | + | ;/mnt/ceph |
| − | :This directory is globally accessible and can be used by everyone for minor important data, which is not worth to be backed up or simply to large for home. This files are not deleted, but if the disk is full we tell you to delete some files. It runs directly as cephfs on our CephCluster | + | :This directory is globally accessible and can be used by everyone for minor important data, which is not worth to be backed up or simply to large for home. This files are not deleted, but if the disk is full we tell you to delete some files. It runs directly as cephfs on our CephCluster. This means a disk or even a host failure will not result in a service interruption. The Ceph Cluster is connected with Infiniband and you may get the best performance when using the login server or the compute nodes. It is currently limited to 30 TB. |
| − | :If you want to use this space please create a | + | :If you want to use this space please create a subdirectory with you login name. All other files and directories will be delete without notice. |
;/home/work-itp (deprecated and replaced by /mnt/ceph) | ;/home/work-itp (deprecated and replaced by /mnt/ceph) | ||
| − | :This directory is globally accessible and can be used by everyone for minor important data which are not worth to be backed up or simply to large for home. | + | :This directory is globally accessible and can be used by everyone for minor important data which are not worth to be backed up or simply to large for your home. These files are not deleted. But, if the disk is full, we tell you to delete some files. It runs on a raid6 system, which means that a disk failure will not result in data loss. It is also served by a dedicated machine (scratch1) and heavy load does not affect the home directories. It is connected with Infiniband and you may get the best performance when using the login server or the compute nodes. It is currently limited to 40 TB. |
| − | |||
;/data (some hosts only) | ;/data (some hosts only) | ||
| − | :This is the local disk. It has the same level like /tmp except that it isn't deleted automatically. Data on this partition is silently deleted when reinstalling or upgrading the machine. | + | :This is the local disk, but not available on all machines. It has the same level like /tmp except that it isn't deleted automatically. Data on this partition is silently deleted when reinstalling or upgrading the machine. |
== Backup == | == Backup == | ||
Latest revision as of 13:11, 10 March 2022
The storage servers and importand hosts (like the login server) are connected with infiniband. Therefor prefer this host for doing larger file I/O as it will be much faster.
Storage
There are different file systems with different lifetimes and redundancies which you can use to store your data.
- your home directory
- This is your primary directory where all settings, your Mail and other important data is saved. It is mounted on every host to /home/YOUR_LOGIN. A backup is done nightly. This file system has an individual limit which you can check with the command 'df'. In change to earlier setups it has no soft limit which you can exceed for a few days. If you have problems with graphical logins or reading mails, first check your disk space.
- /tmp
- This is the counterpart to your home. It is located on the local disk and should be used for temporary data. Unused files are deleted after 14 days.
- /mnt/ceph
- This directory is globally accessible and can be used by everyone for minor important data, which is not worth to be backed up or simply to large for home. This files are not deleted, but if the disk is full we tell you to delete some files. It runs directly as cephfs on our CephCluster. This means a disk or even a host failure will not result in a service interruption. The Ceph Cluster is connected with Infiniband and you may get the best performance when using the login server or the compute nodes. It is currently limited to 30 TB.
- If you want to use this space please create a subdirectory with you login name. All other files and directories will be delete without notice.
- /home/work-itp (deprecated and replaced by /mnt/ceph)
- This directory is globally accessible and can be used by everyone for minor important data which are not worth to be backed up or simply to large for your home. These files are not deleted. But, if the disk is full, we tell you to delete some files. It runs on a raid6 system, which means that a disk failure will not result in data loss. It is also served by a dedicated machine (scratch1) and heavy load does not affect the home directories. It is connected with Infiniband and you may get the best performance when using the login server or the compute nodes. It is currently limited to 40 TB.
- /data (some hosts only)
- This is the local disk, but not available on all machines. It has the same level like /tmp except that it isn't deleted automatically. Data on this partition is silently deleted when reinstalling or upgrading the machine.
Backup
Backups of the home directory and several other important system directories is done nightly.
- There are held 3 generations of a file.
- Overwritten files are stored for 90 days
- A deleted file is stored for 90 days
If you have delete some files and want them restored, please contact the system administrator.
