Difference between revisions of "NPS Disk Space"
From HallCWiki
(→Hall C cdaq-cluster filesystems) |
(→Cache allocations) |
||
| (5 intermediate revisions by the same user not shown) | |||
| Line 8: | Line 8: | ||
/home/cdaq/nps-2023/ | /home/cdaq/nps-2023/ | ||
2. output directory (ROOTfiles, REPORT_OUTPUT, PDF files, etc.): | 2. output directory (ROOTfiles, REPORT_OUTPUT, PDF files, etc.): | ||
| − | /net/cdaq/cdaql1data/cdaq/hallc-online- | + | /net/cdaq/cdaql1data/cdaq/hallc-online-nps2023/ |
;NOTES: | ;NOTES: | ||
| − | * Please do take special care to avoid modifying any files outside of your ~/ | + | * Please do take special care to avoid modifying any files outside of your ~/nps-2023 directory. |
| − | * File storage directories during the online analysis of | + | * File storage directories during the online analysis of NPS. |
* There is a README in the first directory with a few notes. | * There is a README in the first directory with a few notes. | ||
| − | * It is recommended to make a '~/ | + | * It is recommended to make a '~/nps-2023/go_analysis_nps' shell script that sets up your environment from scratch. For example, no changes/additions to anything in ~/bin, changes to init files like .tcshrc, .bashrc, etc, text editor config files, etc. |
== Farm/CUE disk allocations == | == Farm/CUE disk allocations == | ||
| − | /volatile/hallc/ | + | /volatile/hallc/nps/ ( 4 TB high quota; 2 TB guarantee ) |
;NOTES: | ;NOTES: | ||
* Files are '''NOT''' backed-up | * Files are '''NOT''' backed-up | ||
| Line 22: | Line 22: | ||
* Files you want to keep should be pushed to tape using ''jput'' | * Files you want to keep should be pushed to tape using ''jput'' | ||
| − | /work/hallc/ | + | /work/hallc/nps/ directory ( 1TB quota ) |
;NOTES: | ;NOTES: | ||
* Files are '''NOT''' backed up. | * Files are '''NOT''' backed up. | ||
| Line 29: | Line 29: | ||
* Nothing that matters should be stored '''only''' on /work. | * Nothing that matters should be stored '''only''' on /work. | ||
| − | /group/ | + | /group/nps/ ( 100GB quota ) |
;NOTES: | ;NOTES: | ||
* Backed up regularly. | * Backed up regularly. | ||
| Line 35: | Line 35: | ||
== Tape allocations == | == Tape allocations == | ||
| − | /mss/hallc/c- | + | /mss/hallc/c-nps/analysis |
* tape volume for analysis output (ie. simulation output, replay output that you want to keep long-term, etc.) | * tape volume for analysis output (ie. simulation output, replay output that you want to keep long-term, etc.) | ||
| − | /mss/hallc/c- | + | /mss/hallc/c-nps/raw |
* tape volume for raw data output (ie. production CODA files from the Hall) | * tape volume for raw data output (ie. production CODA files from the Hall) | ||
* Raw data are NOT accessible directly from tape. To make them accessible, they must be copied over to cache. | * Raw data are NOT accessible directly from tape. To make them accessible, they must be copied over to cache. | ||
| − | + | ||
== Cache allocations == | == Cache allocations == | ||
| − | /cache/hallc/c- | + | /cache/hallc/c-nps/analysis/ |
| − | /cache/hallc/c- | + | /cache/hallc/c-nps/raw/ |
;NOTES | ;NOTES | ||
| Line 50: | Line 50: | ||
* Cache has a certain quota, which if it is exceeded, the raw data files will be automatically copied to tape. See cache data policy: https://scicomp.jlab.org/docs/filesystemsv2#cacheDataPolicy | * Cache has a certain quota, which if it is exceeded, the raw data files will be automatically copied to tape. See cache data policy: https://scicomp.jlab.org/docs/filesystemsv2#cacheDataPolicy | ||
* to recover the raw data back to cache, the user will have to use the 'jcache get' command to copy over the files from tape to cache. See https://scicomp.jlab.org/docs/node/586 | * to recover the raw data back to cache, the user will have to use the 'jcache get' command to copy over the files from tape to cache. See https://scicomp.jlab.org/docs/node/586 | ||
| + | |||
| + | ==Replayed run information== | ||
| + | |||
| + | https://redmine.jlab.org/projects/nps-farm-setup/wiki/Wiki | ||
Latest revision as of 11:13, 10 October 2023
Disk space allocation directories for the Deuteron experiment. Directory paths marked with [*] are not confirmed as of Feb 09, 2023
For additional info on filesystems, see: https://scicomp.jlab.org/docs/filesystemsv2
Contents
Hall C cdaq-cluster filesystems
1. main directory (for online analyzer, DBs, replay scripts, etc.):
/home/cdaq/nps-2023/
2. output directory (ROOTfiles, REPORT_OUTPUT, PDF files, etc.):
/net/cdaq/cdaql1data/cdaq/hallc-online-nps2023/
- NOTES
- Please do take special care to avoid modifying any files outside of your ~/nps-2023 directory.
- File storage directories during the online analysis of NPS.
- There is a README in the first directory with a few notes.
- It is recommended to make a '~/nps-2023/go_analysis_nps' shell script that sets up your environment from scratch. For example, no changes/additions to anything in ~/bin, changes to init files like .tcshrc, .bashrc, etc, text editor config files, etc.
Farm/CUE disk allocations
/volatile/hallc/nps/ ( 4 TB high quota; 2 TB guarantee )
- NOTES
- Files are NOT backed-up
- Use for all large file output from analysis or simulation jobs. When guarantee threshold is exceeded it is possible to have files auto-cleaned up (removed). See policy: https://scicomp.jlab.org/docs/volatile_disk_pool
- Files you want to keep should be pushed to tape using jput
/work/hallc/nps/ directory ( 1TB quota )
- NOTES
- Files are NOT backed up.
- Good place for analysis software, database files, etc.
- These files should be in GitHub or have similar backups.
- Nothing that matters should be stored only on /work.
/group/nps/ ( 100GB quota )
- NOTES
- Backed up regularly.
- Best place for analysis/replay scripts; software that is being actively developed, etc. (But, still use GitHub!)
Tape allocations
/mss/hallc/c-nps/analysis
- tape volume for analysis output (ie. simulation output, replay output that you want to keep long-term, etc.)
/mss/hallc/c-nps/raw
- tape volume for raw data output (ie. production CODA files from the Hall)
- Raw data are NOT accessible directly from tape. To make them accessible, they must be copied over to cache.
Cache allocations
/cache/hallc/c-nps/analysis/
/cache/hallc/c-nps/raw/
- NOTES
- During online analysis, raw data from tape will be automatically copied over to cache.
- Cache has a certain quota, which if it is exceeded, the raw data files will be automatically copied to tape. See cache data policy: https://scicomp.jlab.org/docs/filesystemsv2#cacheDataPolicy
- to recover the raw data back to cache, the user will have to use the 'jcache get' command to copy over the files from tape to cache. See https://scicomp.jlab.org/docs/node/586
