Multi-Petabyte tertiary automated tape store for worldwide HEP and other scientific endeavors. High Availability (24 x 7) Local and Grid access Fermilab Data Movement & Storage n n n Peaks of > 25 TB to & from tape/day 15 TB/day 4. 5 PB on tape n n n Scalable Hardware and Software Architecture Front-end disk caching Evolves to meet evolving requirements
Local Sources The DZero Experiment Remote Sources The CMS Experiment Sloan Digital Sky Survey The CDF Experiment And Many Others DES, KTe. V, MINOS, LQCD, Mini. Boo. NE, …
• Users write RAW data to Mass Storage, analyze/reanalyze it in real-time on PC “farms”, then write results back into the Mass Storage • ~3 bytes read for every byte written to tape • Lifetime of data exceeds 5 years.
Front-end disk caching Rate adapting, Fast access to frequently requested files + load balancing CMS 250 TB read from cache in one day 50 TB > 800 TB and growing in volatile & tape backed raid disk
Tape Backed Disk Cache CMS CSA 061 from CERN to Fermilab Network to disk cache ~ 250 MB/s Data backed from disk to tape (Pink) Disk Cache to tape ~ 250 MB/s 1. Computing, Software, Analysis Challenge, October 2006
Future capacity through 2010 n n n Currently ~ 4. 5 PB on ~35000 tapes, accessed by 120 tape drives from 9 libraries Currently more than 500 TB in front-end disk cache Expecting additional ~30 PB or more on tape by 2010 Will have > 1 PB in disk cache serving US CMS community Will need to acquire a tape library and dozens of tape drives per year to accommodate
Long-term Retention & Data Integrity recycle clone R/W FULL Randomly select files, read them and check their integrity (calculated CRC) Tape Transport problem “ 3 strikes” NOACCESS FULL @ capacity Selective CRC error Manage tape Life Cycle. “Clone” tapes with too many mounts, Write protect full tapes to prevent accidental erasure. Protected tapes Newly filled tapes needing protection Automated safeguards and audits detect problem tapes and drives (3 tries) and denies access to them Drop from tape aide write protecting tapes on a ticket Tickets generated automatically to physically write protect filled tapes
Скачать презентацию Multi-Petabyte tertiary automated tape store for worldwide HEP
1298f5c0dea50b6bc88cfc1a7e2343ba.ppt