HPSS I/O
HTAR : save/restore private directory (large set of small files) in HPSS
For saving all files in directory 'aaa' on disk to your HPSS target bbb/ccc.tar and keep log file in ~/0x/ddd.log1 (needed if you want to kill/quit the window) do:
cd aaa
du -hs .
htar -cf bbb/ccc.tar file1 file2 fileX* ... >& ~/0x/ddd.log1 &
(wait until job is done, for 50GB make take severa hours)
hsi (goes directly to HPSS, be careful!)
?ls -l bbb/
?quit (exit hsi)
You should see 2 files : ccc.tar & ccc.tar.idx, the size of file ccc.tar should be close to the size of your oryginal directory.
The .tar file shows up immediately but sinking is NOT finished until you see .idx file.
The log file should contain the string: HTAR: HTAR SUCCESSFUL , no warnings.
However the most reliable way to verify the storage did not failed is to:
retrieving files from HPSS to disk directory eee do:
mkdir eee
cd eee
htar -xf bbb/ccc.tar >& ~/0x/ddd.log2 &
Other options of HTAR command are described here:
http://drupal.star.bnl.gov/STAR/comp/sofi/hpss/htar
Known problems:
WARNING: htar_PreallocateSpace: HPSS OUT-OF-SPACE error preallocating
**** bytes for file=[/home/salur/myfile.tar]
This is a directory with 350G. Do I have to divide my files into smaller.
there is a 60GB max file size limit [when creating tarballs], Another
limit to be aware of, however, is an 8GB limit on member files
Mass restoring of ~100GB files from HPSS using Data Carusel
Prepare input file xxx.hpss with list of source +destination paths for all files you need, one line per file
Execute the command (at any location at rcas6nnn)
hpss_user.pl -f xxx.hpss
Wait 10-1000 minutes fro the files to show up at your destination (if all paths are correct)
Instruction for new users uploading FGT documents
Instruction for remote uploading of documents to the common FGT Drupal place to complement missing features of fgt-hn
First time setup
- You need to login to drupal as you. The password is different than your regular RCF password. If you do not remember Drupal password select 'forgot password' and new one will be sent to you by e-mail
- got to the mother-web page:
http://drupal.star.bnl.gov/STAR/subsys/upgr/fgt/hn-upload-contributions
you can also navigate: SubSystems --> Detctor Upgrades --> FGT --> FGT-HN Upload - Got to the bottom and select 'Add child page'
- Set the following fields:
* Title: yourName-FGT-HN-Uploads (edit it)
* Parent: ------FGT-HN-Upload contributions (selectable from menu)
* STAR: --FGT (selectable)
* Body: Your Name : list of attachements sent to FGT-HN (edit it, just 1 line)
Upload a test jpg file using menu at the very bottom of the page. To attache new file, first 'Choose File', next 'Attach' - Finally at the very bottom 'Submit' the whole page to Drupal.
- go to the 'mother-page' and you should see:
* in the upper part your JPG
* in the lower part (below bold text) your sub-page among other people who already joined, alphabetical order.
If you can't see your contribution at this point contact me (Jan). Do not try it over - it will add mess to Drupal. If you add more than one page only the first one will be considered
Uploading subsequent documents
- Login to drupal and go to your sub-page with this test JPEG
- Click Edit
- scroll down to upload section and add as many documents as needed, one by one
- you may change the name of uploaded file - just edit it
- Click 'Submit'
- go to the 'mother-page' and you should see your new documents listed in chronological order.
- Now you can sent an e-mail to FGT-HN referring your new uploaded documents as:
.... new docs are uploaded to:
http://drupal.star.bnl.gov/STAR/subsys/upgr/fgt/hn-upload-contributions
under myName, files xxx-yyy-jpg and zzz-uuu-mmm.pdf
Report to me if there are any problems,
Jan