Data Sharing at UChicago: Work between UChicago and CERN¶

This guide covers how to use the XCache server and manage authentication credentials for data access.

For accessing CERN EOS storage, see CERN EOS Access.

Using XCache¶

See the XCache documentation for information on using the XCache server at UChicago.

Always check your proxy certificate¶

If you are trying to use HTCondor driver to run some jobs that need data access authorization, for example on rucio, always check the status of your proxy grid certificate.

voms-proxy-init -voms atlas
# stores in /home/$USER/x509_u$UID

Once you renew your proxy certificate, add the following line to your job submit file so that HTCondor configures the job environment automatically for x509 authenticated data access:

use_x509userproxy = true  # Required for x509 authentication

If you need to point at a different location than the default above, you can use an additional line

x509userproxy = /different/path/to/x509proxy  # Path to your proxy certificate

Accessing data directly from DDMs¶

If your dataset is available at the local DDM endpoint (eg. MWT2_UC_LOCALGROUPDISK), it would be very performant to read the data directly from it. First you need to discover paths to the files of your dataset. This is easiest done in Rucio. You would issue a command like:

setupATLAS
lsetup rucio
rucio
# get your grid proxy
voms-proxy-init -voms atlas
rucio list-file-replicas user.ivukotic:xcache.test.dat --protocols root --pfns

This will give you a list of paths to all of the files and all the replicas in your dataset. You can also limit it to only paths to specific RSES:

rucio list-file-replicas user.ivukotic:xcache.test.dat --protocols root --pfns --rses MWT2_UC_LOCALGROUPDISK
# output will look like this:
# root://fax.mwt2.org:1094//pnfs/uchicago.edu/atlaslocalgroupdisk/rucio/user/ivukotic/7d/9b/xcache.test.dat

If you need only some files, you can simply grep or awk for them and save filepaths to txt file that your jobs will use.