justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Workflow 1910, Stage 1

Priority50
Processors1
Wall seconds80000
RSS bytes2097152000 (2000 MiB)
Max distance for inputs100.0
Enabled input RSEs
Enabled output RSEs
Enabled sites
Scopeusertests
Events for this stage

Output patterns

 DestinationPatternLifetimeFor next stage
0https://fndcadoor.fnal.gov:2880/dune/scratch/users/lavaut/01910/1*_reco_data_*.root

Environment variables

NameValue
COUNT3
INPUT_TAR_DIR_LOCAL/cvmfs/fifeuser1.opensciencegrid.org/sw/dune/fb0e75b32061b2cb708dd31eb66e78dc35648d97

File states

Total filesFindingUnallocatedAllocatedOutputtingProcessedNot foundFailed
30000300

Job states

TotalSubmittedStartedProcessingOutputtingFinishedNotusedAbortedStalledJobscript errorOutputting failedNone processed
1100004700000
Files processed000.20.20.40.40.60.60.80.8111.21.21.41.41.61.61.81.8222.22.22.42.42.62.62.82.833Apr-23 14:00Apr-23 15:00Apr-23 16:00Files processedBin start timesNumber per binUS_PuertoRicoUK_Lancaster

RSEs used

NameInputsOutputs
MONTECARLO30

Stats of processed input files as CSV or JSON, and of uploaded output files as CSV or JSON (up to 10000 files included)

Jobscript

#!/bin/sh
:<<'EOF'
Test of RCDS/cvmfs tar files access from JUSTIN jobs. You need to 
set up the justin commands 

source /cvmfs/dune.opensciencegrid.org/products/dune/setup_dune.sh
setup justin

Then

1) Create a tar file 
   mkdir somedir
   cd somedir
   date > hello_world.txt
   tar cvf hello_world.tar *
2) Create an X.509 proxy (no need for VOMS)
   kx509
3) Upload to cvmfs
   INPUT_TAR_DIR_LOCAL=`justin-cvmfs-upload hello_world.tar`
4) Wait until  ls $INPUT_TAR_DIR_LOCAL  works (minutes?)
5) Create and run a workflow
   justin simple-workflow --monte-carlo 1 \
   --env INPUT_TAR_DIR_LOCAL="$INPUT_TAR_DIR_LOCAL" \
   --jobscript-id testpro:cvmfs-hello-world

Check the output through the dashboard: the date in hello_world.txt 
should beprinted.

EOF

# Get an unprocessed file from this stage and say we processed it
did_pfn_rse=`$JUSTIN_PATH/justin-get-file`

if [ "$did_pfn_rse" != "" ] ; then
  did=`echo $did_pfn_rse | cut -f1 -d' '`
  echo "$did" > justin-processed-dids.txt

  # Look at the file in cvmfs
  for RUN in $(seq 0 $COUNT);
  do
    echo "Contents of $INPUT_TAR_DIR_LOCAL/hello_world.txt"
    cat <$INPUT_TAR_DIR_LOCAL/hello_world_${RUN}.txt >${RUN}.txt

done
fi
exit 0
justIN time: 2024-11-24 11:21:33 UTC       justIN version: 01.01.09