justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Workflow 1909, Stage 1

Priority50
Processors1
Wall seconds80000
RSS bytes2097152000 (2000 MiB)
Max distance for inputs100.0
Enabled input RSEs
Enabled output RSEs
Enabled sites
Scopeusertests
Events for this stage

Output patterns

 DestinationPatternLifetimeFor next stage
0https://fndcadoor.fnal.gov:2880/dune/scratch/users/lavaut/01909/1*_reco_data_*.root

Environment variables

NameValue
COUNT10
INPUT_TAR_DIR_LOCAL/cvmfs/fifeuser2.opensciencegrid.org/sw/dune/297debcc8506666450357d80c3266560fe9c12f7

File states

Total filesFindingUnallocatedAllocatedOutputtingProcessedNot foundFailed
1000001000

Job states

TotalSubmittedStartedProcessingOutputtingFinishedNotusedAbortedStalledJobscript errorOutputting failedNone processed
15000011400000
Files processed001122334455667788991010Apr-23 14:00Apr-23 15:00Apr-23 16:00Files processedBin start timesNumber per binUS_PuertoRicoNL_NIKHEF

RSEs used

NameInputsOutputs
MONTECARLO100

Stats of processed input files as CSV or JSON, and of uploaded output files as CSV or JSON (up to 10000 files included)

Jobscript

#!/bin/sh
:<<'EOF'
Test of RCDS/cvmfs tar files access from JUSTIN jobs. You need to 
set up the justin commands 

source /cvmfs/dune.opensciencegrid.org/products/dune/setup_dune.sh
setup justin

Then

1) Create a tar file 
   mkdir somedir
   cd somedir
   date > hello_world.txt
   tar cvf hello_world.tar *
2) Create an X.509 proxy (no need for VOMS)
   kx509
3) Upload to cvmfs
   INPUT_TAR_DIR_LOCAL=`justin-cvmfs-upload hello_world.tar`
4) Wait until  ls $INPUT_TAR_DIR_LOCAL  works (minutes?)
5) Create and run a workflow
   justin simple-workflow --monte-carlo 1 \
   --env INPUT_TAR_DIR_LOCAL="$INPUT_TAR_DIR_LOCAL" \
   --jobscript-id testpro:cvmfs-hello-world

Check the output through the dashboard: the date in hello_world.txt 
should beprinted.

EOF

# Get an unprocessed file from this stage and say we processed it
did_pfn_rse=`$JUSTIN_PATH/justin-get-file`

if [ "$did_pfn_rse" != "" ] ; then
  did=`echo $did_pfn_rse | cut -f1 -d' '`
  echo "$did" > justin-processed-dids.txt

  # Look at the file in cvmfs
  for RUN in $(seq 0 $COUNT);
  do
    echo "Contents of $INPUT_TAR_DIR_LOCAL/hello_world.txt"
    cat <$INPUT_TAR_DIR_LOCAL/hello_world_${RUN}.txt >${RUN}.txt

done
fi
exit 0
justIN time: 2024-09-29 09:28:11 UTC       justIN version: 01.01.08