justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 72769.158@justin-prod-sched02.dune.hep.ac.uk

Jobsub ID72769.158@justin-prod-sched02.dune.hep.ac.uk
Workflow ID3262
Stage ID1
User nameamoor@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
RSS bytes5242880000 (5000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2024-09-16 14:04:10
SiteUS_UChicago
EntryEngage_US_MWT2_iut2_condce_mcore
Last heartbeat2024-09-16 17:52:50
From worker nodeHostnameiut2-c418.iu.edu
cpuinfoAMD EPYC 7443 24-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes5242880000 (5000 MiB)
Wall seconds limit86400 (24 hours)
Inner Apptainer?True
Job statefinished
Allocator namejustin-allocator-pro.dune.hep.ac.uk
Started2024-09-16 14:26:32
Input filesusertests:000591_reco_data_2024-08-16T_162554Z_reco_data_2024-08-19T_105838Z_reco_data_2024-08-20T_140218Z_reco_data_2024-08-22T_092056Z.root
JobscriptExit code0
Real time3h (12335s)
CPU time3h (12045s = 97%)
Outputting started2024-09-16 17:52:09
Output filesusertests:000591_reco_data_2024-08-16T_162554Z_reco_data_2024-08-19T_105838Z_reco_data_2024-08-20T_140218Z_reco_data_2024-08-22T_092056Z_reco_data_2024-09-16T_142636Z.root
Finished2024-09-16 17:52:50
Saved logsjustin-logs:72769.158-justin-prod-sched02.dune.hep.ac.uk.logs.tgz
List job events     Wrapper job log

Jobscript log (last 10,000 characters)

AFM reco2 jobscript.
Input PFN = root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/usertests/6a/c7/000591_reco_data_2024-08-16T_162554Z_reco_data_2024-08-19T_105838Z_reco_data_2024-08-20T_140218Z_reco_data_2024-08-22T_092056Z.root
Setting up larsoft UPS area... /cvmfs/larsoft.opensciencegrid.org
Setting up DUNE UPS area... /cvmfs/dune.opensciencegrid.org/products/dune/
/cvmfs/larsoft.opensciencegrid.org/products/xrootd/v5_4_3b/Linux64bit+3.10-2.17-e20-p3913-prof/lib/libXrdPosixPreload.so
=== Start last 50 lines of lar log file ===
%MSG-e ShowerProducedPtrsHolder:   LArPandoraModularShowerCreation:pandoraShower@BeginModule  16-Sep-2024 13:52:02 EDT run: 20000031 subRun: 0 event: 934
Trying to add data product: InitialTrack. This element does not exist in the element holder
%MSG
%MSG-e ShowerProducedPtrsHolder:   LArPandoraModularShowerCreation:pandoraShower@BeginModule  16-Sep-2024 13:52:02 EDT run: 20000031 subRun: 0 event: 934
Trying to add data product: InitialTrack. This element does not exist in the element holder
%MSG
%MSG-e ShowerProducedPtrsHolder:   LArPandoraModularShowerCreation:pandoraShower@BeginModule  16-Sep-2024 13:52:02 EDT run: 20000031 subRun: 0 event: 934
Trying to add data product: InitialTrack. This element does not exist in the element holder
%MSG
%MSG-e ShowerProducedPtrsHolder:   LArPandoraModularShowerCreation:pandoraShower@BeginModule  16-Sep-2024 13:52:02 EDT run: 20000031 subRun: 0 event: 934
Trying to add data product: InitialTrack. This element does not exist in the element holder
%MSG
%MSG-e ShowerProducedPtrsHolder:   LArPandoraModularShowerCreation:pandoraShower@BeginModule  16-Sep-2024 13:52:02 EDT run: 20000031 subRun: 0 event: 934
Trying to add data product: InitialTrack. This element does not exist in the element holder
%MSG
%MSG-e ShowerProducedPtrsHolder:   LArPandoraModularShowerCreation:pandoraShower@BeginModule  16-Sep-2024 13:52:02 EDT run: 20000031 subRun: 0 event: 934
Trying to add data product: InitialTrack. This element does not exist in the element holder
%MSG
16-Sep-2024 13:52:02 EDT  Closed output file "000591_reco_data_2024-08-16T_162554Z_reco_data_2024-08-19T_105838Z_reco_data_2024-08-20T_140218Z_reco_data_2024-08-22T_092056Z_reco_data_2024-09-16T_142636Z.root"
16-Sep-2024 13:52:02 EDT  Closed input file "root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/usertests/6a/c7/000591_reco_data_2024-08-16T_162554Z_reco_data_2024-08-19T_105838Z_reco_data_2024-08-20T_140218Z_reco_data_2024-08-22T_092056Z.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                             0.121614       1022.04       12043.7       1.42695       3323.54        12     
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.00597335     0.0146681     0.0388397    0.00673378     0.0121621       12     
reco:linecluster:LineCluster                           0.0577141      45.6589       475.768      0.117521       131.078        12     
reco:trajcluster:TrajCluster                           0.0168975      63.8289       697.317      0.414735       191.678        12     
reco:pandora:StandardPandora                          0.00171947      910.013       10848.7      0.386593       2996.65        12     
reco:pandoraTrack:LArPandoraTrackCreation             0.000167134    0.942447       8.8403       0.0542677      2.41831        12     
reco:pandoraShower:LArPandoraModularShowerCreation    0.000185899    0.155784      0.373118      0.160261      0.126189        12     
reco:pandoracalo:Calorimetry                          0.000118342    0.403538       3.66136      0.0584778     0.997057        12     
reco:pandorapid:Chi2ParticleID                         3.717e-05    0.00153835    0.00640052    0.000752588   0.00176272       12     
reco:trkshowersplit:TrackShowerHits                   0.000221045    0.778872       7.96459      0.0209917      2.1833         12     
reco:ophit:OpHitFinder                                 0.0333801     0.0661288     0.113751      0.0671706     0.0214381       12     
reco:opflash:OpFlashFinder                            0.00021859    0.00100645    0.00423817    0.000739238   0.00101228       12     
reco:rns:RandomNumberSaver                            2.5377e-05    6.73506e-05   0.000321905   4.2735e-05    7.8766e-05       12     
[art]:TriggerResults:TriggerResultInserter            1.1292e-05    1.94567e-05   5.9893e-05    1.4492e-05    1.30498e-05      12     
end_path:out1:RootOutput                               4.257e-06    1.06156e-05   3.3262e-05     6.237e-06     9.557e-06       12     
end_path:out1:RootOutput(write)                       0.00369731     0.174143       1.07021      0.087372      0.280706        12     
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 2822.57 MB
  Peak resident set size usage (VmHWM): 1785.6 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 50 lines of lar log file ===
lar exit code 0
extractor_prod.py exit code 0
pdjson2metadata exit code 0
Metadata extraction succeeds
===Metadata JSON===
{
    "metadata": {
        "DUNE.campaign": "justIN.w3262s1",
        "art.file_format_era": "ART_2011a",
        "art.file_format_version": 15,
        "art.first_event": 71,
        "art.last_event": 934,
        "art.process_name": "Reco2",
        "art.run_type": "physics",
        "core.application": "art.reco",
        "core.application.family": "art",
        "core.application.name": "reco",
        "core.application.version": "v09_75_03d00",
        "core.data_stream": "out1",
        "core.data_tier": "full-reconstructed",
        "core.end_time": 1726509122.0,
        "core.event_count": 12,
        "core.file_format": "artroot",
        "core.file_type": "mc",
        "core.first_event_number": 71,
        "core.last_event_number": 934,
        "core.run_type": "physics",
        "core.runs": [
            20000031
        ],
        "core.runs_subruns": [
            2000003100000
        ],
        "core.start_time": 1726496837.0
    },
    "parents": [
        {
            "did": "usertests:000591_reco_data_2024-08-16T_162554Z_reco_data_2024-08-19T_105838Z_reco_data_2024-08-20T_140218Z_reco_data_2024-08-22T_092056Z.root"
        }
    ]
}
===================
.:
total 1337220
-rw-r--r-- 1 dune osgvo 1368962741 Sep 16 13:52 000591_reco_data_2024-08-16T_162554Z_reco_data_2024-08-19T_105838Z_reco_data_2024-08-20T_140218Z_reco_data_2024-08-22T_092056Z_reco_data_2024-09-16T_142636Z.root
-rw-r--r-- 1 dune osgvo     304689 Sep 16 13:52 000591_reco_data_2024-08-16T_162554Z_reco_data_2024-08-19T_105838Z_reco_data_2024-08-20T_140218Z_reco_data_2024-08-22T_092056Z_reco_2024-09-16T_142636Z.log
-rw-r--r-- 1 dune osgvo       7560 Sep 16 13:52 reco2_hist.root
-rw-r--r-- 1 dune osgvo       7139 Sep 16 13:52 jobscript.log
-rw-r--r-- 1 dune osgvo       1172 Sep 16 13:52 000591_reco_data_2024-08-16T_162554Z_reco_data_2024-08-19T_105838Z_reco_data_2024-08-20T_140218Z_reco_data_2024-08-22T_092056Z_reco_data_2024-09-16T_142636Z.root.json
-rw-r--r-- 1 dune osgvo       1047 Sep 16 13:52 000591_reco_data_2024-08-16T_162554Z_reco_data_2024-08-19T_105838Z_reco_data_2024-08-20T_140218Z_reco_data_2024-08-22T_092056Z_reco_data_2024-09-16T_142636Z.root.ext.json
-rw-r--r-- 1 dune osgvo        284 Sep 16 10:26 all-input-dids.txt
-rw-r--r-- 1 dune osgvo        218 Sep 16 13:52 justin-processed-pfns.txt
-rw-r--r-- 1 dune osgvo          0 Sep 16 10:26 debugprod.log
justIN time: 2024-09-29 13:22:31 UTC       justIN version: 01.01.08