justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 257505.72@justin-prod-sched01.dune.hep.ac.uk

Jobsub ID257505.72@justin-prod-sched01.dune.hep.ac.uk
Workflow ID3262
Stage ID1
User nameamoor@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
RSS bytes5242880000 (5000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2024-09-16 13:47:19
SiteUS_UChicago
EntryEngage_US_MWT2_uct2_gk02_condce_mcore
Last heartbeat2024-09-16 23:05:20
From worker nodeHostnameuct2-c605.mwt2.org
cpuinfoAMD EPYC 7302 16-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes5242880000 (5000 MiB)
Wall seconds limit86400 (24 hours)
Inner Apptainer?True
Job statefinished
Allocator namejustin-allocator-pro.dune.hep.ac.uk
Started2024-09-16 14:06:35
Input filesusertests:000387_reco_data_2024-08-16T_162223Z_reco_data_2024-08-19T_110508Z_reco_data_2024-08-20T_124138Z_reco_data_2024-08-22T_092959Z.root
JobscriptExit code0
Real time8h (32258s)
CPU time8h (31374s = 97%)
Outputting started2024-09-16 23:04:15
Output filesusertests:000387_reco_data_2024-08-16T_162223Z_reco_data_2024-08-19T_110508Z_reco_data_2024-08-20T_124138Z_reco_data_2024-08-22T_092959Z_reco_data_2024-09-16T_140659Z.root
Finished2024-09-16 23:05:20
Saved logsjustin-logs:257505.72-justin-prod-sched01.dune.hep.ac.uk.logs.tgz
List job events     Wrapper job log

Jobscript log (last 10,000 characters)

AFM reco2 jobscript.
Input PFN = root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/usertests/92/4f/000387_reco_data_2024-08-16T_162223Z_reco_data_2024-08-19T_110508Z_reco_data_2024-08-20T_124138Z_reco_data_2024-08-22T_092959Z.root
Setting up larsoft UPS area... /cvmfs/larsoft.opensciencegrid.org
Setting up DUNE UPS area... /cvmfs/dune.opensciencegrid.org/products/dune/
/cvmfs/larsoft.opensciencegrid.org/products/xrootd/v5_4_3b/Linux64bit+3.10-2.17-e20-p3913-prof/lib/libXrdPosixPreload.so
=== Start last 50 lines of lar log file ===
%MSG-e ShowerProducedPtrsHolder:   LArPandoraModularShowerCreation:pandoraShower@BeginModule  16-Sep-2024 18:04:04 CDT run: 20000031 subRun: 0 event: 997
Trying to add data product: InitialTrack. This element does not exist in the element holder
%MSG
%MSG-e ShowerProducedPtrsHolder:   LArPandoraModularShowerCreation:pandoraShower@BeginModule  16-Sep-2024 18:04:04 CDT run: 20000031 subRun: 0 event: 997
Trying to add data product: InitialTrack. This element does not exist in the element holder
%MSG
%MSG-e ShowerProducedPtrsHolder:   LArPandoraModularShowerCreation:pandoraShower@BeginModule  16-Sep-2024 18:04:04 CDT run: 20000031 subRun: 0 event: 997
Trying to add data product: InitialTrack. This element does not exist in the element holder
%MSG
%MSG-e ShowerProducedPtrsHolder:   LArPandoraModularShowerCreation:pandoraShower@BeginModule  16-Sep-2024 18:04:04 CDT run: 20000031 subRun: 0 event: 997
Trying to add data product: InitialTrack. This element does not exist in the element holder
%MSG
%MSG-e ShowerProducedPtrsHolder:   LArPandoraModularShowerCreation:pandoraShower@BeginModule  16-Sep-2024 18:04:04 CDT run: 20000031 subRun: 0 event: 997
Trying to add data product: InitialTrack. This element does not exist in the element holder
%MSG
%MSG-e ShowerProducedPtrsHolder:   LArPandoraModularShowerCreation:pandoraShower@BeginModule  16-Sep-2024 18:04:04 CDT run: 20000031 subRun: 0 event: 997
Trying to add data product: InitialTrack. This element does not exist in the element holder
%MSG
16-Sep-2024 18:04:06 CDT  Closed output file "000387_reco_data_2024-08-16T_162223Z_reco_data_2024-08-19T_110508Z_reco_data_2024-08-20T_124138Z_reco_data_2024-08-22T_092959Z_reco_data_2024-09-16T_140659Z.root"
16-Sep-2024 18:04:06 CDT  Closed input file "root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/usertests/92/4f/000387_reco_data_2024-08-16T_162223Z_reco_data_2024-08-19T_110508Z_reco_data_2024-08-20T_124138Z_reco_data_2024-08-22T_092959Z.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                             0.088622       2293.84       31652.9       2.22404       8143.27        14     
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.00328575    0.00715368     0.028808     0.00407057    0.00668881       14     
reco:linecluster:LineCluster                           0.0334646      274.83        3694.59      0.148468       949.119        14     
reco:trajcluster:TrajCluster                          0.00925721      210.479       2758.81      0.879138       707.697        14     
reco:pandora:StandardPandora                          0.00501098      1799.88       25089.7      0.610302       6459.46        14     
reco:pandoraTrack:LArPandoraTrackCreation             0.000767711     1.92211       22.9692      0.138195       5.86082        14     
reco:pandoraShower:LArPandoraModularShowerCreation    0.00194847     0.135037       0.68015      0.074745      0.177446        14     
reco:pandoracalo:Calorimetry                          0.000494559    0.761981       9.02977      0.0694569      2.30231        14     
reco:pandorapid:Chi2ParticleID                        0.000109325   0.00209465     0.0145201    0.000810701   0.00362509       14     
reco:trkshowersplit:TrackShowerHits                   0.000252814     1.19177       14.3356      0.0215691      3.66395        14     
reco:ophit:OpHitFinder                                 0.0210421      4.40148       61.139       0.0397691      15.7362        14     
reco:opflash:OpFlashFinder                            0.000314099   0.000854197   0.00292823    0.000579619   0.000744178      14     
reco:rns:RandomNumberSaver                            1.6992e-05    3.83313e-05   0.000178284   2.79225e-05   3.96122e-05      14     
[art]:TriggerResults:TriggerResultInserter             8.927e-06    1.58742e-05   4.1097e-05    1.3225e-05    7.78336e-06      14     
end_path:out1:RootOutput                               2.835e-06    8.36779e-06   3.3052e-05    3.5365e-06    9.22834e-06      14     
end_path:out1:RootOutput(write)                       0.00919386     0.234686       1.6321       0.090135      0.411851        14     
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 3251.45 MB
  Peak resident set size usage (VmHWM): 2079.41 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 50 lines of lar log file ===
lar exit code 0
extractor_prod.py exit code 0
pdjson2metadata exit code 0
Metadata extraction succeeds
===Metadata JSON===
{
    "metadata": {
        "DUNE.campaign": "justIN.w3262s1",
        "art.file_format_era": "ART_2011a",
        "art.file_format_version": 15,
        "art.first_event": 390,
        "art.last_event": 997,
        "art.process_name": "Reco2",
        "art.run_type": "physics",
        "core.application": "art.reco",
        "core.application.family": "art",
        "core.application.name": "reco",
        "core.application.version": "v09_75_03d00",
        "core.data_stream": "out1",
        "core.data_tier": "full-reconstructed",
        "core.end_time": 1726527846.0,
        "core.event_count": 14,
        "core.file_format": "artroot",
        "core.file_type": "mc",
        "core.first_event_number": 390,
        "core.last_event_number": 997,
        "core.run_type": "physics",
        "core.runs": [
            20000031
        ],
        "core.runs_subruns": [
            2000003100000
        ],
        "core.start_time": 1726496064.0
    },
    "parents": [
        {
            "did": "usertests:000387_reco_data_2024-08-16T_162223Z_reco_data_2024-08-19T_110508Z_reco_data_2024-08-20T_124138Z_reco_data_2024-08-22T_092959Z.root"
        }
    ]
}
===================
.:
total 1839476
-rw-r--r--. 1 dune osgvo 1883143881 Sep 16 18:04 000387_reco_data_2024-08-16T_162223Z_reco_data_2024-08-19T_110508Z_reco_data_2024-08-20T_124138Z_reco_data_2024-08-22T_092959Z_reco_data_2024-09-16T_140659Z.root
-rw-r--r--. 1 dune osgvo     439093 Sep 16 18:04 000387_reco_data_2024-08-16T_162223Z_reco_data_2024-08-19T_110508Z_reco_data_2024-08-20T_124138Z_reco_data_2024-08-22T_092959Z_reco_2024-09-16T_140659Z.log
-rw-r--r--. 1 dune osgvo       7560 Sep 16 18:04 reco2_hist.root
-rw-r--r--. 1 dune osgvo       7142 Sep 16 18:04 jobscript.log
-rw-r--r--. 1 dune osgvo       1174 Sep 16 18:04 000387_reco_data_2024-08-16T_162223Z_reco_data_2024-08-19T_110508Z_reco_data_2024-08-20T_124138Z_reco_data_2024-08-22T_092959Z_reco_data_2024-09-16T_140659Z.root.json
-rw-r--r--. 1 dune osgvo       1049 Sep 16 18:04 000387_reco_data_2024-08-16T_162223Z_reco_data_2024-08-19T_110508Z_reco_data_2024-08-20T_124138Z_reco_data_2024-08-22T_092959Z_reco_data_2024-09-16T_140659Z.root.ext.json
-rw-r--r--. 1 dune osgvo        284 Sep 16 09:06 all-input-dids.txt
-rw-r--r--. 1 dune osgvo        218 Sep 16 18:04 justin-processed-pfns.txt
-rw-r--r--. 1 dune osgvo          0 Sep 16 09:07 debugprod.log
justIN time: 2024-11-17 07:13:15 UTC       justIN version: 01.01.09