justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 359717.23@justin-prod-sched01.dune.hep.ac.uk

Jobsub ID359717.23@justin-prod-sched01.dune.hep.ac.uk
Workflow ID5846
Stage ID1
User nameamoor@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
GPUNo
RSS bytes6291456000 (6000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2025-04-01 14:55:31
SiteES_PIC
EntryDUNE_T1_ES_PIC_ce15-multicore
Last heartbeat2025-04-01 14:59:59
From worker nodeHostnamehnode70.pic.es
cpuinfoAMD EPYC 7443P 24-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes6291456000 (6000 MiB)
Wall seconds limit216000 (60 hours)
GPU
Inner Apptainer?True
Job statefinished
Allocator namejustin-allocator-pro.dune.hep.ac.uk
Started2025-04-01 14:56:42
Input filesusertests:000012_reco_data_2025-03-31T_142801Z_reco_data_2025-03-31T_150020Z_reco_data_2025-04-01T_083018Z_reco_data_2025-04-01T_101328Z.root
JobscriptExit code0
Real time2m (126s)
CPU time0m (31s = 24%)
Max RSS bytes1172078592 (1117 MiB)
Outputting started2025-04-01 14:58:49
Output filesusertests:000012_reco_data_2025-03-31T_142801Z_reco_data_2025-03-31T_150020Z_reco_data_2025-04-01T_083018Z_reco_data_2025-04-01T_101328Z_reco_data_2025-04-01T_145645Z.root
Finished2025-04-01 14:59:59
Saved logsjustin-logs:359717.23-justin-prod-sched01.dune.hep.ac.uk.logs.tgz
List job events     Wrapper job log

Jobscript log (last 10,000 characters)

AFM reco2 jobscript.
Input PFN = root://xrootd-archive.cr.cnaf.infn.it:1096//dune/usertests/d9/fb/000012_reco_data_2025-03-31T_142801Z_reco_data_2025-03-31T_150020Z_reco_data_2025-04-01T_083018Z_reco_data_2025-04-01T_101328Z.root
Setting up larsoft UPS area... /cvmfs/larsoft.opensciencegrid.org
Setting up DUNE UPS area... /cvmfs/dune.opensciencegrid.org/products/dune/
/cvmfs/larsoft.opensciencegrid.org/products/xrootd/v5_5_5a/Linux64bit+3.10-2.17-e26-p3915-prof/lib/libXrdPosixPreload.so
=== Start last 50 lines of lar log file ===
Trying to add data product: InitialTrack. This element does not exist in the element holder
%MSG
%MSG-e ShowerProducedPtrsHolder:   LArPandoraModularShowerCreation:pandoraShower@BeginModule  01-Apr-2025 16:58:47 CEST run: 20000031 subRun: 0 event: 20
Trying to add data product: InitialTrack. This element does not exist in the element holder
%MSG
%MSG-e ShowerProducedPtrsHolder:   LArPandoraModularShowerCreation:pandoraShower@BeginModule  01-Apr-2025 16:58:47 CEST run: 20000031 subRun: 0 event: 20
Trying to add data product: InitialTrack. This element does not exist in the element holder
%MSG
%MSG-e ShowerProducedPtrsHolder:   LArPandoraModularShowerCreation:pandoraShower@BeginModule  01-Apr-2025 16:58:47 CEST run: 20000031 subRun: 0 event: 20
Trying to add data product: InitialTrack. This element does not exist in the element holder
%MSG
%MSG-e ShowerProducedPtrsHolder:   LArPandoraModularShowerCreation:pandoraShower@BeginModule  01-Apr-2025 16:58:47 CEST run: 20000031 subRun: 0 event: 20
Trying to add data product: InitialTrack. This element does not exist in the element holder
%MSG
%MSG-e ShowerProducedPtrsHolder:   LArPandoraModularShowerCreation:pandoraShower@BeginModule  01-Apr-2025 16:58:47 CEST run: 20000031 subRun: 0 event: 20
Trying to add data product: InitialTrack. This element does not exist in the element holder
%MSG
%MSG-e ShowerProducedPtrsHolder:   LArPandoraModularShowerCreation:pandoraShower@BeginModule  01-Apr-2025 16:58:47 CEST run: 20000031 subRun: 0 event: 20
Trying to add data product: InitialTrack. This element does not exist in the element holder
%MSG
01-Apr-2025 16:58:47 CEST  Closed output file "000012_reco_data_2025-03-31T_142801Z_reco_data_2025-03-31T_150020Z_reco_data_2025-04-01T_083018Z_reco_data_2025-04-01T_101328Z_reco_data_2025-04-01T_145645Z.root"
01-Apr-2025 16:58:48 CEST  Closed input file "root://xrootd-archive.cr.cnaf.infn.it:1096//dune/usertests/d9/fb/000012_reco_data_2025-03-31T_142801Z_reco_data_2025-03-31T_150020Z_reco_data_2025-04-01T_083018Z_reco_data_2025-04-01T_101328Z.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                             0.376819      0.761434       1.69735      0.418883      0.466744        20     
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                 0.0373806     0.0434437     0.0755882     0.0375838     0.0133229       20     
reco:linecluster:LineCluster                           0.223748       0.32625      0.520918      0.227194      0.113922        20     
reco:trajcluster:TrajCluster                            0.11185      0.213787      0.560813      0.118026      0.144688        20     
reco:pandora:StandardPandora                          0.000609075    0.0815035     0.412696      0.0104766     0.118814        20     
reco:pandoraTrack:LArPandoraTrackCreation             7.2232e-05     0.0135772     0.0764681    0.000456441    0.0218036       20     
reco:pandoraShower:LArPandoraModularShowerCreation    0.000104362    0.0513427     0.188761     0.000636086    0.0687757       20     
reco:pandoracalo:Calorimetry                          0.000145393   0.00937823     0.0354271    0.000508922    0.0126166       20     
reco:pandorapid:Chi2ParticleID                         2.144e-05    0.000157971   0.000566294   5.69565e-05   0.000170548      20     
reco:trkshowersplit:TrackShowerHits                   0.000149373   0.00495919     0.023909     0.000304058   0.00717684       20     
reco:rns:RandomNumberSaver                            1.0061e-05    1.61211e-05   0.000107142    1.113e-05    2.09024e-05      20     
[art]:TriggerResults:TriggerResultInserter             5.77e-06     7.54615e-06    2.496e-05     6.505e-06    4.03261e-06      20     
end_path:out1:RootOutput                               1.471e-06    2.45455e-06     9.4e-06      2.095e-06    1.62242e-06      20     
end_path:out1:RootOutput(write)                       0.00178721     0.0167316     0.0481409    0.00281952     0.0184194       20     
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 2270.93 MB
  Peak resident set size usage (VmHWM): 1172.08 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 50 lines of lar log file ===
lar exit code 0
extractor_prod.py exit code 0
pdjson2metadata exit code 0
Metadata extraction succeeds
===Metadata JSON===
{
    "metadata": {
        "DUNE.campaign": "justIN.w5846s1",
        "art.file_format_era": "ART_2011a",
        "art.file_format_version": 15,
        "art.first_event": 1,
        "art.last_event": 20,
        "art.process_name": "Reco2",
        "art.run_type": "physics",
        "core.application": "art.reco",
        "core.application.family": "art",
        "core.application.name": "reco",
        "core.application.version": "v10_04_06d00",
        "core.data_stream": "out1",
        "core.data_tier": "full-reconstructed",
        "core.end_time": 1743519527.0,
        "core.event_count": 20,
        "core.file_format": "artroot",
        "core.file_type": "mc",
        "core.first_event_number": 1,
        "core.last_event_number": 20,
        "core.run_type": "physics",
        "core.runs": [
            20000031
        ],
        "core.runs_subruns": [
            2000003100000
        ],
        "core.start_time": 1743519442.0
    },
    "parents": [
        {
            "did": "usertests:000012_reco_data_2025-03-31T_142801Z_reco_data_2025-03-31T_150020Z_reco_data_2025-04-01T_083018Z_reco_data_2025-04-01T_101328Z.root"
        }
    ]
}
===================
.:
total 163108
-rw-r--r-- 1 dune001 dune 166867037 Apr  1 16:58 000012_reco_data_2025-03-31T_142801Z_reco_data_2025-03-31T_150020Z_reco_data_2025-04-01T_083018Z_reco_data_2025-04-01T_101328Z_reco_data_2025-04-01T_145645Z.root
-rw-r--r-- 1 dune001 dune    106615 Apr  1 16:58 000012_reco_data_2025-03-31T_142801Z_reco_data_2025-03-31T_150020Z_reco_data_2025-04-01T_083018Z_reco_data_2025-04-01T_101328Z_reco_2025-04-01T_145645Z.log
-rw-r--r-- 1 dune001 dune      7560 Apr  1 16:58 reco2_hist.root
-rw-r--r-- 1 dune001 dune      6923 Apr  1 16:58 jobscript.log
-rw-r--r-- 1 dune001 dune      1168 Apr  1 16:58 000012_reco_data_2025-03-31T_142801Z_reco_data_2025-03-31T_150020Z_reco_data_2025-04-01T_083018Z_reco_data_2025-04-01T_101328Z_reco_data_2025-04-01T_145645Z.root.json
-rw-r--r-- 1 dune001 dune      1042 Apr  1 16:58 000012_reco_data_2025-03-31T_142801Z_reco_data_2025-03-31T_150020Z_reco_data_2025-04-01T_083018Z_reco_data_2025-04-01T_101328Z_reco_data_2025-04-01T_145645Z.root.ext.json
-rw-r--r-- 1 dune001 dune       284 Apr  1 16:56 all-input-dids.txt
-rw-r--r-- 1 dune001 dune       197 Apr  1 16:58 justin-processed-pfns.txt
-rw-r--r-- 1 dune001 dune         0 Apr  1 16:56 debugprod.log
justIN time: 2025-04-03 08:24:36 UTC       justIN version: 01.03.00