justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

21 July 2025: This instance at RAL is read-only. Please do not try submitting new workflows for now.

Jobsub ID 179009.2@justin-prod-sched02.dune.hep.ac.uk

Jobsub ID179009.2@justin-prod-sched02.dune.hep.ac.uk
Workflow ID6304
Stage ID1
User nameamoor@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
GPUNo
RSS bytes5242880000 (5000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2025-04-17 16:23:33
SiteUS_FNAL-FermiGrid
EntryFNAL_GPGrid_ce03_mcore_op_duneonly
Last heartbeat2025-04-17 19:40:22
From worker nodeHostnamedunegli-4595642-0-fnpc23016.fnal.gov
cpuinfoAMD EPYC 7543 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes5242880000 (5000 MiB)
Wall seconds limit172800 (48 hours)
GPU
Inner Apptainer?True
Job statefinished
Allocator namejustin-allocator-pro.dune.hep.ac.uk
Started2025-04-17 18:58:46
Input filesusertests:000599_reco_data_2025-04-02T_083608Z_reco_data_2025-04-02T_210906Z_reco_data_2025-04-04T_114458Z_reco_data_2025-04-07T_111010Z_reco_data_2025-04-08T_101717Z.root
JobscriptExit code0
Real time41m (2484s)
CPU time39m (2384s = 95%)
Max RSS bytes2519089152 (2402 MiB)
Outputting started2025-04-17 19:40:12
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/amoor/06304/1/001/Mergeana-20250417T194010_ana_hist.root
Finished2025-04-17 19:40:22
Saved logsjustin-logs:179009.2-justin-prod-sched02.dune.hep.ac.uk.logs.tgz
List job events     Wrapper job log

Jobscript log (last 10,000 characters)

ning: there was no vertex found for PFParticle with ID 33
Warning: there was no track found for track-like PFParticle with ID 33
Begin processing the 984th record. run: 20000031 subRun: 0 event: 984 at 17-Apr-2025 19:39:03 UTC
Analysing.

Begin processing the 985th record. run: 20000031 subRun: 0 event: 985 at 17-Apr-2025 19:39:03 UTC
Analysing.

Begin processing the 986th record. run: 20000031 subRun: 0 event: 986 at 17-Apr-2025 19:39:06 UTC
Analysing.

Warning: there were 48 reconstructed PFParticle daughters; only the first 10 being stored in tree
Warning: there were 21 reconstructed PFParticle clusters; only the first 10 being stored in tree
Begin processing the 987th record. run: 20000031 subRun: 0 event: 987 at 17-Apr-2025 19:39:12 UTC
Analysing.

Begin processing the 988th record. run: 20000031 subRun: 0 event: 988 at 17-Apr-2025 19:39:17 UTC
Analysing.

Warning: there were 28 reconstructed PFParticle daughters; only the first 10 being stored in tree
Warning: there were 18 reconstructed PFParticle clusters; only the first 10 being stored in tree
Begin processing the 989th record. run: 20000031 subRun: 0 event: 989 at 17-Apr-2025 19:39:23 UTC
Analysing.

Warning: there were 16 reconstructed PFParticle daughters; only the first 10 being stored in tree
Begin processing the 990th record. run: 20000031 subRun: 0 event: 990 at 17-Apr-2025 19:39:29 UTC
Analysing.

Begin processing the 991st record. run: 20000031 subRun: 0 event: 991 at 17-Apr-2025 19:39:29 UTC
Analysing.

Begin processing the 992nd record. run: 20000031 subRun: 0 event: 992 at 17-Apr-2025 19:39:29 UTC
Analysing.

Begin processing the 993rd record. run: 20000031 subRun: 0 event: 993 at 17-Apr-2025 19:39:31 UTC
Analysing.

Warning: there were 62 reconstructed PFParticle daughters; only the first 10 being stored in tree
Warning: there was no vertex found for PFParticle with ID 56
Warning: there was no track found for track-like PFParticle with ID 56
Begin processing the 994th record. run: 20000031 subRun: 0 event: 994 at 17-Apr-2025 19:39:41 UTC
Analysing.

Warning: there were 18 reconstructed PFParticle daughters; only the first 10 being stored in tree
Begin processing the 995th record. run: 20000031 subRun: 0 event: 995 at 17-Apr-2025 19:39:48 UTC
Analysing.

Begin processing the 996th record. run: 20000031 subRun: 0 event: 996 at 17-Apr-2025 19:39:48 UTC
Analysing.

Warning: there were 29 reconstructed PFParticle daughters; only the first 10 being stored in tree
Warning: there were 23 reconstructed PFParticle clusters; only the first 10 being stored in tree
Warning: there was no vertex found for PFParticle with ID 20
Warning: there was no track found for track-like PFParticle with ID 20
Begin processing the 997th record. run: 20000031 subRun: 0 event: 997 at 17-Apr-2025 19:39:57 UTC
Analysing.

Warning: there were 42 reconstructed PFParticle daughters; only the first 10 being stored in tree
Warning: there were 21 reconstructed PFParticle clusters; only the first 10 being stored in tree
Warning: there was no vertex found for PFParticle with ID 40
Warning: there was no track found for track-like PFParticle with ID 40
%MSG-e AnalysisTree:limits:  AnalysisTree:analysistree@BeginModule  17-Apr-2025 19:40:08 UTC run: 20000031 subRun: 0 event: 997
the pandoraTrack track #0 has 2787 hits on calorimetry plane #0, only 2000 stored in tree
%MSG
Begin processing the 998th record. run: 20000031 subRun: 0 event: 998 at 17-Apr-2025 19:40:08 UTC
Analysing.

Begin processing the 999th record. run: 20000031 subRun: 0 event: 999 at 17-Apr-2025 19:40:08 UTC
Analysing.

Begin processing the 1000th record. run: 20000031 subRun: 0 event: 1000 at 17-Apr-2025 19:40:08 UTC
Analysing.

17-Apr-2025 19:40:09 UTC  Closed output file "000599_reco_data_2025-04-02T_083608Z_reco_data_2025-04-02T_210906Z_reco_data_2025-04-04T_114458Z_reco_data_2025-04-07T_111010Z_reco_data_2025-04-08T_101717Z_reco_data_2025-04-17T_185849Z.root"
17-Apr-2025 19:40:09 UTC  Closed input file "root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/usertests/b5/5f/000599_reco_data_2025-04-02T_083608Z_reco_data_2025-04-02T_210906Z_reco_data_2025-04-04T_114458Z_reco_data_2025-04-07T_111010Z_reco_data_2025-04-08T_101717Z.root"

==================================================================================================================================
TimeTracker printout (sec)                          Min           Avg           Max         Median          RMS         nEvts   
==================================================================================================================================
Full event                                       0.0121559      2.32222       47.2183      0.0349286      4.35734       1000    
----------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                          0.000360427   0.000798798    0.0198902    0.000676153   0.00100537      1000    
end_path:out1:RootOutput                         1.553e-06    4.99435e-06   9.2769e-05     5.31e-06     3.13132e-06     1000    
end_path:analysistree:AnalysisTree              0.00994919      2.30203       46.1117      0.0131635      4.33309       1000    
end_path:pmtrackeff:NeutrinoTrackingEff         0.000128238   0.000211234   0.00196466    0.000175579   0.000121777     1000    
end_path:pmtracktceff:NeutrinoTrackingEff       0.000114873   0.000132662   0.00122059    0.000123229   4.23458e-05     1000    
end_path:pandoraeff:NeutrinoTrackingEff         0.000110343   0.000125097   0.000994433   0.000118835   3.63361e-05     1000    
end_path:pmtrajfiteff:NeutrinoTrackingEff       0.000108049   0.00012099    0.000868559   0.000115904   3.05232e-05     1000    
end_path:pmtrajfittceff:NeutrinoTrackingEff     0.000106315   0.000118298   0.000782263   0.000113735   2.82251e-05     1000    
end_path:pandorashweff:NeutrinoShowerEff        1.7264e-05    2.53874e-05   0.000500387   2.1146e-05    2.73809e-05     1000    
end_path:emshowershweff:NeutrinoShowerEff       1.4168e-05    1.9452e-05    0.00030466    1.5861e-05    1.92149e-05     1000    
end_path:trajclustershweff:NeutrinoShowerEff    1.4247e-05    1.91337e-05   0.000293006   1.5901e-05    1.77476e-05     1000    
end_path:runcheckereco:CheckRecoEnergy           5.581e-06    7.33993e-06   0.000177734    6.943e-06    5.51109e-06     1000    
end_path:flashmatchAna:FlashMatchAna            4.7853e-05     0.0170656      1.12103     0.00278034     0.0570638      1000    
end_path:out1:RootOutput(write)                 0.000718249   0.00123177    0.00940686    0.00115582    0.000465291     1000    
==================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 3826.67 MB
  Peak resident set size usage (VmHWM): 2519.09 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 200 lines of lar log file ===
lar exit code 0
extractor_prod.py exit code 0
pdjson2metadata exit code 0
Metadata extraction succeeds
===Metadata JSON===
{
    "metadata": {
        "DUNE.campaign": "justIN.w6304s1",
        "art.file_format_era": "ART_2011a",
        "art.file_format_version": 15,
        "art.first_event": 1,
        "art.last_event": 1000,
        "art.process_name": "Mergeana",
        "art.run_type": "physics",
        "core.application": "art.reco",
        "core.application.family": "art",
        "core.application.name": "reco",
        "core.application.version": "v10_04_06d00",
        "core.data_stream": "out1",
        "core.data_tier": "full-reconstructed",
        "core.end_time": 1744918809.0,
        "core.event_count": 1000,
        "core.file_format": "artroot",
        "core.file_type": "mc",
        "core.first_event_number": 1,
        "core.last_event_number": 1000,
        "core.run_type": "physics",
        "core.runs": [
            20000031
        ],
        "core.runs_subruns": [
            2000003100000
        ],
        "core.start_time": 1744916340.0
    },
    "parents": [
        {
            "did": "usertests:000599_reco_data_2025-04-02T_083608Z_reco_data_2025-04-02T_210906Z_reco_data_2025-04-04T_114458Z_reco_data_2025-04-07T_111010Z_reco_data_2025-04-08T_101717Z.root"
        }
    ]
}
===================
.:
total 8116260
-rw-r--r-- 1 dunegli fnalgrid 8201603350 Apr 17 19:40 000599_reco_data_2025-04-02T_083608Z_reco_data_2025-04-02T_210906Z_reco_data_2025-04-04T_114458Z_reco_data_2025-04-07T_111010Z_reco_data_2025-04-08T_101717Z_reco_data_2025-04-17T_185849Z.root
-rw-r--r-- 1 dunegli fnalgrid  109179809 Apr 17 19:40 Mergeana-20250417T194010_ana_hist.root
-rw-r--r-- 1 dunegli fnalgrid     228154 Apr 17 19:40 000599_reco_data_2025-04-02T_083608Z_reco_data_2025-04-02T_210906Z_reco_data_2025-04-04T_114458Z_reco_data_2025-04-07T_111010Z_reco_data_2025-04-08T_101717Z_reco_2025-04-17T_185849Z.log
-rw-r--r-- 1 dunegli fnalgrid      16117 Apr 17 19:40 jobscript.log
-rw-r--r-- 1 dunegli fnalgrid       1207 Apr 17 19:40 000599_reco_data_2025-04-02T_083608Z_reco_data_2025-04-02T_210906Z_reco_data_2025-04-04T_114458Z_reco_data_2025-04-07T_111010Z_reco_data_2025-04-08T_101717Z_reco_data_2025-04-17T_185849Z.root.json
-rw-r--r-- 1 dunegli fnalgrid       1112 Apr 17 19:40 000599_reco_data_2025-04-02T_083608Z_reco_data_2025-04-02T_210906Z_reco_data_2025-04-04T_114458Z_reco_data_2025-04-07T_111010Z_reco_data_2025-04-08T_101717Z_reco_data_2025-04-17T_185849Z.root.ext.json
-rw-r--r-- 1 dunegli fnalgrid        344 Apr 17 18:58 all-input-dids.txt
-rw-r--r-- 1 dunegli fnalgrid        248 Apr 17 19:40 justin-processed-pfns.txt
-rw-r--r-- 1 dunegli fnalgrid          0 Apr 17 18:58 debugprod.log
justIN time: 2025-08-21 20:04:52 UTC       justIN version: 01.03.02