justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

21 July 2025: This instance at RAL is read-only. Please do not try submitting new workflows for now.

Jobsub ID 234185.139@justin-prod-sched02.dune.hep.ac.uk

Jobsub ID234185.139@justin-prod-sched02.dune.hep.ac.uk
Workflow ID8057
Stage ID1
User nameepennacc@fnal.gov
HTCondor Groupgroup_dune.prod_mcsim
RequestedProcessors1
GPUNo
RSS bytes4194304000 (4000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2025-07-03 02:11:37
SiteUK_RAL-Tier1
EntryLIGO_UK_RAL_arc_ce05
Last heartbeat2025-07-03 13:24:31
From worker nodeHostnamedune001-6582799.0-lcg2696.gridpp.rl.ac.uk
cpuinfoAMD EPYC 9654 96-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit216000 (60 hours)
GPU
Inner Apptainer?True
Job stateaborted
Allocator namejustin-allocator-pro.dune.hep.ac.uk
Started2025-07-03 07:08:48
Input filesfardet-vd:prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250131T214401Z_gen_001609_supernova_g4stage1_g4stage2_detsim_reco.root
Outputting started2025-07-03 13:17:19
Output files
Finished2025-07-03 13:24:31
List job events     Wrapper job log

Jobscript log (last 10,000 characters)

same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  03-Jul-2025 13:11:42 UTC run: 5013 subRun: 0 event: 16090
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  03-Jul-2025 13:11:42 UTC run: 5013 subRun: 0 event: 16090
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  03-Jul-2025 13:11:42 UTC run: 5013 subRun: 0 event: 16090
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  03-Jul-2025 13:11:42 UTC run: 5013 subRun: 0 event: 16090
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  03-Jul-2025 13:11:42 UTC run: 5013 subRun: 0 event: 16090
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  03-Jul-2025 13:11:42 UTC run: 5013 subRun: 0 event: 16090
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  03-Jul-2025 13:11:42 UTC run: 5013 subRun: 0 event: 16090
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  03-Jul-2025 13:11:42 UTC run: 5013 subRun: 0 event: 16090
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  03-Jul-2025 13:11:42 UTC run: 5013 subRun: 0 event: 16090
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  03-Jul-2025 13:11:42 UTC run: 5013 subRun: 0 event: 16090
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  03-Jul-2025 13:11:42 UTC run: 5013 subRun: 0 event: 16090
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  03-Jul-2025 13:11:42 UTC run: 5013 subRun: 0 event: 16090
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  03-Jul-2025 13:11:42 UTC run: 5013 subRun: 0 event: 16090
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  03-Jul-2025 13:11:42 UTC run: 5013 subRun: 0 event: 16090
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  03-Jul-2025 13:11:42 UTC run: 5013 subRun: 0 event: 16090
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  03-Jul-2025 13:11:42 UTC run: 5013 subRun: 0 event: 16090
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  03-Jul-2025 13:11:42 UTC run: 5013 subRun: 0 event: 16090
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  03-Jul-2025 13:11:42 UTC run: 5013 subRun: 0 event: 16090
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  03-Jul-2025 13:11:42 UTC run: 5013 subRun: 0 event: 16090
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  03-Jul-2025 13:11:42 UTC run: 5013 subRun: 0 event: 16090
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  03-Jul-2025 13:11:42 UTC run: 5013 subRun: 0 event: 16090
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  03-Jul-2025 13:11:43 UTC run: 5013 subRun: 0 event: 16090
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  03-Jul-2025 13:11:43 UTC run: 5013 subRun: 0 event: 16090
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  03-Jul-2025 13:11:43 UTC run: 5013 subRun: 0 event: 16090
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  03-Jul-2025 13:11:43 UTC run: 5013 subRun: 0 event: 16090
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  03-Jul-2025 13:11:43 UTC run: 5013 subRun: 0 event: 16090
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  03-Jul-2025 13:11:43 UTC run: 5013 subRun: 0 event: 16090
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  03-Jul-2025 13:11:43 UTC run: 5013 subRun: 0 event: 16090
Comparing two wires in the same plane: return failure
%MSG
03-Jul-2025 13:15:09 UTC  Closed output file "prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250131T214401Z_gen_001609_supernova_g4stage1_g4stage2_detsim_reco_20250703T070855Z_reco2.root"
03-Jul-2025 13:15:09 UTC  Closed input file "root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-vd/73/b0/prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250131T214401Z_gen_001609_supernova_g4stage1_g4stage2_detsim_reco.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              1868.96       2152.33       2755.79       2029.43       285.547        10     
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                 0.0108585     0.0545236     0.174632      0.0339981     0.0527186       10     
reco:pandora:StandardPandora                            1652.93       1896.74       2456.11       1745.7        272.75         10     
reco:pandoraTrack:LArPandoraTrackCreation             0.000431429    0.0267869     0.157808      0.0148054     0.0447432       10     
reco:pandoraShower:LArPandoraModularShowerCreation    0.000355865   0.00271908    0.00811451    0.00170749    0.00233027       10     
reco:pandoracalo:Calorimetry                          0.000552641   0.00243037     0.0150796    0.000803208   0.00425914       10     
reco:pandorapid:Chi2ParticleID                        8.1302e-05    0.000257627   0.000714213   0.000151363   0.000228283      10     
reco:linecluster:LineCluster                            1.15733       1.25478       1.71596       1.21231      0.155217        10     
reco:trajcluster:TrajCluster                           0.295546      0.355842      0.458539      0.347137      0.0551025       10     
reco:pmtracktc:PMAlgTrackMaker                          3.77841       5.9535        7.40371       6.38903       1.11994        10     
reco:emtrkmichelid:EmTrackMichelId                      179.663       246.255       290.049       254.451       39.874         10     
reco:solarflash:SolarOpFlash                            1.36381       1.42307       1.55419       1.40477      0.0570013       10     
[art]:TriggerResults:TriggerResultInserter            5.1267e-05    8.39472e-05   0.000121754   8.5559e-05    1.93705e-05      10     
end_path:out1:RootOutput                              1.4702e-05    2.62666e-05   6.9975e-05    2.4662e-05    1.54436e-05      10     
end_path:out1:RootOutput(write)                         0.23062      0.254085      0.307463      0.253074      0.0212546       10     
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4076.42 MB
  Peak resident set size usage (VmHWM): 2474.11 MB
====================================================================================================
Art has completed and will exit with status 0.
Moving on to metadata extractor 
============OUTPUT FILE:  prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250131T214401Z_gen_001609_supernova_g4stage1_g4stage2_detsim_reco_20250703T070855Z_reco2.root
Extracted metadata
checking file====== fardet-vd:prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250131T214401Z_gen_001609_supernova_g4stage1_g4stage2_detsim_reco.root
MD conversion to MetaCat OK
all-input-dids.txt
debugprod.log
jobscript.log
justin-processed-pfns.txt
old_md.json
prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250131T214401Z_gen_001609_supernova_g4stage1_g4stage2_detsim_reco_20250703T070855Z_reco2.root
prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250131T214401Z_gen_001609_supernova_g4stage1_g4stage2_detsim_reco_20250703T070855Z_reco2.root.ext.json
prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250131T214401Z_gen_001609_supernova_g4stage1_g4stage2_detsim_reco_20250703T070855Z_reco2.root.json
reco2_hist.root
justIN time: 2025-08-15 10:20:01 UTC       justIN version: 01.03.02