justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

21 July 2025: This instance at RAL is read-only. Please do not try submitting new workflows for now.

Jobsub ID 426673.8@justin-prod-sched01.dune.hep.ac.uk

Jobsub ID426673.8@justin-prod-sched01.dune.hep.ac.uk
Workflow ID8051
Stage ID1
User namehiguera@fnal.gov
HTCondor Groupgroup_dune.prod_mcsim
RequestedProcessors1
GPUNo
RSS bytes4194304000 (4000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2025-07-03 05:55:00
SiteNL_SURFsara
EntryDUNE_SurfSARA_arc01
Last heartbeat2025-07-03 13:17:46
From worker nodeHostnamewn-la-03.gina.surf.nl
cpuinfoAMD EPYC 9754 128-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit129600 (36 hours)
GPU
Inner Apptainer?True
Job statefinished
Allocator namejustin-allocator-pro.dune.hep.ac.uk
Started2025-07-03 06:53:58
Input filesfardet-vd:prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250205T151311Z_gen_002518_supernova_g4stage1_g4stage2_detsim_reco.root
JobscriptExit code0
Real time6h (22347s)
CPU time6h (21913s = 98%)
Max RSS bytes2564804608 (2445 MiB)
Outputting started2025-07-03 13:06:25
Output filesfardet-vd:prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250205T151311Z_gen_002518_supernova_g4stage1_g4stage2_detsim_reco_20250703T065409Z_reco2.root
Finished2025-07-03 13:17:46
Saved logsjustin-logs:426673.8-justin-prod-sched01.dune.hep.ac.uk.logs.tgz
List job events     Wrapper job log

Jobscript log (last 10,000 characters)

MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  03-Jul-2025 15:03:47 CEST run: 5067 subRun: 0 event: 25180
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  03-Jul-2025 15:03:47 CEST run: 5067 subRun: 0 event: 25180
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  03-Jul-2025 15:03:47 CEST run: 5067 subRun: 0 event: 25180
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  03-Jul-2025 15:03:47 CEST run: 5067 subRun: 0 event: 25180
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  03-Jul-2025 15:03:47 CEST run: 5067 subRun: 0 event: 25180
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  03-Jul-2025 15:03:47 CEST run: 5067 subRun: 0 event: 25180
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  03-Jul-2025 15:03:47 CEST run: 5067 subRun: 0 event: 25180
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  03-Jul-2025 15:03:47 CEST run: 5067 subRun: 0 event: 25180
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  03-Jul-2025 15:03:47 CEST run: 5067 subRun: 0 event: 25180
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  03-Jul-2025 15:03:47 CEST run: 5067 subRun: 0 event: 25180
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  03-Jul-2025 15:03:47 CEST run: 5067 subRun: 0 event: 25180
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  03-Jul-2025 15:03:47 CEST run: 5067 subRun: 0 event: 25180
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  03-Jul-2025 15:03:47 CEST run: 5067 subRun: 0 event: 25180
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  03-Jul-2025 15:03:47 CEST run: 5067 subRun: 0 event: 25180
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  03-Jul-2025 15:03:47 CEST run: 5067 subRun: 0 event: 25180
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  03-Jul-2025 15:03:47 CEST run: 5067 subRun: 0 event: 25180
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  03-Jul-2025 15:03:47 CEST run: 5067 subRun: 0 event: 25180
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  03-Jul-2025 15:03:47 CEST run: 5067 subRun: 0 event: 25180
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  03-Jul-2025 15:03:47 CEST run: 5067 subRun: 0 event: 25180
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  03-Jul-2025 15:03:47 CEST run: 5067 subRun: 0 event: 25180
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  03-Jul-2025 15:03:47 CEST run: 5067 subRun: 0 event: 25180
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  03-Jul-2025 15:03:47 CEST run: 5067 subRun: 0 event: 25180
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  03-Jul-2025 15:03:47 CEST run: 5067 subRun: 0 event: 25180
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  03-Jul-2025 15:03:47 CEST run: 5067 subRun: 0 event: 25180
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  03-Jul-2025 15:03:47 CEST run: 5067 subRun: 0 event: 25180
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  03-Jul-2025 15:03:47 CEST run: 5067 subRun: 0 event: 25180
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  03-Jul-2025 15:03:47 CEST run: 5067 subRun: 0 event: 25180
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  03-Jul-2025 15:03:47 CEST run: 5067 subRun: 0 event: 25180
Comparing two wires in the same plane: return failure
%MSG
03-Jul-2025 15:06:17 CEST  Closed output file "prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250205T151311Z_gen_002518_supernova_g4stage1_g4stage2_detsim_reco_20250703T065409Z_reco2.root"
03-Jul-2025 15:06:18 CEST  Closed input file "root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-vd/8b/ef/prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250205T151311Z_gen_002518_supernova_g4stage1_g4stage2_detsim_reco.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              1505.79       2198.37       3013.32       2184.12       391.065        10     
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.00930971     0.0459856     0.158748      0.0225289     0.0512822       10     
reco:pandora:StandardPandora                            1348.15       1967.99       2755.98       1937.05       376.734        10     
reco:pandoraTrack:LArPandoraTrackCreation             0.000365879    0.0218627      0.15189     0.00218062     0.044427        10     
reco:pandoraShower:LArPandoraModularShowerCreation    0.000277095   0.00302921     0.0141719    0.000869032   0.00430426       10     
reco:pandoracalo:Calorimetry                          0.000256604   0.00685158     0.0206041    0.00292864    0.00739311       10     
reco:pandorapid:Chi2ParticleID                        5.0797e-05    0.000264766   0.00121282     9.043e-05    0.00037673       10     
reco:linecluster:LineCluster                            1.15334       1.22834       1.74738       1.16953      0.173473        10     
reco:trajcluster:TrajCluster                           0.315878      0.395385      0.432573      0.406484      0.0310292       10     
reco:pmtracktc:PMAlgTrackMaker                          5.14621       7.20267       8.57406       7.32959       1.01214        10     
reco:emtrkmichelid:EmTrackMichelId                      146.092       219.782       281.361       231.831       41.9087        10     
reco:solarflash:SolarOpFlash                            1.26751       1.4279        1.65964       1.41056      0.0923054       10     
[art]:TriggerResults:TriggerResultInserter            2.9294e-05    4.54122e-05   9.0626e-05    4.0536e-05    1.65057e-05      10     
end_path:out1:RootOutput                               6.43e-06     1.19528e-05   3.6404e-05    9.8845e-06    8.30408e-06      10     
end_path:out1:RootOutput(write)                        0.220793      0.267876       0.28661      0.271328      0.0170047       10     
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4034.08 MB
  Peak resident set size usage (VmHWM): 2564.8 MB
====================================================================================================
Art has completed and will exit with status 0.
Moving on to metadata extractor 
============OUTPUT FILE:  prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250205T151311Z_gen_002518_supernova_g4stage1_g4stage2_detsim_reco_20250703T065409Z_reco2.root
Extracted metadata
checking file====== fardet-vd:prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250205T151311Z_gen_002518_supernova_g4stage1_g4stage2_detsim_reco.root
MD conversion to MetaCat OK
all-input-dids.txt
debugprod.log
jobscript.log
justin-processed-pfns.txt
old_md.json
prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250205T151311Z_gen_002518_supernova_g4stage1_g4stage2_detsim_reco_20250703T065409Z_reco2.root
prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250205T151311Z_gen_002518_supernova_g4stage1_g4stage2_detsim_reco_20250703T065409Z_reco2.root.ext.json
prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250205T151311Z_gen_002518_supernova_g4stage1_g4stage2_detsim_reco_20250703T065409Z_reco2.root.json
reco2_hist.root
justIN time: 2025-08-14 20:37:24 UTC       justIN version: 01.03.02