justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

21 July 2025: This instance at RAL is read-only. Please do not try submitting new workflows for now.

Jobsub ID 233491.103@justin-prod-sched02.dune.hep.ac.uk

Jobsub ID233491.103@justin-prod-sched02.dune.hep.ac.uk
Workflow ID8047
Stage ID1
User nameepennacc@fnal.gov
HTCondor Groupgroup_dune.prod_mcsim
RequestedProcessors1
GPUNo
RSS bytes4194304000 (4000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2025-07-01 23:20:12
SiteUK_Durham
EntryDUNE_UK_SGridDurham_ce3
Last heartbeat2025-07-02 06:59:45
From worker nodeHostnamen263.dur.scotgrid.ac.uk
cpuinfoAMD EPYC 9534 64-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit171000 (47 hours)
GPU
Inner Apptainer?True
Job statefinished
Allocator namejustin-allocator-pro.dune.hep.ac.uk
Started2025-07-02 03:39:16
Input filesfardet-vd:prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250128T174311Z_gen_001699_supernova_g4stage1_g4stage2_detsim_reco.root
JobscriptExit code0
Real time3h (11789s)
CPU time2h (8704s = 73%)
Max RSS bytes2578042880 (2458 MiB)
Outputting started2025-07-02 06:55:46
Output filesfardet-vd:prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250128T174311Z_gen_001699_supernova_g4stage1_g4stage2_detsim_reco_20250702T034001Z_reco2.root
Finished2025-07-02 06:59:33
Saved logsjustin-logs:233491.103-justin-prod-sched02.dune.hep.ac.uk.logs.tgz
List job events     Wrapper job log

Jobscript log (last 10,000 characters)

same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  02-Jul-2025 07:54:19 BST run: 4969 subRun: 0 event: 16990
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  02-Jul-2025 07:54:19 BST run: 4969 subRun: 0 event: 16990
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  02-Jul-2025 07:54:19 BST run: 4969 subRun: 0 event: 16990
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  02-Jul-2025 07:54:19 BST run: 4969 subRun: 0 event: 16990
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  02-Jul-2025 07:54:19 BST run: 4969 subRun: 0 event: 16990
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  02-Jul-2025 07:54:19 BST run: 4969 subRun: 0 event: 16990
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  02-Jul-2025 07:54:19 BST run: 4969 subRun: 0 event: 16990
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  02-Jul-2025 07:54:19 BST run: 4969 subRun: 0 event: 16990
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  02-Jul-2025 07:54:19 BST run: 4969 subRun: 0 event: 16990
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  02-Jul-2025 07:54:19 BST run: 4969 subRun: 0 event: 16990
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  02-Jul-2025 07:54:19 BST run: 4969 subRun: 0 event: 16990
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  02-Jul-2025 07:54:19 BST run: 4969 subRun: 0 event: 16990
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  02-Jul-2025 07:54:19 BST run: 4969 subRun: 0 event: 16990
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  02-Jul-2025 07:54:19 BST run: 4969 subRun: 0 event: 16990
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  02-Jul-2025 07:54:19 BST run: 4969 subRun: 0 event: 16990
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  02-Jul-2025 07:54:19 BST run: 4969 subRun: 0 event: 16990
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  02-Jul-2025 07:54:19 BST run: 4969 subRun: 0 event: 16990
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  02-Jul-2025 07:54:19 BST run: 4969 subRun: 0 event: 16990
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  02-Jul-2025 07:54:20 BST run: 4969 subRun: 0 event: 16990
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  02-Jul-2025 07:54:20 BST run: 4969 subRun: 0 event: 16990
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  02-Jul-2025 07:54:20 BST run: 4969 subRun: 0 event: 16990
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  02-Jul-2025 07:54:20 BST run: 4969 subRun: 0 event: 16990
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  02-Jul-2025 07:54:20 BST run: 4969 subRun: 0 event: 16990
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  02-Jul-2025 07:54:20 BST run: 4969 subRun: 0 event: 16990
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  02-Jul-2025 07:54:20 BST run: 4969 subRun: 0 event: 16990
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  02-Jul-2025 07:54:20 BST run: 4969 subRun: 0 event: 16990
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  02-Jul-2025 07:54:20 BST run: 4969 subRun: 0 event: 16990
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  02-Jul-2025 07:54:20 BST run: 4969 subRun: 0 event: 16990
Comparing two wires in the same plane: return failure
%MSG
02-Jul-2025 07:55:43 BST  Closed output file "prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250128T174311Z_gen_001699_supernova_g4stage1_g4stage2_detsim_reco_20250702T034001Z_reco2.root"
02-Jul-2025 07:55:44 BST  Closed input file "root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-vd/40/4f/prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250128T174311Z_gen_001699_supernova_g4stage1_g4stage2_detsim_reco.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              714.976       886.394       1027.54       879.529       100.935        10     
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.00246143     0.0360924     0.160659     0.00657826     0.0608089       10     
reco:pandora:StandardPandora                            600.274       780.269       911.729       786.003       102.621        10     
reco:pandoraTrack:LArPandoraTrackCreation             0.00044655    0.00542145     0.0413136    0.00165978     0.0119699       10     
reco:pandoraShower:LArPandoraModularShowerCreation    0.000595889   0.00159898     0.0082115    0.000873079   0.00220885       10     
reco:pandoracalo:Calorimetry                          0.00023169    0.00047492    0.00223894    0.000279375   0.000588806      10     
reco:pandorapid:Chi2ParticleID                         6.285e-05    0.000159962   0.000833299   8.4955e-05    0.000224811      10     
reco:linecluster:LineCluster                            1.86653       2.86717       3.53707       2.89983      0.485826        10     
reco:trajcluster:TrajCluster                           0.166357      0.227236      0.258934      0.243838      0.0332911       10     
reco:pmtracktc:PMAlgTrackMaker                           3.319        5.5954        6.75907       6.33862       1.35125        10     
reco:emtrkmichelid:EmTrackMichelId                      66.8003       94.7715       108.017       103.282       14.6929        10     
reco:solarflash:SolarOpFlash                            2.03675       2.43994       2.97569       2.43953      0.266613        10     
[art]:TriggerResults:TriggerResultInserter             1.676e-05     2.414e-05     4.455e-05    2.2105e-05    7.7651e-06       10     
end_path:out1:RootOutput                               3.63e-06     7.9222e-06     2.414e-05    6.2205e-06    5.51796e-06      10     
end_path:out1:RootOutput(write)                        0.136166      0.179226      0.197869      0.192499      0.0229709       10     
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4050.26 MB
  Peak resident set size usage (VmHWM): 2578.04 MB
====================================================================================================
Art has completed and will exit with status 0.
Moving on to metadata extractor 
============OUTPUT FILE:  prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250128T174311Z_gen_001699_supernova_g4stage1_g4stage2_detsim_reco_20250702T034001Z_reco2.root
Extracted metadata
checking file====== fardet-vd:prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250128T174311Z_gen_001699_supernova_g4stage1_g4stage2_detsim_reco.root
MD conversion to MetaCat OK
all-input-dids.txt
debugprod.log
jobscript.log
justin-processed-pfns.txt
old_md.json
prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250128T174311Z_gen_001699_supernova_g4stage1_g4stage2_detsim_reco_20250702T034001Z_reco2.root
prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250128T174311Z_gen_001699_supernova_g4stage1_g4stage2_detsim_reco_20250702T034001Z_reco2.root.ext.json
prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250128T174311Z_gen_001699_supernova_g4stage1_g4stage2_detsim_reco_20250702T034001Z_reco2.root.json
reco2_hist.root
justIN time: 2025-08-14 20:34:03 UTC       justIN version: 01.03.02