justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

21 July 2025: This instance at RAL is read-only. Please do not try submitting new workflows for now.

Jobsub ID 229470.0@justin-prod-sched02.dune.hep.ac.uk

Jobsub ID229470.0@justin-prod-sched02.dune.hep.ac.uk
Workflow ID7846
Stage ID1
User nameepennacc@fnal.gov
HTCondor Groupgroup_dune.prod_mcsim
RequestedProcessors1
GPUNo
RSS bytes6291456000 (6000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2025-06-25 17:17:44
SiteUK_RAL-Tier1
EntryLIGO_UK_RAL_arc_ce03
Last heartbeat2025-06-26 00:07:20
From worker nodeHostnamedune001-7143231.0-lcg2710.gridpp.rl.ac.uk
cpuinfoAMD EPYC 9654 96-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes6291456000 (6000 MiB)
Wall seconds limit216000 (60 hours)
GPU
Inner Apptainer?True
Job statefinished
Allocator namejustin-allocator-pro.dune.hep.ac.uk
Started2025-06-25 20:48:01
Input filesfardet-vd:prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250205T044331Z_gen_000255_supernova_g4stage1_g4stage2_detsim_reco.root
JobscriptExit code0
Real time3h (11799s)
CPU time3h (11396s = 96%)
Max RSS bytes2358824960 (2249 MiB)
Outputting started2025-06-26 00:04:40
Output filesfardet-vd:prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250205T044331Z_gen_000255_supernova_g4stage1_g4stage2_detsim_reco_20250625T204806Z_reco2.root
Finished2025-06-26 00:07:20
Saved logsjustin-logs:229470.0-justin-prod-sched02.dune.hep.ac.uk.logs.tgz
List job events     Wrapper job log

Jobscript log (last 10,000 characters)

0
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  26-Jun-2025 00:02:44 UTC run: 5057 subRun: 0 event: 2550
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  26-Jun-2025 00:02:44 UTC run: 5057 subRun: 0 event: 2550
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  26-Jun-2025 00:02:44 UTC run: 5057 subRun: 0 event: 2550
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  26-Jun-2025 00:02:44 UTC run: 5057 subRun: 0 event: 2550
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  26-Jun-2025 00:02:44 UTC run: 5057 subRun: 0 event: 2550
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  26-Jun-2025 00:02:44 UTC run: 5057 subRun: 0 event: 2550
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  26-Jun-2025 00:02:44 UTC run: 5057 subRun: 0 event: 2550
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  26-Jun-2025 00:02:44 UTC run: 5057 subRun: 0 event: 2550
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  26-Jun-2025 00:02:44 UTC run: 5057 subRun: 0 event: 2550
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  26-Jun-2025 00:02:44 UTC run: 5057 subRun: 0 event: 2550
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  26-Jun-2025 00:02:44 UTC run: 5057 subRun: 0 event: 2550
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  26-Jun-2025 00:02:44 UTC run: 5057 subRun: 0 event: 2550
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  26-Jun-2025 00:02:44 UTC run: 5057 subRun: 0 event: 2550
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  26-Jun-2025 00:02:44 UTC run: 5057 subRun: 0 event: 2550
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  26-Jun-2025 00:02:44 UTC run: 5057 subRun: 0 event: 2550
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  26-Jun-2025 00:02:44 UTC run: 5057 subRun: 0 event: 2550
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  26-Jun-2025 00:02:44 UTC run: 5057 subRun: 0 event: 2550
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  26-Jun-2025 00:02:44 UTC run: 5057 subRun: 0 event: 2550
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  26-Jun-2025 00:02:44 UTC run: 5057 subRun: 0 event: 2550
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  26-Jun-2025 00:02:44 UTC run: 5057 subRun: 0 event: 2550
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  26-Jun-2025 00:02:44 UTC run: 5057 subRun: 0 event: 2550
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  26-Jun-2025 00:02:44 UTC run: 5057 subRun: 0 event: 2550
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  26-Jun-2025 00:02:44 UTC run: 5057 subRun: 0 event: 2550
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  26-Jun-2025 00:02:44 UTC run: 5057 subRun: 0 event: 2550
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  26-Jun-2025 00:02:44 UTC run: 5057 subRun: 0 event: 2550
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  26-Jun-2025 00:02:44 UTC run: 5057 subRun: 0 event: 2550
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  26-Jun-2025 00:02:44 UTC run: 5057 subRun: 0 event: 2550
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  26-Jun-2025 00:02:44 UTC run: 5057 subRun: 0 event: 2550
Comparing two wires in the same plane: return failure
%MSG
26-Jun-2025 00:04:35 UTC  Closed output file "prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250205T044331Z_gen_000255_supernova_g4stage1_g4stage2_detsim_reco_20250625T204806Z_reco2.root"
26-Jun-2025 00:04:35 UTC  Closed input file "root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-vd/cb/83/prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250205T044331Z_gen_000255_supernova_g4stage1_g4stage2_detsim_reco.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              1001.14       1153.71       1297.93       1149.41       86.8679        10     
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.00397304     0.040448      0.142953      0.0187773     0.0509485       10     
reco:pandora:StandardPandora                            860.581       1029.62       1159.31       1027.04       90.112         10     
reco:pandoraTrack:LArPandoraTrackCreation             0.000257449    0.0105777     0.0505753    0.00739952     0.0143732       10     
reco:pandoraShower:LArPandoraModularShowerCreation    0.00026483    0.00174093    0.00633162    0.00103308    0.00168892       10     
reco:pandoracalo:Calorimetry                          0.000294905    0.0023947     0.014619     0.000733758   0.00416821       10     
reco:pandorapid:Chi2ParticleID                         4.644e-05    0.000195644   0.000448477   0.000122866   0.000144457      10     
reco:linecluster:LineCluster                            1.13439       1.43092       3.20972       1.17326      0.617037        10     
reco:trajcluster:TrajCluster                           0.218071      0.315736      0.354548      0.325087      0.0384287       10     
reco:pmtracktc:PMAlgTrackMaker                          3.76741       6.21298       7.99828       6.2975        1.4173         10     
reco:emtrkmichelid:EmTrackMichelId                      98.8946       114.307       131.304       110.433       11.2276        10     
reco:solarflash:SolarOpFlash                            1.22176       1.54118       2.5433        1.43071      0.358831        10     
[art]:TriggerResults:TriggerResultInserter            2.4366e-05    4.42886e-05   7.1458e-05    3.95195e-05   1.40283e-05      10     
end_path:out1:RootOutput                               7.581e-06    1.39993e-05   3.4973e-05    1.1473e-05    7.4753e-06       10     
end_path:out1:RootOutput(write)                         0.15548      0.226955      0.284198      0.243295      0.0405117       10     
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4069.88 MB
  Peak resident set size usage (VmHWM): 2358.82 MB
====================================================================================================
Art has completed and will exit with status 0.
Moving on to metadata extractor 
============OUTPUT FILE:  prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250205T044331Z_gen_000255_supernova_g4stage1_g4stage2_detsim_reco_20250625T204806Z_reco2.root
Extracted metadata
checking file====== fardet-vd:prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250205T044331Z_gen_000255_supernova_g4stage1_g4stage2_detsim_reco.root
MD conversion to MetaCat OK
all-input-dids.txt
debugprod.log
jobscript.log
justin-processed-pfns.txt
old_md.json
prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250205T044331Z_gen_000255_supernova_g4stage1_g4stage2_detsim_reco_20250625T204806Z_reco2.root
prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250205T044331Z_gen_000255_supernova_g4stage1_g4stage2_detsim_reco_20250625T204806Z_reco2.root.ext.json
prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250205T044331Z_gen_000255_supernova_g4stage1_g4stage2_detsim_reco_20250625T204806Z_reco2.root.json
reco2_hist.root
justIN time: 2025-08-14 23:47:13 UTC       justIN version: 01.03.02