21 July 2025: This instance at RAL is read-only. Please do not try submitting new workflows for now.
Jobsub ID 425578.122@justin-prod-sched01.dune.hep.ac.uk
| Jobsub ID | 425578.122@justin-prod-sched01.dune.hep.ac.uk | |
| Workflow ID | 8047 | |
| Stage ID | 1 | |
| User name | epennacc@fnal.gov | |
| HTCondor Group | group_dune.prod_mcsim | |
| Requested | Processors | 1 |
| GPU | No | |
| RSS bytes | 4194304000 (4000 MiB) | |
| Wall seconds limit | 80000 (22 hours) | |
| Submitted time | 2025-07-01 12:28:56 | |
| Site | US_FNAL-FermiGrid | |
| Entry | FNAL_GPGrid_ce03_mcore_op_duneonly | |
| Last heartbeat | 2025-07-01 15:08:41 | |
| From worker node | Hostname | dunegli-5845335-0-fnpc23006.fnal.gov |
| cpuinfo | AMD EPYC 7543 32-Core Processor | |
| OS release | Scientific Linux release 7.9 (Nitrogen) | |
| Processors | 1 | |
| RSS bytes | 4194304000 (4000 MiB) | |
| Wall seconds limit | 172800 (48 hours) | |
| GPU | ||
| Inner Apptainer? | True | |
| Job state | outputting_failed | |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk | |
| Started | 2025-07-01 12:31:42 | |
| Input files | fardet-vd:prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250128T080212Z_gen_002144_supernova_g4stage1_g4stage2_detsim_reco.root | |
| Jobscript | Exit code | 0 |
| Real time | 2h (9359s) | |
| CPU time | 2h (9221s = 98%) | |
| Max RSS bytes | 2583838720 (2464 MiB) | |
| Outputting started | 2025-07-01 15:07:51 | |
| Output files | ||
| Finished | 2025-07-01 15:08:41 | |
| List job events Wrapper job log | ||
Jobscript log (last 10,000 characters)
same plane: return failure %MSG %MSG-e WireIDIntersectionCheck: PMAlgTrackMaker:pmtracktc@BeginModule 01-Jul-2025 15:06:05 UTC run: 4963 subRun: 0 event: 21440 Comparing two wires in the same plane: return failure %MSG %MSG-e WireIDIntersectionCheck: PMAlgTrackMaker:pmtracktc@BeginModule 01-Jul-2025 15:06:05 UTC run: 4963 subRun: 0 event: 21440 Comparing two wires in the same plane: return failure %MSG %MSG-e WireIDIntersectionCheck: PMAlgTrackMaker:pmtracktc@BeginModule 01-Jul-2025 15:06:05 UTC run: 4963 subRun: 0 event: 21440 Comparing two wires in the same plane: return failure %MSG %MSG-e WireIDIntersectionCheck: PMAlgTrackMaker:pmtracktc@BeginModule 01-Jul-2025 15:06:05 UTC run: 4963 subRun: 0 event: 21440 Comparing two wires in the same plane: return failure %MSG %MSG-e WireIDIntersectionCheck: PMAlgTrackMaker:pmtracktc@BeginModule 01-Jul-2025 15:06:05 UTC run: 4963 subRun: 0 event: 21440 Comparing two wires in the same plane: return failure %MSG %MSG-e WireIDIntersectionCheck: PMAlgTrackMaker:pmtracktc@BeginModule 01-Jul-2025 15:06:05 UTC run: 4963 subRun: 0 event: 21440 Comparing two wires in the same plane: return failure %MSG %MSG-e WireIDIntersectionCheck: PMAlgTrackMaker:pmtracktc@BeginModule 01-Jul-2025 15:06:05 UTC run: 4963 subRun: 0 event: 21440 Comparing two wires in the same plane: return failure %MSG %MSG-e WireIDIntersectionCheck: PMAlgTrackMaker:pmtracktc@BeginModule 01-Jul-2025 15:06:05 UTC run: 4963 subRun: 0 event: 21440 Comparing two wires in the same plane: return failure %MSG %MSG-e WireIDIntersectionCheck: PMAlgTrackMaker:pmtracktc@BeginModule 01-Jul-2025 15:06:05 UTC run: 4963 subRun: 0 event: 21440 Comparing two wires in the same plane: return failure %MSG %MSG-e WireIDIntersectionCheck: PMAlgTrackMaker:pmtracktc@BeginModule 01-Jul-2025 15:06:05 UTC run: 4963 subRun: 0 event: 21440 Comparing two wires in the same plane: return failure %MSG %MSG-e WireIDIntersectionCheck: PMAlgTrackMaker:pmtracktc@BeginModule 01-Jul-2025 15:06:05 UTC run: 4963 subRun: 0 event: 21440 Comparing two wires in the same plane: return failure %MSG %MSG-e WireIDIntersectionCheck: PMAlgTrackMaker:pmtracktc@BeginModule 01-Jul-2025 15:06:05 UTC run: 4963 subRun: 0 event: 21440 Comparing two wires in the same plane: return failure %MSG %MSG-e WireIDIntersectionCheck: PMAlgTrackMaker:pmtracktc@BeginModule 01-Jul-2025 15:06:05 UTC run: 4963 subRun: 0 event: 21440 Comparing two wires in the same plane: return failure %MSG %MSG-e WireIDIntersectionCheck: PMAlgTrackMaker:pmtracktc@BeginModule 01-Jul-2025 15:06:05 UTC run: 4963 subRun: 0 event: 21440 Comparing two wires in the same plane: return failure %MSG %MSG-e WireIDIntersectionCheck: PMAlgTrackMaker:pmtracktc@BeginModule 01-Jul-2025 15:06:05 UTC run: 4963 subRun: 0 event: 21440 Comparing two wires in the same plane: return failure %MSG %MSG-e WireIDIntersectionCheck: PMAlgTrackMaker:pmtracktc@BeginModule 01-Jul-2025 15:06:05 UTC run: 4963 subRun: 0 event: 21440 Comparing two wires in the same plane: return failure %MSG %MSG-e WireIDIntersectionCheck: PMAlgTrackMaker:pmtracktc@BeginModule 01-Jul-2025 15:06:05 UTC run: 4963 subRun: 0 event: 21440 Comparing two wires in the same plane: return failure %MSG %MSG-e WireIDIntersectionCheck: PMAlgTrackMaker:pmtracktc@BeginModule 01-Jul-2025 15:06:06 UTC run: 4963 subRun: 0 event: 21440 Comparing two wires in the same plane: return failure %MSG %MSG-e WireIDIntersectionCheck: PMAlgTrackMaker:pmtracktc@BeginModule 01-Jul-2025 15:06:06 UTC run: 4963 subRun: 0 event: 21440 Comparing two wires in the same plane: return failure %MSG %MSG-e WireIDIntersectionCheck: PMAlgTrackMaker:pmtracktc@BeginModule 01-Jul-2025 15:06:06 UTC run: 4963 subRun: 0 event: 21440 Comparing two wires in the same plane: return failure %MSG %MSG-e WireIDIntersectionCheck: PMAlgTrackMaker:pmtracktc@BeginModule 01-Jul-2025 15:06:06 UTC run: 4963 subRun: 0 event: 21440 Comparing two wires in the same plane: return failure %MSG %MSG-e WireIDIntersectionCheck: PMAlgTrackMaker:pmtracktc@BeginModule 01-Jul-2025 15:06:06 UTC run: 4963 subRun: 0 event: 21440 Comparing two wires in the same plane: return failure %MSG %MSG-e WireIDIntersectionCheck: PMAlgTrackMaker:pmtracktc@BeginModule 01-Jul-2025 15:06:06 UTC run: 4963 subRun: 0 event: 21440 Comparing two wires in the same plane: return failure %MSG %MSG-e WireIDIntersectionCheck: PMAlgTrackMaker:pmtracktc@BeginModule 01-Jul-2025 15:06:06 UTC run: 4963 subRun: 0 event: 21440 Comparing two wires in the same plane: return failure %MSG %MSG-e WireIDIntersectionCheck: PMAlgTrackMaker:pmtracktc@BeginModule 01-Jul-2025 15:06:06 UTC run: 4963 subRun: 0 event: 21440 Comparing two wires in the same plane: return failure %MSG %MSG-e WireIDIntersectionCheck: PMAlgTrackMaker:pmtracktc@BeginModule 01-Jul-2025 15:06:06 UTC run: 4963 subRun: 0 event: 21440 Comparing two wires in the same plane: return failure %MSG %MSG-e WireIDIntersectionCheck: PMAlgTrackMaker:pmtracktc@BeginModule 01-Jul-2025 15:06:06 UTC run: 4963 subRun: 0 event: 21440 Comparing two wires in the same plane: return failure %MSG %MSG-e WireIDIntersectionCheck: PMAlgTrackMaker:pmtracktc@BeginModule 01-Jul-2025 15:06:06 UTC run: 4963 subRun: 0 event: 21440 Comparing two wires in the same plane: return failure %MSG 01-Jul-2025 15:07:40 UTC Closed output file "prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250128T080212Z_gen_002144_supernova_g4stage1_g4stage2_detsim_reco_20250701T123145Z_reco2.root" 01-Jul-2025 15:07:40 UTC Closed input file "root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-vd/8e/4a/prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250128T080212Z_gen_002144_supernova_g4stage1_g4stage2_detsim_reco.root" ======================================================================================================================================== TimeTracker printout (sec) Min Avg Max Median RMS nEvts ======================================================================================================================================== Full event 791.952 929.169 1178.58 893.397 108.369 10 ---------------------------------------------------------------------------------------------------------------------------------------- source:RootInput(read) 0.00404207 0.0124195 0.0312082 0.00917481 0.00860507 10 reco:pandora:StandardPandora 649.126 810.107 1055.46 778.047 110.86 10 reco:pandoraTrack:LArPandoraTrackCreation 0.000253956 0.0080554 0.0452441 0.00239897 0.0129557 10 reco:pandoraShower:LArPandoraModularShowerCreation 0.000226584 0.00218449 0.00837501 0.000842803 0.00247016 10 reco:pandoracalo:Calorimetry 0.000153744 0.000615214 0.00268832 0.000296812 0.000754943 10 reco:pandorapid:Chi2ParticleID 4.3544e-05 0.000206162 0.000889212 8.1862e-05 0.000270026 10 reco:linecluster:LineCluster 0.113739 0.171487 0.5352 0.125011 0.122559 10 reco:trajcluster:TrajCluster 0.180913 0.202702 0.241235 0.196367 0.0196084 10 reco:pmtracktc:PMAlgTrackMaker 3.3783 3.8241 4.33489 3.79216 0.255524 10 reco:emtrkmichelid:EmTrackMichelId 92.284 114.368 138.512 113.445 12.6373 10 reco:solarflash:SolarOpFlash 0.288576 0.324435 0.539662 0.298848 0.0723964 10 [art]:TriggerResults:TriggerResultInserter 2.2283e-05 3.66722e-05 8.7167e-05 3.14555e-05 1.82382e-05 10 end_path:out1:RootOutput 5.27e-06 1.03168e-05 3.8073e-05 7.329e-06 9.31572e-06 10 end_path:out1:RootOutput(write) 0.138189 0.146869 0.152962 0.148131 0.00491764 10 ======================================================================================================================================== ==================================================================================================== MemoryTracker summary (base-10 MB units used) Peak virtual memory usage (VmPeak) : 4042.02 MB Peak resident set size usage (VmHWM): 2583.84 MB ==================================================================================================== Art has completed and will exit with status 0. Moving on to metadata extractor ============OUTPUT FILE: prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250128T080212Z_gen_002144_supernova_g4stage1_g4stage2_detsim_reco_20250701T123145Z_reco2.root Extracted metadata checking file====== fardet-vd:prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250128T080212Z_gen_002144_supernova_g4stage1_g4stage2_detsim_reco.root MD conversion to MetaCat OK all-input-dids.txt debugprod.log jobscript.log justin-processed-pfns.txt old_md.json prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250128T080212Z_gen_002144_supernova_g4stage1_g4stage2_detsim_reco_20250701T123145Z_reco2.root prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250128T080212Z_gen_002144_supernova_g4stage1_g4stage2_detsim_reco_20250701T123145Z_reco2.root.ext.json prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250128T080212Z_gen_002144_supernova_g4stage1_g4stage2_detsim_reco_20250701T123145Z_reco2.root.json reco2_hist.root