justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

21 July 2025: This instance at RAL is read-only. Please do not try submitting new workflows for now.

Jobsub ID 423447.0@justin-prod-sched01.dune.hep.ac.uk

Jobsub ID423447.0@justin-prod-sched01.dune.hep.ac.uk
Workflow ID7836
Stage ID1
User nameepennacc@fnal.gov
HTCondor Groupgroup_dune.prod_mcsim
RequestedProcessors1
GPUNo
RSS bytes6291456000 (6000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2025-06-27 06:51:50
SiteUS_Colorado
EntryCMSHTPC_T3_US_Colorado_heposg01-colorado
Last heartbeat2025-06-27 21:55:53
From worker nodeHostnamelnxfarm240.colorado.edu
cpuinfoIntel(R) Core(TM) i7-2600K CPU @ 3.40GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes6291456000 (6000 MiB)
Wall seconds limit171000 (47 hours)
GPU
Inner Apptainer?True
Job statefinished
Allocator namejustin-allocator-pro.dune.hep.ac.uk
Started2025-06-27 11:29:42
Input filesfardet-vd:prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250122T011709Z_gen_000894_supernova_g4stage1_g4stage2_detsim_reco.root
JobscriptExit code0
Real time10h (37185s)
CPU time9h (34555s = 92%)
Max RSS bytes2502520832 (2386 MiB)
Outputting started2025-06-27 21:49:28
Output filesfardet-vd:prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250122T011709Z_gen_000894_supernova_g4stage1_g4stage2_detsim_reco_20250627T112945Z_reco2.root
Finished2025-06-27 21:55:53
Saved logsjustin-logs:423447.0-justin-prod-sched01.dune.hep.ac.uk.logs.tgz
List job events     Wrapper job log

Jobscript log (last 10,000 characters)

0
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  27-Jun-2025 15:44:52 MDT run: 4789 subRun: 0 event: 8940
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  27-Jun-2025 15:44:52 MDT run: 4789 subRun: 0 event: 8940
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  27-Jun-2025 15:44:52 MDT run: 4789 subRun: 0 event: 8940
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  27-Jun-2025 15:44:52 MDT run: 4789 subRun: 0 event: 8940
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  27-Jun-2025 15:44:52 MDT run: 4789 subRun: 0 event: 8940
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  27-Jun-2025 15:44:52 MDT run: 4789 subRun: 0 event: 8940
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  27-Jun-2025 15:44:52 MDT run: 4789 subRun: 0 event: 8940
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  27-Jun-2025 15:44:52 MDT run: 4789 subRun: 0 event: 8940
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  27-Jun-2025 15:44:52 MDT run: 4789 subRun: 0 event: 8940
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  27-Jun-2025 15:44:52 MDT run: 4789 subRun: 0 event: 8940
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  27-Jun-2025 15:44:52 MDT run: 4789 subRun: 0 event: 8940
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  27-Jun-2025 15:44:52 MDT run: 4789 subRun: 0 event: 8940
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  27-Jun-2025 15:44:52 MDT run: 4789 subRun: 0 event: 8940
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  27-Jun-2025 15:44:52 MDT run: 4789 subRun: 0 event: 8940
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  27-Jun-2025 15:44:52 MDT run: 4789 subRun: 0 event: 8940
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  27-Jun-2025 15:44:52 MDT run: 4789 subRun: 0 event: 8940
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  27-Jun-2025 15:44:52 MDT run: 4789 subRun: 0 event: 8940
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  27-Jun-2025 15:44:52 MDT run: 4789 subRun: 0 event: 8940
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  27-Jun-2025 15:44:52 MDT run: 4789 subRun: 0 event: 8940
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  27-Jun-2025 15:44:52 MDT run: 4789 subRun: 0 event: 8940
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  27-Jun-2025 15:44:52 MDT run: 4789 subRun: 0 event: 8940
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  27-Jun-2025 15:44:52 MDT run: 4789 subRun: 0 event: 8940
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  27-Jun-2025 15:44:52 MDT run: 4789 subRun: 0 event: 8940
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  27-Jun-2025 15:44:52 MDT run: 4789 subRun: 0 event: 8940
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  27-Jun-2025 15:44:52 MDT run: 4789 subRun: 0 event: 8940
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  27-Jun-2025 15:44:52 MDT run: 4789 subRun: 0 event: 8940
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  27-Jun-2025 15:44:52 MDT run: 4789 subRun: 0 event: 8940
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  27-Jun-2025 15:44:52 MDT run: 4789 subRun: 0 event: 8940
Comparing two wires in the same plane: return failure
%MSG
27-Jun-2025 15:49:24 MDT  Closed output file "prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250122T011709Z_gen_000894_supernova_g4stage1_g4stage2_detsim_reco_20250627T112945Z_reco2.root"
27-Jun-2025 15:49:24 MDT  Closed input file "root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-vd/44/15/prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250122T011709Z_gen_000894_supernova_g4stage1_g4stage2_detsim_reco.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              3297.29       3480.52       3528.65       3498.23       62.8254        10     
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.00242977     0.0159396     0.0384305     0.0135575     0.0113039       10     
reco:pandora:StandardPandora                            3012.3        3206.54       3275.86       3219.5        68.4426        10     
reco:pandoraTrack:LArPandoraTrackCreation             0.000187053    0.0117307     0.0779561    0.00367291     0.0225574       10     
reco:pandoraShower:LArPandoraModularShowerCreation    0.000191501   0.00243769     0.0163525    0.000987906   0.00468121       10     
reco:pandoracalo:Calorimetry                          0.000363792   0.00348392    0.00815728    0.00329828    0.00287082       10     
reco:pandorapid:Chi2ParticleID                        4.7281e-05    0.000288484   0.00117104    9.62665e-05   0.000389552      10     
reco:linecluster:LineCluster                           0.421621      0.984802       1.86152      0.770495      0.515794        10     
reco:trajcluster:TrajCluster                           0.344342      0.404277      0.428996      0.413114      0.025987        10     
reco:pmtracktc:PMAlgTrackMaker                          6.95413       7.84653       8.92284       7.6319       0.659235        10     
reco:emtrkmichelid:EmTrackMichelId                      211.354       263.188        278.4        267.286       17.9827        10     
reco:solarflash:SolarOpFlash                           0.770535       1.22681       1.95145       1.04914      0.393285        10     
[art]:TriggerResults:TriggerResultInserter            1.8277e-05    3.12893e-05    7.426e-05    2.7287e-05    1.57471e-05      10     
end_path:out1:RootOutput                               2.641e-06    7.6431e-06    2.1374e-05     5.085e-06    6.35581e-06      10     
end_path:out1:RootOutput(write)                        0.215395      0.295424      0.917302      0.225948      0.207454        10     
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4040.58 MB
  Peak resident set size usage (VmHWM): 2502.52 MB
====================================================================================================
Art has completed and will exit with status 0.
Moving on to metadata extractor 
============OUTPUT FILE:  prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250122T011709Z_gen_000894_supernova_g4stage1_g4stage2_detsim_reco_20250627T112945Z_reco2.root
Extracted metadata
checking file====== fardet-vd:prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250122T011709Z_gen_000894_supernova_g4stage1_g4stage2_detsim_reco.root
MD conversion to MetaCat OK
all-input-dids.txt
debugprod.log
jobscript.log
justin-processed-pfns.txt
old_md.json
prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250122T011709Z_gen_000894_supernova_g4stage1_g4stage2_detsim_reco_20250627T112945Z_reco2.root
prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250122T011709Z_gen_000894_supernova_g4stage1_g4stage2_detsim_reco_20250627T112945Z_reco2.root.ext.json
prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250122T011709Z_gen_000894_supernova_g4stage1_g4stage2_detsim_reco_20250627T112945Z_reco2.root.json
reco2_hist.root
justIN time: 2025-08-14 21:50:21 UTC       justIN version: 01.03.02