justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

21 July 2025: This instance at RAL is read-only. Please do not try submitting new workflows for now.

Jobsub ID 424562.1@justin-prod-sched01.dune.hep.ac.uk

Jobsub ID424562.1@justin-prod-sched01.dune.hep.ac.uk
Workflow ID7958
Stage ID1
User nameepennacc@fnal.gov
HTCondor Groupgroup_dune.prod_mcsim
RequestedProcessors1
GPUNo
RSS bytes4194304000 (4000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2025-06-29 10:26:16
SiteUS_Colorado
EntryCMSHTPC_T3_US_Colorado_heposg01-colorado
Last heartbeat2025-06-29 19:27:30
From worker nodeHostnamelnxfarm226.colorado.edu
cpuinfoIntel(R) Core(TM) i7-2600K CPU @ 3.40GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit171000 (47 hours)
GPU
Inner Apptainer?True
Job statefinished
Allocator namejustin-allocator-pro.dune.hep.ac.uk
Started2025-06-29 10:27:50
Input filesfardet-vd:prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250203T030708Z_gen_004643_supernova_g4stage1_g4stage2_detsim_reco.root
JobscriptExit code0
Real time8h (32126s)
CPU time8h (31517s = 98%)
Max RSS bytes2571964416 (2452 MiB)
Outputting started2025-06-29 19:23:18
Output filesfardet-vd:prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250203T030708Z_gen_004643_supernova_g4stage1_g4stage2_detsim_reco_20250629T102753Z_reco2.root
Finished2025-06-29 19:27:30
Saved logsjustin-logs:424562.1-justin-prod-sched01.dune.hep.ac.uk.logs.tgz
List job events     Wrapper job log

Jobscript log (last 10,000 characters)

same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  29-Jun-2025 13:19:37 MDT run: 5027 subRun: 0 event: 46430
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  29-Jun-2025 13:19:37 MDT run: 5027 subRun: 0 event: 46430
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  29-Jun-2025 13:19:37 MDT run: 5027 subRun: 0 event: 46430
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  29-Jun-2025 13:19:37 MDT run: 5027 subRun: 0 event: 46430
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  29-Jun-2025 13:19:37 MDT run: 5027 subRun: 0 event: 46430
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  29-Jun-2025 13:19:37 MDT run: 5027 subRun: 0 event: 46430
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  29-Jun-2025 13:19:37 MDT run: 5027 subRun: 0 event: 46430
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  29-Jun-2025 13:19:37 MDT run: 5027 subRun: 0 event: 46430
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  29-Jun-2025 13:19:37 MDT run: 5027 subRun: 0 event: 46430
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  29-Jun-2025 13:19:37 MDT run: 5027 subRun: 0 event: 46430
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  29-Jun-2025 13:19:37 MDT run: 5027 subRun: 0 event: 46430
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  29-Jun-2025 13:19:37 MDT run: 5027 subRun: 0 event: 46430
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  29-Jun-2025 13:19:37 MDT run: 5027 subRun: 0 event: 46430
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  29-Jun-2025 13:19:37 MDT run: 5027 subRun: 0 event: 46430
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  29-Jun-2025 13:19:37 MDT run: 5027 subRun: 0 event: 46430
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  29-Jun-2025 13:19:37 MDT run: 5027 subRun: 0 event: 46430
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  29-Jun-2025 13:19:37 MDT run: 5027 subRun: 0 event: 46430
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  29-Jun-2025 13:19:37 MDT run: 5027 subRun: 0 event: 46430
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  29-Jun-2025 13:19:37 MDT run: 5027 subRun: 0 event: 46430
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  29-Jun-2025 13:19:37 MDT run: 5027 subRun: 0 event: 46430
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  29-Jun-2025 13:19:37 MDT run: 5027 subRun: 0 event: 46430
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  29-Jun-2025 13:19:37 MDT run: 5027 subRun: 0 event: 46430
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  29-Jun-2025 13:19:37 MDT run: 5027 subRun: 0 event: 46430
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  29-Jun-2025 13:19:37 MDT run: 5027 subRun: 0 event: 46430
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  29-Jun-2025 13:19:37 MDT run: 5027 subRun: 0 event: 46430
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  29-Jun-2025 13:19:37 MDT run: 5027 subRun: 0 event: 46430
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  29-Jun-2025 13:19:37 MDT run: 5027 subRun: 0 event: 46430
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  29-Jun-2025 13:19:37 MDT run: 5027 subRun: 0 event: 46430
Comparing two wires in the same plane: return failure
%MSG
29-Jun-2025 13:23:14 MDT  Closed output file "prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250203T030708Z_gen_004643_supernova_g4stage1_g4stage2_detsim_reco_20250629T102753Z_reco2.root"
29-Jun-2025 13:23:14 MDT  Closed input file "root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-vd/2b/11/prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250203T030708Z_gen_004643_supernova_g4stage1_g4stage2_detsim_reco.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              1861.91       3174.96       3516.19       3480.67       583.298        10     
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.00200795     0.0199626     0.0541886     0.0160154     0.0145101       10     
reco:pandora:StandardPandora                            1697.96       2916.7        3259.58       3219.18       573.342        10     
reco:pandoraTrack:LArPandoraTrackCreation             0.000157323   0.00490346     0.0369971    0.000365802    0.0109209       10     
reco:pandoraShower:LArPandoraModularShowerCreation    0.000165113   0.00186669    0.00682244    0.00073348    0.00226355       10     
reco:pandoracalo:Calorimetry                          0.000205627   0.00252706    0.00639709    0.00158871    0.00227919       10     
reco:pandorapid:Chi2ParticleID                        4.2123e-05    0.000145924   0.000672518   7.80965e-05   0.00018474       10     
reco:linecluster:LineCluster                            0.40323      0.487962      0.600328      0.488219      0.0589621       10     
reco:trajcluster:TrajCluster                           0.286868      0.364895      0.448924      0.369421      0.0426331       10     
reco:pmtracktc:PMAlgTrackMaker                          4.93015       7.46605       8.94798       7.67826       1.16299        10     
reco:emtrkmichelid:EmTrackMichelId                      156.144       248.756       317.081       255.261       39.647         10     
reco:solarflash:SolarOpFlash                           0.770602      0.915455       1.02203       0.91365      0.0689856       10     
[art]:TriggerResults:TriggerResultInserter             1.772e-05    3.13183e-05   3.8997e-05    3.3381e-05    7.00771e-06      10     
end_path:out1:RootOutput                               3.231e-06     7.085e-06    1.0515e-05     7.252e-06    2.0693e-06       10     
end_path:out1:RootOutput(write)                        0.209386      0.238887      0.255211      0.246144      0.0150292       10     
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4047.37 MB
  Peak resident set size usage (VmHWM): 2571.96 MB
====================================================================================================
Art has completed and will exit with status 0.
Moving on to metadata extractor 
============OUTPUT FILE:  prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250203T030708Z_gen_004643_supernova_g4stage1_g4stage2_detsim_reco_20250629T102753Z_reco2.root
Extracted metadata
checking file====== fardet-vd:prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250203T030708Z_gen_004643_supernova_g4stage1_g4stage2_detsim_reco.root
MD conversion to MetaCat OK
all-input-dids.txt
debugprod.log
jobscript.log
justin-processed-pfns.txt
old_md.json
prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250203T030708Z_gen_004643_supernova_g4stage1_g4stage2_detsim_reco_20250629T102753Z_reco2.root
prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250203T030708Z_gen_004643_supernova_g4stage1_g4stage2_detsim_reco_20250629T102753Z_reco2.root.ext.json
prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250203T030708Z_gen_004643_supernova_g4stage1_g4stage2_detsim_reco_20250629T102753Z_reco2.root.json
reco2_hist.root
justIN time: 2025-08-15 04:43:36 UTC       justIN version: 01.03.02