justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 108740.20@justin-prod-sched02.dune.hep.ac.uk

Jobsub ID108740.20@justin-prod-sched02.dune.hep.ac.uk
Workflow ID4213
Stage ID1
User nameismerio@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2024-11-23 04:25:44
SiteCERN
EntryCMSHTPC_T2_CH_CERN_ce506
Last heartbeat2024-11-23 05:30:42
From worker nodeHostnameb9p11p6612.cern.ch
cpuinfoAMD EPYC 7543 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit171000 (47 hours)
Inner Apptainer?True
Job stateoutputting_failed
Allocator namejustin-allocator-pro.dune.hep.ac.uk
Started2024-11-23 04:37:19
Input filesfardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50607041_50_20231207T033735Z_gen_g4_detsim_hitreco__20240510T055907Z_reco2.root
JobscriptExit code0
Real time53m (3185s)
CPU time54m (3255s = 102%)
Outputting started2024-11-23 05:30:24
Output files
Finished2024-11-23 05:30:42
List job events     Wrapper job log

Jobscript log (last 10,000 characters)

its 29 (17, 0, 12), nPfoHits 30 (18, 0, 12)

---SUMMARY--------------------------------------------------------------------------------------
#CorrectNu: 1/1, Fraction: 1
------------------------------------------------------------------------------------------------

Found: 2212 with KE=0.138525 GeV, E=1.0768 GeV, p2=0.279137
Found: 2212 with KE=0.106776 GeV, E=1.04505 GeV, p2=0.21177
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtrack@BeginModule  23-Nov-2024 06:30:16 CET run: 50607041 subRun: 1 event: 5100
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrajFitter:pmtrajfit@BeginModule  23-Nov-2024 06:30:16 CET run: 50607041 subRun: 1 event: 5100
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrajFitter:pmtrajfittc@BeginModule  23-Nov-2024 06:30:17 CET run: 50607041 subRun: 1 event: 5100
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrajFitter:pmtrajfittc@BeginModule  23-Nov-2024 06:30:17 CET run: 50607041 subRun: 1 event: 5100
Comparing two wires in the same plane: return failure
%MSG
Boundary wire vector sizes: 64, 31, 48
minwire 0: 1503
minwire 1: 1668
minwire 2: 1307
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 199
Used alternate method to get min and max wires due to vertex determination failure: 0, 199
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 199
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 199
Classifier summary: 
Output 0: 0.963328, 0.00462474, 0.032047, 
Output 1: 0.0315155, 0.305181, 0.651987, 0.0113158, 
Output 2: 0.987053, 0.012947, 

571
***Wiener Filter*******ApplyPostfilter***
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 1984!
23-Nov-2024 06:30:23 CET  Closed output file "atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50607041_50_20231207T033735Z_gen_g4_detsim_hitreco__20240510T055907Z_reco2_reco_data_2024-11-23T_043723Z.root"
23-Nov-2024 06:30:23 CET  Closed input file "root://golias100.farm.particle.cz:1094/dpm/farm.particle.cz/home/dune/RSE/fardet-hd/6b/ee/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50607041_50_20231207T033735Z_gen_g4_detsim_hitreco__20240510T055907Z_reco2.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              1.88266       29.8086       487.201       10.1197       69.4601        100    
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                 0.0169048     0.0442473     0.164316      0.0430888     0.0195019       100    
reco:linecluster:LineCluster                           0.149296      0.561058       26.5311      0.181225       2.73906        100    
reco:trajcluster:TrajCluster                           0.0476816      1.22499       51.8272      0.0649533      6.51067        100    
reco:pandora:StandardPandora                           0.867924       2.27104       36.9385       1.26556       5.07865        100    
reco:pandoraTrack:LArPandoraTrackCreation             0.000128481   0.00757307     0.184914     0.00180755     0.0215174       100    
reco:pandoraShower:LArPandoraModularShowerCreation    0.000192042    0.0241953     0.445646     0.00438336     0.0665501       100    
reco:pandoracalo:Calorimetry                          0.000107431    0.0294005      1.01599     0.00169119     0.122738        100    
reco:pandorapid:Chi2ParticleID                        3.7771e-05    0.000182307   0.00198292    0.000126291   0.000229835      100    
reco:trkshowersplit:TrackShowerHits                    9.481e-05     0.0356366      1.21728     0.00114665     0.166519        100    
reco:pmtrack:PMAlgTrackMaker                          0.00100158      2.86027       70.215       0.288139       9.74637        100    
reco:pmtrackcalo:Calorimetry                          5.8001e-05     0.0130408     0.421952     0.00109379     0.0493953       100    
reco:pmtrackpid:Chi2ParticleID                         2.298e-05    0.00029181    0.00386129    0.000130436   0.000501862      100    
reco:pmtrajfit:PMAlgTrajFitter                        0.000169182      2.72         65.831       0.342909       9.22415        100    
reco:pmtrajfitcalo:Calorimetry                        5.6551e-05     0.0259654     0.787679     0.00177373     0.103789        100    
reco:pmtrajfitpid:Chi2ParticleID                       2.331e-05    0.000187595   0.00136448    0.000142927   0.000175357      100    
reco:pmtracktc:PMAlgTrackMaker                        0.000795758     3.35549       85.2058      0.320709       11.5065        100    
reco:pmtracktccalo:Calorimetry                        6.2331e-05     0.0087326     0.192004     0.00118982     0.0247422       100    
reco:pmtracktcpid:Chi2ParticleID                       2.45e-05     0.000246914   0.00177149    0.000136767   0.000292621      100    
reco:pmtrajfittc:PMAlgTrajFitter                      0.000133061     1.57932       23.862       0.301455       3.76391        100    
reco:pmtrajfittccalo:Calorimetry                      5.6941e-05     0.0174477     0.516793     0.00151345     0.061103        100    
reco:pmtrajfittcpid:Chi2ParticleID                     2.402e-05    0.000374707   0.00437515    0.000168246   0.00062506       100    
reco:blurredcluster:BlurredClustering                 0.00820492      1.07038       7.42817      0.503103       1.47935        100    
reco:emtrkmichelid:EmTrackMichelId                     0.202155       10.4603       117.71        3.08393       19.745         100    
reco:emshower:EMShower                                0.000255962    0.0260317     0.731528      0.0052624      0.07763        100    
reco:cvnmap:CVNMapper                                  2.731e-05     0.0112239     0.0370831     0.0136637    0.00817105       100    
reco:cvneva:CVNEvaluator                               1.853e-05     0.683505       1.27565      0.828351      0.398193        100    
reco:energyrecnumu:EnergyReco                         0.000383384    0.0853253      4.24382     0.00369452     0.443572        100    
reco:energyrecnue:EnergyReco                          0.000154891   0.00231843     0.0596776    0.000642056   0.00638033       100    
reco:energyrecnc:EnergyReco                            0.0017206     0.0046962     0.0658769    0.00322984    0.00679763       100    
reco:energyrecnumurange:EnergyReco                    0.000212862    0.0781662      4.12599     0.00294058     0.425124        100    
reco:energyrecnumumcs:EnergyReco                      0.00136446     0.0691573      3.07253     0.00370637     0.326079        100    
reco:opdec:Deconvolution                               0.068507      0.279536      0.463608      0.284152      0.0842216       100    
reco:ophitspe:OpHitFinderDeco                          0.199297       1.99996       4.22028       1.94281      0.872784        100    
reco:opflash:OpFlashFinder                            0.000259893    0.0014653    0.00375938    0.00133688    0.000621858      100    
reco:rns:RandomNumberSaver                            1.8661e-05    3.96014e-05   0.000224202   3.6185e-05    2.09403e-05      100    
reco:anglereconue:NuAngularReco                       0.000147792   0.00177758     0.056397     0.000406139   0.00592772       100    
reco:anglereconumu:NuAngularReco                       6.943e-05    0.00153458     0.0552731    0.000255792   0.00582577       100    
reco:anglereconuepfps:NuAngularReco                   0.000114461   0.00163514     0.0557157    0.000369693   0.00577961       100    
reco:anglereconumupfps:NuAngularReco                   9.33e-05     0.00153995     0.056319     0.000315503   0.00580229       100    
[art]:TriggerResults:TriggerResultInserter             8.62e-06     1.34011e-05   3.3961e-05     1.297e-05    3.57845e-06      100    
end_path:out1:RootOutput                               2.55e-06     6.40256e-06    2.204e-05     6.285e-06    2.22692e-06      100    
end_path:out1:RootOutput(write)                        0.014965      0.248286       1.25537      0.233474      0.186617        100    
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 21673.8 MB
  Peak resident set size usage (VmHWM): 2252.34 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
Validation_ccnc_atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50607041_50_20231207T033735Z_gen_g4_detsim_hitreco__20240510T055907Z_reco2.root
all-input-dids.txt
atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50607041_50_20231207T033735Z_gen_g4_detsim_hitreco__20240510T055907Z_reco2_reco_2024-11-23T_043723Z.log
atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50607041_50_20231207T033735Z_gen_g4_detsim_hitreco__20240510T055907Z_reco2_reco_data_2024-11-23T_043723Z.root
debugprod.log
jobscript.log
justin-processed-pfns.txt
reco2_hist.root
MyPandoraSettings_Master_Atmos_DUNEFD.xml
MyPandoraSettings_Master_DUNEFD.xml
build_slf7.x86_64
localProducts_larsoft_v09_91_02_e26_prof
setup_env-testreco.sh
srcs
temp.txt
temp2.txt
test
work
justIN time: 2024-11-23 13:58:30 UTC       justIN version: 01.01.09