justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 108850.9@justin-prod-sched02.dune.hep.ac.uk

Jobsub ID108850.9@justin-prod-sched02.dune.hep.ac.uk
Workflow ID4213
Stage ID1
User nameismerio@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2024-11-23 06:01:13
SiteUK_Lancaster
EntryUBoone_UK_Lancaster_HEC_grendel_ce02
Last heartbeat2024-11-23 07:37:30
From worker nodeHostnamecomp23-03
cpuinfoIntel(R) Xeon(R) Gold 6248 CPU @ 2.50GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit257400 (71 hours)
Inner Apptainer?True
Job stateoutputting_failed
Allocator namejustin-allocator-pro.dune.hep.ac.uk
Started2024-11-23 06:02:32
Input filesfardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74506302_411_20231202T180522Z_gen_g4_detsim_hitreco__20240508T063926Z_reco2.root
JobscriptExit code0
Real time1h (5681s)
CPU time1h (4934s = 86%)
Outputting started2024-11-23 07:37:14
Output files
Finished2024-11-23 07:37:30
List job events     Wrapper job log

Jobscript log (last 10,000 characters)

23-Nov-2024 07:36:59 GMT run: 74506302 subRun: 1 event: 41200
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrajFitter:pmtrajfittc@BeginModule  23-Nov-2024 07:36:59 GMT run: 74506302 subRun: 1 event: 41200
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  EMShower:emshower@BeginModule  23-Nov-2024 07:37:07 GMT run: 74506302 subRun: 1 event: 41200
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  EMShower:emshower@BeginModule  23-Nov-2024 07:37:07 GMT run: 74506302 subRun: 1 event: 41200
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  EMShower:emshower@BeginModule  23-Nov-2024 07:37:07 GMT run: 74506302 subRun: 1 event: 41200
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  EMShower:emshower@BeginModule  23-Nov-2024 07:37:07 GMT run: 74506302 subRun: 1 event: 41200
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  EMShower:emshower@BeginModule  23-Nov-2024 07:37:07 GMT run: 74506302 subRun: 1 event: 41200
Comparing two wires in the same plane: return failure
%MSG
Boundary wire vector sizes: 220, 256, 231
minwire 0: 1562
minwire 1: 125
minwire 2: 1637
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 199
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 199
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 199
Classifier summary: 
Output 0: 0.0829756, 0.911529, 0.00549516, 
Output 1: 0.0132391, 0.132858, 0.787198, 0.066704, 
Output 2: 0.928597, 0.0714032, 

446
***Wiener Filter*******ApplyPostfilter***
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 1350!
23-Nov-2024 07:37:12 GMT  Closed output file "atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74506302_411_20231202T180522Z_gen_g4_detsim_hitreco__20240508T063926Z_reco2_reco_data_2024-11-23T_060236Z.root"
23-Nov-2024 07:37:12 GMT  Closed input file "root://golias100.farm.particle.cz:1094/dpm/farm.particle.cz/home/dune/RSE/fardet-hd/17/c7/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74506302_411_20231202T180522Z_gen_g4_detsim_hitreco__20240508T063926Z_reco2.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              1.09028       49.8443       1801.9        10.6231       204.017        100    
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                 0.0285966     0.0698471     0.445967      0.0612855     0.051734        100    
reco:linecluster:LineCluster                           0.162472      0.731032       22.0474       0.33558       2.35569        100    
reco:trajcluster:TrajCluster                           0.081932       3.33347       196.556      0.104103       20.8674        100    
reco:pandora:StandardPandora                           0.103534       13.3517       689.505       1.23763       82.0257        100    
reco:pandoraTrack:LArPandoraTrackCreation             0.000135527    0.0114992     0.518306     0.00167611     0.0543065       100    
reco:pandoraShower:LArPandoraModularShowerCreation    0.000170912   0.00769391     0.112997     0.00260265     0.0155402       100    
reco:pandoracalo:Calorimetry                          0.00010002     0.0709238      4.33362     0.00104758      0.46062        100    
reco:pandorapid:Chi2ParticleID                        3.6645e-05    0.00016849    0.00108368     0.0001224    0.000163452      100    
reco:trkshowersplit:TrackShowerHits                   0.000102635    0.033982       1.65903     0.000637118    0.178629        100    
reco:pmtrack:PMAlgTrackMaker                          0.00117351      2.90156       118.202      0.139079       13.4717        100    
reco:pmtrackcalo:Calorimetry                          8.6068e-05     0.0275695      1.51966     0.000996507    0.160902        100    
reco:pmtrackpid:Chi2ParticleID                        3.0413e-05    0.000332856   0.00513862    0.00015192    0.000673233      100    
reco:pmtrajfit:PMAlgTrajFitter                        0.000219436     3.48923       173.854       0.27923       17.9881        100    
reco:pmtrajfitcalo:Calorimetry                        6.6189e-05     0.0718599      4.32809     0.00141343     0.460537        100    
reco:pmtrajfitpid:Chi2ParticleID                      3.0072e-05    0.000223488   0.00203405    0.000148139   0.000277081      100    
reco:pmtracktc:PMAlgTrackMaker                         0.001257       3.91019       163.422      0.199481       18.4231        100    
reco:pmtracktccalo:Calorimetry                        7.9745e-05     0.0158242     0.825436     0.000883799    0.0862769       100    
reco:pmtracktcpid:Chi2ParticleID                       3.026e-05    0.000283074   0.00395792    0.000143239   0.000498348      100    
reco:pmtrajfittc:PMAlgTrajFitter                      0.000199985     1.88291       65.7394      0.281256       7.39584        100    
reco:pmtrajfittccalo:Calorimetry                      6.8764e-05     0.0344125      1.80151     0.00147324     0.192686        100    
reco:pmtrajfittcpid:Chi2ParticleID                    2.8858e-05    0.000377627   0.00539398    0.000168022   0.000733544      100    
reco:blurredcluster:BlurredClustering                 8.2739e-05      1.0512        12.9083      0.461137       1.70966        100    
reco:emtrkmichelid:EmTrackMichelId                    0.000132143     15.1147       331.617       3.44575       41.1351        100    
reco:emshower:EMShower                                0.000207013    0.0343036     0.732592      0.0123648     0.0845963       100    
reco:cvnmap:CVNMapper                                 2.3795e-05     0.0189542     0.0598653     0.0195439     0.0145879       100    
reco:cvneva:CVNEvaluator                              1.5493e-05     0.597275       1.64094      0.746449       0.3598         100    
reco:energyrecnumu:EnergyReco                         0.000254485    0.0476986      1.05934     0.00539885     0.175123        100    
reco:energyrecnue:EnergyReco                          0.000124603   0.00388281     0.184211     0.000524631    0.0192059       100    
reco:energyrecnc:EnergyReco                           0.00177091    0.00557904     0.186783     0.00215998     0.0192921       100    
reco:energyrecnumurange:EnergyReco                    0.000182207    0.0448798      1.06374     0.00198738     0.174781        100    
reco:energyrecnumumcs:EnergyReco                      0.00180053     0.0459946      1.04727     0.00253398       0.174         100    
reco:opdec:Deconvolution                               0.110823      0.441379       1.0245        0.43527      0.144511        100    
reco:ophitspe:OpHitFinderDeco                          0.419337       2.25615       4.36787       2.22038      0.744874        100    
reco:opflash:OpFlashFinder                            0.000265852    0.0014896     0.0050572    0.00139281    0.000727367      100    
reco:rns:RandomNumberSaver                            1.9205e-05    4.24385e-05   0.000259434   3.86795e-05   2.55465e-05      100    
reco:anglereconue:NuAngularReco                       0.000150868    0.0033606     0.170369     0.000413707    0.0175626       100    
reco:anglereconumu:NuAngularReco                      6.7412e-05    0.00310974     0.169256     0.000220349    0.0174427       100    
reco:anglereconuepfps:NuAngularReco                   0.000114108   0.00322987      0.16851     0.000344899    0.0173616       100    
reco:anglereconumupfps:NuAngularReco                  9.9591e-05    0.00314301     0.168606     0.000279797    0.0173747       100    
[art]:TriggerResults:TriggerResultInserter             8.262e-06    1.29256e-05   4.4728e-05    1.1822e-05    4.68588e-06      100    
end_path:out1:RootOutput                               2.461e-06    6.58736e-06   1.5744e-05     6.364e-06    2.38137e-06      100    
end_path:out1:RootOutput(write)                        0.0142281     0.220587       1.64172      0.205379      0.199635        100    
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 9990.16 MB
  Peak resident set size usage (VmHWM): 2826.88 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
Validation_ccnc_atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74506302_411_20231202T180522Z_gen_g4_detsim_hitreco__20240508T063926Z_reco2.root
all-input-dids.txt
atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74506302_411_20231202T180522Z_gen_g4_detsim_hitreco__20240508T063926Z_reco2_reco_2024-11-23T_060236Z.log
atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74506302_411_20231202T180522Z_gen_g4_detsim_hitreco__20240508T063926Z_reco2_reco_data_2024-11-23T_060236Z.root
debugprod.log
jobscript.log
justin-processed-pfns.txt
reco2_hist.root
MyPandoraSettings_Master_Atmos_DUNEFD.xml
MyPandoraSettings_Master_DUNEFD.xml
build_slf7.x86_64
localProducts_larsoft_v09_91_02_e26_prof
setup_env-testreco.sh
srcs
temp.txt
temp2.txt
test
work
justIN time: 2024-11-23 11:40:21 UTC       justIN version: 01.01.09