justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 108807.18@justin-prod-sched02.dune.hep.ac.uk

Jobsub ID108807.18@justin-prod-sched02.dune.hep.ac.uk
Workflow ID4213
Stage ID1
User nameismerio@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2024-11-23 05:23:22
SiteCERN
EntryCMSHTPC_T2_CH_CERN_ce511
Last heartbeat2024-11-23 06:14:50
From worker nodeHostnameb9p08p7686.cern.ch
cpuinfoAMD EPYC 7543 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit171000 (47 hours)
Inner Apptainer?True
Job stateoutputting_failed
Allocator namejustin-allocator-pro.dune.hep.ac.uk
Started2024-11-23 05:24:44
Input filesfardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50601160_989_20231205T045230Z_gen_g4_detsim_hitreco__20240510T021547Z_reco2.root
JobscriptExit code0
Real time49m (2989s)
CPU time51m (3111s = 104%)
Outputting started2024-11-23 06:14:33
Output files
Finished2024-11-23 06:14:50
List job events     Wrapper job log

Jobscript log (last 10,000 characters)

imaryId 1, Nu 1, CR 0, MCPDG 2212, Energy 0.963555, Dist. 0.696985, nMCHits 20 (12, 6, 2)
-MatchedPfoId 1, Nu 1 [NuId: 1], CR 0, PDG 13, nMatchedHits 6 (3, 3, 0), nPfoHits 6 (3, 3, 0)

---SUMMARY--------------------------------------------------------------------------------------
#CorrectNu: 1/1, Fraction: 1
------------------------------------------------------------------------------------------------

Found: 2212 with KE=0.0252833 GeV, E=0.963555 GeV, p2=0.0480846
%MSG-e ShowerProducedPtrsHolder:   LArPandoraModularShowerCreation:pandoraShower@BeginModule  23-Nov-2024 07:14:31 CET run: 50601160 subRun: 1 event: 99000
Trying to add data product: InitialTrack. This element does not exist in the element holder
%MSG
%MSG-e ShowerProducedPtrsHolder:   LArPandoraModularShowerCreation:pandoraShower@BeginModule  23-Nov-2024 07:14:31 CET run: 50601160 subRun: 1 event: 99000
Trying to add data product: ShowerPCA. This element does not exist in the element holder
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrajFitter:pmtrajfit@BeginModule  23-Nov-2024 07:14:31 CET run: 50601160 subRun: 1 event: 99000
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrajFitter:pmtrajfit@BeginModule  23-Nov-2024 07:14:31 CET run: 50601160 subRun: 1 event: 99000
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrajFitter:pmtrajfittc@BeginModule  23-Nov-2024 07:14:31 CET run: 50601160 subRun: 1 event: 99000
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrajFitter:pmtrajfittc@BeginModule  23-Nov-2024 07:14:31 CET run: 50601160 subRun: 1 event: 99000
Comparing two wires in the same plane: return failure
%MSG
126
***Wiener Filter*******ApplyPostfilter***
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 315!
23-Nov-2024 07:14:32 CET  Closed output file "atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50601160_989_20231205T045230Z_gen_g4_detsim_hitreco__20240510T021547Z_reco2_reco_data_2024-11-23T_052449Z.root"
23-Nov-2024 07:14:32 CET  Closed input file "root://golias100.farm.particle.cz:1094/dpm/farm.particle.cz/home/dune/RSE/fardet-hd/18/c4/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50601160_989_20231205T045230Z_gen_g4_detsim_hitreco__20240510T021547Z_reco2.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              1.78565       27.8337       379.412       8.7761        59.5458        100    
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                 0.016756      0.0453689     0.177266      0.0439089     0.0226586       100    
reco:linecluster:LineCluster                           0.147981      0.413327       6.61262      0.177873      0.938936        100    
reco:trajcluster:TrajCluster                           0.0475317     0.922871       17.3306      0.0624969      3.03297        100    
reco:pandora:StandardPandora                            0.79306       2.2796        39.0942       1.12427       5.13553        100    
reco:pandoraTrack:LArPandoraTrackCreation             0.000119321   0.00702389     0.145627     0.00143362     0.0193372       100    
reco:pandoraShower:LArPandoraModularShowerCreation    0.000180881    0.0136142      0.34854     0.00377225     0.0396833       100    
reco:pandoracalo:Calorimetry                          9.7141e-05     0.0256365     0.763018     0.00101243     0.0957508       100    
reco:pandorapid:Chi2ParticleID                         3.382e-05    0.000149484   0.000976085   0.000121076   0.00012215       100    
reco:trkshowersplit:TrackShowerHits                   9.0011e-05     0.0294896     0.553984     0.000796379    0.0973342       100    
reco:pmtrack:PMAlgTrackMaker                          0.000933605     2.79264       58.1803      0.227481       8.29428        100    
reco:pmtrackcalo:Calorimetry                          5.6881e-05     0.0129932     0.314755     0.00093585     0.044858        100    
reco:pmtrackpid:Chi2ParticleID                         2.315e-05    0.000289584   0.00303746    0.000140671   0.000479965      100    
reco:pmtrajfit:PMAlgTrajFitter                        0.000151051     2.45278       57.5211      0.269157       7.31976        100    
reco:pmtrajfitcalo:Calorimetry                         5.513e-05     0.0276437     0.674912     0.00142751     0.100528        100    
reco:pmtrajfitpid:Chi2ParticleID                      2.3361e-05    0.00017399    0.00102994    0.000135161   0.000157344      100    
reco:pmtracktc:PMAlgTrackMaker                        0.000746305     3.27694       51.6689      0.262245       9.4128         100    
reco:pmtracktccalo:Calorimetry                         5.63e-05     0.00842372      0.14802     0.00096519     0.0255763       100    
reco:pmtracktcpid:Chi2ParticleID                       2.193e-05    0.000257119   0.00197852    0.000135871   0.000335354      100    
reco:pmtrajfittc:PMAlgTrajFitter                      0.00013961      1.76791       30.3043      0.315977       4.54484        100    
reco:pmtrajfittccalo:Calorimetry                      5.3821e-05     0.0187994     0.439215      0.0010894     0.0649995       100    
reco:pmtrajfittcpid:Chi2ParticleID                     2.435e-05    0.000346898   0.00335571    0.000147911   0.000575882      100    
reco:blurredcluster:BlurredClustering                 0.00549055     0.906961       7.78207       0.39654       1.28393        100    
reco:emtrkmichelid:EmTrackMichelId                     0.159582       9.88202       107.698       2.41075       19.945         100    
reco:emshower:EMShower                                0.000226141    0.0258041     0.306057     0.00490532     0.0524098       100    
reco:cvnmap:CVNMapper                                  2.082e-05     0.0104751     0.0300816     0.0113221    0.00797439       100    
reco:cvneva:CVNEvaluator                               1.408e-05     0.642524       1.39084      0.766246      0.367976        100    
reco:energyrecnumu:EnergyReco                         0.000242172    0.0236504     0.631571     0.00521894      0.0699         100    
reco:energyrecnue:EnergyReco                          0.00011602    0.00254932     0.0743756    0.000466332   0.00881932       100    
reco:energyrecnc:EnergyReco                           0.00169475    0.00428894     0.051141     0.00279076    0.00650509       100    
reco:energyrecnumurange:EnergyReco                    0.000119341    0.0210709     0.633088     0.00289258     0.0705378       100    
reco:energyrecnumumcs:EnergyReco                      0.000619223    0.0222632     0.631854     0.00349087     0.0695078       100    
reco:opdec:Deconvolution                               0.074236      0.288399       0.94776      0.272864      0.117842        100    
reco:ophitspe:OpHitFinderDeco                          0.289847       1.67612       3.36142       1.62578      0.684623        100    
reco:opflash:OpFlashFinder                            0.000247211   0.00135726    0.00381948    0.00125161    0.000620538      100    
reco:rns:RandomNumberSaver                             1.749e-05    3.61111e-05   0.000245761   3.3755e-05    2.25676e-05      100    
reco:anglereconue:NuAngularReco                       0.000153571    0.0016982     0.0409506    0.000393367   0.00504351       100    
reco:anglereconumu:NuAngularReco                       6.752e-05    0.00149271     0.0399011    0.000214761   0.00489566       100    
reco:anglereconuepfps:NuAngularReco                   0.00011057     0.0016037     0.0399149    0.000330811   0.00489423       100    
reco:anglereconumupfps:NuAngularReco                  8.7511e-05    0.00154141     0.0397952    0.000283007   0.00489563       100    
[art]:TriggerResults:TriggerResultInserter             7.55e-06     1.18208e-05    3.255e-05     1.081e-05    3.85316e-06      100    
end_path:out1:RootOutput                               2.47e-06     6.49243e-06   2.4271e-05     6.33e-06     2.47052e-06      100    
end_path:out1:RootOutput(write)                        0.0164703     0.221939       1.02564      0.202623      0.157886        100    
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 21888.3 MB
  Peak resident set size usage (VmHWM): 2465.62 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
Validation_ccnc_atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50601160_989_20231205T045230Z_gen_g4_detsim_hitreco__20240510T021547Z_reco2.root
all-input-dids.txt
atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50601160_989_20231205T045230Z_gen_g4_detsim_hitreco__20240510T021547Z_reco2_reco_2024-11-23T_052449Z.log
atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50601160_989_20231205T045230Z_gen_g4_detsim_hitreco__20240510T021547Z_reco2_reco_data_2024-11-23T_052449Z.root
debugprod.log
jobscript.log
justin-processed-pfns.txt
reco2_hist.root
MyPandoraSettings_Master_Atmos_DUNEFD.xml
MyPandoraSettings_Master_DUNEFD.xml
build_slf7.x86_64
localProducts_larsoft_v09_91_02_e26_prof
setup_env-testreco.sh
srcs
temp.txt
temp2.txt
test
work
justIN time: 2024-11-23 14:35:57 UTC       justIN version: 01.01.09