justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 280012.110@justin-prod-sched01.dune.hep.ac.uk

Jobsub ID280012.110@justin-prod-sched01.dune.hep.ac.uk
Workflow ID3836
Stage ID1
User nameismerio@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2024-10-25 15:19:05
SiteCERN
EntryCMSHTPC_T2_CH_CERN_ce510
Last heartbeat2024-10-26 09:35:13
From worker nodeHostnameb9p08p3749.cern.ch
cpuinfoAMD EPYC 7543 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit171000 (47 hours)
Inner Apptainer?True
Job statefinished
Allocator namejustin-allocator-pro.dune.hep.ac.uk
Started2024-10-25 15:47:44
Input filesfardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6430442_1069_20231205T105950Z_gen_g4_detsim_hitreco__20240509T222603Z_reco2.root
JobscriptExit code0
Real time17h (64028s)
CPU time17h (63115s = 98%)
Outputting started2024-10-26 09:34:52
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/ismerio/03836/1/Validation_ccnc_atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6430442_1069_20231205T105950Z_gen_g4_detsim_hitreco__20240509T222603Z_reco2.root
Finished2024-10-26 09:35:13
Saved logsjustin-logs:280012.110-justin-prod-sched01.dune.hep.ac.uk.logs.tgz
List job events     Wrapper job log

Jobscript log (last 10,000 characters)

g Ophitfinder with InputDigiType = 'recob'
Found hits: 2777!
Begin processing the 100th record. run: 6430442 subRun: 1 event: 107000 at 26-Oct-2024 11:34:46 CEST
> Running Algorithm: Alg0001, LArPreProcessing
> Running Algorithm: Alg0002, LArDLMaster
> Running Algorithm: Alg0003, LArNeutrinoEventValidation
---RAW-MATCHING-OUTPUT--------------------------------------------------------------------------
Failure in algorithm Alg0003, LArNeutrinoEventValidation, STATUS_CODE_NOT_INITIALIZED
%MSG-e ShowerProducedPtrsHolder:   LArPandoraModularShowerCreation:pandoraShower@BeginModule  26-Oct-2024 11:34:47 CEST run: 6430442 subRun: 1 event: 107000
Trying to add data product: InitialTrack. This element does not exist in the element holder
%MSG
%MSG-e ShowerProducedPtrsHolder:   LArPandoraModularShowerCreation:pandoraShower@BeginModule  26-Oct-2024 11:34:47 CEST run: 6430442 subRun: 1 event: 107000
Trying to add data product: ShowerPCA. This element does not exist in the element holder
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrajFitter:pmtrajfit@BeginModule  26-Oct-2024 11:34:47 CEST run: 6430442 subRun: 1 event: 107000
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  26-Oct-2024 11:34:47 CEST run: 6430442 subRun: 1 event: 107000
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  26-Oct-2024 11:34:47 CEST run: 6430442 subRun: 1 event: 107000
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrajFitter:pmtrajfittc@BeginModule  26-Oct-2024 11:34:47 CEST run: 6430442 subRun: 1 event: 107000
Comparing two wires in the same plane: return failure
%MSG
154
***Wiener Filter*******ApplyPostfilter***
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 305!
26-Oct-2024 11:34:49 CEST  Closed output file "atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6430442_1069_20231205T105950Z_gen_g4_detsim_hitreco__20240509T222603Z_reco2_reco_data_2024-10-25T_154750Z.root"
26-Oct-2024 11:34:49 CEST  Closed input file "root://golias100.farm.particle.cz:1094/dpm/farm.particle.cz/home/dune/RSE/fardet-hd/63/2d/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6430442_1069_20231205T105950Z_gen_g4_detsim_hitreco__20240509T222603Z_reco2.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                             0.580899       638.159       62336.1       7.04985       6200.91        100    
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                 0.0174501     0.054123      0.922597      0.0434944     0.0916317       100    
reco:linecluster:LineCluster                           0.0944743      0.56315       36.5419      0.172875       3.61781        100    
reco:trajcluster:TrajCluster                           0.0472526     0.497031       29.7818      0.0551693      2.99882        100    
reco:pandora:StandardPandora                           0.0551972      621.441       62029.8      0.941529       6171.77        100    
reco:pandoraTrack:LArPandoraTrackCreation             0.000108431   0.00338276     0.0437588    0.000901414   0.00622336       100    
reco:pandoraShower:LArPandoraModularShowerCreation    0.000152882    0.0105311     0.169821     0.00166603     0.0236392       100    
reco:pandoracalo:Calorimetry                          8.4262e-05     0.0102814     0.443694     0.000613894    0.0458255       100    
reco:pandorapid:Chi2ParticleID                        3.3231e-05    0.000139296   0.00162858    9.7462e-05    0.000210955      100    
reco:trkshowersplit:TrackShowerHits                   9.9992e-05     0.0130901     0.633084     0.000532073    0.0654938       100    
reco:pmtrack:PMAlgTrackMaker                          0.000917645     1.31587       39.0016      0.106803       4.40642        100    
reco:pmtrackcalo:Calorimetry                           5.827e-05    0.00387681     0.0707678    0.000509818   0.00948151       100    
reco:pmtrackpid:Chi2ParticleID                         2.244e-05    0.000169248   0.00121035    0.000103787   0.000204486      100    
reco:pmtrajfit:PMAlgTrajFitter                        0.000147902     1.19039       41.3674      0.128325       4.37656        100    
reco:pmtrajfitcalo:Calorimetry                        5.0401e-05     0.0106169     0.436853     0.000891744    0.0453964       100    
reco:pmtrajfitpid:Chi2ParticleID                      2.1211e-05    0.000139053   0.000951146   0.000115902   0.00012143       100    
reco:pmtracktc:PMAlgTrackMaker                        0.000735311     1.58481       39.4123      0.116733       4.83906        100    
reco:pmtracktccalo:Calorimetry                        5.7541e-05     0.0036307     0.046894     0.000638851   0.00786302       100    
reco:pmtracktcpid:Chi2ParticleID                       2.277e-05    0.000166059   0.000963986   0.000109376   0.000168277      100    
reco:pmtrajfittc:PMAlgTrajFitter                      0.000130362    0.859008       17.6366      0.182337       2.08645        100    
reco:pmtrajfittccalo:Calorimetry                       5.357e-05    0.00794637     0.341531     0.000814447    0.0355359       100    
reco:pmtrajfittcpid:Chi2ParticleID                    2.2631e-05    0.000230918   0.00397305    0.000129197   0.000438468      100    
reco:blurredcluster:BlurredClustering                 0.000125072    0.771941       4.19524      0.374079      0.918457        100    
reco:emtrkmichelid:EmTrackMichelId                    0.000104942     7.3746        87.3284       2.08227       13.1237        100    
reco:emshower:EMShower                                0.000151453    0.0164053     0.154093     0.00234962     0.0281091       100    
reco:cvnmap:CVNMapper                                 2.1151e-05    0.00872119      0.08639     0.00636928     0.0113955       100    
reco:cvneva:CVNEvaluator                               1.62e-05      0.640234       5.2674       0.704824      0.686846        100    
reco:energyrecnumu:EnergyReco                         0.000318675    0.0230279     0.367987     0.00317936     0.0609277       100    
reco:energyrecnue:EnergyReco                          0.000120722   0.00120558     0.0111762    0.000517944    0.0017394       100    
reco:energyrecnc:EnergyReco                           0.00150783    0.00301635     0.014018     0.00224964    0.00195233       100    
reco:energyrecnumurange:EnergyReco                    0.000128813    0.0215459     0.356639     0.00178416     0.0616662       100    
reco:energyrecnumumcs:EnergyReco                      0.000834984    0.0222181     0.359754     0.00299982     0.0601568       100    
reco:opdec:Deconvolution                               0.0570431     0.248335      0.457178      0.244287      0.0803063       100    
reco:ophitspe:OpHitFinderDeco                          0.193021       1.26986       2.60205       1.23061      0.500806        100    
reco:opflash:OpFlashFinder                            0.000264694   0.00119903    0.00468992    0.00107705    0.000635595      100    
reco:rns:RandomNumberSaver                             1.699e-05    4.21356e-05   0.000508398   2.73155e-05   6.08224e-05      100    
reco:anglereconue:NuAngularReco                       0.000116752   0.000811821   0.00708943    0.000359766   0.00115287       100    
reco:anglereconumu:NuAngularReco                      5.9981e-05    0.000565892    0.0058124    0.000160852   0.000969112      100    
reco:anglereconuepfps:NuAngularReco                   0.000102652   0.000705675   0.00583796    0.000310545   0.00100931       100    
reco:anglereconumupfps:NuAngularReco                  8.0821e-05    0.000617501   0.00581061    0.000215794   0.00100114       100    
[art]:TriggerResults:TriggerResultInserter             7.94e-06     1.32968e-05   0.000104862   1.12555e-05   1.03163e-05      100    
end_path:out1:RootOutput                               2.53e-06     6.96522e-06   0.000276875   3.7055e-06    2.7188e-05       100    
end_path:out1:RootOutput(write)                        0.0129737     0.182352      0.904918      0.174399      0.126099        100    
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 25779.8 MB
  Peak resident set size usage (VmHWM): 5343.35 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
Validation_ccnc_atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6430442_1069_20231205T105950Z_gen_g4_detsim_hitreco__20240509T222603Z_reco2.root
all-input-dids.txt
atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6430442_1069_20231205T105950Z_gen_g4_detsim_hitreco__20240509T222603Z_reco2_reco_2024-10-25T_154750Z.log
atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6430442_1069_20231205T105950Z_gen_g4_detsim_hitreco__20240509T222603Z_reco2_reco_data_2024-10-25T_154750Z.root
debugprod.log
jobscript.log
justin-processed-pfns.txt
reco2_hist.root
MyPandoraSettings_Master_Atmos_DUNEFD.xml
MyPandoraSettings_Master_DUNEFD.xml
build_slf7.x86_64
localProducts_larsoft_v09_91_02_e26_prof
setup_env-testreco.sh
srcs
work
justIN time: 2024-11-17 09:25:04 UTC       justIN version: 01.01.09