justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 301489.115@justin-prod-sched01.dune.hep.ac.uk

Jobsub ID301489.115@justin-prod-sched01.dune.hep.ac.uk
Workflow ID4213
Stage ID1
User nameismerio@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2024-11-22 23:05:17
SiteUK_RAL-Tier1
EntryLIGO_UK_RAL_arc_ce02
Last heartbeat2024-11-23 00:06:16
From worker nodeHostnamedune001-3353644.0-lcg2697.gridpp.rl.ac.uk
cpuinfoAMD EPYC 9654 96-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit216000 (60 hours)
Inner Apptainer?True
Job statefinished
Allocator namejustin-allocator-pro.dune.hep.ac.uk
Started2024-11-22 23:22:42
Input filesfardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6467376_666_20231207T073802Z_gen_g4_detsim_hitreco__20240510T055018Z_reco2.root
JobscriptExit code0
Real time36m (2162s)
CPU time35m (2148s = 99%)
Outputting started2024-11-22 23:58:45
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/ismerio/04213/1/Validation_ccnc_atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6467376_666_20231207T073802Z_gen_g4_detsim_hitreco__20240510T055018Z_reco2.root
Finished2024-11-23 00:06:16
Saved logsjustin-logs:301489.115-justin-prod-sched01.dune.hep.ac.uk.logs.tgz
List job events     Wrapper job log

Jobscript log (last 10,000 characters)

same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  22-Nov-2024 23:58:35 UTC run: 6467376 subRun: 1 event: 66700
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrajFitter:pmtrajfittc@BeginModule  22-Nov-2024 23:58:35 UTC run: 6467376 subRun: 1 event: 66700
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrajFitter:pmtrajfittc@BeginModule  22-Nov-2024 23:58:35 UTC run: 6467376 subRun: 1 event: 66700
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrajFitter:pmtrajfittc@BeginModule  22-Nov-2024 23:58:35 UTC run: 6467376 subRun: 1 event: 66700
Comparing two wires in the same plane: return failure
%MSG
%MSG-e pma::Track3D:  PMAlgTrajFitter:pmtrajfittc@BeginModule  22-Nov-2024 23:58:35 UTC run: 6467376 subRun: 1 event: 66700
0 enabled hits in AverageDist2 calculation.
%MSG
%MSG-e pma::Track3D:  PMAlgTrajFitter:pmtrajfittc@BeginModule  22-Nov-2024 23:58:35 UTC run: 6467376 subRun: 1 event: 66700
Track empty.
%MSG
%MSG-e pma::Track3D:  PMAlgTrajFitter:pmtrajfittc@BeginModule  22-Nov-2024 23:58:35 UTC run: 6467376 subRun: 1 event: 66700
TuneFullTree failed.
%MSG
Boundary wire vector sizes: 119, 120, 81
minwire 0: 1692
minwire 1: 477
minwire 2: 1582
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 199
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 199
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 199
Classifier summary: 
Output 0: 0.438943, 0.0575287, 0.503528, 
Output 1: 0.124049, 0.373548, 0.348669, 0.153734, 
Output 2: 0.281969, 0.718031, 

422
***Wiener Filter*******ApplyPostfilter***
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 1344!
22-Nov-2024 23:58:44 UTC  Closed output file "atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6467376_666_20231207T073802Z_gen_g4_detsim_hitreco__20240510T055018Z_reco2_reco_data_2024-11-22T_232247Z.root"
22-Nov-2024 23:58:44 UTC  Closed input file "root://xrootd.echo.stfc.ac.uk:1094/dune:/protodune/RSE/fardet-hd/d3/23/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6467376_666_20231207T073802Z_gen_g4_detsim_hitreco__20240510T055018Z_reco2.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                             0.226027       20.409        220.73        9.07137       38.9768        100    
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.00134083     0.103069      0.410741      0.0787097     0.0772428       100    
reco:linecluster:LineCluster                          0.00525327     0.346691       7.61774      0.154932      0.844715        100    
reco:trajcluster:TrajCluster                          0.000976468    0.553199        16.13       0.0622261      2.16904        100    
reco:pandora:StandardPandora                           0.0274172      1.79324       16.7391       1.31591       2.12493        100    
reco:pandoraTrack:LArPandoraTrackCreation             0.000121983   0.00511639     0.0564564    0.00158482     0.0107376       100    
reco:pandoraShower:LArPandoraModularShowerCreation    0.000187272   0.00753784     0.057941     0.00307571     0.0114304       100    
reco:pandoracalo:Calorimetry                          9.5543e-05     0.0138185     0.362053      0.0010449     0.0510285       100    
reco:pandorapid:Chi2ParticleID                        3.8488e-05    0.000170182   0.00123341    0.000135078   0.000148238      100    
reco:trkshowersplit:TrackShowerHits                   5.8659e-05     0.0194687     0.558929     0.000714468    0.0781251       100    
reco:pmtrack:PMAlgTrackMaker                          0.000946182     1.94092       38.2445      0.202255       6.1689         100    
reco:pmtrackcalo:Calorimetry                          5.4612e-05    0.00649826     0.126644     0.000865992    0.0199955       100    
reco:pmtrackpid:Chi2ParticleID                        2.7291e-05    0.000261021    0.0021129    0.000157221   0.000382947      100    
reco:pmtrajfit:PMAlgTrajFitter                        0.000174633     1.47608       24.9641       0.29697       3.86154        100    
reco:pmtrajfitcalo:Calorimetry                        4.5809e-05     0.0147657     0.371801     0.00151099     0.0541673       100    
reco:pmtrajfitpid:Chi2ParticleID                      2.4837e-05    0.00018959    0.000748976   0.00015298    0.000140481      100    
reco:pmtracktc:PMAlgTrackMaker                        0.000872171     2.19797       50.8908      0.206427       7.09612        100    
reco:pmtracktccalo:Calorimetry                        4.8473e-05    0.00461293     0.0999252    0.000886704    0.0136323       100    
reco:pmtracktcpid:Chi2ParticleID                      2.1022e-05    0.000240375   0.00188352    0.000151338   0.000311617      100    
reco:pmtrajfittc:PMAlgTrajFitter                      0.000160632     1.15546       19.8214      0.218247       3.04692        100    
reco:pmtrajfittccalo:Calorimetry                      5.4983e-05    0.00852483     0.183102      0.0011124     0.027128        100    
reco:pmtrajfittcpid:Chi2ParticleID                    2.3425e-05    0.000296323   0.00280589    0.000163641   0.000425215      100    
reco:blurredcluster:BlurredClustering                 0.000218338    0.920129       5.13207      0.419966       1.15211        100    
reco:emtrkmichelid:EmTrackMichelId                    0.000121402     7.0488        73.1881       2.38045       13.2973        100    
reco:emshower:EMShower                                0.000165859    0.0269659     0.207767     0.00421143     0.0431827       100    
reco:cvnmap:CVNMapper                                 1.6525e-05    0.00800809     0.0462522    0.00574416    0.00865176       100    
reco:cvneva:CVNEvaluator                              1.6676e-05      0.55272      0.987811      0.765884      0.375695        100    
reco:energyrecnumu:EnergyReco                         0.000352781    0.0269493     0.655768     0.00614048     0.0795027       100    
reco:energyrecnue:EnergyReco                          0.000138348   0.00162814     0.0193588    0.000494469    0.0032298       100    
reco:energyrecnc:EnergyReco                           0.00142093    0.00370116     0.0223698    0.00263853    0.00350242       100    
reco:energyrecnumurange:EnergyReco                    0.000176776    0.0236795     0.667121     0.00220885     0.0807028       100    
reco:energyrecnumumcs:EnergyReco                      0.00090485     0.0249556     0.656802      0.0029814     0.0797278       100    
reco:opdec:Deconvolution                               0.0248281     0.268307       1.06969       0.25592      0.144738        100    
reco:ophitspe:OpHitFinderDeco                          0.0229538      1.62957       3.27394       1.52699       0.73435        100    
reco:opflash:OpFlashFinder                            0.000149194   0.00147979     0.0064599    0.00138405    0.000804513      100    
reco:rns:RandomNumberSaver                            1.8678e-05    3.71179e-05   0.00038586    3.0571e-05    3.67829e-05      100    
reco:anglereconue:NuAngularReco                       0.000167332   0.00130012     0.0159461    0.000446972    0.0027655       100    
reco:anglereconumu:NuAngularReco                       6.655e-05    0.00105339     0.0157803    0.000258044   0.00266312       100    
reco:anglereconuepfps:NuAngularReco                   0.000127932    0.0012097     0.0161677    0.000391228   0.00270505       100    
reco:anglereconumupfps:NuAngularReco                  9.0627e-05    0.00111601     0.0161471    0.000313452   0.00268376       100    
[art]:TriggerResults:TriggerResultInserter            1.0676e-05    1.52353e-05   6.1252e-05    1.42115e-05   5.95628e-06      100    
end_path:out1:RootOutput                               3.715e-06    6.13445e-06   2.8493e-05     5.704e-06    3.16493e-06      100    
end_path:out1:RootOutput(write)                        0.0135016     0.217024      0.699961      0.230024      0.147089        100    
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 40169.8 MB
  Peak resident set size usage (VmHWM): 2385.22 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
Validation_ccnc_atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6467376_666_20231207T073802Z_gen_g4_detsim_hitreco__20240510T055018Z_reco2.root
all-input-dids.txt
atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6467376_666_20231207T073802Z_gen_g4_detsim_hitreco__20240510T055018Z_reco2_reco_2024-11-22T_232247Z.log
atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6467376_666_20231207T073802Z_gen_g4_detsim_hitreco__20240510T055018Z_reco2_reco_data_2024-11-22T_232247Z.root
debugprod.log
jobscript.log
justin-processed-pfns.txt
reco2_hist.root
MyPandoraSettings_Master_Atmos_DUNEFD.xml
MyPandoraSettings_Master_DUNEFD.xml
build_slf7.x86_64
localProducts_larsoft_v09_91_02_e26_prof
setup_env-testreco.sh
srcs
temp.txt
temp2.txt
test
work
justIN time: 2024-11-24 15:37:47 UTC       justIN version: 01.01.09