justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 301487.25@justin-prod-sched01.dune.hep.ac.uk

Jobsub ID301487.25@justin-prod-sched01.dune.hep.ac.uk
Workflow ID4213
Stage ID1
User nameismerio@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2024-11-22 22:57:53
SiteCERN
EntryCMSHTPC_T2_CH_CERN_ce511
Last heartbeat2024-11-23 00:26:32
From worker nodeHostnameb9p12p1803.cern.ch
cpuinfoAMD EPYC 7543 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit171000 (47 hours)
Inner Apptainer?True
Job statefinished
Allocator namejustin-allocator-pro.dune.hep.ac.uk
Started2024-11-22 23:11:26
Input filesfardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6413332_463_20231203T043256Z_gen_g4_detsim_hitreco__20240508T064253Z_reco2.root
JobscriptExit code0
Real time57m (3436s)
CPU time55m (3332s = 96%)
Outputting started2024-11-23 00:08:43
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/ismerio/04213/1/Validation_ccnc_atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6413332_463_20231203T043256Z_gen_g4_detsim_hitreco__20240508T064253Z_reco2.root
Finished2024-11-23 00:26:32
Saved logsjustin-logs:301487.25-justin-prod-sched01.dune.hep.ac.uk.logs.tgz
List job events     Wrapper job log

Jobscript log (last 10,000 characters)

pmtrajfittc@BeginModule  23-Nov-2024 01:08:35 CET run: 6413332 subRun: 1 event: 46400
0 enabled hits in AverageDist2 calculation.
%MSG
%MSG-e pma::Track3D:  PMAlgTrajFitter:pmtrajfittc@BeginModule  23-Nov-2024 01:08:35 CET run: 6413332 subRun: 1 event: 46400
Track empty.
%MSG
%MSG-e pma::Track3D:  PMAlgTrajFitter:pmtrajfittc@BeginModule  23-Nov-2024 01:08:35 CET run: 6413332 subRun: 1 event: 46400
TuneFullTree failed.
%MSG
%MSG-e pma::Track3D:  PMAlgTrajFitter:pmtrajfittc@BeginModule  23-Nov-2024 01:08:35 CET run: 6413332 subRun: 1 event: 46400
Flip, endpoint closer to vStart.
%MSG
%MSG-e pma::Track3D:  PMAlgTrajFitter:pmtrajfittc@BeginModule  23-Nov-2024 01:08:35 CET run: 6413332 subRun: 1 event: 46400
0 enabled hits in AverageDist2 calculation.
%MSG
%MSG-e pma::Track3D:  PMAlgTrajFitter:pmtrajfittc@BeginModule  23-Nov-2024 01:08:35 CET run: 6413332 subRun: 1 event: 46400
Track empty.
%MSG
%MSG-e pma::Track3D:  PMAlgTrajFitter:pmtrajfittc@BeginModule  23-Nov-2024 01:08:35 CET run: 6413332 subRun: 1 event: 46400
TuneFullTree failed.
%MSG
%MSG-e WireIDIntersectionCheck:  EMShower:emshower@BeginModule  23-Nov-2024 01:08:37 CET run: 6413332 subRun: 1 event: 46400
Comparing two wires in the same plane: return failure
%MSG
Boundary wire vector sizes: 123, 129, 107
minwire 0: 2366
minwire 1: 1277
minwire 2: 2055
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 199
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 199
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 199
Classifier summary: 
Output 0: 0.00109242, 0.998681, 0.000226652, 
Output 1: 0.00395515, 0.994913, 0.00112477, 7.21375e-06, 
Output 2: 0.99934, 0.000659533, 

545
***Wiener Filter*******ApplyPostfilter***
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 1894!
23-Nov-2024 01:08:42 CET  Closed output file "atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6413332_463_20231203T043256Z_gen_g4_detsim_hitreco__20240508T064253Z_reco2_reco_data_2024-11-22T_231131Z.root"
23-Nov-2024 01:08:42 CET  Closed input file "root://golias100.farm.particle.cz:1094/dpm/farm.particle.cz/home/dune/RSE/fardet-hd/37/9e/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6413332_463_20231203T043256Z_gen_g4_detsim_hitreco__20240508T064253Z_reco2.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                             0.562835       31.6223       776.774       9.49155       85.0436        100    
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                 0.0164326     0.0522036     0.310879      0.0404294     0.0513702       100    
reco:linecluster:LineCluster                           0.0939716     0.478359       22.1838      0.170026       2.20279        100    
reco:trajcluster:TrajCluster                           0.047164       1.11945       61.6734      0.058329       6.24435        100    
reco:pandora:StandardPandora                           0.0719713      3.04782       95.0351       1.31025       9.83934        100    
reco:pandoraTrack:LArPandoraTrackCreation             0.000128452   0.00664955     0.108401     0.00143167     0.0157392       100    
reco:pandoraShower:LArPandoraModularShowerCreation    0.000163452    0.0153735     0.387651     0.00268078     0.0433812       100    
reco:pandoracalo:Calorimetry                          0.000103561    0.0340459      1.96354     0.00102906     0.197887        100    
reco:pandorapid:Chi2ParticleID                         3.619e-05    0.00018936    0.00160108    0.000140792   0.000185493      100    
reco:trkshowersplit:TrackShowerHits                   0.000105401    0.0331201      1.35926     0.000762479    0.144931        100    
reco:pmtrack:PMAlgTrackMaker                          0.00108615      2.90199       97.9413      0.175011       10.6412        100    
reco:pmtrackcalo:Calorimetry                          6.4101e-05     0.0199839      1.04607     0.000882305    0.107783        100    
reco:pmtrackpid:Chi2ParticleID                        2.5381e-05    0.000339095   0.00572006    0.000145197   0.000687164      100    
reco:pmtrajfit:PMAlgTrajFitter                        0.000174062     2.66987       76.2991      0.218026       8.72511        100    
reco:pmtrajfitcalo:Calorimetry                        5.6581e-05     0.0345853      1.91847     0.00135803     0.194248        100    
reco:pmtrajfitpid:Chi2ParticleID                       2.324e-05    0.00020308    0.00081116    0.000153727   0.000174918      100    
reco:pmtracktc:PMAlgTrackMaker                        0.00100822      3.94844       148.865      0.174371       15.8546        100    
reco:pmtracktccalo:Calorimetry                        6.5431e-05     0.0101212     0.385318     0.000908111    0.0414961       100    
reco:pmtracktcpid:Chi2ParticleID                       2.846e-05    0.000297917   0.00428748    0.000150766   0.000505519      100    
reco:pmtrajfittc:PMAlgTrajFitter                      0.000218733     2.11597       64.9322      0.318857       7.1435         100    
reco:pmtrajfittccalo:Calorimetry                      9.8001e-05     0.019916      0.810845     0.00116722     0.0862998       100    
reco:pmtrajfittcpid:Chi2ParticleID                    4.0721e-05    0.00039157    0.00400128    0.000181933   0.000636955      100    
reco:blurredcluster:BlurredClustering                 0.000125652     1.03704       9.51758       0.40566       1.44234        100    
reco:emtrkmichelid:EmTrackMichelId                    0.000155062     10.5456       184.56        2.3215        24.2102        100    
reco:emshower:EMShower                                0.000227853    0.0214008     0.132395     0.00407567     0.0327774       100    
reco:cvnmap:CVNMapper                                 2.2001e-05     0.0119948     0.0402481     0.0142454     0.0090257       100    
reco:cvneva:CVNEvaluator                               1.566e-05     0.732118       1.23861      0.956757      0.477182        100    
reco:energyrecnumu:EnergyReco                         0.000386205    0.0269649     0.447527     0.00374077     0.0660699       100    
reco:energyrecnue:EnergyReco                          0.000118492   0.00227223     0.0348875    0.000468436   0.00549448       100    
reco:energyrecnc:EnergyReco                           0.00159641    0.00505971     0.0397111    0.00344333    0.00597761       100    
reco:energyrecnumurange:EnergyReco                    0.000187062    0.0247365     0.431729     0.00234262     0.0644564       100    
reco:energyrecnumumcs:EnergyReco                      0.00112914     0.026786      0.453224     0.00367849     0.0655428       100    
reco:opdec:Deconvolution                               0.0610196     0.284359      0.545716       0.27651      0.0925688       100    
reco:ophitspe:OpHitFinderDeco                          0.140923       2.11805       4.87309       1.96416      0.938147        100    
reco:opflash:OpFlashFinder                            0.000169912   0.00165673    0.00423552    0.00160957    0.000695538      100    
reco:rns:RandomNumberSaver                             1.894e-05    4.53446e-05   0.000335284   4.3821e-05    3.09258e-05      100    
reco:anglereconue:NuAngularReco                       0.000164762   0.00187624     0.0349058    0.000462896   0.00478366       100    
reco:anglereconumu:NuAngularReco                      6.5931e-05    0.00161108     0.0327371    0.000231198   0.00469672       100    
reco:anglereconuepfps:NuAngularReco                   0.000125631   0.00174887     0.0299021    0.000392905   0.00450368       100    
reco:anglereconumupfps:NuAngularReco                  8.9181e-05    0.00164222     0.0329056    0.000315754    0.0045499       100    
[art]:TriggerResults:TriggerResultInserter             9.34e-06     1.61248e-05   3.7781e-05    1.5735e-05    4.72374e-06      100    
end_path:out1:RootOutput                               2.61e-06     6.32132e-06    2.293e-05     6.19e-06     2.96983e-06      100    
end_path:out1:RootOutput(write)                        0.0116936     0.265787       1.30821      0.264912      0.205039        100    
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 21734.3 MB
  Peak resident set size usage (VmHWM): 2264.1 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
Validation_ccnc_atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6413332_463_20231203T043256Z_gen_g4_detsim_hitreco__20240508T064253Z_reco2.root
all-input-dids.txt
atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6413332_463_20231203T043256Z_gen_g4_detsim_hitreco__20240508T064253Z_reco2_reco_2024-11-22T_231131Z.log
atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6413332_463_20231203T043256Z_gen_g4_detsim_hitreco__20240508T064253Z_reco2_reco_data_2024-11-22T_231131Z.root
debugprod.log
jobscript.log
justin-processed-pfns.txt
reco2_hist.root
MyPandoraSettings_Master_Atmos_DUNEFD.xml
MyPandoraSettings_Master_DUNEFD.xml
build_slf7.x86_64
localProducts_larsoft_v09_91_02_e26_prof
setup_env-testreco.sh
srcs
temp.txt
temp2.txt
test
work
justIN time: 2024-11-23 14:43:35 UTC       justIN version: 01.01.09