justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 108593.67@justin-prod-sched02.dune.hep.ac.uk

Jobsub ID108593.67@justin-prod-sched02.dune.hep.ac.uk
Workflow ID4213
Stage ID1
User nameismerio@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2024-11-22 23:25:00
SiteUK_RAL-Tier1
EntryLIGO_UK_RAL_arc_ce04
Last heartbeat2024-11-23 00:18:14
From worker nodeHostnamedune001-2993277.0-lcg2719.gridpp.rl.ac.uk
cpuinfoAMD EPYC 9654 96-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit216000 (60 hours)
Inner Apptainer?True
Job statefinished
Allocator namejustin-allocator-pro.dune.hep.ac.uk
Started2024-11-22 23:35:56
Input filesfardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74510902_665_20231203T112055Z_gen_g4_detsim_hitreco__20240509T190709Z_reco2.root
JobscriptExit code0
Real time33m (2037s)
CPU time36m (2164s = 106%)
Outputting started2024-11-23 00:09:54
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/ismerio/04213/1/Validation_ccnc_atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74510902_665_20231203T112055Z_gen_g4_detsim_hitreco__20240509T190709Z_reco2.root
Finished2024-11-23 00:18:14
Saved logsjustin-logs:108593.67-justin-prod-sched02.dune.hep.ac.uk.logs.tgz
List job events     Wrapper job log

Jobscript log (last 10,000 characters)

ireIDIntersectionCheck:  PMAlgTrajFitter:pmtrajfittc@BeginModule  23-Nov-2024 00:09:47 UTC run: 74510902 subRun: 1 event: 66600
1st wire C:0 T:10 P:2 W:716 does not exist (max wire number: 480)
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrajFitter:pmtrajfittc@BeginModule  23-Nov-2024 00:09:47 UTC run: 74510902 subRun: 1 event: 66600
1st wire C:0 T:10 P:2 W:980 does not exist (max wire number: 480)
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrajFitter:pmtrajfittc@BeginModule  23-Nov-2024 00:09:47 UTC run: 74510902 subRun: 1 event: 66600
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrajFitter:pmtrajfittc@BeginModule  23-Nov-2024 00:09:47 UTC run: 74510902 subRun: 1 event: 66600
Comparing two wires in the same plane: return failure
%MSG
%MSG-e pma::Track3D:  PMAlgTrajFitter:pmtrajfittc@BeginModule  23-Nov-2024 00:09:48 UTC run: 74510902 subRun: 1 event: 66600
0 enabled hits in AverageDist2 calculation.
%MSG
%MSG-e pma::Track3D:  PMAlgTrajFitter:pmtrajfittc@BeginModule  23-Nov-2024 00:09:48 UTC run: 74510902 subRun: 1 event: 66600
Track empty.
%MSG
%MSG-e pma::Track3D:  PMAlgTrajFitter:pmtrajfittc@BeginModule  23-Nov-2024 00:09:48 UTC run: 74510902 subRun: 1 event: 66600
TuneFullTree failed.
%MSG
Boundary wire vector sizes: 101, 97, 95
minwire 0: 1515
minwire 1: 2161
minwire 2: 1028
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 199
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 199
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 199
Classifier summary: 
Output 0: 0.0853858, 0.0972081, 0.817406, 
Output 1: 0.0378804, 0.640155, 0.2981, 0.0238643, 
Output 2: 0.0784007, 0.921599, 

533
***Wiener Filter*******ApplyPostfilter***
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 1685!
23-Nov-2024 00:09:53 UTC  Closed output file "atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74510902_665_20231203T112055Z_gen_g4_detsim_hitreco__20240509T190709Z_reco2_reco_data_2024-11-22T_233559Z.root"
23-Nov-2024 00:09:53 UTC  Closed input file "root://xrootd.echo.stfc.ac.uk:1094/dune:/protodune/RSE/fardet-hd/0e/9c/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74510902_665_20231203T112055Z_gen_g4_detsim_hitreco__20240509T190709Z_reco2.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              1.93599       19.1273       200.021       8.01663       28.1691        100    
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.00109278     0.0968364      0.44585      0.0824963     0.0681431       100    
reco:linecluster:LineCluster                           0.0365424      0.34401       4.72556      0.222787      0.499986        100    
reco:trajcluster:TrajCluster                          0.00719872     0.494901       13.5333      0.122694       1.50816        100    
reco:pandora:StandardPandora                           0.895201       1.58885       7.59749       1.25782       1.01441        100    
reco:pandoraTrack:LArPandoraTrackCreation             0.000130406   0.00443046     0.0298884    0.00164348    0.00657273       100    
reco:pandoraShower:LArPandoraModularShowerCreation    0.000176095   0.00958751     0.149423     0.00262067     0.0205342       100    
reco:pandoracalo:Calorimetry                           9.961e-05     0.0101598     0.185769     0.000963537    0.026064        100    
reco:pandorapid:Chi2ParticleID                        3.6735e-05    0.00016551    0.00104731    0.000136756   0.000127101      100    
reco:trkshowersplit:TrackShowerHits                   0.000124077    0.0135969      0.26556     0.000650395    0.0363252       100    
reco:pmtrack:PMAlgTrackMaker                          0.000985926     1.44495        22.13       0.167406       3.24016        100    
reco:pmtrackcalo:Calorimetry                          5.6254e-05    0.00538145     0.0669764    0.000888694    0.0113959       100    
reco:pmtrackpid:Chi2ParticleID                        2.2494e-05    0.000254491   0.00179239    0.000144998   0.000285274      100    
reco:pmtrajfit:PMAlgTrajFitter                        0.000205801     1.41328       23.9615      0.224618       3.14849        100    
reco:pmtrajfitcalo:Calorimetry                        4.8774e-05     0.0110552      0.23133     0.00122241     0.0310002       100    
reco:pmtrajfitpid:Chi2ParticleID                      2.1713e-05    0.000184721   0.000901287   0.00015269    0.000128352      100    
reco:pmtracktc:PMAlgTrackMaker                        0.00104847      1.81698       43.8123      0.209545       5.2224         100    
reco:pmtracktccalo:Calorimetry                        6.2785e-05    0.00421294     0.0427393    0.000849385    0.0080786       100    
reco:pmtracktcpid:Chi2ParticleID                      2.4737e-05    0.000235996   0.00135423    0.000156661   0.000228801      100    
reco:pmtrajfittc:PMAlgTrajFitter                      0.000179441     1.1012        14.3494      0.247174       1.9974         100    
reco:pmtrajfittccalo:Calorimetry                      4.6951e-05    0.00735321     0.0999093    0.00105195     0.0156177       100    
reco:pmtrajfittcpid:Chi2ParticleID                    1.9509e-05    0.000304699   0.00192807    0.000166946   0.000338243      100    
reco:blurredcluster:BlurredClustering                 0.00367747     0.857475       4.58674      0.537911      0.999543        100    
reco:emtrkmichelid:EmTrackMichelId                     0.166666       7.37388       59.3972       2.33977       11.2748        100    
reco:emshower:EMShower                                0.000316176    0.0186887     0.153951     0.00675859     0.0271966       100    
reco:cvnmap:CVNMapper                                 2.0912e-05    0.00806984     0.0353906    0.00419674    0.00901968       100    
reco:cvneva:CVNEvaluator                              1.7066e-05     0.491211       1.0247       0.590313      0.316067        100    
reco:energyrecnumu:EnergyReco                         0.000307193    0.0224445     0.399307     0.00670081     0.0538357       100    
reco:energyrecnue:EnergyReco                          0.000101433   0.00121027     0.0116851    0.000432963   0.00189893       100    
reco:energyrecnc:EnergyReco                           0.00132083    0.00293475     0.0130195    0.00247135     0.001883        100    
reco:energyrecnumurange:EnergyReco                    0.000171098    0.0165585     0.239251      0.002073      0.0427289       100    
reco:energyrecnumumcs:EnergyReco                      0.00130323     0.0177364     0.243088     0.00298055     0.0427257       100    
reco:opdec:Deconvolution                               0.0707905     0.290998      0.658436      0.268889      0.139265        100    
reco:ophitspe:OpHitFinderDeco                          0.204788       1.44905       2.88605       1.34857       0.6594         100    
reco:opflash:OpFlashFinder                            0.000402106   0.00133079    0.00372552    0.00136163    0.000547462      100    
reco:rns:RandomNumberSaver                            2.1953e-05    3.84969e-05   0.000324979   3.21135e-05   3.72333e-05      100    
reco:anglereconue:NuAngularReco                       0.000169476   0.00107673     0.0115341    0.000419918   0.00175885       100    
reco:anglereconumu:NuAngularReco                      5.9259e-05    0.000850182    0.0114507    0.000219716   0.00172256       100    
reco:anglereconuepfps:NuAngularReco                   0.000113131   0.000986703    0.0110122    0.000342757   0.00170114       100    
reco:anglereconumupfps:NuAngularReco                  8.4257e-05    0.000903319    0.0116654    0.000268129   0.00174427       100    
[art]:TriggerResults:TriggerResultInserter             9.144e-06    1.39634e-05   4.5588e-05    1.25235e-05   5.57565e-06      100    
end_path:out1:RootOutput                               3.295e-06    4.46991e-06   2.7121e-05     4.061e-06    2.78584e-06      100    
end_path:out1:RootOutput(write)                        0.0254322     0.201826      0.699823       0.21173      0.120803        100    
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 40115.6 MB
  Peak resident set size usage (VmHWM): 2306.87 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
Validation_ccnc_atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74510902_665_20231203T112055Z_gen_g4_detsim_hitreco__20240509T190709Z_reco2.root
all-input-dids.txt
atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74510902_665_20231203T112055Z_gen_g4_detsim_hitreco__20240509T190709Z_reco2_reco_2024-11-22T_233559Z.log
atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74510902_665_20231203T112055Z_gen_g4_detsim_hitreco__20240509T190709Z_reco2_reco_data_2024-11-22T_233559Z.root
debugprod.log
jobscript.log
justin-processed-pfns.txt
reco2_hist.root
MyPandoraSettings_Master_Atmos_DUNEFD.xml
MyPandoraSettings_Master_DUNEFD.xml
build_slf7.x86_64
localProducts_larsoft_v09_91_02_e26_prof
setup_env-testreco.sh
srcs
temp.txt
temp2.txt
test
work
justIN time: 2024-11-24 03:58:12 UTC       justIN version: 01.01.09