justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 108607.53@justin-prod-sched02.dune.hep.ac.uk

Jobsub ID108607.53@justin-prod-sched02.dune.hep.ac.uk
Workflow ID4213
Stage ID1
User nameismerio@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2024-11-22 23:37:42
SiteUK_RAL-Tier1
EntryLIGO_UK_RAL_arc_ce04
Last heartbeat2024-11-23 00:17:09
From worker nodeHostnamedune001-2995519.0-lcg2699.gridpp.rl.ac.uk
cpuinfoAMD EPYC 9654 96-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit216000 (60 hours)
Inner Apptainer?True
Job statefinished
Allocator namejustin-allocator-pro.dune.hep.ac.uk
Started2024-11-22 23:55:16
Input filesfardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6472619_318_20231208T061844Z_gen_g4_detsim_hitreco__20240510T045730Z_reco2.root
JobscriptExit code0
Real time16m (971s)
CPU time20m (1238s = 127%)
Outputting started2024-11-23 00:11:27
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/ismerio/04213/1/Validation_ccnc_atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6472619_318_20231208T061844Z_gen_g4_detsim_hitreco__20240510T045730Z_reco2.root
Finished2024-11-23 00:17:09
Saved logsjustin-logs:108607.53-justin-prod-sched02.dune.hep.ac.uk.logs.tgz
List job events     Wrapper job log

Jobscript log (last 10,000 characters)

abled hits in AverageDist2 calculation.
%MSG
%MSG-e pma::Track3D:  PMAlgTrajFitter:pmtrajfittc@BeginModule  23-Nov-2024 00:11:02 UTC run: 6472619 subRun: 1 event: 31900
Track empty.
%MSG
%MSG-e pma::Track3D:  PMAlgTrajFitter:pmtrajfittc@BeginModule  23-Nov-2024 00:11:02 UTC run: 6472619 subRun: 1 event: 31900
TuneFullTree failed.
%MSG
%MSG-e pma::Track3D:  PMAlgTrajFitter:pmtrajfittc@BeginModule  23-Nov-2024 00:11:02 UTC run: 6472619 subRun: 1 event: 31900
0 enabled hits in AverageDist2 calculation.
%MSG
%MSG-e pma::Track3D:  PMAlgTrajFitter:pmtrajfittc@BeginModule  23-Nov-2024 00:11:02 UTC run: 6472619 subRun: 1 event: 31900
Track empty.
%MSG
%MSG-e pma::Track3D:  PMAlgTrajFitter:pmtrajfittc@BeginModule  23-Nov-2024 00:11:02 UTC run: 6472619 subRun: 1 event: 31900
TuneFullTree failed.
%MSG
%MSG-e pma::Track3D:  PMAlgTrajFitter:pmtrajfittc@BeginModule  23-Nov-2024 00:11:02 UTC run: 6472619 subRun: 1 event: 31900
0 enabled hits in AverageDist2 calculation.
%MSG
%MSG-e pma::Track3D:  PMAlgTrajFitter:pmtrajfittc@BeginModule  23-Nov-2024 00:11:02 UTC run: 6472619 subRun: 1 event: 31900
Track empty.
%MSG
%MSG-e pma::Track3D:  PMAlgTrajFitter:pmtrajfittc@BeginModule  23-Nov-2024 00:11:02 UTC run: 6472619 subRun: 1 event: 31900
TuneFullTree failed.
%MSG
Boundary wire vector sizes: 1621, 1741, 1283
minwire 0: 1848
minwire 1: 489
minwire 2: 1913
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 199
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 199
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 199
Classifier summary: 
Output 0: 0.123931, 0.158563, 0.717506, 
Output 1: 0.254804, 0.576527, 0.152053, 0.0166164, 
Output 2: 0.214299, 0.785701, 

672
***Wiener Filter*******ApplyPostfilter***
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 2386!
23-Nov-2024 00:11:26 UTC  Closed output file "atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6472619_318_20231208T061844Z_gen_g4_detsim_hitreco__20240510T045730Z_reco2_reco_data_2024-11-22T_235520Z.root"
23-Nov-2024 00:11:26 UTC  Closed input file "root://mover.pp.rl.ac.uk:1094/pnfs/pp.rl.ac.uk/data/dune/fardet-hd/4a/59/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6472619_318_20231208T061844Z_gen_g4_detsim_hitreco__20240510T045730Z_reco2.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              1.33584       9.22355       65.9637       6.28577       9.42684        100    
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                 0.001425      0.017559      0.206075      0.0167831     0.0212263       100    
reco:linecluster:LineCluster                          0.00740627     0.0192631     0.106067      0.0146887     0.0147648       100    
reco:trajcluster:TrajCluster                          0.00139303     0.044989       1.17352     0.00802889     0.128094        100    
reco:pandora:StandardPandora                           0.839749       1.16117       2.06727       1.10738      0.230639        100    
reco:pandoraTrack:LArPandoraTrackCreation             0.000116035   0.00242475     0.0217502    0.00125412    0.00347853       100    
reco:pandoraShower:LArPandoraModularShowerCreation    0.000155003   0.00486763     0.0551322     0.0019829    0.00844462       100    
reco:pandoracalo:Calorimetry                          8.5339e-05     0.0023755     0.0350376    0.000628611   0.00524077       100    
reco:pandorapid:Chi2ParticleID                        2.9394e-05    0.000112912   0.000477248   9.50585e-05   7.4064e-05       100    
reco:trkshowersplit:TrackShowerHits                   7.4993e-05    0.00259176     0.047535     0.000468535   0.00602397       100    
reco:pmtrack:PMAlgTrackMaker                          0.00100806     0.433582        5.788       0.109669      0.827957        100    
reco:pmtrackcalo:Calorimetry                          5.4142e-05    0.00173775     0.0319886    0.000731435   0.00373526       100    
reco:pmtrackpid:Chi2ParticleID                         2.017e-05    0.000154906   0.000764791   0.000124026   0.000134743      100    
reco:pmtrajfit:PMAlgTrajFitter                        0.000129454    0.441816       3.44658      0.133818      0.633391        100    
reco:pmtrajfitcalo:Calorimetry                        4.1262e-05    0.00274796     0.0369345    0.00104915    0.00495024       100    
reco:pmtrajfitpid:Chi2ParticleID                      1.8247e-05    0.000134282   0.000404588   0.000117547   8.39534e-05      100    
reco:pmtracktc:PMAlgTrackMaker                        0.000767976    0.482595       8.48889      0.117776       1.03937        100    
reco:pmtracktccalo:Calorimetry                        4.5168e-05    0.00135574     0.0126052    0.000817745   0.00179867       100    
reco:pmtracktcpid:Chi2ParticleID                       1.951e-05    0.000145421   0.000599913   0.000136306   9.86879e-05      100    
reco:pmtrajfittc:PMAlgTrajFitter                      0.000128723    0.420947       4.38357      0.122209      0.644347        100    
reco:pmtrajfittccalo:Calorimetry                      4.1873e-05    0.00221415     0.0369425    0.000824651   0.00439437       100    
reco:pmtrajfittcpid:Chi2ParticleID                    1.7527e-05    0.000178666   0.000972794   0.000120817   0.00015957       100    
reco:blurredcluster:BlurredClustering                 0.00630656     0.634245       4.79904      0.315816      0.794383        100    
reco:emtrkmichelid:EmTrackMichelId                     0.172304       3.68135       35.1494       1.73614       5.23733        100    
reco:emshower:EMShower                                0.000285089    0.0135446     0.115098     0.00432402     0.0216553       100    
reco:cvnmap:CVNMapper                                  2.004e-05     0.0061792     0.0297199     0.004052     0.00752521       100    
reco:cvneva:CVNEvaluator                              1.6495e-05     0.469728      0.924915      0.568148      0.311274        100    
reco:energyrecnumu:EnergyReco                         0.000297909    0.0141321     0.343417     0.00560918     0.0421973       100    
reco:energyrecnue:EnergyReco                          0.000105128   0.000791541   0.00727026    0.000373662   0.00100545       100    
reco:energyrecnc:EnergyReco                            0.0012733    0.00229296     0.0101376     0.0021674    0.00119372       100    
reco:energyrecnumurange:EnergyReco                    0.000129945    0.0105072     0.346141      0.0015166     0.0419928       100    
reco:energyrecnumumcs:EnergyReco                      0.00123946     0.011515      0.341218     0.00243221     0.0416089       100    
reco:opdec:Deconvolution                               0.0137667     0.114975      0.223203      0.115171      0.045947        100    
reco:ophitspe:OpHitFinderDeco                          0.081099       1.05839       2.44033      0.999123       0.51174        100    
reco:opflash:OpFlashFinder                            0.000240092   0.00108607    0.00255275    0.00102918    0.000441436      100    
reco:rns:RandomNumberSaver                            1.8688e-05    3.03424e-05   0.000213582   2.49275e-05   2.1227e-05       100    
reco:anglereconue:NuAngularReco                       0.000152719   0.000587615   0.00684123    0.000315159   0.000808442      100    
reco:anglereconumu:NuAngularReco                       5.315e-05    0.000399526   0.00568024    0.000138738   0.000721086      100    
reco:anglereconuepfps:NuAngularReco                   9.6686e-05    0.000512618   0.00600988    0.000258233   0.000760992      100    
reco:anglereconumupfps:NuAngularReco                  7.0235e-05    0.00042722    0.00578878    0.000179561   0.000715269      100    
[art]:TriggerResults:TriggerResultInserter             9.173e-06    1.3516e-05    4.1222e-05    1.23435e-05   4.62958e-06      100    
end_path:out1:RootOutput                               2.955e-06    4.56955e-06   1.7366e-05     4.006e-06    1.68103e-06      100    
end_path:out1:RootOutput(write)                        0.0160788     0.158128      0.519555      0.167299      0.0861315       100    
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 39841 MB
  Peak resident set size usage (VmHWM): 2043.25 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
Validation_ccnc_atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6472619_318_20231208T061844Z_gen_g4_detsim_hitreco__20240510T045730Z_reco2.root
all-input-dids.txt
atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6472619_318_20231208T061844Z_gen_g4_detsim_hitreco__20240510T045730Z_reco2_reco_2024-11-22T_235520Z.log
atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6472619_318_20231208T061844Z_gen_g4_detsim_hitreco__20240510T045730Z_reco2_reco_data_2024-11-22T_235520Z.root
debugprod.log
jobscript.log
justin-processed-pfns.txt
reco2_hist.root
MyPandoraSettings_Master_Atmos_DUNEFD.xml
MyPandoraSettings_Master_DUNEFD.xml
build_slf7.x86_64
localProducts_larsoft_v09_91_02_e26_prof
setup_env-testreco.sh
srcs
temp.txt
temp2.txt
test
work
justIN time: 2024-11-24 04:08:33 UTC       justIN version: 01.01.09