justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 301553.84@justin-prod-sched01.dune.hep.ac.uk

Jobsub ID301553.84@justin-prod-sched01.dune.hep.ac.uk
Workflow ID4213
Stage ID1
User nameismerio@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2024-11-22 23:36:34
SiteUK_RAL-Tier1
EntryLIGO_UK_RAL_arc_ce02
Last heartbeat2024-11-23 00:26:29
From worker nodeHostnamedune001-3353833.0-lcg2725.gridpp.rl.ac.uk
cpuinfoAMD EPYC 9654 96-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit216000 (60 hours)
Inner Apptainer?True
Job statefinished
Allocator namejustin-allocator-pro.dune.hep.ac.uk
Started2024-11-22 23:51:11
Input filesfardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6414162_669_20231203T143600Z_gen_g4_detsim_hitreco__20240508T074619Z_reco2.root
JobscriptExit code0
Real time28m (1684s)
CPU time31m (1873s = 111%)
Outputting started2024-11-23 00:19:16
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/ismerio/04213/1/Validation_ccnc_atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6414162_669_20231203T143600Z_gen_g4_detsim_hitreco__20240508T074619Z_reco2.root
Finished2024-11-23 00:26:29
Saved logsjustin-logs:301553.84-justin-prod-sched01.dune.hep.ac.uk.logs.tgz
List job events     Wrapper job log

Jobscript log (last 10,000 characters)

pandoraShower@BeginModule  23-Nov-2024 00:19:13 UTC run: 6414162 subRun: 1 event: 67000
Trying to add data product: InitialTrack. This element does not exist in the element holder
%MSG
%MSG-e ShowerProducedPtrsHolder:   LArPandoraModularShowerCreation:pandoraShower@BeginModule  23-Nov-2024 00:19:13 UTC run: 6414162 subRun: 1 event: 67000
Trying to add data product: ShowerPCA. This element does not exist in the element holder
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrajFitter:pmtrajfit@BeginModule  23-Nov-2024 00:19:13 UTC run: 6414162 subRun: 1 event: 67000
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  23-Nov-2024 00:19:13 UTC run: 6414162 subRun: 1 event: 67000
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  23-Nov-2024 00:19:13 UTC run: 6414162 subRun: 1 event: 67000
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  23-Nov-2024 00:19:13 UTC run: 6414162 subRun: 1 event: 67000
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrajFitter:pmtrajfittc@BeginModule  23-Nov-2024 00:19:13 UTC run: 6414162 subRun: 1 event: 67000
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrajFitter:pmtrajfittc@BeginModule  23-Nov-2024 00:19:13 UTC run: 6414162 subRun: 1 event: 67000
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  EMShower:emshower@BeginModule  23-Nov-2024 00:19:13 UTC run: 6414162 subRun: 1 event: 67000
Comparing two wires in the same plane: return failure
%MSG
364
***Wiener Filter*******ApplyPostfilter***
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 898!
23-Nov-2024 00:19:14 UTC  Closed output file "atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6414162_669_20231203T143600Z_gen_g4_detsim_hitreco__20240508T074619Z_reco2_reco_data_2024-11-22T_235115Z.root"
23-Nov-2024 00:19:14 UTC  Closed input file "root://mover.pp.rl.ac.uk:1094/pnfs/pp.rl.ac.uk/data/dune/fardet-hd/44/cf/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6414162_669_20231203T143600Z_gen_g4_detsim_hitreco__20240508T074619Z_reco2.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                             0.0711037      16.2559       161.25        7.99328       26.4874        100    
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.00164471     0.0201993     0.570439      0.0176089     0.0563496       100    
reco:linecluster:LineCluster                          0.00184725     0.0875459      2.81194      0.0164243     0.345384        100    
reco:trajcluster:TrajCluster                          0.000967473    0.280007       5.05796      0.012889      0.960435        100    
reco:pandora:StandardPandora                           0.0149494      1.42716       8.96827       1.1493        1.15331        100    
reco:pandoraTrack:LArPandoraTrackCreation             0.000141784   0.00362483     0.0329431    0.00151188    0.00599815       100    
reco:pandoraShower:LArPandoraModularShowerCreation    0.000191198   0.00766334     0.174761     0.00229588     0.0195062       100    
reco:pandoracalo:Calorimetry                          8.7432e-05    0.00948017     0.213791     0.000992295    0.0313465       100    
reco:pandorapid:Chi2ParticleID                         3.302e-05    0.000133193   0.000832348   0.000108208   0.000103702      100    
reco:trkshowersplit:TrackShowerHits                   5.7807e-05     0.0124728     0.248164     0.000670974    0.0417464       100    
reco:pmtrack:PMAlgTrackMaker                          0.00109239      1.30623       23.4577      0.209527       3.53169        100    
reco:pmtrackcalo:Calorimetry                          6.3897e-05    0.00461427     0.056811     0.000951993    0.0110921       100    
reco:pmtrackpid:Chi2ParticleID                        2.3215e-05    0.000225323   0.00132862     0.0001422    0.000265552      100    
reco:pmtrajfit:PMAlgTrajFitter                        0.000196246     1.25703       15.5731      0.302744       2.77071        100    
reco:pmtrajfitcalo:Calorimetry                        6.5079e-05    0.00904871     0.213065     0.00160426     0.0284602       100    
reco:pmtrajfitpid:Chi2ParticleID                      2.4758e-05    0.000165087   0.000495298   0.000139681   9.66729e-05      100    
reco:pmtracktc:PMAlgTrackMaker                        0.00105697      1.58943       27.5441      0.196867       4.40827        100    
reco:pmtracktccalo:Calorimetry                        6.0271e-05    0.00377301     0.0530876    0.000981969   0.00930325       100    
reco:pmtracktcpid:Chi2ParticleID                      2.0241e-05    0.00021602    0.00151696    0.00015204    0.000243501      100    
reco:pmtrajfittc:PMAlgTrajFitter                       0.0001658     0.838099       12.3974       0.26686       1.87351        100    
reco:pmtrajfittccalo:Calorimetry                       5.984e-05    0.00628882     0.108184     0.00121989     0.017887        100    
reco:pmtrajfittcpid:Chi2ParticleID                    2.4267e-05    0.000259282    0.002158     0.000154904   0.000335048      100    
reco:blurredcluster:BlurredClustering                 0.000213473    0.850716       4.94186      0.388025       1.09716        100    
reco:emtrkmichelid:EmTrackMichelId                    9.7588e-05      6.2718        63.5179       2.5242        10.9234        100    
reco:emshower:EMShower                                0.000146402    0.0221969     0.200392     0.00552764     0.0377075       100    
reco:cvnmap:CVNMapper                                 1.7426e-05    0.00651066     0.0400052    0.00401217    0.00839191       100    
reco:cvneva:CVNEvaluator                              1.4972e-05     0.524451       0.95512       0.61417      0.336294        100    
reco:energyrecnumu:EnergyReco                         0.000299322    0.0201353     0.360266     0.00647664     0.0481767       100    
reco:energyrecnue:EnergyReco                          9.4002e-05    0.00109866     0.0126923    0.000409389   0.00199438       100    
reco:energyrecnc:EnergyReco                           0.00136372    0.00291307     0.0164825    0.00242359     0.0022772       100    
reco:energyrecnumurange:EnergyReco                    0.000154574    0.0187287     0.536839     0.00233355     0.0624055       100    
reco:energyrecnumumcs:EnergyReco                      0.000612976    0.0197541     0.528682     0.00292715     0.0610949       100    
reco:opdec:Deconvolution                              0.00449427     0.135274      0.292462      0.132379      0.0539839       100    
reco:ophitspe:OpHitFinderDeco                         0.00553521      1.32051       3.03633       1.2099       0.612657        100    
reco:opflash:OpFlashFinder                            0.000104317   0.00121011    0.00369708    0.00115495    0.00053085       100    
reco:rns:RandomNumberSaver                            1.6976e-05    2.81578e-05   0.000199652   2.3841e-05    1.91641e-05      100    
reco:anglereconue:NuAngularReco                       0.00014572    0.000876915    0.0118803    0.00036061    0.00167582       100    
reco:anglereconumu:NuAngularReco                      5.4092e-05    0.000686705    0.0106275    0.000181004    0.0015759       100    
reco:anglereconuepfps:NuAngularReco                   0.000113852   0.000799274    0.0106797    0.000293344   0.00156347       100    
reco:anglereconumupfps:NuAngularReco                  7.4062e-05    0.000731419    0.0107019    0.000229893    0.0015605       100    
[art]:TriggerResults:TriggerResultInserter             8.093e-06    1.11976e-05   3.1938e-05    1.0712e-05    3.05735e-06      100    
end_path:out1:RootOutput                               2.644e-06    3.97212e-06   2.1763e-05     3.31e-06     2.43939e-06      100    
end_path:out1:RootOutput(write)                        0.014119      0.192097      0.737353      0.193579      0.121209        100    
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 40186.3 MB
  Peak resident set size usage (VmHWM): 2419.73 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
Validation_ccnc_atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6414162_669_20231203T143600Z_gen_g4_detsim_hitreco__20240508T074619Z_reco2.root
all-input-dids.txt
atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6414162_669_20231203T143600Z_gen_g4_detsim_hitreco__20240508T074619Z_reco2_reco_2024-11-22T_235115Z.log
atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6414162_669_20231203T143600Z_gen_g4_detsim_hitreco__20240508T074619Z_reco2_reco_data_2024-11-22T_235115Z.root
debugprod.log
jobscript.log
justin-processed-pfns.txt
reco2_hist.root
MyPandoraSettings_Master_Atmos_DUNEFD.xml
MyPandoraSettings_Master_DUNEFD.xml
build_slf7.x86_64
localProducts_larsoft_v09_91_02_e26_prof
setup_env-testreco.sh
srcs
temp.txt
temp2.txt
test
work
justIN time: 2024-11-24 16:06:51 UTC       justIN version: 01.01.09