justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 108512.134@justin-prod-sched02.dune.hep.ac.uk

Jobsub ID108512.134@justin-prod-sched02.dune.hep.ac.uk
Workflow ID4213
Stage ID1
User nameismerio@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2024-11-22 22:14:30
SiteUK_Durham
EntryDUNE_UK_SGridDurham_ce3
Last heartbeat2024-11-22 23:34:13
From worker nodeHostnamen260.dur.scotgrid.ac.uk
cpuinfoAMD EPYC 7763 64-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit171000 (47 hours)
Inner Apptainer?True
Job statefinished
Allocator namejustin-allocator-pro.dune.hep.ac.uk
Started2024-11-22 22:34:12
Input filesfardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50620606_413_20231207T105608Z_gen_g4_detsim_hitreco__20240510T040035Z_reco2.root
JobscriptExit code0
Real time58m (3501s)
CPU time51m (3108s = 88%)
Outputting started2024-11-22 23:32:33
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/ismerio/04213/1/Validation_ccnc_atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50620606_413_20231207T105608Z_gen_g4_detsim_hitreco__20240510T040035Z_reco2.root
Finished2024-11-22 23:34:05
Saved logsjustin-logs:108512.134-justin-prod-sched02.dune.hep.ac.uk.logs.tgz
List job events     Wrapper job log

Jobscript log (last 10,000 characters)

ane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrajFitter:pmtrajfit@BeginModule  22-Nov-2024 23:32:29 GMT run: 50620606 subRun: 1 event: 41400
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  22-Nov-2024 23:32:29 GMT run: 50620606 subRun: 1 event: 41400
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  22-Nov-2024 23:32:29 GMT run: 50620606 subRun: 1 event: 41400
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  22-Nov-2024 23:32:29 GMT run: 50620606 subRun: 1 event: 41400
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  22-Nov-2024 23:32:29 GMT run: 50620606 subRun: 1 event: 41400
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  22-Nov-2024 23:32:29 GMT run: 50620606 subRun: 1 event: 41400
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrajFitter:pmtrajfittc@BeginModule  22-Nov-2024 23:32:29 GMT run: 50620606 subRun: 1 event: 41400
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrajFitter:pmtrajfittc@BeginModule  22-Nov-2024 23:32:29 GMT run: 50620606 subRun: 1 event: 41400
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrajFitter:pmtrajfittc@BeginModule  22-Nov-2024 23:32:29 GMT run: 50620606 subRun: 1 event: 41400
Comparing two wires in the same plane: return failure
%MSG
345
***Wiener Filter*******ApplyPostfilter***
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 1002!
22-Nov-2024 23:32:32 GMT  Closed output file "atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50620606_413_20231207T105608Z_gen_g4_detsim_hitreco__20240510T040035Z_reco2_reco_data_2024-11-22T_223417Z.root"
22-Nov-2024 23:32:32 GMT  Closed input file "root://mover.pp.rl.ac.uk:1094/pnfs/pp.rl.ac.uk/data/dune/fardet-hd/c3/d9/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50620606_413_20231207T105608Z_gen_g4_detsim_hitreco__20240510T040035Z_reco2.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              2.38001       31.0089       400.55        9.65788       62.597         100    
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                 0.0179865     0.039803      0.0718611     0.0373751     0.0134855       100    
reco:linecluster:LineCluster                           0.176275      0.335328       3.84891      0.196338      0.510562        100    
reco:trajcluster:TrajCluster                           0.0513862     0.763823       24.1976      0.0619832      2.92179        100    
reco:pandora:StandardPandora                            1.08557       2.13402       27.2636       1.22616       3.42103        100    
reco:pandoraTrack:LArPandoraTrackCreation             0.000147398   0.00799509      0.17446      0.0015001     0.0212629       100    
reco:pandoraShower:LArPandoraModularShowerCreation    0.00021836     0.0168555      0.18784     0.00341722     0.0344907       100    
reco:pandoracalo:Calorimetry                          0.000117401    0.0217814     0.415293     0.00111746     0.0667936       100    
reco:pandorapid:Chi2ParticleID                         4.265e-05    0.000214296   0.00256826    0.000151435   0.000282687      100    
reco:trkshowersplit:TrackShowerHits                   0.000148239    0.0301356     0.783626     0.000866261    0.104983        100    
reco:pmtrack:PMAlgTrackMaker                          0.00138473      2.85683       55.241       0.243027       8.04446        100    
reco:pmtrackcalo:Calorimetry                          9.3396e-05     0.0156473     0.380608     0.00120602     0.0528736       100    
reco:pmtrackpid:Chi2ParticleID                         3.723e-05    0.000372146   0.00326763    0.000181422   0.000590276      100    
reco:pmtrajfit:PMAlgTrajFitter                        0.000216448     2.54427       47.879       0.370869       6.96898        100    
reco:pmtrajfitcalo:Calorimetry                        8.0672e-05     0.0228645     0.433181     0.00170445     0.0692142       100    
reco:pmtrajfitpid:Chi2ParticleID                      3.4154e-05    0.000241671   0.00154872    0.000189446   0.00021282       100    
reco:pmtracktc:PMAlgTrackMaker                        0.00124288      3.28793       58.1821      0.192979       9.41734        100    
reco:pmtracktccalo:Calorimetry                        8.7134e-05     0.0113546     0.304873     0.00108075     0.0371626       100    
reco:pmtracktcpid:Chi2ParticleID                      3.5708e-05    0.000348295   0.00314464    0.000179492   0.000501388      100    
reco:pmtrajfittc:PMAlgTrajFitter                      0.000201078     1.89038       29.9956      0.344291       4.6673         100    
reco:pmtrajfittccalo:Calorimetry                      8.1944e-05     0.0192435      0.4488      0.00133893     0.0621725       100    
reco:pmtrajfittcpid:Chi2ParticleID                    3.5637e-05    0.000451697   0.00395052    0.000209399   0.000735235      100    
reco:blurredcluster:BlurredClustering                 0.00313642      1.08624       8.51381      0.528621       1.5228         100    
reco:emtrkmichelid:EmTrackMichelId                     0.283326       12.2134       150.453       2.82043       24.8695        100    
reco:emshower:EMShower                                0.000329089    0.0268651     0.235002     0.00459016     0.0452284       100    
reco:cvnmap:CVNMapper                                 3.3353e-05     0.0103131     0.0432027    0.00944337    0.00890871       100    
reco:cvneva:CVNEvaluator                              2.1811e-05     0.658227       1.20551      0.871499      0.415272        100    
reco:energyrecnumu:EnergyReco                         0.000401504    0.0680778      1.50808     0.00453633     0.225632        100    
reco:energyrecnue:EnergyReco                          0.000195097   0.00256019     0.0481098    0.000500306   0.00605453       100    
reco:energyrecnc:EnergyReco                           0.00249225    0.00511299     0.0530587    0.00310173    0.00638341       100    
reco:energyrecnumurange:EnergyReco                    0.000203713    0.065898       1.4959      0.00298323     0.224737        100    
reco:energyrecnumumcs:EnergyReco                      0.000854278    0.0678928      1.49778     0.00401954     0.224959        100    
reco:opdec:Deconvolution                               0.117105      0.303138      0.534924      0.298787       0.1004         100    
reco:ophitspe:OpHitFinderDeco                          0.486942       2.22669       4.25762       2.24091      0.881841        100    
reco:opflash:OpFlashFinder                            0.000406734   0.00168256    0.00639205    0.00154427    0.00081985       100    
reco:rns:RandomNumberSaver                            2.2743e-05    4.60312e-05   0.000436451    4.237e-05    4.13903e-05      100    
reco:anglereconue:NuAngularReco                       0.000192122   0.00198306     0.0429517    0.000442252    0.0052282       100    
reco:anglereconumu:NuAngularReco                      0.000104917    0.0017428     0.0427822    0.000299714   0.00517654       100    
reco:anglereconuepfps:NuAngularReco                   0.000146626   0.00190962     0.0427704    0.000432213   0.00518447       100    
reco:anglereconumupfps:NuAngularReco                  0.000113975   0.00182293     0.0426517    0.000360623   0.00518235       100    
[art]:TriggerResults:TriggerResultInserter            1.0419e-05    1.57931e-05   5.9622e-05    1.4868e-05    5.10941e-06      100    
end_path:out1:RootOutput                               2.875e-06    6.03081e-06   1.9206e-05     6.192e-06    2.57741e-06      100    
end_path:out1:RootOutput(write)                        0.0258233     0.263199       1.05565      0.255558      0.191848        100    
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 38410.5 MB
  Peak resident set size usage (VmHWM): 2349.35 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
Validation_ccnc_atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50620606_413_20231207T105608Z_gen_g4_detsim_hitreco__20240510T040035Z_reco2.root
all-input-dids.txt
atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50620606_413_20231207T105608Z_gen_g4_detsim_hitreco__20240510T040035Z_reco2_reco_2024-11-22T_223417Z.log
atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50620606_413_20231207T105608Z_gen_g4_detsim_hitreco__20240510T040035Z_reco2_reco_data_2024-11-22T_223417Z.root
debugprod.log
jobscript.log
justin-processed-pfns.txt
reco2_hist.root
MyPandoraSettings_Master_Atmos_DUNEFD.xml
MyPandoraSettings_Master_DUNEFD.xml
build_slf7.x86_64
localProducts_larsoft_v09_91_02_e26_prof
setup_env-testreco.sh
srcs
temp.txt
temp2.txt
test
work
justIN time: 2024-11-23 12:24:13 UTC       justIN version: 01.01.09