justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 108609.32@justin-prod-sched02.dune.hep.ac.uk

Jobsub ID108609.32@justin-prod-sched02.dune.hep.ac.uk
Workflow ID4213
Stage ID1
User nameismerio@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2024-11-22 23:38:51
SiteUK_RAL-Tier1
EntryLIGO_UK_RAL_arc_ce04
Last heartbeat2024-11-23 00:26:50
From worker nodeHostnamedune001-2994123.0-lcg2695.gridpp.rl.ac.uk
cpuinfoAMD EPYC 9654 96-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit216000 (60 hours)
Inner Apptainer?True
Job statefinished
Allocator namejustin-allocator-pro.dune.hep.ac.uk
Started2024-11-22 23:55:41
Input filesfardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74525168_357_20231206T014538Z_gen_g4_detsim_hitreco__20240510T062709Z_reco2.root
JobscriptExit code0
Real time26m (1570s)
CPU time30m (1826s = 116%)
Outputting started2024-11-23 00:26:24
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/ismerio/04213/1/Validation_ccnc_atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74525168_357_20231206T014538Z_gen_g4_detsim_hitreco__20240510T062709Z_reco2.root
Finished2024-11-23 00:26:43
Saved logsjustin-logs:108609.32-justin-prod-sched02.dune.hep.ac.uk.logs.tgz
List job events     Wrapper job log

Jobscript log (last 10,000 characters)

00:21:48 UTC run: 74525168 subRun: 1 event: 35800
1st wire C:0 T:18 P:2 W:606 does not exist (max wire number: 480)
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrajFitter:pmtrajfittc@BeginModule  23-Nov-2024 00:21:48 UTC run: 74525168 subRun: 1 event: 35800
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrajFitter:pmtrajfittc@BeginModule  23-Nov-2024 00:21:48 UTC run: 74525168 subRun: 1 event: 35800
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrajFitter:pmtrajfittc@BeginModule  23-Nov-2024 00:21:48 UTC run: 74525168 subRun: 1 event: 35800
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  EMShower:emshower@BeginModule  23-Nov-2024 00:21:49 UTC run: 74525168 subRun: 1 event: 35800
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  EMShower:emshower@BeginModule  23-Nov-2024 00:21:49 UTC run: 74525168 subRun: 1 event: 35800
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  EMShower:emshower@BeginModule  23-Nov-2024 00:21:49 UTC run: 74525168 subRun: 1 event: 35800
Comparing two wires in the same plane: return failure
%MSG
Boundary wire vector sizes: 161, 79, 93
minwire 0: 1847
minwire 1: 950
minwire 2: 1991
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 199
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 199
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 199
Classifier summary: 
Output 0: 0.0491719, 0.940774, 0.0100536, 
Output 1: 0.0297884, 0.416182, 0.472001, 0.0820283, 
Output 2: 0.93699, 0.0630095, 

527
***Wiener Filter*******ApplyPostfilter***
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 1730!
23-Nov-2024 00:21:51 UTC  Closed output file "atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74525168_357_20231206T014538Z_gen_g4_detsim_hitreco__20240510T062709Z_reco2_reco_data_2024-11-22T_235545Z.root"
23-Nov-2024 00:21:51 UTC  Closed input file "root://mover.pp.rl.ac.uk:1094/pnfs/pp.rl.ac.uk/data/dune/fardet-hd/64/59/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74525168_357_20231206T014538Z_gen_g4_detsim_hitreco__20240510T062709Z_reco2.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                             0.106742       15.1805       365.658        5.604        42.9208        100    
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.00138348      0.0117       0.0433613     0.0041925     0.0106928       100    
reco:linecluster:LineCluster                          0.00141209     0.322977       27.1642      0.0137661      2.70696        100    
reco:trajcluster:TrajCluster                          0.000815217    0.531442       38.8219     0.00765914      3.91007        100    
reco:pandora:StandardPandora                           0.0117753      1.52162       27.9363       1.00619       3.17674        100    
reco:pandoraTrack:LArPandoraTrackCreation             0.000104798   0.00386798     0.0939899     0.0010548     0.0119442       100    
reco:pandoraShower:LArPandoraModularShowerCreation    0.000137007   0.00646812     0.141233      0.0016481     0.0165444       100    
reco:pandoracalo:Calorimetry                          8.1743e-05     0.0146967     0.763392     0.00062146     0.0844371       100    
reco:pandorapid:Chi2ParticleID                        3.2259e-05    0.000121452   0.00129654    8.3886e-05    0.000145775      100    
reco:trkshowersplit:TrackShowerHits                   5.4752e-05     0.0142003      0.70859     0.000476823    0.0766757       100    
reco:pmtrack:PMAlgTrackMaker                          0.000893675     1.42349       59.3355       0.11163       6.63525        100    
reco:pmtrackcalo:Calorimetry                          5.0917e-05    0.00585783     0.200712     0.000637745    0.0272852       100    
reco:pmtrackpid:Chi2ParticleID                        1.8999e-05    0.000183458   0.00214689    0.000111097   0.000300095      100    
reco:pmtrajfit:PMAlgTrajFitter                        0.000122054    0.935465       19.7269      0.140229       3.08094        100    
reco:pmtrajfitcalo:Calorimetry                        3.9018e-05     0.0136299     0.705978     0.000976219    0.0766778       100    
reco:pmtrajfitpid:Chi2ParticleID                      1.8147e-05    0.000136347   0.000792162   0.00011603    0.00011274       100    
reco:pmtracktc:PMAlgTrackMaker                        0.000753674     1.63922       73.9121       0.13494       7.98215        100    
reco:pmtracktccalo:Calorimetry                        4.4387e-05    0.00332137     0.0930716    0.000749323    0.0113346       100    
reco:pmtracktcpid:Chi2ParticleID                      1.8818e-05    0.000167698   0.00121618    0.000119485   0.000199023      100    
reco:pmtrajfittc:PMAlgTrajFitter                      0.00011317     0.805457       17.5683      0.157582       2.37854        100    
reco:pmtrajfittccalo:Calorimetry                      3.9159e-05    0.00813349     0.259432     0.000922278    0.0344344       100    
reco:pmtrajfittcpid:Chi2ParticleID                    1.6896e-05    0.000226413   0.00222799    0.000140186   0.000369413      100    
reco:blurredcluster:BlurredClustering                 0.000134973    0.745667       5.0892       0.376637       1.03875        100    
reco:emtrkmichelid:EmTrackMichelId                    7.7096e-05      5.4562        89.2819       1.55262       13.2979        100    
reco:emshower:EMShower                                0.00010581     0.0179556     0.242616     0.00302643      0.03562        100    
reco:cvnmap:CVNMapper                                 1.2159e-05    0.00578562     0.0416059    0.00384627    0.00683101       100    
reco:cvneva:CVNEvaluator                              1.2279e-05     0.424346      0.988573      0.548304      0.305283        100    
reco:energyrecnumu:EnergyReco                         0.000306922    0.0186832     0.690019      0.0046356     0.0748089       100    
reco:energyrecnue:EnergyReco                          0.000110467   0.00148614     0.0299322    0.000473187   0.00408254       100    
reco:energyrecnc:EnergyReco                           0.00115947     0.0028623     0.0337222    0.00197249    0.00434342       100    
reco:energyrecnumurange:EnergyReco                    0.000141402    0.0168743     0.781327     0.00143417     0.083274        100    
reco:energyrecnumumcs:EnergyReco                      0.00116611     0.0164621     0.643359     0.00225644     0.070657        100    
reco:opdec:Deconvolution                              0.00858961     0.108209      0.279795      0.106095      0.0530864       100    
reco:ophitspe:OpHitFinderDeco                          0.051774      0.942918       2.43263      0.889544      0.526203        100    
reco:opflash:OpFlashFinder                            0.000182735   0.00102582    0.00322666    0.000971361   0.000525246      100    
reco:rns:RandomNumberSaver                            1.6715e-05    2.67627e-05   0.000216446   2.18075e-05   2.06557e-05      100    
reco:anglereconue:NuAngularReco                       0.000130095   0.000980993    0.0214818    0.000291163   0.00283056       100    
reco:anglereconumu:NuAngularReco                       4.66e-05     0.000811847    0.0213462    0.000143906   0.00279364       100    
reco:anglereconuepfps:NuAngularReco                   9.0066e-05    0.000925185    0.0215295    0.000236451   0.00281798       100    
reco:anglereconumupfps:NuAngularReco                  6.1823e-05    0.000861945    0.0213701    0.000192204   0.00280776       100    
[art]:TriggerResults:TriggerResultInserter             7.521e-06    1.05664e-05   3.3791e-05    1.0065e-05    3.71134e-06      100    
end_path:out1:RootOutput                               2.494e-06    3.98198e-06   1.5073e-05     3.36e-06     1.60523e-06      100    
end_path:out1:RootOutput(write)                        0.0128723     0.154592      0.656796      0.160863      0.115814        100    
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 40372.6 MB
  Peak resident set size usage (VmHWM): 2557.91 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
Validation_ccnc_atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74525168_357_20231206T014538Z_gen_g4_detsim_hitreco__20240510T062709Z_reco2.root
all-input-dids.txt
atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74525168_357_20231206T014538Z_gen_g4_detsim_hitreco__20240510T062709Z_reco2_reco_2024-11-22T_235545Z.log
atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74525168_357_20231206T014538Z_gen_g4_detsim_hitreco__20240510T062709Z_reco2_reco_data_2024-11-22T_235545Z.root
debugprod.log
jobscript.log
justin-processed-pfns.txt
reco2_hist.root
MyPandoraSettings_Master_Atmos_DUNEFD.xml
MyPandoraSettings_Master_DUNEFD.xml
build_slf7.x86_64
localProducts_larsoft_v09_91_02_e26_prof
setup_env-testreco.sh
srcs
temp.txt
temp2.txt
test
work
justIN time: 2024-11-24 03:37:54 UTC       justIN version: 01.01.09