justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 301492.102@justin-prod-sched01.dune.hep.ac.uk

Jobsub ID301492.102@justin-prod-sched01.dune.hep.ac.uk
Workflow ID4213
Stage ID1
User nameismerio@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2024-11-22 23:11:25
SiteUK_RAL-Tier1
EntryLIGO_UK_RAL_arc_ce02
Last heartbeat2024-11-23 00:07:25
From worker nodeHostnamedune001-3354763.0-lcg2598.gridpp.rl.ac.uk
cpuinfoAMD EPYC 7763 64-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit216000 (60 hours)
Inner Apptainer?True
Job statefinished
Allocator namejustin-allocator-pro.dune.hep.ac.uk
Started2024-11-22 23:27:44
Input filesfardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50574832_830_20231203T133317Z_gen_g4_detsim_hitreco__20240508T052758Z_reco2.root
JobscriptExit code0
Real time35m (2134s)
CPU time55m (3329s = 155%)
Outputting started2024-11-23 00:03:18
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/ismerio/04213/1/Validation_ccnc_atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50574832_830_20231203T133317Z_gen_g4_detsim_hitreco__20240508T052758Z_reco2.root
Finished2024-11-23 00:07:25
Saved logsjustin-logs:301492.102-justin-prod-sched01.dune.hep.ac.uk.logs.tgz
List job events     Wrapper job log

Jobscript log (last 10,000 characters)

eginModule  23-Nov-2024 00:02:57 UTC run: 50574832 subRun: 1 event: 83100
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrajFitter:pmtrajfittc@BeginModule  23-Nov-2024 00:02:57 UTC run: 50574832 subRun: 1 event: 83100
Comparing two wires in the same plane: return failure
%MSG
%MSG-e pma::Track3D:  PMAlgTrajFitter:pmtrajfittc@BeginModule  23-Nov-2024 00:02:57 UTC run: 50574832 subRun: 1 event: 83100
0 enabled hits in AverageDist2 calculation.
%MSG
%MSG-e pma::Track3D:  PMAlgTrajFitter:pmtrajfittc@BeginModule  23-Nov-2024 00:02:57 UTC run: 50574832 subRun: 1 event: 83100
Track empty.
%MSG
%MSG-e pma::Track3D:  PMAlgTrajFitter:pmtrajfittc@BeginModule  23-Nov-2024 00:02:57 UTC run: 50574832 subRun: 1 event: 83100
TuneFullTree failed.
%MSG
%MSG-e pma::Track3D:  PMAlgTrajFitter:pmtrajfittc@BeginModule  23-Nov-2024 00:02:57 UTC run: 50574832 subRun: 1 event: 83100
0 enabled hits in AverageDist2 calculation.
%MSG
%MSG-e pma::Track3D:  PMAlgTrajFitter:pmtrajfittc@BeginModule  23-Nov-2024 00:02:57 UTC run: 50574832 subRun: 1 event: 83100
Track empty.
%MSG
%MSG-e pma::Track3D:  PMAlgTrajFitter:pmtrajfittc@BeginModule  23-Nov-2024 00:02:57 UTC run: 50574832 subRun: 1 event: 83100
TuneFullTree failed.
%MSG
Boundary wire vector sizes: 806, 708, 486
minwire 0: 1665
minwire 1: 109
minwire 2: 1188
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 199
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 199
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 199
Classifier summary: 
Output 0: 0.043201, 0.0225483, 0.934251, 
Output 1: 0.0897776, 0.346251, 0.314672, 0.249299, 
Output 2: 0.0488028, 0.951197, 

585
***Wiener Filter*******ApplyPostfilter***
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 2021!
23-Nov-2024 00:03:17 UTC  Closed output file "atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50574832_830_20231203T133317Z_gen_g4_detsim_hitreco__20240508T052758Z_reco2_reco_data_2024-11-22T_232805Z.root"
23-Nov-2024 00:03:17 UTC  Closed input file "root://xrootd.echo.stfc.ac.uk:1094/dune:/protodune/RSE/fardet-hd/12/f4/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50574832_830_20231203T133317Z_gen_g4_detsim_hitreco__20240508T052758Z_reco2.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                             0.949043       19.7411       228.311       10.8058       28.5711        100    
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.00167563     0.0892651     0.331803      0.0748873     0.0587792       100    
reco:linecluster:LineCluster                           0.0608669     0.300148      0.857249      0.257542      0.172271        100    
reco:trajcluster:TrajCluster                          0.00112036     0.309069       8.23468      0.116299      0.878561        100    
reco:pandora:StandardPandora                           0.174079       1.72698       13.0284       1.43174       1.43308        100    
reco:pandoraTrack:LArPandoraTrackCreation             0.000125542   0.00477826     0.0753516    0.00161898     0.0107825       100    
reco:pandoraShower:LArPandoraModularShowerCreation    0.000159493   0.00838002     0.100937     0.00316193     0.0160206       100    
reco:pandoracalo:Calorimetry                          9.4212e-05    0.00790462     0.288874     0.00117539     0.0299614       100    
reco:pandorapid:Chi2ParticleID                        3.8541e-05    0.000188553   0.00238961    0.000141063   0.00025335       100    
reco:trkshowersplit:TrackShowerHits                   8.9042e-05     0.0106927     0.296865     0.000770477    0.0361563       100    
reco:pmtrack:PMAlgTrackMaker                          0.00104241      1.41381       27.6758      0.256303       3.5051         100    
reco:pmtrackcalo:Calorimetry                          6.7881e-05    0.00505482     0.127439     0.00111702     0.0148739       100    
reco:pmtrackpid:Chi2ParticleID                        2.6181e-05    0.000269979   0.00168225    0.000178174   0.000288654      100    
reco:pmtrajfit:PMAlgTrajFitter                        0.000161353     1.14883       11.6905      0.320502       1.86367        100    
reco:pmtrajfitcalo:Calorimetry                        6.3681e-05    0.00729381     0.180128     0.00165107     0.0197365       100    
reco:pmtrajfitpid:Chi2ParticleID                      2.5051e-05    0.000208891   0.000935559   0.000182284   0.000141511      100    
reco:pmtracktc:PMAlgTrackMaker                        0.000839378     1.54059       36.7405      0.263304       4.22389        100    
reco:pmtracktccalo:Calorimetry                        6.3041e-05     0.0041683     0.0862586    0.00116031     0.0106111       100    
reco:pmtracktcpid:Chi2ParticleID                       2.585e-05    0.000270026     0.00195     0.000184323   0.000290817      100    
reco:pmtrajfittc:PMAlgTrajFitter                      0.000141774     1.03729       17.2666      0.352152       2.10524        100    
reco:pmtrajfittccalo:Calorimetry                      6.1411e-05    0.00649752     0.154435     0.00123402     0.0183076       100    
reco:pmtrajfittcpid:Chi2ParticleID                     2.376e-05    0.000314155   0.00207338    0.000187529   0.000346154      100    
reco:blurredcluster:BlurredClustering                 0.000198985    0.958177       6.2986       0.476519       1.20727        100    
reco:emtrkmichelid:EmTrackMichelId                    0.000144783     7.43597       103.088       2.89836       13.4802        100    
reco:emshower:EMShower                                0.000222154    0.0285595     0.194553     0.00726793     0.0426406       100    
reco:cvnmap:CVNMapper                                  2.422e-05    0.00841345     0.041097     0.00624992    0.00772045       100    
reco:cvneva:CVNEvaluator                              1.6411e-05     0.893827       1.42762       1.05604      0.489728        100    
reco:energyrecnumu:EnergyReco                         0.000340667    0.0168227     0.327976     0.00548604     0.0417516       100    
reco:energyrecnue:EnergyReco                          0.000158813    0.0015833     0.0231573    0.000465515   0.00327953       100    
reco:energyrecnc:EnergyReco                           0.00172144    0.00402048     0.0235144    0.00338556    0.00336463       100    
reco:energyrecnumurange:EnergyReco                    0.000182544    0.0137549     0.219928     0.00264299     0.0349169       100    
reco:energyrecnumumcs:EnergyReco                      0.00124074     0.0147832     0.199978     0.00361877     0.0324066       100    
reco:opdec:Deconvolution                               0.0651815     0.385469       1.61415      0.337548      0.230378        100    
reco:ophitspe:OpHitFinderDeco                          0.169104       2.10145       4.16391       2.0404        0.88187        100    
reco:opflash:OpFlashFinder                            0.000247306   0.00142081    0.00494573    0.00135658    0.000656646      100    
reco:rns:RandomNumberSaver                            1.7881e-05    4.18482e-05   0.000358638   3.31505e-05   4.45384e-05      100    
reco:anglereconue:NuAngularReco                       0.000165763   0.00115379     0.0184268    0.000380958   0.00249224       100    
reco:anglereconumu:NuAngularReco                      8.2322e-05    0.000976061    0.0201232    0.000243635   0.00255308       100    
reco:anglereconuepfps:NuAngularReco                   0.000127723   0.00108937     0.0191325    0.000356837   0.00247205       100    
reco:anglereconumupfps:NuAngularReco                  0.000116632   0.00101892     0.0183375    0.000306032   0.00241644       100    
[art]:TriggerResults:TriggerResultInserter             9.051e-06    1.62539e-05   7.0942e-05     1.428e-05    7.66537e-06      100    
end_path:out1:RootOutput                               2.94e-06     4.65293e-06    2.252e-05     3.935e-06    2.40597e-06      100    
end_path:out1:RootOutput(write)                        0.0224659     0.248417      0.734429      0.263913      0.127323        100    
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 39254.6 MB
  Peak resident set size usage (VmHWM): 2082.7 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
Validation_ccnc_atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50574832_830_20231203T133317Z_gen_g4_detsim_hitreco__20240508T052758Z_reco2.root
all-input-dids.txt
atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50574832_830_20231203T133317Z_gen_g4_detsim_hitreco__20240508T052758Z_reco2_reco_2024-11-22T_232805Z.log
atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50574832_830_20231203T133317Z_gen_g4_detsim_hitreco__20240508T052758Z_reco2_reco_data_2024-11-22T_232805Z.root
debugprod.log
jobscript.log
justin-processed-pfns.txt
reco2_hist.root
MyPandoraSettings_Master_Atmos_DUNEFD.xml
MyPandoraSettings_Master_DUNEFD.xml
build_slf7.x86_64
localProducts_larsoft_v09_91_02_e26_prof
setup_env-testreco.sh
srcs
temp.txt
temp2.txt
test
work
justIN time: 2024-11-24 16:05:40 UTC       justIN version: 01.01.09