justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 301751.74@justin-prod-sched01.dune.hep.ac.uk

Jobsub ID301751.74@justin-prod-sched01.dune.hep.ac.uk
Workflow ID4213
Stage ID1
User nameismerio@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2024-11-23 05:14:46
SiteUK_Lancaster
EntryUBoone_UK_Lancaster_HEC_grendel_ce02
Last heartbeat2024-11-23 06:25:27
From worker nodeHostnamecomp08-16
cpuinfoIntel(R) Xeon(R) Gold 6248 CPU @ 2.50GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit257400 (71 hours)
Inner Apptainer?True
Job stateoutputting_failed
Allocator namejustin-allocator-pro.dune.hep.ac.uk
Started2024-11-23 05:16:18
Input filesfardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50542214_754_20231201T224522Z_gen_g4_detsim_hitreco__20240507T205316Z_reco2.root
JobscriptExit code0
Real time1h (4133s)
CPU time1h (3764s = 91%)
Outputting started2024-11-23 06:25:12
Output files
Finished2024-11-23 06:25:27
List job events     Wrapper job log

Jobscript log (last 10,000 characters)

: 75500
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrajFitter:pmtrajfittc@BeginModule  23-Nov-2024 06:25:02 GMT run: 50542214 subRun: 1 event: 75500
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrajFitter:pmtrajfittc@BeginModule  23-Nov-2024 06:25:02 GMT run: 50542214 subRun: 1 event: 75500
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrajFitter:pmtrajfittc@BeginModule  23-Nov-2024 06:25:02 GMT run: 50542214 subRun: 1 event: 75500
1st wire C:0 T:7 P:2 W:591 does not exist (max wire number: 480)
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrajFitter:pmtrajfittc@BeginModule  23-Nov-2024 06:25:02 GMT run: 50542214 subRun: 1 event: 75500
1st wire C:0 T:7 P:2 W:689 does not exist (max wire number: 480)
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrajFitter:pmtrajfittc@BeginModule  23-Nov-2024 06:25:03 GMT run: 50542214 subRun: 1 event: 75500
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrajFitter:pmtrajfittc@BeginModule  23-Nov-2024 06:25:03 GMT run: 50542214 subRun: 1 event: 75500
1st wire C:0 T:7 P:2 W:497 does not exist (max wire number: 480)
%MSG
Boundary wire vector sizes: 131, 93, 65
minwire 0: 746
minwire 1: 2305
minwire 2: 173
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 199
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 199
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 199
Classifier summary: 
Output 0: 0.196573, 0.00194819, 0.801478, 
Output 1: 0.992382, 0.00746816, 0.000129889, 2.03446e-05, 
Output 2: 0.855707, 0.144293, 

472
***Wiener Filter*******ApplyPostfilter***
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 1632!
23-Nov-2024 06:25:10 GMT  Closed output file "atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50542214_754_20231201T224522Z_gen_g4_detsim_hitreco__20240507T205316Z_reco2_reco_data_2024-11-23T_051621Z.root"
23-Nov-2024 06:25:10 GMT  Closed input file "root://xrootd1.esc.qmul.ac.uk:1094//dune/RSE/fardet-hd/c2/45/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50542214_754_20231201T224522Z_gen_g4_detsim_hitreco__20240507T205316Z_reco2.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                             0.401107       37.6916       912.164       10.6455       103.26         100    
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                 0.011175      0.0497337     0.417128      0.0264378     0.0731464       100    
reco:linecluster:LineCluster                           0.0650429      1.30063       89.1868      0.159927       8.85459        100    
reco:trajcluster:TrajCluster                           0.0289545      1.80894       105.981      0.0469135      10.7972        100    
reco:pandora:StandardPandora                           0.0558595      4.13591       171.31        1.15772       18.7451        100    
reco:pandoraTrack:LArPandoraTrackCreation             0.000141636   0.00765261      0.15963     0.00213429     0.0196853       100    
reco:pandoraShower:LArPandoraModularShowerCreation    0.000192527   0.00973826     0.0839812    0.00278003     0.0167759       100    
reco:pandoracalo:Calorimetry                          0.000104615    0.0396367      2.37365     0.000967637    0.240503        100    
reco:pandorapid:Chi2ParticleID                        3.7906e-05    0.000165681   0.00225078    0.000113323   0.000229947      100    
reco:trkshowersplit:TrackShowerHits                   9.9274e-05      0.02925       1.27347     0.000723867    0.135234        100    
reco:pmtrack:PMAlgTrackMaker                          0.00134324      3.10248       128.719       0.18855       13.5501        100    
reco:pmtrackcalo:Calorimetry                          8.8863e-05     0.0117859      0.33276     0.000904638    0.0422367       100    
reco:pmtrackpid:Chi2ParticleID                        3.3518e-05    0.000280958   0.00257749    0.000140242   0.000419601      100    
reco:pmtrajfit:PMAlgTrajFitter                        0.000200011     2.63491       90.9947      0.251331       9.83985        100    
reco:pmtrajfitcalo:Calorimetry                        6.6704e-05     0.0397296      2.29676     0.00150093     0.233373        100    
reco:pmtrajfitpid:Chi2ParticleID                      2.7691e-05    0.000184507   0.000912965   0.00014896    0.00013952       100    
reco:pmtracktc:PMAlgTrackMaker                        0.00115738      3.12056       86.858       0.229018       10.3063        100    
reco:pmtracktccalo:Calorimetry                        7.6429e-05    0.00800804     0.221906     0.00106059     0.0253111       100    
reco:pmtracktcpid:Chi2ParticleID                      2.9421e-05    0.000265649   0.00247465    0.000149928   0.000360155      100    
reco:pmtrajfittc:PMAlgTrajFitter                      0.000186862     1.57673       29.5646      0.290743       4.01471        100    
reco:pmtrajfittccalo:Calorimetry                      6.6247e-05     0.0220265     0.662301     0.00123258     0.0829979       100    
reco:pmtrajfittcpid:Chi2ParticleID                    2.7073e-05    0.000367437   0.00371136    0.000158334   0.000580354      100    
reco:blurredcluster:BlurredClustering                 6.6368e-05      1.15029       8.66854      0.476767       1.71324        100    
reco:emtrkmichelid:EmTrackMichelId                    0.000110711     15.2906       188.513       3.67995       30.4174        100    
reco:emshower:EMShower                                0.000162837    0.0298036     0.205478     0.00711961     0.0431692       100    
reco:cvnmap:CVNMapper                                 1.8707e-05     0.014138      0.0462062     0.0136172     0.0101897       100    
reco:cvneva:CVNEvaluator                               1.64e-05      0.600966       2.45283      0.772944      0.385728        100    
reco:energyrecnumu:EnergyReco                         0.000348555    0.0359647     0.481121     0.00665297     0.0885945       100    
reco:energyrecnue:EnergyReco                          0.000148934   0.00246843     0.0605291    0.000616818   0.00678324       100    
reco:energyrecnc:EnergyReco                           0.00168257    0.00392752     0.0570871    0.00204768    0.00642714       100    
reco:energyrecnumurange:EnergyReco                    0.000273233    0.0327469     0.485858     0.00190802     0.0885321       100    
reco:energyrecnumumcs:EnergyReco                      0.00167474     0.0337551     0.482663     0.00291509     0.0883456       100    
reco:opdec:Deconvolution                               0.0602131     0.341766       1.53761      0.260819      0.253632        100    
reco:ophitspe:OpHitFinderDeco                          0.0872318      2.02756       3.83745       1.98212       0.82397        100    
reco:opflash:OpFlashFinder                            0.000297339   0.00137786    0.00600489    0.00134279    0.000686519      100    
reco:rns:RandomNumberSaver                            1.8813e-05    3.2105e-05    0.000313607   2.6713e-05    2.93603e-05      100    
reco:anglereconue:NuAngularReco                       0.000156748   0.00192985     0.0530996    0.000362343   0.00584012       100    
reco:anglereconumu:NuAngularReco                      7.0525e-05    0.00166249     0.0489953    0.000206721    0.0053963       100    
reco:anglereconuepfps:NuAngularReco                   0.000113986   0.00178405     0.0521696    0.000309864   0.00567231       100    
reco:anglereconumupfps:NuAngularReco                  9.7789e-05    0.00174342     0.0499438    0.000288022   0.00547457       100    
[art]:TriggerResults:TriggerResultInserter             8.655e-06    1.25154e-05   7.4054e-05    1.0728e-05    6.71177e-06      100    
end_path:out1:RootOutput                               2.31e-06     4.4817e-06    2.1638e-05    3.3925e-06    2.45624e-06      100    
end_path:out1:RootOutput(write)                        0.0117151     0.218298      0.938363      0.202557      0.157002        100    
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 9635.7 MB
  Peak resident set size usage (VmHWM): 2530.57 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
Validation_ccnc_atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50542214_754_20231201T224522Z_gen_g4_detsim_hitreco__20240507T205316Z_reco2.root
all-input-dids.txt
atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50542214_754_20231201T224522Z_gen_g4_detsim_hitreco__20240507T205316Z_reco2_reco_2024-11-23T_051621Z.log
atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50542214_754_20231201T224522Z_gen_g4_detsim_hitreco__20240507T205316Z_reco2_reco_data_2024-11-23T_051621Z.root
debugprod.log
jobscript.log
justin-processed-pfns.txt
reco2_hist.root
MyPandoraSettings_Master_Atmos_DUNEFD.xml
MyPandoraSettings_Master_DUNEFD.xml
build_slf7.x86_64
localProducts_larsoft_v09_91_02_e26_prof
setup_env-testreco.sh
srcs
temp.txt
temp2.txt
test
work
justIN time: 2024-11-23 12:08:18 UTC       justIN version: 01.01.09