Jobsub ID 301437.82@justin-prod-sched01.dune.hep.ac.uk

Jobsub ID	301437.82@justin-prod-sched01.dune.hep.ac.uk
Workflow ID	4213
Stage ID	1
User name	ismerio@fnal.gov
HTCondor Group	group_dune
Requested	Processors	1
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	80000 (22 hours)
Submitted time	2024-11-22 22:04:16
Site	UK_RAL-Tier1
Entry	LIGO_UK_RAL_arc_ce04
Last heartbeat	2024-11-22 23:28:23
From worker node	Hostname	dune001-2993327.0-lcg2699.gridpp.rl.ac.uk
	cpuinfo	AMD EPYC 9654 96-Core Processor
	OS release	Scientific Linux release 7.9 (Nitrogen)
	Processors	1
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	216000 (60 hours)
	Inner Apptainer?	True
Job state	aborted
Allocator name	justin-allocator-pro.dune.hep.ac.uk
Started	2024-11-22 22:18:02
Input files	fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74552621_510_20231207T124655Z_gen_g4_detsim_hitreco__20240510T061623Z_reco2.root
Outputting started	2024-11-22 23:22:43
Output files
Finished	2024-11-22 23:28:23
List job events Wrapper job log
Jobscript log (last 10,000 characters)

1 event: 51100
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrajFitter:pmtrajfittc@BeginModule  22-Nov-2024 23:22:35 UTC run: 74552621 subRun: 1 event: 51100
1st wire C:0 T:2 P:2 W:521 does not exist (max wire number: 480)
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrajFitter:pmtrajfittc@BeginModule  22-Nov-2024 23:22:35 UTC run: 74552621 subRun: 1 event: 51100
1st wire C:0 T:2 P:2 W:561 does not exist (max wire number: 480)
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrajFitter:pmtrajfittc@BeginModule  22-Nov-2024 23:22:35 UTC run: 74552621 subRun: 1 event: 51100
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrajFitter:pmtrajfittc@BeginModule  22-Nov-2024 23:22:35 UTC run: 74552621 subRun: 1 event: 51100
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrajFitter:pmtrajfittc@BeginModule  22-Nov-2024 23:22:35 UTC run: 74552621 subRun: 1 event: 51100
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  EMShower:emshower@BeginModule  22-Nov-2024 23:22:39 UTC run: 74552621 subRun: 1 event: 51100
Comparing two wires in the same plane: return failure
%MSG
Boundary wire vector sizes: 130, 146, 128
minwire 0: 367
minwire 1: 2252
minwire 2: 193
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 199
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 199
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 199
Classifier summary: 
Output 0: 0.0377564, 0.960726, 0.00151749, 
Output 1: 0.0023275, 0.997142, 0.000527803, 2.27263e-06, 
Output 2: 0.996901, 0.00309906, 

411
***Wiener Filter*******ApplyPostfilter***
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 1236!
22-Nov-2024 23:22:42 UTC  Closed output file "atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74552621_510_20231207T124655Z_gen_g4_detsim_hitreco__20240510T061623Z_reco2_reco_data_2024-11-22T_221806Z.root"
22-Nov-2024 23:22:42 UTC  Closed input file "root://xrootd.echo.stfc.ac.uk:1094/dune:/protodune/RSE/fardet-hd/64/96/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74552621_510_20231207T124655Z_gen_g4_detsim_hitreco__20240510T061623Z_reco2.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              1.73862       37.3819       577.405       7.67431       95.1024        100    
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.00140025     0.0871748     0.615361      0.0729094     0.0787496       100    
reco:linecluster:LineCluster                           0.0388219      1.42631       40.5307      0.220267       5.74268        100    
reco:trajcluster:TrajCluster                          0.00722676      2.08016       46.6518      0.0964714      7.44421        100    
reco:pandora:StandardPandora                           0.875081       3.41485       70.3338       1.19892       8.82105        100    
reco:pandoraTrack:LArPandoraTrackCreation             0.000140401   0.00868086     0.130021     0.00149033     0.0214568       100    
reco:pandoraShower:LArPandoraModularShowerCreation    0.000184738    0.0113637     0.223886     0.00301604     0.0277164       100    
reco:pandoracalo:Calorimetry                          0.000114082    0.043125      0.890432     0.000829083    0.145667        100    
reco:pandorapid:Chi2ParticleID                        4.1383e-05    0.000184814   0.00193609    0.000118729   0.000226664      100    
reco:trkshowersplit:TrackShowerHits                   0.000136015    0.0531597     0.975802     0.000652502    0.171675        100    
reco:pmtrack:PMAlgTrackMaker                          0.00116472      4.10923       87.9267      0.141302       13.2066        100    
reco:pmtrackcalo:Calorimetry                          5.5874e-05     0.0167359     0.454721     0.000848146    0.0588007       100    
reco:pmtrackpid:Chi2ParticleID                        2.0822e-05    0.000323118   0.00367709    0.000137051   0.000557081      100    
reco:pmtrajfit:PMAlgTrajFitter                        0.000174132     4.07515       77.3407      0.203908       13.5287        100    
reco:pmtrajfitcalo:Calorimetry                        4.4988e-05     0.0440091      1.17554     0.00141993     0.166562        100    
reco:pmtrajfitpid:Chi2ParticleID                      1.9139e-05    0.000186266   0.00126601    0.000141733   0.000178122      100    
reco:pmtracktc:PMAlgTrackMaker                        0.000893054     4.89121       116.315      0.179584       16.4077        100    
reco:pmtracktccalo:Calorimetry                        4.6039e-05    0.00968239     0.239482     0.000958513    0.0304262       100    
reco:pmtracktcpid:Chi2ParticleID                       2.028e-05    0.00027869    0.00277778    0.00014968    0.000408677      100    
reco:pmtrajfittc:PMAlgTrajFitter                       0.0001459      2.11626       28.9683       0.22288       5.88515        100    
reco:pmtrajfittccalo:Calorimetry                      4.2845e-05     0.0266751     0.496723     0.00108729     0.0867924       100    
reco:pmtrajfittcpid:Chi2ParticleID                    1.7456e-05    0.000406665   0.00353925    0.000151913   0.000722686      100    
reco:blurredcluster:BlurredClustering                 0.00410903     0.978648       7.95482      0.374702       1.41989        100    
reco:emtrkmichelid:EmTrackMichelId                     0.181677       11.3124       121.991       2.21169       25.0215        100    
reco:emshower:EMShower                                0.000266932    0.0296195     0.322945     0.00384006     0.0561082       100    
reco:cvnmap:CVNMapper                                 2.1152e-05    0.00618895     0.0351332    0.00416766    0.00736948       100    
reco:cvneva:CVNEvaluator                              1.5543e-05     0.537469      0.996082      0.666575      0.352272        100    
reco:energyrecnumu:EnergyReco                         0.000377488    0.0435621     0.882303     0.00755774     0.142182        100    
reco:energyrecnue:EnergyReco                          0.00012585    0.00285787     0.0430779    0.000454203   0.00748807       100    
reco:energyrecnc:EnergyReco                            0.0014198    0.00466223     0.0438189    0.00243181    0.00760529       100    
reco:energyrecnumurange:EnergyReco                    0.000175333    0.0383786     0.867089     0.00234817     0.139679        100    
reco:energyrecnumumcs:EnergyReco                      0.000975528    0.0395343     0.871325     0.00265818     0.139187        100    
reco:opdec:Deconvolution                               0.0452199     0.261513      0.905032      0.231224      0.146607        100    
reco:ophitspe:OpHitFinderDeco                          0.0696274      1.47715       3.25737       1.3763        0.68434        100    
reco:opflash:OpFlashFinder                            0.000230657   0.00135166    0.00555735    0.00121483    0.000734435      100    
reco:rns:RandomNumberSaver                            2.0571e-05    3.37454e-05   0.00037226    2.7401e-05    3.50912e-05      100    
reco:anglereconue:NuAngularReco                       0.00014604    0.00259519     0.0523286    0.000388785    0.0075824       100    
reco:anglereconumu:NuAngularReco                      5.0986e-05    0.00232938     0.0520047    0.000200317   0.00742653       100    
reco:anglereconuepfps:NuAngularReco                   0.000101362   0.00243868     0.0529573    0.000314533   0.00751522       100    
reco:anglereconumupfps:NuAngularReco                  7.0947e-05    0.00232426     0.0502942    0.000241634   0.00740064       100    
[art]:TriggerResults:TriggerResultInserter             8.343e-06    1.27309e-05   6.2024e-05    1.18175e-05   5.61988e-06      100    
end_path:out1:RootOutput                               3.255e-06    4.44158e-06   2.4026e-05     4.011e-06    2.71242e-06      100    
end_path:out1:RootOutput(write)                        0.0146561     0.221728       1.08515      0.197599      0.195783        100    
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 40561 MB
  Peak resident set size usage (VmHWM): 2770.89 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
Validation_ccnc_atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74552621_510_20231207T124655Z_gen_g4_detsim_hitreco__20240510T061623Z_reco2.root
all-input-dids.txt
atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74552621_510_20231207T124655Z_gen_g4_detsim_hitreco__20240510T061623Z_reco2_reco_2024-11-22T_221806Z.log
atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74552621_510_20231207T124655Z_gen_g4_detsim_hitreco__20240510T061623Z_reco2_reco_data_2024-11-22T_221806Z.root
debugprod.log
jobscript.log
justin-processed-pfns.txt
reco2_hist.root
MyPandoraSettings_Master_Atmos_DUNEFD.xml
MyPandoraSettings_Master_DUNEFD.xml
build_slf7.x86_64
localProducts_larsoft_v09_91_02_e26_prof
setup_env-testreco.sh
srcs
temp.txt
temp2.txt
test
work