Jobsub ID 108827.79@justin-prod-sched02.dune.hep.ac.uk

Jobsub ID	108827.79@justin-prod-sched02.dune.hep.ac.uk
Workflow ID	4213
Stage ID	1
User name	ismerio@fnal.gov
HTCondor Group	group_dune
Requested	Processors	1
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	80000 (22 hours)
Submitted time	2024-11-23 05:25:34
Site	UK_Brunel
Entry	CMSHTPC_T2_UK_London_Brunel_dc2_22
Last heartbeat	2024-11-23 06:22:25
From worker node	Hostname	wn-b2-15-00.brunel.ac.uk
	cpuinfo	Intel(R) Xeon(R) CPU E5-2630 v3 @ 2.40GHz
	OS release	Scientific Linux release 7.9 (Nitrogen)
	Processors	1
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	171000 (47 hours)
	Inner Apptainer?	True
Job state	outputting_failed
Allocator name	justin-allocator-pro.dune.hep.ac.uk
Started	2024-11-23 05:27:25
Input files	fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74552621_91_20231207T121428Z_gen_g4_detsim_hitreco__20240510T063020Z_reco2.root
Jobscript	Exit code	0
	Real time	54m (3284s)
	CPU time	49m (2983s = 90%)
Outputting started	2024-11-23 06:22:10
Output files
Finished	2024-11-23 06:22:25
List job events Wrapper job log
Jobscript log (last 10,000 characters)

subRun: 1 event: 9200
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrajFitter:pmtrajfittc@BeginModule  23-Nov-2024 06:21:25 GMT run: 74552621 subRun: 1 event: 9200
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrajFitter:pmtrajfittc@BeginModule  23-Nov-2024 06:21:25 GMT run: 74552621 subRun: 1 event: 9200
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrajFitter:pmtrajfittc@BeginModule  23-Nov-2024 06:21:25 GMT run: 74552621 subRun: 1 event: 9200
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrajFitter:pmtrajfittc@BeginModule  23-Nov-2024 06:21:25 GMT run: 74552621 subRun: 1 event: 9200
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrajFitter:pmtrajfittc@BeginModule  23-Nov-2024 06:21:27 GMT run: 74552621 subRun: 1 event: 9200
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrajFitter:pmtrajfittc@BeginModule  23-Nov-2024 06:21:27 GMT run: 74552621 subRun: 1 event: 9200
Comparing two wires in the same plane: return failure
%MSG
Boundary wire vector sizes: 1259, 1366, 1511
minwire 0: 173
minwire 1: 1093
minwire 2: 70
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 199
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 199
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 199
Classifier summary: 
Output 0: 0.00271105, 0.00414077, 0.993148, 
Output 1: 0.373257, 0.421672, 0.121824, 0.0832464, 
Output 2: 0.771204, 0.228796, 

607
***Wiener Filter*******ApplyPostfilter***
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 2163!
23-Nov-2024 06:22:09 GMT  Closed output file "atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74552621_91_20231207T121428Z_gen_g4_detsim_hitreco__20240510T063020Z_reco2_reco_data_2024-11-23T_052728Z.root"
23-Nov-2024 06:22:09 GMT  Closed input file "root://golias100.farm.particle.cz:1094/dpm/farm.particle.cz/home/dune/RSE/fardet-hd/a5/95/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74552621_91_20231207T121428Z_gen_g4_detsim_hitreco__20240510T063020Z_reco2.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                             0.718814       28.903        364.943       11.9334       50.1294        100    
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                 0.0218925     0.0461356     0.106889      0.0485865     0.0156121       100    
reco:linecluster:LineCluster                           0.128098      0.410382       9.03161      0.234449      0.931415        100    
reco:trajcluster:TrajCluster                           0.0640455      0.83984       20.1477      0.0888032      2.79084        100    
reco:pandora:StandardPandora                           0.0739105      2.17624       39.1636       1.29367       4.57921        100    
reco:pandoraTrack:LArPandoraTrackCreation             0.000125155   0.00647059     0.0986641    0.00230156     0.0140311       100    
reco:pandoraShower:LArPandoraModularShowerCreation    0.000167887    0.0129916     0.0925866    0.00383534     0.0204485       100    
reco:pandoracalo:Calorimetry                          9.8907e-05     0.0240221     0.663885     0.00214372     0.0948339       100    
reco:pandorapid:Chi2ParticleID                        3.9691e-05    0.000143026   0.00114744     0.0001212    0.000125868      100    
reco:trkshowersplit:TrackShowerHits                    6.83e-05      0.0188876     0.473636     0.00119633     0.0655662       100    
reco:pmtrack:PMAlgTrackMaker                           0.0011459      2.12379       35.5243       0.39019       4.73176        100    
reco:pmtrackcalo:Calorimetry                           7.746e-05     0.0104153     0.272824     0.00135876     0.0321012       100    
reco:pmtrackpid:Chi2ParticleID                        3.2302e-05    0.000258946   0.00242536    0.000140313   0.000350406      100    
reco:pmtrajfit:PMAlgTrajFitter                        0.00019201      1.94926       31.272        0.5236        4.73529        100    
reco:pmtrajfitcalo:Calorimetry                        7.4158e-05     0.0242845      0.66672     0.00264869     0.0925238       100    
reco:pmtrajfitpid:Chi2ParticleID                       3.175e-05    0.00015601    0.000571119   0.000139518   0.000106089      100    
reco:pmtracktc:PMAlgTrackMaker                        0.00101439      2.5871        50.0855      0.389913       6.8315         100    
reco:pmtracktccalo:Calorimetry                        7.8153e-05     0.0086074      0.14292     0.00150234     0.0204909       100    
reco:pmtracktcpid:Chi2ParticleID                      3.2212e-05    0.000254129   0.00179016    0.000146128   0.000304543      100    
reco:pmtrajfittc:PMAlgTrajFitter                      0.000183835     1.71728       28.8151      0.434348       3.7144         100    
reco:pmtrajfittccalo:Calorimetry                       7.467e-05     0.0182906      0.55011     0.00161634     0.0664608       100    
reco:pmtrajfittcpid:Chi2ParticleID                    3.1348e-05    0.000340868   0.00344652    0.000163727   0.000510905      100    
reco:blurredcluster:BlurredClustering                 8.4453e-05      1.16581       6.87424      0.628568       1.36083        100    
reco:emtrkmichelid:EmTrackMichelId                    0.000139753     12.9841       139.427       4.82229       21.1471        100    
reco:emshower:EMShower                                0.000186061    0.0326416     0.233694     0.00684783     0.0490087       100    
reco:cvnmap:CVNMapper                                 2.4421e-05     0.0123183     0.0435126     0.0107495    0.00861726       100    
reco:cvneva:CVNEvaluator                              1.8074e-05     0.556862       1.53277      0.654597       0.28964        100    
reco:energyrecnumu:EnergyReco                         0.00026365     0.0489002      1.02378     0.00386901     0.140019        100    
reco:energyrecnue:EnergyReco                          0.000150057    0.0021538     0.028607     0.000965058   0.00412545       100    
reco:energyrecnc:EnergyReco                           0.00202827    0.00399162     0.0326988    0.00254468    0.00445535       100    
reco:energyrecnumurange:EnergyReco                    0.000154722    0.047218       1.01825     0.00229612     0.138822        100    
reco:energyrecnumumcs:EnergyReco                      0.000938625    0.0481967      1.01875      0.003961      0.138268        100    
reco:opdec:Deconvolution                               0.0870382     0.328608      0.534931      0.330972      0.0912315       100    
reco:ophitspe:OpHitFinderDeco                          0.192079       1.45854        2.429        1.51456      0.526136        100    
reco:opflash:OpFlashFinder                            0.000151988   0.00117517    0.00380013    0.00113827    0.000557777      100    
reco:rns:RandomNumberSaver                            1.5131e-05    2.37403e-05   0.000218085   2.02365e-05   2.02804e-05      100    
reco:anglereconue:NuAngularReco                       9.3837e-05    0.00141548     0.024175     0.000331462   0.00345157       100    
reco:anglereconumu:NuAngularReco                      7.8841e-05    0.00135249     0.0235985    0.000319443   0.00338427       100    
reco:anglereconuepfps:NuAngularReco                   0.000111537   0.00145422     0.0239837    0.000391592   0.00341048       100    
reco:anglereconumupfps:NuAngularReco                  0.000108234   0.00143141      0.02389     0.00039414    0.00341724       100    
[art]:TriggerResults:TriggerResultInserter             9.581e-06    1.16839e-05   3.6304e-05    1.0966e-05    3.19704e-06      100    
end_path:out1:RootOutput                               2.702e-06    4.05105e-06   1.1866e-05     3.46e-06     1.2852e-06       100    
end_path:out1:RootOutput(write)                        0.0105943     0.230321      0.794095      0.222239      0.133861        100    
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 8297.75 MB
  Peak resident set size usage (VmHWM): 2205.75 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
Validation_ccnc_atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74552621_91_20231207T121428Z_gen_g4_detsim_hitreco__20240510T063020Z_reco2.root
all-input-dids.txt
atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74552621_91_20231207T121428Z_gen_g4_detsim_hitreco__20240510T063020Z_reco2_reco_2024-11-23T_052728Z.log
atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74552621_91_20231207T121428Z_gen_g4_detsim_hitreco__20240510T063020Z_reco2_reco_data_2024-11-23T_052728Z.root
debugprod.log
jobscript.log
justin-processed-pfns.txt
reco2_hist.root
MyPandoraSettings_Master_Atmos_DUNEFD.xml
MyPandoraSettings_Master_DUNEFD.xml
build_slf7.x86_64
localProducts_larsoft_v09_91_02_e26_prof
setup_env-testreco.sh
srcs
temp.txt
temp2.txt
test
work