Jobsub ID 108744.157@justin-prod-sched02.dune.hep.ac.uk

Jobsub ID	108744.157@justin-prod-sched02.dune.hep.ac.uk
Workflow ID	4213
Stage ID	1
User name	ismerio@fnal.gov
HTCondor Group	group_dune
Requested	Processors	1
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	80000 (22 hours)
Submitted time	2024-11-23 04:38:45
Site	CERN
Entry	CMSHTPC_T2_CH_CERN_ce512
Last heartbeat	2024-11-23 06:18:22
From worker node	Hostname	b9p10p4272.cern.ch
	cpuinfo	AMD EPYC 7543 32-Core Processor
	OS release	Scientific Linux release 7.9 (Nitrogen)
	Processors	1
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	171000 (47 hours)
	Inner Apptainer?	True
Job state	outputting_failed
Allocator name	justin-allocator-pro.dune.hep.ac.uk
Started	2024-11-23 04:43:10
Input files	fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74494002_729_20231201T170150Z_gen_g4_detsim_hitreco__20240507T195919Z_reco2.root
Jobscript	Exit code	0
	Real time	1h (5693s)
	CPU time	1h (5622s = 98%)
Outputting started	2024-11-23 06:18:04
Output files
Finished	2024-11-23 06:18:22
List job events Wrapper job log
Jobscript log (last 10,000 characters)

-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  23-Nov-2024 07:17:56 CET run: 74494002 subRun: 1 event: 73000
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  23-Nov-2024 07:17:56 CET run: 74494002 subRun: 1 event: 73000
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  23-Nov-2024 07:17:56 CET run: 74494002 subRun: 1 event: 73000
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  23-Nov-2024 07:17:56 CET run: 74494002 subRun: 1 event: 73000
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrajFitter:pmtrajfittc@BeginModule  23-Nov-2024 07:17:56 CET run: 74494002 subRun: 1 event: 73000
Comparing two wires in the same plane: return failure
%MSG
Boundary wire vector sizes: 43, 43, 44
minwire 0: 2602
minwire 1: 647
minwire 2: 2474
Used alternate method to get min and max wires due to vertex determination failure: 0, 199
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 199
Used alternate method to get min and max wires due to vertex determination failure: 96, 295
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 199
Used alternate method to get min and max wires due to vertex determination failure: 12, 211
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 199
Classifier summary: 
Output 0: 0.989359, 0.000986313, 0.00965442, 
Output 1: 0.931145, 0.0637616, 0.00430631, 0.000787554, 
Output 2: 0.997454, 0.00254616, 

344
***Wiener Filter*******ApplyPostfilter***
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 995!
23-Nov-2024 07:18:02 CET  Closed output file "atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74494002_729_20231201T170150Z_gen_g4_detsim_hitreco__20240507T195919Z_reco2_reco_data_2024-11-23T_044314Z.root"
23-Nov-2024 07:18:02 CET  Closed input file "root://golias100.farm.particle.cz:1094/dpm/farm.particle.cz/home/dune/RSE/fardet-hd/8b/5b/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74494002_729_20231201T170150Z_gen_g4_detsim_hitreco__20240507T195919Z_reco2.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                             0.508479       54.6967       1214.9        9.68702       177.18         100    
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                 0.0170185     0.134608       8.93456      0.0427056     0.884917        100    
reco:linecluster:LineCluster                           0.0944712      1.5274        55.2095      0.174781       6.77545        100    
reco:trajcluster:TrajCluster                           0.0475481      3.6456        113.387      0.0608346      16.4696        100    
reco:pandora:StandardPandora                           0.0560115      8.5891        489.348       1.32464       50.6887        100    
reco:pandoraTrack:LArPandoraTrackCreation             0.000150892    0.0101879     0.230799     0.00172294     0.0305305       100    
reco:pandoraShower:LArPandoraModularShowerCreation    0.000206833    0.0142946     0.145675     0.00360305     0.0286307       100    
reco:pandoracalo:Calorimetry                          0.000120821    0.0609583      2.08874     0.00119841     0.278405        100    
reco:pandorapid:Chi2ParticleID                         4.664e-05    0.000209491   0.00224917    0.000155727   0.000253883      100    
reco:trkshowersplit:TrackShowerHits                   6.3421e-05     0.0840714      2.75121     0.000923856    0.355762        100    
reco:pmtrack:PMAlgTrackMaker                          0.000978031      5.868        137.241      0.203183       21.2445        100    
reco:pmtrackcalo:Calorimetry                           5.913e-05     0.0292881      1.08972     0.00107535     0.129073        100    
reco:pmtrackpid:Chi2ParticleID                         2.306e-05    0.000381315   0.00433677    0.000151286   0.000756335      100    
reco:pmtrajfit:PMAlgTrajFitter                        0.000223403     6.86128       320.421      0.291733       34.0971        100    
reco:pmtrajfitcalo:Calorimetry                        7.4471e-05     0.0618794      2.09232     0.00156419     0.274482        100    
reco:pmtrajfitpid:Chi2ParticleID                       3.109e-05    0.000225442   0.00161668    0.000178163   0.000235705      100    
reco:pmtracktc:PMAlgTrackMaker                        0.000977432     7.07451       135.589      0.202227       22.843         100    
reco:pmtracktccalo:Calorimetry                        6.8801e-05     0.0116501     0.266223     0.00110217     0.0409826       100    
reco:pmtracktcpid:Chi2ParticleID                      2.6931e-05    0.000309163   0.00337531    0.000160437   0.000499536      100    
reco:pmtrajfittc:PMAlgTrajFitter                      0.000194682     2.72308       57.7642      0.273141       8.21044        100    
reco:pmtrajfittccalo:Calorimetry                      6.2021e-05     0.0338586     0.750791     0.00128514     0.123143        100    
reco:pmtrajfittcpid:Chi2ParticleID                    2.5801e-05    0.000457366   0.00463875    0.000183347   0.000842212      100    
reco:blurredcluster:BlurredClustering                 0.000115421     1.25845       8.10364      0.573886       1.76288        100    
reco:emtrkmichelid:EmTrackMichelId                    0.000118421     13.273        169.791       2.45328       31.2457        100    
reco:emshower:EMShower                                0.000182913    0.0365488     0.432299     0.00523228     0.0703081       100    
reco:cvnmap:CVNMapper                                  2.048e-05     0.012398      0.0399047     0.0150134     0.0102184       100    
reco:cvneva:CVNEvaluator                              1.5261e-05     0.728286       5.1253       0.885695      0.634035        100    
reco:energyrecnumu:EnergyReco                         0.000295793    0.0459596      1.15371     0.00448954     0.149286        100    
reco:energyrecnue:EnergyReco                          0.000127672   0.00330957     0.0698126    0.000530086   0.00929347       100    
reco:energyrecnc:EnergyReco                           0.00167798    0.00569721     0.075929     0.00316558    0.00983056       100    
reco:energyrecnumurange:EnergyReco                    0.000139191    0.0418657      1.12796     0.00281143      0.14341        100    
reco:energyrecnumumcs:EnergyReco                      0.000735199    0.0428443      1.15567     0.00345511     0.142894        100    
reco:opdec:Deconvolution                               0.0510481     0.281792      0.569293      0.276153      0.0917727       100    
reco:ophitspe:OpHitFinderDeco                           0.14504       1.95661       4.03919       1.86773       0.83057        100    
reco:opflash:OpFlashFinder                            0.000128041   0.00158415    0.00518921    0.00147451    0.000819216      100    
reco:rns:RandomNumberSaver                            1.6941e-05    4.25171e-05   0.000345574   3.98555e-05   3.30961e-05      100    
reco:anglereconue:NuAngularReco                       0.000149411   0.00243401     0.0421981    0.000448845   0.00658087       100    
reco:anglereconumu:NuAngularReco                       6.518e-05    0.00213101     0.0386314    0.000264373   0.00623345       100    
reco:anglereconuepfps:NuAngularReco                   0.000113531   0.00224936     0.0389324    0.000391875   0.00625415       100    
reco:anglereconumupfps:NuAngularReco                  8.9061e-05    0.00216233     0.0391697    0.000317624   0.00624585       100    
[art]:TriggerResults:TriggerResultInserter             8.03e-06     1.34184e-05   7.2301e-05     1.244e-05    6.85498e-06      100    
end_path:out1:RootOutput                               2.13e-06     7.56577e-06   6.8811e-05    7.1605e-06    6.57447e-06      100    
end_path:out1:RootOutput(write)                        0.0106129     0.265931       1.45866      0.242586      0.233766        100    
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 21932.4 MB
  Peak resident set size usage (VmHWM): 2511.46 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
Validation_ccnc_atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74494002_729_20231201T170150Z_gen_g4_detsim_hitreco__20240507T195919Z_reco2.root
all-input-dids.txt
atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74494002_729_20231201T170150Z_gen_g4_detsim_hitreco__20240507T195919Z_reco2_reco_2024-11-23T_044314Z.log
atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74494002_729_20231201T170150Z_gen_g4_detsim_hitreco__20240507T195919Z_reco2_reco_data_2024-11-23T_044314Z.root
debugprod.log
jobscript.log
justin-processed-pfns.txt
reco2_hist.root
MyPandoraSettings_Master_Atmos_DUNEFD.xml
MyPandoraSettings_Master_DUNEFD.xml
build_slf7.x86_64
localProducts_larsoft_v09_91_02_e26_prof
setup_env-testreco.sh
srcs
temp.txt
temp2.txt
test
work