Jobsub ID 108605.17@justin-prod-sched02.dune.hep.ac.uk

Jobsub ID	108605.17@justin-prod-sched02.dune.hep.ac.uk
Workflow ID	4213
Stage ID	1
User name	ismerio@fnal.gov
HTCondor Group	group_dune
Requested	Processors	1
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	80000 (22 hours)
Submitted time	2024-11-22 23:35:25
Site	UK_Brunel
Entry	CMSHTPC_T2_UK_London_Brunel_dc2_22
Last heartbeat	2024-11-23 00:21:45
From worker node	Hostname	wn-a3-37-01.brunel.ac.uk
	cpuinfo	Intel(R) Xeon(R) Gold 6126 CPU @ 2.60GHz
	OS release	Scientific Linux release 7.9 (Nitrogen)
	Processors	1
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	171000 (47 hours)
	Inner Apptainer?	True
Job state	stalled
Allocator name	justin-allocator-pro.dune.hep.ac.uk
Started	2024-11-22 23:39:30
Input files	fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50574864_541_20231203T160247Z_gen_g4_detsim_hitreco__20240508T033500Z_reco2.root
Outputting started	2024-11-23 00:14:36
Output files
Finished	2024-11-23 00:34:32
List job events Wrapper job log
Jobscript log (last 10,000 characters)

s not exist (max wire number: 480)
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrajFitter:pmtrajfittc@BeginModule  23-Nov-2024 00:14:24 GMT run: 50574864 subRun: 1 event: 54200
1st wire C:0 T:2 P:2 W:646 does not exist (max wire number: 480)
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrajFitter:pmtrajfittc@BeginModule  23-Nov-2024 00:14:24 GMT run: 50574864 subRun: 1 event: 54200
1st wire C:0 T:2 P:2 W:695 does not exist (max wire number: 480)
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrajFitter:pmtrajfittc@BeginModule  23-Nov-2024 00:14:24 GMT run: 50574864 subRun: 1 event: 54200
1st wire C:0 T:2 P:2 W:556 does not exist (max wire number: 480)
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrajFitter:pmtrajfittc@BeginModule  23-Nov-2024 00:14:24 GMT run: 50574864 subRun: 1 event: 54200
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrajFitter:pmtrajfittc@BeginModule  23-Nov-2024 00:14:24 GMT run: 50574864 subRun: 1 event: 54200
1st wire C:0 T:2 P:2 W:536 does not exist (max wire number: 480)
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrajFitter:pmtrajfittc@BeginModule  23-Nov-2024 00:14:24 GMT run: 50574864 subRun: 1 event: 54200
1st wire C:0 T:2 P:2 W:646 does not exist (max wire number: 480)
%MSG
Boundary wire vector sizes: 331, 330, 332
minwire 0: 396
minwire 1: 2573
minwire 2: 42
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 199
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 199
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 199
Classifier summary: 
Output 0: 0.13151, 0.0883926, 0.780098, 
Output 1: 0.0223746, 0.231918, 0.615306, 0.130402, 
Output 2: 0.715342, 0.284658, 

445
***Wiener Filter*******ApplyPostfilter***
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 1458!
23-Nov-2024 00:14:35 GMT  Closed output file "atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50574864_541_20231203T160247Z_gen_g4_detsim_hitreco__20240508T033500Z_reco2_reco_data_2024-11-22T_233933Z.root"
23-Nov-2024 00:14:35 GMT  Closed input file "root://mover.pp.rl.ac.uk:1094/pnfs/pp.rl.ac.uk/data/dune/fardet-hd/41/3b/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50574864_541_20231203T160247Z_gen_g4_detsim_hitreco__20240508T033500Z_reco2.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              1.60504       19.873        303.191       7.86945       41.8124        100    
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.00550017     0.0220799     0.0630105     0.0228637    0.00966912       100    
reco:linecluster:LineCluster                           0.0486058     0.104376       1.56655      0.0604897     0.191689        100    
reco:trajcluster:TrajCluster                           0.0139144     0.496132       15.8515      0.019806       2.05296        100    
reco:pandora:StandardPandora                           0.901217       1.81078       57.1408       1.0112        5.68287        100    
reco:pandoraTrack:LArPandoraTrackCreation             0.000125112   0.00421164     0.0835151    0.00121429     0.011564        100    
reco:pandoraShower:LArPandoraModularShowerCreation    0.000175505   0.00985077     0.237755     0.00216246     0.0273493       100    
reco:pandoracalo:Calorimetry                          9.9877e-05     0.0111526     0.293298     0.000707527    0.0439832       100    
reco:pandorapid:Chi2ParticleID                        3.6659e-05    0.00012154     0.001061     0.00010199    0.000120727      100    
reco:trkshowersplit:TrackShowerHits                   9.2024e-05     0.0109196     0.369084     0.000399074    0.0463033       100    
reco:pmtrack:PMAlgTrackMaker                          0.00105733      1.15838       22.0227      0.130214       3.44898        100    
reco:pmtrackcalo:Calorimetry                          6.9249e-05    0.00747407     0.221073     0.000531916    0.029275        100    
reco:pmtrackpid:Chi2ParticleID                        2.6993e-05    0.00019449     0.0021076    0.000101423   0.000315215      100    
reco:pmtrajfit:PMAlgTrajFitter                        0.000155029     1.03968       21.8328      0.180178       3.00122        100    
reco:pmtrajfitcalo:Calorimetry                        5.3749e-05     0.011308      0.285513     0.00101592     0.0434647       100    
reco:pmtrajfitpid:Chi2ParticleID                      2.4304e-05    0.000129654   0.000609472   0.000111631   0.000101682      100    
reco:pmtracktc:PMAlgTrackMaker                        0.000869778     1.42748       30.8942      0.140807       4.40276        100    
reco:pmtracktccalo:Calorimetry                        6.0859e-05     0.0056645     0.133504     0.00062498     0.0183992       100    
reco:pmtracktcpid:Chi2ParticleID                      2.4793e-05    0.000190489   0.00180647    0.000112615   0.000271765      100    
reco:pmtrajfittc:PMAlgTrajFitter                      0.000142111    0.991506       18.5002      0.161248       2.85044        100    
reco:pmtrajfittccalo:Calorimetry                      5.3901e-05     0.011007      0.333441     0.000779448    0.0453214       100    
reco:pmtrajfittcpid:Chi2ParticleID                    2.3044e-05    0.000241421   0.00294965    0.000111927   0.00043297       100    
reco:blurredcluster:BlurredClustering                 0.00287539     0.826095       7.74779      0.336873       1.22885        100    
reco:emtrkmichelid:EmTrackMichelId                     0.240573       9.13029       126.194       2.38136       19.5492        100    
reco:emshower:EMShower                                0.000213008    0.0176836     0.173026     0.00326727     0.0307266       100    
reco:cvnmap:CVNMapper                                 1.9771e-05     0.0114248     0.0423826     0.0120902     0.010062        100    
reco:cvneva:CVNEvaluator                              1.5751e-05     0.437195      0.758435      0.644897      0.321867        100    
reco:energyrecnumu:EnergyReco                         0.000243438    0.0434691      1.98625     0.00368586     0.210134        100    
reco:energyrecnue:EnergyReco                          0.000117059   0.00146936     0.0279702    0.000469772   0.00367995       100    
reco:energyrecnc:EnergyReco                           0.00161789    0.00302037     0.0310865    0.00201568    0.00388052       100    
reco:energyrecnumurange:EnergyReco                    0.000170012     0.0449        2.15316      0.0017559     0.227389        100    
reco:energyrecnumumcs:EnergyReco                      0.00102325     0.0460356      2.1581      0.00253333     0.227701        100    
reco:opdec:Deconvolution                               0.0339737      0.17797      0.315425      0.183497      0.0712256       100    
reco:ophitspe:OpHitFinderDeco                          0.188615       1.82761       3.53548       1.87015      0.837254        100    
reco:opflash:OpFlashFinder                             0.0001976    0.00110917    0.00348631    0.00112882    0.000538917      100    
reco:rns:RandomNumberSaver                            1.6583e-05    2.89168e-05   0.00024986    2.47925e-05   2.36433e-05      100    
reco:anglereconue:NuAngularReco                       0.000120081   0.00106268     0.0237775    0.000307981   0.00312487       100    
reco:anglereconumu:NuAngularReco                      6.0428e-05    0.000913337    0.023232     0.000164482   0.00303316       100    
reco:anglereconuepfps:NuAngularReco                   0.000102617   0.00102911     0.0236738    0.000262241   0.00306734       100    
reco:anglereconumupfps:NuAngularReco                  8.4273e-05    0.000974993    0.0231426    0.000223359   0.00303528       100    
[art]:TriggerResults:TriggerResultInserter             8.334e-06    1.06959e-05   3.5278e-05    1.01445e-05   3.11408e-06      100    
end_path:out1:RootOutput                               2.69e-06     5.0384e-06    1.5839e-05    3.7825e-06    2.65147e-06      100    
end_path:out1:RootOutput(write)                         0.01406      0.176437      0.722461      0.190153      0.129826        100    
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 10737.1 MB
  Peak resident set size usage (VmHWM): 2421.19 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
Validation_ccnc_atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50574864_541_20231203T160247Z_gen_g4_detsim_hitreco__20240508T033500Z_reco2.root
all-input-dids.txt
atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50574864_541_20231203T160247Z_gen_g4_detsim_hitreco__20240508T033500Z_reco2_reco_2024-11-22T_233933Z.log
atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50574864_541_20231203T160247Z_gen_g4_detsim_hitreco__20240508T033500Z_reco2_reco_data_2024-11-22T_233933Z.root
debugprod.log
jobscript.log
justin-processed-pfns.txt
reco2_hist.root
MyPandoraSettings_Master_Atmos_DUNEFD.xml
MyPandoraSettings_Master_DUNEFD.xml
build_slf7.x86_64
localProducts_larsoft_v09_91_02_e26_prof
setup_env-testreco.sh
srcs
temp.txt
temp2.txt
test
work