Jobsub ID 108885.34@justin-prod-sched02.dune.hep.ac.uk
Jobscript log (last 10,000 characters)
8365 subRun: 1 event: 47303
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck: PMAlgTrajFitter:pmtrajfittc@BeginModule 23-Nov-2024 08:37:58 CET run: 6418365 subRun: 1 event: 47303
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck: PMAlgTrajFitter:pmtrajfittc@BeginModule 23-Nov-2024 08:37:58 CET run: 6418365 subRun: 1 event: 47303
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck: PMAlgTrajFitter:pmtrajfittc@BeginModule 23-Nov-2024 08:37:58 CET run: 6418365 subRun: 1 event: 47303
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck: PMAlgTrajFitter:pmtrajfittc@BeginModule 23-Nov-2024 08:37:58 CET run: 6418365 subRun: 1 event: 47303
Comparing two wires in the same plane: return failure
%MSG
%MSG-e pma::Track3D: PMAlgTrajFitter:pmtrajfittc@BeginModule 23-Nov-2024 08:37:58 CET run: 6418365 subRun: 1 event: 47303
0 enabled hits in AverageDist2 calculation.
%MSG
%MSG-e pma::Track3D: PMAlgTrajFitter:pmtrajfittc@BeginModule 23-Nov-2024 08:37:58 CET run: 6418365 subRun: 1 event: 47303
Track empty.
%MSG
%MSG-e pma::Track3D: PMAlgTrajFitter:pmtrajfittc@BeginModule 23-Nov-2024 08:37:58 CET run: 6418365 subRun: 1 event: 47303
TuneFullTree failed.
%MSG
23-Nov-2024 08:37:58 CET Closed input file "root://penguin12.grid.surfsara.nl:21094/pnfs/grid.sara.nl/data/dune/disk/RSE/fardet-hd/29/17/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6418365_473_20231203T111345Z_gen_g4_detsim_hitreco__20240508T044205Z_reco2.root"
========================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
========================================================================================================================================
Full event 0.00180073 15.401 36.7306 9.47073 15.5698 3
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.00127637 0.001531 0.00180073 0.00151588 0.000214336 3
reco:linecluster:LineCluster 0.012472 0.032059 0.0530054 0.0306995 0.0165756 3
reco:trajcluster:TrajCluster 0.00357441 0.133412 0.370651 0.0260098 0.168003 3
reco:pandora:StandardPandora 1.5559 2.40365 3.1371 2.51795 0.65056 3
reco:pandoraTrack:LArPandoraTrackCreation 0.000166333 0.00530491 0.00945139 0.00629702 0.00385498 3
reco:pandoraShower:LArPandoraModularShowerCreation 0.000603473 0.0148382 0.0257151 0.0181962 0.0105232 3
reco:pandoracalo:Calorimetry 0.000117199 0.00428455 0.00767951 0.00505694 0.00313524 3
reco:pandorapid:Chi2ParticleID 4.9804e-05 0.000690016 0.00174987 0.000270376 0.000754819 3
reco:trkshowersplit:TrackShowerHits 0.000311656 0.00502869 0.0121956 0.00257881 0.00515159 3
reco:pmtrack:PMAlgTrackMaker 0.00735893 0.671064 1.90987 0.0959581 0.876718 3
reco:pmtrackcalo:Calorimetry 0.000309157 0.00479597 0.0131592 0.000919522 0.00591897 3
reco:pmtrackpid:Chi2ParticleID 9.2037e-05 0.00024023 0.00050802 0.000120633 0.000189716 3
reco:pmtrajfit:PMAlgTrajFitter 0.0241663 0.626951 1.43387 0.422814 0.593337 3
reco:pmtrajfitcalo:Calorimetry 0.000292207 0.00375004 0.00972226 0.00123564 0.00424053 3
reco:pmtrajfitpid:Chi2ParticleID 8.3916e-05 0.000195108 0.000362245 0.000139162 0.000120317 3
reco:pmtracktc:PMAlgTrackMaker 0.0197463 0.673943 1.89407 0.108015 0.863511 3
reco:pmtracktccalo:Calorimetry 0.000528123 0.00501309 0.0134048 0.00110632 0.00593855 3
reco:pmtracktcpid:Chi2ParticleID 0.000122106 0.000283983 0.000602559 0.000127283 0.000225277 3
reco:pmtrajfittc:PMAlgTrajFitter 0.0745702 0.782666 2.13189 0.141533 0.954441 3
reco:pmtrajfittccalo:Calorimetry 0.000507675 0.00586758 0.0160866 0.0010085 0.00722881 3
reco:pmtrajfittcpid:Chi2ParticleID 0.000120904 0.00030414 0.000668177 0.000123339 0.000257415 3
reco:blurredcluster:BlurredClustering 0.281987 1.25562 2.22925 1.25562 0.973631 2
reco:emtrkmichelid:EmTrackMichelId 2.5324 11.1514 19.7703 11.1514 8.61897 2
reco:emshower:EMShower 0.00503467 0.0350095 0.0649843 0.0350095 0.0299748 2
reco:cvnmap:CVNMapper 0.0120042 0.0238108 0.0356175 0.0238108 0.0118066 2
reco:cvneva:CVNEvaluator 1.1729 1.19994 1.22697 1.19994 0.0270387 2
reco:energyrecnumu:EnergyReco 0.0170039 0.0633792 0.109754 0.0633792 0.0463753 2
reco:energyrecnue:EnergyReco 0.000652673 0.00156437 0.00247607 0.00156437 0.000911701 2
reco:energyrecnc:EnergyReco 0.00319198 0.00407431 0.00495665 0.00407431 0.000882336 2
reco:energyrecnumurange:EnergyReco 0.00880645 0.0550626 0.101319 0.0550626 0.0462561 2
reco:energyrecnumumcs:EnergyReco 0.0111734 0.0568884 0.102603 0.0568884 0.045715 2
reco:opdec:Deconvolution 0.146858 0.211507 0.276156 0.211507 0.0646485 2
reco:ophitspe:OpHitFinderDeco 0.962579 1.5059 2.04921 1.5059 0.543317 2
reco:opflash:OpFlashFinder 0.00198615 0.00323243 0.00447871 0.00323243 0.00124628 2
reco:rns:RandomNumberSaver 2.9014e-05 0.000175701 0.000322388 0.000175701 0.000146687 2
reco:anglereconue:NuAngularReco 0.00137212 0.00155455 0.00173697 0.00155455 0.000182424 2
reco:anglereconumu:NuAngularReco 0.000377224 0.00101368 0.00165014 0.00101368 0.000636459 2
reco:anglereconuepfps:NuAngularReco 0.00179133 0.00233134 0.00287135 0.00233134 0.000540011 2
reco:anglereconumupfps:NuAngularReco 0.000450178 0.00111457 0.00177896 0.00111457 0.000664389 2
[art]:TriggerResults:TriggerResultInserter 1.2047e-05 3.4223e-05 5.6399e-05 3.4223e-05 2.2176e-05 2
end_path:out1:RootOutput 5.161e-06 1.17425e-05 1.8324e-05 1.17425e-05 6.5815e-06 2
end_path:out1:RootOutput(write) 0.247685 0.312713 0.377741 0.312713 0.0650283 2
========================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 6422.01 MB
Peak resident set size usage (VmHWM): 1609.12 MB
====================================================================================================
%MSG-s ArtException: PostEndJob 23-Nov-2024 08:37:59 CET ModuleEndJob
---- EventProcessorFailure BEGIN
EventProcessor: an exception occurred during current event processing
---- ScheduleExecutionFailure BEGIN
Path: ProcessingStopped.
---- BadAlloc BEGIN
A bad_alloc exception was thrown while processing module BlurredClustering/blurredcluster run: 6418365 subRun: 1 event: 47303
The job has probably exhausted the virtual memory available to the process.
---- BadAlloc END
Exception going through path reco
---- ScheduleExecutionFailure END
---- EventProcessorFailure END
---- FatalRootError BEGIN
Fatal Root Error: TTree::SetEntries
Tree branches have different numbers of entries, eg EventAuxiliary has 2 entries while sim::OpDetDivRecs_opdigi__detsim. has 100 entries.
ROOT severity: 2000
---- FatalRootError END
%MSG
Art has completed and will exit with status 1.
=== End last 100 lines of lar log file ===
RootOutput-ac06-723a-6685-6738.root
Validation_ccnc_atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6418365_473_20231203T111345Z_gen_g4_detsim_hitreco__20240508T044205Z_reco2.root
all-input-dids.txt
atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6418365_473_20231203T111345Z_gen_g4_detsim_hitreco__20240508T044205Z_reco2_reco_2024-11-23T_073511Z.log
debugprod.log
jobscript.log
justin-processed-pfns.txt
reco2_hist.root
MyPandoraSettings_Master_Atmos_DUNEFD.xml
MyPandoraSettings_Master_DUNEFD.xml
build_slf7.x86_64
localProducts_larsoft_v09_91_02_e26_prof
setup_env-testreco.sh
srcs
temp.txt
temp2.txt
test
work