Jobsub ID 66569.92@justin-prod-sched02.dune.hep.ac.uk
Jobscript log (last 10,000 characters)
AFM reco2 jobscript.
Input PFN = root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/usertests/9e/49/000732_reco_data_2024-08-16T_162803Z_reco_data_2024-08-19T_140943Z_reco_data_2024-08-20T_124959Z_reco_data_2024-08-22T_093236Z.root
Setting up larsoft UPS area... /cvmfs/larsoft.opensciencegrid.org
Setting up DUNE UPS area... /cvmfs/dune.opensciencegrid.org/products/dune/
/cvmfs/larsoft.opensciencegrid.org/products/xrootd/v5_4_3b/Linux64bit+3.10-2.17-e20-p3913-prof/lib/libXrdPosixPreload.so
=== Start last 50 lines of lar log file ===
reco:pmtrack:PMAlgTrackMaker 0.000859634 6.24402 18.4932 1.51393 7.84076 6
reco:pmtrackcalo:Calorimetry 7.3529e-05 0.0126476 0.0518028 0.00451828 0.0181933 6
reco:pmtrackpid:Chi2ParticleID 2.8283e-05 0.000475673 0.00165015 0.000179514 0.000568568 6
reco:pmtrajfit:PMAlgTrajFitter 0.0180812 3.12927 13.0301 0.701219 4.63328 6
reco:pmtrajfitcalo:Calorimetry 0.0006403 0.0632639 0.316528 0.00654649 0.114473 6
reco:pmtrajfitpid:Chi2ParticleID 0.000131398 0.000880251 0.00329219 0.000266773 0.00113508 6
reco:pmtracktc:PMAlgTrackMaker 0.00241503 5.72966 23.3155 0.87787 8.50303 6
reco:pmtracktccalo:Calorimetry 0.000137871 0.00781196 0.0187539 0.00487638 0.00785647 6
reco:pmtracktcpid:Chi2ParticleID 7.0723e-05 0.000383529 0.000988747 0.000200608 0.000348345 6
reco:pmtrajfittc:PMAlgTrajFitter 0.0299646 2.56616 9.83462 0.804694 3.51849 6
reco:pmtrajfittccalo:Calorimetry 0.000253198 0.0267944 0.131116 0.00392142 0.047227 6
reco:pmtrajfittcpid:Chi2ParticleID 7.4341e-05 0.000660609 0.00260671 0.000225666 0.000896124 6
reco:cvnmap:CVNMapper 3.0127e-05 0.0611668 0.0826692 0.0724099 0.027746 6
reco:cvneva:CVNEvaluator 1.6982e-05 1.13662 2.58417 1.05501 0.754174 6
reco:energyrecnumu:EnergyReco 0.00452344 0.392708 1.81024 0.0565223 0.709188 5
reco:energyrecnue:EnergyReco 0.0007201 0.0625767 0.272269 0.00478638 0.105594 5
reco:energyrecnc:EnergyReco 0.00334572 0.065544 0.275954 0.00764693 0.105954 5
reco:ophit:OpHitFinder 0.182464 0.215029 0.236969 0.221466 0.0194779 5
reco:opflash:OpFlashFinder 0.000292692 0.00105058 0.00285756 0.000801144 0.000928161 5
reco:rns:RandomNumberSaver 1.9116e-05 6.88544e-05 0.000216629 3.7722e-05 7.43014e-05 5
[art]:TriggerResults:TriggerResultInserter 1.1131e-05 1.79338e-05 3.5777e-05 1.4918e-05 9.09104e-06 5
end_path:out1:RootOutput 3.968e-06 8.0716e-06 1.7734e-05 6.202e-06 4.91626e-06 5
end_path:out1:RootOutput(write) 0.500337 0.599557 0.819191 0.543546 0.116479 5
========================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 3430.52 MB
Peak resident set size usage (VmHWM): 1971.56 MB
====================================================================================================
%MSG-s ArtException: PostEndJob 05-Sep-2024 10:30:21 CDT ModuleEndJob
---- EventProcessorFailure BEGIN
EventProcessor: an exception occurred during current event processing
---- ScheduleExecutionFailure BEGIN
Path: ProcessingStopped.
---- StdException BEGIN
An exception was thrown while processing module EnergyReco/energyrecnumu run: 20000031 subRun: 0 event: 623
vector::_M_range_check: __n (which is 37) >= this->size() (which is 19)
---- StdException END
Exception going through path reco
---- ScheduleExecutionFailure END
---- EventProcessorFailure END
---- FatalRootError BEGIN
Fatal Root Error: TTree::SetEntries
Tree branches have different numbers of entries, eg EventAuxiliary has 5 entries while sim::OpDetDivRecs_opdigi__detsim. has 15 entries.
ROOT severity: 2000
---- FatalRootError END
%MSG
Art has completed and will exit with status 1.
=== End last 50 lines of lar log file ===
lar exit code 0
Traceback (most recent call last):
File "/cvmfs/dune.opensciencegrid.org/products/dune/duneutil/v09_75_03d00/bin/extractor_prod.py", line 434, in <module>
main()
File "/cvmfs/dune.opensciencegrid.org/products/dune/duneutil/v09_75_03d00/bin/extractor_prod.py", line 373, in main
mddict = expSpecificMetadata.getmetadata()
File "/cvmfs/dune.opensciencegrid.org/products/dune/duneutil/v09_75_03d00/bin/extractor_prod.py", line 344, in getmetadata
jobt = self.get_job(proc)
File "/cvmfs/dune.opensciencegrid.org/products/dune/duneutil/v09_75_03d00/bin/extractor_prod.py", line 69, in get_job
raise RuntimeError('sam_metadata_dumper returned nonzero exit status {}.'.format(rc))
RuntimeError: sam_metadata_dumper returned nonzero exit status 1.
extractor_prod.py exit code 1
Error reading metadata from file: Expecting value: line 1 column 1 (char 0)
pdjson2metadata exit code 1
.:
total 223076
-rw-r--r-- 1 slot1_8 slot1_8 227538799 Sep 5 10:30 RootOutput-2e22-3b2f-8cbb-1e8e.root
-rw-r--r-- 1 slot1_8 slot1_8 856454 Sep 5 10:30 000732_reco_data_2024-08-16T_162803Z_reco_data_2024-08-19T_140943Z_reco_data_2024-08-20T_124959Z_reco_data_2024-08-22T_093236Z_reco_2024-09-05T_152655Z.log
-rw-r--r-- 1 slot1_8 slot1_8 7560 Sep 5 10:30 reco2_hist.root
-rw-r--r-- 1 slot1_8 slot1_8 6005 Sep 5 10:30 jobscript.log
-rw-r--r-- 1 slot1_8 slot1_8 284 Sep 5 10:26 all-input-dids.txt
-rw-r--r-- 1 slot1_8 slot1_8 0 Sep 5 10:30 000732_reco_data_2024-08-16T_162803Z_reco_data_2024-08-19T_140943Z_reco_data_2024-08-20T_124959Z_reco_data_2024-08-22T_093236Z_reco_data_2024-09-05T_152655Z.root.ext.json
-rw-r--r-- 1 slot1_8 slot1_8 0 Sep 5 10:30 000732_reco_data_2024-08-16T_162803Z_reco_data_2024-08-19T_140943Z_reco_data_2024-08-20T_124959Z_reco_data_2024-08-22T_093236Z_reco_data_2024-09-05T_152655Z.root.json
-rw-r--r-- 1 slot1_8 slot1_8 0 Sep 5 10:27 debugprod.log 