Jobsub ID 92378.1@justin-prod-sched02.dune.hep.ac.uk
| Jobsub ID | 92378.1@justin-prod-sched02.dune.hep.ac.uk |
| Workflow ID | 3920 |
| Stage ID | 1 |
| User name | ismerio@fnal.gov |
| HTCondor Group | group_dune |
| Requested | Processors | 1 |
| RSS bytes | 4194304000 (4000 MiB) |
| Wall seconds limit | 80000 (22 hours) |
| Submitted time | 2024-11-06 22:01:58 |
| Site | US_UConn-HPC |
| Entry | GLUEX_US_UConn-HPC_osgce |
| Last heartbeat | 2024-11-07 00:51:14 |
| From worker node | Hostname | cn438 |
| cpuinfo | AMD EPYC 7452 32-Core Processor |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4194304000 (4000 MiB) |
| Wall seconds limit | 172800 (48 hours) |
| Inner Apptainer? | True |
| Job state | outputting_failed |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk |
| Started | 2024-11-06 22:04:19 |
| Input files | fardet-hd:anutau_dune10kt_1x2x6_1127_165_20230828T190254Z_gen_g4_detsim_hitreco__20240221T023848Z_reco2.root
|
| Jobscript | Exit code | 0 |
| Real time | 2h (9997s) |
| CPU time | 2h (9459s = 94%) |
| Outputting started | 2024-11-07 00:50:57 |
| Output files | |
| Finished | 2024-11-07 00:51:14 |
| List job events Wrapper job log |
Jobscript log (last 10,000 characters)
lation.
%MSG
%MSG-e pma::Track3D: PMAlgTrajFitter:pmtrajfittc@BeginModule 06-Nov-2024 19:50:37 EST run: 1127 subRun: 1 event: 16600
Track empty.
%MSG
%MSG-e pma::Track3D: PMAlgTrajFitter:pmtrajfittc@BeginModule 06-Nov-2024 19:50:37 EST run: 1127 subRun: 1 event: 16600
TuneFullTree failed.
%MSG
%MSG-e pma::Track3D: PMAlgTrajFitter:pmtrajfittc@BeginModule 06-Nov-2024 19:50:37 EST run: 1127 subRun: 1 event: 16600
0 enabled hits in AverageDist2 calculation.
%MSG
%MSG-e pma::Track3D: PMAlgTrajFitter:pmtrajfittc@BeginModule 06-Nov-2024 19:50:37 EST run: 1127 subRun: 1 event: 16600
Track empty.
%MSG
%MSG-e pma::Track3D: PMAlgTrajFitter:pmtrajfittc@BeginModule 06-Nov-2024 19:50:37 EST run: 1127 subRun: 1 event: 16600
TuneFullTree failed.
%MSG
%MSG-e pma::Track3D: PMAlgTrajFitter:pmtrajfittc@BeginModule 06-Nov-2024 19:50:37 EST run: 1127 subRun: 1 event: 16600
0 enabled hits in AverageDist2 calculation.
%MSG
%MSG-e pma::Track3D: PMAlgTrajFitter:pmtrajfittc@BeginModule 06-Nov-2024 19:50:37 EST run: 1127 subRun: 1 event: 16600
Track empty.
%MSG
%MSG-e pma::Track3D: PMAlgTrajFitter:pmtrajfittc@BeginModule 06-Nov-2024 19:50:37 EST run: 1127 subRun: 1 event: 16600
TuneFullTree failed.
%MSG
Shower cones:
3
9
10
11
16
22
1
6
8
18
19
Event 16600 track shower separation:
Shower initial tracks are:
2
4
Track tracks are:
0
5
7
13
14
15
17
20
21
%MSG-e WireIDIntersectionCheck: EMShower:emshower@BeginModule 06-Nov-2024 19:50:51 EST run: 1127 subRun: 1 event: 16600
Comparing two wires in the same plane: return failure
%MSG
Boundary wire vector sizes: 481, 562, 533
minwire 0: 2357
minwire 1: 55
minwire 2: 2603
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max wires due to vertex determination failure: 2379, 2878
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary:
Output 0: 0.113989,
Output 1: 0.000695361, 0.00257896, 0.125435, 0.871291,
Output 2: 0.798496, 0.0326562, 0.0671356, 0.101712,
Output 3: 0.927665, 0.0705996, 0.00151292, 0.000222514,
Output 4: 0.0211922, 0.967965, 0.0102652, 0.000577384,
Output 5: 0.0103312, 0.985763, 0.00368777, 0.000217958,
Output 6: 0.107973, 0.82175, 0.0623337, 0.00794264,
421
***Wiener Filter*******ApplyPostfilter***
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 1228!
06-Nov-2024 19:50:56 EST Closed output file "anutau_dune10kt_1x2x6_1127_165_20230828T190254Z_gen_g4_detsim_hitreco__20240221T023848Z_reco2_reco_data_2024-11-06T_220423Z.root"
06-Nov-2024 19:50:56 EST Closed input file "root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-hd/4e/f8/anutau_dune10kt_1x2x6_1127_165_20230828T190254Z_gen_g4_detsim_hitreco__20240221T023848Z_reco2.root"
========================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
========================================================================================================================================
Full event 0.868144 96.5029 918.071 28.8763 165.618 100
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.0259283 0.0634888 0.122683 0.0618597 0.0131768 100
reco:linecluster:LineCluster 0.145836 0.965091 17.3373 0.276474 2.62545 100
reco:trajcluster:TrajCluster 0.0728643 6.30598 115.509 0.164268 18.7508 100
reco:pandora:StandardPandora 0.0844546 7.14019 190.347 1.76986 22.1798 100
reco:pandoraTrack:LArPandoraTrackCreation 0.00015869 0.0258333 0.287967 0.00618421 0.0462691 100
reco:pandoraShower:LArPandoraModularShowerCreation 0.000210217 0.0478165 0.448846 0.0104722 0.0887931 100
reco:pandoracalo:Calorimetry 0.000108735 0.0730567 1.02858 0.00591208 0.165561 100
reco:pandorapid:Chi2ParticleID 4.6849e-05 0.000354471 0.00271696 0.000255893 0.00035487 100
reco:trkshowersplit:TrackShowerHits 8.0011e-05 0.0876773 1.39953 0.00505249 0.217277 100
reco:pmtrack:PMAlgTrackMaker 0.00160119 10.8426 127.307 1.66834 22.2072 100
reco:pmtrackcalo:Calorimetry 8.4068e-05 0.0663388 1.08651 0.0041891 0.160175 100
reco:pmtrackpid:Chi2ParticleID 3.3734e-05 0.000908198 0.00568528 0.000409111 0.00119167 100
reco:pmtrajfit:PMAlgTrajFitter 0.000237509 6.01185 59.1361 1.62854 10.9099 100
reco:pmtrajfitcalo:Calorimetry 7.9009e-05 0.0750571 1.13707 0.00716911 0.172759 100
reco:pmtrajfitpid:Chi2ParticleID 3.2291e-05 0.000397139 0.00166976 0.000285559 0.000334355 100
reco:pmtracktc:PMAlgTrackMaker 0.00146223 13.2149 180.159 1.69563 27.5937 100
reco:pmtracktccalo:Calorimetry 0.000109056 0.0383149 0.478955 0.00521546 0.0757336 100
reco:pmtracktcpid:Chi2ParticleID 3.753e-05 0.000745337 0.00397829 0.000410184 0.000836404 100
reco:pmtrajfittc:PMAlgTrajFitter 0.00020692 6.14285 58.1633 1.54068 10.9609 100
reco:pmtrajfittccalo:Calorimetry 7.9641e-05 0.07335 1.18892 0.00680368 0.17207 100
reco:pmtrajfittcpid:Chi2ParticleID 3.3363e-05 0.000966314 0.00630075 0.000430157 0.00124696 100
reco:blurredcluster:BlurredClustering 0.000233691 2.29533 11.7959 1.465 2.37565 100
reco:emtrkmichelid:EmTrackMichelId 0.000127871 37.8225 268.29 12.0458 54.0193 100
reco:emshower:EMShower 0.000202342 0.0567038 0.351092 0.00977907 0.082237 100
reco:cvnmap:CVNMapper 1.9357e-05 0.0329054 0.105688 0.0303435 0.0208057 100
reco:cvneva:CVNEvaluator 1.7082e-05 1.06145 2.04313 1.22919 0.450068 100
reco:energyrecnumu:EnergyReco 0.000414713 0.230528 3.3086 0.0206192 0.576798 100
reco:energyrecnue:EnergyReco 0.000174779 0.00863777 0.134975 0.00236481 0.0185691 100
reco:energyrecnc:EnergyReco 0.00235515 0.0115467 0.131858 0.00438285 0.0186099 100
reco:energyrecnumurange:EnergyReco 0.000198664 0.225258 3.2686 0.0100932 0.583433 100
reco:energyrecnumumcs:EnergyReco 0.0028477 0.236149 3.99523 0.0134464 0.627463 100
reco:opdec:Deconvolution 0.080976 0.38816 0.654835 0.388008 0.0977994 100
reco:ophitspe:OpHitFinderDeco 0.218691 2.32327 3.81029 2.31836 0.781262 100
reco:opflash:OpFlashFinder 0.000192704 0.00182353 0.00506343 0.00185004 0.000725646 100
reco:rns:RandomNumberSaver 2.0829e-05 4.60842e-05 0.000296329 4.1498e-05 3.07512e-05 100
[art]:TriggerResults:TriggerResultInserter 1.2153e-05 1.72006e-05 4.1809e-05 1.6326e-05 4.34412e-06 100
end_path:out1:RootOutput 3.547e-06 6.64159e-06 1.9537e-05 6.803e-06 2.4135e-06 100
end_path:out1:RootOutput(write) 0.00882159 0.628047 1.81631 0.589305 0.352021 100
========================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 21665.7 MB
Peak resident set size usage (VmHWM): 2793.98 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
Validation_ccnc_anutau_dune10kt_1x2x6_1127_165_20230828T190254Z_gen_g4_detsim_hitreco__20240221T023848Z_reco2.root
all-input-dids.txt
anutau_dune10kt_1x2x6_1127_165_20230828T190254Z_gen_g4_detsim_hitreco__20240221T023848Z_reco2_reco_2024-11-06T_220423Z.log
anutau_dune10kt_1x2x6_1127_165_20230828T190254Z_gen_g4_detsim_hitreco__20240221T023848Z_reco2_reco_data_2024-11-06T_220423Z.root
debugprod.log
jobscript.log
justin-processed-pfns.txt
reco2_hist.root
MyPandoraSettings_Master_Atmos_DUNEFD.xml
MyPandoraSettings_Master_DUNEFD.xml
build_slf7.x86_64
localProducts_larsoft_v09_91_02_e26_prof
setup_env-testreco.sh
srcs
work 