Jobsub ID 78742.49@justin-prod-sched02.dune.hep.ac.uk
| Jobsub ID | 78742.49@justin-prod-sched02.dune.hep.ac.uk |
| Workflow ID | 3448 |
| Stage ID | 1 |
| User name | calcuttj@fnal.gov |
| HTCondor Group | group_dune.prod_mcsim |
| Requested | Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 18000 (5 hours) |
| Submitted time | 2024-09-26 12:05:15 |
| Site | ES_PIC |
| Entry | DUNE_T1_ES_PIC_ce14-multicore |
| Last heartbeat | 2024-09-26 13:59:47 |
| From worker node | Hostname | tds211.pic.es |
| cpuinfo | Intel(R) Xeon(R) CPU E5-2640 v3 @ 2.60GHz |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 216000 (60 hours) |
| Inner Apptainer? | True |
| Job state | jobscript_error |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk |
| Started | 2024-09-26 12:35:50 |
| Input files | hd-protodune:np04hd_raw_run029092_0916_dataflow4_datawriter_0_20240906T073858.hdf5
|
| Jobscript | Exit code | 1 |
| Real time | 0m (0s) |
| CPU time | 0m (0s = 0%) |
| Outputting started | |
| Output files | |
| Finished | 2024-09-26 13:59:47 |
| Saved logs | justin-logs:78742.49-justin-prod-sched02.dune.hep.ac.uk.logs.tgz |
| List job events Wrapper job log |
Jobscript log (last 10,000 characters)
5859 ticks
wclsFrameSaver: saving 44822 traces tagged "gauss"
FrameSaver: q=978142 n=863921 tag=gauss
wclsFrameSaver: saving 57921 traces tagged "wiener"
FrameSaver: q=1.05243e+06 n=792564 tag=wiener
4533 hits in TPC 0
4047 hits in TPC 1
3653 hits in TPC 2
2397 hits in TPC 3
4096 hits in TPC 4
5725 hits in TPC 5
4250 hits in TPC 6
3370 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 849 XUs and 1151 XVs -> 54 XUVs
C:0 T:2 7790 XUs and 13051 XVs -> 555 XUVs
C:0 T:3 84 XUs and 88 XVs -> 1 XUVs
C:0 T:4 86 XUs and 86 XVs -> 1 XUVs
C:0 T:5 30610 XUs and 20859 XVs -> 577 XUVs
C:0 T:6 13230 XUs and 11467 XVs -> 281 XUVs
C:0 T:7 522 XUs and 515 XVs -> 16 XUVs
1485 XUVs total
887 collection wire objects
1485 potential space points
Neighbour search...
37227 tests to find 22942 neighbours
Iterating with no regularization...
Begin: 5.1691e+06
0 4.8514e+06
1 4.82993e+06
2 4.82587e+06
Now with regularization...
Begin: 4.5901e+06
0 4.58834e+06
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Begin processing the 30th record. run: 29092 subRun: 1 event: 220076 at 26-Sep-2024 15:57:28 CEST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 78932 traces, 78932 summary
Retagger: tagging trace set: gauss with 58051 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 58051 traces tagged "gauss"
FrameSaver: q=1.38224e+06 n=1103989 tag=gauss
wclsFrameSaver: saving 78932 traces tagged "wiener"
FrameSaver: q=1.49623e+06 n=1003493 tag=wiener
2440 hits in TPC 0
2525 hits in TPC 1
5835 hits in TPC 2
3630 hits in TPC 3
4133 hits in TPC 4
6323 hits in TPC 5
9759 hits in TPC 6
7169 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:2 48705 XUs and 18024 XVs -> 590 XUVs
C:0 T:3 70 XUs and 83 XVs -> 1 XUVs
C:0 T:4 142 XUs and 157 XVs -> 9 XUVs
C:0 T:5 26519 XUs and 35162 XVs -> 744 XUVs
C:0 T:6 72762 XUs and 140477 XVs -> 5110 XUVs
C:0 T:7 310 XUs and 887 XVs -> 49 XUVs
6503 XUVs total
2098 collection wire objects
6503 potential space points
Neighbour search...
385407 tests to find 210116 neighbours
Iterating with no regularization...
Begin: 2.16006e+07
0 1.76939e+07
1 1.75027e+07
2 1.74736e+07
3 1.74653e+07
Now with regularization...
Begin: 1.66521e+07
0 1.66468e+07
TwoViewTransverseTracksAlgorithm: failed to calculate correlation coefficient p-value for these numbers
----view 0: 0.00144756 0.00144762 0.00144756 0.00144756 0.00144762 0.00144756 0.00144756 0.00136143 0.00124109 0.00124103 0.00124109
----view 1: 0 0 0.00985837 0.00478625 0.0632095 0.0640869 0.0513096 0.0664158 0.0562859 0.0338116 0.00838017
TwoViewTransverseTracksAlgorithm: failed to calculate correlation coefficient p-value for these numbers
----view 0: 0.00144762 0.00144756 0.00144756 0.00144762 0.00144756 0.00144756 0.00136143 0.00124109 0.00124103 0.00124109 0.00124109
----view 1: 0 0.00985837 0.00478625 0.0632095 0.0640869 0.0513096 0.0664158 0.0562859 0.0338116 0.00838017 0.0136907
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
no of hits closeby 0 event 220076 TrkackID 13 startx, y, z 14.0173 592.929 445.011 wireno, peakt tpcno 299 5 5 dist 0min T, max_T5 2993.5
no of hits closeby 6 event 220076 TrkackID 17 startx, y, z -11.6904 597.79 451.402 wireno, peakt tpcno 460 1842.18 6 dist 9.63614min T, max_T1791.27 3194.68
no of hits closeby 0 event 220076 TrkackID 13 startx, y, z 14.0173 592.929 445.011 wireno, peakt tpcno 299 5 5 dist 0min T, max_T5 2993.5
no of hits closeby 6 event 220076 TrkackID 17 startx, y, z -11.6904 597.79 451.402 wireno, peakt tpcno 460 1842.18 6 dist 9.63614min T, max_T1791.27 3194.68
26-Sep-2024 15:59:46 CEST Closed output file "np04hd_raw_run029092_0916_dataflow4_datawriter_0_20240906T073858_reco_stage1_reco_stage2_20240926T135946_keepup.root"
26-Sep-2024 15:59:46 CEST Closed input file "np04hd_raw_run029092_0916_dataflow4_datawriter_0_20240906T073858_reco_stage1.root"
========================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
========================================================================================================================================
Full event 62.3097 92.0754 137.687 91.0063 16.8779 30
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.000444886 0.0022161 0.00362228 0.00258244 0.00100777 30
produce:fembfilter:PDHDFEMBFilter 0.563222 0.609898 0.676551 0.598206 0.0307088 30
produce:wclsdatahd:WireCellToolkit 38.3442 43.5672 52.7686 42.3133 4.32128 30
produce:gaushit:GausHitFinder 0.35091 0.759257 1.16239 0.748315 0.172047 30
produce:nhitsfilter:NumberOfHitsFilter 0.000216156 0.000370739 0.000612471 0.000371375 9.09733e-05 30
produce:reco3d:SpacePointSolver 0.0764115 0.867449 2.86735 0.581642 0.732921 30
produce:hitpdune:DisambigFromSpacePoints 0.0658941 0.536741 1.19802 0.450534 0.282155 30
produce:pandora:StandardPandora 4.54095 13.3507 29.9979 12.5637 5.72862 30
produce:pandoraWriter:StandardPandora 0.0886743 0.14759 0.213124 0.141919 0.0356575 30
produce:pandoraTrack:LArPandoraTrackCreation 0.464632 1.34333 2.61909 1.35784 0.548894 30
produce:pandoraShower:LArPandoraShowerCreation 0.369345 1.56553 4.31192 1.351 0.911563 30
produce:pandoracalo:Calorimetry 0.242485 0.656363 1.16096 0.65105 0.237051 30
produce:pandoracalonosce:Calorimetry 0.221988 0.601926 1.12871 0.604206 0.224181 30
produce:pandorapid:Chi2ParticleID 0.00112081 0.0018049 0.00250785 0.0017528 0.000337887 30
produce:pandoraShowercalo:ShowerCalorimetry 0.326982 1.43031 3.53207 1.28164 0.813196 30
produce:pandoraShowercalonosce:ShowerCalorimetry 0.328555 1.39551 3.46011 1.26394 0.77289 30
produce:pandoraGnocchiCalo:GnocchiCalorimetry 0.0603472 0.131473 0.200824 0.127277 0.0339076 30
produce:pandoraGnocchiCalonosce:GnocchiCalorimetry 0.0251611 0.045489 0.0695587 0.0427311 0.0117622 30
produce:emtrkmichelid:EmTrackMichelId 13.4151 23.5645 34.9699 23.0827 5.62055 30
[art]:TriggerResults:TriggerResultInserter 1.9348e-05 5.85946e-05 0.000842936 2.75895e-05 0.000146105 30
end_path:out1:RootOutput 7.272e-06 1.04416e-05 3.8258e-05 8.87e-06 5.66582e-06 30
end_path:michelremoving:michelremoving 0.0176031 0.0939921 0.447319 0.0360191 0.100497 30
end_path:michelremovinggnocchi:michelremoving 0.0156645 0.0919692 0.458881 0.0348421 0.10184 30
end_path:calibanagnocchi:CalibAnaTree 0.181693 0.495517 1.05082 0.483266 0.182932 30
end_path:out1:RootOutput(write) 0.328686 0.72862 1.07346 0.722382 0.178632 30
========================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 4934.95 MB
Peak resident set size usage (VmHWM): 2583.77 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np04hd_raw_run029092_0916_dataflow4_datawriter_0_20240906T073858_reco_stage1_reco_stage2_20240926T135946_keepup.root
\tHists: np04hd_raw_run029092_0916_dataflow4_datawriter_0_20240906T073858_reco_stage1_reco_stage2_20240926T135946_keepup_hists.root
Forming reco metadata
/cvmfs/larsoft.opensciencegrid.org/products/python/v3_9_15/Linux64bit+3.10-2.17/bin/python: No module named meta_maker
Error in reco metadata
