Jobsub ID 263276.91@justin-prod-sched01.dune.hep.ac.uk
| Jobsub ID | 263276.91@justin-prod-sched01.dune.hep.ac.uk |
| Workflow ID | 3447 |
| Stage ID | 1 |
| User name | calcuttj@fnal.gov |
| HTCondor Group | group_dune.prod_mcsim |
| Requested | Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 18000 (5 hours) |
| Submitted time | 2024-09-26 09:54:33 |
| Site | FR_CCIN2P3 |
| Entry | DUNE_FR_CCIN2P3_cccondorce02 |
| Last heartbeat | 2024-09-26 11:10:45 |
| From worker node | Hostname | ccwcondor0559 |
| cpuinfo | AMD EPYC 7453 28-Core Processor |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 106200 (29 hours) |
| Inner Apptainer? | True |
| Job state | jobscript_error |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk |
| Started | 2024-09-26 09:55:57 |
| Input files | hd-protodune:np04hd_raw_run029092_0570_dataflow7_datawriter_0_20240906T021031.hdf5
|
| Jobscript | Exit code | 1 |
| Real time | 0m (0s) |
| CPU time | 0m (0s = 0%) |
| Outputting started | |
| Output files | |
| Finished | 2024-09-26 11:10:45 |
| Saved logs | justin-logs:263276.91-justin-prod-sched01.dune.hep.ac.uk.logs.tgz |
| List job events Wrapper job log |
Jobscript log (last 10,000 characters)
in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 14080 XUs and 12661 XVs -> 2372 XUVs
C:0 T:2 74836 XUs and 64351 XVs -> 2772 XUVs
C:0 T:3 131 XUs and 124 XVs -> 4 XUVs
C:0 T:4 13 XUs and 16 XVs -> 2 XUVs
C:0 T:5 32664 XUs and 49803 XVs -> 3705 XUVs
C:0 T:6 102115 XUs and 111853 XVs -> 1924 XUVs
C:0 T:7 396 XUs and 357 XVs -> 14 XUVs
10793 XUVs total
2697 collection wire objects
10793 potential space points
Neighbour search...
1216739 tests to find 563886 neighbours
Iterating with no regularization...
Begin: 1.23715e+07
0 1.11226e+07
1 1.10511e+07
2 1.10412e+07
Now with regularization...
Begin: 1.04822e+07
0 1.04778e+07
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Begin processing the 29th record. run: 29092 subRun: 1 event: 137031 at 26-Sep-2024 12:58:39 CEST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 51260 traces, 51260 summary
Retagger: tagging trace set: gauss with 39114 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 39114 traces tagged "gauss"
FrameSaver: q=862549 n=727940 tag=gauss
wclsFrameSaver: saving 51260 traces tagged "wiener"
FrameSaver: q=928570 n=663054 tag=wiener
2200 hits in TPC 0
3001 hits in TPC 1
4483 hits in TPC 2
2604 hits in TPC 3
3181 hits in TPC 4
5511 hits in TPC 5
2936 hits in TPC 6
2396 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 463 XUs and 475 XVs -> 23 XUVs
C:0 T:2 9919 XUs and 7289 XVs -> 234 XUVs
C:0 T:3 341 XUs and 551 XVs -> 11 XUVs
C:0 T:4 24 XUs and 26 XVs -> 0 XUVs
C:0 T:5 18829 XUs and 14930 XVs -> 460 XUVs
C:0 T:6 20171 XUs and 16625 XVs -> 1143 XUVs
C:0 T:7 68 XUs and 51 XVs -> 1 XUVs
1872 XUVs total
724 collection wire objects
1872 potential space points
Neighbour search...
119366 tests to find 61360 neighbours
Iterating with no regularization...
Begin: 1.8329e+07
0 1.70014e+07
1 1.69514e+07
2 1.69373e+07
Now with regularization...
Begin: 1.64664e+07
0 1.6459e+07
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Begin processing the 30th record. run: 29092 subRun: 1 event: 137039 at 26-Sep-2024 12:59:55 CEST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 66224 traces, 66224 summary
Retagger: tagging trace set: gauss with 48492 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 48492 traces tagged "gauss"
FrameSaver: q=1.0892e+06 n=879097 tag=gauss
wclsFrameSaver: saving 66224 traces tagged "wiener"
FrameSaver: q=1.16062e+06 n=786796 tag=wiener
2398 hits in TPC 0
1751 hits in TPC 1
7055 hits in TPC 2
5311 hits in TPC 3
2358 hits in TPC 4
4238 hits in TPC 5
5729 hits in TPC 6
3827 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:0 18 XUs and 0 XVs -> 0 XUVs
C:0 T:2 31956 XUs and 33655 XVs -> 3109 XUVs
C:0 T:3 106 XUs and 77 XVs -> 1 XUVs
C:0 T:4 358 XUs and 307 XVs -> 16 XUVs
C:0 T:5 15045 XUs and 12406 XVs -> 113 XUVs
C:0 T:6 16161 XUs and 13607 XVs -> 1394 XUVs
C:0 T:7 191 XUs and 185 XVs -> 11 XUVs
4644 XUVs total
1192 collection wire objects
4644 potential space points
Neighbour search...
292586 tests to find 177118 neighbours
Iterating with no regularization...
Begin: 8.25957e+06
0 7.52964e+06
1 7.46702e+06
2 7.45495e+06
3 7.45094e+06
Now with regularization...
Begin: 6.82805e+06
0 6.82527e+06
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
no of hits closeby 0 event 137039 TrkackID 15 startx, y, z 111.084 207.508 457.631 wireno, peakt tpcno 289 4010.72 5 dist 0min T, max_T2828.14 4230.13
no of hits closeby 0 event 137039 TrkackID 15 startx, y, z 111.084 207.508 457.631 wireno, peakt tpcno 289 4010.72 5 dist 0min T, max_T2828.14 4230.13
26-Sep-2024 13:01:34 CEST Closed output file "np04hd_raw_run029092_0570_dataflow7_datawriter_0_20240906T021031_reco_stage1_reco_stage2_20240926T110134_keepup.root"
26-Sep-2024 13:01:34 CEST Closed input file "np04hd_raw_run029092_0570_dataflow7_datawriter_0_20240906T021031_reco_stage1.root"
========================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
========================================================================================================================================
Full event 51.9534 97.5705 184.035 95.1831 24.2404 30
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.00043353 0.00335799 0.00470308 0.00370963 0.00100371 30
produce:fembfilter:PDHDFEMBFilter 0.499519 0.639768 0.840196 0.628295 0.0899094 30
produce:wclsdatahd:WireCellToolkit 33.5348 40.2715 47.6578 40.1253 3.30432 30
produce:gaushit:GausHitFinder 0.406423 0.764087 2.64977 0.673118 0.403898 30
produce:nhitsfilter:NumberOfHitsFilter 0.00021397 0.000349534 0.00107299 0.000322126 0.000148118 30
produce:reco3d:SpacePointSolver 0.193884 1.35345 5.37952 0.968963 1.31184 29
produce:hitpdune:DisambigFromSpacePoints 0.171807 0.542963 1.76778 0.477912 0.333767 29
produce:pandora:StandardPandora 5.2181 13.2993 43.3707 11.2632 7.40146 29
produce:pandoraWriter:StandardPandora 0.108858 0.160916 0.263694 0.158388 0.0357058 29
produce:pandoraTrack:LArPandoraTrackCreation 0.78693 1.82523 4.61284 1.65082 0.901172 29
produce:pandoraShower:LArPandoraShowerCreation 0.588661 2.2726 7.51523 1.82831 1.47086 29
produce:pandoracalo:Calorimetry 0.361604 0.849804 2.45331 0.695988 0.433391 29
produce:pandoracalonosce:Calorimetry 0.319275 0.767374 2.278 0.699826 0.386729 29
produce:pandorapid:Chi2ParticleID 0.00115679 0.00201702 0.00315472 0.00198061 0.000537865 29
produce:pandoraShowercalo:ShowerCalorimetry 0.496181 1.81863 6.91024 1.58756 1.19725 29
produce:pandoraShowercalonosce:ShowerCalorimetry 0.600982 1.79303 6.9865 1.40437 1.20289 29
produce:pandoraGnocchiCalo:GnocchiCalorimetry 0.0826223 0.141646 0.230091 0.123834 0.0419892 29
produce:pandoraGnocchiCalonosce:GnocchiCalorimetry 0.0266277 0.0513265 0.0991824 0.0463445 0.0165484 29
produce:emtrkmichelid:EmTrackMichelId 19.0677 31.2462 53.7564 30.7328 7.45727 29
[art]:TriggerResults:TriggerResultInserter 2.3223e-05 5.81616e-05 0.000127237 5.4983e-05 1.94009e-05 30
end_path:out1:RootOutput 8.465e-06 1.64036e-05 2.5017e-05 1.67615e-05 4.16078e-06 30
end_path:michelremoving:michelremoving 4.9833e-05 0.0756519 0.454401 0.0451662 0.0904022 30
end_path:michelremovinggnocchi:michelremoving 3.2601e-05 0.0639814 0.29794 0.0396802 0.0680442 30
end_path:calibanagnocchi:CalibAnaTree 3.4325e-05 0.540269 1.59247 0.479101 0.297357 30
end_path:out1:RootOutput(write) 0.414003 0.747141 1.45225 0.735979 0.199388 30
========================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 4947.42 MB
Peak resident set size usage (VmHWM): 2369.42 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np04hd_raw_run029092_0570_dataflow7_datawriter_0_20240906T021031_reco_stage1_reco_stage2_20240926T110134_keepup.root
\tHists: np04hd_raw_run029092_0570_dataflow7_datawriter_0_20240906T021031_reco_stage1_reco_stage2_20240926T110134_keepup_hists.root
Forming reco metadata
/cvmfs/larsoft.opensciencegrid.org/products/python/v3_9_15/Linux64bit+3.10-2.17/bin/python: No module named meta_maker
Error in reco metadata
