Jobsub ID 263295.145@justin-prod-sched01.dune.hep.ac.uk
| Jobsub ID | 263295.145@justin-prod-sched01.dune.hep.ac.uk |
| Workflow ID | 3453 |
| Stage ID | 1 |
| User name | calcuttj@fnal.gov |
| HTCondor Group | group_dune.prod_mcsim |
| Requested | Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 18000 (5 hours) |
| Submitted time | 2024-09-26 09:58:35 |
| Site | FR_CCIN2P3 |
| Entry | DUNE_FR_CCIN2P3_cccondorce01 |
| Last heartbeat | 2024-09-26 11:07:01 |
| From worker node | Hostname | ccwcondor0564 |
| cpuinfo | AMD EPYC 7453 28-Core Processor |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 106200 (29 hours) |
| Inner Apptainer? | True |
| Job state | jobscript_error |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk |
| Started | 2024-09-26 10:04:23 |
| Input files | hd-protodune:np04hd_raw_run029107_3522_dataflow2_datawriter_0_20240909T023153.hdf5
|
| Jobscript | Exit code | 1 |
| Real time | 0m (0s) |
| CPU time | 0m (0s = 0%) |
| Outputting started | |
| Output files | |
| Finished | 2024-09-26 11:07:01 |
| Saved logs | justin-logs:263295.145-justin-prod-sched01.dune.hep.ac.uk.logs.tgz |
| List job events Wrapper job log |
Jobscript log (last 10,000 characters)
XVs -> 164 XUVs
C:0 T:2 14364 XUs and 19328 XVs -> 1947 XUVs
C:0 T:3 381 XUs and 471 XVs -> 7 XUVs
C:0 T:4 100 XUs and 132 XVs -> 2 XUVs
C:0 T:5 11190 XUs and 9487 XVs -> 173 XUVs
C:0 T:6 18897 XUs and 11706 XVs -> 395 XUVs
C:0 T:7 70 XUs and 55 XVs -> 3 XUVs
2691 XUVs total
1005 collection wire objects
2691 potential space points
Neighbour search...
62149 tests to find 40374 neighbours
Iterating with no regularization...
Begin: 1.93191e+07
0 1.89118e+07
1 1.88725e+07
2 1.88675e+07
Now with regularization...
Begin: 1.80238e+07
0 1.80219e+07
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Begin processing the 29th record. run: 29107 subRun: 1 event: 845538 at 26-Sep-2024 12:54:22 CEST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 57398 traces, 57398 summary
Retagger: tagging trace set: gauss with 41462 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 41462 traces tagged "gauss"
FrameSaver: q=845870 n=746353 tag=gauss
wclsFrameSaver: saving 57398 traces tagged "wiener"
FrameSaver: q=921792 n=668437 tag=wiener
4865 hits in TPC 0
3922 hits in TPC 1
6252 hits in TPC 2
3124 hits in TPC 3
2770 hits in TPC 4
5523 hits in TPC 5
2188 hits in TPC 6
1112 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 1543 XUs and 2910 XVs -> 116 XUVs
C:0 T:2 21398 XUs and 15019 XVs -> 300 XUVs
C:0 T:3 71 XUs and 89 XVs -> 3 XUVs
C:0 T:4 1037 XUs and 1530 XVs -> 168 XUVs
C:0 T:5 24057 XUs and 34071 XVs -> 1986 XUVs
C:0 T:6 2716 XUs and 1890 XVs -> 41 XUVs
C:0 T:7 87 XUs and 101 XVs -> 2 XUVs
2616 XUVs total
779 collection wire objects
2616 potential space points
Neighbour search...
285202 tests to find 134094 neighbours
Iterating with no regularization...
Begin: 2.96256e+06
0 2.66062e+06
1 2.63559e+06
2 2.63273e+06
3 2.63189e+06
Now with regularization...
Begin: 2.48046e+06
0 2.47983e+06
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
no of hits closeby 0 event 845538 TrkackID 0 startx, y, z -32.9083 287.057 439.645 wireno, peakt tpcno 12 1311.06 2 dist 0min T, max_T1305.07 2601.71
no of hits closeby 0 event 845538 TrkackID 0 startx, y, z -32.9083 287.057 439.645 wireno, peakt tpcno 12 1311.06 2 dist 0min T, max_T1305.07 2601.71
Begin processing the 30th record. run: 29107 subRun: 1 event: 845546 at 26-Sep-2024 12:55:33 CEST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 74113 traces, 74113 summary
Retagger: tagging trace set: gauss with 54411 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 54411 traces tagged "gauss"
FrameSaver: q=1.41166e+06 n=1019563 tag=gauss
wclsFrameSaver: saving 74113 traces tagged "wiener"
FrameSaver: q=1.49452e+06 n=917092 tag=wiener
2363 hits in TPC 0
3075 hits in TPC 1
5604 hits in TPC 2
2764 hits in TPC 3
5040 hits in TPC 4
12389 hits in TPC 5
5302 hits in TPC 6
2871 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 447 XUs and 282 XVs -> 31 XUVs
C:0 T:2 10981 XUs and 13952 XVs -> 190 XUVs
C:0 T:3 110 XUs and 86 XVs -> 4 XUVs
C:0 T:4 375 XUs and 435 XVs -> 25 XUVs
C:0 T:5 48767 XUs and 60995 XVs -> 5084 XUVs
C:0 T:6 9675 XUs and 19199 XVs -> 843 XUVs
C:0 T:7 493 XUs and 535 XVs -> 22 XUVs
6199 XUVs total
1219 collection wire objects
6199 potential space points
Neighbour search...
784085 tests to find 383344 neighbours
Iterating with no regularization...
Begin: 5.44392e+07
0 4.71083e+07
1 4.64539e+07
2 4.6381e+07
3 4.63589e+07
Now with regularization...
Begin: 4.54142e+07
0 4.53999e+07
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
26-Sep-2024 12:57:11 CEST Closed output file "np04hd_raw_run029107_3522_dataflow2_datawriter_0_20240909T023153_reco_stage1_reco_stage2_20240926T105711_keepup.root"
26-Sep-2024 12:57:11 CEST Closed input file "np04hd_raw_run029107_3522_dataflow2_datawriter_0_20240909T023153_reco_stage1.root"
========================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
========================================================================================================================================
Full event 59.5921 75.2426 109.572 73.7488 11.9095 30
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.000585734 0.00249199 0.00523658 0.00276051 0.00111392 30
produce:fembfilter:PDHDFEMBFilter 0.49193 0.528406 0.672123 0.504743 0.0508834 30
produce:wclsdatahd:WireCellToolkit 30.7937 32.519 34.135 32.6524 1.0376 30
produce:gaushit:GausHitFinder 0.339499 0.500263 0.724432 0.50133 0.107914 30
produce:nhitsfilter:NumberOfHitsFilter 0.000222065 0.000284401 0.000549387 0.000267545 5.941e-05 30
produce:reco3d:SpacePointSolver 0.186511 0.610453 1.84081 0.490272 0.398824 30
produce:hitpdune:DisambigFromSpacePoints 0.149432 0.327551 0.704684 0.320648 0.138411 30
produce:pandora:StandardPandora 4.59871 8.87875 21.2058 7.98987 3.7092 30
produce:pandoraWriter:StandardPandora 0.079449 0.119595 0.195847 0.117568 0.027402 30
produce:pandoraTrack:LArPandoraTrackCreation 0.531997 1.3051 2.76462 1.16782 0.516324 30
produce:pandoraShower:LArPandoraShowerCreation 0.572193 1.52582 3.76539 1.45025 0.819385 30
produce:pandoracalo:Calorimetry 0.279863 0.600061 1.38643 0.548783 0.233037 30
produce:pandoracalonosce:Calorimetry 0.243808 0.55563 1.29813 0.531333 0.216189 30
produce:pandorapid:Chi2ParticleID 0.00111057 0.0016494 0.00354126 0.00154718 0.000502728 30
produce:pandoraShowercalo:ShowerCalorimetry 0.534694 1.24744 2.97195 1.08916 0.565858 30
produce:pandoraShowercalonosce:ShowerCalorimetry 0.532765 1.17392 2.4092 1.11591 0.498012 30
produce:pandoraGnocchiCalo:GnocchiCalorimetry 0.0643306 0.0933412 0.127778 0.0943673 0.0174312 30
produce:pandoraGnocchiCalonosce:GnocchiCalorimetry 0.0250547 0.0369539 0.0832945 0.0368108 0.0108201 30
produce:emtrkmichelid:EmTrackMichelId 17.3014 23.8584 34.8423 23.1359 4.58127 30
[art]:TriggerResults:TriggerResultInserter 2.7602e-05 6.16759e-05 0.00012301 5.9997e-05 2.06483e-05 30
end_path:out1:RootOutput 7.675e-06 1.77139e-05 3.6729e-05 1.7167e-05 5.69702e-06 30
end_path:michelremoving:michelremoving 0.0225005 0.0811665 0.279185 0.053051 0.0608282 30
end_path:michelremovinggnocchi:michelremoving 0.0208855 0.0716245 0.219254 0.042483 0.0525032 30
end_path:calibanagnocchi:CalibAnaTree 0.229881 0.442783 0.843503 0.421465 0.146965 30
end_path:out1:RootOutput(write) 0.392292 0.555071 0.867924 0.555745 0.108695 30
========================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 4927.84 MB
Peak resident set size usage (VmHWM): 2352.38 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np04hd_raw_run029107_3522_dataflow2_datawriter_0_20240909T023153_reco_stage1_reco_stage2_20240926T105711_keepup.root
\tHists: np04hd_raw_run029107_3522_dataflow2_datawriter_0_20240909T023153_reco_stage1_reco_stage2_20240926T105711_keepup_hists.root
Forming reco metadata
/cvmfs/larsoft.opensciencegrid.org/products/python/v3_9_15/Linux64bit+3.10-2.17/bin/python: No module named meta_maker
Error in reco metadata
