Jobsub ID 263865.2@justin-prod-sched01.dune.hep.ac.uk
| Jobsub ID | 263865.2@justin-prod-sched01.dune.hep.ac.uk |
| Workflow ID | 3448 |
| Stage ID | 1 |
| User name | calcuttj@fnal.gov |
| HTCondor Group | group_dune.prod_mcsim |
| Requested | Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 18000 (5 hours) |
| Submitted time | 2024-09-26 13:56:43 |
| Site | IT_CNAF |
| Entry | CMSHTPC_T1_IT_CNAF_condor_ce02 |
| Last heartbeat | 2024-09-26 15:07:28 |
| From worker node | Hostname | cn-leo-005.cr.cnaf.infn.it |
| cpuinfo | Intel(R) Xeon(R) Platinum 8480+ |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 216000 (60 hours) |
| Inner Apptainer? | True |
| Job state | jobscript_error |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk |
| Started | 2024-09-26 13:57:20 |
| Input files | hd-protodune:np04hd_raw_run029094_0020_dataflow3_datawriter_0_20240906T084225.hdf5
|
| Jobscript | Exit code | 1 |
| Real time | 0m (0s) |
| CPU time | 0m (0s = 0%) |
| Outputting started | |
| Output files | |
| Finished | 2024-09-26 15:07:28 |
| Saved logs | justin-logs:263865.2-justin-prod-sched01.dune.hep.ac.uk.logs.tgz |
| List job events Wrapper job log |
Jobscript log (last 10,000 characters)
Begin: 2.51348e+07
0 2.51099e+07
BdtBeamParticleIdTool::SliceFeatures::GetLArTPCIntercepts - inconsistent number of intercepts between a line and the LArTPC
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
no of hits closeby 21 event 5019 TrkackID 3 startx, y, z 34.3601 592.226 368.553 wireno, peakt tpcno 369 1140 5 dist 0min T, max_T1140 3869.83
no of hits closeby 21 event 5019 TrkackID 3 startx, y, z 34.3601 592.226 368.553 wireno, peakt tpcno 369 1140 5 dist 0min T, max_T1140 3869.83
Begin processing the 29th record. run: 29094 subRun: 1 event: 5027 at 26-Sep-2024 17:03:25 CEST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 36319 traces, 36319 summary
Retagger: tagging trace set: gauss with 27300 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 27300 traces tagged "gauss"
FrameSaver: q=527768 n=509523 tag=gauss
wclsFrameSaver: saving 36319 traces tagged "wiener"
FrameSaver: q=563525 n=461787 tag=wiener
3068 hits in TPC 0
2927 hits in TPC 1
1521 hits in TPC 2
1243 hits in TPC 3
2833 hits in TPC 4
2605 hits in TPC 5
3295 hits in TPC 6
1957 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 335 XUs and 159 XVs -> 2 XUVs
C:0 T:2 4129 XUs and 6321 XVs -> 105 XUVs
C:0 T:3 33 XUs and 33 XVs -> 3 XUVs
C:0 T:4 66 XUs and 95 XVs -> 7 XUVs
C:0 T:5 7058 XUs and 5789 XVs -> 287 XUVs
C:0 T:6 8845 XUs and 9585 XVs -> 375 XUVs
C:0 T:7 52 XUs and 39 XVs -> 2 XUVs
781 XUVs total
420 collection wire objects
781 potential space points
Neighbour search...
18791 tests to find 11846 neighbours
Iterating with no regularization...
Begin: 1.37377e+06
0 1.32561e+06
1 1.3225e+06
2 1.32132e+06
Now with regularization...
Begin: 1.17681e+06
0 1.17619e+06
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Begin processing the 30th record. run: 29094 subRun: 1 event: 5035 at 26-Sep-2024 17:04:44 CEST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 80196 traces, 80196 summary
Retagger: tagging trace set: gauss with 60124 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 60124 traces tagged "gauss"
FrameSaver: q=1.21264e+06 n=1045604 tag=gauss
wclsFrameSaver: saving 80196 traces tagged "wiener"
FrameSaver: q=1.31739e+06 n=939769 tag=wiener
5387 hits in TPC 0
3482 hits in TPC 1
6269 hits in TPC 2
5258 hits in TPC 3
5583 hits in TPC 4
7532 hits in TPC 5
4596 hits in TPC 6
2601 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 680 XUs and 377 XVs -> 12 XUVs
C:0 T:2 19368 XUs and 25347 XVs -> 628 XUVs
C:0 T:3 241 XUs and 325 XVs -> 9 XUVs
C:0 T:4 106 XUs and 104 XVs -> 5 XUVs
C:0 T:5 36919 XUs and 32513 XVs -> 1339 XUVs
C:0 T:6 9658 XUs and 10113 XVs -> 206 XUVs
C:0 T:7 25 XUs and 61 XVs -> 3 XUVs
2202 XUVs total
1306 collection wire objects
2202 potential space points
Neighbour search...
50600 tests to find 27514 neighbours
Iterating with no regularization...
Begin: 7.71446e+06
0 6.89297e+06
1 6.84918e+06
2 6.84567e+06
Now with regularization...
Begin: 6.28665e+06
0 6.28553e+06
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
no of hits closeby 0 event 5035 TrkackID 0 startx, y, z 284.863 322.304 1.8744 wireno, peakt tpcno 457 4737.45 5 dist 0min T, max_T1227.14 4868.19
no of hits closeby 0 event 5035 TrkackID 0 startx, y, z 284.863 322.304 1.8744 wireno, peakt tpcno 457 4737.45 5 dist 0min T, max_T1227.14 4868.19
26-Sep-2024 17:06:37 CEST Closed output file "np04hd_raw_run029094_0020_dataflow3_datawriter_0_20240906T084225_reco_stage1_reco_stage2_20240926T150637_keepup.root"
26-Sep-2024 17:06:37 CEST Closed input file "np04hd_raw_run029094_0020_dataflow3_datawriter_0_20240906T084225_reco_stage1.root"
========================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
========================================================================================================================================
Full event 64.5143 93.9754 122.101 93.6068 13.1455 30
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.000441777 0.00224222 0.0076432 0.00171043 0.00170023 30
produce:fembfilter:PDHDFEMBFilter 0.66562 0.708261 0.76132 0.702056 0.020456 30
produce:wclsdatahd:WireCellToolkit 56.5125 59.7155 64.2454 59.4583 1.54561 30
produce:gaushit:GausHitFinder 0.287775 0.511994 1.14099 0.496732 0.16402 30
produce:nhitsfilter:NumberOfHitsFilter 0.000307283 0.000500184 0.000807373 0.00049145 0.0001124 30
produce:reco3d:SpacePointSolver 0.11292 0.540318 1.1208 0.507381 0.270964 29
produce:hitpdune:DisambigFromSpacePoints 0.0944332 0.26167 0.516624 0.222735 0.107045 29
produce:pandora:StandardPandora 2.68638 7.74768 14.8279 6.58937 3.36145 29
produce:pandoraWriter:StandardPandora 0.0875036 0.144968 0.215773 0.139852 0.0351555 29
produce:pandoraTrack:LArPandoraTrackCreation 0.581882 1.97214 4.05354 1.94236 0.877942 29
produce:pandoraShower:LArPandoraShowerCreation 0.556532 2.38522 4.49991 2.28868 1.12633 29
produce:pandoracalo:Calorimetry 0.271712 0.847283 2.75685 0.776611 0.471285 29
produce:pandoracalonosce:Calorimetry 0.239131 0.7445 1.3984 0.74225 0.298187 29
produce:pandorapid:Chi2ParticleID 0.000897905 0.00152873 0.00266918 0.00147875 0.000361176 29
produce:pandoraShowercalo:ShowerCalorimetry 0.430123 1.87947 3.64232 1.81451 0.899524 29
produce:pandoraShowercalonosce:ShowerCalorimetry 0.4439 1.83786 3.73254 1.70341 0.889367 29
produce:pandoraGnocchiCalo:GnocchiCalorimetry 0.0427514 0.0867759 0.120712 0.0873689 0.0221649 29
produce:pandoraGnocchiCalonosce:GnocchiCalorimetry 0.0195407 0.0393796 0.0589658 0.0399676 0.0105401 29
produce:emtrkmichelid:EmTrackMichelId 7.53086 13.9914 19.9148 14.5376 3.45722 29
[art]:TriggerResults:TriggerResultInserter 4.748e-05 6.89644e-05 0.00010341 6.58085e-05 1.39602e-05 30
end_path:out1:RootOutput 1.5539e-05 2.23214e-05 3.7404e-05 2.00105e-05 5.48557e-06 30
end_path:michelremoving:michelremoving 6.5991e-05 0.0825544 0.304625 0.0587494 0.0593313 30
end_path:michelremovinggnocchi:michelremoving 5.2641e-05 0.0802199 0.303477 0.0654973 0.0586785 30
end_path:calibanagnocchi:CalibAnaTree 4.6903e-05 0.586709 1.14628 0.573731 0.243938 30
end_path:out1:RootOutput(write) 0.295906 0.551615 0.823788 0.536698 0.144028 30
========================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 6026.11 MB
Peak resident set size usage (VmHWM): 2695.89 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np04hd_raw_run029094_0020_dataflow3_datawriter_0_20240906T084225_reco_stage1_reco_stage2_20240926T150637_keepup.root
\tHists: np04hd_raw_run029094_0020_dataflow3_datawriter_0_20240906T084225_reco_stage1_reco_stage2_20240926T150637_keepup_hists.root
Forming reco metadata
/cvmfs/larsoft.opensciencegrid.org/products/python/v3_9_15/Linux64bit+3.10-2.17/bin/python: No module named meta_maker
Error in reco metadata
