Jobsub ID 263757.103@justin-prod-sched01.dune.hep.ac.uk
| Jobsub ID | 263757.103@justin-prod-sched01.dune.hep.ac.uk |
| Workflow ID | 3448 |
| Stage ID | 1 |
| User name | calcuttj@fnal.gov |
| HTCondor Group | group_dune.prod_mcsim |
| Requested | Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 18000 (5 hours) |
| Submitted time | 2024-09-26 13:23:33 |
| Site | UK_Durham |
| Entry | DUNE_UK_SGridDurham_ce3 |
| Last heartbeat | 2024-09-26 15:23:33 |
| From worker node | Hostname | n187.dur.scotgrid.ac.uk |
| cpuinfo | Intel(R) Xeon(R) Gold 5220 CPU @ 2.20GHz |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 171000 (47 hours) |
| Inner Apptainer? | True |
| Job state | jobscript_error |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk |
| Started | 2024-09-26 13:25:03 |
| Input files | hd-protodune:np04hd_raw_run029092_0926_dataflow7_datawriter_0_20240906T075019.hdf5
|
| Jobscript | Exit code | 1 |
| Real time | 0m (0s) |
| CPU time | 0m (0s = 0%) |
| Outputting started | |
| Output files | |
| Finished | 2024-09-26 15:23:33 |
| Saved logs | justin-logs:263757.103-justin-prod-sched01.dune.hep.ac.uk.logs.tgz |
| List job events Wrapper job log |
Jobscript log (last 10,000 characters)
14 XUVs
C:0 T:5 12610 XUs and 12146 XVs -> 152 XUVs
C:0 T:6 5943 XUs and 7290 XVs -> 266 XUVs
C:0 T:7 81 XUs and 88 XVs -> 8 XUVs
837 XUVs total
441 collection wire objects
837 potential space points
Neighbour search...
20925 tests to find 13108 neighbours
Iterating with no regularization...
Begin: 2.7051e+06
0 2.57421e+06
1 2.57058e+06
2 2.5701e+06
Now with regularization...
Begin: 2.42441e+06
0 2.42425e+06
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
no of hits closeby 0 event 222463 TrkackID 3 startx, y, z 84.1487 598.644 99.1342 wireno, peakt tpcno 257 3017.95 2 dist 387.227min T, max_T1238.66 3073.64
no of hits closeby 0 event 222463 TrkackID 3 startx, y, z 84.1487 598.644 99.1342 wireno, peakt tpcno 257 3017.95 2 dist 387.227min T, max_T1238.66 3073.64
Begin processing the 29th record. run: 29092 subRun: 1 event: 222471 at 26-Sep-2024 16:18:13 BST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 69834 traces, 69834 summary
Retagger: tagging trace set: gauss with 51143 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 51143 traces tagged "gauss"
FrameSaver: q=1.16325e+06 n=956853 tag=gauss
wclsFrameSaver: saving 69834 traces tagged "wiener"
FrameSaver: q=1.27064e+06 n=877730 tag=wiener
3315 hits in TPC 0
3865 hits in TPC 1
7607 hits in TPC 2
3996 hits in TPC 3
4298 hits in TPC 4
6239 hits in TPC 5
3550 hits in TPC 6
2422 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 153 XUs and 123 XVs -> 6 XUVs
C:0 T:2 27393 XUs and 81985 XVs -> 4516 XUVs
C:0 T:3 14 XUs and 20 XVs -> 3 XUVs
C:0 T:4 80 XUs and 88 XVs -> 1 XUVs
C:0 T:5 22349 XUs and 30828 XVs -> 3382 XUVs
C:0 T:6 9614 XUs and 6858 XVs -> 166 XUVs
C:0 T:7 395 XUs and 641 XVs -> 9 XUVs
8083 XUVs total
1817 collection wire objects
8083 potential space points
Neighbour search...
672741 tests to find 396316 neighbours
Iterating with no regularization...
Begin: 7.50562e+06
0 6.0675e+06
1 5.9765e+06
2 5.95591e+06
3 5.94814e+06
4 5.94436e+06
Now with regularization...
Begin: 5.4499e+06
0 5.44621e+06
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Begin processing the 30th record. run: 29092 subRun: 1 event: 222479 at 26-Sep-2024 16:20:21 BST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 78310 traces, 78310 summary
Retagger: tagging trace set: gauss with 59460 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 59460 traces tagged "gauss"
FrameSaver: q=1.24972e+06 n=1124379 tag=gauss
wclsFrameSaver: saving 78310 traces tagged "wiener"
FrameSaver: q=1.3443e+06 n=1019831 tag=wiener
4304 hits in TPC 0
3894 hits in TPC 1
12115 hits in TPC 2
4435 hits in TPC 3
2052 hits in TPC 4
4462 hits in TPC 5
8461 hits in TPC 6
3529 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 746 XUs and 1722 XVs -> 72 XUVs
C:0 T:2 49406 XUs and 88565 XVs -> 2579 XUVs
C:0 T:3 803 XUs and 967 XVs -> 44 XUVs
C:0 T:4 21 XUs and 13 XVs -> 1 XUVs
C:0 T:5 11240 XUs and 15069 XVs -> 396 XUVs
C:0 T:6 49642 XUs and 63336 XVs -> 2212 XUVs
C:0 T:7 548 XUs and 602 XVs -> 13 XUVs
5317 XUVs total
2243 collection wire objects
5317 potential space points
Neighbour search...
274917 tests to find 147606 neighbours
Iterating with no regularization...
Begin: 1.17084e+07
0 1.03433e+07
1 1.0281e+07
2 1.02708e+07
Now with regularization...
Begin: 9.69632e+06
0 9.69235e+06
BdtBeamParticleIdTool::SliceFeatures::GetLArTPCIntercepts - inconsistent number of intercepts between a line and the LArTPC
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
26-Sep-2024 16:22:59 BST Closed output file "np04hd_raw_run029092_0926_dataflow7_datawriter_0_20240906T075019_reco_stage1_reco_stage2_20240926T152259_keepup.root"
26-Sep-2024 16:22:59 BST Closed input file "np04hd_raw_run029092_0926_dataflow7_datawriter_0_20240906T075019_reco_stage1.root"
========================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
========================================================================================================================================
Full event 95.9397 127.686 171.488 125.334 20.1816 30
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.00034337 0.00136364 0.00353359 0.00105073 0.000882857 30
produce:fembfilter:PDHDFEMBFilter 0.790836 1.09553 1.17816 1.10553 0.0764978 30
produce:wclsdatahd:WireCellToolkit 62.8007 64.7922 66.6872 65.073 0.976567 30
produce:gaushit:GausHitFinder 0.661323 1.17899 1.8473 1.13868 0.269729 30
produce:nhitsfilter:NumberOfHitsFilter 0.000325772 0.000484253 0.000717477 0.000485527 9.01732e-05 30
produce:reco3d:SpacePointSolver 0.25583 1.45029 3.62783 1.3101 0.941761 30
produce:hitpdune:DisambigFromSpacePoints 0.274347 1.04344 2.21684 0.991132 0.46898 30
produce:pandora:StandardPandora 7.85503 20.2488 36.7651 18.6814 8.10621 30
produce:pandoraWriter:StandardPandora 0.131385 0.227181 0.328474 0.225216 0.0518252 30
produce:pandoraTrack:LArPandoraTrackCreation 0.667018 1.99287 4.17257 1.89937 0.821734 30
produce:pandoraShower:LArPandoraShowerCreation 0.524015 2.31066 5.3877 2.17936 1.18576 30
produce:pandoracalo:Calorimetry 0.349739 0.929723 1.63073 0.887099 0.32532 30
produce:pandoracalonosce:Calorimetry 0.308115 0.848316 1.53048 0.813523 0.308171 30
produce:pandorapid:Chi2ParticleID 0.00185432 0.00278357 0.00480434 0.00272113 0.000582076 30
produce:pandoraShowercalo:ShowerCalorimetry 0.462514 2.0287 4.45115 1.98213 0.97677 30
produce:pandoraShowercalonosce:ShowerCalorimetry 0.460936 1.99991 4.34738 1.94243 0.956167 30
produce:pandoraGnocchiCalo:GnocchiCalorimetry 0.117159 0.214032 0.305876 0.222012 0.0501384 30
produce:pandoraGnocchiCalonosce:GnocchiCalorimetry 0.0463382 0.0844076 0.121763 0.0866369 0.0193972 30
produce:emtrkmichelid:EmTrackMichelId 15.1618 24.8316 36.1492 24.358 5.39885 30
[art]:TriggerResults:TriggerResultInserter 3.6092e-05 0.000155614 0.000504573 4.7199e-05 0.000182694 30
end_path:out1:RootOutput 8.805e-06 1.77565e-05 0.000128386 1.14755e-05 2.11861e-05 30
end_path:michelremoving:michelremoving 0.0228773 0.143123 0.504677 0.0654272 0.14057 30
end_path:michelremovinggnocchi:michelremoving 0.0204499 0.143293 0.502839 0.0544782 0.145778 30
end_path:calibanagnocchi:CalibAnaTree 0.229458 0.705111 1.36076 0.692271 0.275977 30
end_path:out1:RootOutput(write) 0.714115 1.23773 1.93778 1.17981 0.36004 30
========================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 5779.33 MB
Peak resident set size usage (VmHWM): 2806.69 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np04hd_raw_run029092_0926_dataflow7_datawriter_0_20240906T075019_reco_stage1_reco_stage2_20240926T152259_keepup.root
\tHists: np04hd_raw_run029092_0926_dataflow7_datawriter_0_20240906T075019_reco_stage1_reco_stage2_20240926T152259_keepup_hists.root
Forming reco metadata
/cvmfs/larsoft.opensciencegrid.org/products/python/v3_9_15/Linux64bit+3.10-2.17/bin/python: No module named meta_maker
Error in reco metadata
