Jobsub ID 263290.25@justin-prod-sched01.dune.hep.ac.uk
| Jobsub ID | 263290.25@justin-prod-sched01.dune.hep.ac.uk |
| Workflow ID | 3461 |
| Stage ID | 1 |
| User name | calcuttj@fnal.gov |
| HTCondor Group | group_dune.prod_mcsim |
| Requested | Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 18000 (5 hours) |
| Submitted time | 2024-09-26 09:57:08 |
| Site | UK_Sheffield |
| Entry | DUNE_UK_Sheffield_lcgce1 |
| Last heartbeat | 2024-09-26 11:24:29 |
| From worker node | Hostname | wn031.hep |
| cpuinfo | Intel(R) Xeon(R) Gold 6226 CPU @ 2.70GHz |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 257400 (71 hours) |
| Inner Apptainer? | True |
| Job state | jobscript_error |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk |
| Started | 2024-09-26 09:59:00 |
| Input files | hd-protodune:np04hd_raw_run027298_0067_dataflow1_datawriter_0_20240619T142840.hdf5
|
| Jobscript | Exit code | 1 |
| Real time | 0m (0s) |
| CPU time | 0m (0s = 0%) |
| Outputting started | |
| Output files | |
| Finished | 2024-09-26 11:24:29 |
| Saved logs | justin-logs:263290.25-justin-prod-sched01.dune.hep.ac.uk.logs.tgz |
| List job events Wrapper job log |
Jobscript log (last 10,000 characters)
...
C:0 T:0 672 XUs and 45 XVs -> 8 XUVs
C:0 T:1 47743 XUs and 7493 XVs -> 1111 XUVs
C:0 T:2 3705 XUs and 8076 XVs -> 173 XUVs
C:0 T:3 73 XUs and 97 XVs -> 2 XUVs
C:0 T:4 237 XUs and 252 XVs -> 10 XUVs
C:0 T:5 48779 XUs and 45018 XVs -> 1251 XUVs
C:0 T:6 35986 XUs and 46821 XVs -> 1992 XUVs
C:0 T:7 1309 XUs and 1579 XVs -> 40 XUVs
4587 XUVs total
1985 collection wire objects
4587 potential space points
Neighbour search...
144085 tests to find 69832 neighbours
Iterating with no regularization...
Begin: 5.18331e+07
0 4.94398e+07
1 4.92254e+07
2 4.92016e+07
Now with regularization...
Begin: 4.69422e+07
0 4.6936e+07
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Begin processing the 34th record. run: 27298 subRun: 1 event: 9513 at 26-Sep-2024 12:14:09 BST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 52249 traces, 52249 summary
Retagger: tagging trace set: gauss with 40884 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 40884 traces tagged "gauss"
FrameSaver: q=1.63696e+06 n=805323 tag=gauss
wclsFrameSaver: saving 52249 traces tagged "wiener"
FrameSaver: q=1.73556e+06 n=736701 tag=wiener
1324 hits in TPC 0
3397 hits in TPC 1
6844 hits in TPC 2
3401 hits in TPC 3
2236 hits in TPC 4
5364 hits in TPC 5
6041 hits in TPC 6
4278 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 2386 XUs and 1023 XVs -> 14 XUVs
C:0 T:2 17700 XUs and 17771 XVs -> 735 XUVs
C:0 T:3 30 XUs and 31 XVs -> 0 XUVs
C:0 T:4 28 XUs and 42 XVs -> 4 XUVs
C:0 T:5 10703 XUs and 10759 XVs -> 594 XUVs
C:0 T:6 33150 XUs and 20453 XVs -> 726 XUVs
C:0 T:7 390 XUs and 312 XVs -> 14 XUVs
2087 XUVs total
1306 collection wire objects
2087 potential space points
Neighbour search...
38025 tests to find 18758 neighbours
Iterating with no regularization...
Begin: 4.06925e+07
0 3.79553e+07
1 3.78132e+07
2 3.7806e+07
Now with regularization...
Begin: 3.52696e+07
0 3.52617e+07
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Begin processing the 35th record. run: 27298 subRun: 1 event: 9517 at 26-Sep-2024 12:15:48 BST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 55567 traces, 55567 summary
Retagger: tagging trace set: gauss with 39290 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 39290 traces tagged "gauss"
FrameSaver: q=1.34752e+06 n=678769 tag=gauss
wclsFrameSaver: saving 55567 traces tagged "wiener"
FrameSaver: q=1.47204e+06 n=606574 tag=wiener
2657 hits in TPC 0
2498 hits in TPC 1
5677 hits in TPC 2
3599 hits in TPC 3
1344 hits in TPC 4
1971 hits in TPC 5
6157 hits in TPC 6
5491 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:0 0 XUs and 0 XVs -> 0 XUVs
C:0 T:1 1840 XUs and 1792 XVs -> 91 XUVs
C:0 T:2 9276 XUs and 25368 XVs -> 2002 XUVs
C:0 T:3 68 XUs and 221 XVs -> 2 XUVs
C:0 T:4 61 XUs and 62 XVs -> 9 XUVs
C:0 T:5 1567 XUs and 1800 XVs -> 57 XUVs
C:0 T:6 24395 XUs and 44913 XVs -> 722 XUVs
C:0 T:7 1031 XUs and 1892 XVs -> 32 XUVs
2915 XUVs total
1425 collection wire objects
2915 potential space points
Neighbour search...
47423 tests to find 28744 neighbours
Iterating with no regularization...
Begin: 1.57315e+07
0 1.41865e+07
1 1.40986e+07
2 1.40912e+07
Now with regularization...
Begin: 1.25149e+07
0 1.25135e+07
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
no of hits closeby 0 event 9517 TrkackID 8 startx, y, z 78.6521 605.298 9.90563 wireno, peakt tpcno 204 2402.74 2 dist 59.7578min T, max_T1415.25 2440.52
no of hits closeby 0 event 9517 TrkackID 8 startx, y, z 78.6521 605.298 9.90563 wireno, peakt tpcno 204 2402.74 2 dist 59.7578min T, max_T1415.25 2440.52
26-Sep-2024 12:17:17 BST Closed output file "np04hd_raw_run027298_0067_dataflow1_datawriter_0_20240619T142840_reco_stage1_reco_stage2_20240926T111717_keepup.root"
26-Sep-2024 12:17:17 BST Closed input file "np04hd_raw_run027298_0067_dataflow1_datawriter_0_20240619T142840_reco_stage1.root"
========================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
========================================================================================================================================
Full event 66.7021 95.5774 134.364 94.2874 14.9346 35
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.000489403 0.00228269 0.00563192 0.00187854 0.00135895 35
produce:fembfilter:PDHDFEMBFilter 0.665038 0.922197 3.38559 0.788473 0.475418 35
produce:wclsdatahd:WireCellToolkit 44.5963 48.8746 51.3224 48.8635 1.50638 35
produce:gaushit:GausHitFinder 0.420904 0.728129 1.06491 0.717015 0.158523 35
produce:nhitsfilter:NumberOfHitsFilter 0.0003217 0.000430926 0.00055115 0.000420992 6.14232e-05 35
produce:reco3d:SpacePointSolver 0.167963 0.771005 2.1281 0.638673 0.462043 35
produce:hitpdune:DisambigFromSpacePoints 0.152244 0.547688 1.00184 0.466246 0.217567 35
produce:pandora:StandardPandora 5.97503 13.6105 25.409 12.0505 5.16711 35
produce:pandoraWriter:StandardPandora 0.0989435 0.162012 0.220815 0.159011 0.0313554 35
produce:pandoraTrack:LArPandoraTrackCreation 0.593359 3.25506 8.65974 3.19631 1.46895 35
produce:pandoraShower:LArPandoraShowerCreation 0.554325 3.51602 9.29209 2.98822 1.89234 35
produce:pandoracalo:Calorimetry 0.31749 1.19082 2.69143 1.14549 0.468847 35
produce:pandoracalonosce:Calorimetry 0.274057 1.10846 2.44529 1.08706 0.430836 35
produce:pandorapid:Chi2ParticleID 0.00117907 0.00195486 0.00288965 0.001929 0.00034245 35
produce:pandoraShowercalo:ShowerCalorimetry 0.457803 2.57409 5.2802 2.54292 1.18531 35
produce:pandoraShowercalonosce:ShowerCalorimetry 0.431889 2.5199 5.07754 2.43111 1.10839 35
produce:pandoraGnocchiCalo:GnocchiCalorimetry 0.0597789 0.112683 0.154484 0.110589 0.0230664 35
produce:pandoraGnocchiCalonosce:GnocchiCalorimetry 0.0223193 0.0462709 0.0631933 0.0467271 0.00979769 35
produce:emtrkmichelid:EmTrackMichelId 8.90842 13.7529 18.214 13.2649 2.52822 35
[art]:TriggerResults:TriggerResultInserter 3.1919e-05 5.02495e-05 7.9473e-05 4.7356e-05 1.06167e-05 35
end_path:out1:RootOutput 9.625e-06 1.61153e-05 2.8325e-05 1.5646e-05 3.87663e-06 35
end_path:michelremoving:michelremoving 0.0246526 0.0745747 0.20346 0.0555351 0.0472284 35
end_path:michelremovinggnocchi:michelremoving 0.0226741 0.0692078 0.191531 0.0500043 0.0445817 35
end_path:calibanagnocchi:CalibAnaTree 0.142999 0.789072 1.75119 0.782608 0.308588 35
end_path:out1:RootOutput(write) 0.398965 0.683689 0.925672 0.642878 0.135705 35
========================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 5787.03 MB
Peak resident set size usage (VmHWM): 2597.22 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np04hd_raw_run027298_0067_dataflow1_datawriter_0_20240619T142840_reco_stage1_reco_stage2_20240926T111717_keepup.root
\tHists: np04hd_raw_run027298_0067_dataflow1_datawriter_0_20240619T142840_reco_stage1_reco_stage2_20240926T111717_keepup_hists.root
Forming reco metadata
/cvmfs/larsoft.opensciencegrid.org/products/python/v3_9_15/Linux64bit+3.10-2.17/bin/python: No module named meta_maker
Error in reco metadata
