Jobsub ID 263931.158@justin-prod-sched01.dune.hep.ac.uk
| Jobsub ID | 263931.158@justin-prod-sched01.dune.hep.ac.uk |
| Workflow ID | 3448 |
| Stage ID | 1 |
| User name | calcuttj@fnal.gov |
| HTCondor Group | group_dune.prod_mcsim |
| Requested | Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 18000 (5 hours) |
| Submitted time | 2024-09-26 14:23:56 |
| Site | UK_Sheffield |
| Entry | DUNE_UK_Sheffield_lcgce1 |
| Last heartbeat | 2024-09-26 15:54:06 |
| From worker node | Hostname | wn096.hep |
| cpuinfo | Intel(R) Xeon(R) CPU E5-2640 v4 @ 2.40GHz |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 257400 (71 hours) |
| Inner Apptainer? | True |
| Job state | jobscript_error |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk |
| Started | 2024-09-26 14:25:33 |
| Input files | hd-protodune:np04hd_raw_run029097_0132_dataflow5_datawriter_0_20240906T130744.hdf5
|
| Jobscript | Exit code | 1 |
| Real time | 0m (0s) |
| CPU time | 0m (0s = 0%) |
| Outputting started | |
| Output files | |
| Finished | 2024-09-26 15:54:06 |
| Saved logs | justin-logs:263931.158-justin-prod-sched01.dune.hep.ac.uk.logs.tgz |
| List job events Wrapper job log |
Jobscript log (last 10,000 characters)
traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 43824 traces tagged "gauss"
FrameSaver: q=960532 n=787721 tag=gauss
wclsFrameSaver: saving 59481 traces tagged "wiener"
FrameSaver: q=1.0484e+06 n=710405 tag=wiener
4404 hits in TPC 0
3564 hits in TPC 1
3245 hits in TPC 2
2114 hits in TPC 3
5224 hits in TPC 4
5989 hits in TPC 5
3594 hits in TPC 6
2670 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 1049 XUs and 748 XVs -> 17 XUVs
C:0 T:2 10191 XUs and 16938 XVs -> 496 XUVs
C:0 T:3 0 XUs and 0 XVs -> 0 XUVs
C:0 T:4 1435 XUs and 1281 XVs -> 132 XUVs
C:0 T:5 23801 XUs and 17518 XVs -> 470 XUVs
C:0 T:6 13036 XUs and 34633 XVs -> 2944 XUVs
C:0 T:7 684 XUs and 625 XVs -> 30 XUVs
4089 XUVs total
1040 collection wire objects
4089 potential space points
Neighbour search...
543117 tests to find 238830 neighbours
Iterating with no regularization...
Begin: 7.91507e+06
0 7.26957e+06
1 7.17716e+06
2 7.17028e+06
Now with regularization...
Begin: 6.84667e+06
0 6.84442e+06
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Begin processing the 29th record. run: 29097 subRun: 1 event: 31957 at 26-Sep-2024 16:50:06 BST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 42267 traces, 42267 summary
Retagger: tagging trace set: gauss with 32037 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 32037 traces tagged "gauss"
FrameSaver: q=661635 n=603744 tag=gauss
wclsFrameSaver: saving 42267 traces tagged "wiener"
FrameSaver: q=718812 n=548558 tag=wiener
2385 hits in TPC 0
2087 hits in TPC 1
2803 hits in TPC 2
1970 hits in TPC 3
3755 hits in TPC 4
4505 hits in TPC 5
3399 hits in TPC 6
1890 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 393 XUs and 291 XVs -> 1 XUVs
C:0 T:2 14445 XUs and 28883 XVs -> 1878 XUVs
C:0 T:3 2 XUs and 6 XVs -> 0 XUVs
C:0 T:4 806 XUs and 1009 XVs -> 57 XUVs
C:0 T:5 14198 XUs and 45222 XVs -> 1672 XUVs
C:0 T:6 18772 XUs and 18224 XVs -> 513 XUVs
C:0 T:7 63 XUs and 94 XVs -> 2 XUVs
4123 XUVs total
1261 collection wire objects
4123 potential space points
Neighbour search...
337245 tests to find 163504 neighbours
Iterating with no regularization...
Begin: 6.11277e+06
0 5.41291e+06
1 5.35544e+06
2 5.34913e+06
3 5.34746e+06
Now with regularization...
Begin: 5.06309e+06
0 5.06221e+06
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Begin processing the 30th record. run: 29097 subRun: 1 event: 31965 at 26-Sep-2024 16:51:45 BST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 56003 traces, 56003 summary
Retagger: tagging trace set: gauss with 42560 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 42560 traces tagged "gauss"
FrameSaver: q=980592 n=862275 tag=gauss
wclsFrameSaver: saving 56003 traces tagged "wiener"
FrameSaver: q=1.05241e+06 n=794354 tag=wiener
3405 hits in TPC 0
2692 hits in TPC 1
6574 hits in TPC 2
3185 hits in TPC 3
2077 hits in TPC 4
3001 hits in TPC 5
6342 hits in TPC 6
3945 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 126 XUs and 223 XVs -> 12 XUVs
C:0 T:2 25846 XUs and 31003 XVs -> 769 XUVs
C:0 T:3 146 XUs and 237 XVs -> 15 XUVs
C:0 T:4 55 XUs and 76 XVs -> 3 XUVs
C:0 T:5 3318 XUs and 5565 XVs -> 286 XUVs
C:0 T:6 44189 XUs and 64640 XVs -> 3515 XUVs
C:0 T:7 1196 XUs and 1140 XVs -> 36 XUVs
4636 XUVs total
1488 collection wire objects
4636 potential space points
Neighbour search...
275118 tests to find 152914 neighbours
Iterating with no regularization...
Begin: 9.0227e+06
0 8.35476e+06
1 8.30334e+06
2 8.29262e+06
3 8.28928e+06
Now with regularization...
Begin: 7.77978e+06
0 7.77735e+06
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
26-Sep-2024 16:53:42 BST Closed output file "np04hd_raw_run029097_0132_dataflow5_datawriter_0_20240906T130744_reco_stage1_reco_stage2_20240926T155342_keepup.root"
26-Sep-2024 16:53:42 BST Closed input file "np04hd_raw_run029097_0132_dataflow5_datawriter_0_20240906T130744_reco_stage1.root"
========================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
========================================================================================================================================
Full event 85.6048 116.992 163.979 114.198 18.0731 30
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.00100042 0.00302834 0.00453825 0.0032359 0.000928181 30
produce:fembfilter:PDHDFEMBFilter 0.68975 0.746623 0.878832 0.745029 0.0377758 30
produce:wclsdatahd:WireCellToolkit 52.8714 57.3783 60.6068 57.7839 1.69274 30
produce:gaushit:GausHitFinder 0.4444 0.75553 1.16511 0.774968 0.163948 30
produce:nhitsfilter:NumberOfHitsFilter 0.000210424 0.000382376 0.000563258 0.000372893 8.05484e-05 30
produce:reco3d:SpacePointSolver 0.147887 0.869808 2.74195 0.753432 0.56468 30
produce:hitpdune:DisambigFromSpacePoints 0.160025 0.546793 1.03667 0.502719 0.247516 30
produce:pandora:StandardPandora 5.13215 14.1985 30.4104 12.8914 6.04355 30
produce:pandoraWriter:StandardPandora 0.0800806 0.161034 0.27929 0.16462 0.040152 30
produce:pandoraTrack:LArPandoraTrackCreation 0.572928 1.50844 3.39621 1.61092 0.602523 30
produce:pandoraShower:LArPandoraShowerCreation 0.489856 1.82856 4.86002 1.66352 1.01536 30
produce:pandoracalo:Calorimetry 0.30418 0.761442 1.41381 0.781445 0.268775 30
produce:pandoracalonosce:Calorimetry 0.264164 0.69049 1.24085 0.705217 0.239682 30
produce:pandorapid:Chi2ParticleID 0.00144748 0.0020402 0.00369301 0.00204824 0.000417733 30
produce:pandoraShowercalo:ShowerCalorimetry 0.447103 1.66918 3.94093 1.49949 0.868002 30
produce:pandoraShowercalonosce:ShowerCalorimetry 0.436434 1.62223 3.81166 1.48985 0.828238 30
produce:pandoraGnocchiCalo:GnocchiCalorimetry 0.0841178 0.140105 0.196568 0.140832 0.0284114 30
produce:pandoraGnocchiCalonosce:GnocchiCalorimetry 0.0305903 0.0503513 0.0689251 0.0482896 0.010285 30
produce:emtrkmichelid:EmTrackMichelId 19.4024 32.436 50.7402 32.6131 7.70506 30
[art]:TriggerResults:TriggerResultInserter 2.6136e-05 3.73504e-05 8.9163e-05 3.52555e-05 1.21085e-05 30
end_path:out1:RootOutput 9.373e-06 1.25156e-05 2.6664e-05 1.1536e-05 3.66211e-06 30
end_path:michelremoving:michelremoving 0.0272242 0.0750684 0.343656 0.049302 0.0681173 30
end_path:michelremovinggnocchi:michelremoving 0.0232359 0.0684636 0.33579 0.0426796 0.0674989 30
end_path:calibanagnocchi:CalibAnaTree 0.227303 0.561968 1.00859 0.580385 0.178446 30
end_path:out1:RootOutput(write) 0.520561 0.799246 1.10805 0.811791 0.162221 30
========================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 4957 MB
Peak resident set size usage (VmHWM): 2347.81 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np04hd_raw_run029097_0132_dataflow5_datawriter_0_20240906T130744_reco_stage1_reco_stage2_20240926T155342_keepup.root
\tHists: np04hd_raw_run029097_0132_dataflow5_datawriter_0_20240906T130744_reco_stage1_reco_stage2_20240926T155342_keepup_hists.root
Forming reco metadata
/cvmfs/larsoft.opensciencegrid.org/products/python/v3_9_15/Linux64bit+3.10-2.17/bin/python: No module named meta_maker
Error in reco metadata
