Jobsub ID 78462.117@justin-prod-sched02.dune.hep.ac.uk
| Jobsub ID | 78462.117@justin-prod-sched02.dune.hep.ac.uk |
| Workflow ID | 3454 |
| Stage ID | 1 |
| User name | calcuttj@fnal.gov |
| HTCondor Group | group_dune.prod_mcsim |
| Requested | Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 18000 (5 hours) |
| Submitted time | 2024-09-26 09:58:37 |
| Site | UK_Sheffield |
| Entry | DUNE_UK_Sheffield_lcgce1 |
| Last heartbeat | 2024-09-26 12:16:49 |
| From worker node | Hostname | wn087.hep |
| cpuinfo | Intel(R) Xeon(R) CPU E5-2650 v2 @ 2.60GHz |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 257400 (71 hours) |
| Inner Apptainer? | True |
| Job state | outputting_failed |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk |
| Started | 2024-09-26 10:16:22 |
| Input files | hd-protodune:np04hd_raw_run029108_0270_dataflow2_datawriter_0_20240909T144146.hdf5
|
| Jobscript | Exit code | 1 |
| Real time | 0m (0s) |
| CPU time | 0m (0s = 0%) |
| Outputting started | |
| Output files | |
| Finished | 2024-09-26 12:16:49 |
| List job events Wrapper job log |
Jobscript log (last 10,000 characters)
0217 traces, 50217 summary
Retagger: tagging trace set: gauss with 38767 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 38767 traces tagged "gauss"
FrameSaver: q=880270 n=728899 tag=gauss
wclsFrameSaver: saving 50217 traces tagged "wiener"
FrameSaver: q=936312 n=666303 tag=wiener
2084 hits in TPC 0
3198 hits in TPC 1
5089 hits in TPC 2
3367 hits in TPC 3
2959 hits in TPC 4
3111 hits in TPC 5
3537 hits in TPC 6
3323 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 3881 XUs and 1498 XVs -> 43 XUVs
C:0 T:2 21307 XUs and 13683 XVs -> 520 XUVs
C:0 T:3 39 XUs and 24 XVs -> 0 XUVs
C:0 T:4 20 XUs and 15 XVs -> 0 XUVs
C:0 T:5 16364 XUs and 10211 XVs -> 578 XUVs
C:0 T:6 6432 XUs and 9013 XVs -> 494 XUVs
C:0 T:7 33 XUs and 92 XVs -> 0 XUVs
1635 XUVs total
933 collection wire objects
1635 potential space points
Neighbour search...
40593 tests to find 24130 neighbours
Iterating with no regularization...
Begin: 5.1394e+06
0 4.86402e+06
1 4.84738e+06
2 4.84551e+06
Now with regularization...
Begin: 4.56773e+06
0 4.56726e+06
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Begin processing the 29th record. run: 29108 subRun: 1 event: 65050 at 26-Sep-2024 12:48:58 BST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 46339 traces, 46339 summary
Retagger: tagging trace set: gauss with 33348 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 33348 traces tagged "gauss"
FrameSaver: q=698956 n=597748 tag=gauss
wclsFrameSaver: saving 46339 traces tagged "wiener"
FrameSaver: q=753718 n=532999 tag=wiener
1490 hits in TPC 0
2231 hits in TPC 1
4480 hits in TPC 2
2696 hits in TPC 3
2384 hits in TPC 4
3616 hits in TPC 5
3824 hits in TPC 6
2294 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 19 XUs and 49 XVs -> 0 XUVs
C:0 T:2 12037 XUs and 24185 XVs -> 705 XUVs
C:0 T:3 96 XUs and 43 XVs -> 1 XUVs
C:0 T:4 18 XUs and 24 XVs -> 2 XUVs
C:0 T:5 10942 XUs and 14763 XVs -> 1446 XUVs
C:0 T:6 7973 XUs and 5997 XVs -> 117 XUVs
C:0 T:7 89 XUs and 210 XVs -> 7 XUVs
2278 XUVs total
779 collection wire objects
2278 potential space points
Neighbour search...
109742 tests to find 66330 neighbours
Iterating with no regularization...
Begin: 4.94475e+06
0 4.68542e+06
1 4.66466e+06
2 4.66049e+06
Now with regularization...
Begin: 4.26272e+06
0 4.26098e+06
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Begin processing the 30th record. run: 29108 subRun: 1 event: 65058 at 26-Sep-2024 12:50:48 BST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 52868 traces, 52868 summary
Retagger: tagging trace set: gauss with 40300 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 40300 traces tagged "gauss"
FrameSaver: q=856473 n=757465 tag=gauss
wclsFrameSaver: saving 52868 traces tagged "wiener"
FrameSaver: q=905963 n=686194 tag=wiener
3618 hits in TPC 0
2394 hits in TPC 1
5359 hits in TPC 2
3436 hits in TPC 3
2534 hits in TPC 4
3674 hits in TPC 5
3421 hits in TPC 6
3228 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 89 XUs and 73 XVs -> 12 XUVs
C:0 T:2 15924 XUs and 12805 XVs -> 492 XUVs
C:0 T:3 575 XUs and 314 XVs -> 5 XUVs
C:0 T:4 17 XUs and 18 XVs -> 0 XUVs
C:0 T:5 11029 XUs and 6559 XVs -> 155 XUVs
C:0 T:6 7354 XUs and 6826 XVs -> 241 XUVs
C:0 T:7 71 XUs and 85 XVs -> 2 XUVs
907 XUVs total
578 collection wire objects
907 potential space points
Neighbour search...
9865 tests to find 6024 neighbours
Iterating with no regularization...
Begin: 5.12117e+06
0 4.95996e+06
1 4.9482e+06
2 4.9473e+06
Now with regularization...
Begin: 4.70316e+06
0 4.70289e+06
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
26-Sep-2024 12:52:44 BST Closed output file "np04hd_raw_run029108_0270_dataflow2_datawriter_0_20240909T144146_reco_stage1_reco_stage2_20240926T115244_keepup.root"
26-Sep-2024 12:52:44 BST Closed input file "np04hd_raw_run029108_0270_dataflow2_datawriter_0_20240909T144146_reco_stage1.root"
========================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
========================================================================================================================================
Full event 84.9935 132.8 623.367 109.789 94.1184 30
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.0004723 0.00261758 0.00437871 0.00249292 0.000941794 30
produce:fembfilter:PDHDFEMBFilter 0.701325 0.736222 0.879352 0.710037 0.0521651 30
produce:wclsdatahd:WireCellToolkit 42.4957 46.9185 56.1 45.8272 3.54279 30
produce:gaushit:GausHitFinder 0.459031 0.765217 1.39564 0.732118 0.204596 30
produce:nhitsfilter:NumberOfHitsFilter 0.000230981 0.000380482 0.000609219 0.000367589 8.55338e-05 30
produce:reco3d:SpacePointSolver 0.102194 0.896582 4.80095 0.534537 0.957878 30
produce:hitpdune:DisambigFromSpacePoints 0.125936 0.601233 2.6622 0.463004 0.463055 30
produce:pandora:StandardPandora 4.55067 28.5967 516.399 10.2029 90.8032 30
produce:pandoraWriter:StandardPandora 0.0803974 0.145998 0.284453 0.13272 0.0394964 30
produce:pandoraTrack:LArPandoraTrackCreation 0.503371 1.35269 4.38874 1.15026 0.707413 30
produce:pandoraShower:LArPandoraShowerCreation 0.347728 1.50041 6.45709 1.23011 1.08149 30
produce:pandoracalo:Calorimetry 0.251753 0.686025 2.08749 0.608221 0.33331 30
produce:pandoracalonosce:Calorimetry 0.216444 0.629337 2.01957 0.533107 0.339167 30
produce:pandorapid:Chi2ParticleID 0.0012486 0.0019432 0.00353396 0.00187651 0.00049472 30
produce:pandoraShowercalo:ShowerCalorimetry 0.341429 1.39116 5.0455 1.15399 0.895867 30
produce:pandoraShowercalonosce:ShowerCalorimetry 0.324702 1.42936 5.22339 1.1965 0.954103 30
produce:pandoraGnocchiCalo:GnocchiCalorimetry 0.0845504 0.145152 0.324418 0.133401 0.0501014 30
produce:pandoraGnocchiCalonosce:GnocchiCalorimetry 0.0245973 0.046919 0.0796596 0.0433098 0.0128546 30
produce:emtrkmichelid:EmTrackMichelId 27.152 45.4204 85.8333 46.0188 11.4377 30
[art]:TriggerResults:TriggerResultInserter 2.7065e-05 3.93154e-05 0.000101341 3.36795e-05 1.51992e-05 30
end_path:out1:RootOutput 9.716e-06 1.19176e-05 3.3564e-05 1.0207e-05 5.49784e-06 30
end_path:michelremoving:michelremoving 0.0198905 0.095817 0.587964 0.0401887 0.116033 30
end_path:michelremovinggnocchi:michelremoving 0.0182567 0.0917412 0.582905 0.0341924 0.11903 30
end_path:calibanagnocchi:CalibAnaTree 0.18612 0.512371 1.97755 0.46095 0.319132 30
end_path:out1:RootOutput(write) 0.407295 0.735391 1.40671 0.68147 0.207086 30
========================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 4921.97 MB
Peak resident set size usage (VmHWM): 2350.72 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np04hd_raw_run029108_0270_dataflow2_datawriter_0_20240909T144146_reco_stage1_reco_stage2_20240926T115244_keepup.root
\tHists: np04hd_raw_run029108_0270_dataflow2_datawriter_0_20240909T144146_reco_stage1_reco_stage2_20240926T115244_keepup_hists.root
Forming reco metadata
/cvmfs/larsoft.opensciencegrid.org/products/python/v3_9_15/Linux64bit+3.10-2.17/bin/python: No module named meta_maker
Error in reco metadata
