Jobsub ID 263763.111@justin-prod-sched01.dune.hep.ac.uk
| Jobsub ID | 263763.111@justin-prod-sched01.dune.hep.ac.uk |
| Workflow ID | 3448 |
| Stage ID | 1 |
| User name | calcuttj@fnal.gov |
| HTCondor Group | group_dune.prod_mcsim |
| Requested | Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 18000 (5 hours) |
| Submitted time | 2024-09-26 13:24:49 |
| Site | UK_Sheffield |
| Entry | DUNE_UK_Sheffield_lcgce1 |
| Last heartbeat | 2024-09-26 14:48:44 |
| From worker node | Hostname | wn029.hep |
| cpuinfo | Intel(R) Xeon(R) Silver 4216 CPU @ 2.10GHz |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 257400 (71 hours) |
| Inner Apptainer? | True |
| Job state | jobscript_error |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk |
| Started | 2024-09-26 13:26:28 |
| Input files | hd-protodune:np04hd_raw_run029097_0014_dataflow4_datawriter_0_20240906T103340.hdf5
|
| Jobscript | Exit code | 1 |
| Real time | 0m (0s) |
| CPU time | 0m (0s = 0%) |
| Outputting started | |
| Output files | |
| Finished | 2024-09-26 14:48:44 |
| Saved logs | justin-logs:263763.111-justin-prod-sched01.dune.hep.ac.uk.logs.tgz |
| List job events Wrapper job log |
Jobscript log (last 10,000 characters)
wire objects
7143 potential space points
Neighbour search...
556023 tests to find 318204 neighbours
Iterating with no regularization...
Begin: 9.51435e+06
0 8.67799e+06
1 8.62836e+06
2 8.61936e+06
3 8.61648e+06
Now with regularization...
Begin: 8.22157e+06
0 8.21976e+06
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Begin processing the 29th record. run: 29097 subRun: 1 event: 3588 at 26-Sep-2024 15:43:55 BST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 71808 traces, 71808 summary
Retagger: tagging trace set: gauss with 52953 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 52953 traces tagged "gauss"
FrameSaver: q=1.16378e+06 n=981475 tag=gauss
wclsFrameSaver: saving 71808 traces tagged "wiener"
FrameSaver: q=1.26747e+06 n=894362 tag=wiener
4218 hits in TPC 0
3579 hits in TPC 1
4613 hits in TPC 2
2500 hits in TPC 3
2960 hits in TPC 4
7414 hits in TPC 5
7377 hits in TPC 6
5700 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:0 40 XUs and 14 XVs -> 0 XUVs
C:0 T:1 330 XUs and 280 XVs -> 11 XUVs
C:0 T:2 14092 XUs and 13425 XVs -> 314 XUVs
C:0 T:3 834 XUs and 834 XVs -> 42 XUVs
C:0 T:4 225 XUs and 375 XVs -> 4 XUVs
C:0 T:5 53310 XUs and 44860 XVs -> 1092 XUVs
C:0 T:6 30364 XUs and 35760 XVs -> 2681 XUVs
C:0 T:7 109 XUs and 156 XVs -> 7 XUVs
4151 XUVs total
1628 collection wire objects
4151 potential space points
Neighbour search...
145927 tests to find 82742 neighbours
Iterating with no regularization...
Begin: 7.21896e+06
0 6.23212e+06
1 6.13777e+06
2 6.12642e+06
3 6.12275e+06
Now with regularization...
Begin: 5.58233e+06
0 5.57996e+06
BdtBeamParticleIdTool::SliceFeatures::GetLArTPCIntercepts - inconsistent number of intercepts between a line and the LArTPC
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Begin processing the 30th record. run: 29097 subRun: 1 event: 3596 at 26-Sep-2024 15:45:41 BST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 96910 traces, 96910 summary
Retagger: tagging trace set: gauss with 73891 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 73891 traces tagged "gauss"
FrameSaver: q=1.64225e+06 n=1394445 tag=gauss
wclsFrameSaver: saving 96910 traces tagged "wiener"
FrameSaver: q=1.76523e+06 n=1277092 tag=wiener
2988 hits in TPC 0
3487 hits in TPC 1
12790 hits in TPC 2
5410 hits in TPC 3
6277 hits in TPC 4
9357 hits in TPC 5
9014 hits in TPC 6
3195 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 205 XUs and 178 XVs -> 0 XUVs
C:0 T:2 82574 XUs and 103774 XVs -> 3333 XUVs
C:0 T:3 439 XUs and 454 XVs -> 7 XUVs
C:0 T:4 378 XUs and 638 XVs -> 35 XUVs
C:0 T:5 47147 XUs and 50423 XVs -> 2382 XUVs
C:0 T:6 62512 XUs and 66485 XVs -> 1482 XUVs
C:0 T:7 192 XUs and 312 XVs -> 3 XUVs
7242 XUVs total
3087 collection wire objects
7242 potential space points
Neighbour search...
275370 tests to find 155388 neighbours
Iterating with no regularization...
Begin: 1.74753e+07
0 1.60928e+07
1 1.59732e+07
2 1.59573e+07
Now with regularization...
Begin: 1.52021e+07
0 1.51971e+07
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
no of hits closeby 0 event 3596 TrkackID 4 startx, y, z 176.969 600.937 349.91 wireno, peakt tpcno 198 5 5 dist 0min T, max_T5 2779.5
no of hits closeby 0 event 3596 TrkackID 9 startx, y, z 148.887 594.268 169.738 wireno, peakt tpcno 361 2490.5 2 dist 165.54min T, max_T573.891 2512.57
no of hits closeby 0 event 3596 TrkackID 4 startx, y, z 176.969 600.937 349.91 wireno, peakt tpcno 198 5 5 dist 0min T, max_T5 2779.5
no of hits closeby 0 event 3596 TrkackID 9 startx, y, z 148.887 594.268 169.738 wireno, peakt tpcno 361 2490.5 2 dist 165.54min T, max_T573.891 2512.57
26-Sep-2024 15:48:21 BST Closed output file "np04hd_raw_run029097_0014_dataflow4_datawriter_0_20240906T103340_reco_stage1_reco_stage2_20240926T144821_keepup.root"
26-Sep-2024 15:48:21 BST Closed input file "np04hd_raw_run029097_0014_dataflow4_datawriter_0_20240906T103340_reco_stage1.root"
========================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
========================================================================================================================================
Full event 71.5394 103.364 159.967 105.169 17.5822 30
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.000341686 0.00130972 0.00403705 0.00108184 0.00100998 30
produce:fembfilter:PDHDFEMBFilter 0.703885 0.720953 0.866732 0.715955 0.0279429 30
produce:wclsdatahd:WireCellToolkit 44.6443 46.6203 55.1637 46.1093 2.06742 30
produce:gaushit:GausHitFinder 0.445814 0.820509 1.20021 0.858108 0.177325 30
produce:nhitsfilter:NumberOfHitsFilter 0.000262948 0.000399584 0.00051594 0.000409403 6.32921e-05 30
produce:reco3d:SpacePointSolver 0.249459 1.27671 4.83969 1.01068 0.988555 30
produce:hitpdune:DisambigFromSpacePoints 0.183452 0.654867 1.44241 0.643399 0.269659 30
produce:pandora:StandardPandora 5.31761 14.41 27.7904 14.4523 5.35645 30
produce:pandoraWriter:StandardPandora 0.093593 0.169369 0.302412 0.17633 0.0407916 30
produce:pandoraTrack:LArPandoraTrackCreation 0.454976 1.86969 5.08195 1.81204 0.953528 30
produce:pandoraShower:LArPandoraShowerCreation 0.553354 2.13841 6.07407 2.02086 1.175 30
produce:pandoracalo:Calorimetry 0.276295 0.802996 2.0149 0.743609 0.353346 30
produce:pandoracalonosce:Calorimetry 0.229889 0.735377 1.94379 0.689407 0.343124 30
produce:pandorapid:Chi2ParticleID 0.00115536 0.00188134 0.00310085 0.00185505 0.000433134 30
produce:pandoraShowercalo:ShowerCalorimetry 0.464392 1.83083 6.27092 1.83709 1.08558 30
produce:pandoraShowercalonosce:ShowerCalorimetry 0.459954 1.79522 6.04669 1.80395 1.04832 30
produce:pandoraGnocchiCalo:GnocchiCalorimetry 0.0770145 0.135076 0.218478 0.134529 0.0301543 30
produce:pandoraGnocchiCalonosce:GnocchiCalorimetry 0.0335246 0.0536176 0.0938451 0.0536528 0.0127475 30
produce:emtrkmichelid:EmTrackMichelId 15.2994 27.5429 48.2557 28.2844 6.25251 30
[art]:TriggerResults:TriggerResultInserter 2.8206e-05 3.97787e-05 7.4817e-05 3.7744e-05 8.97295e-06 30
end_path:out1:RootOutput 6.676e-06 1.48902e-05 0.00018633 8.503e-06 3.19455e-05 30
end_path:michelremoving:michelremoving 0.0222316 0.081383 0.457142 0.0393947 0.0995213 30
end_path:michelremovinggnocchi:michelremoving 0.0186713 0.0685226 0.454229 0.0360004 0.0855816 30
end_path:calibanagnocchi:CalibAnaTree 0.168872 0.581733 1.54817 0.566529 0.270286 30
end_path:out1:RootOutput(write) 0.499724 0.862841 1.46109 0.852885 0.213452 30
========================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 5803.43 MB
Peak resident set size usage (VmHWM): 2635.78 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np04hd_raw_run029097_0014_dataflow4_datawriter_0_20240906T103340_reco_stage1_reco_stage2_20240926T144821_keepup.root
\tHists: np04hd_raw_run029097_0014_dataflow4_datawriter_0_20240906T103340_reco_stage1_reco_stage2_20240926T144821_keepup_hists.root
Forming reco metadata
/cvmfs/larsoft.opensciencegrid.org/products/python/v3_9_15/Linux64bit+3.10-2.17/bin/python: No module named meta_maker
Error in reco metadata
