Jobsub ID 263290.64@justin-prod-sched01.dune.hep.ac.uk
| Jobsub ID | 263290.64@justin-prod-sched01.dune.hep.ac.uk |
| Workflow ID | 3461 |
| Stage ID | 1 |
| User name | calcuttj@fnal.gov |
| HTCondor Group | group_dune.prod_mcsim |
| Requested | Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 18000 (5 hours) |
| Submitted time | 2024-09-26 09:57:08 |
| Site | UK_Sheffield |
| Entry | DUNE_UK_Sheffield_lcgce1 |
| Last heartbeat | 2024-09-26 11:34:33 |
| From worker node | Hostname | wn029.hep |
| cpuinfo | Intel(R) Xeon(R) Silver 4216 CPU @ 2.10GHz |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 257400 (71 hours) |
| Inner Apptainer? | True |
| Job state | jobscript_error |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk |
| Started | 2024-09-26 09:59:40 |
| Input files | hd-protodune:np04hd_raw_run027298_0195_dataflow3_datawriter_0_20240619T145832.hdf5
|
| Jobscript | Exit code | 1 |
| Real time | 0m (0s) |
| CPU time | 0m (0s = 0%) |
| Outputting started | |
| Output files | |
| Finished | 2024-09-26 11:34:33 |
| Saved logs | justin-logs:263290.64-justin-prod-sched01.dune.hep.ac.uk.logs.tgz |
| List job events Wrapper job log |
Jobscript log (last 10,000 characters)
61 XVs -> 15 XUVs
C:0 T:2 9307 XUs and 15259 XVs -> 490 XUVs
C:0 T:3 48 XUs and 72 XVs -> 2 XUVs
C:0 T:4 486 XUs and 334 XVs -> 12 XUVs
C:0 T:5 25152 XUs and 24839 XVs -> 1126 XUVs
C:0 T:6 48518 XUs and 22984 XVs -> 719 XUVs
C:0 T:7 522 XUs and 600 XVs -> 10 XUVs
2374 XUVs total
1396 collection wire objects
2374 potential space points
Neighbour search...
58508 tests to find 35994 neighbours
Iterating with no regularization...
Begin: 2.96235e+07
0 2.77484e+07
1 2.76467e+07
2 2.76394e+07
Now with regularization...
Begin: 2.56096e+07
0 2.56033e+07
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
no of hits closeby 2 event 27431 TrkackID 5 startx, y, z 87.179 372.688 449.629 wireno, peakt tpcno 1 497.097 5 dist 0min T, max_T220.296 1361.21
no of hits closeby 2 event 27431 TrkackID 5 startx, y, z 87.179 372.688 449.629 wireno, peakt tpcno 1 497.097 5 dist 0min T, max_T220.296 1361.21
Begin processing the 34th record. run: 27298 subRun: 1 event: 27435 at 26-Sep-2024 12:23:19 BST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 55717 traces, 55717 summary
Retagger: tagging trace set: gauss with 42879 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 42879 traces tagged "gauss"
FrameSaver: q=1.5449e+06 n=795777 tag=gauss
wclsFrameSaver: saving 55717 traces tagged "wiener"
FrameSaver: q=1.65583e+06 n=724678 tag=wiener
2278 hits in TPC 0
2662 hits in TPC 1
5635 hits in TPC 2
2644 hits in TPC 3
3222 hits in TPC 4
6204 hits in TPC 5
4007 hits in TPC 6
3477 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:0 5 XUs and 6 XVs -> 0 XUVs
C:0 T:1 208 XUs and 169 XVs -> 4 XUVs
C:0 T:2 8812 XUs and 13246 XVs -> 257 XUVs
C:0 T:3 30 XUs and 29 XVs -> 1 XUVs
C:0 T:4 233 XUs and 195 XVs -> 4 XUVs
C:0 T:5 17064 XUs and 14226 XVs -> 552 XUVs
C:0 T:6 10797 XUs and 12110 XVs -> 461 XUVs
C:0 T:7 115 XUs and 114 XVs -> 4 XUVs
1283 XUVs total
868 collection wire objects
1283 potential space points
Neighbour search...
14803 tests to find 8432 neighbours
Iterating with no regularization...
Begin: 1.65184e+07
0 1.59491e+07
1 1.59163e+07
2 1.59142e+07
Now with regularization...
Begin: 1.50998e+07
0 1.50993e+07
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Begin processing the 35th record. run: 27298 subRun: 1 event: 27439 at 26-Sep-2024 12:25:05 BST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 68762 traces, 68762 summary
Retagger: tagging trace set: gauss with 50607 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 50607 traces tagged "gauss"
FrameSaver: q=1.90138e+06 n=885968 tag=gauss
wclsFrameSaver: saving 68762 traces tagged "wiener"
FrameSaver: q=2.1055e+06 n=804731 tag=wiener
4553 hits in TPC 0
6382 hits in TPC 1
7821 hits in TPC 2
4915 hits in TPC 3
1663 hits in TPC 4
2903 hits in TPC 5
5324 hits in TPC 6
4736 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 15458 XUs and 16719 XVs -> 485 XUVs
C:0 T:2 28314 XUs and 24460 XVs -> 640 XUVs
C:0 T:3 459 XUs and 354 XVs -> 19 XUVs
C:0 T:4 27 XUs and 52 XVs -> 0 XUVs
C:0 T:5 5580 XUs and 4040 XVs -> 324 XUVs
C:0 T:6 14354 XUs and 13610 XVs -> 1110 XUVs
C:0 T:7 10304 XUs and 5369 XVs -> 697 XUVs
3275 XUVs total
1158 collection wire objects
3275 potential space points
Neighbour search...
111315 tests to find 61472 neighbours
Iterating with no regularization...
Begin: 4.97302e+07
0 4.77204e+07
1 4.75289e+07
2 4.75134e+07
Now with regularization...
Begin: 4.55275e+07
0 4.55204e+07
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
26-Sep-2024 12:26:59 BST Closed output file "np04hd_raw_run027298_0195_dataflow3_datawriter_0_20240619T145832_reco_stage1_reco_stage2_20240926T112659_keepup.root"
26-Sep-2024 12:26:59 BST Closed input file "np04hd_raw_run027298_0195_dataflow3_datawriter_0_20240619T145832_reco_stage1.root"
========================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
========================================================================================================================================
Full event 63.5562 103.663 172.285 102.982 21.8121 35
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.000394417 0.00146229 0.00417005 0.00118838 0.000993955 35
produce:fembfilter:PDHDFEMBFilter 0.767363 0.907012 1.11734 0.907122 0.0711804 35
produce:wclsdatahd:WireCellToolkit 43.6094 47.7545 58.4679 46.5558 3.81435 35
produce:gaushit:GausHitFinder 0.305581 0.835102 1.35125 0.802002 0.232029 35
produce:nhitsfilter:NumberOfHitsFilter 0.000238186 0.00041311 0.000743372 0.000394172 9.8767e-05 35
produce:reco3d:SpacePointSolver 0.183979 0.917666 3.88408 0.719958 0.7269 35
produce:hitpdune:DisambigFromSpacePoints 0.105389 0.611399 1.68338 0.559494 0.309482 35
produce:pandora:StandardPandora 4.64127 14.4447 35.6096 13.275 6.39114 35
produce:pandoraWriter:StandardPandora 0.0920071 0.169667 0.268773 0.171611 0.0408547 35
produce:pandoraTrack:LArPandoraTrackCreation 0.451248 2.696 5.62903 2.60209 1.23166 35
produce:pandoraShower:LArPandoraShowerCreation 0.271421 3.19262 11.1331 2.5627 2.27923 35
produce:pandoracalo:Calorimetry 0.1833 1.00145 2.47819 0.973407 0.485382 35
produce:pandoracalonosce:Calorimetry 0.159491 0.922105 1.91514 0.911089 0.425517 35
produce:pandorapid:Chi2ParticleID 0.00122421 0.00204608 0.00329582 0.00197243 0.000462383 35
produce:pandoraShowercalo:ShowerCalorimetry 0.17692 2.38894 7.85026 2.10741 1.59405 35
produce:pandoraShowercalonosce:ShowerCalorimetry 0.171663 2.33511 7.30845 1.99415 1.50465 35
produce:pandoraGnocchiCalo:GnocchiCalorimetry 0.0439105 0.123069 0.196971 0.127993 0.0368722 35
produce:pandoraGnocchiCalonosce:GnocchiCalorimetry 0.017659 0.0492674 0.136182 0.0496969 0.0188445 35
produce:emtrkmichelid:EmTrackMichelId 10.8755 23.4864 40.2656 23.4548 5.58734 35
[art]:TriggerResults:TriggerResultInserter 3.4126e-05 5.56263e-05 0.000109656 5.5871e-05 1.21035e-05 35
end_path:out1:RootOutput 9.874e-06 1.61483e-05 3.2664e-05 1.6376e-05 3.88171e-06 35
end_path:michelremoving:michelremoving 0.0141585 0.0691949 0.225413 0.0459372 0.0594614 35
end_path:michelremovinggnocchi:michelremoving 0.0131888 0.0622805 0.222035 0.0417287 0.052675 35
end_path:calibanagnocchi:CalibAnaTree 0.134466 0.681441 1.54687 0.69507 0.301211 35
end_path:out1:RootOutput(write) 0.331444 0.784432 1.38926 0.805993 0.20891 35
========================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 5810.41 MB
Peak resident set size usage (VmHWM): 2645.54 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np04hd_raw_run027298_0195_dataflow3_datawriter_0_20240619T145832_reco_stage1_reco_stage2_20240926T112659_keepup.root
\tHists: np04hd_raw_run027298_0195_dataflow3_datawriter_0_20240619T145832_reco_stage1_reco_stage2_20240926T112659_keepup_hists.root
Forming reco metadata
/cvmfs/larsoft.opensciencegrid.org/products/python/v3_9_15/Linux64bit+3.10-2.17/bin/python: No module named meta_maker
Error in reco metadata
