Jobsub ID 263765.64@justin-prod-sched01.dune.hep.ac.uk
| Jobsub ID | 263765.64@justin-prod-sched01.dune.hep.ac.uk |
| Workflow ID | 3451 |
| Stage ID | 1 |
| User name | calcuttj@fnal.gov |
| HTCondor Group | group_dune.prod_mcsim |
| Requested | Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 18000 (5 hours) |
| Submitted time | 2024-09-26 13:24:52 |
| Site | UK_Sheffield |
| Entry | DUNE_UK_Sheffield_lcgce1 |
| Last heartbeat | 2024-09-26 14:56:41 |
| From worker node | Hostname | wn095.hep |
| cpuinfo | Intel(R) Xeon(R) CPU E5-2640 v4 @ 2.40GHz |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 257400 (71 hours) |
| Inner Apptainer? | True |
| Job state | jobscript_error |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk |
| Started | 2024-09-26 13:25:52 |
| Input files | hd-protodune:np04hd_raw_run029107_1946_dataflow2_datawriter_0_20240907T233259.hdf5
|
| Jobscript | Exit code | 1 |
| Real time | 0m (0s) |
| CPU time | 0m (0s = 0%) |
| Outputting started | |
| Output files | |
| Finished | 2024-09-26 14:56:41 |
| Saved logs | justin-logs:263765.64-justin-prod-sched01.dune.hep.ac.uk.logs.tgz |
| List job events Wrapper job log |
Jobscript log (last 10,000 characters)
set: gauss with 42115 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 42115 traces tagged "gauss"
FrameSaver: q=842163 n=734215 tag=gauss
wclsFrameSaver: saving 56478 traces tagged "wiener"
FrameSaver: q=917168 n=661376 tag=wiener
3063 hits in TPC 0
4257 hits in TPC 1
6022 hits in TPC 2
2178 hits in TPC 3
3360 hits in TPC 4
6253 hits in TPC 5
3135 hits in TPC 6
1499 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 500 XUs and 261 XVs -> 6 XUVs
C:0 T:2 44611 XUs and 32210 XVs -> 1159 XUVs
C:0 T:3 9 XUs and 9 XVs -> 0 XUVs
C:0 T:4 202 XUs and 159 XVs -> 7 XUVs
C:0 T:5 39425 XUs and 25598 XVs -> 1564 XUVs
C:0 T:6 54715 XUs and 42383 XVs -> 963 XUVs
C:0 T:7 93 XUs and 88 XVs -> 3 XUVs
3702 XUVs total
1219 collection wire objects
3702 potential space points
Neighbour search...
374230 tests to find 179980 neighbours
Iterating with no regularization...
Begin: 6.76336e+06
0 6.36929e+06
1 6.33613e+06
2 6.33207e+06
Now with regularization...
Begin: 6.07246e+06
0 6.07083e+06
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Begin processing the 29th record. run: 29107 subRun: 1 event: 467290 at 26-Sep-2024 15:51:45 BST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 57825 traces, 57825 summary
Retagger: tagging trace set: gauss with 43393 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 43393 traces tagged "gauss"
FrameSaver: q=941031 n=822655 tag=gauss
wclsFrameSaver: saving 57825 traces tagged "wiener"
FrameSaver: q=1.0021e+06 n=743969 tag=wiener
3623 hits in TPC 0
3118 hits in TPC 1
3340 hits in TPC 2
2300 hits in TPC 3
2305 hits in TPC 4
6244 hits in TPC 5
6133 hits in TPC 6
3474 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 14 XUs and 4 XVs -> 0 XUVs
C:0 T:2 5166 XUs and 5328 XVs -> 364 XUVs
C:0 T:3 222 XUs and 200 XVs -> 19 XUVs
C:0 T:4 252 XUs and 202 XVs -> 12 XUVs
C:0 T:5 15887 XUs and 14065 XVs -> 504 XUVs
C:0 T:6 16382 XUs and 23352 XVs -> 1686 XUVs
C:0 T:7 476 XUs and 510 XVs -> 18 XUVs
2603 XUVs total
1340 collection wire objects
2603 potential space points
Neighbour search...
57767 tests to find 33936 neighbours
Iterating with no regularization...
Begin: 8.80338e+06
0 8.31542e+06
1 8.28989e+06
2 8.28607e+06
Now with regularization...
Begin: 7.65899e+06
0 7.6574e+06
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Begin processing the 30th record. run: 29107 subRun: 1 event: 467298 at 26-Sep-2024 15:53:57 BST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 78423 traces, 78423 summary
Retagger: tagging trace set: gauss with 58164 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 58164 traces tagged "gauss"
FrameSaver: q=1.37893e+06 n=1123858 tag=gauss
wclsFrameSaver: saving 78423 traces tagged "wiener"
FrameSaver: q=1.47876e+06 n=1026514 tag=wiener
2938 hits in TPC 0
3100 hits in TPC 1
5342 hits in TPC 2
4569 hits in TPC 3
6158 hits in TPC 4
12116 hits in TPC 5
4359 hits in TPC 6
3252 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 1558 XUs and 3998 XVs -> 130 XUVs
C:0 T:2 12729 XUs and 18282 XVs -> 796 XUVs
C:0 T:3 334 XUs and 284 XVs -> 20 XUVs
C:0 T:4 106 XUs and 154 XVs -> 7 XUVs
C:0 T:5 51617 XUs and 91532 XVs -> 6740 XUVs
C:0 T:6 9050 XUs and 14061 XVs -> 648 XUVs
C:0 T:7 32 XUs and 50 XVs -> 0 XUVs
8341 XUVs total
1915 collection wire objects
8341 potential space points
Neighbour search...
1134495 tests to find 556100 neighbours
Iterating with no regularization...
Begin: 1.35888e+07
0 1.27493e+07
1 1.26846e+07
2 1.2677e+07
Now with regularization...
Begin: 1.20332e+07
0 1.20289e+07
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
26-Sep-2024 15:56:18 BST Closed output file "np04hd_raw_run029107_1946_dataflow2_datawriter_0_20240907T233259_reco_stage1_reco_stage2_20240926T145618_keepup.root"
26-Sep-2024 15:56:18 BST Closed input file "np04hd_raw_run029107_1946_dataflow2_datawriter_0_20240907T233259_reco_stage1.root"
========================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
========================================================================================================================================
Full event 83.4735 119.491 154.408 117.807 17.5211 30
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.000540694 0.00286597 0.00494979 0.0031324 0.00123937 30
produce:fembfilter:PDHDFEMBFilter 0.749952 0.771079 0.874863 0.761999 0.0253934 30
produce:wclsdatahd:WireCellToolkit 55.6347 57.153 59.0172 57.0378 0.819259 30
produce:gaushit:GausHitFinder 0.441965 0.811752 1.174 0.798913 0.184922 30
produce:nhitsfilter:NumberOfHitsFilter 0.00023214 0.000403568 0.000575151 0.000411339 7.67617e-05 30
produce:reco3d:SpacePointSolver 0.103889 1.0141 3.66664 0.90604 0.737134 30
produce:hitpdune:DisambigFromSpacePoints 0.121701 0.592799 1.45286 0.541521 0.280222 30
produce:pandora:StandardPandora 5.35525 15.4208 32.1121 14.827 6.62557 30
produce:pandoraWriter:StandardPandora 0.0854318 0.167409 0.229347 0.170162 0.0366232 30
produce:pandoraTrack:LArPandoraTrackCreation 0.507796 1.65711 3.23034 1.58993 0.635807 30
produce:pandoraShower:LArPandoraShowerCreation 0.34417 1.94018 3.99558 1.9303 0.944304 30
produce:pandoracalo:Calorimetry 0.285468 0.844317 1.44458 0.816629 0.286567 30
produce:pandoracalonosce:Calorimetry 0.256616 0.7639 1.31748 0.737493 0.264318 30
produce:pandorapid:Chi2ParticleID 0.0014366 0.00204724 0.0031738 0.00194366 0.000357672 30
produce:pandoraShowercalo:ShowerCalorimetry 0.322324 1.80438 4.01987 1.76975 0.841213 30
produce:pandoraShowercalonosce:ShowerCalorimetry 0.314521 1.71258 3.54018 1.71699 0.734796 30
produce:pandoraGnocchiCalo:GnocchiCalorimetry 0.080011 0.145916 0.224035 0.146808 0.03182 30
produce:pandoraGnocchiCalonosce:GnocchiCalorimetry 0.0287503 0.052859 0.0719195 0.0536891 0.0108536 30
produce:emtrkmichelid:EmTrackMichelId 17.3258 32.9498 47.5202 32.2189 6.76612 30
[art]:TriggerResults:TriggerResultInserter 2.2666e-05 3.39501e-05 7.9098e-05 3.1215e-05 1.08054e-05 30
end_path:out1:RootOutput 9.147e-06 1.86886e-05 0.000226889 1.07635e-05 3.88063e-05 30
end_path:michelremoving:michelremoving 0.022146 0.0910078 0.521225 0.0475265 0.0994453 30
end_path:michelremovinggnocchi:michelremoving 0.0207115 0.0860391 0.511791 0.0398324 0.0984232 30
end_path:calibanagnocchi:CalibAnaTree 0.173825 0.563642 0.973953 0.565085 0.195428 30
end_path:out1:RootOutput(write) 0.431083 0.830127 1.12933 0.804971 0.171974 30
========================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 4942.37 MB
Peak resident set size usage (VmHWM): 2351.95 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np04hd_raw_run029107_1946_dataflow2_datawriter_0_20240907T233259_reco_stage1_reco_stage2_20240926T145618_keepup.root
\tHists: np04hd_raw_run029107_1946_dataflow2_datawriter_0_20240907T233259_reco_stage1_reco_stage2_20240926T145618_keepup_hists.root
Forming reco metadata
/cvmfs/larsoft.opensciencegrid.org/products/python/v3_9_15/Linux64bit+3.10-2.17/bin/python: No module named meta_maker
Error in reco metadata
