Jobsub ID 263681.21@justin-prod-sched01.dune.hep.ac.uk
| Jobsub ID | 263681.21@justin-prod-sched01.dune.hep.ac.uk |
| Workflow ID | 3451 |
| Stage ID | 1 |
| User name | calcuttj@fnal.gov |
| HTCondor Group | group_dune.prod_mcsim |
| Requested | Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 18000 (5 hours) |
| Submitted time | 2024-09-26 13:07:21 |
| Site | UK_Durham |
| Entry | DUNE_UK_SGridDurham_ce4 |
| Last heartbeat | 2024-09-26 14:55:33 |
| From worker node | Hostname | n185.dur.scotgrid.ac.uk |
| cpuinfo | Intel(R) Xeon(R) Gold 5220 CPU @ 2.20GHz |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 171000 (47 hours) |
| Inner Apptainer? | True |
| Job state | jobscript_error |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk |
| Started | 2024-09-26 13:07:58 |
| Input files | hd-protodune:np04hd_raw_run029107_2112_dataflow7_datawriter_0_20240908T015503.hdf5
|
| Jobscript | Exit code | 1 |
| Real time | 0m (0s) |
| CPU time | 0m (0s = 0%) |
| Outputting started | |
| Output files | |
| Finished | 2024-09-26 14:55:33 |
| Saved logs | justin-logs:263681.21-justin-prod-sched01.dune.hep.ac.uk.logs.tgz |
| List job events Wrapper job log |
Jobscript log (last 10,000 characters)
lsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 42293 traces tagged "gauss"
FrameSaver: q=857387 n=783101 tag=gauss
wclsFrameSaver: saving 56078 traces tagged "wiener"
FrameSaver: q=909121 n=701479 tag=wiener
2789 hits in TPC 0
2337 hits in TPC 1
7029 hits in TPC 2
5037 hits in TPC 3
3092 hits in TPC 4
5319 hits in TPC 5
2273 hits in TPC 6
896 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 111 XUs and 100 XVs -> 2 XUVs
C:0 T:2 30604 XUs and 34212 XVs -> 992 XUVs
C:0 T:3 743 XUs and 541 XVs -> 12 XUVs
C:0 T:4 92 XUs and 111 XVs -> 4 XUVs
C:0 T:5 16620 XUs and 18482 XVs -> 293 XUVs
C:0 T:6 10898 XUs and 19831 XVs -> 355 XUVs
C:0 T:7 3 XUs and 2 XVs -> 0 XUVs
1658 XUVs total
922 collection wire objects
1658 potential space points
Neighbour search...
70992 tests to find 45846 neighbours
Iterating with no regularization...
Begin: 3.13515e+06
0 3.01202e+06
1 3.0081e+06
2 3.00738e+06
Now with regularization...
Begin: 2.83796e+06
0 2.83763e+06
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Begin processing the 29th record. run: 29107 subRun: 1 event: 507127 at 26-Sep-2024 15:51:37 BST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 28346 traces, 28346 summary
Retagger: tagging trace set: gauss with 21017 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 21017 traces tagged "gauss"
FrameSaver: q=442534 n=380840 tag=gauss
wclsFrameSaver: saving 28346 traces tagged "wiener"
FrameSaver: q=467709 n=339739 tag=wiener
1092 hits in TPC 0
1416 hits in TPC 1
2805 hits in TPC 2
1910 hits in TPC 3
1458 hits in TPC 4
3219 hits in TPC 5
1136 hits in TPC 6
666 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 33 XUs and 24 XVs -> 3 XUVs
C:0 T:2 5849 XUs and 12225 XVs -> 588 XUVs
C:0 T:3 28 XUs and 45 XVs -> 1 XUVs
C:0 T:4 74 XUs and 127 XVs -> 4 XUVs
C:0 T:5 5199 XUs and 3493 XVs -> 134 XUVs
C:0 T:6 837 XUs and 2667 XVs -> 48 XUVs
C:0 T:7 0 XUs and 0 XVs -> 0 XUVs
778 XUVs total
527 collection wire objects
778 potential space points
Neighbour search...
8640 tests to find 4836 neighbours
Iterating with no regularization...
Begin: 2.93669e+06
0 2.76862e+06
1 2.76444e+06
2 2.76423e+06
Now with regularization...
Begin: 2.48292e+06
0 2.48287e+06
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Begin processing the 30th record. run: 29107 subRun: 1 event: 507135 at 26-Sep-2024 15:52:51 BST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 74677 traces, 74677 summary
Retagger: tagging trace set: gauss with 56003 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 56003 traces tagged "gauss"
FrameSaver: q=1.24354e+06 n=1066068 tag=gauss
wclsFrameSaver: saving 74677 traces tagged "wiener"
FrameSaver: q=1.33408e+06 n=967865 tag=wiener
2887 hits in TPC 0
2866 hits in TPC 1
7690 hits in TPC 2
5672 hits in TPC 3
4725 hits in TPC 4
6973 hits in TPC 5
5766 hits in TPC 6
4012 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 90 XUs and 226 XVs -> 3 XUVs
C:0 T:2 27638 XUs and 32356 XVs -> 2214 XUVs
C:0 T:3 399 XUs and 332 XVs -> 4 XUVs
C:0 T:4 212 XUs and 185 XVs -> 9 XUVs
C:0 T:5 29266 XUs and 65348 XVs -> 1313 XUVs
C:0 T:6 19464 XUs and 14862 XVs -> 839 XUVs
C:0 T:7 78 XUs and 57 XVs -> 1 XUVs
4383 XUVs total
1897 collection wire objects
4383 potential space points
Neighbour search...
139007 tests to find 79552 neighbours
Iterating with no regularization...
Begin: 1.59156e+07
0 1.42259e+07
1 1.40301e+07
2 1.40195e+07
Now with regularization...
Begin: 1.33918e+07
0 1.33883e+07
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
26-Sep-2024 15:55:05 BST Closed output file "np04hd_raw_run029107_2112_dataflow7_datawriter_0_20240908T015503_reco_stage1_reco_stage2_20240926T145505_keepup.root"
26-Sep-2024 15:55:05 BST Closed input file "np04hd_raw_run029107_2112_dataflow7_datawriter_0_20240908T015503_reco_stage1.root"
========================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
========================================================================================================================================
Full event 74.2514 107.865 149.951 104.265 19.4971 30
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.000331463 0.00118744 0.00367486 0.000982224 0.000921407 30
produce:fembfilter:PDHDFEMBFilter 0.702388 0.816037 1.06655 0.787092 0.126467 30
produce:wclsdatahd:WireCellToolkit 45.4136 53.5113 63.8994 54.5829 6.28906 30
produce:gaushit:GausHitFinder 0.464763 0.971442 1.64072 1.00824 0.252535 30
produce:nhitsfilter:NumberOfHitsFilter 0.000261981 0.000418708 0.000717586 0.00041433 8.64823e-05 30
produce:reco3d:SpacePointSolver 0.176715 1.19241 3.40821 0.961013 0.830876 30
produce:hitpdune:DisambigFromSpacePoints 0.186814 0.765722 1.48794 0.735639 0.332159 30
produce:pandora:StandardPandora 3.78113 16.6374 36.9697 15.1925 7.97186 30
produce:pandoraWriter:StandardPandora 0.0882451 0.187882 0.281298 0.1915 0.0528399 30
produce:pandoraTrack:LArPandoraTrackCreation 0.473568 2.03014 4.30685 1.8461 0.973914 30
produce:pandoraShower:LArPandoraShowerCreation 0.359407 2.12271 4.82885 1.835 1.12217 30
produce:pandoracalo:Calorimetry 0.264048 0.863634 1.82597 0.807571 0.376583 30
produce:pandoracalonosce:Calorimetry 0.230803 0.802948 1.6974 0.735732 0.364997 30
produce:pandorapid:Chi2ParticleID 0.00127785 0.00231664 0.0044921 0.0020357 0.000762853 30
produce:pandoraShowercalo:ShowerCalorimetry 0.318672 1.90137 4.67263 1.69322 1.0067 30
produce:pandoraShowercalonosce:ShowerCalorimetry 0.309478 1.84918 4.5696 1.68613 0.97825 30
produce:pandoraGnocchiCalo:GnocchiCalorimetry 0.0822122 0.170508 0.289074 0.153161 0.0549897 30
produce:pandoraGnocchiCalonosce:GnocchiCalorimetry 0.0317412 0.06643 0.112401 0.0573048 0.0232456 30
produce:emtrkmichelid:EmTrackMichelId 11.1075 22.0903 34.184 21.4276 5.45341 30
[art]:TriggerResults:TriggerResultInserter 2.8945e-05 3.87666e-05 7.0109e-05 3.9412e-05 8.27195e-06 30
end_path:out1:RootOutput 7.547e-06 1.38891e-05 0.000123104 9.205e-06 2.04722e-05 30
end_path:michelremoving:michelremoving 0.0192372 0.0677371 0.227885 0.0466745 0.0536001 30
end_path:michelremovinggnocchi:michelremoving 0.016815 0.0631193 0.219708 0.0419206 0.0531672 30
end_path:calibanagnocchi:CalibAnaTree 0.189507 0.646079 1.47636 0.593468 0.302583 30
end_path:out1:RootOutput(write) 0.419209 0.93371 1.71442 0.901023 0.304902 30
========================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 5761.62 MB
Peak resident set size usage (VmHWM): 2849.41 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np04hd_raw_run029107_2112_dataflow7_datawriter_0_20240908T015503_reco_stage1_reco_stage2_20240926T145505_keepup.root
\tHists: np04hd_raw_run029107_2112_dataflow7_datawriter_0_20240908T015503_reco_stage1_reco_stage2_20240926T145505_keepup_hists.root
Forming reco metadata
/cvmfs/larsoft.opensciencegrid.org/products/python/v3_9_15/Linux64bit+3.10-2.17/bin/python: No module named meta_maker
Error in reco metadata
