Jobsub ID 264031.2@justin-prod-sched01.dune.hep.ac.uk
| Jobsub ID | 264031.2@justin-prod-sched01.dune.hep.ac.uk |
| Workflow ID | 3460 |
| Stage ID | 1 |
| User name | calcuttj@fnal.gov |
| HTCondor Group | group_dune.prod_mcsim |
| Requested | Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 18000 (5 hours) |
| Submitted time | 2024-09-26 15:00:44 |
| Site | UK_Manchester |
| Entry | UBoone_T2_UK_Manchester_ce01 |
| Last heartbeat | 2024-09-26 16:12:07 |
| From worker node | Hostname | wn2207291.tier2.hep.manchester.ac.uk |
| cpuinfo | AMD EPYC 7513 32-Core Processor |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 257400 (71 hours) |
| Inner Apptainer? | True |
| Job state | jobscript_error |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk |
| Started | 2024-09-26 15:02:04 |
| Input files | hd-protodune:np04hd_raw_run029218_0100_dataflow0_datawriter_0_20240917T165336.hdf5
|
| Jobscript | Exit code | 1 |
| Real time | 0m (0s) |
| CPU time | 0m (0s = 0%) |
| Outputting started | |
| Output files | |
| Finished | 2024-09-26 16:12:07 |
| Saved logs | justin-logs:264031.2-justin-prod-sched01.dune.hep.ac.uk.logs.tgz |
| List job events Wrapper job log |
Jobscript log (last 10,000 characters)
iener"
FrameSaver: q=1.17754e+06 n=842082 tag=wiener
1644 hits in TPC 0
1275 hits in TPC 1
4950 hits in TPC 2
2567 hits in TPC 3
3922 hits in TPC 4
6693 hits in TPC 5
8690 hits in TPC 6
6570 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 106 XUs and 56 XVs -> 8 XUVs
C:0 T:2 31100 XUs and 44795 XVs -> 601 XUVs
C:0 T:3 148 XUs and 308 XVs -> 14 XUVs
C:0 T:4 857 XUs and 2716 XVs -> 120 XUVs
C:0 T:5 19243 XUs and 35153 XVs -> 3602 XUVs
C:0 T:6 57223 XUs and 102037 XVs -> 2409 XUVs
C:0 T:7 570 XUs and 696 XVs -> 40 XUVs
6794 XUVs total
1986 collection wire objects
6794 potential space points
Neighbour search...
551828 tests to find 290770 neighbours
Iterating with no regularization...
Begin: 9.80559e+06
0 8.39598e+06
1 8.30528e+06
2 8.28801e+06
3 8.28164e+06
Now with regularization...
Begin: 7.65856e+06
0 7.65469e+06
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Begin processing the 30th record. run: 29218 subRun: 1 event: 25040 at 26-Sep-2024 17:08:39 BST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 65348 traces, 65348 summary
Retagger: tagging trace set: gauss with 47374 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 47374 traces tagged "gauss"
FrameSaver: q=1.06672e+06 n=881255 tag=gauss
wclsFrameSaver: saving 65348 traces tagged "wiener"
FrameSaver: q=1.15165e+06 n=782672 tag=wiener
2744 hits in TPC 0
2076 hits in TPC 1
6750 hits in TPC 2
5068 hits in TPC 3
3639 hits in TPC 4
5921 hits in TPC 5
5835 hits in TPC 6
2555 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 81 XUs and 114 XVs -> 18 XUVs
C:0 T:2 38334 XUs and 33472 XVs -> 2079 XUVs
C:0 T:3 23 XUs and 39 XVs -> 0 XUVs
C:0 T:4 114 XUs and 130 XVs -> 5 XUVs
C:0 T:5 59091 XUs and 46281 XVs -> 4250 XUVs
C:0 T:6 12974 XUs and 12602 XVs -> 228 XUVs
C:0 T:7 108 XUs and 167 XVs -> 4 XUVs
6584 XUVs total
1446 collection wire objects
6584 potential space points
Neighbour search...
688160 tests to find 320522 neighbours
Iterating with no regularization...
Begin: 1.41456e+07
0 1.16079e+07
1 1.13665e+07
2 1.13169e+07
3 1.13039e+07
4 1.12995e+07
Now with regularization...
Begin: 1.02965e+07
0 1.02866e+07
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Begin processing the 31st record. run: 29218 subRun: 1 event: 25056 at 26-Sep-2024 17:10:14 BST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 60828 traces, 60828 summary
Retagger: tagging trace set: gauss with 46379 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 46379 traces tagged "gauss"
FrameSaver: q=1.03527e+06 n=890580 tag=gauss
wclsFrameSaver: saving 60828 traces tagged "wiener"
FrameSaver: q=1.12662e+06 n=824149 tag=wiener
3520 hits in TPC 0
2550 hits in TPC 1
3786 hits in TPC 2
3035 hits in TPC 3
3015 hits in TPC 4
7628 hits in TPC 5
7163 hits in TPC 6
4015 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 3166 XUs and 3419 XVs -> 63 XUVs
C:0 T:2 9717 XUs and 8228 XVs -> 262 XUVs
C:0 T:3 142 XUs and 171 XVs -> 12 XUVs
C:0 T:4 852 XUs and 1454 XVs -> 79 XUVs
C:0 T:5 27157 XUs and 25754 XVs -> 907 XUVs
C:0 T:6 30388 XUs and 28705 XVs -> 913 XUVs
C:0 T:7 483 XUs and 353 XVs -> 11 XUVs
2247 XUVs total
1429 collection wire objects
2247 potential space points
Neighbour search...
35887 tests to find 20670 neighbours
Iterating with no regularization...
Begin: 7.70874e+06
0 7.16557e+06
1 7.13984e+06
2 7.13742e+06
Now with regularization...
Begin: 6.72824e+06
0 6.72759e+06
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
26-Sep-2024 17:11:38 BST Closed output file "np04hd_raw_run029218_0100_dataflow0_datawriter_0_20240917T165336_reco_stage1_reco_stage2_20240926T161138_keepup.root"
26-Sep-2024 17:11:38 BST Closed input file "np04hd_raw_run029218_0100_dataflow0_datawriter_0_20240917T165336_reco_stage1.root"
========================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
========================================================================================================================================
Full event 50.0354 88.0332 291.483 81.0487 39.2972 31
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.000420599 0.00299927 0.0068172 0.00273702 0.00128386 31
produce:fembfilter:PDHDFEMBFilter 0.492092 0.702443 0.832678 0.743326 0.104774 31
produce:wclsdatahd:WireCellToolkit 27.8729 36.1901 45.0165 36.4418 3.54037 31
produce:gaushit:GausHitFinder 0.294974 0.66278 1.29394 0.631548 0.224195 31
produce:nhitsfilter:NumberOfHitsFilter 0.0002396 0.000383848 0.00065631 0.000349435 0.000106807 31
produce:reco3d:SpacePointSolver 0.129277 0.977842 5.08367 0.545374 1.01218 31
produce:hitpdune:DisambigFromSpacePoints 0.128381 0.519846 2.54299 0.415953 0.421695 31
produce:pandora:StandardPandora 4.24822 15.5256 176.716 9.97048 29.6161 31
produce:pandoraWriter:StandardPandora 0.0843772 0.166256 0.34258 0.148837 0.0592316 31
produce:pandoraTrack:LArPandoraTrackCreation 0.323245 1.11553 4.66034 0.893502 0.797057 31
produce:pandoraShower:LArPandoraShowerCreation 0.324128 1.28292 5.04776 1.06664 0.963428 31
produce:pandoracalo:Calorimetry 0.168491 0.594809 1.79608 0.492769 0.330365 31
produce:pandoracalonosce:Calorimetry 0.147943 0.539163 1.71002 0.434155 0.317804 31
produce:pandorapid:Chi2ParticleID 0.00101635 0.00199015 0.0036229 0.00181843 0.000639841 31
produce:pandoraShowercalo:ShowerCalorimetry 0.243358 1.19587 3.81245 0.995666 0.772005 31
produce:pandoraShowercalonosce:ShowerCalorimetry 0.240535 1.16722 3.87789 1.01859 0.769506 31
produce:pandoraGnocchiCalo:GnocchiCalorimetry 0.0463648 0.134836 0.27427 0.140353 0.0568409 31
produce:pandoraGnocchiCalonosce:GnocchiCalorimetry 0.0174925 0.0441905 0.0856339 0.0442479 0.0172551 31
produce:emtrkmichelid:EmTrackMichelId 13.6187 25.5785 42.0119 25.5149 6.08488 31
[art]:TriggerResults:TriggerResultInserter 2.3714e-05 5.80596e-05 0.000147427 5.6055e-05 2.64225e-05 31
end_path:out1:RootOutput 8.305e-06 1.53846e-05 5.7488e-05 1.2854e-05 8.8902e-06 31
end_path:michelremoving:michelremoving 0.0195118 0.0822573 0.379711 0.035346 0.0874157 31
end_path:michelremovinggnocchi:michelremoving 0.0187114 0.0770696 0.377908 0.036623 0.0861629 31
end_path:calibanagnocchi:CalibAnaTree 0.194035 0.47157 1.55091 0.426156 0.256439 31
end_path:out1:RootOutput(write) 0.420038 0.857616 1.72826 0.837333 0.26434 31
========================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 4950.34 MB
Peak resident set size usage (VmHWM): 2577.3 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np04hd_raw_run029218_0100_dataflow0_datawriter_0_20240917T165336_reco_stage1_reco_stage2_20240926T161138_keepup.root
\tHists: np04hd_raw_run029218_0100_dataflow0_datawriter_0_20240917T165336_reco_stage1_reco_stage2_20240926T161138_keepup_hists.root
Forming reco metadata
/cvmfs/larsoft.opensciencegrid.org/products/python/v3_9_15/Linux64bit+3.10-2.17/bin/python: No module named meta_maker
Error in reco metadata
