Jobsub ID 263843.5@justin-prod-sched01.dune.hep.ac.uk
| Jobsub ID | 263843.5@justin-prod-sched01.dune.hep.ac.uk |
| Workflow ID | 3448 |
| Stage ID | 1 |
| User name | calcuttj@fnal.gov |
| HTCondor Group | group_dune.prod_mcsim |
| Requested | Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 18000 (5 hours) |
| Submitted time | 2024-09-26 13:50:49 |
| Site | IT_CNAF |
| Entry | CMSHTPC_T1_IT_CNAF_condor_ce02 |
| Last heartbeat | 2024-09-26 15:45:24 |
| From worker node | Hostname | cn-608-06-07.cr.cnaf.infn.it |
| cpuinfo | Intel(R) Xeon(R) CPU E5-2697 v4 @ 2.30GHz |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 216000 (60 hours) |
| Inner Apptainer? | True |
| Job state | jobscript_error |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk |
| Started | 2024-09-26 13:52:11 |
| Input files | hd-protodune:np04hd_raw_run029097_0269_dataflow6_datawriter_0_20240906T155447.hdf5
|
| Jobscript | Exit code | 1 |
| Real time | 0m (0s) |
| CPU time | 0m (0s = 0%) |
| Outputting started | |
| Output files | |
| Finished | 2024-09-26 15:45:24 |
| Saved logs | justin-logs:263843.5-justin-prod-sched01.dune.hep.ac.uk.logs.tgz |
| List job events Wrapper job log |
Jobscript log (last 10,000 characters)
6
5625 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:0 11 XUs and 2 XVs -> 1 XUVs
C:0 T:1 50 XUs and 28 XVs -> 2 XUVs
C:0 T:2 17629 XUs and 29230 XVs -> 1132 XUVs
C:0 T:3 5 XUs and 17 XVs -> 0 XUVs
C:0 T:4 49 XUs and 84 XVs -> 2 XUVs
C:0 T:5 25749 XUs and 22529 XVs -> 915 XUVs
C:0 T:6 34669 XUs and 52241 XVs -> 4223 XUVs
C:0 T:7 1776 XUs and 2968 XVs -> 60 XUVs
6335 XUVs total
1899 collection wire objects
6335 potential space points
Neighbour search...
575971 tests to find 313528 neighbours
Iterating with no regularization...
Begin: 9.14564e+06
0 7.75488e+06
1 7.65774e+06
2 7.64074e+06
3 7.6349e+06
Now with regularization...
Begin: 6.8782e+06
0 6.87382e+06
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Begin processing the 29th record. run: 29097 subRun: 1 event: 64790 at 26-Sep-2024 17:38:59 CEST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 60815 traces, 60815 summary
Retagger: tagging trace set: gauss with 44486 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 44486 traces tagged "gauss"
FrameSaver: q=982919 n=844404 tag=gauss
wclsFrameSaver: saving 60815 traces tagged "wiener"
FrameSaver: q=1.09459e+06 n=777056 tag=wiener
2020 hits in TPC 0
2986 hits in TPC 1
6414 hits in TPC 2
4610 hits in TPC 3
3100 hits in TPC 4
5359 hits in TPC 5
4086 hits in TPC 6
4149 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 4247 XUs and 9673 XVs -> 1296 XUVs
C:0 T:2 66592 XUs and 29789 XVs -> 1222 XUVs
C:0 T:3 46 XUs and 44 XVs -> 0 XUVs
C:0 T:4 58 XUs and 132 XVs -> 1 XUVs
C:0 T:5 10662 XUs and 34806 XVs -> 1139 XUVs
C:0 T:6 50678 XUs and 26275 XVs -> 795 XUVs
C:0 T:7 293 XUs and 270 XVs -> 9 XUVs
4462 XUVs total
1806 collection wire objects
4462 potential space points
Neighbour search...
203152 tests to find 122726 neighbours
Iterating with no regularization...
Begin: 9.36756e+06
0 8.79368e+06
1 8.75546e+06
2 8.7519e+06
Now with regularization...
Begin: 8.31672e+06
0 8.31543e+06
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Begin processing the 30th record. run: 29097 subRun: 1 event: 64798 at 26-Sep-2024 17:41:18 CEST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 89105 traces, 89105 summary
Retagger: tagging trace set: gauss with 67317 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 67317 traces tagged "gauss"
FrameSaver: q=1.54945e+06 n=1279208 tag=gauss
wclsFrameSaver: saving 89105 traces tagged "wiener"
FrameSaver: q=1.64481e+06 n=1159710 tag=wiener
4820 hits in TPC 0
3932 hits in TPC 1
4726 hits in TPC 2
3202 hits in TPC 3
6083 hits in TPC 4
9041 hits in TPC 5
9070 hits in TPC 6
5225 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:0 54 XUs and 74 XVs -> 5 XUVs
C:0 T:1 300 XUs and 361 XVs -> 4 XUVs
C:0 T:2 11905 XUs and 7627 XVs -> 214 XUVs
C:0 T:3 142 XUs and 143 XVs -> 1 XUVs
C:0 T:4 453 XUs and 548 XVs -> 11 XUVs
C:0 T:5 38169 XUs and 43808 XVs -> 1364 XUVs
C:0 T:6 27671 XUs and 29435 XVs -> 1844 XUVs
C:0 T:7 175 XUs and 205 XVs -> 12 XUVs
3455 XUVs total
1959 collection wire objects
3455 potential space points
Neighbour search...
59595 tests to find 34850 neighbours
Iterating with no regularization...
Begin: 1.17056e+07
0 1.09139e+07
1 1.08614e+07
2 1.08571e+07
Now with regularization...
Begin: 1.0043e+07
0 1.00415e+07
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
no of hits closeby 0 event 64798 TrkackID 31 startx, y, z -31.0729 111.411 240.827 wireno, peakt tpcno 72 5637.15 6 dist 0min T, max_T5471.87 5855.74
no of hits closeby 0 event 64798 TrkackID 31 startx, y, z -31.0729 111.411 240.827 wireno, peakt tpcno 72 5637.15 6 dist 0min T, max_T5471.87 5855.74
26-Sep-2024 17:44:55 CEST Closed output file "np04hd_raw_run029097_0269_dataflow6_datawriter_0_20240906T155447_reco_stage1_reco_stage2_20240926T154455_keepup.root"
26-Sep-2024 17:44:55 CEST Closed input file "np04hd_raw_run029097_0269_dataflow6_datawriter_0_20240906T155447_reco_stage1.root"
========================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
========================================================================================================================================
Full event 104.501 167.403 233.413 167.984 31.1627 30
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.000560188 0.00314817 0.00472349 0.00331672 0.00107183 30
produce:fembfilter:PDHDFEMBFilter 0.701074 1.03105 1.31761 1.06538 0.133984 30
produce:wclsdatahd:WireCellToolkit 62.2924 73.8679 84.8615 75.2128 5.79983 30
produce:gaushit:GausHitFinder 0.59164 1.15066 1.66825 1.17439 0.307057 30
produce:nhitsfilter:NumberOfHitsFilter 0.000358382 0.000562192 0.0011497 0.000557644 0.000147753 30
produce:reco3d:SpacePointSolver 0.313475 2.12844 7.79226 1.63277 1.81883 30
produce:hitpdune:DisambigFromSpacePoints 0.343367 1.11082 2.26034 1.10052 0.570479 30
produce:pandora:StandardPandora 7.58857 21.9482 48.1199 19.1163 9.95935 30
produce:pandoraWriter:StandardPandora 0.101728 0.241932 0.409802 0.253587 0.0725805 30
produce:pandoraTrack:LArPandoraTrackCreation 0.993152 3.4112 6.60818 3.67757 1.45625 30
produce:pandoraShower:LArPandoraShowerCreation 0.708712 3.95973 11.7882 3.99114 2.13821 30
produce:pandoracalo:Calorimetry 0.390451 1.54344 3.05721 1.58465 0.682612 30
produce:pandoracalonosce:Calorimetry 0.354535 1.43488 2.94574 1.44914 0.661397 30
produce:pandorapid:Chi2ParticleID 0.00154579 0.00313679 0.00518448 0.0033528 0.000996413 30
produce:pandoraShowercalo:ShowerCalorimetry 0.593621 3.35174 10.4007 3.29088 1.91622 30
produce:pandoraShowercalonosce:ShowerCalorimetry 0.630037 3.25846 10.1251 3.36106 1.80668 30
produce:pandoraGnocchiCalo:GnocchiCalorimetry 0.0788608 0.234877 0.450979 0.245264 0.0762357 30
produce:pandoraGnocchiCalonosce:GnocchiCalorimetry 0.0294496 0.0879466 0.146627 0.0904624 0.0259001 30
produce:emtrkmichelid:EmTrackMichelId 19.7769 45.855 72.6195 46.3646 11.5301 30
[art]:TriggerResults:TriggerResultInserter 4.0761e-05 6.30148e-05 0.000135809 6.29425e-05 1.77109e-05 30
end_path:out1:RootOutput 8.703e-06 1.58867e-05 4.6111e-05 1.34475e-05 7.89866e-06 30
end_path:michelremoving:michelremoving 0.0317209 0.123219 0.295002 0.0913034 0.0787846 30
end_path:michelremovinggnocchi:michelremoving 0.0297164 0.114555 0.291995 0.0831092 0.0750849 30
end_path:calibanagnocchi:CalibAnaTree 0.321436 1.13624 2.3545 1.0962 0.514613 30
end_path:out1:RootOutput(write) 0.448772 1.21588 1.98147 1.19847 0.358653 30
========================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 4926.19 MB
Peak resident set size usage (VmHWM): 2329.27 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np04hd_raw_run029097_0269_dataflow6_datawriter_0_20240906T155447_reco_stage1_reco_stage2_20240926T154455_keepup.root
\tHists: np04hd_raw_run029097_0269_dataflow6_datawriter_0_20240906T155447_reco_stage1_reco_stage2_20240926T154455_keepup_hists.root
Forming reco metadata
/cvmfs/larsoft.opensciencegrid.org/products/python/v3_9_15/Linux64bit+3.10-2.17/bin/python: No module named meta_maker
Error in reco metadata
