Jobsub ID 78687.113@justin-prod-sched02.dune.hep.ac.uk
| Jobsub ID | 78687.113@justin-prod-sched02.dune.hep.ac.uk |
| Workflow ID | 3447 |
| Stage ID | 1 |
| User name | calcuttj@fnal.gov |
| HTCondor Group | group_dune.prod_mcsim |
| Requested | Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 18000 (5 hours) |
| Submitted time | 2024-09-26 11:27:14 |
| Site | IT_CNAF |
| Entry | CMSHTPC_T1_IT_CNAF_condor_ce02 |
| Last heartbeat | 2024-09-26 14:15:02 |
| From worker node | Hostname | cn-608-05-05.cr.cnaf.infn.it |
| cpuinfo | Intel(R) Xeon(R) CPU E5-2697 v4 @ 2.30GHz |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 216000 (60 hours) |
| Inner Apptainer? | True |
| Job state | jobscript_error |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk |
| Started | 2024-09-26 12:14:34 |
| Input files | hd-protodune:np04hd_raw_run029081_0568_dataflow1_datawriter_0_20240905T014253.hdf5
|
| Jobscript | Exit code | 1 |
| Real time | 0m (0s) |
| CPU time | 0m (0s = 0%) |
| Outputting started | |
| Output files | |
| Finished | 2024-09-26 14:15:02 |
| Saved logs | justin-logs:78687.113-justin-prod-sched02.dune.hep.ac.uk.logs.tgz |
| List job events Wrapper job log |
Jobscript log (last 10,000 characters)
mParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
no of hits closeby 0 event 136561 TrkackID 20 startx, y, z 47.9344 149.677 212.707 wireno, peakt tpcno 440 606.5 2 dist 22.4653min T, max_T6.20323 608.231
no of hits closeby 0 event 136561 TrkackID 20 startx, y, z 47.9344 149.677 212.707 wireno, peakt tpcno 440 606.5 2 dist 22.4653min T, max_T6.20323 608.231
Begin processing the 29th record. run: 29081 subRun: 1 event: 136569 at 26-Sep-2024 16:09:11 CEST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 79924 traces, 79924 summary
Retagger: tagging trace set: gauss with 58608 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 58608 traces tagged "gauss"
FrameSaver: q=1.17987e+06 n=1054211 tag=gauss
wclsFrameSaver: saving 79924 traces tagged "wiener"
FrameSaver: q=1.2661e+06 n=940233 tag=wiener
3124 hits in TPC 0
4075 hits in TPC 1
8969 hits in TPC 2
5436 hits in TPC 3
2869 hits in TPC 4
4947 hits in TPC 5
7159 hits in TPC 6
3219 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 1189 XUs and 1574 XVs -> 286 XUVs
C:0 T:2 71324 XUs and 68137 XVs -> 1429 XUVs
C:0 T:3 541 XUs and 509 XVs -> 6 XUVs
C:0 T:4 29 XUs and 38 XVs -> 3 XUVs
C:0 T:5 18718 XUs and 27122 XVs -> 1078 XUVs
C:0 T:6 37239 XUs and 42973 XVs -> 1979 XUVs
C:0 T:7 335 XUs and 327 XVs -> 9 XUVs
4790 XUVs total
1504 collection wire objects
4790 potential space points
Neighbour search...
317272 tests to find 175672 neighbours
Iterating with no regularization...
Begin: 8.78457e+06
0 7.82405e+06
1 7.74854e+06
2 7.73929e+06
3 7.73653e+06
Now with regularization...
Begin: 7.49228e+06
0 7.49056e+06
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Begin processing the 30th record. run: 29081 subRun: 1 event: 136577 at 26-Sep-2024 16:12:02 CEST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 61736 traces, 61736 summary
Retagger: tagging trace set: gauss with 46657 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 46657 traces tagged "gauss"
FrameSaver: q=1.05429e+06 n=869293 tag=gauss
wclsFrameSaver: saving 61736 traces tagged "wiener"
FrameSaver: q=1.12847e+06 n=788856 tag=wiener
3779 hits in TPC 0
4071 hits in TPC 1
2744 hits in TPC 2
2242 hits in TPC 3
4364 hits in TPC 4
5595 hits in TPC 5
5596 hits in TPC 6
4478 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 6063 XUs and 8515 XVs -> 552 XUVs
C:0 T:2 3960 XUs and 4615 XVs -> 109 XUVs
C:0 T:3 125 XUs and 203 XVs -> 5 XUVs
C:0 T:4 215 XUs and 377 XVs -> 27 XUVs
C:0 T:5 24608 XUs and 16168 XVs -> 670 XUVs
C:0 T:6 13361 XUs and 17683 XVs -> 1037 XUVs
C:0 T:7 57 XUs and 97 XVs -> 3 XUVs
2403 XUVs total
1334 collection wire objects
2403 potential space points
Neighbour search...
54837 tests to find 33612 neighbours
Iterating with no regularization...
Begin: 6.61384e+06
0 6.27833e+06
1 6.25689e+06
2 6.25513e+06
Now with regularization...
Begin: 5.83378e+06
0 5.83315e+06
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
no of hits closeby 2 event 136577 TrkackID 0 startx, y, z 88.4163 409.335 3.95287 wireno, peakt tpcno 443 3906.4 5 dist 0min T, max_T3747.2 4867.47
no of hits closeby 0 event 136577 TrkackID 21 startx, y, z 14.4999 595.445 90.5342 wireno, peakt tpcno 194 4642.37 2 dist 111.897min T, max_T3856.51 4681.49
no of hits closeby 2 event 136577 TrkackID 0 startx, y, z 88.4163 409.335 3.95287 wireno, peakt tpcno 443 3906.4 5 dist 0min T, max_T3747.2 4867.47
no of hits closeby 0 event 136577 TrkackID 21 startx, y, z 14.4999 595.445 90.5342 wireno, peakt tpcno 194 4642.37 2 dist 111.897min T, max_T3856.51 4681.49
26-Sep-2024 16:14:35 CEST Closed output file "np04hd_raw_run029081_0568_dataflow1_datawriter_0_20240905T014253_reco_stage1_reco_stage2_20240926T141435_keepup.root"
26-Sep-2024 16:14:35 CEST Closed input file "np04hd_raw_run029081_0568_dataflow1_datawriter_0_20240905T014253_reco_stage1.root"
========================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
========================================================================================================================================
Full event 105.228 182.825 295.547 172.823 44.5362 30
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.000485672 0.00322976 0.00614337 0.0030004 0.00161258 30
produce:fembfilter:PDHDFEMBFilter 0.679798 0.941759 1.17409 1.04966 0.173923 30
produce:wclsdatahd:WireCellToolkit 67.2899 74.6731 81.6875 75.28 3.30367 30
produce:gaushit:GausHitFinder 0.469622 1.24274 2.12208 1.216 0.384073 30
produce:nhitsfilter:NumberOfHitsFilter 0.000279877 0.000556508 0.000871535 0.000534986 0.000142805 30
produce:reco3d:SpacePointSolver 0.298883 2.22865 10.3098 1.69399 1.88244 30
produce:hitpdune:DisambigFromSpacePoints 0.323648 1.52822 4.88598 1.27894 1.12313 30
produce:pandora:StandardPandora 8.17835 28.3728 69.4929 23.2672 16.034 30
produce:pandoraWriter:StandardPandora 0.131256 0.261393 0.448289 0.251265 0.0741047 30
produce:pandoraTrack:LArPandoraTrackCreation 1.06865 3.81964 8.45995 3.25076 2.04838 30
produce:pandoraShower:LArPandoraShowerCreation 0.681974 4.7506 12.8524 4.22099 3.15973 30
produce:pandoracalo:Calorimetry 0.502781 1.7876 4.35447 1.59904 0.9524 30
produce:pandoracalonosce:Calorimetry 0.43292 1.63998 4.03276 1.44885 0.896226 30
produce:pandorapid:Chi2ParticleID 0.0013084 0.00320628 0.0050845 0.00320154 0.000895962 30
produce:pandoraShowercalo:ShowerCalorimetry 0.533631 4.1566 10.8296 3.86765 2.70232 30
produce:pandoraShowercalonosce:ShowerCalorimetry 0.524759 4.05719 11.511 3.59422 2.76636 30
produce:pandoraGnocchiCalo:GnocchiCalorimetry 0.0877418 0.243729 0.438209 0.22949 0.0935122 30
produce:pandoraGnocchiCalonosce:GnocchiCalorimetry 0.0347374 0.0927064 0.163565 0.0932857 0.0349212 30
produce:emtrkmichelid:EmTrackMichelId 18.0935 50.0818 86.3259 47.7726 14.3964 30
[art]:TriggerResults:TriggerResultInserter 3.9893e-05 7.19396e-05 0.000125785 6.8049e-05 2.10854e-05 30
end_path:out1:RootOutput 8.993e-06 1.71474e-05 3.5545e-05 1.5514e-05 5.96629e-06 30
end_path:michelremoving:michelremoving 0.0292267 0.164586 0.594267 0.0918826 0.150696 30
end_path:michelremovinggnocchi:michelremoving 0.0274467 0.153707 0.593144 0.0847002 0.143984 30
end_path:calibanagnocchi:CalibAnaTree 0.281461 1.24553 3.11658 1.15273 0.701245 30
end_path:out1:RootOutput(write) 0.502678 1.19122 1.92178 1.15035 0.409149 30
========================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 4973.79 MB
Peak resident set size usage (VmHWM): 2394.46 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np04hd_raw_run029081_0568_dataflow1_datawriter_0_20240905T014253_reco_stage1_reco_stage2_20240926T141435_keepup.root
\tHists: np04hd_raw_run029081_0568_dataflow1_datawriter_0_20240905T014253_reco_stage1_reco_stage2_20240926T141435_keepup_hists.root
Forming reco metadata
/cvmfs/larsoft.opensciencegrid.org/products/python/v3_9_15/Linux64bit+3.10-2.17/bin/python: No module named meta_maker
Error in reco metadata
