Jobsub ID 78692.97@justin-prod-sched02.dune.hep.ac.uk
| Jobsub ID | 78692.97@justin-prod-sched02.dune.hep.ac.uk |
| Workflow ID | 3447 |
| Stage ID | 1 |
| User name | calcuttj@fnal.gov |
| HTCondor Group | group_dune.prod_mcsim |
| Requested | Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 18000 (5 hours) |
| Submitted time | 2024-09-26 11:28:49 |
| Site | IT_CNAF |
| Entry | CMSHTPC_T1_IT_CNAF_condor_ce02 |
| Last heartbeat | 2024-09-26 14:23:30 |
| From worker node | Hostname | cn-609-02-01.cr.cnaf.infn.it |
| cpuinfo | Intel(R) Xeon(R) CPU E5-2697 v4 @ 2.30GHz |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 216000 (60 hours) |
| Inner Apptainer? | True |
| Job state | jobscript_error |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk |
| Started | 2024-09-26 12:17:55 |
| Input files | hd-protodune:np04hd_raw_run029081_0549_dataflow3_datawriter_0_20240905T012746.hdf5
|
| Jobscript | Exit code | 1 |
| Real time | 0m (0s) |
| CPU time | 0m (0s = 0%) |
| Outputting started | |
| Output files | |
| Finished | 2024-09-26 14:23:30 |
| Saved logs | justin-logs:78692.97-justin-prod-sched02.dune.hep.ac.uk.logs.tgz |
| List job events Wrapper job log |
Jobscript log (last 10,000 characters)
06
Now with regularization...
Begin: 4.74299e+06
0 4.7415e+06
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
no of hits closeby 1 event 131979 TrkackID 5 startx, y, z 10.9309 589.544 299.576 wireno, peakt tpcno 399 10.6044 5 dist 0min T, max_T0.866623 1875.32
no of hits closeby 1 event 131979 TrkackID 5 startx, y, z 10.9309 589.544 299.576 wireno, peakt tpcno 399 10.6044 5 dist 0min T, max_T0.866623 1875.32
Begin processing the 29th record. run: 29081 subRun: 1 event: 131987 at 26-Sep-2024 16:18:02 CEST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 39655 traces, 39655 summary
Retagger: tagging trace set: gauss with 30908 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 30908 traces tagged "gauss"
FrameSaver: q=672597 n=615663 tag=gauss
wclsFrameSaver: saving 39655 traces tagged "wiener"
FrameSaver: q=709685 n=565298 tag=wiener
1914 hits in TPC 0
1904 hits in TPC 1
2813 hits in TPC 2
1339 hits in TPC 3
3990 hits in TPC 4
5326 hits in TPC 5
3401 hits in TPC 6
1878 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 7 XUs and 7 XVs -> 0 XUVs
C:0 T:2 4215 XUs and 11383 XVs -> 214 XUVs
C:0 T:3 36 XUs and 62 XVs -> 9 XUVs
C:0 T:4 273 XUs and 441 XVs -> 20 XUVs
C:0 T:5 10131 XUs and 14571 XVs -> 898 XUVs
C:0 T:6 4521 XUs and 20422 XVs -> 252 XUVs
C:0 T:7 74 XUs and 102 XVs -> 2 XUVs
1395 XUVs total
634 collection wire objects
1395 potential space points
Neighbour search...
34041 tests to find 21024 neighbours
Iterating with no regularization...
Begin: 3.5127e+06
0 3.08118e+06
1 3.05434e+06
2 3.04936e+06
3 3.04773e+06
Now with regularization...
Begin: 2.83805e+06
0 2.83732e+06
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
no of hits closeby 2 event 131987 TrkackID 1 startx, y, z 98.0863 593.226 420.334 wireno, peakt tpcno 431 13.5 5 dist 0.379335min T, max_T5 1433.5
no of hits closeby 0 event 131987 TrkackID 3 startx, y, z 57.8553 593.855 304.647 wireno, peakt tpcno 2 5103.52 6 dist 363.424min T, max_T3760.4 5165.36
no of hits closeby 2 event 131987 TrkackID 1 startx, y, z 98.0863 593.226 420.334 wireno, peakt tpcno 431 13.5 5 dist 0.379335min T, max_T5 1433.5
no of hits closeby 0 event 131987 TrkackID 3 startx, y, z 57.8553 593.855 304.647 wireno, peakt tpcno 2 5103.52 6 dist 363.424min T, max_T3760.4 5165.36
Begin processing the 30th record. run: 29081 subRun: 1 event: 132003 at 26-Sep-2024 16:20:38 CEST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 44074 traces, 44074 summary
Retagger: tagging trace set: gauss with 33817 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 33817 traces tagged "gauss"
FrameSaver: q=705974 n=644053 tag=gauss
wclsFrameSaver: saving 44074 traces tagged "wiener"
FrameSaver: q=742133 n=583269 tag=wiener
2159 hits in TPC 0
1757 hits in TPC 1
3953 hits in TPC 2
2638 hits in TPC 3
2153 hits in TPC 4
3369 hits in TPC 5
4584 hits in TPC 6
2163 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 12 XUs and 27 XVs -> 1 XUVs
C:0 T:2 9119 XUs and 9663 XVs -> 616 XUVs
C:0 T:3 70 XUs and 85 XVs -> 0 XUVs
C:0 T:4 42 XUs and 46 XVs -> 2 XUVs
C:0 T:5 4780 XUs and 10481 XVs -> 265 XUVs
C:0 T:6 15019 XUs and 12465 XVs -> 239 XUVs
C:0 T:7 125 XUs and 100 XVs -> 0 XUVs
1123 XUVs total
799 collection wire objects
1123 potential space points
Neighbour search...
11885 tests to find 6768 neighbours
Iterating with no regularization...
Begin: 2.44297e+06
0 2.35298e+06
1 2.34738e+06
2 2.34679e+06
Now with regularization...
Begin: 2.10036e+06
0 2.10018e+06
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
26-Sep-2024 16:22:58 CEST Closed output file "np04hd_raw_run029081_0549_dataflow3_datawriter_0_20240905T012746_reco_stage1_reco_stage2_20240926T142258_keepup.root"
26-Sep-2024 16:22:58 CEST Closed input file "np04hd_raw_run029081_0549_dataflow3_datawriter_0_20240905T012746_reco_stage1.root"
========================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
========================================================================================================================================
Full event 114.478 187.703 282.861 182.101 41.6483 30
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.00092939 0.00291997 0.00608881 0.00285426 0.00137355 30
produce:fembfilter:PDHDFEMBFilter 0.686218 1.02365 1.28744 1.07616 0.139927 30
produce:wclsdatahd:WireCellToolkit 60.9328 80.3821 90.7985 81.1729 6.34259 30
produce:gaushit:GausHitFinder 0.607877 1.27283 1.93702 1.2621 0.374091 30
produce:nhitsfilter:NumberOfHitsFilter 0.000367728 0.000600042 0.00104335 0.000591136 0.000174565 30
produce:reco3d:SpacePointSolver 0.315358 1.44911 4.9969 1.19305 1.01108 30
produce:hitpdune:DisambigFromSpacePoints 0.34044 1.26655 3.0612 1.06612 0.739404 30
produce:pandora:StandardPandora 9.32117 25.6568 56.5163 23.4782 12.0336 30
produce:pandoraWriter:StandardPandora 0.128904 0.261018 0.442579 0.248916 0.0778363 30
produce:pandoraTrack:LArPandoraTrackCreation 1.13517 4.03346 9.39967 3.58633 1.96254 30
produce:pandoraShower:LArPandoraShowerCreation 1.12218 4.94498 14.4667 4.21857 3.01883 30
produce:pandoracalo:Calorimetry 0.582353 1.79444 4.50574 1.704 0.811309 30
produce:pandoracalonosce:Calorimetry 0.481807 1.68049 4.28878 1.61976 0.788453 30
produce:pandorapid:Chi2ParticleID 0.00155167 0.00334881 0.0052578 0.00353052 0.000898013 30
produce:pandoraShowercalo:ShowerCalorimetry 0.865657 4.27979 14.3452 3.73762 2.72502 30
produce:pandoraShowercalonosce:ShowerCalorimetry 0.869562 4.17066 12.5355 3.67022 2.52322 30
produce:pandoraGnocchiCalo:GnocchiCalorimetry 0.106221 0.253903 0.459404 0.241696 0.0754889 30
produce:pandoraGnocchiCalonosce:GnocchiCalorimetry 0.0462903 0.101095 0.156472 0.0997139 0.0297226 30
produce:emtrkmichelid:EmTrackMichelId 27.895 51.8767 84.4442 49.0354 15.3625 30
[art]:TriggerResults:TriggerResultInserter 4.1972e-05 7.65682e-05 0.000157797 7.79235e-05 2.48086e-05 30
end_path:out1:RootOutput 7.865e-06 1.76693e-05 4.1458e-05 1.8023e-05 6.95732e-06 30
end_path:michelremoving:michelremoving 0.0349178 0.163824 0.492995 0.111706 0.123592 30
end_path:michelremovinggnocchi:michelremoving 0.0346263 0.160804 0.48027 0.0987683 0.123502 30
end_path:calibanagnocchi:CalibAnaTree 0.40273 1.38699 3.30688 1.2943 0.690274 30
end_path:out1:RootOutput(write) 0.735982 1.3259 2.08156 1.29021 0.370459 30
========================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 4944.83 MB
Peak resident set size usage (VmHWM): 2370.01 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np04hd_raw_run029081_0549_dataflow3_datawriter_0_20240905T012746_reco_stage1_reco_stage2_20240926T142258_keepup.root
\tHists: np04hd_raw_run029081_0549_dataflow3_datawriter_0_20240905T012746_reco_stage1_reco_stage2_20240926T142258_keepup_hists.root
Forming reco metadata
/cvmfs/larsoft.opensciencegrid.org/products/python/v3_9_15/Linux64bit+3.10-2.17/bin/python: No module named meta_maker
Error in reco metadata
