Jobsub ID 263254.159@justin-prod-sched01.dune.hep.ac.uk
| Jobsub ID | 263254.159@justin-prod-sched01.dune.hep.ac.uk |
| Workflow ID | 3447 |
| Stage ID | 1 |
| User name | calcuttj@fnal.gov |
| HTCondor Group | group_dune.prod_mcsim |
| Requested | Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 18000 (5 hours) |
| Submitted time | 2024-09-26 09:35:43 |
| Site | FR_CCIN2P3 |
| Entry | DUNE_FR_CCIN2P3_cccondorce02 |
| Last heartbeat | 2024-09-26 10:46:29 |
| From worker node | Hostname | ccwcondor0779 |
| cpuinfo | AMD EPYC 7302 16-Core Processor |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 106200 (29 hours) |
| Inner Apptainer? | True |
| Job state | jobscript_error |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk |
| Started | 2024-09-26 09:37:47 |
| Input files | hd-protodune:np04hd_raw_run029081_0570_dataflow6_datawriter_0_20240905T014432.hdf5
|
| Jobscript | Exit code | 1 |
| Real time | 0m (0s) |
| CPU time | 0m (0s = 0%) |
| Outputting started | |
| Output files | |
| Finished | 2024-09-26 10:46:29 |
| Saved logs | justin-logs:263254.159-justin-prod-sched01.dune.hep.ac.uk.logs.tgz |
| List job events Wrapper job log |
Jobscript log (last 10,000 characters)
-----------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
no of hits closeby 13 event 137022 TrkackID 2 startx, y, z -149.152 436.003 375.236 wireno, peakt tpcno 20 118.887 2 dist 0min T, max_T5.85126 1918.74
no of hits closeby 13 event 137022 TrkackID 2 startx, y, z -149.152 436.003 375.236 wireno, peakt tpcno 20 118.887 2 dist 0min T, max_T5.85126 1918.74
Begin processing the 29th record. run: 29081 subRun: 1 event: 137030 at 26-Sep-2024 12:34:44 CEST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 67699 traces, 67699 summary
Retagger: tagging trace set: gauss with 49365 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 49365 traces tagged "gauss"
FrameSaver: q=1.07703e+06 n=871399 tag=gauss
wclsFrameSaver: saving 67699 traces tagged "wiener"
FrameSaver: q=1.17604e+06 n=789039 tag=wiener
3718 hits in TPC 0
3156 hits in TPC 1
6996 hits in TPC 2
4598 hits in TPC 3
3268 hits in TPC 4
4281 hits in TPC 5
4348 hits in TPC 6
3279 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 4767 XUs and 3656 XVs -> 117 XUVs
C:0 T:2 23319 XUs and 22553 XVs -> 884 XUVs
C:0 T:3 72 XUs and 106 XVs -> 6 XUVs
C:0 T:4 139 XUs and 268 XVs -> 5 XUVs
C:0 T:5 28501 XUs and 24679 XVs -> 931 XUVs
C:0 T:6 41461 XUs and 21428 XVs -> 457 XUVs
C:0 T:7 94 XUs and 199 XVs -> 5 XUVs
2405 XUVs total
1172 collection wire objects
2405 potential space points
Neighbour search...
77591 tests to find 38996 neighbours
Iterating with no regularization...
Begin: 1.40383e+07
0 1.35709e+07
1 1.35304e+07
2 1.35243e+07
Now with regularization...
Begin: 1.29341e+07
0 1.29324e+07
BdtBeamParticleIdTool::SliceFeatures::GetLArTPCIntercepts - inconsistent number of intercepts between a line and the LArTPC
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
no of hits closeby 1 event 137030 TrkackID 9 startx, y, z -117.052 283.981 251.324 wireno, peakt tpcno 36 1148.86 2 dist 0min T, max_T7.05432 1505
no of hits closeby 1 event 137030 TrkackID 9 startx, y, z -117.052 283.981 251.324 wireno, peakt tpcno 36 1148.86 2 dist 0min T, max_T7.05432 1505
Begin processing the 30th record. run: 29081 subRun: 1 event: 137054 at 26-Sep-2024 12:36:06 CEST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 61333 traces, 61333 summary
Retagger: tagging trace set: gauss with 43362 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 43362 traces tagged "gauss"
FrameSaver: q=917613 n=723030 tag=gauss
wclsFrameSaver: saving 61333 traces tagged "wiener"
FrameSaver: q=1.00291e+06 n=638317 tag=wiener
1677 hits in TPC 0
1983 hits in TPC 1
5392 hits in TPC 2
3068 hits in TPC 3
3927 hits in TPC 4
5972 hits in TPC 5
5147 hits in TPC 6
3276 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 1226 XUs and 516 XVs -> 21 XUVs
C:0 T:2 37139 XUs and 64781 XVs -> 2558 XUVs
C:0 T:3 353 XUs and 454 XVs -> 11 XUVs
C:0 T:4 41 XUs and 68 XVs -> 2 XUVs
C:0 T:5 39621 XUs and 12172 XVs -> 410 XUVs
C:0 T:6 29593 XUs and 40191 XVs -> 1013 XUVs
C:0 T:7 382 XUs and 282 XVs -> 9 XUVs
4024 XUVs total
1241 collection wire objects
4024 potential space points
Neighbour search...
176662 tests to find 93668 neighbours
Iterating with no regularization...
Begin: 1.14545e+07
0 1.00627e+07
1 9.97327e+06
2 9.95435e+06
3 9.94652e+06
Now with regularization...
Begin: 9.53015e+06
0 9.52489e+06
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
no of hits closeby 0 event 137054 TrkackID 13 startx, y, z 10.8614 595.992 123.636 wireno, peakt tpcno 368 3589.41 2 dist 115.416min T, max_T3473.76 3616.8
no of hits closeby 0 event 137054 TrkackID 13 startx, y, z 10.8614 595.992 123.636 wireno, peakt tpcno 368 3589.41 2 dist 115.416min T, max_T3473.76 3616.8
26-Sep-2024 12:37:18 CEST Closed output file "np04hd_raw_run029081_0570_dataflow6_datawriter_0_20240905T014432_reco_stage1_reco_stage2_20240926T103718_keepup.root"
26-Sep-2024 12:37:18 CEST Closed input file "np04hd_raw_run029081_0570_dataflow6_datawriter_0_20240905T014432_reco_stage1.root"
========================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
========================================================================================================================================
Full event 65.298 87.6299 122.411 86.4922 13.6195 30
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.000567428 0.00251644 0.00529009 0.00241883 0.00136533 30
produce:fembfilter:PDHDFEMBFilter 0.535813 0.629711 0.811632 0.612735 0.0610931 30
produce:wclsdatahd:WireCellToolkit 36.2959 40.0012 42.1397 40.2329 1.32951 30
produce:gaushit:GausHitFinder 0.346661 0.578 0.902131 0.578907 0.140222 30
produce:nhitsfilter:NumberOfHitsFilter 0.000242656 0.000521486 0.00351947 0.00041967 0.000566013 30
produce:reco3d:SpacePointSolver 0.177666 0.868347 2.50877 0.812301 0.528833 30
produce:hitpdune:DisambigFromSpacePoints 0.138817 0.440721 0.947364 0.429131 0.193168 30
produce:pandora:StandardPandora 4.34207 11.5375 32.1979 9.92993 5.82015 30
produce:pandoraWriter:StandardPandora 0.0757935 0.140197 0.272533 0.140633 0.0402086 30
produce:pandoraTrack:LArPandoraTrackCreation 0.373411 0.991588 1.77298 0.89183 0.374024 30
produce:pandoraShower:LArPandoraShowerCreation 0.446591 1.21178 3.46926 1.01731 0.611728 30
produce:pandoracalo:Calorimetry 0.203306 0.514948 0.978616 0.510818 0.166518 30
produce:pandoracalonosce:Calorimetry 0.190786 0.461624 1.01587 0.473636 0.170172 30
produce:pandorapid:Chi2ParticleID 0.000985073 0.00161091 0.00292765 0.00146646 0.00049258 30
produce:pandoraShowercalo:ShowerCalorimetry 0.395967 1.11187 2.91382 1.05314 0.511456 30
produce:pandoraShowercalonosce:ShowerCalorimetry 0.342127 1.06247 2.62589 1.01581 0.446571 30
produce:pandoraGnocchiCalo:GnocchiCalorimetry 0.0704291 0.119136 0.192872 0.112534 0.0289522 30
produce:pandoraGnocchiCalonosce:GnocchiCalorimetry 0.0238692 0.0421845 0.0772461 0.0403538 0.0122037 30
produce:emtrkmichelid:EmTrackMichelId 17.347 26.5024 42.2098 25.9793 5.27607 30
[art]:TriggerResults:TriggerResultInserter 1.7853e-05 4.19278e-05 9.9016e-05 3.54215e-05 2.18352e-05 30
end_path:out1:RootOutput 7.454e-06 1.16032e-05 3.217e-05 9.663e-06 5.50524e-06 30
end_path:michelremoving:michelremoving 0.0188028 0.0801302 0.245417 0.0431065 0.0644506 30
end_path:michelremovinggnocchi:michelremoving 0.0157469 0.0770689 0.280022 0.0444578 0.0663325 30
end_path:calibanagnocchi:CalibAnaTree 0.187294 0.375929 0.758464 0.387569 0.12358 30
end_path:out1:RootOutput(write) 0.421871 0.723663 1.04482 0.722799 0.157263 30
========================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 4947.48 MB
Peak resident set size usage (VmHWM): 2367.77 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np04hd_raw_run029081_0570_dataflow6_datawriter_0_20240905T014432_reco_stage1_reco_stage2_20240926T103718_keepup.root
\tHists: np04hd_raw_run029081_0570_dataflow6_datawriter_0_20240905T014432_reco_stage1_reco_stage2_20240926T103718_keepup_hists.root
Forming reco metadata
/cvmfs/larsoft.opensciencegrid.org/products/python/v3_9_15/Linux64bit+3.10-2.17/bin/python: No module named meta_maker
Error in reco metadata
