Jobsub ID 263270.148@justin-prod-sched01.dune.hep.ac.uk
| Jobsub ID | 263270.148@justin-prod-sched01.dune.hep.ac.uk |
| Workflow ID | 3447 |
| Stage ID | 1 |
| User name | calcuttj@fnal.gov |
| HTCondor Group | group_dune.prod_mcsim |
| Requested | Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 18000 (5 hours) |
| Submitted time | 2024-09-26 09:52:09 |
| Site | FR_CCIN2P3 |
| Entry | DUNE_FR_CCIN2P3_cccondorce01 |
| Last heartbeat | 2024-09-26 11:09:46 |
| From worker node | Hostname | ccwcondor0577 |
| cpuinfo | AMD EPYC 7453 28-Core Processor |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 106200 (29 hours) |
| Inner Apptainer? | True |
| Job state | jobscript_error |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk |
| Started | 2024-09-26 09:53:03 |
| Input files | hd-protodune:np04hd_raw_run029081_0863_dataflow6_datawriter_0_20240905T062327.hdf5
|
| Jobscript | Exit code | 1 |
| Real time | 0m (0s) |
| CPU time | 0m (0s = 0%) |
| Outputting started | |
| Output files | |
| Finished | 2024-09-26 11:09:46 |
| Saved logs | justin-logs:263270.148-justin-prod-sched01.dune.hep.ac.uk.logs.tgz |
| List job events Wrapper job log |
Jobscript log (last 10,000 characters)
------
------------------------------------------------------------------------------------------------
Begin processing the 29th record. run: 29081 subRun: 1 event: 207350 at 26-Sep-2024 12:56:56 CEST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 69193 traces, 69193 summary
Retagger: tagging trace set: gauss with 51207 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 51207 traces tagged "gauss"
FrameSaver: q=1.1838e+06 n=959418 tag=gauss
wclsFrameSaver: saving 69193 traces tagged "wiener"
FrameSaver: q=1.26026e+06 n=866283 tag=wiener
1991 hits in TPC 0
2068 hits in TPC 1
7861 hits in TPC 2
4671 hits in TPC 3
3392 hits in TPC 4
6258 hits in TPC 5
4906 hits in TPC 6
3595 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 292 XUs and 350 XVs -> 4 XUVs
C:0 T:2 32759 XUs and 26164 XVs -> 993 XUVs
C:0 T:3 293 XUs and 434 XVs -> 7 XUVs
C:0 T:4 128 XUs and 206 XVs -> 11 XUVs
C:0 T:5 34774 XUs and 30197 XVs -> 663 XUVs
C:0 T:6 11760 XUs and 12362 XVs -> 364 XUVs
C:0 T:7 288 XUs and 543 XVs -> 69 XUVs
2111 XUVs total
1221 collection wire objects
2111 potential space points
Neighbour search...
33909 tests to find 20790 neighbours
Iterating with no regularization...
Begin: 8.14586e+06
0 7.72562e+06
1 7.70824e+06
2 7.70617e+06
Now with regularization...
Begin: 7.1756e+06
0 7.17465e+06
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
no of hits closeby 0 event 207350 TrkackID 6 startx, y, z 210.827 404.532 460.541 wireno, peakt tpcno 268 4705.58 5 dist 0min T, max_T3009.26 5675.23
no of hits closeby 0 event 207350 TrkackID 13 startx, y, z -198.237 174.112 454.231 wireno, peakt tpcno 297 2384.44 6 dist 0min T, max_T9.04773 2560
no of hits closeby 0 event 207350 TrkackID 6 startx, y, z 210.827 404.532 460.541 wireno, peakt tpcno 268 4705.58 5 dist 0min T, max_T3009.26 5675.23
no of hits closeby 0 event 207350 TrkackID 13 startx, y, z -198.237 174.112 454.231 wireno, peakt tpcno 297 2384.44 6 dist 0min T, max_T9.04773 2560
Begin processing the 30th record. run: 29081 subRun: 1 event: 207406 at 26-Sep-2024 12:58:15 CEST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 43035 traces, 43035 summary
Retagger: tagging trace set: gauss with 32055 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 32055 traces tagged "gauss"
FrameSaver: q=612334 n=561330 tag=gauss
wclsFrameSaver: saving 43035 traces tagged "wiener"
FrameSaver: q=669065 n=506524 tag=wiener
4325 hits in TPC 0
2796 hits in TPC 1
4027 hits in TPC 2
1824 hits in TPC 3
2935 hits in TPC 4
5072 hits in TPC 5
627 hits in TPC 6
177 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 1008 XUs and 589 XVs -> 45 XUVs
C:0 T:2 14453 XUs and 13716 XVs -> 493 XUVs
C:0 T:3 42 XUs and 29 XVs -> 0 XUVs
C:0 T:4 189 XUs and 329 XVs -> 9 XUVs
C:0 T:5 13787 XUs and 15808 XVs -> 264 XUVs
C:0 T:6 242 XUs and 393 XVs -> 54 XUVs
C:0 T:7 26 XUs and 58 XVs -> 2 XUVs
867 XUVs total
478 collection wire objects
867 potential space points
Neighbour search...
12789 tests to find 7940 neighbours
Iterating with no regularization...
Begin: 2.94685e+06
0 2.82288e+06
1 2.81417e+06
2 2.81201e+06
Now with regularization...
Begin: 2.67957e+06
0 2.67845e+06
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
no of hits closeby 0 event 207406 TrkackID 5 startx, y, z 32.5006 594.993 114.716 wireno, peakt tpcno 263 1007.35 2 dist 285.09min T, max_T4.85969 1080.33
no of hits closeby 0 event 207406 TrkackID 7 startx, y, z 184.062 604.701 77.6402 wireno, peakt tpcno 314 3491.51 2 dist 35.3109min T, max_T1182.01 3531.6
no of hits closeby 0 event 207406 TrkackID 5 startx, y, z 32.5006 594.993 114.716 wireno, peakt tpcno 263 1007.35 2 dist 285.09min T, max_T4.85969 1080.33
no of hits closeby 0 event 207406 TrkackID 7 startx, y, z 184.062 604.701 77.6402 wireno, peakt tpcno 314 3491.51 2 dist 35.3109min T, max_T1182.01 3531.6
26-Sep-2024 12:59:12 CEST Closed output file "np04hd_raw_run029081_0863_dataflow6_datawriter_0_20240905T062327_reco_stage1_reco_stage2_20240926T105912_keepup.root"
26-Sep-2024 12:59:13 CEST Closed input file "np04hd_raw_run029081_0863_dataflow6_datawriter_0_20240905T062327_reco_stage1.root"
========================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
========================================================================================================================================
Full event 57.0272 95.9482 142.04 91.1657 19.1032 30
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.000524212 0.00296995 0.0052669 0.00295169 0.00121585 30
produce:fembfilter:PDHDFEMBFilter 0.504171 1.05479 8.41366 0.731074 1.41428 30
produce:wclsdatahd:WireCellToolkit 31.546 40.266 46.8563 40.4076 3.55635 30
produce:gaushit:GausHitFinder 0.286796 0.665468 0.967444 0.652134 0.176278 30
produce:nhitsfilter:NumberOfHitsFilter 0.000195807 0.000320734 0.000662702 0.000308018 8.34892e-05 30
produce:reco3d:SpacePointSolver 0.119878 1.1272 2.99991 0.802135 0.83199 30
produce:hitpdune:DisambigFromSpacePoints 0.1126 0.481874 0.980969 0.463852 0.24465 30
produce:pandora:StandardPandora 3.66026 11.64 24.0931 10.6348 5.10785 30
produce:pandoraWriter:StandardPandora 0.0744997 0.149689 0.260227 0.147811 0.046241 30
produce:pandoraTrack:LArPandoraTrackCreation 0.512907 1.83076 4.45146 1.63702 0.818438 30
produce:pandoraShower:LArPandoraShowerCreation 0.366306 2.11936 6.02812 1.76334 1.16911 30
produce:pandoracalo:Calorimetry 0.243908 0.894333 2.25061 0.841297 0.391057 30
produce:pandoracalonosce:Calorimetry 0.217493 0.851286 2.20122 0.824315 0.369696 30
produce:pandorapid:Chi2ParticleID 0.00109664 0.00220574 0.00531801 0.00206563 0.000760139 30
produce:pandoraShowercalo:ShowerCalorimetry 0.336784 1.93331 4.80115 1.57262 1.11317 30
produce:pandoraShowercalonosce:ShowerCalorimetry 0.298751 1.777 4.17959 1.52059 1.01187 30
produce:pandoraGnocchiCalo:GnocchiCalorimetry 0.0525536 0.138843 0.283104 0.140646 0.0507212 30
produce:pandoraGnocchiCalonosce:GnocchiCalorimetry 0.0205188 0.0492769 0.0811799 0.0466343 0.0163994 30
produce:emtrkmichelid:EmTrackMichelId 15.8441 29.2507 44.6468 28.3055 7.19572 30
[art]:TriggerResults:TriggerResultInserter 4.3091e-05 6.5288e-05 9.1742e-05 6.4596e-05 1.08593e-05 30
end_path:out1:RootOutput 1.2112e-05 1.98712e-05 7.1384e-05 1.77635e-05 1.08787e-05 30
end_path:michelremoving:michelremoving 0.0232527 0.100282 0.368206 0.0590049 0.0812414 30
end_path:michelremovinggnocchi:michelremoving 0.0225133 0.0971227 0.320584 0.0504035 0.0785205 30
end_path:calibanagnocchi:CalibAnaTree 0.231704 0.602271 1.44982 0.538004 0.283565 30
end_path:out1:RootOutput(write) 0.353361 0.693457 1.00037 0.716224 0.178653 30
========================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 4926.47 MB
Peak resident set size usage (VmHWM): 2347.01 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np04hd_raw_run029081_0863_dataflow6_datawriter_0_20240905T062327_reco_stage1_reco_stage2_20240926T105912_keepup.root
\tHists: np04hd_raw_run029081_0863_dataflow6_datawriter_0_20240905T062327_reco_stage1_reco_stage2_20240926T105912_keepup_hists.root
Forming reco metadata
/cvmfs/larsoft.opensciencegrid.org/products/python/v3_9_15/Linux64bit+3.10-2.17/bin/python: No module named meta_maker
Error in reco metadata
