Jobsub ID 263266.22@justin-prod-sched01.dune.hep.ac.uk
| Jobsub ID | 263266.22@justin-prod-sched01.dune.hep.ac.uk |
| Workflow ID | 3447 |
| Stage ID | 1 |
| User name | calcuttj@fnal.gov |
| HTCondor Group | group_dune.prod_mcsim |
| Requested | Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 18000 (5 hours) |
| Submitted time | 2024-09-26 09:49:48 |
| Site | FR_CCIN2P3 |
| Entry | DUNE_FR_CCIN2P3_cccondorce03 |
| Last heartbeat | 2024-09-26 11:06:55 |
| From worker node | Hostname | ccwcondor0788 |
| cpuinfo | AMD EPYC 7453 28-Core Processor |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 106200 (29 hours) |
| Inner Apptainer? | True |
| Job state | jobscript_error |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk |
| Started | 2024-09-26 09:50:28 |
| Input files | hd-protodune:np04hd_raw_run029092_0450_dataflow4_datawriter_0_20240906T002937.hdf5
|
| Jobscript | Exit code | 1 |
| Real time | 0m (0s) |
| CPU time | 0m (0s = 0%) |
| Outputting started | |
| Output files | |
| Finished | 2024-09-26 11:06:55 |
| Saved logs | justin-logs:263266.22-justin-prod-sched01.dune.hep.ac.uk.logs.tgz |
| List job events Wrapper job log |
Jobscript log (last 10,000 characters)
3770 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 291 XUs and 235 XVs -> 7 XUVs
C:0 T:2 19716 XUs and 24967 XVs -> 1067 XUVs
C:0 T:3 148 XUs and 187 XVs -> 4 XUVs
C:0 T:4 5 XUs and 4 XVs -> 0 XUVs
C:0 T:5 4199 XUs and 6679 XVs -> 128 XUVs
C:0 T:6 22339 XUs and 24994 XVs -> 980 XUVs
C:0 T:7 206 XUs and 225 XVs -> 8 XUVs
2194 XUVs total
1325 collection wire objects
2194 potential space points
Neighbour search...
59286 tests to find 32100 neighbours
Iterating with no regularization...
Begin: 5.92996e+06
0 5.74907e+06
1 5.73835e+06
2 5.73681e+06
Now with regularization...
Begin: 5.28432e+06
0 5.28378e+06
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
no of hits closeby 5 event 108220 TrkackID 12 startx, y, z 90.6446 172.235 228.1 wireno, peakt tpcno 340 1545.15 2 dist 86.6691min T, max_T530 1803.54
no of hits closeby 5 event 108220 TrkackID 12 startx, y, z 90.6446 172.235 228.1 wireno, peakt tpcno 340 1545.15 2 dist 86.6691min T, max_T530 1803.54
Begin processing the 29th record. run: 29092 subRun: 1 event: 108228 at 26-Sep-2024 12:54:32 CEST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 59258 traces, 59258 summary
Retagger: tagging trace set: gauss with 45210 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 45210 traces tagged "gauss"
FrameSaver: q=924022 n=848521 tag=gauss
wclsFrameSaver: saving 59258 traces tagged "wiener"
FrameSaver: q=993510 n=773465 tag=wiener
5500 hits in TPC 0
4456 hits in TPC 1
5075 hits in TPC 2
2545 hits in TPC 3
5418 hits in TPC 4
4701 hits in TPC 5
2348 hits in TPC 6
1471 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 5579 XUs and 3588 XVs -> 120 XUVs
C:0 T:2 10128 XUs and 11480 XVs -> 261 XUVs
C:0 T:3 429 XUs and 486 XVs -> 23 XUVs
C:0 T:4 29 XUs and 45 XVs -> 3 XUVs
C:0 T:5 24498 XUs and 26696 XVs -> 1196 XUVs
C:0 T:6 2782 XUs and 3969 XVs -> 66 XUVs
C:0 T:7 79 XUs and 141 XVs -> 3 XUVs
1672 XUVs total
987 collection wire objects
1672 potential space points
Neighbour search...
48020 tests to find 18440 neighbours
Iterating with no regularization...
Begin: 5.89765e+06
0 5.10746e+06
1 5.0639e+06
2 5.06145e+06
Now with regularization...
Begin: 4.47858e+06
0 4.47673e+06
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Begin processing the 30th record. run: 29092 subRun: 1 event: 108236 at 26-Sep-2024 12:55:53 CEST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 57554 traces, 57554 summary
Retagger: tagging trace set: gauss with 43885 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 43885 traces tagged "gauss"
FrameSaver: q=882076 n=806570 tag=gauss
wclsFrameSaver: saving 57554 traces tagged "wiener"
FrameSaver: q=937989 n=721126 tag=wiener
3186 hits in TPC 0
3254 hits in TPC 1
4135 hits in TPC 2
1505 hits in TPC 3
3730 hits in TPC 4
4761 hits in TPC 5
7065 hits in TPC 6
2338 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:0 18 XUs and 11 XVs -> 0 XUVs
C:0 T:1 193 XUs and 101 XVs -> 2 XUVs
C:0 T:2 23733 XUs and 52403 XVs -> 1340 XUVs
C:0 T:3 21 XUs and 30 XVs -> 3 XUVs
C:0 T:4 30 XUs and 29 XVs -> 1 XUVs
C:0 T:5 26813 XUs and 31941 XVs -> 998 XUVs
C:0 T:6 32792 XUs and 39503 XVs -> 1552 XUVs
C:0 T:7 98 XUs and 92 XVs -> 2 XUVs
3898 XUVs total
1189 collection wire objects
3898 potential space points
Neighbour search...
409254 tests to find 194306 neighbours
Iterating with no regularization...
Begin: 6.76348e+06
0 5.6195e+06
1 5.56185e+06
2 5.55662e+06
Now with regularization...
Begin: 5.39876e+06
0 5.3969e+06
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
26-Sep-2024 12:57:16 CEST Closed output file "np04hd_raw_run029092_0450_dataflow4_datawriter_0_20240906T002937_reco_stage1_reco_stage2_20240926T105716_keepup.root"
26-Sep-2024 12:57:16 CEST Closed input file "np04hd_raw_run029092_0450_dataflow4_datawriter_0_20240906T002937_reco_stage1.root"
========================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
========================================================================================================================================
Full event 69.8799 103.023 149.018 101.838 20.53 30
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.000562722 0.00342104 0.00871483 0.00321403 0.0016096 30
produce:fembfilter:PDHDFEMBFilter 0.503113 0.705716 0.928545 0.7446 0.0992962 30
produce:wclsdatahd:WireCellToolkit 35.5902 42.5572 48.2954 43.0989 3.85221 30
produce:gaushit:GausHitFinder 0.499764 0.753782 1.16884 0.766805 0.191366 30
produce:nhitsfilter:NumberOfHitsFilter 0.00020829 0.000362874 0.000565988 0.00035307 8.202e-05 30
produce:reco3d:SpacePointSolver 0.228718 1.25216 3.73116 0.911866 0.904096 30
produce:hitpdune:DisambigFromSpacePoints 0.181432 0.631322 1.58879 0.564356 0.316546 30
produce:pandora:StandardPandora 4.71142 13.131 25.2452 13.1266 5.50847 30
produce:pandoraWriter:StandardPandora 0.0804675 0.156383 0.275852 0.156628 0.0454404 30
produce:pandoraTrack:LArPandoraTrackCreation 0.655588 1.75352 3.39924 1.64712 0.701039 30
produce:pandoraShower:LArPandoraShowerCreation 0.598786 2.38737 4.56816 2.23826 1.24529 30
produce:pandoracalo:Calorimetry 0.342297 0.911228 1.56277 0.914791 0.325827 30
produce:pandoracalonosce:Calorimetry 0.288773 0.795473 1.47947 0.813524 0.306268 30
produce:pandorapid:Chi2ParticleID 0.00135064 0.00222983 0.00420888 0.00224352 0.000636321 30
produce:pandoraShowercalo:ShowerCalorimetry 0.455157 1.98438 4.18726 1.85714 0.997264 30
produce:pandoraShowercalonosce:ShowerCalorimetry 0.488828 1.95651 5.18627 1.79106 1.0656 30
produce:pandoraGnocchiCalo:GnocchiCalorimetry 0.0829018 0.169482 0.254804 0.171189 0.0478296 30
produce:pandoraGnocchiCalonosce:GnocchiCalorimetry 0.0289121 0.0590574 0.0885903 0.0604915 0.0177558 30
produce:emtrkmichelid:EmTrackMichelId 20.5907 32.0014 48.0988 32.133 7.29141 30
[art]:TriggerResults:TriggerResultInserter 2.8073e-05 5.90912e-05 9.9165e-05 5.78385e-05 1.60738e-05 30
end_path:out1:RootOutput 1.0589e-05 1.92439e-05 5.2929e-05 1.84495e-05 7.73002e-06 30
end_path:michelremoving:michelremoving 0.0243551 0.0888133 0.299119 0.055067 0.0713099 30
end_path:michelremovinggnocchi:michelremoving 0.0184944 0.0817622 0.235237 0.0491233 0.0655958 30
end_path:calibanagnocchi:CalibAnaTree 0.24349 0.634634 1.93066 0.581299 0.33686 30
end_path:out1:RootOutput(write) 0.52365 0.801978 1.20901 0.814003 0.201259 30
========================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 4904.07 MB
Peak resident set size usage (VmHWM): 2327.15 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np04hd_raw_run029092_0450_dataflow4_datawriter_0_20240906T002937_reco_stage1_reco_stage2_20240926T105716_keepup.root
\tHists: np04hd_raw_run029092_0450_dataflow4_datawriter_0_20240906T002937_reco_stage1_reco_stage2_20240926T105716_keepup_hists.root
Forming reco metadata
/cvmfs/larsoft.opensciencegrid.org/products/python/v3_9_15/Linux64bit+3.10-2.17/bin/python: No module named meta_maker
Error in reco metadata
