Jobsub ID 78455.44@justin-prod-sched02.dune.hep.ac.uk
| Jobsub ID | 78455.44@justin-prod-sched02.dune.hep.ac.uk |
| Workflow ID | 3451 |
| Stage ID | 1 |
| User name | calcuttj@fnal.gov |
| HTCondor Group | group_dune.prod_mcsim |
| Requested | Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 18000 (5 hours) |
| Submitted time | 2024-09-26 09:57:01 |
| Site | FR_CCIN2P3 |
| Entry | DUNE_FR_CCIN2P3_cccondorce02 |
| Last heartbeat | 2024-09-26 11:14:18 |
| From worker node | Hostname | ccwcondor0635 |
| cpuinfo | AMD EPYC 7453 28-Core Processor |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 106200 (29 hours) |
| Inner Apptainer? | True |
| Job state | jobscript_error |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk |
| Started | 2024-09-26 10:08:37 |
| Input files | hd-protodune:np04hd_raw_run029107_2259_dataflow0_datawriter_0_20240908T042901.hdf5
|
| Jobscript | Exit code | 1 |
| Real time | 0m (0s) |
| CPU time | 0m (0s = 0%) |
| Outputting started | |
| Output files | |
| Finished | 2024-09-26 11:14:18 |
| Saved logs | justin-logs:78455.44-justin-prod-sched02.dune.hep.ac.uk.logs.tgz |
| List job events Wrapper job log |
Jobscript log (last 10,000 characters)
0 XVs -> 10 XUVs
C:0 T:5 50157 XUs and 61109 XVs -> 3428 XUVs
C:0 T:6 28113 XUs and 12934 XVs -> 227 XUVs
C:0 T:7 122 XUs and 213 XVs -> 3 XUVs
4204 XUVs total
1118 collection wire objects
4204 potential space points
Neighbour search...
217218 tests to find 123512 neighbours
Iterating with no regularization...
Begin: 3.80769e+07
0 3.61137e+07
1 3.59097e+07
2 3.58769e+07
Now with regularization...
Begin: 3.50995e+07
0 3.5088e+07
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Begin processing the 29th record. run: 29107 subRun: 1 event: 542416 at 26-Sep-2024 13:10:20 CEST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 58565 traces, 58565 summary
Retagger: tagging trace set: gauss with 45733 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 45733 traces tagged "gauss"
FrameSaver: q=932948 n=849017 tag=gauss
wclsFrameSaver: saving 58565 traces tagged "wiener"
FrameSaver: q=989868 n=770189 tag=wiener
3913 hits in TPC 0
3068 hits in TPC 1
2784 hits in TPC 2
1645 hits in TPC 3
4690 hits in TPC 4
6597 hits in TPC 5
6005 hits in TPC 6
2531 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 7617 XUs and 5345 XVs -> 169 XUVs
C:0 T:2 4992 XUs and 3361 XVs -> 167 XUVs
C:0 T:3 11 XUs and 7 XVs -> 0 XUVs
C:0 T:4 87 XUs and 102 XVs -> 1 XUVs
C:0 T:5 24907 XUs and 20986 XVs -> 582 XUVs
C:0 T:6 15965 XUs and 14500 XVs -> 305 XUVs
C:0 T:7 38 XUs and 25 XVs -> 4 XUVs
1228 XUVs total
944 collection wire objects
1228 potential space points
Neighbour search...
11878 tests to find 6858 neighbours
Iterating with no regularization...
Begin: 2.87735e+06
0 2.78578e+06
1 2.78273e+06
2 2.78257e+06
Now with regularization...
Begin: 2.55072e+06
0 2.55062e+06
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
no of hits closeby 0 event 542416 TrkackID 0 startx, y, z 84.0339 592.579 291.599 wireno, peakt tpcno 103 44 5 dist 2.86226min T, max_T12.9644 3880.66
no of hits closeby 1 event 542416 TrkackID 1 startx, y, z 15.2883 589.294 206.698 wireno, peakt tpcno 464 2622.07 5 dist 0min T, max_T2581.47 4707.11
no of hits closeby 0 event 542416 TrkackID 0 startx, y, z 84.0339 592.579 291.599 wireno, peakt tpcno 103 44 5 dist 2.86226min T, max_T12.9644 3880.66
no of hits closeby 1 event 542416 TrkackID 1 startx, y, z 15.2883 589.294 206.698 wireno, peakt tpcno 464 2622.07 5 dist 0min T, max_T2581.47 4707.11
Begin processing the 30th record. run: 29107 subRun: 1 event: 542424 at 26-Sep-2024 13:11:34 CEST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 54478 traces, 54478 summary
Retagger: tagging trace set: gauss with 41540 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 41540 traces tagged "gauss"
FrameSaver: q=911596 n=798196 tag=gauss
wclsFrameSaver: saving 54478 traces tagged "wiener"
FrameSaver: q=982043 n=729318 tag=wiener
3390 hits in TPC 0
3064 hits in TPC 1
5685 hits in TPC 2
3772 hits in TPC 3
2250 hits in TPC 4
3742 hits in TPC 5
4176 hits in TPC 6
3740 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 393 XUs and 236 XVs -> 13 XUVs
C:0 T:2 20018 XUs and 22836 XVs -> 1596 XUVs
C:0 T:3 336 XUs and 496 XVs -> 47 XUVs
C:0 T:4 215 XUs and 270 XVs -> 6 XUVs
C:0 T:5 6685 XUs and 8725 XVs -> 178 XUVs
C:0 T:6 8977 XUs and 8766 XVs -> 336 XUVs
C:0 T:7 2210 XUs and 3500 XVs -> 212 XUVs
2388 XUVs total
797 collection wire objects
2388 potential space points
Neighbour search...
111122 tests to find 58760 neighbours
Iterating with no regularization...
Begin: 7.96872e+06
0 7.13503e+06
1 7.06739e+06
2 7.05485e+06
3 7.0515e+06
Now with regularization...
Begin: 6.52714e+06
0 6.52489e+06
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
26-Sep-2024 13:12:47 CEST Closed output file "np04hd_raw_run029107_2259_dataflow0_datawriter_0_20240908T042901_reco_stage1_reco_stage2_20240926T111247_keepup.root"
26-Sep-2024 13:12:47 CEST Closed input file "np04hd_raw_run029107_2259_dataflow0_datawriter_0_20240908T042901_reco_stage1.root"
========================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
========================================================================================================================================
Full event 56.0967 87.5409 137.461 88.0923 20.9252 30
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.000585983 0.00312221 0.00603641 0.00335762 0.00134789 30
produce:fembfilter:PDHDFEMBFilter 0.492166 0.676994 0.962928 0.668515 0.119577 30
produce:wclsdatahd:WireCellToolkit 29.5366 37.6421 46.3324 36.7548 4.83762 30
produce:gaushit:GausHitFinder 0.29337 0.660717 1.09436 0.630703 0.22865 30
produce:nhitsfilter:NumberOfHitsFilter 0.000211916 0.000347284 0.000757705 0.000330407 0.000101697 30
produce:reco3d:SpacePointSolver 0.105132 1.04393 2.57047 0.943593 0.655185 30
produce:hitpdune:DisambigFromSpacePoints 0.113306 0.511177 1.08677 0.464473 0.270847 30
produce:pandora:StandardPandora 4.50659 11.2223 26.0354 10.2437 5.36808 30
produce:pandoraWriter:StandardPandora 0.0842821 0.159148 0.245765 0.157249 0.0483079 30
produce:pandoraTrack:LArPandoraTrackCreation 0.584572 1.86705 4.05045 1.80747 0.712797 30
produce:pandoraShower:LArPandoraShowerCreation 0.607079 2.35576 5.50491 1.99307 1.10161 30
produce:pandoracalo:Calorimetry 0.329937 0.865273 1.78376 0.806724 0.363645 30
produce:pandoracalonosce:Calorimetry 0.267891 0.805474 1.62268 0.811642 0.33351 30
produce:pandorapid:Chi2ParticleID 0.00122193 0.00211319 0.00522892 0.00196906 0.000827623 30
produce:pandoraShowercalo:ShowerCalorimetry 0.467771 2.00132 4.37818 2.00231 0.940629 30
produce:pandoraShowercalonosce:ShowerCalorimetry 0.476105 1.95175 4.2024 1.79439 0.903852 30
produce:pandoraGnocchiCalo:GnocchiCalorimetry 0.0716869 0.140957 0.2604 0.129447 0.047517 30
produce:pandoraGnocchiCalonosce:GnocchiCalorimetry 0.0244599 0.05022 0.0893402 0.048198 0.0160937 30
produce:emtrkmichelid:EmTrackMichelId 13.3672 23.8503 47.0823 22.5889 6.9646 30
[art]:TriggerResults:TriggerResultInserter 2.3945e-05 5.88573e-05 0.000149098 5.6405e-05 2.30783e-05 30
end_path:out1:RootOutput 7.254e-06 1.57581e-05 4.9993e-05 1.4547e-05 7.44811e-06 30
end_path:michelremoving:michelremoving 0.0181751 0.0866032 0.301468 0.051545 0.0773984 30
end_path:michelremovinggnocchi:michelremoving 0.0171627 0.0832006 0.319832 0.0511554 0.0732672 30
end_path:calibanagnocchi:CalibAnaTree 0.186786 0.620409 1.73863 0.569434 0.295933 30
end_path:out1:RootOutput(write) 0.378841 0.723096 1.22822 0.714896 0.198718 30
========================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 4922.85 MB
Peak resident set size usage (VmHWM): 2350.51 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np04hd_raw_run029107_2259_dataflow0_datawriter_0_20240908T042901_reco_stage1_reco_stage2_20240926T111247_keepup.root
\tHists: np04hd_raw_run029107_2259_dataflow0_datawriter_0_20240908T042901_reco_stage1_reco_stage2_20240926T111247_keepup_hists.root
Forming reco metadata
/cvmfs/larsoft.opensciencegrid.org/products/python/v3_9_15/Linux64bit+3.10-2.17/bin/python: No module named meta_maker
Error in reco metadata
