Jobsub ID 78434.9@justin-prod-sched02.dune.hep.ac.uk
| Jobsub ID | 78434.9@justin-prod-sched02.dune.hep.ac.uk |
| Workflow ID | 3451 |
| Stage ID | 1 |
| User name | calcuttj@fnal.gov |
| HTCondor Group | group_dune.prod_mcsim |
| Requested | Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 18000 (5 hours) |
| Submitted time | 2024-09-26 09:34:34 |
| Site | FR_CCIN2P3 |
| Entry | DUNE_FR_CCIN2P3_cccondorce02 |
| Last heartbeat | 2024-09-26 11:16:03 |
| From worker node | Hostname | ccwcondor0591 |
| cpuinfo | AMD EPYC 7453 28-Core Processor |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 106200 (29 hours) |
| Inner Apptainer? | True |
| Job state | jobscript_error |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk |
| Started | 2024-09-26 09:52:28 |
| Input files | hd-protodune:np04hd_raw_run029107_1905_dataflow5_datawriter_0_20240907T225851.hdf5
|
| Jobscript | Exit code | 1 |
| Real time | 0m (0s) |
| CPU time | 0m (0s = 0%) |
| Outputting started | |
| Output files | |
| Finished | 2024-09-26 11:16:03 |
| Saved logs | justin-logs:78434.9-justin-prod-sched02.dune.hep.ac.uk.logs.tgz |
| List job events Wrapper job log |
Jobscript log (last 10,000 characters)
s
C:0 T:3 468 XUs and 396 XVs -> 3 XUVs
C:0 T:4 291 XUs and 385 XVs -> 17 XUVs
C:0 T:5 21542 XUs and 31933 XVs -> 1908 XUVs
C:0 T:6 22868 XUs and 26561 XVs -> 1401 XUVs
C:0 T:7 259 XUs and 754 XVs -> 21 XUVs
4096 XUVs total
1678 collection wire objects
4096 potential space points
Neighbour search...
123410 tests to find 77152 neighbours
Iterating with no regularization...
Begin: 8.71773e+06
0 8.17933e+06
1 8.1242e+06
2 8.11479e+06
3 8.11215e+06
Now with regularization...
Begin: 7.47448e+06
0 7.4728e+06
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Begin processing the 29th record. run: 29107 subRun: 1 event: 457453 at 26-Sep-2024 13:00:39 CEST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 67135 traces, 67135 summary
Retagger: tagging trace set: gauss with 48983 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 48983 traces tagged "gauss"
FrameSaver: q=988698 n=903744 tag=gauss
wclsFrameSaver: saving 67135 traces tagged "wiener"
FrameSaver: q=1.07741e+06 n=817577 tag=wiener
2908 hits in TPC 0
2142 hits in TPC 1
9827 hits in TPC 2
4017 hits in TPC 3
1635 hits in TPC 4
3632 hits in TPC 5
7864 hits in TPC 6
3955 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 54 XUs and 48 XVs -> 1 XUVs
C:0 T:2 94932 XUs and 96677 XVs -> 2987 XUVs
C:0 T:3 486 XUs and 433 XVs -> 9 XUVs
C:0 T:4 138 XUs and 199 XVs -> 7 XUVs
C:0 T:5 4043 XUs and 4618 XVs -> 159 XUVs
C:0 T:6 52944 XUs and 53778 XVs -> 1016 XUVs
C:0 T:7 619 XUs and 600 XVs -> 17 XUVs
4196 XUVs total
1421 collection wire objects
4196 potential space points
Neighbour search...
270888 tests to find 144070 neighbours
Iterating with no regularization...
Begin: 6.25193e+06
0 5.54881e+06
1 5.48833e+06
2 5.47781e+06
3 5.47409e+06
Now with regularization...
Begin: 5.22515e+06
0 5.22311e+06
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Begin processing the 30th record. run: 29107 subRun: 1 event: 457461 at 26-Sep-2024 13:02:12 CEST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 60032 traces, 60032 summary
Retagger: tagging trace set: gauss with 43862 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 43862 traces tagged "gauss"
FrameSaver: q=1.01725e+06 n=857684 tag=gauss
wclsFrameSaver: saving 60032 traces tagged "wiener"
FrameSaver: q=1.09823e+06 n=785945 tag=wiener
3395 hits in TPC 0
2484 hits in TPC 1
5365 hits in TPC 2
5281 hits in TPC 3
3575 hits in TPC 4
4923 hits in TPC 5
3563 hits in TPC 6
3271 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:2 20671 XUs and 20302 XVs -> 1238 XUVs
C:0 T:3 107 XUs and 124 XVs -> 6 XUVs
C:0 T:4 378 XUs and 405 XVs -> 22 XUVs
C:0 T:5 26907 XUs and 24928 XVs -> 3154 XUVs
C:0 T:6 15411 XUs and 11749 XVs -> 164 XUVs
C:0 T:7 41 XUs and 66 XVs -> 3 XUVs
4587 XUVs total
1000 collection wire objects
4587 potential space points
Neighbour search...
419899 tests to find 228356 neighbours
Iterating with no regularization...
Begin: 7.668e+06
0 6.85897e+06
1 6.76858e+06
2 6.74851e+06
3 6.74149e+06
4 6.73779e+06
Now with regularization...
Begin: 6.16862e+06
0 6.16507e+06
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
no of hits closeby 0 event 457461 TrkackID 1 startx, y, z 288.818 335.17 132.945 wireno, peakt tpcno 291 3605.5 2 dist 342.122min T, max_T5 3654.69
no of hits closeby 1 event 457461 TrkackID 3 startx, y, z -259.759 542.445 192.522 wireno, peakt tpcno 10 1446.82 2 dist 0min T, max_T22.231 3355.84
no of hits closeby 0 event 457461 TrkackID 1 startx, y, z 288.818 335.17 132.945 wireno, peakt tpcno 291 3605.5 2 dist 342.122min T, max_T5 3654.69
no of hits closeby 1 event 457461 TrkackID 3 startx, y, z -259.759 542.445 192.522 wireno, peakt tpcno 10 1446.82 2 dist 0min T, max_T22.231 3355.84
26-Sep-2024 13:03:47 CEST Closed output file "np04hd_raw_run029107_1905_dataflow5_datawriter_0_20240907T225851_reco_stage1_reco_stage2_20240926T110347_keepup.root"
26-Sep-2024 13:03:47 CEST Closed input file "np04hd_raw_run029107_1905_dataflow5_datawriter_0_20240907T225851_reco_stage1.root"
========================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
========================================================================================================================================
Full event 72.6083 105.003 153.772 98.8791 21.4828 30
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.000951721 0.00319142 0.00604996 0.00329063 0.00129449 30
produce:fembfilter:PDHDFEMBFilter 0.510388 0.652243 0.773318 0.649739 0.0799476 30
produce:wclsdatahd:WireCellToolkit 35.3473 40.4104 48.3237 40.6843 3.43166 30
produce:gaushit:GausHitFinder 0.394873 0.748993 1.05148 0.789348 0.190309 30
produce:nhitsfilter:NumberOfHitsFilter 0.000246672 0.000371841 0.000650397 0.000354805 8.49325e-05 30
produce:reco3d:SpacePointSolver 0.365386 1.4278 3.84659 0.943916 1.09413 30
produce:hitpdune:DisambigFromSpacePoints 0.227405 0.632274 1.36436 0.52603 0.305904 30
produce:pandora:StandardPandora 6.09468 13.945 28.247 12.9071 5.93704 30
produce:pandoraWriter:StandardPandora 0.0961487 0.173242 0.279782 0.169979 0.0499194 30
produce:pandoraTrack:LArPandoraTrackCreation 1.0005 2.15357 4.68627 1.98844 0.958703 30
produce:pandoraShower:LArPandoraShowerCreation 0.683781 2.99873 6.71279 2.57113 1.69784 30
produce:pandoracalo:Calorimetry 0.404668 1.04607 2.09594 0.984051 0.465298 30
produce:pandoracalonosce:Calorimetry 0.372433 0.961393 1.94504 0.857342 0.449249 30
produce:pandorapid:Chi2ParticleID 0.00100647 0.00245462 0.00709548 0.00235187 0.00113733 30
produce:pandoraShowercalo:ShowerCalorimetry 0.680385 2.5206 7.41684 2.20574 1.51079 30
produce:pandoraShowercalonosce:ShowerCalorimetry 0.729002 2.39614 6.10054 1.92816 1.36726 30
produce:pandoraGnocchiCalo:GnocchiCalorimetry 0.0754518 0.167525 0.298841 0.151707 0.0509395 30
produce:pandoraGnocchiCalonosce:GnocchiCalorimetry 0.0279096 0.0595686 0.0941184 0.0571268 0.0158482 30
produce:emtrkmichelid:EmTrackMichelId 21.5026 32.8401 49.8936 31.2952 7.6453 30
[art]:TriggerResults:TriggerResultInserter 5.4051e-05 7.08298e-05 0.000102622 6.943e-05 1.07015e-05 30
end_path:out1:RootOutput 1.1011e-05 2.02324e-05 5.0704e-05 1.94365e-05 6.36142e-06 30
end_path:michelremoving:michelremoving 0.0296592 0.0894619 0.308796 0.0563688 0.0819067 30
end_path:michelremovinggnocchi:michelremoving 0.0212405 0.0826193 0.334199 0.0447225 0.0849684 30
end_path:calibanagnocchi:CalibAnaTree 0.232833 0.684356 1.3251 0.603198 0.318944 30
end_path:out1:RootOutput(write) 0.47153 0.764928 1.09486 0.750527 0.180715 30
========================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 4947.86 MB
Peak resident set size usage (VmHWM): 2368.52 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np04hd_raw_run029107_1905_dataflow5_datawriter_0_20240907T225851_reco_stage1_reco_stage2_20240926T110347_keepup.root
\tHists: np04hd_raw_run029107_1905_dataflow5_datawriter_0_20240907T225851_reco_stage1_reco_stage2_20240926T110347_keepup_hists.root
Forming reco metadata
/cvmfs/larsoft.opensciencegrid.org/products/python/v3_9_15/Linux64bit+3.10-2.17/bin/python: No module named meta_maker
Error in reco metadata
