Jobsub ID 78830.75@justin-prod-sched02.dune.hep.ac.uk
| Jobsub ID | 78830.75@justin-prod-sched02.dune.hep.ac.uk |
| Workflow ID | 3449 |
| Stage ID | 1 |
| User name | calcuttj@fnal.gov |
| HTCondor Group | group_dune.prod_mcsim |
| Requested | Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 18000 (5 hours) |
| Submitted time | 2024-09-26 13:07:16 |
| Site | UK_Durham |
| Entry | DUNE_UK_SGridDurham_ce4 |
| Last heartbeat | 2024-09-26 14:38:43 |
| From worker node | Hostname | n234.dur.scotgrid.ac.uk |
| cpuinfo | AMD EPYC 7352 24-Core Processor |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 171000 (47 hours) |
| Inner Apptainer? | True |
| Job state | jobscript_error |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk |
| Started | 2024-09-26 13:08:28 |
| Input files | hd-protodune:np04hd_raw_run029107_0754_dataflow6_datawriter_0_20240907T032011.hdf5
|
| Jobscript | Exit code | 1 |
| Real time | 0m (0s) |
| CPU time | 0m (0s = 0%) |
| Outputting started | |
| Output files | |
| Finished | 2024-09-26 14:38:43 |
| Saved logs | justin-logs:78830.75-justin-prod-sched02.dune.hep.ac.uk.logs.tgz |
| List job events Wrapper job log |
Jobscript log (last 10,000 characters)
s is
Retagger: tagging trace set: wiener with 46450 traces, 46450 summary
Retagger: tagging trace set: gauss with 34940 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 34940 traces tagged "gauss"
FrameSaver: q=743214 n=650696 tag=gauss
wclsFrameSaver: saving 46450 traces tagged "wiener"
FrameSaver: q=791464 n=590643 tag=wiener
2465 hits in TPC 0
2710 hits in TPC 1
3221 hits in TPC 2
2163 hits in TPC 3
2172 hits in TPC 4
3778 hits in TPC 5
4175 hits in TPC 6
3515 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 821 XUs and 539 XVs -> 3 XUVs
C:0 T:2 5588 XUs and 6811 XVs -> 394 XUVs
C:0 T:3 85 XUs and 169 XVs -> 3 XUVs
C:0 T:4 17 XUs and 15 XVs -> 1 XUVs
C:0 T:5 10357 XUs and 22024 XVs -> 754 XUVs
C:0 T:6 13920 XUs and 10446 XVs -> 516 XUVs
C:0 T:7 198 XUs and 314 XVs -> 9 XUVs
1680 XUVs total
933 collection wire objects
1680 potential space points
Neighbour search...
60342 tests to find 33830 neighbours
Iterating with no regularization...
Begin: 3.92841e+06
0 3.71342e+06
1 3.70295e+06
2 3.70124e+06
Now with regularization...
Begin: 3.40438e+06
0 3.40367e+06
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Begin processing the 29th record. run: 29107 subRun: 1 event: 181214 at 26-Sep-2024 15:35:16 BST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 55129 traces, 55129 summary
Retagger: tagging trace set: gauss with 43812 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 43812 traces tagged "gauss"
FrameSaver: q=997423 n=884656 tag=gauss
wclsFrameSaver: saving 55129 traces tagged "wiener"
FrameSaver: q=1.06093e+06 n=820530 tag=wiener
3019 hits in TPC 0
3800 hits in TPC 1
3313 hits in TPC 2
2480 hits in TPC 3
4962 hits in TPC 4
7818 hits in TPC 5
3311 hits in TPC 6
2845 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 691 XUs and 626 XVs -> 17 XUVs
C:0 T:2 6897 XUs and 9709 XVs -> 327 XUVs
C:0 T:3 118 XUs and 161 XVs -> 6 XUVs
C:0 T:4 323 XUs and 335 XVs -> 5 XUVs
C:0 T:5 30271 XUs and 26957 XVs -> 649 XUVs
C:0 T:6 5489 XUs and 5669 XVs -> 505 XUVs
C:0 T:7 155 XUs and 286 XVs -> 8 XUVs
1517 XUVs total
1043 collection wire objects
1517 potential space points
Neighbour search...
27655 tests to find 15698 neighbours
Iterating with no regularization...
Begin: 4.73341e+06
0 4.41143e+06
1 4.39467e+06
2 4.39335e+06
Now with regularization...
Begin: 4.09575e+06
0 4.09525e+06
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Begin processing the 30th record. run: 29107 subRun: 1 event: 181222 at 26-Sep-2024 15:36:36 BST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 70040 traces, 70040 summary
Retagger: tagging trace set: gauss with 51169 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 51169 traces tagged "gauss"
FrameSaver: q=1.06429e+06 n=880901 tag=gauss
wclsFrameSaver: saving 70040 traces tagged "wiener"
FrameSaver: q=1.15498e+06 n=784001 tag=wiener
3257 hits in TPC 0
4457 hits in TPC 1
5867 hits in TPC 2
2776 hits in TPC 3
2986 hits in TPC 4
5184 hits in TPC 5
5896 hits in TPC 6
3963 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 428 XUs and 359 XVs -> 2 XUVs
C:0 T:2 45558 XUs and 18109 XVs -> 512 XUVs
C:0 T:3 11 XUs and 8 XVs -> 0 XUVs
C:0 T:4 72 XUs and 34 XVs -> 1 XUVs
C:0 T:5 17654 XUs and 7047 XVs -> 92 XUVs
C:0 T:6 19002 XUs and 24755 XVs -> 1369 XUVs
C:0 T:7 449 XUs and 548 XVs -> 15 XUVs
1991 XUVs total
735 collection wire objects
1991 potential space points
Neighbour search...
128339 tests to find 70078 neighbours
Iterating with no regularization...
Begin: 6.32172e+06
0 5.82603e+06
1 5.7988e+06
2 5.79545e+06
Now with regularization...
Begin: 5.61357e+06
0 5.61226e+06
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
26-Sep-2024 15:38:02 BST Closed output file "np04hd_raw_run029107_0754_dataflow6_datawriter_0_20240907T032011_reco_stage1_reco_stage2_20240926T143802_keepup.root"
26-Sep-2024 15:38:02 BST Closed input file "np04hd_raw_run029107_0754_dataflow6_datawriter_0_20240907T032011_reco_stage1.root"
========================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
========================================================================================================================================
Full event 68.5723 88.8893 116.013 86.9201 13.2006 30
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.000487818 0.00246644 0.00413003 0.00235108 0.00106677 30
produce:fembfilter:PDHDFEMBFilter 0.518822 0.678461 0.909765 0.707011 0.0891076 30
produce:wclsdatahd:WireCellToolkit 30.2883 38.4145 40.7258 38.8845 1.99088 30
produce:gaushit:GausHitFinder 0.51449 0.758502 1.11917 0.71655 0.161301 30
produce:nhitsfilter:NumberOfHitsFilter 0.000316575 0.000429836 0.000662026 0.000427835 7.75544e-05 30
produce:reco3d:SpacePointSolver 0.32249 1.14373 3.01048 0.949518 0.739645 30
produce:hitpdune:DisambigFromSpacePoints 0.258741 0.526176 0.973195 0.498978 0.204556 30
produce:pandora:StandardPandora 7.35647 12.5875 21.8672 11.7212 3.84756 30
produce:pandoraWriter:StandardPandora 0.113389 0.181267 0.242372 0.176913 0.0370706 30
produce:pandoraTrack:LArPandoraTrackCreation 0.492511 1.19217 2.04216 1.13596 0.401151 30
produce:pandoraShower:LArPandoraShowerCreation 0.469342 1.35179 2.84829 1.30127 0.607408 30
produce:pandoracalo:Calorimetry 0.295792 0.618275 0.988395 0.611258 0.169942 30
produce:pandoracalonosce:Calorimetry 0.24405 0.546787 0.861929 0.528897 0.166288 30
produce:pandorapid:Chi2ParticleID 0.00151316 0.00415606 0.061332 0.00218943 0.0106277 30
produce:pandoraShowercalo:ShowerCalorimetry 0.429704 1.18984 2.37895 1.16167 0.523796 30
produce:pandoraShowercalonosce:ShowerCalorimetry 0.40228 1.14037 2.22265 1.09283 0.488536 30
produce:pandoraGnocchiCalo:GnocchiCalorimetry 0.0912767 0.172445 0.249596 0.17486 0.038829 30
produce:pandoraGnocchiCalonosce:GnocchiCalorimetry 0.0318169 0.0565287 0.0820187 0.0568241 0.0129559 30
produce:emtrkmichelid:EmTrackMichelId 14.604 26.8066 37.8724 26.8401 6.18481 30
[art]:TriggerResults:TriggerResultInserter 2.0158e-05 3.55937e-05 8.4889e-05 3.10085e-05 1.60119e-05 30
end_path:out1:RootOutput 6.392e-06 1.87937e-05 0.00026537 9.012e-06 4.59898e-05 30
end_path:michelremoving:michelremoving 0.0185101 0.0846035 0.626961 0.0331509 0.117794 30
end_path:michelremovinggnocchi:michelremoving 0.0164244 0.0777346 0.608021 0.0286412 0.115286 30
end_path:calibanagnocchi:CalibAnaTree 0.169112 0.398973 0.641659 0.383409 0.116459 30
end_path:out1:RootOutput(write) 0.605482 0.832763 1.16291 0.840325 0.141206 30
========================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 4924.08 MB
Peak resident set size usage (VmHWM): 2594.82 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np04hd_raw_run029107_0754_dataflow6_datawriter_0_20240907T032011_reco_stage1_reco_stage2_20240926T143802_keepup.root
\tHists: np04hd_raw_run029107_0754_dataflow6_datawriter_0_20240907T032011_reco_stage1_reco_stage2_20240926T143802_keepup_hists.root
Forming reco metadata
/cvmfs/larsoft.opensciencegrid.org/products/python/v3_9_15/Linux64bit+3.10-2.17/bin/python: No module named meta_maker
Error in reco metadata
