Jobsub ID 263331.41@justin-prod-sched01.dune.hep.ac.uk
| Jobsub ID | 263331.41@justin-prod-sched01.dune.hep.ac.uk |
| Workflow ID | 3448 |
| Stage ID | 1 |
| User name | calcuttj@fnal.gov |
| HTCondor Group | group_dune.prod_mcsim |
| Requested | Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 18000 (5 hours) |
| Submitted time | 2024-09-26 10:10:04 |
| Site | UK_Sheffield |
| Entry | DUNE_UK_Sheffield_lcgce1 |
| Last heartbeat | 2024-09-26 12:21:01 |
| From worker node | Hostname | wn032.hep |
| cpuinfo | Intel(R) Xeon(R) Gold 6226 CPU @ 2.70GHz |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 257400 (71 hours) |
| Inner Apptainer? | True |
| Job state | outputting_failed |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk |
| Started | 2024-09-26 10:43:55 |
| Input files | hd-protodune:np04hd_raw_run029094_0075_dataflow5_datawriter_0_20240906T090951.hdf5
|
| Jobscript | Exit code | 1 |
| Real time | 0m (0s) |
| CPU time | 0m (0s = 0%) |
| Outputting started | |
| Output files | |
| Finished | 2024-09-26 12:21:01 |
| List job events Wrapper job log |
Jobscript log (last 10,000 characters)
5 XUVs
C:0 T:6 16716 XUs and 21775 XVs -> 470 XUVs
C:0 T:7 341 XUs and 273 XVs -> 17 XUVs
2329 XUVs total
1508 collection wire objects
2329 potential space points
Neighbour search...
53331 tests to find 25658 neighbours
Iterating with no regularization...
Begin: 9.28462e+06
0 8.55436e+06
1 8.51335e+06
2 8.50808e+06
Now with regularization...
Begin: 7.91295e+06
0 7.91049e+06
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
no of hits closeby 0 event 18221 TrkackID 7 startx, y, z 200.594 450.947 110.149 wireno, peakt tpcno 423 2538.26 2 dist 302.631min T, max_T10.1568 2561.39
no of hits closeby 0 event 18221 TrkackID 7 startx, y, z 200.594 450.947 110.149 wireno, peakt tpcno 423 2538.26 2 dist 302.631min T, max_T10.1568 2561.39
Begin processing the 29th record. run: 29094 subRun: 1 event: 18229 at 26-Sep-2024 12:53:59 BST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 51048 traces, 51048 summary
Retagger: tagging trace set: gauss with 37744 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 37744 traces tagged "gauss"
FrameSaver: q=808600 n=672605 tag=gauss
wclsFrameSaver: saving 51048 traces tagged "wiener"
FrameSaver: q=866263 n=603170 tag=wiener
2760 hits in TPC 0
1954 hits in TPC 1
4864 hits in TPC 2
2781 hits in TPC 3
2369 hits in TPC 4
5053 hits in TPC 5
3398 hits in TPC 6
1988 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 384 XUs and 419 XVs -> 16 XUVs
C:0 T:2 19977 XUs and 16921 XVs -> 228 XUVs
C:0 T:3 202 XUs and 268 XVs -> 3 XUVs
C:0 T:4 55 XUs and 46 XVs -> 0 XUVs
C:0 T:5 16244 XUs and 13573 XVs -> 744 XUVs
C:0 T:6 11168 XUs and 11939 XVs -> 459 XUVs
C:0 T:7 964 XUs and 634 XVs -> 31 XUVs
1481 XUVs total
857 collection wire objects
1481 potential space points
Neighbour search...
20469 tests to find 13424 neighbours
Iterating with no regularization...
Begin: 9.07552e+06
0 8.58032e+06
1 8.54567e+06
2 8.53918e+06
Now with regularization...
Begin: 8.09244e+06
0 8.09069e+06
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Begin processing the 30th record. run: 29094 subRun: 1 event: 18245 at 26-Sep-2024 12:55:17 BST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 125993 traces, 125993 summary
Retagger: tagging trace set: gauss with 95983 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 95983 traces tagged "gauss"
FrameSaver: q=2.43912e+06 n=1766789 tag=gauss
wclsFrameSaver: saving 125993 traces tagged "wiener"
FrameSaver: q=2.62673e+06 n=1596405 tag=wiener
3037 hits in TPC 0
3640 hits in TPC 1
18683 hits in TPC 2
13295 hits in TPC 3
6900 hits in TPC 4
10882 hits in TPC 5
7187 hits in TPC 6
5311 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 432 XUs and 226 XVs -> 13 XUVs
C:0 T:2 220007 XUs and 303637 XVs -> 17711 XUVs
C:0 T:3 290 XUs and 475 XVs -> 14 XUVs
C:0 T:4 647 XUs and 950 XVs -> 21 XUVs
C:0 T:5 67090 XUs and 47837 XVs -> 7340 XUVs
C:0 T:6 18699 XUs and 25293 XVs -> 1820 XUVs
C:0 T:7 581 XUs and 1176 XVs -> 21 XUVs
26940 XUVs total
5942 collection wire objects
26940 potential space points
Neighbour search...
5053148 tests to find 1677040 neighbours
Iterating with no regularization...
Begin: 5.00139e+07
0 3.19129e+07
1 3.04073e+07
2 3.01308e+07
3 3.00333e+07
4 2.99856e+07
5 2.99586e+07
Now with regularization...
Begin: 2.71883e+07
0 2.7137e+07
1 2.71099e+07
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLArTPCIntercepts - inconsistent number of intercepts between a line and the LArTPC
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
26-Sep-2024 13:00:06 BST Closed output file "np04hd_raw_run029094_0075_dataflow5_datawriter_0_20240906T090951_reco_stage1_reco_stage2_20240926T120006_keepup.root"
26-Sep-2024 13:00:06 BST Closed input file "np04hd_raw_run029094_0075_dataflow5_datawriter_0_20240906T090951_reco_stage1.root"
========================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
========================================================================================================================================
Full event 69.5064 94.0012 287.904 86.1023 38.6586 30
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.000434964 0.0015104 0.00410137 0.0013933 0.000926639 30
produce:fembfilter:PDHDFEMBFilter 0.626417 0.654572 0.739976 0.651443 0.0222148 30
produce:wclsdatahd:WireCellToolkit 46.2692 49.2799 53.7924 49.117 1.30394 30
produce:gaushit:GausHitFinder 0.413871 0.658697 1.3297 0.628878 0.195173 30
produce:nhitsfilter:NumberOfHitsFilter 0.000286967 0.000405564 0.000689792 0.000400376 8.98452e-05 30
produce:reco3d:SpacePointSolver 0.17045 0.953164 7.53897 0.772441 1.27347 30
produce:hitpdune:DisambigFromSpacePoints 0.174291 0.575625 4.56386 0.435398 0.758213 30
produce:pandora:StandardPandora 4.64213 12.8446 88.1693 10.0159 14.5087 30
produce:pandoraWriter:StandardPandora 0.100759 0.153106 0.325246 0.141661 0.0461817 30
produce:pandoraTrack:LArPandoraTrackCreation 0.893666 2.88248 19.7469 2.19833 3.29776 30
produce:pandoraShower:LArPandoraShowerCreation 0.673511 3.11957 21.765 2.32495 3.78947 30
produce:pandoracalo:Calorimetry 0.36912 1.13664 7.64189 0.957805 1.27014 30
produce:pandoracalonosce:Calorimetry 0.315777 1.08787 7.60064 0.858261 1.27722 30
produce:pandorapid:Chi2ParticleID 0.0012266 0.00185509 0.00420361 0.0017262 0.000629117 30
produce:pandoraShowercalo:ShowerCalorimetry 0.510952 2.74827 17.9426 2.02404 3.22053 30
produce:pandoraShowercalonosce:ShowerCalorimetry 0.55148 2.68832 17.2867 2.05074 3.12104 30
produce:pandoraGnocchiCalo:GnocchiCalorimetry 0.0643217 0.118296 0.237708 0.115751 0.0354164 30
produce:pandoraGnocchiCalonosce:GnocchiCalorimetry 0.0283276 0.0479264 0.0962543 0.0473838 0.0144095 30
produce:emtrkmichelid:EmTrackMichelId 8.24456 13.1697 30.9532 12.1672 4.39825 30
[art]:TriggerResults:TriggerResultInserter 3.2598e-05 5.20805e-05 9.4813e-05 5.12275e-05 1.22501e-05 30
end_path:out1:RootOutput 7.492e-06 1.36833e-05 2.5823e-05 1.3673e-05 3.91655e-06 30
end_path:michelremoving:michelremoving 0.0252349 0.0710058 0.284705 0.0546986 0.053506 30
end_path:michelremovinggnocchi:michelremoving 0.0282505 0.0643417 0.296515 0.0468522 0.0546847 30
end_path:calibanagnocchi:CalibAnaTree 0.244269 0.808051 6.16886 0.571881 1.0317 30
end_path:out1:RootOutput(write) 0.40349 0.695521 1.47934 0.686073 0.211538 30
========================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 5792.22 MB
Peak resident set size usage (VmHWM): 2627.62 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np04hd_raw_run029094_0075_dataflow5_datawriter_0_20240906T090951_reco_stage1_reco_stage2_20240926T120006_keepup.root
\tHists: np04hd_raw_run029094_0075_dataflow5_datawriter_0_20240906T090951_reco_stage1_reco_stage2_20240926T120006_keepup_hists.root
Forming reco metadata
/cvmfs/larsoft.opensciencegrid.org/products/python/v3_9_15/Linux64bit+3.10-2.17/bin/python: No module named meta_maker
Error in reco metadata
