Jobsub ID 78469.22@justin-prod-sched02.dune.hep.ac.uk
| Jobsub ID | 78469.22@justin-prod-sched02.dune.hep.ac.uk |
| Workflow ID | 3449 |
| Stage ID | 1 |
| User name | calcuttj@fnal.gov |
| HTCondor Group | group_dune.prod_mcsim |
| Requested | Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 18000 (5 hours) |
| Submitted time | 2024-09-26 10:01:18 |
| Site | IT_CNAF |
| Entry | CMSHTPC_T1_IT_CNAF_condor_ce02 |
| Last heartbeat | 2024-09-26 12:10:04 |
| From worker node | Hostname | cn-leo-109.cr.cnaf.infn.it |
| cpuinfo | Intel(R) Xeon(R) Platinum 8480+ |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 216000 (60 hours) |
| Inner Apptainer? | True |
| Job state | outputting_failed |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk |
| Started | 2024-09-26 10:23:30 |
| Input files | hd-protodune:np04hd_raw_run029107_0836_dataflow3_datawriter_0_20240907T042528.hdf5
|
| Jobscript | Exit code | 1 |
| Real time | 0m (0s) |
| CPU time | 0m (0s = 0%) |
| Outputting started | |
| Output files | |
| Finished | 2024-09-26 12:10:04 |
| List job events Wrapper job log |
Jobscript log (last 10,000 characters)
9 XVs -> 4363 XUVs
C:0 T:6 27793 XUs and 17803 XVs -> 272 XUVs
C:0 T:7 20 XUs and 9 XVs -> 1 XUVs
5046 XUVs total
1468 collection wire objects
5046 potential space points
Neighbour search...
370084 tests to find 225460 neighbours
Iterating with no regularization...
Begin: 7.11868e+06
0 6.62136e+06
1 6.57729e+06
2 6.56721e+06
3 6.56306e+06
Now with regularization...
Begin: 6.10499e+06
0 6.10182e+06
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
no of hits closeby 1 event 200883 TrkackID 10 startx, y, z 111.742 434.037 316.997 wireno, peakt tpcno 87 1206.5 5 dist 3.50916min T, max_T0.889132 1423.86
no of hits closeby 1 event 200883 TrkackID 10 startx, y, z 111.742 434.037 316.997 wireno, peakt tpcno 87 1206.5 5 dist 3.50916min T, max_T0.889132 1423.86
Begin processing the 29th record. run: 29107 subRun: 1 event: 200891 at 26-Sep-2024 13:32:14 CEST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 76251 traces, 76251 summary
Retagger: tagging trace set: gauss with 54744 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 54744 traces tagged "gauss"
FrameSaver: q=1.07073e+06 n=964638 tag=gauss
wclsFrameSaver: saving 76251 traces tagged "wiener"
FrameSaver: q=1.18487e+06 n=872130 tag=wiener
3227 hits in TPC 0
5300 hits in TPC 1
4451 hits in TPC 2
2963 hits in TPC 3
4116 hits in TPC 4
7805 hits in TPC 5
6658 hits in TPC 6
4992 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 1811 XUs and 7530 XVs -> 117 XUVs
C:0 T:2 10557 XUs and 32471 XVs -> 1509 XUVs
C:0 T:3 28 XUs and 52 XVs -> 1 XUVs
C:0 T:4 271 XUs and 328 XVs -> 12 XUVs
C:0 T:5 55595 XUs and 65716 XVs -> 1260 XUVs
C:0 T:6 17205 XUs and 35297 XVs -> 1073 XUVs
C:0 T:7 830 XUs and 888 XVs -> 25 XUVs
3997 XUVs total
1682 collection wire objects
3997 potential space points
Neighbour search...
190541 tests to find 108786 neighbours
Iterating with no regularization...
Begin: 1.16447e+07
0 9.4187e+06
1 9.21998e+06
2 9.20515e+06
3 9.201e+06
Now with regularization...
Begin: 8.83684e+06
0 8.83439e+06
TwoViewTransverseTracksAlgorithm: failed to calculate correlation coefficient p-value for these numbers
----view 0: 0.045114 0.282448 0.108151 0.0613157 0.0679321 0.0709943 0.0835267 0.0392572 0.0636792 0.0614263 0.0300882
----view 1: 0.0027304 0.00273043 0.00273041 0.00273042 0.00273041 0.00273041 0.00273041 0.00273042 0.00273041 0.00273041 0.00273042
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Begin processing the 30th record. run: 29107 subRun: 1 event: 200899 at 26-Sep-2024 13:34:11 CEST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 65245 traces, 65245 summary
Retagger: tagging trace set: gauss with 47543 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 47543 traces tagged "gauss"
FrameSaver: q=1.05663e+06 n=865277 tag=gauss
wclsFrameSaver: saving 65245 traces tagged "wiener"
FrameSaver: q=1.16584e+06 n=786317 tag=wiener
3065 hits in TPC 0
3338 hits in TPC 1
4836 hits in TPC 2
2396 hits in TPC 3
4832 hits in TPC 4
7695 hits in TPC 5
5831 hits in TPC 6
2355 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 232 XUs and 218 XVs -> 11 XUVs
C:0 T:2 46677 XUs and 87655 XVs -> 1283 XUVs
C:0 T:3 177 XUs and 133 XVs -> 5 XUVs
C:0 T:4 365 XUs and 524 XVs -> 24 XUVs
C:0 T:5 31676 XUs and 39142 XVs -> 3039 XUVs
C:0 T:6 61150 XUs and 57712 XVs -> 1742 XUVs
C:0 T:7 342 XUs and 270 XVs -> 2 XUVs
6106 XUVs total
1718 collection wire objects
6106 potential space points
Neighbour search...
295812 tests to find 170934 neighbours
Iterating with no regularization...
Begin: 1.25476e+07
0 1.13861e+07
1 1.12703e+07
2 1.12531e+07
3 1.12482e+07
Now with regularization...
Begin: 1.06069e+07
0 1.06034e+07
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
26-Sep-2024 13:36:02 CEST Closed output file "np04hd_raw_run029107_0836_dataflow3_datawriter_0_20240907T042528_reco_stage1_reco_stage2_20240926T113602_keepup.root"
26-Sep-2024 13:36:02 CEST Closed input file "np04hd_raw_run029107_0836_dataflow3_datawriter_0_20240907T042528_reco_stage1.root"
========================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
========================================================================================================================================
Full event 86.0752 104.076 142.702 102.478 13.2787 30
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.000598172 0.00290801 0.00792277 0.00218928 0.00231396 30
produce:fembfilter:PDHDFEMBFilter 0.666069 0.715248 0.801024 0.714792 0.0305092 30
produce:wclsdatahd:WireCellToolkit 56.1381 60.8805 64.9303 61.1389 2.46951 30
produce:gaushit:GausHitFinder 0.390967 0.58632 0.824341 0.583732 0.107052 30
produce:nhitsfilter:NumberOfHitsFilter 0.000414575 0.000592728 0.000875713 0.00056551 0.00013104 30
produce:reco3d:SpacePointSolver 0.26055 1.28388 6.34993 0.898633 1.14306 30
produce:hitpdune:DisambigFromSpacePoints 0.160086 0.372609 0.75956 0.349359 0.13211 30
produce:pandora:StandardPandora 5.68972 10.317 18.937 9.54191 3.74914 30
produce:pandoraWriter:StandardPandora 0.107872 0.173713 0.347912 0.162459 0.0466348 30
produce:pandoraTrack:LArPandoraTrackCreation 1.10214 2.46953 5.76319 2.22446 1.0016 30
produce:pandoraShower:LArPandoraShowerCreation 0.781179 3.10558 8.14414 2.72036 1.52654 30
produce:pandoracalo:Calorimetry 0.434347 0.965308 1.97947 0.847465 0.354399 30
produce:pandoracalonosce:Calorimetry 0.40652 0.89959 1.88692 0.790002 0.345207 30
produce:pandorapid:Chi2ParticleID 0.0011259 0.0017119 0.00315669 0.0016127 0.000402259 30
produce:pandoraShowercalo:ShowerCalorimetry 0.632197 2.3683 6.51917 1.96455 1.16024 30
produce:pandoraShowercalonosce:ShowerCalorimetry 0.602499 2.30211 6.47726 1.92597 1.13482 30
produce:pandoraGnocchiCalo:GnocchiCalorimetry 0.0644501 0.102587 0.156204 0.0964905 0.0201503 30
produce:pandoraGnocchiCalonosce:GnocchiCalorimetry 0.0268959 0.0475091 0.0719041 0.0453945 0.0107882 30
produce:emtrkmichelid:EmTrackMichelId 10.9473 15.537 22.69 14.7668 3.30461 30
[art]:TriggerResults:TriggerResultInserter 4.7939e-05 7.89048e-05 0.000124317 8.27535e-05 1.71328e-05 30
end_path:out1:RootOutput 1.3149e-05 2.39074e-05 3.8062e-05 2.48115e-05 6.10041e-06 30
end_path:michelremoving:michelremoving 0.0298886 0.0992761 0.441503 0.0630804 0.0890125 30
end_path:michelremovinggnocchi:michelremoving 0.0247966 0.0955832 0.424847 0.0582476 0.0874504 30
end_path:calibanagnocchi:CalibAnaTree 0.335073 0.678854 1.52968 0.608976 0.260556 30
end_path:out1:RootOutput(write) 0.478389 0.646254 0.980312 0.619351 0.138247 30
========================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 5785.35 MB
Peak resident set size usage (VmHWM): 2623.58 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np04hd_raw_run029107_0836_dataflow3_datawriter_0_20240907T042528_reco_stage1_reco_stage2_20240926T113602_keepup.root
\tHists: np04hd_raw_run029107_0836_dataflow3_datawriter_0_20240907T042528_reco_stage1_reco_stage2_20240926T113602_keepup_hists.root
Forming reco metadata
/cvmfs/larsoft.opensciencegrid.org/products/python/v3_9_15/Linux64bit+3.10-2.17/bin/python: No module named meta_maker
Error in reco metadata
