Jobsub ID 79005.0@justin-prod-sched02.dune.hep.ac.uk
| Jobsub ID | 79005.0@justin-prod-sched02.dune.hep.ac.uk |
| Workflow ID | 3448 |
| Stage ID | 1 |
| User name | calcuttj@fnal.gov |
| HTCondor Group | group_dune.prod_mcsim |
| Requested | Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 18000 (5 hours) |
| Submitted time | 2024-09-26 13:55:33 |
| Site | IT_CNAF |
| Entry | CMSHTPC_T1_IT_CNAF_condor_ce02 |
| Last heartbeat | 2024-09-26 15:04:10 |
| From worker node | Hostname | cn-leo-187.cr.cnaf.infn.it |
| cpuinfo | Intel(R) Xeon(R) Platinum 8480+ |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 216000 (60 hours) |
| Inner Apptainer? | True |
| Job state | jobscript_error |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk |
| Started | 2024-09-26 13:56:26 |
| Input files | hd-protodune:np04hd_raw_run029094_0112_dataflow1_datawriter_0_20240906T092756.hdf5
|
| Jobscript | Exit code | 1 |
| Real time | 0m (0s) |
| CPU time | 0m (0s = 0%) |
| Outputting started | |
| Output files | |
| Finished | 2024-09-26 15:04:10 |
| Saved logs | justin-logs:79005.0-justin-prod-sched02.dune.hep.ac.uk.logs.tgz |
| List job events Wrapper job log |
Jobscript log (last 10,000 characters)
6
3245 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 22 XUs and 43 XVs -> 2 XUVs
C:0 T:2 27703 XUs and 22646 XVs -> 800 XUVs
C:0 T:3 356 XUs and 281 XVs -> 4 XUVs
C:0 T:4 209 XUs and 197 XVs -> 11 XUVs
C:0 T:5 66930 XUs and 59077 XVs -> 3047 XUVs
C:0 T:6 17513 XUs and 11368 XVs -> 259 XUVs
C:0 T:7 491 XUs and 250 XVs -> 18 XUVs
4141 XUVs total
1923 collection wire objects
4141 potential space points
Neighbour search...
142353 tests to find 66818 neighbours
Iterating with no regularization...
Begin: 8.41116e+06
0 7.42515e+06
1 7.3042e+06
2 7.2926e+06
3 7.29021e+06
Now with regularization...
Begin: 6.70362e+06
0 6.70153e+06
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Begin processing the 29th record. run: 29094 subRun: 1 event: 27113 at 26-Sep-2024 17:00:14 CEST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 49131 traces, 49131 summary
Retagger: tagging trace set: gauss with 35896 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 35896 traces tagged "gauss"
FrameSaver: q=868666 n=685750 tag=gauss
wclsFrameSaver: saving 49131 traces tagged "wiener"
FrameSaver: q=937111 n=626798 tag=wiener
1857 hits in TPC 0
1884 hits in TPC 1
5046 hits in TPC 2
4995 hits in TPC 3
2308 hits in TPC 4
3829 hits in TPC 5
4280 hits in TPC 6
2718 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 97 XUs and 54 XVs -> 0 XUVs
C:0 T:2 13719 XUs and 35348 XVs -> 1510 XUVs
C:0 T:3 351 XUs and 840 XVs -> 52 XUVs
C:0 T:4 9 XUs and 17 XVs -> 3 XUVs
C:0 T:5 8278 XUs and 11476 XVs -> 2482 XUVs
C:0 T:6 7370 XUs and 11898 XVs -> 752 XUVs
C:0 T:7 60 XUs and 75 XVs -> 5 XUVs
4804 XUVs total
1812 collection wire objects
4804 potential space points
Neighbour search...
202690 tests to find 111806 neighbours
Iterating with no regularization...
Begin: 8.86526e+06
0 7.65833e+06
1 7.55119e+06
2 7.53311e+06
3 7.52759e+06
Now with regularization...
Begin: 6.55734e+06
0 6.55265e+06
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Begin processing the 30th record. run: 29094 subRun: 1 event: 27121 at 26-Sep-2024 17:01:47 CEST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 72546 traces, 72546 summary
Retagger: tagging trace set: gauss with 53280 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 53280 traces tagged "gauss"
FrameSaver: q=1.24096e+06 n=1005769 tag=gauss
wclsFrameSaver: saving 72546 traces tagged "wiener"
FrameSaver: q=1.32582e+06 n=907148 tag=wiener
4838 hits in TPC 0
2446 hits in TPC 1
9585 hits in TPC 2
7016 hits in TPC 3
2589 hits in TPC 4
3552 hits in TPC 5
5344 hits in TPC 6
3188 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 297 XUs and 177 XVs -> 7 XUVs
C:0 T:2 45102 XUs and 49481 XVs -> 1141 XUVs
C:0 T:3 394 XUs and 556 XVs -> 17 XUVs
C:0 T:4 5 XUs and 12 XVs -> 0 XUVs
C:0 T:5 6853 XUs and 10039 XVs -> 605 XUVs
C:0 T:6 18130 XUs and 19674 XVs -> 473 XUVs
C:0 T:7 104 XUs and 107 XVs -> 2 XUVs
2245 XUVs total
1434 collection wire objects
2245 potential space points
Neighbour search...
26131 tests to find 15322 neighbours
Iterating with no regularization...
Begin: 8.83957e+06
0 8.40238e+06
1 8.38367e+06
2 8.38222e+06
Now with regularization...
Begin: 7.87395e+06
0 7.87353e+06
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
no of hits closeby 1 event 27121 TrkackID 3 startx, y, z -151.239 465.073 456.606 wireno, peakt tpcno 61 3684.68 6 dist 0min T, max_T3171.46 5125.56
no of hits closeby 1 event 27121 TrkackID 3 startx, y, z -151.239 465.073 456.606 wireno, peakt tpcno 61 3684.68 6 dist 0min T, max_T3171.46 5125.56
26-Sep-2024 17:03:37 CEST Closed output file "np04hd_raw_run029094_0112_dataflow1_datawriter_0_20240906T092756_reco_stage1_reco_stage2_20240926T150337_keepup.root"
26-Sep-2024 17:03:37 CEST Closed input file "np04hd_raw_run029094_0112_dataflow1_datawriter_0_20240906T092756_reco_stage1.root"
========================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
========================================================================================================================================
Full event 75.2005 93.0091 117.896 93.0453 10.5784 30
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.000578039 0.00228144 0.00687209 0.00200855 0.00167411 30
produce:fembfilter:PDHDFEMBFilter 0.654901 0.709951 1.03701 0.699527 0.0789719 30
produce:wclsdatahd:WireCellToolkit 54.5458 59.2878 63.1539 60.0798 2.50386 30
produce:gaushit:GausHitFinder 0.266207 0.506029 0.725279 0.505087 0.102398 30
produce:nhitsfilter:NumberOfHitsFilter 0.00028999 0.000491393 0.000713952 0.0004942 9.12578e-05 30
produce:reco3d:SpacePointSolver 0.0803857 0.69581 2.25878 0.551267 0.462565 30
produce:hitpdune:DisambigFromSpacePoints 0.0872763 0.300585 0.52762 0.29323 0.105437 30
produce:pandora:StandardPandora 2.35564 7.82464 13.371 7.66538 2.27749 30
produce:pandoraWriter:StandardPandora 0.069316 0.143619 0.301204 0.141767 0.0459234 30
produce:pandoraTrack:LArPandoraTrackCreation 0.652135 1.97377 5.39459 1.80549 1.0373 30
produce:pandoraShower:LArPandoraShowerCreation 0.66823 2.51422 6.1003 2.55384 1.26065 30
produce:pandoracalo:Calorimetry 0.277031 0.792954 1.55699 0.782574 0.317926 30
produce:pandoracalonosce:Calorimetry 0.254637 0.737923 1.56303 0.730217 0.314812 30
produce:pandorapid:Chi2ParticleID 0.00108204 0.00627526 0.144855 0.00146907 0.0257354 30
produce:pandoraShowercalo:ShowerCalorimetry 0.557465 1.96332 3.73277 2.01203 0.897355 30
produce:pandoraShowercalonosce:ShowerCalorimetry 0.448481 1.89053 3.62244 1.95144 0.85203 30
produce:pandoraGnocchiCalo:GnocchiCalorimetry 0.0403964 0.0907765 0.142251 0.0890411 0.023449 30
produce:pandoraGnocchiCalonosce:GnocchiCalorimetry 0.0205086 0.0407582 0.062334 0.0402898 0.0109543 30
produce:emtrkmichelid:EmTrackMichelId 6.33818 11.8926 17.2263 12.1896 2.16505 30
[art]:TriggerResults:TriggerResultInserter 3.5069e-05 6.71761e-05 0.000118014 7.13135e-05 1.86918e-05 30
end_path:out1:RootOutput 1.1392e-05 2.04833e-05 3.8519e-05 2.10075e-05 6.47739e-06 30
end_path:michelremoving:michelremoving 0.0211863 0.0730994 0.196504 0.0572215 0.0476653 30
end_path:michelremovinggnocchi:michelremoving 0.017271 0.0697636 0.190167 0.0511021 0.0477314 30
end_path:calibanagnocchi:CalibAnaTree 0.212775 0.559463 1.15601 0.54531 0.255115 30
end_path:out1:RootOutput(write) 0.31277 0.575229 0.854191 0.570934 0.111916 30
========================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 5771.42 MB
Peak resident set size usage (VmHWM): 2535 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np04hd_raw_run029094_0112_dataflow1_datawriter_0_20240906T092756_reco_stage1_reco_stage2_20240926T150337_keepup.root
\tHists: np04hd_raw_run029094_0112_dataflow1_datawriter_0_20240906T092756_reco_stage1_reco_stage2_20240926T150337_keepup_hists.root
Forming reco metadata
/cvmfs/larsoft.opensciencegrid.org/products/python/v3_9_15/Linux64bit+3.10-2.17/bin/python: No module named meta_maker
Error in reco metadata
