Jobsub ID 78933.1@justin-prod-sched02.dune.hep.ac.uk
| Jobsub ID | 78933.1@justin-prod-sched02.dune.hep.ac.uk |
| Workflow ID | 3449 |
| Stage ID | 1 |
| User name | calcuttj@fnal.gov |
| HTCondor Group | group_dune.prod_mcsim |
| Requested | Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 18000 (5 hours) |
| Submitted time | 2024-09-26 13:39:54 |
| Site | IT_CNAF |
| Entry | CMSHTPC_T1_IT_CNAF_condor_ce02 |
| Last heartbeat | 2024-09-26 15:31:22 |
| From worker node | Hostname | cn-314-04-08.cr.cnaf.infn.it |
| cpuinfo | Intel(R) Xeon(R) CPU E5-2697 v4 @ 2.30GHz |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 216000 (60 hours) |
| Inner Apptainer? | True |
| Job state | jobscript_error |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk |
| Started | 2024-09-26 13:40:24 |
| Input files | hd-protodune:np04hd_raw_run029107_0656_dataflow1_datawriter_0_20240907T020226.hdf5
|
| Jobscript | Exit code | 1 |
| Real time | 0m (0s) |
| CPU time | 0m (0s = 0%) |
| Outputting started | |
| Output files | |
| Finished | 2024-09-26 15:31:22 |
| Saved logs | justin-logs:78933.1-justin-prod-sched02.dune.hep.ac.uk.logs.tgz |
| List job events Wrapper job log |
Jobscript log (last 10,000 characters)
XUVs
C:0 T:5 31852 XUs and 20298 XVs -> 1263 XUVs
C:0 T:6 33710 XUs and 45267 XVs -> 1200 XUVs
C:0 T:7 511 XUs and 717 XVs -> 19 XUVs
3926 XUVs total
1410 collection wire objects
3926 potential space points
Neighbour search...
146342 tests to find 85102 neighbours
Iterating with no regularization...
Begin: 9.95635e+06
0 9.07007e+06
1 8.99745e+06
2 8.98265e+06
3 8.97758e+06
Now with regularization...
Begin: 8.50678e+06
0 8.50404e+06
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Begin processing the 29th record. run: 29107 subRun: 1 event: 157689 at 26-Sep-2024 17:25:59 CEST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 77410 traces, 77410 summary
Retagger: tagging trace set: gauss with 54207 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 54207 traces tagged "gauss"
FrameSaver: q=1.059e+06 n=933962 tag=gauss
wclsFrameSaver: saving 77410 traces tagged "wiener"
FrameSaver: q=1.15235e+06 n=826329 tag=wiener
2174 hits in TPC 0
2159 hits in TPC 1
9480 hits in TPC 2
3568 hits in TPC 3
3347 hits in TPC 4
6863 hits in TPC 5
6290 hits in TPC 6
3611 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 121 XUs and 156 XVs -> 11 XUVs
C:0 T:2 46968 XUs and 91456 XVs -> 1182 XUVs
C:0 T:3 2994 XUs and 4352 XVs -> 204 XUVs
C:0 T:4 68 XUs and 74 XVs -> 2 XUVs
C:0 T:5 19842 XUs and 35280 XVs -> 683 XUVs
C:0 T:6 35221 XUs and 40585 XVs -> 561 XUVs
C:0 T:7 1592 XUs and 1240 XVs -> 50 XUVs
2693 XUVs total
1336 collection wire objects
2693 potential space points
Neighbour search...
64987 tests to find 37454 neighbours
Iterating with no regularization...
Begin: 6.47842e+06
0 5.74997e+06
1 5.7209e+06
2 5.71804e+06
Now with regularization...
Begin: 5.53163e+06
0 5.53066e+06
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Begin processing the 30th record. run: 29107 subRun: 1 event: 157697 at 26-Sep-2024 17:29:00 CEST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 35496 traces, 35496 summary
Retagger: tagging trace set: gauss with 26281 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 26281 traces tagged "gauss"
FrameSaver: q=567661 n=502384 tag=gauss
wclsFrameSaver: saving 35496 traces tagged "wiener"
FrameSaver: q=607703 n=457524 tag=wiener
2687 hits in TPC 0
1785 hits in TPC 1
5156 hits in TPC 2
2591 hits in TPC 3
2661 hits in TPC 4
2876 hits in TPC 5
833 hits in TPC 6
153 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 14 XUs and 34 XVs -> 2 XUVs
C:0 T:2 12972 XUs and 12579 XVs -> 430 XUVs
C:0 T:3 15 XUs and 12 XVs -> 0 XUVs
C:0 T:4 340 XUs and 565 XVs -> 87 XUVs
C:0 T:5 23333 XUs and 24648 XVs -> 568 XUVs
C:0 T:6 319 XUs and 315 XVs -> 45 XUVs
C:0 T:7 2 XUs and 1 XVs -> 0 XUVs
1132 XUVs total
549 collection wire objects
1132 potential space points
Neighbour search...
30126 tests to find 19170 neighbours
Iterating with no regularization...
Begin: 3.25099e+06
0 2.96777e+06
1 2.94164e+06
2 2.93892e+06
Now with regularization...
Begin: 2.76041e+06
0 2.75968e+06
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
no of hits closeby 0 event 157697 TrkackID 12 startx, y, z 55.5689 587.016 233.122 wireno, peakt tpcno 76 5275.51 5 dist 0min T, max_T4706.04 5404.82
no of hits closeby 0 event 157697 TrkackID 12 startx, y, z 55.5689 587.016 233.122 wireno, peakt tpcno 76 5275.51 5 dist 0min T, max_T4706.04 5404.82
26-Sep-2024 17:30:55 CEST Closed output file "np04hd_raw_run029107_0656_dataflow1_datawriter_0_20240907T020226_reco_stage1_reco_stage2_20240926T153055_keepup.root"
26-Sep-2024 17:30:55 CEST Closed input file "np04hd_raw_run029107_0656_dataflow1_datawriter_0_20240907T020226_reco_stage1.root"
========================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
========================================================================================================================================
Full event 111.431 166.513 249.203 165.781 32.5414 30
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.000812837 0.00337983 0.00598568 0.00356456 0.00125241 30
produce:fembfilter:PDHDFEMBFilter 0.694966 0.975392 1.27983 1.04477 0.1523 30
produce:wclsdatahd:WireCellToolkit 67.7903 74.8337 80.905 74.4294 3.82254 30
produce:gaushit:GausHitFinder 0.649238 1.22487 1.86443 1.22488 0.315659 30
produce:nhitsfilter:NumberOfHitsFilter 0.000313208 0.000556086 0.000886371 0.000539314 0.000142105 30
produce:reco3d:SpacePointSolver 0.2487 1.50623 5.54842 1.17083 1.14397 30
produce:hitpdune:DisambigFromSpacePoints 0.256521 1.24004 2.98011 1.14899 0.675361 30
produce:pandora:StandardPandora 8.41382 23.1861 73.8412 21.4955 12.2927 30
produce:pandoraWriter:StandardPandora 0.132564 0.245931 0.429967 0.24309 0.0653782 30
produce:pandoraTrack:LArPandoraTrackCreation 0.837914 2.88034 6.21788 2.8802 1.15329 30
produce:pandoraShower:LArPandoraShowerCreation 0.656472 3.54346 8.54953 3.305 1.9566 30
produce:pandoracalo:Calorimetry 0.457759 1.39774 2.44551 1.40297 0.526525 30
produce:pandoracalonosce:Calorimetry 0.410073 1.27753 2.27103 1.2961 0.473226 30
produce:pandorapid:Chi2ParticleID 0.00160883 0.00314251 0.00555192 0.00311336 0.000774918 30
produce:pandoraShowercalo:ShowerCalorimetry 0.54603 3.11053 8.1122 3.05331 1.66643 30
produce:pandoraShowercalonosce:ShowerCalorimetry 0.544663 3.03703 6.59194 3.08923 1.4793 30
produce:pandoraGnocchiCalo:GnocchiCalorimetry 0.109308 0.251125 0.42561 0.251732 0.077793 30
produce:pandoraGnocchiCalonosce:GnocchiCalorimetry 0.0462456 0.0928752 0.136156 0.0922906 0.0257409 30
produce:emtrkmichelid:EmTrackMichelId 22.6661 45.2632 82.2272 42.1663 12.3226 30
[art]:TriggerResults:TriggerResultInserter 2.5132e-05 5.54369e-05 0.000151698 5.18995e-05 2.3008e-05 30
end_path:out1:RootOutput 9.087e-06 2.03978e-05 0.000219234 1.219e-05 3.75101e-05 30
end_path:michelremoving:michelremoving 0.0280997 0.114329 0.349684 0.0679955 0.09868 30
end_path:michelremovinggnocchi:michelremoving 0.0266298 0.120689 0.658142 0.0604548 0.132689 30
end_path:calibanagnocchi:CalibAnaTree 0.299609 0.904338 1.80937 0.864231 0.388709 30
end_path:out1:RootOutput(write) 0.406701 1.14552 1.9297 1.11529 0.399664 30
========================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 4950.07 MB
Peak resident set size usage (VmHWM): 2296.72 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np04hd_raw_run029107_0656_dataflow1_datawriter_0_20240907T020226_reco_stage1_reco_stage2_20240926T153055_keepup.root
\tHists: np04hd_raw_run029107_0656_dataflow1_datawriter_0_20240907T020226_reco_stage1_reco_stage2_20240926T153055_keepup_hists.root
Forming reco metadata
/cvmfs/larsoft.opensciencegrid.org/products/python/v3_9_15/Linux64bit+3.10-2.17/bin/python: No module named meta_maker
Error in reco metadata
