Jobsub ID 263851.4@justin-prod-sched01.dune.hep.ac.uk
| Jobsub ID | 263851.4@justin-prod-sched01.dune.hep.ac.uk |
| Workflow ID | 3449 |
| Stage ID | 1 |
| User name | calcuttj@fnal.gov |
| HTCondor Group | group_dune.prod_mcsim |
| Requested | Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 18000 (5 hours) |
| Submitted time | 2024-09-26 13:52:03 |
| Site | IT_CNAF |
| Entry | CMSHTPC_T1_IT_CNAF_condor_ce02 |
| Last heartbeat | 2024-09-26 15:05:17 |
| From worker node | Hostname | cn-leo-199.cr.cnaf.infn.it |
| cpuinfo | Intel(R) Xeon(R) Platinum 8480+ |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 216000 (60 hours) |
| Inner Apptainer? | True |
| Job state | jobscript_error |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk |
| Started | 2024-09-26 13:52:55 |
| Input files | hd-protodune:np04hd_raw_run029107_0504_dataflow3_datawriter_0_20240906T235941.hdf5
|
| Jobscript | Exit code | 1 |
| Real time | 0m (0s) |
| CPU time | 0m (0s = 0%) |
| Outputting started | |
| Output files | |
| Finished | 2024-09-26 15:05:17 |
| Saved logs | justin-logs:263851.4-justin-prod-sched01.dune.hep.ac.uk.logs.tgz |
| List job events Wrapper job log |
Jobscript log (last 10,000 characters)
06
Now with regularization...
Begin: 5.22098e+06
0 5.21364e+06
1 5.20756e+06
2 5.20282e+06
BdtBeamParticleIdTool::SliceFeatures::GetLArTPCIntercepts - inconsistent number of intercepts between a line and the LArTPC
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
no of hits closeby 4 event 121203 TrkackID 2 startx, y, z -160.149 554.681 2.03132 wireno, peakt tpcno 477 3506.72 6 dist 0.252974min T, max_T2632.84 4693.82
no of hits closeby 4 event 121203 TrkackID 2 startx, y, z -160.149 554.681 2.03132 wireno, peakt tpcno 477 3506.72 6 dist 0.252974min T, max_T2632.84 4693.82
Begin processing the 29th record. run: 29107 subRun: 1 event: 121211 at 26-Sep-2024 17:01:46 CEST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 79483 traces, 79483 summary
Retagger: tagging trace set: gauss with 58318 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 58318 traces tagged "gauss"
FrameSaver: q=1.36198e+06 n=1063255 tag=gauss
wclsFrameSaver: saving 79483 traces tagged "wiener"
FrameSaver: q=1.46941e+06 n=956808 tag=wiener
2791 hits in TPC 0
2739 hits in TPC 1
6383 hits in TPC 2
5222 hits in TPC 3
4309 hits in TPC 4
6508 hits in TPC 5
8769 hits in TPC 6
4523 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 3187 XUs and 527 XVs -> 39 XUVs
C:0 T:2 22989 XUs and 32288 XVs -> 963 XUVs
C:0 T:3 32 XUs and 61 XVs -> 5 XUVs
C:0 T:4 47 XUs and 68 XVs -> 5 XUVs
C:0 T:5 37539 XUs and 20881 XVs -> 1454 XUVs
C:0 T:6 34975 XUs and 34497 XVs -> 1353 XUVs
C:0 T:7 207 XUs and 241 XVs -> 7 XUVs
3826 XUVs total
1914 collection wire objects
3826 potential space points
Neighbour search...
96810 tests to find 58248 neighbours
Iterating with no regularization...
Begin: 1.68333e+07
0 1.57123e+07
1 1.56331e+07
2 1.56261e+07
Now with regularization...
Begin: 1.4611e+07
0 1.46082e+07
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLArTPCIntercepts - inconsistent number of intercepts between a line and the LArTPC
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
no of hits closeby 6 event 121211 TrkackID 9 startx, y, z 94.2368 588.829 226.15 wireno, peakt tpcno 459 5183 2 dist 381.385min T, max_T2941.04 5205.25
no of hits closeby 6 event 121211 TrkackID 9 startx, y, z 94.2368 588.829 226.15 wireno, peakt tpcno 459 5183 2 dist 381.385min T, max_T2941.04 5205.25
Begin processing the 30th record. run: 29107 subRun: 1 event: 121219 at 26-Sep-2024 17:03:37 CEST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 32540 traces, 32540 summary
Retagger: tagging trace set: gauss with 23496 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 23496 traces tagged "gauss"
FrameSaver: q=471842 n=414397 tag=gauss
wclsFrameSaver: saving 32540 traces tagged "wiener"
FrameSaver: q=516686 n=370861 tag=wiener
2192 hits in TPC 0
3015 hits in TPC 1
1616 hits in TPC 2
687 hits in TPC 3
2052 hits in TPC 4
3868 hits in TPC 5
1345 hits in TPC 6
1601 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 1658 XUs and 1682 XVs -> 18 XUVs
C:0 T:2 2171 XUs and 1123 XVs -> 80 XUVs
C:0 T:3 59 XUs and 78 XVs -> 3 XUVs
C:0 T:4 16 XUs and 46 XVs -> 2 XUVs
C:0 T:5 13193 XUs and 25853 XVs -> 2613 XUVs
C:0 T:6 2105 XUs and 1651 XVs -> 123 XUVs
C:0 T:7 193 XUs and 188 XVs -> 60 XUVs
2899 XUVs total
575 collection wire objects
2899 potential space points
Neighbour search...
451973 tests to find 207108 neighbours
Iterating with no regularization...
Begin: 4.36595e+06
0 4.04429e+06
1 4.01838e+06
2 4.01489e+06
Now with regularization...
Begin: 3.85898e+06
0 3.85709e+06
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
26-Sep-2024 17:04:54 CEST Closed output file "np04hd_raw_run029107_0504_dataflow3_datawriter_0_20240906T235941_reco_stage1_reco_stage2_20240926T150454_keepup.root"
26-Sep-2024 17:04:54 CEST Closed input file "np04hd_raw_run029107_0504_dataflow3_datawriter_0_20240906T235941_reco_stage1.root"
========================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
========================================================================================================================================
Full event 75.9063 98.1052 160.629 96.7529 15.5209 30
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.000500356 0.00255248 0.00654122 0.00209229 0.00184781 30
produce:fembfilter:PDHDFEMBFilter 0.701185 0.724004 0.75615 0.722286 0.0123323 30
produce:wclsdatahd:WireCellToolkit 58.1797 62.0057 65.0443 62.2715 1.66325 30
produce:gaushit:GausHitFinder 0.233097 0.534645 0.976185 0.54964 0.140793 30
produce:nhitsfilter:NumberOfHitsFilter 0.000325933 0.000519815 0.00076354 0.000517969 9.87219e-05 30
produce:reco3d:SpacePointSolver 0.24296 0.904378 3.49798 0.697807 0.711149 30
produce:hitpdune:DisambigFromSpacePoints 0.101604 0.336422 1.08056 0.300201 0.191561 30
produce:pandora:StandardPandora 3.4556 8.90736 30.5497 8.07426 4.92056 30
produce:pandoraWriter:StandardPandora 0.0873537 0.149327 0.245681 0.150834 0.0370392 30
produce:pandoraTrack:LArPandoraTrackCreation 0.58025 1.99917 7.41327 1.95042 1.16885 30
produce:pandoraShower:LArPandoraShowerCreation 0.508324 2.62607 8.92323 2.53585 1.54163 30
produce:pandoracalo:Calorimetry 0.242337 0.843769 3.04969 0.839255 0.471283 30
produce:pandoracalonosce:Calorimetry 0.219428 0.79729 2.94035 0.788311 0.456329 30
produce:pandorapid:Chi2ParticleID 0.000987585 0.00160058 0.00335167 0.00151407 0.000450589 30
produce:pandoraShowercalo:ShowerCalorimetry 0.351453 2.11721 7.97296 1.9426 1.37031 30
produce:pandoraShowercalonosce:ShowerCalorimetry 0.342907 2.06577 7.55434 1.90402 1.30453 30
produce:pandoraGnocchiCalo:GnocchiCalorimetry 0.0397792 0.0945826 0.144447 0.0942315 0.0233666 30
produce:pandoraGnocchiCalonosce:GnocchiCalorimetry 0.0195393 0.0402594 0.0637558 0.0398781 0.00942221 30
produce:emtrkmichelid:EmTrackMichelId 6.29783 12.2795 19.257 11.9812 2.82144 30
[art]:TriggerResults:TriggerResultInserter 5.1669e-05 7.07397e-05 0.000100754 6.8495e-05 1.06184e-05 30
end_path:out1:RootOutput 1.6359e-05 2.27039e-05 3.6311e-05 2.14485e-05 4.66027e-06 30
end_path:michelremoving:michelremoving 0.0246029 0.0682068 0.220224 0.0500623 0.0431089 30
end_path:michelremovinggnocchi:michelremoving 0.0242451 0.0652392 0.217189 0.0467371 0.0431886 30
end_path:calibanagnocchi:CalibAnaTree 0.205499 0.619122 2.1735 0.55125 0.366056 30
end_path:out1:RootOutput(write) 0.273837 0.610992 1.03141 0.598253 0.159822 30
========================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 5739.12 MB
Peak resident set size usage (VmHWM): 2577.62 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np04hd_raw_run029107_0504_dataflow3_datawriter_0_20240906T235941_reco_stage1_reco_stage2_20240926T150454_keepup.root
\tHists: np04hd_raw_run029107_0504_dataflow3_datawriter_0_20240906T235941_reco_stage1_reco_stage2_20240926T150454_keepup_hists.root
Forming reco metadata
/cvmfs/larsoft.opensciencegrid.org/products/python/v3_9_15/Linux64bit+3.10-2.17/bin/python: No module named meta_maker
Error in reco metadata
