Jobsub ID 78936.28@justin-prod-sched02.dune.hep.ac.uk
| Jobsub ID | 78936.28@justin-prod-sched02.dune.hep.ac.uk |
| Workflow ID | 3453 |
| Stage ID | 1 |
| User name | calcuttj@fnal.gov |
| HTCondor Group | group_dune.prod_mcsim |
| Requested | Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 18000 (5 hours) |
| Submitted time | 2024-09-26 13:39:58 |
| Site | IT_CNAF |
| Entry | CMSHTPC_T1_IT_CNAF_condor_ce02 |
| Last heartbeat | 2024-09-26 14:55:27 |
| From worker node | Hostname | cn-leo-040.cr.cnaf.infn.it |
| cpuinfo | Intel(R) Xeon(R) Platinum 8480+ |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 216000 (60 hours) |
| Inner Apptainer? | True |
| Job state | jobscript_error |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk |
| Started | 2024-09-26 13:40:56 |
| Input files | hd-protodune:np04hd_raw_run029107_3260_dataflow3_datawriter_0_20240908T223042.hdf5
|
| Jobscript | Exit code | 1 |
| Real time | 0m (0s) |
| CPU time | 0m (0s = 0%) |
| Outputting started | |
| Output files | |
| Finished | 2024-09-26 14:55:27 |
| Saved logs | justin-logs:78936.28-justin-prod-sched02.dune.hep.ac.uk.logs.tgz |
| List job events Wrapper job log |
Jobscript log (last 10,000 characters)
1738 XUVs
C:0 T:6 43384 XUs and 93877 XVs -> 3152 XUVs
C:0 T:7 28 XUs and 32 XVs -> 2 XUVs
6307 XUVs total
2248 collection wire objects
6307 potential space points
Neighbour search...
686371 tests to find 332352 neighbours
Iterating with no regularization...
Begin: 1.41829e+07
0 1.32237e+07
1 1.3162e+07
2 1.31518e+07
Now with regularization...
Begin: 1.25118e+07
0 1.25085e+07
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Begin processing the 29th record. run: 29107 subRun: 1 event: 782683 at 26-Sep-2024 16:50:53 CEST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 53971 traces, 53971 summary
Retagger: tagging trace set: gauss with 38884 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 38884 traces tagged "gauss"
FrameSaver: q=811999 n=692273 tag=gauss
wclsFrameSaver: saving 53971 traces tagged "wiener"
FrameSaver: q=874269 n=618323 tag=wiener
2234 hits in TPC 0
2891 hits in TPC 1
4123 hits in TPC 2
2755 hits in TPC 3
3069 hits in TPC 4
7148 hits in TPC 5
3707 hits in TPC 6
2011 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 33 XUs and 35 XVs -> 0 XUVs
C:0 T:2 8041 XUs and 8899 XVs -> 911 XUVs
C:0 T:3 26 XUs and 45 XVs -> 2 XUVs
C:0 T:4 2511 XUs and 2307 XVs -> 563 XUVs
C:0 T:5 60520 XUs and 75360 XVs -> 2584 XUVs
C:0 T:6 4099 XUs and 5558 XVs -> 425 XUVs
C:0 T:7 329 XUs and 354 XVs -> 11 XUVs
4496 XUVs total
1330 collection wire objects
4496 potential space points
Neighbour search...
336338 tests to find 175280 neighbours
Iterating with no regularization...
Begin: 4.63392e+06
0 4.03533e+06
1 3.98362e+06
2 3.97484e+06
3 3.9718e+06
Now with regularization...
Begin: 3.57703e+06
0 3.57489e+06
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Begin processing the 30th record. run: 29107 subRun: 1 event: 782691 at 26-Sep-2024 16:52:41 CEST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 79173 traces, 79173 summary
Retagger: tagging trace set: gauss with 57645 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 57645 traces tagged "gauss"
FrameSaver: q=1.24271e+06 n=1058050 tag=gauss
wclsFrameSaver: saving 79173 traces tagged "wiener"
FrameSaver: q=1.34416e+06 n=965757 tag=wiener
2743 hits in TPC 0
2410 hits in TPC 1
6434 hits in TPC 2
4140 hits in TPC 3
3981 hits in TPC 4
5507 hits in TPC 5
10391 hits in TPC 6
5652 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 38 XUs and 23 XVs -> 0 XUVs
C:0 T:2 29730 XUs and 33396 XVs -> 496 XUVs
C:0 T:3 24 XUs and 48 XVs -> 0 XUVs
C:0 T:4 461 XUs and 596 XVs -> 17 XUVs
C:0 T:5 12465 XUs and 21890 XVs -> 445 XUVs
C:0 T:6 41753 XUs and 47356 XVs -> 2628 XUVs
C:0 T:7 281 XUs and 347 XVs -> 10 XUVs
3596 XUVs total
1463 collection wire objects
3596 potential space points
Neighbour search...
120694 tests to find 75344 neighbours
Iterating with no regularization...
Begin: 7.31466e+06
0 6.77509e+06
1 6.70856e+06
2 6.69785e+06
3 6.6949e+06
Now with regularization...
Begin: 6.25004e+06
0 6.24839e+06
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLArTPCIntercepts - inconsistent number of intercepts between a line and the LArTPC
BdtBeamParticleIdTool::SliceFeatures::GetLArTPCIntercepts - inconsistent number of intercepts between a line and the LArTPC
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
26-Sep-2024 16:55:01 CEST Closed output file "np04hd_raw_run029107_3260_dataflow3_datawriter_0_20240908T223042_reco_stage1_reco_stage2_20240926T145501_keepup.root"
26-Sep-2024 16:55:01 CEST Closed input file "np04hd_raw_run029107_3260_dataflow3_datawriter_0_20240908T223042_reco_stage1.root"
========================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
========================================================================================================================================
Full event 69.3423 102.18 137.357 99.3004 16.9038 30
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.000709313 0.00215436 0.00745148 0.00181851 0.00140175 30
produce:fembfilter:PDHDFEMBFilter 0.709418 0.783534 1.24174 0.764012 0.0937878 30
produce:wclsdatahd:WireCellToolkit 59.4636 63.2558 72.5467 62.8991 2.53357 30
produce:gaushit:GausHitFinder 0.271558 0.631455 2.89788 0.522766 0.447546 30
produce:nhitsfilter:NumberOfHitsFilter 0.000371483 0.000667235 0.00305317 0.000538616 0.000467653 30
produce:reco3d:SpacePointSolver 0.0866514 0.954545 3.48094 0.552783 0.818152 29
produce:hitpdune:DisambigFromSpacePoints 0.0873713 0.339189 0.717877 0.284764 0.140131 29
produce:pandora:StandardPandora 3.92905 9.49384 19.4504 9.79347 3.76509 29
produce:pandoraWriter:StandardPandora 0.0760585 0.160848 0.252672 0.149845 0.0453244 29
produce:pandoraTrack:LArPandoraTrackCreation 0.803219 2.43148 5.84585 1.85216 1.31998 29
produce:pandoraShower:LArPandoraShowerCreation 0.605175 3.23939 8.58521 2.53177 2.16762 29
produce:pandoracalo:Calorimetry 0.322729 0.976081 2.05946 0.79628 0.478279 29
produce:pandoracalonosce:Calorimetry 0.290077 0.921849 1.89527 0.75207 0.453628 29
produce:pandorapid:Chi2ParticleID 0.00111789 0.00455504 0.0844319 0.00154821 0.0151048 29
produce:pandoraShowercalo:ShowerCalorimetry 0.447185 2.40017 5.47141 1.92241 1.48665 29
produce:pandoraShowercalonosce:ShowerCalorimetry 0.434522 2.43132 5.58076 1.98638 1.4971 29
produce:pandoraGnocchiCalo:GnocchiCalorimetry 0.0475729 0.142401 1.35649 0.104166 0.230827 29
produce:pandoraGnocchiCalonosce:GnocchiCalorimetry 0.0202803 0.043136 0.0688648 0.0414972 0.0121477 29
produce:emtrkmichelid:EmTrackMichelId 6.27142 13.3689 18.9648 13.6566 3.20539 29
[art]:TriggerResults:TriggerResultInserter 5.8882e-05 8.08623e-05 0.000124242 7.78185e-05 1.46975e-05 30
end_path:out1:RootOutput 1.6403e-05 6.75394e-05 0.000933816 2.3906e-05 0.000175486 30
end_path:michelremoving:michelremoving 6.8776e-05 0.0709881 0.277649 0.0665535 0.0491231 30
end_path:michelremovinggnocchi:michelremoving 5.1469e-05 0.061413 0.157322 0.057585 0.0295181 30
end_path:calibanagnocchi:CalibAnaTree 5.1585e-05 0.642893 1.4646 0.523208 0.356231 30
end_path:out1:RootOutput(write) 0.297048 0.66449 1.22181 0.672703 0.2009 30
========================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 5939.37 MB
Peak resident set size usage (VmHWM): 2742.26 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np04hd_raw_run029107_3260_dataflow3_datawriter_0_20240908T223042_reco_stage1_reco_stage2_20240926T145501_keepup.root
\tHists: np04hd_raw_run029107_3260_dataflow3_datawriter_0_20240908T223042_reco_stage1_reco_stage2_20240926T145501_keepup_hists.root
Forming reco metadata
/cvmfs/larsoft.opensciencegrid.org/products/python/v3_9_15/Linux64bit+3.10-2.17/bin/python: No module named meta_maker
Error in reco metadata
