Jobsub ID 78442.172@justin-prod-sched02.dune.hep.ac.uk
| Jobsub ID | 78442.172@justin-prod-sched02.dune.hep.ac.uk |
| Workflow ID | 3447 |
| Stage ID | 1 |
| User name | calcuttj@fnal.gov |
| HTCondor Group | group_dune.prod_mcsim |
| Requested | Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 18000 (5 hours) |
| Submitted time | 2024-09-26 09:48:32 |
| Site | FR_CCIN2P3 |
| Entry | DUNE_FR_CCIN2P3_cccondorce03 |
| Last heartbeat | 2024-09-26 11:04:02 |
| From worker node | Hostname | ccwcondor0551 |
| cpuinfo | AMD EPYC 7453 28-Core Processor |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 106200 (29 hours) |
| Inner Apptainer? | True |
| Job state | jobscript_error |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk |
| Started | 2024-09-26 09:51:13 |
| Input files | hd-protodune:np04hd_raw_run029081_0610_dataflow5_datawriter_0_20240905T022622.hdf5
|
| Jobscript | Exit code | 1 |
| Real time | 0m (0s) |
| CPU time | 0m (0s = 0%) |
| Outputting started | |
| Output files | |
| Finished | 2024-09-26 11:04:02 |
| Saved logs | justin-logs:78442.172-justin-prod-sched02.dune.hep.ac.uk.logs.tgz |
| List job events Wrapper job log |
Jobscript log (last 10,000 characters)
1 XUVs
C:0 T:3 51 XUs and 47 XVs -> 6 XUVs
C:0 T:4 138 XUs and 179 XVs -> 7 XUVs
C:0 T:5 16918 XUs and 16236 XVs -> 495 XUVs
C:0 T:6 19921 XUs and 20129 XVs -> 288 XUVs
C:0 T:7 58 XUs and 39 XVs -> 1 XUVs
1459 XUVs total
818 collection wire objects
1459 potential space points
Neighbour search...
40753 tests to find 23878 neighbours
Iterating with no regularization...
Begin: 5.9647e+06
0 5.52732e+06
1 5.48901e+06
2 5.48546e+06
Now with regularization...
Begin: 5.22153e+06
0 5.22043e+06
BdtBeamParticleIdTool::SliceFeatures::GetLArTPCIntercepts - inconsistent number of intercepts between a line and the LArTPC
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Begin processing the 29th record. run: 29081 subRun: 1 event: 146637 at 26-Sep-2024 12:51:03 CEST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 68929 traces, 68929 summary
Retagger: tagging trace set: gauss with 50247 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 50247 traces tagged "gauss"
FrameSaver: q=1.04628e+06 n=944307 tag=gauss
wclsFrameSaver: saving 68929 traces tagged "wiener"
FrameSaver: q=1.15524e+06 n=859327 tag=wiener
3875 hits in TPC 0
4820 hits in TPC 1
9059 hits in TPC 2
4089 hits in TPC 3
2628 hits in TPC 4
5083 hits in TPC 5
5634 hits in TPC 6
2659 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 10771 XUs and 13488 XVs -> 755 XUVs
C:0 T:2 53574 XUs and 113083 XVs -> 1920 XUVs
C:0 T:3 94 XUs and 97 XVs -> 1 XUVs
C:0 T:4 133 XUs and 240 XVs -> 12 XUVs
C:0 T:5 10148 XUs and 15236 XVs -> 464 XUVs
C:0 T:6 62603 XUs and 152778 XVs -> 5854 XUVs
C:0 T:7 290 XUs and 264 XVs -> 12 XUVs
9018 XUVs total
2130 collection wire objects
9018 potential space points
Neighbour search...
683738 tests to find 372882 neighbours
Iterating with no regularization...
Begin: 7.72071e+06
0 6.65219e+06
1 6.56449e+06
2 6.54924e+06
3 6.54397e+06
Now with regularization...
Begin: 6.18911e+06
0 6.18572e+06
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
no of hits closeby 1 event 146637 TrkackID 2 startx, y, z 180.287 603.074 144.827 wireno, peakt tpcno 240 3771.35 2 dist 360.15min T, max_T9.67372 3807.53
no of hits closeby 1 event 146637 TrkackID 2 startx, y, z 180.287 603.074 144.827 wireno, peakt tpcno 240 3771.35 2 dist 360.15min T, max_T9.67372 3807.53
Begin processing the 30th record. run: 29081 subRun: 1 event: 146645 at 26-Sep-2024 12:52:26 CEST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 62121 traces, 62121 summary
Retagger: tagging trace set: gauss with 45408 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 45408 traces tagged "gauss"
FrameSaver: q=959734 n=861624 tag=gauss
wclsFrameSaver: saving 62121 traces tagged "wiener"
FrameSaver: q=1.05075e+06 n=786078 tag=wiener
4354 hits in TPC 0
3108 hits in TPC 1
6821 hits in TPC 2
6252 hits in TPC 3
2772 hits in TPC 4
4140 hits in TPC 5
3792 hits in TPC 6
2071 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 192 XUs and 200 XVs -> 7 XUVs
C:0 T:2 33837 XUs and 94435 XVs -> 3678 XUVs
C:0 T:3 363 XUs and 878 XVs -> 14 XUVs
C:0 T:4 40 XUs and 54 XVs -> 0 XUVs
C:0 T:5 9329 XUs and 16888 XVs -> 456 XUVs
C:0 T:6 7480 XUs and 16481 XVs -> 533 XUVs
C:0 T:7 2 XUs and 1 XVs -> 0 XUVs
4688 XUVs total
1928 collection wire objects
4688 potential space points
Neighbour search...
206268 tests to find 98886 neighbours
Iterating with no regularization...
Begin: 1.15316e+07
0 9.73432e+06
1 9.5838e+06
2 9.57362e+06
3 9.57127e+06
Now with regularization...
Begin: 9.08935e+06
0 9.08796e+06
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
26-Sep-2024 12:53:39 CEST Closed output file "np04hd_raw_run029081_0610_dataflow5_datawriter_0_20240905T022622_reco_stage1_reco_stage2_20240926T105339_keepup.root"
26-Sep-2024 12:53:39 CEST Closed input file "np04hd_raw_run029081_0610_dataflow5_datawriter_0_20240905T022622_reco_stage1.root"
========================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
========================================================================================================================================
Full event 63.7612 89.7183 127.664 86.362 17.6872 30
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.000691161 0.0029325 0.00489939 0.00319105 0.00109712 30
produce:fembfilter:PDHDFEMBFilter 0.495321 0.56276 0.761558 0.534428 0.0721034 30
produce:wclsdatahd:WireCellToolkit 31.2957 34.76 40.914 34.261 2.43068 30
produce:gaushit:GausHitFinder 0.411478 0.592135 1.14203 0.527829 0.179281 30
produce:nhitsfilter:NumberOfHitsFilter 0.000227214 0.000346996 0.000691062 0.000327788 9.56643e-05 30
produce:reco3d:SpacePointSolver 0.224238 1.32378 5.86791 0.947941 1.23349 30
produce:hitpdune:DisambigFromSpacePoints 0.193892 0.494993 1.3668 0.40221 0.281805 30
produce:pandora:StandardPandora 4.87888 11.8774 25.7198 10.0005 5.39173 30
produce:pandoraWriter:StandardPandora 0.0778493 0.142143 0.238927 0.13706 0.0371308 30
produce:pandoraTrack:LArPandoraTrackCreation 0.705805 1.97946 4.30568 1.77456 0.907213 30
produce:pandoraShower:LArPandoraShowerCreation 0.684755 2.41495 5.8362 2.01042 1.34605 30
produce:pandoracalo:Calorimetry 0.282466 0.858796 1.72443 0.808033 0.380024 30
produce:pandoracalonosce:Calorimetry 0.334213 0.804842 1.46734 0.730762 0.331924 30
produce:pandorapid:Chi2ParticleID 0.00123914 0.001912 0.00402797 0.00180839 0.000586896 30
produce:pandoraShowercalo:ShowerCalorimetry 0.598115 2.05799 4.98323 1.80879 1.12643 30
produce:pandoraShowercalonosce:ShowerCalorimetry 0.474732 1.89555 4.61113 1.75228 1.04484 30
produce:pandoraGnocchiCalo:GnocchiCalorimetry 0.0631694 0.117581 0.248139 0.100728 0.0416676 30
produce:pandoraGnocchiCalonosce:GnocchiCalorimetry 0.0241553 0.0446268 0.079781 0.0407224 0.0125819 30
produce:emtrkmichelid:EmTrackMichelId 16.7648 28.1945 39.1515 27.8839 5.94319 30
[art]:TriggerResults:TriggerResultInserter 4.0025e-05 6.40777e-05 0.000101129 6.22315e-05 1.50555e-05 30
end_path:out1:RootOutput 8.596e-06 1.62731e-05 4.7388e-05 1.52935e-05 6.58745e-06 30
end_path:michelremoving:michelremoving 0.0208516 0.0597249 0.197935 0.0441121 0.0428892 30
end_path:michelremovinggnocchi:michelremoving 0.0191312 0.0580591 0.26328 0.0387676 0.0508489 30
end_path:calibanagnocchi:CalibAnaTree 0.227428 0.609318 1.33374 0.518251 0.269069 30
end_path:out1:RootOutput(write) 0.381692 0.644663 1.04511 0.613383 0.164904 30
========================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 4966.65 MB
Peak resident set size usage (VmHWM): 2387.55 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np04hd_raw_run029081_0610_dataflow5_datawriter_0_20240905T022622_reco_stage1_reco_stage2_20240926T105339_keepup.root
\tHists: np04hd_raw_run029081_0610_dataflow5_datawriter_0_20240905T022622_reco_stage1_reco_stage2_20240926T105339_keepup_hists.root
Forming reco metadata
/cvmfs/larsoft.opensciencegrid.org/products/python/v3_9_15/Linux64bit+3.10-2.17/bin/python: No module named meta_maker
Error in reco metadata
