Jobsub ID 78460.43@justin-prod-sched02.dune.hep.ac.uk
| Jobsub ID | 78460.43@justin-prod-sched02.dune.hep.ac.uk |
| Workflow ID | 3447 |
| Stage ID | 1 |
| User name | calcuttj@fnal.gov |
| HTCondor Group | group_dune.prod_mcsim |
| Requested | Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 18000 (5 hours) |
| Submitted time | 2024-09-26 09:58:17 |
| Site | FR_CCIN2P3 |
| Entry | DUNE_FR_CCIN2P3_cccondorce01 |
| Last heartbeat | 2024-09-26 11:16:24 |
| From worker node | Hostname | ccwcondor0748 |
| cpuinfo | AMD EPYC 7302 16-Core Processor |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 106200 (29 hours) |
| Inner Apptainer? | True |
| Job state | jobscript_error |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk |
| Started | 2024-09-26 10:10:51 |
| Input files | hd-protodune:np04hd_raw_run029092_0450_dataflow5_datawriter_0_20240906T002937.hdf5
|
| Jobscript | Exit code | 1 |
| Real time | 0m (0s) |
| CPU time | 0m (0s = 0%) |
| Outputting started | |
| Output files | |
| Finished | 2024-09-26 11:16:24 |
| Saved logs | justin-logs:78460.43-justin-prod-sched02.dune.hep.ac.uk.logs.tgz |
| List job events Wrapper job log |
Jobscript log (last 10,000 characters)
ag=gauss
wclsFrameSaver: saving 60288 traces tagged "wiener"
FrameSaver: q=1.03041e+06 n=753816 tag=wiener
3033 hits in TPC 0
2318 hits in TPC 1
5807 hits in TPC 2
3512 hits in TPC 3
2203 hits in TPC 4
5593 hits in TPC 5
7163 hits in TPC 6
3014 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:0 8 XUs and 3 XVs -> 0 XUVs
C:0 T:1 946 XUs and 1825 XVs -> 151 XUVs
C:0 T:2 13600 XUs and 20370 XVs -> 512 XUVs
C:0 T:3 3005 XUs and 2661 XVs -> 275 XUVs
C:0 T:4 250 XUs and 505 XVs -> 12 XUVs
C:0 T:5 9478 XUs and 9748 XVs -> 727 XUVs
C:0 T:6 21639 XUs and 41125 XVs -> 580 XUVs
C:0 T:7 509 XUs and 656 XVs -> 22 XUVs
2279 XUVs total
1365 collection wire objects
2279 potential space points
Neighbour search...
45525 tests to find 25512 neighbours
Iterating with no regularization...
Begin: 5.77335e+06
0 5.45113e+06
1 5.43587e+06
2 5.43408e+06
Now with regularization...
Begin: 5.04329e+06
0 5.04275e+06
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Begin processing the 29th record. run: 29092 subRun: 1 event: 108229 at 26-Sep-2024 13:12:42 CEST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 58109 traces, 58109 summary
Retagger: tagging trace set: gauss with 41222 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 41222 traces tagged "gauss"
FrameSaver: q=865199 n=704860 tag=gauss
wclsFrameSaver: saving 58109 traces tagged "wiener"
FrameSaver: q=946441 n=633207 tag=wiener
3379 hits in TPC 0
3128 hits in TPC 1
6409 hits in TPC 2
3946 hits in TPC 3
2795 hits in TPC 4
3615 hits in TPC 5
4065 hits in TPC 6
1286 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:0 175 XUs and 27 XVs -> 9 XUVs
C:0 T:1 2957 XUs and 1734 XVs -> 38 XUVs
C:0 T:2 34235 XUs and 27955 XVs -> 1164 XUVs
C:0 T:3 1704 XUs and 1031 XVs -> 17 XUVs
C:0 T:4 81 XUs and 53 XVs -> 1 XUVs
C:0 T:5 30307 XUs and 18554 XVs -> 601 XUVs
C:0 T:6 10753 XUs and 16506 XVs -> 1049 XUVs
C:0 T:7 161 XUs and 165 XVs -> 1 XUVs
2880 XUVs total
1176 collection wire objects
2880 potential space points
Neighbour search...
134998 tests to find 75032 neighbours
Iterating with no regularization...
Begin: 8.83576e+06
0 7.81007e+06
1 7.72621e+06
2 7.71923e+06
Now with regularization...
Begin: 7.31378e+06
0 7.31166e+06
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Begin processing the 30th record. run: 29092 subRun: 1 event: 108237 at 26-Sep-2024 13:13:48 CEST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 68930 traces, 68930 summary
Retagger: tagging trace set: gauss with 50899 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 50899 traces tagged "gauss"
FrameSaver: q=1.03764e+06 n=927527 tag=gauss
wclsFrameSaver: saving 68930 traces tagged "wiener"
FrameSaver: q=1.12278e+06 n=830995 tag=wiener
2765 hits in TPC 0
4006 hits in TPC 1
9983 hits in TPC 2
4872 hits in TPC 3
1816 hits in TPC 4
5041 hits in TPC 5
6148 hits in TPC 6
1888 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 712 XUs and 772 XVs -> 65 XUVs
C:0 T:2 39462 XUs and 60322 XVs -> 1097 XUVs
C:0 T:3 738 XUs and 898 XVs -> 33 XUVs
C:0 T:4 18 XUs and 17 XVs -> 0 XUVs
C:0 T:5 15203 XUs and 38631 XVs -> 753 XUVs
C:0 T:6 16315 XUs and 35524 XVs -> 865 XUVs
C:0 T:7 135 XUs and 147 XVs -> 5 XUVs
2818 XUVs total
1332 collection wire objects
2818 potential space points
Neighbour search...
86826 tests to find 52706 neighbours
Iterating with no regularization...
Begin: 8.06414e+06
0 7.53502e+06
1 7.49437e+06
2 7.48798e+06
Now with regularization...
Begin: 7.12217e+06
0 7.11978e+06
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
26-Sep-2024 13:15:05 CEST Closed output file "np04hd_raw_run029092_0450_dataflow5_datawriter_0_20240906T002937_reco_stage1_reco_stage2_20240926T111505_keepup.root"
26-Sep-2024 13:15:05 CEST Closed input file "np04hd_raw_run029092_0450_dataflow5_datawriter_0_20240906T002937_reco_stage1.root"
========================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
========================================================================================================================================
Full event 55.0939 87.2663 122.946 84.0452 17.7466 30
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.000382248 0.00221334 0.00565434 0.00235918 0.001326 30
produce:fembfilter:PDHDFEMBFilter 0.535005 0.644075 0.864245 0.600166 0.09873 30
produce:wclsdatahd:WireCellToolkit 34.663 40.6821 50.0855 38.6069 5.43253 30
produce:gaushit:GausHitFinder 0.332576 0.597087 0.989442 0.529116 0.176735 30
produce:nhitsfilter:NumberOfHitsFilter 0.000198854 0.00039034 0.000684406 0.000366208 0.00011794 30
produce:reco3d:SpacePointSolver 0.112949 0.863622 3.60488 0.773277 0.709486 30
produce:hitpdune:DisambigFromSpacePoints 0.148637 0.428022 0.915656 0.360477 0.210436 30
produce:pandora:StandardPandora 3.40461 10.8332 27.8864 9.31593 4.94398 30
produce:pandoraWriter:StandardPandora 0.0568447 0.133537 0.212383 0.116107 0.0458399 30
produce:pandoraTrack:LArPandoraTrackCreation 0.369299 1.01761 2.14914 0.939492 0.423684 30
produce:pandoraShower:LArPandoraShowerCreation 0.319623 1.1515 2.26447 1.09232 0.538226 30
produce:pandoracalo:Calorimetry 0.205342 0.525423 0.925416 0.504558 0.18647 30
produce:pandoracalonosce:Calorimetry 0.178726 0.484553 1.05788 0.438378 0.203486 30
produce:pandorapid:Chi2ParticleID 0.000882227 0.00188261 0.00635775 0.00160338 0.00104223 30
produce:pandoraShowercalo:ShowerCalorimetry 0.290312 1.06779 2.04954 1.0789 0.465588 30
produce:pandoraShowercalonosce:ShowerCalorimetry 0.26151 1.02622 2.07189 0.938788 0.481588 30
produce:pandoraGnocchiCalo:GnocchiCalorimetry 0.061515 0.134836 0.248807 0.119481 0.0465516 30
produce:pandoraGnocchiCalonosce:GnocchiCalorimetry 0.0202884 0.0436491 0.0866123 0.0384179 0.015527 30
produce:emtrkmichelid:EmTrackMichelId 13.0389 26.2949 40.2045 24.2007 6.75396 30
[art]:TriggerResults:TriggerResultInserter 1.6291e-05 3.81065e-05 8.8867e-05 3.4019e-05 1.96945e-05 30
end_path:out1:RootOutput 7.364e-06 1.0774e-05 2.673e-05 9.563e-06 3.86912e-06 30
end_path:michelremoving:michelremoving 0.0128851 0.0619618 0.191659 0.0308102 0.0557483 30
end_path:michelremovinggnocchi:michelremoving 0.0117385 0.0571825 0.192256 0.0254118 0.0559801 30
end_path:calibanagnocchi:CalibAnaTree 0.16194 0.370209 0.674565 0.340448 0.131968 30
end_path:out1:RootOutput(write) 0.394011 0.722358 1.09276 0.694184 0.187521 30
========================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 4910.95 MB
Peak resident set size usage (VmHWM): 2332.64 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np04hd_raw_run029092_0450_dataflow5_datawriter_0_20240906T002937_reco_stage1_reco_stage2_20240926T111505_keepup.root
\tHists: np04hd_raw_run029092_0450_dataflow5_datawriter_0_20240906T002937_reco_stage1_reco_stage2_20240926T111505_keepup_hists.root
Forming reco metadata
/cvmfs/larsoft.opensciencegrid.org/products/python/v3_9_15/Linux64bit+3.10-2.17/bin/python: No module named meta_maker
Error in reco metadata
