Jobsub ID 263295.140@justin-prod-sched01.dune.hep.ac.uk
| Jobsub ID | 263295.140@justin-prod-sched01.dune.hep.ac.uk |
| Workflow ID | 3453 |
| Stage ID | 1 |
| User name | calcuttj@fnal.gov |
| HTCondor Group | group_dune.prod_mcsim |
| Requested | Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 18000 (5 hours) |
| Submitted time | 2024-09-26 09:58:35 |
| Site | FR_CCIN2P3 |
| Entry | DUNE_FR_CCIN2P3_cccondorce01 |
| Last heartbeat | 2024-09-26 11:09:35 |
| From worker node | Hostname | ccwcondor0612 |
| cpuinfo | AMD EPYC 7453 28-Core Processor |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 106200 (29 hours) |
| Inner Apptainer? | True |
| Job state | jobscript_error |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk |
| Started | 2024-09-26 10:04:09 |
| Input files | hd-protodune:np04hd_raw_run029107_3563_dataflow7_datawriter_0_20240909T032551.hdf5
|
| Jobscript | Exit code | 1 |
| Real time | 0m (0s) |
| CPU time | 0m (0s = 0%) |
| Outputting started | |
| Output files | |
| Finished | 2024-09-26 11:09:35 |
| Saved logs | justin-logs:263295.140-justin-prod-sched01.dune.hep.ac.uk.logs.tgz |
| List job events Wrapper job log |
Jobscript log (last 10,000 characters)
eSaver: saving 57589 traces tagged "wiener"
FrameSaver: q=991875 n=702577 tag=wiener
4222 hits in TPC 0
2611 hits in TPC 1
4624 hits in TPC 2
2102 hits in TPC 3
3769 hits in TPC 4
5067 hits in TPC 5
5181 hits in TPC 6
2445 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 1913 XUs and 1658 XVs -> 101 XUVs
C:0 T:2 35556 XUs and 32531 XVs -> 604 XUVs
C:0 T:3 141 XUs and 165 XVs -> 6 XUVs
C:0 T:4 73 XUs and 86 XVs -> 8 XUVs
C:0 T:5 15295 XUs and 23422 XVs -> 825 XUVs
C:0 T:6 53851 XUs and 42774 XVs -> 985 XUVs
C:0 T:7 219 XUs and 412 XVs -> 7 XUVs
2536 XUVs total
1165 collection wire objects
2536 potential space points
Neighbour search...
82910 tests to find 40654 neighbours
Iterating with no regularization...
Begin: 7.9815e+06
0 7.5185e+06
1 7.49443e+06
2 7.49082e+06
Now with regularization...
Begin: 7.17771e+06
0 7.17626e+06
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Begin processing the 29th record. run: 29107 subRun: 1 event: 855367 at 26-Sep-2024 12:56:51 CEST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 67294 traces, 67294 summary
Retagger: tagging trace set: gauss with 49366 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 49366 traces tagged "gauss"
FrameSaver: q=1.11383e+06 n=872576 tag=gauss
wclsFrameSaver: saving 67294 traces tagged "wiener"
FrameSaver: q=1.23323e+06 n=791771 tag=wiener
4054 hits in TPC 0
3732 hits in TPC 1
1979 hits in TPC 2
1935 hits in TPC 3
5692 hits in TPC 4
5505 hits in TPC 5
6927 hits in TPC 6
4679 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 3538 XUs and 3510 XVs -> 344 XUVs
C:0 T:2 15246 XUs and 8367 XVs -> 477 XUVs
C:0 T:3 22 XUs and 9 XVs -> 0 XUVs
C:0 T:4 104 XUs and 237 XVs -> 14 XUVs
C:0 T:5 46004 XUs and 57969 XVs -> 4790 XUVs
C:0 T:6 33161 XUs and 20736 XVs -> 1511 XUVs
C:0 T:7 174 XUs and 142 XVs -> 8 XUVs
7144 XUVs total
1688 collection wire objects
7144 potential space points
Neighbour search...
1499166 tests to find 461158 neighbours
Iterating with no regularization...
Begin: 1.85648e+07
0 1.50366e+07
1 1.46579e+07
2 1.45976e+07
3 1.45816e+07
4 1.4575e+07
Now with regularization...
Begin: 1.37227e+07
0 1.37125e+07
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Begin processing the 30th record. run: 29107 subRun: 1 event: 855375 at 26-Sep-2024 12:58:07 CEST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 44159 traces, 44159 summary
Retagger: tagging trace set: gauss with 32976 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 32976 traces tagged "gauss"
FrameSaver: q=787828 n=662721 tag=gauss
wclsFrameSaver: saving 44159 traces tagged "wiener"
FrameSaver: q=833581 n=607445 tag=wiener
2203 hits in TPC 0
1252 hits in TPC 1
2832 hits in TPC 2
2183 hits in TPC 3
2586 hits in TPC 4
4158 hits in TPC 5
5910 hits in TPC 6
3805 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 20 XUs and 31 XVs -> 4 XUVs
C:0 T:2 6361 XUs and 4731 XVs -> 198 XUVs
C:0 T:3 265 XUs and 401 XVs -> 16 XUVs
C:0 T:4 28 XUs and 13 XVs -> 0 XUVs
C:0 T:5 21372 XUs and 23084 XVs -> 2749 XUVs
C:0 T:6 18049 XUs and 10468 XVs -> 552 XUVs
C:0 T:7 369 XUs and 258 XVs -> 8 XUVs
3527 XUVs total
930 collection wire objects
3527 potential space points
Neighbour search...
456323 tests to find 223586 neighbours
Iterating with no regularization...
Begin: 3.67325e+06
0 3.4551e+06
1 3.44072e+06
2 3.43874e+06
Now with regularization...
Begin: 3.13967e+06
0 3.13848e+06
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
26-Sep-2024 12:59:04 CEST Closed output file "np04hd_raw_run029107_3563_dataflow7_datawriter_0_20240909T032551_reco_stage1_reco_stage2_20240926T105904_keepup.root"
26-Sep-2024 12:59:04 CEST Closed input file "np04hd_raw_run029107_3563_dataflow7_datawriter_0_20240909T032551_reco_stage1.root"
========================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
========================================================================================================================================
Full event 54.4399 75.4159 104.385 74.0952 10.9203 30
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.000344054 0.00239628 0.0039122 0.00256809 0.00103896 30
produce:fembfilter:PDHDFEMBFilter 0.498598 0.529944 0.62466 0.512433 0.0366558 30
produce:wclsdatahd:WireCellToolkit 30.391 33.0911 39.8765 32.524 2.06578 30
produce:gaushit:GausHitFinder 0.280997 0.515219 0.952258 0.509756 0.12363 30
produce:nhitsfilter:NumberOfHitsFilter 0.000208981 0.000293431 0.000549037 0.000270035 6.82894e-05 30
produce:reco3d:SpacePointSolver 0.133212 0.869335 2.26041 0.67482 0.604049 30
produce:hitpdune:DisambigFromSpacePoints 0.128259 0.3545 1.45388 0.277502 0.231753 30
produce:pandora:StandardPandora 3.25444 8.41804 19.9348 8.14052 3.15008 30
produce:pandoraWriter:StandardPandora 0.070449 0.119901 0.186503 0.113555 0.0272476 30
produce:pandoraTrack:LArPandoraTrackCreation 0.491541 1.22535 2.32081 1.19533 0.425647 30
produce:pandoraShower:LArPandoraShowerCreation 0.412589 1.37517 2.56641 1.2356 0.58854 30
produce:pandoracalo:Calorimetry 0.246516 0.589963 0.980361 0.601269 0.18472 30
produce:pandoracalonosce:Calorimetry 0.231657 0.540028 1.03166 0.546782 0.179186 30
produce:pandorapid:Chi2ParticleID 0.00112829 0.00163921 0.0042758 0.00153308 0.000563351 30
produce:pandoraShowercalo:ShowerCalorimetry 0.356351 1.23178 2.10137 1.14622 0.46372 30
produce:pandoraShowercalonosce:ShowerCalorimetry 0.346701 1.1624 1.96538 1.06421 0.43428 30
produce:pandoraGnocchiCalo:GnocchiCalorimetry 0.0636748 0.0995563 0.130976 0.097804 0.017641 30
produce:pandoraGnocchiCalonosce:GnocchiCalorimetry 0.025 0.0392397 0.0601003 0.0388701 0.00743425 30
produce:emtrkmichelid:EmTrackMichelId 15.1227 23.9517 35.0964 23.0991 4.56249 30
[art]:TriggerResults:TriggerResultInserter 2.3484e-05 5.10138e-05 0.000114154 4.5896e-05 1.98359e-05 30
end_path:out1:RootOutput 7.454e-06 1.43773e-05 3.6609e-05 1.2829e-05 5.74955e-06 30
end_path:michelremoving:michelremoving 0.0184849 0.0721385 0.200379 0.0356098 0.0567043 30
end_path:michelremovinggnocchi:michelremoving 0.0171594 0.0686531 0.18411 0.0316625 0.0551784 30
end_path:calibanagnocchi:CalibAnaTree 0.165475 0.411829 0.854061 0.400462 0.131852 30
end_path:out1:RootOutput(write) 0.351365 0.572405 0.847229 0.560227 0.103146 30
========================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 4929.74 MB
Peak resident set size usage (VmHWM): 2342.13 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np04hd_raw_run029107_3563_dataflow7_datawriter_0_20240909T032551_reco_stage1_reco_stage2_20240926T105904_keepup.root
\tHists: np04hd_raw_run029107_3563_dataflow7_datawriter_0_20240909T032551_reco_stage1_reco_stage2_20240926T105904_keepup_hists.root
Forming reco metadata
/cvmfs/larsoft.opensciencegrid.org/products/python/v3_9_15/Linux64bit+3.10-2.17/bin/python: No module named meta_maker
Error in reco metadata
