Jobsub ID 263295.2@justin-prod-sched01.dune.hep.ac.uk
| Jobsub ID | 263295.2@justin-prod-sched01.dune.hep.ac.uk |
| Workflow ID | 3453 |
| Stage ID | 1 |
| User name | calcuttj@fnal.gov |
| HTCondor Group | group_dune.prod_mcsim |
| Requested | Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 18000 (5 hours) |
| Submitted time | 2024-09-26 09:58:35 |
| Site | FR_CCIN2P3 |
| Entry | DUNE_FR_CCIN2P3_cccondorce03 |
| Last heartbeat | 2024-09-26 11:12:50 |
| From worker node | Hostname | ccwcondor0723.in2p3.fr |
| cpuinfo | AMD EPYC 7302 16-Core Processor |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 106200 (29 hours) |
| Inner Apptainer? | True |
| Job state | jobscript_error |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk |
| Started | 2024-09-26 10:03:07 |
| Input files | hd-protodune:np04hd_raw_run029107_3652_dataflow5_datawriter_0_20240909T044315.hdf5
|
| Jobscript | Exit code | 1 |
| Real time | 0m (0s) |
| CPU time | 0m (0s = 0%) |
| Outputting started | |
| Output files | |
| Finished | 2024-09-26 11:12:50 |
| Saved logs | justin-logs:263295.2-justin-prod-sched01.dune.hep.ac.uk.logs.tgz |
| List job events Wrapper job log |
Jobscript log (last 10,000 characters)
al space points
Neighbour search...
2496 tests to find 1388 neighbours
Iterating with no regularization...
Begin: 1.09389e+06
0 1.07005e+06
1 1.06903e+06
Now with regularization...
Begin: 1.00463e+06
0 1.00458e+06
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
no of hits closeby 0 event 876725 TrkackID 1 startx, y, z 202.59 603.776 424.227 wireno, peakt tpcno 211 5.67218 5 dist 0min T, max_T5.67218 2569.61
no of hits closeby 0 event 876725 TrkackID 1 startx, y, z 202.59 603.776 424.227 wireno, peakt tpcno 211 5.67218 5 dist 0min T, max_T5.67218 2569.61
Begin processing the 29th record. run: 29107 subRun: 1 event: 876733 at 26-Sep-2024 13:01:05 CEST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 45738 traces, 45738 summary
Retagger: tagging trace set: gauss with 33940 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 33940 traces tagged "gauss"
FrameSaver: q=680397 n=621869 tag=gauss
wclsFrameSaver: saving 45738 traces tagged "wiener"
FrameSaver: q=732320 n=561133 tag=wiener
2115 hits in TPC 0
3850 hits in TPC 1
6964 hits in TPC 2
3350 hits in TPC 3
1203 hits in TPC 4
2440 hits in TPC 5
3224 hits in TPC 6
1224 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 683 XUs and 1360 XVs -> 35 XUVs
C:0 T:2 33125 XUs and 58551 XVs -> 2639 XUVs
C:0 T:3 73 XUs and 64 XVs -> 3 XUVs
C:0 T:4 18 XUs and 29 XVs -> 7 XUVs
C:0 T:5 4109 XUs and 6011 XVs -> 117 XUVs
C:0 T:6 17005 XUs and 27899 XVs -> 1429 XUVs
C:0 T:7 6 XUs and 7 XVs -> 0 XUVs
4230 XUVs total
1039 collection wire objects
4230 potential space points
Neighbour search...
394752 tests to find 222832 neighbours
Iterating with no regularization...
Begin: 6.48949e+06
0 5.52975e+06
1 5.48743e+06
2 5.47993e+06
3 5.47758e+06
Now with regularization...
Begin: 5.18702e+06
0 5.18548e+06
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
no of hits closeby 0 event 876733 TrkackID 1 startx, y, z 187.229 195.109 460.779 wireno, peakt tpcno 62 3927.46 5 dist 0min T, max_T3081 5446.34
no of hits closeby 0 event 876733 TrkackID 1 startx, y, z 187.229 195.109 460.779 wireno, peakt tpcno 62 3927.46 5 dist 0min T, max_T3081 5446.34
Begin processing the 30th record. run: 29107 subRun: 1 event: 876741 at 26-Sep-2024 13:02:08 CEST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 78734 traces, 78734 summary
Retagger: tagging trace set: gauss with 58078 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 58078 traces tagged "gauss"
FrameSaver: q=1.22878e+06 n=1086370 tag=gauss
wclsFrameSaver: saving 78734 traces tagged "wiener"
FrameSaver: q=1.31276e+06 n=976224 tag=wiener
4166 hits in TPC 0
3973 hits in TPC 1
7983 hits in TPC 2
4046 hits in TPC 3
6004 hits in TPC 4
9675 hits in TPC 5
4830 hits in TPC 6
1624 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 24 XUs and 19 XVs -> 1 XUVs
C:0 T:2 31064 XUs and 60930 XVs -> 1264 XUVs
C:0 T:3 2952 XUs and 6769 XVs -> 395 XUVs
C:0 T:4 573 XUs and 853 XVs -> 57 XUVs
C:0 T:5 34829 XUs and 32841 XVs -> 645 XUVs
C:0 T:6 9758 XUs and 36691 XVs -> 954 XUVs
C:0 T:7 93 XUs and 143 XVs -> 3 XUVs
3319 XUVs total
1388 collection wire objects
3319 potential space points
Neighbour search...
167359 tests to find 97282 neighbours
Iterating with no regularization...
Begin: 6.49387e+06
0 5.91721e+06
1 5.87746e+06
2 5.87101e+06
3 5.86872e+06
Now with regularization...
Begin: 5.57267e+06
0 5.5711e+06
BdtBeamParticleIdTool::SliceFeatures::GetLArTPCIntercepts - inconsistent number of intercepts between a line and the LArTPC
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
26-Sep-2024 13:03:37 CEST Closed output file "np04hd_raw_run029107_3652_dataflow5_datawriter_0_20240909T044315_reco_stage1_reco_stage2_20240926T110337_keepup.root"
26-Sep-2024 13:03:37 CEST Closed input file "np04hd_raw_run029107_3652_dataflow5_datawriter_0_20240909T044315_reco_stage1.root"
========================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
========================================================================================================================================
Full event 53.1774 85.4351 110.749 87.4726 15.4292 30
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.000552102 0.00273057 0.00678858 0.00244249 0.00130545 30
produce:fembfilter:PDHDFEMBFilter 0.573954 0.663528 0.87624 0.654587 0.0636278 30
produce:wclsdatahd:WireCellToolkit 36.4425 40.4984 45.617 40.5224 2.45292 30
produce:gaushit:GausHitFinder 0.233177 0.577442 0.949741 0.607647 0.172526 30
produce:nhitsfilter:NumberOfHitsFilter 0.000167606 0.000384156 0.000653463 0.000359914 0.000112569 30
produce:reco3d:SpacePointSolver 0.0508797 0.784073 3.61586 0.59773 0.752417 30
produce:hitpdune:DisambigFromSpacePoints 0.0590825 0.387117 0.761966 0.397283 0.173197 30
produce:pandora:StandardPandora 2.31226 10.065 19.1405 9.89848 4.51973 30
produce:pandoraWriter:StandardPandora 0.0543545 0.131313 0.226785 0.135253 0.041428 30
produce:pandoraTrack:LArPandoraTrackCreation 0.279581 0.895294 1.59426 0.859408 0.34604 30
produce:pandoraShower:LArPandoraShowerCreation 0.231456 1.12929 2.49583 1.04602 0.564833 30
produce:pandoracalo:Calorimetry 0.17007 0.519332 0.95312 0.500657 0.188748 30
produce:pandoracalonosce:Calorimetry 0.125443 0.446304 0.899472 0.456561 0.181693 30
produce:pandorapid:Chi2ParticleID 0.000959961 0.00166506 0.00269398 0.00166046 0.000413187 30
produce:pandoraShowercalo:ShowerCalorimetry 0.174102 1.00858 2.09017 1.00868 0.508184 30
produce:pandoraShowercalonosce:ShowerCalorimetry 0.141777 0.96864 2.12503 0.963602 0.488006 30
produce:pandoraGnocchiCalo:GnocchiCalorimetry 0.0516183 0.121643 0.234862 0.113155 0.0422165 30
produce:pandoraGnocchiCalonosce:GnocchiCalorimetry 0.0132922 0.0418012 0.069253 0.0378945 0.0147542 30
produce:emtrkmichelid:EmTrackMichelId 11.2537 25.8404 40.2514 26.6931 7.36726 30
[art]:TriggerResults:TriggerResultInserter 1.7593e-05 4.35388e-05 0.000109366 3.19055e-05 2.36715e-05 30
end_path:out1:RootOutput 6.262e-06 1.03114e-05 4.1057e-05 8.1355e-06 6.77483e-06 30
end_path:michelremoving:michelremoving 0.0156465 0.0774993 0.386388 0.0321493 0.100036 30
end_path:michelremovinggnocchi:michelremoving 0.0141805 0.0702787 0.375234 0.030069 0.0900673 30
end_path:calibanagnocchi:CalibAnaTree 0.125225 0.360249 0.632581 0.361268 0.132845 30
end_path:out1:RootOutput(write) 0.306127 0.719867 1.15473 0.747376 0.206849 30
========================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 4896.35 MB
Peak resident set size usage (VmHWM): 2321.98 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np04hd_raw_run029107_3652_dataflow5_datawriter_0_20240909T044315_reco_stage1_reco_stage2_20240926T110337_keepup.root
\tHists: np04hd_raw_run029107_3652_dataflow5_datawriter_0_20240909T044315_reco_stage1_reco_stage2_20240926T110337_keepup_hists.root
Forming reco metadata
/cvmfs/larsoft.opensciencegrid.org/products/python/v3_9_15/Linux64bit+3.10-2.17/bin/python: No module named meta_maker
Error in reco metadata
