Jobsub ID 263967.0@justin-prod-sched01.dune.hep.ac.uk
| Jobsub ID | 263967.0@justin-prod-sched01.dune.hep.ac.uk |
| Workflow ID | 3448 |
| Stage ID | 1 |
| User name | calcuttj@fnal.gov |
| HTCondor Group | group_dune.prod_mcsim |
| Requested | Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 18000 (5 hours) |
| Submitted time | 2024-09-26 14:32:17 |
| Site | ES_PIC |
| Entry | DUNE_T1_ES_PIC_ce14-multicore |
| Last heartbeat | 2024-09-26 15:53:16 |
| From worker node | Hostname | tds222.pic.es |
| cpuinfo | Intel(R) Xeon(R) CPU E5-2640 v3 @ 2.60GHz |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 216000 (60 hours) |
| Inner Apptainer? | True |
| Job state | jobscript_error |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk |
| Started | 2024-09-26 14:33:04 |
| Input files | hd-protodune:np04hd_raw_run029097_0011_dataflow3_datawriter_0_20240906T102934.hdf5
|
| Jobscript | Exit code | 1 |
| Real time | 0m (0s) |
| CPU time | 0m (0s = 0%) |
| Outputting started | |
| Output files | |
| Finished | 2024-09-26 15:53:16 |
| Saved logs | justin-logs:263967.0-justin-prod-sched01.dune.hep.ac.uk.logs.tgz |
| List job events Wrapper job log |
Jobscript log (last 10,000 characters)
in: 1.06911e+07
0 1.06865e+07
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Begin processing the 29th record. run: 29097 subRun: 1 event: 2867 at 26-Sep-2024 17:49:58 CEST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 67145 traces, 67145 summary
Retagger: tagging trace set: gauss with 49953 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 49953 traces tagged "gauss"
FrameSaver: q=1.02377e+06 n=915158 tag=gauss
wclsFrameSaver: saving 67145 traces tagged "wiener"
FrameSaver: q=1.10472e+06 n=821756 tag=wiener
5498 hits in TPC 0
4426 hits in TPC 1
5529 hits in TPC 2
2878 hits in TPC 3
2967 hits in TPC 4
4298 hits in TPC 5
6178 hits in TPC 6
2993 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 3690 XUs and 3648 XVs -> 204 XUVs
C:0 T:2 34931 XUs and 38328 XVs -> 1452 XUVs
C:0 T:3 66 XUs and 138 XVs -> 5 XUVs
C:0 T:4 28 XUs and 38 XVs -> 5 XUVs
C:0 T:5 19540 XUs and 16144 XVs -> 299 XUVs
C:0 T:6 30645 XUs and 30887 XVs -> 419 XUVs
C:0 T:7 458 XUs and 573 XVs -> 10 XUVs
2394 XUVs total
1082 collection wire objects
2394 potential space points
Neighbour search...
67926 tests to find 41162 neighbours
Iterating with no regularization...
Begin: 3.78551e+06
0 3.47338e+06
1 3.4509e+06
2 3.4464e+06
3 3.44508e+06
Now with regularization...
Begin: 3.23079e+06
0 3.23011e+06
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
no of hits closeby 0 event 2867 TrkackID 12 startx, y, z -14.6984 163.796 445.895 wireno, peakt tpcno 320 1257.66 6 dist 0min T, max_T1253 2486.36
no of hits closeby 0 event 2867 TrkackID 16 startx, y, z 79.198 360.138 10.4709 wireno, peakt tpcno 109 2974.44 2 dist 58.1696min T, max_T2012.33 3005.18
no of hits closeby 0 event 2867 TrkackID 12 startx, y, z -14.6984 163.796 445.895 wireno, peakt tpcno 320 1257.66 6 dist 0min T, max_T1253 2486.36
no of hits closeby 0 event 2867 TrkackID 16 startx, y, z 79.198 360.138 10.4709 wireno, peakt tpcno 109 2974.44 2 dist 58.1696min T, max_T2012.33 3005.18
Begin processing the 30th record. run: 29097 subRun: 1 event: 2883 at 26-Sep-2024 17:51:26 CEST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 57055 traces, 57055 summary
Retagger: tagging trace set: gauss with 42926 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 42926 traces tagged "gauss"
FrameSaver: q=908229 n=831200 tag=gauss
wclsFrameSaver: saving 57055 traces tagged "wiener"
FrameSaver: q=979857 n=761942 tag=wiener
2998 hits in TPC 0
3652 hits in TPC 1
3267 hits in TPC 2
779 hits in TPC 3
2752 hits in TPC 4
6032 hits in TPC 5
7594 hits in TPC 6
4426 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 386 XUs and 330 XVs -> 5 XUVs
C:0 T:2 9393 XUs and 5234 XVs -> 209 XUVs
C:0 T:3 103 XUs and 147 XVs -> 8 XUVs
C:0 T:4 19 XUs and 24 XVs -> 0 XUVs
C:0 T:5 31259 XUs and 49913 XVs -> 761 XUVs
C:0 T:6 41095 XUs and 24815 XVs -> 576 XUVs
C:0 T:7 782 XUs and 1114 XVs -> 37 XUVs
1596 XUVs total
854 collection wire objects
1596 potential space points
Neighbour search...
62424 tests to find 30996 neighbours
Iterating with no regularization...
Begin: 4.88602e+06
0 4.72868e+06
1 4.71925e+06
2 4.71868e+06
Now with regularization...
Begin: 4.39801e+06
0 4.39777e+06
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
26-Sep-2024 17:52:52 CEST Closed output file "np04hd_raw_run029097_0011_dataflow3_datawriter_0_20240906T102934_reco_stage1_reco_stage2_20240926T155252_keepup.root"
26-Sep-2024 17:52:52 CEST Closed input file "np04hd_raw_run029097_0011_dataflow3_datawriter_0_20240906T102934_reco_stage1.root"
========================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
========================================================================================================================================
Full event 63.2258 97.2377 135.106 95.3717 17.1224 30
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.000356738 0.00191968 0.00669142 0.00189208 0.00119264 30
produce:fembfilter:PDHDFEMBFilter 0.527731 0.617627 0.659743 0.626043 0.0285258 30
produce:wclsdatahd:WireCellToolkit 38.1555 43.6169 46.3946 43.8787 1.63491 30
produce:gaushit:GausHitFinder 0.378787 0.853837 1.21932 0.87006 0.188316 30
produce:nhitsfilter:NumberOfHitsFilter 0.00016667 0.000354017 0.000539125 0.000373049 9.36867e-05 30
produce:reco3d:SpacePointSolver 0.203839 1.28968 4.46438 0.812308 1.11627 30
produce:hitpdune:DisambigFromSpacePoints 0.186267 0.717687 1.38637 0.638362 0.348222 30
produce:pandora:StandardPandora 3.80953 15.3827 35.9656 14.7254 6.33055 30
produce:pandoraWriter:StandardPandora 0.0905629 0.156137 0.241541 0.15387 0.0369115 30
produce:pandoraTrack:LArPandoraTrackCreation 0.413417 1.36676 2.75433 1.32881 0.541772 30
produce:pandoraShower:LArPandoraShowerCreation 0.364831 1.60856 3.57541 1.38806 0.792766 30
produce:pandoracalo:Calorimetry 0.215418 0.654377 1.31249 0.637497 0.222105 30
produce:pandoracalonosce:Calorimetry 0.176949 0.606319 1.16007 0.552164 0.222426 30
produce:pandorapid:Chi2ParticleID 0.000998258 0.00181974 0.0027835 0.00176729 0.000355808 30
produce:pandoraShowercalo:ShowerCalorimetry 0.284421 1.40742 3.9146 1.25702 0.696994 30
produce:pandoraShowercalonosce:ShowerCalorimetry 0.269319 1.39037 3.8406 1.26336 0.691619 30
produce:pandoraGnocchiCalo:GnocchiCalorimetry 0.0822501 0.142812 0.215416 0.144205 0.0318472 30
produce:pandoraGnocchiCalonosce:GnocchiCalorimetry 0.0257603 0.0483741 0.0725465 0.0489404 0.0108408 30
produce:emtrkmichelid:EmTrackMichelId 13.1434 25.7693 39.8039 24.8732 5.54732 30
[art]:TriggerResults:TriggerResultInserter 1.6843e-05 2.52963e-05 6.694e-05 2.168e-05 1.09017e-05 30
end_path:out1:RootOutput 5.979e-06 8.2801e-06 2.1694e-05 7.609e-06 2.82145e-06 30
end_path:michelremoving:michelremoving 0.0158121 0.100897 0.429331 0.0374432 0.116417 30
end_path:michelremovinggnocchi:michelremoving 0.0147392 0.0969523 0.416843 0.0346803 0.115398 30
end_path:calibanagnocchi:CalibAnaTree 0.174081 0.517852 1.01852 0.488717 0.180464 30
end_path:out1:RootOutput(write) 0.415529 0.815842 1.16608 0.79667 0.165857 30
========================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 4948.73 MB
Peak resident set size usage (VmHWM): 2598.67 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np04hd_raw_run029097_0011_dataflow3_datawriter_0_20240906T102934_reco_stage1_reco_stage2_20240926T155252_keepup.root
\tHists: np04hd_raw_run029097_0011_dataflow3_datawriter_0_20240906T102934_reco_stage1_reco_stage2_20240926T155252_keepup_hists.root
Forming reco metadata
/cvmfs/larsoft.opensciencegrid.org/products/python/v3_9_15/Linux64bit+3.10-2.17/bin/python: No module named meta_maker
Error in reco metadata
