Jobsub ID 263680.182@justin-prod-sched01.dune.hep.ac.uk
| Jobsub ID | 263680.182@justin-prod-sched01.dune.hep.ac.uk |
| Workflow ID | 3448 |
| Stage ID | 1 |
| User name | calcuttj@fnal.gov |
| HTCondor Group | group_dune.prod_mcsim |
| Requested | Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 18000 (5 hours) |
| Submitted time | 2024-09-26 13:07:13 |
| Site | ES_PIC |
| Entry | DUNE_T1_ES_PIC_ce14-multicore |
| Last heartbeat | 2024-09-26 14:30:13 |
| From worker node | Hostname | tds207.pic.es |
| cpuinfo | Intel(R) Xeon(R) CPU E5-2640 v3 @ 2.60GHz |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 216000 (60 hours) |
| Inner Apptainer? | True |
| Job state | jobscript_error |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk |
| Started | 2024-09-26 13:08:20 |
| Input files | hd-protodune:np04hd_raw_run029094_0012_dataflow5_datawriter_0_20240906T083825.hdf5
|
| Jobscript | Exit code | 1 |
| Real time | 0m (0s) |
| CPU time | 0m (0s = 0%) |
| Outputting started | |
| Output files | |
| Finished | 2024-09-26 14:30:13 |
| Saved logs | justin-logs:263680.182-justin-prod-sched01.dune.hep.ac.uk.logs.tgz |
| List job events Wrapper job log |
Jobscript log (last 10,000 characters)
n TPC 6
4806 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 220 XUs and 183 XVs -> 31 XUVs
C:0 T:2 5215 XUs and 8288 XVs -> 201 XUVs
C:0 T:3 83 XUs and 102 XVs -> 5 XUVs
C:0 T:4 32 XUs and 36 XVs -> 1 XUVs
C:0 T:5 23761 XUs and 21856 XVs -> 786 XUVs
C:0 T:6 13134 XUs and 18742 XVs -> 430 XUVs
C:0 T:7 377 XUs and 420 XVs -> 11 XUVs
1465 XUVs total
1047 collection wire objects
1465 potential space points
Neighbour search...
14539 tests to find 7844 neighbours
Iterating with no regularization...
Begin: 5.30393e+06
0 5.16958e+06
1 5.16406e+06
2 5.16369e+06
Now with regularization...
Begin: 4.7493e+06
0 4.74922e+06
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
no of hits closeby 0 event 3101 TrkackID 1 startx, y, z -76.107 486.56 231.832 wireno, peakt tpcno 353 633.469 6 dist 5.86235min T, max_T567 5005.02
no of hits closeby 0 event 3101 TrkackID 1 startx, y, z -76.107 486.56 231.832 wireno, peakt tpcno 353 633.469 6 dist 5.86235min T, max_T567 5005.02
Begin processing the 29th record. run: 29094 subRun: 1 event: 3109 at 26-Sep-2024 16:24:46 CEST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 67116 traces, 67116 summary
Retagger: tagging trace set: gauss with 50456 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 50456 traces tagged "gauss"
FrameSaver: q=1.26872e+06 n=946427 tag=gauss
wclsFrameSaver: saving 67116 traces tagged "wiener"
FrameSaver: q=1.38863e+06 n=858105 tag=wiener
2357 hits in TPC 0
2075 hits in TPC 1
5409 hits in TPC 2
3428 hits in TPC 3
2995 hits in TPC 4
5137 hits in TPC 5
10336 hits in TPC 6
6067 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:2 32272 XUs and 36559 XVs -> 1945 XUVs
C:0 T:3 1 XUs and 2 XVs -> 0 XUVs
C:0 T:4 49 XUs and 43 XVs -> 1 XUVs
C:0 T:5 15216 XUs and 12022 XVs -> 690 XUVs
C:0 T:6 183100 XUs and 132777 XVs -> 12566 XUVs
C:0 T:7 225 XUs and 207 XVs -> 11 XUVs
15213 XUVs total
2817 collection wire objects
15213 potential space points
Neighbour search...
2904505 tests to find 731500 neighbours
Iterating with no regularization...
Begin: 4.64988e+07
0 2.87022e+07
1 2.67639e+07
2 2.65653e+07
3 2.65175e+07
4 2.64995e+07
Now with regularization...
Begin: 2.49337e+07
0 2.4906e+07
1 2.48961e+07
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Begin processing the 30th record. run: 29094 subRun: 1 event: 3117 at 26-Sep-2024 16:26:40 CEST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 65321 traces, 65321 summary
Retagger: tagging trace set: gauss with 49660 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 49660 traces tagged "gauss"
FrameSaver: q=1.08902e+06 n=900448 tag=gauss
wclsFrameSaver: saving 65321 traces tagged "wiener"
FrameSaver: q=1.15631e+06 n=809102 tag=wiener
3114 hits in TPC 0
2898 hits in TPC 1
6743 hits in TPC 2
4812 hits in TPC 3
3212 hits in TPC 4
6241 hits in TPC 5
3236 hits in TPC 6
2192 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 3880 XUs and 6588 XVs -> 2209 XUVs
C:0 T:2 19093 XUs and 17795 XVs -> 597 XUVs
C:0 T:3 153 XUs and 130 XVs -> 2 XUVs
C:0 T:4 212 XUs and 355 XVs -> 15 XUVs
C:0 T:5 20683 XUs and 16627 XVs -> 370 XUVs
C:0 T:6 5481 XUs and 5775 XVs -> 144 XUVs
C:0 T:7 35 XUs and 50 XVs -> 3 XUVs
3340 XUVs total
941 collection wire objects
3340 potential space points
Neighbour search...
207102 tests to find 129632 neighbours
Iterating with no regularization...
Begin: 5.6327e+06
0 5.42065e+06
1 5.40422e+06
2 5.40253e+06
Now with regularization...
Begin: 5.15875e+06
0 5.15853e+06
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
26-Sep-2024 16:28:10 CEST Closed output file "np04hd_raw_run029094_0012_dataflow5_datawriter_0_20240906T083825_reco_stage1_reco_stage2_20240926T142810_keepup.root"
26-Sep-2024 16:28:10 CEST Closed input file "np04hd_raw_run029094_0012_dataflow5_datawriter_0_20240906T083825_reco_stage1.root"
========================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
========================================================================================================================================
Full event 71.609 91.8364 136.125 88.0971 15.5067 30
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.000273541 0.00127989 0.0019134 0.00137394 0.000496157 30
produce:fembfilter:PDHDFEMBFilter 0.605412 0.627538 0.664509 0.629526 0.011895 30
produce:wclsdatahd:WireCellToolkit 42.7647 45.7682 47.773 45.6611 1.19625 30
produce:gaushit:GausHitFinder 0.479147 0.74049 1.08547 0.716878 0.179709 30
produce:nhitsfilter:NumberOfHitsFilter 0.000173342 0.000274591 0.000415472 0.000257079 7.22411e-05 30
produce:reco3d:SpacePointSolver 0.155469 0.578201 2.70593 0.381095 0.495369 30
produce:hitpdune:DisambigFromSpacePoints 0.1946 0.524267 1.21295 0.4205 0.278681 30
produce:pandora:StandardPandora 5.79527 12.4443 27.8035 10.8332 5.62793 30
produce:pandoraWriter:StandardPandora 0.0837794 0.128363 0.229115 0.120186 0.0329954 30
produce:pandoraTrack:LArPandoraTrackCreation 0.516789 1.08133 2.29493 1.03062 0.42759 30
produce:pandoraShower:LArPandoraShowerCreation 0.472845 1.35464 3.45863 1.16188 0.672384 30
produce:pandoracalo:Calorimetry 0.273227 0.585241 1.27704 0.533385 0.219924 30
produce:pandoracalonosce:Calorimetry 0.238229 0.528504 1.17422 0.488173 0.201102 30
produce:pandorapid:Chi2ParticleID 0.00120565 0.00166634 0.00263505 0.0015639 0.000318928 30
produce:pandoraShowercalo:ShowerCalorimetry 0.419412 1.28321 3.70135 1.13753 0.670509 30
produce:pandoraShowercalonosce:ShowerCalorimetry 0.412108 1.25881 3.66017 1.08794 0.661646 30
produce:pandoraGnocchiCalo:GnocchiCalorimetry 0.0789714 0.131218 0.206997 0.123286 0.0312765 30
produce:pandoraGnocchiCalonosce:GnocchiCalorimetry 0.0268771 0.0441725 0.0727307 0.040812 0.0103439 30
produce:emtrkmichelid:EmTrackMichelId 15.6324 23.3765 39.8726 22.5812 5.63587 30
[art]:TriggerResults:TriggerResultInserter 1.8157e-05 2.20859e-05 6.0195e-05 2.05755e-05 7.40316e-06 30
end_path:out1:RootOutput 5.574e-06 9.01363e-06 2.0672e-05 8.376e-06 3.00452e-06 30
end_path:michelremoving:michelremoving 0.0184439 0.069333 0.264725 0.031956 0.0696385 30
end_path:michelremovinggnocchi:michelremoving 0.0173238 0.0673699 0.26985 0.0292148 0.070391 30
end_path:calibanagnocchi:CalibAnaTree 0.185901 0.451015 1.08964 0.415472 0.183904 30
end_path:out1:RootOutput(write) 0.46201 0.729969 1.11985 0.69133 0.16835 30
========================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 4941.16 MB
Peak resident set size usage (VmHWM): 2582.75 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np04hd_raw_run029094_0012_dataflow5_datawriter_0_20240906T083825_reco_stage1_reco_stage2_20240926T142810_keepup.root
\tHists: np04hd_raw_run029094_0012_dataflow5_datawriter_0_20240906T083825_reco_stage1_reco_stage2_20240926T142810_keepup_hists.root
Forming reco metadata
/cvmfs/larsoft.opensciencegrid.org/products/python/v3_9_15/Linux64bit+3.10-2.17/bin/python: No module named meta_maker
Error in reco metadata
