Jobsub ID 263763.79@justin-prod-sched01.dune.hep.ac.uk
| Jobsub ID | 263763.79@justin-prod-sched01.dune.hep.ac.uk |
| Workflow ID | 3448 |
| Stage ID | 1 |
| User name | calcuttj@fnal.gov |
| HTCondor Group | group_dune.prod_mcsim |
| Requested | Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 18000 (5 hours) |
| Submitted time | 2024-09-26 13:24:49 |
| Site | UK_Durham |
| Entry | DUNE_UK_SGridDurham_ce3 |
| Last heartbeat | 2024-09-26 15:05:01 |
| From worker node | Hostname | n231.dur.scotgrid.ac.uk |
| cpuinfo | Intel(R) Xeon(R) Gold 5220 CPU @ 2.20GHz |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 171000 (47 hours) |
| Inner Apptainer? | True |
| Job state | jobscript_error |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk |
| Started | 2024-09-26 13:25:55 |
| Input files | hd-protodune:np04hd_raw_run029094_0041_dataflow5_datawriter_0_20240906T085251.hdf5
|
| Jobscript | Exit code | 1 |
| Real time | 0m (0s) |
| CPU time | 0m (0s = 0%) |
| Outputting started | |
| Output files | |
| Finished | 2024-09-26 15:05:01 |
| Saved logs | justin-logs:263763.79-justin-prod-sched01.dune.hep.ac.uk.logs.tgz |
| List job events Wrapper job log |
Jobscript log (last 10,000 characters)
:SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
no of hits closeby 0 event 10061 TrkackID 5 startx, y, z 117.664 510.498 258.907 wireno, peakt tpcno 107 1191.79 5 dist 0min T, max_T4.65031 1509.89
no of hits closeby 0 event 10061 TrkackID 5 startx, y, z 117.664 510.498 258.907 wireno, peakt tpcno 107 1191.79 5 dist 0min T, max_T4.65031 1509.89
Begin processing the 29th record. run: 29094 subRun: 1 event: 10069 at 26-Sep-2024 16:00:00 BST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 95007 traces, 95007 summary
Retagger: tagging trace set: gauss with 71318 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 71318 traces tagged "gauss"
FrameSaver: q=1.52606e+06 n=1348707 tag=gauss
wclsFrameSaver: saving 95007 traces tagged "wiener"
FrameSaver: q=1.61819e+06 n=1210827 tag=wiener
5869 hits in TPC 0
5374 hits in TPC 1
8915 hits in TPC 2
5750 hits in TPC 3
6565 hits in TPC 4
9410 hits in TPC 5
5080 hits in TPC 6
3203 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 580 XUs and 672 XVs -> 52 XUVs
C:0 T:2 47750 XUs and 35797 XVs -> 659 XUVs
C:0 T:3 229 XUs and 246 XVs -> 3 XUVs
C:0 T:4 1028 XUs and 1321 XVs -> 28 XUVs
C:0 T:5 47002 XUs and 43251 XVs -> 636 XUVs
C:0 T:6 11109 XUs and 12672 XVs -> 356 XUVs
C:0 T:7 219 XUs and 414 XVs -> 8 XUVs
1742 XUVs total
1291 collection wire objects
1742 potential space points
Neighbour search...
13440 tests to find 7822 neighbours
Iterating with no regularization...
Begin: 6.00304e+06
0 5.73032e+06
1 5.72047e+06
2 5.71979e+06
Now with regularization...
Begin: 5.44642e+06
0 5.44619e+06
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
no of hits closeby 0 event 10069 TrkackID 1 startx, y, z -55.9507 299.192 441.087 wireno, peakt tpcno 38 1491.64 2 dist 0min T, max_T1449.39 2171.27
no of hits closeby 0 event 10069 TrkackID 23 startx, y, z -72.4411 128.801 352.002 wireno, peakt tpcno 59 504.826 6 dist 0min T, max_T5 932.5
no of hits closeby 0 event 10069 TrkackID 1 startx, y, z -55.9507 299.192 441.087 wireno, peakt tpcno 38 1491.64 2 dist 0min T, max_T1449.39 2171.27
no of hits closeby 0 event 10069 TrkackID 23 startx, y, z -72.4411 128.801 352.002 wireno, peakt tpcno 59 504.826 6 dist 0min T, max_T5 932.5
Begin processing the 30th record. run: 29094 subRun: 1 event: 10077 at 26-Sep-2024 16:02:48 BST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 54815 traces, 54815 summary
Retagger: tagging trace set: gauss with 40054 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 40054 traces tagged "gauss"
FrameSaver: q=883981 n=702352 tag=gauss
wclsFrameSaver: saving 54815 traces tagged "wiener"
FrameSaver: q=975184 n=643648 tag=wiener
2594 hits in TPC 0
3216 hits in TPC 1
5585 hits in TPC 2
4317 hits in TPC 3
2536 hits in TPC 4
3077 hits in TPC 5
3774 hits in TPC 6
2056 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 863 XUs and 669 XVs -> 25 XUVs
C:0 T:2 18696 XUs and 23893 XVs -> 2260 XUVs
C:0 T:3 31 XUs and 36 XVs -> 0 XUVs
C:0 T:4 190 XUs and 251 XVs -> 21 XUVs
C:0 T:5 9960 XUs and 10200 XVs -> 808 XUVs
C:0 T:6 6519 XUs and 4683 XVs -> 158 XUVs
C:0 T:7 144 XUs and 78 XVs -> 3 XUVs
3275 XUVs total
1037 collection wire objects
3275 potential space points
Neighbour search...
111613 tests to find 65030 neighbours
Iterating with no regularization...
Begin: 5.98905e+06
0 5.31383e+06
1 5.2312e+06
2 5.21514e+06
3 5.21017e+06
Now with regularization...
Begin: 4.69843e+06
0 4.69531e+06
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
26-Sep-2024 16:04:31 BST Closed output file "np04hd_raw_run029094_0041_dataflow5_datawriter_0_20240906T085251_reco_stage1_reco_stage2_20240926T150431_keepup.root"
26-Sep-2024 16:04:31 BST Closed input file "np04hd_raw_run029094_0041_dataflow5_datawriter_0_20240906T085251_reco_stage1.root"
========================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
========================================================================================================================================
Full event 70.8361 101.214 167.781 103.291 22.5806 30
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.000393253 0.00126381 0.00368119 0.000937608 0.000845379 30
produce:fembfilter:PDHDFEMBFilter 0.700175 0.892484 1.12786 0.841349 0.177511 30
produce:wclsdatahd:WireCellToolkit 43.7994 54.8853 71.0416 51.6669 8.86435 30
produce:gaushit:GausHitFinder 0.490629 0.931266 1.69759 0.919297 0.288366 30
produce:nhitsfilter:NumberOfHitsFilter 0.000251511 0.000408324 0.000720662 0.000404425 0.000112849 30
produce:reco3d:SpacePointSolver 0.166809 0.770658 2.41943 0.646585 0.516814 30
produce:hitpdune:DisambigFromSpacePoints 0.212701 0.738461 1.54221 0.645375 0.393661 30
produce:pandora:StandardPandora 5.66277 13.6189 32.3153 12.1422 6.60906 30
produce:pandoraWriter:StandardPandora 0.0951169 0.174007 0.360238 0.163688 0.0607399 30
produce:pandoraTrack:LArPandoraTrackCreation 0.632939 1.60699 4.34666 1.34142 0.792671 30
produce:pandoraShower:LArPandoraShowerCreation 0.634462 1.82839 6.15048 1.51393 1.11515 30
produce:pandoracalo:Calorimetry 0.309245 0.744928 1.96512 0.671116 0.335355 30
produce:pandoracalonosce:Calorimetry 0.274483 0.678415 1.79691 0.615578 0.306642 30
produce:pandorapid:Chi2ParticleID 0.00131005 0.00223635 0.00502433 0.00195918 0.000848726 30
produce:pandoraShowercalo:ShowerCalorimetry 0.515087 1.63793 5.41627 1.44965 0.964343 30
produce:pandoraShowercalonosce:ShowerCalorimetry 0.508944 1.58087 4.98472 1.33421 0.896983 30
produce:pandoraGnocchiCalo:GnocchiCalorimetry 0.0814457 0.172399 0.346224 0.153447 0.067963 30
produce:pandoraGnocchiCalonosce:GnocchiCalorimetry 0.0313345 0.0641726 0.124302 0.0541399 0.0250717 30
produce:emtrkmichelid:EmTrackMichelId 11.4474 18.957 33.5522 18.3266 4.62058 30
[art]:TriggerResults:TriggerResultInserter 2.9792e-05 4.11357e-05 9.1342e-05 3.95085e-05 1.15062e-05 30
end_path:out1:RootOutput 7.731e-06 1.11654e-05 2.8711e-05 1.03495e-05 3.92322e-06 30
end_path:michelremoving:michelremoving 0.0242504 0.121493 0.467398 0.0549147 0.113971 30
end_path:michelremovinggnocchi:michelremoving 0.0205596 0.112419 0.331968 0.048754 0.101457 30
end_path:calibanagnocchi:CalibAnaTree 0.202489 0.567201 1.31226 0.548078 0.220917 30
end_path:out1:RootOutput(write) 0.502528 0.977301 1.76232 0.951382 0.296519 30
========================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 5737.59 MB
Peak resident set size usage (VmHWM): 2812.57 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np04hd_raw_run029094_0041_dataflow5_datawriter_0_20240906T085251_reco_stage1_reco_stage2_20240926T150431_keepup.root
\tHists: np04hd_raw_run029094_0041_dataflow5_datawriter_0_20240906T085251_reco_stage1_reco_stage2_20240926T150431_keepup_hists.root
Forming reco metadata
/cvmfs/larsoft.opensciencegrid.org/products/python/v3_9_15/Linux64bit+3.10-2.17/bin/python: No module named meta_maker
Error in reco metadata
