Jobsub ID 263795.0@justin-prod-sched01.dune.hep.ac.uk
| Jobsub ID | 263795.0@justin-prod-sched01.dune.hep.ac.uk |
| Workflow ID | 3460 |
| Stage ID | 1 |
| User name | calcuttj@fnal.gov |
| HTCondor Group | group_dune.prod_mcsim |
| Requested | Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 18000 (5 hours) |
| Submitted time | 2024-09-26 13:28:44 |
| Site | CZ_FZU |
| Entry | Nova_CZ_FZU_osgce1 |
| Last heartbeat | 2024-09-26 15:20:52 |
| From worker node | Hostname | mikan28.farm.particle.cz |
| cpuinfo | Intel(R) Xeon(R) CPU E5-2650 v4 @ 2.20GHz |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 172800 (48 hours) |
| Inner Apptainer? | True |
| Job state | jobscript_error |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk |
| Started | 2024-09-26 13:34:31 |
| Input files | hd-protodune:np04hd_raw_run029218_0097_dataflow4_datawriter_0_20240917T165137.hdf5
|
| Jobscript | Exit code | 1 |
| Real time | 0m (0s) |
| CPU time | 0m (0s = 0%) |
| Outputting started | |
| Output files | |
| Finished | 2024-09-26 15:20:52 |
| Saved logs | justin-logs:263795.0-justin-prod-sched01.dune.hep.ac.uk.logs.tgz |
| List job events Wrapper job log |
Jobscript log (last 10,000 characters)
TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:0 11 XUs and 13 XVs -> 2 XUVs
C:0 T:1 296 XUs and 298 XVs -> 49 XUVs
C:0 T:2 48752 XUs and 18513 XVs -> 1257 XUVs
C:0 T:3 55 XUs and 97 XVs -> 4 XUVs
C:0 T:4 18 XUs and 12 XVs -> 1 XUVs
C:0 T:5 8288 XUs and 6774 XVs -> 301 XUVs
C:0 T:6 14416 XUs and 9659 XVs -> 294 XUVs
C:0 T:7 22 XUs and 44 XVs -> 2 XUVs
1910 XUVs total
995 collection wire objects
1910 potential space points
Neighbour search...
73510 tests to find 42548 neighbours
Iterating with no regularization...
Begin: 7.04288e+06
0 6.56461e+06
1 6.54586e+06
2 6.54259e+06
Now with regularization...
Begin: 6.13703e+06
0 6.13559e+06
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
no of hits closeby 0 event 24284 TrkackID 11 startx, y, z 88.8418 594.378 418.237 wireno, peakt tpcno 445 1212.19 5 dist 0min T, max_T286.149 1383.86
no of hits closeby 0 event 24284 TrkackID 11 startx, y, z 88.8418 594.378 418.237 wireno, peakt tpcno 445 1212.19 5 dist 0min T, max_T286.149 1383.86
Begin processing the 30th record. run: 29218 subRun: 1 event: 24292 at 26-Sep-2024 17:16:35 CEST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 44268 traces, 44268 summary
Retagger: tagging trace set: gauss with 34035 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 34035 traces tagged "gauss"
FrameSaver: q=778033 n=684713 tag=gauss
wclsFrameSaver: saving 44268 traces tagged "wiener"
FrameSaver: q=824780 n=629127 tag=wiener
3811 hits in TPC 0
2847 hits in TPC 1
3272 hits in TPC 2
2585 hits in TPC 3
1478 hits in TPC 4
3832 hits in TPC 5
3877 hits in TPC 6
2808 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 609 XUs and 865 XVs -> 106 XUVs
C:0 T:2 4363 XUs and 4722 XVs -> 228 XUVs
C:0 T:3 135 XUs and 123 XVs -> 11 XUVs
C:0 T:4 55 XUs and 45 XVs -> 0 XUVs
C:0 T:5 8679 XUs and 4719 XVs -> 315 XUVs
C:0 T:6 8859 XUs and 7612 XVs -> 332 XUVs
C:0 T:7 90 XUs and 94 XVs -> 8 XUVs
1000 XUVs total
614 collection wire objects
1000 potential space points
Neighbour search...
10982 tests to find 6794 neighbours
Iterating with no regularization...
Begin: 3.08567e+06
0 2.91704e+06
1 2.90446e+06
2 2.90359e+06
Now with regularization...
Begin: 2.72902e+06
0 2.72881e+06
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Begin processing the 31st record. run: 29218 subRun: 1 event: 24300 at 26-Sep-2024 17:18:32 CEST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 40880 traces, 40880 summary
Retagger: tagging trace set: gauss with 31076 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 31076 traces tagged "gauss"
FrameSaver: q=726409 n=620792 tag=gauss
wclsFrameSaver: saving 40880 traces tagged "wiener"
FrameSaver: q=774531 n=568975 tag=wiener
1953 hits in TPC 0
1867 hits in TPC 1
4479 hits in TPC 2
3809 hits in TPC 3
1254 hits in TPC 4
3597 hits in TPC 5
3559 hits in TPC 6
2727 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:0 4 XUs and 4 XVs -> 0 XUVs
C:0 T:1 122 XUs and 129 XVs -> 42 XUVs
C:0 T:2 18590 XUs and 11413 XVs -> 849 XUVs
C:0 T:3 75 XUs and 209 XVs -> 7 XUVs
C:0 T:4 54 XUs and 44 XVs -> 5 XUVs
C:0 T:5 4913 XUs and 4310 XVs -> 138 XUVs
C:0 T:6 8925 XUs and 6389 XVs -> 592 XUVs
C:0 T:7 99 XUs and 70 XVs -> 8 XUVs
1641 XUVs total
939 collection wire objects
1641 potential space points
Neighbour search...
22023 tests to find 12582 neighbours
Iterating with no regularization...
Begin: 4.55848e+06
0 4.3126e+06
1 4.29874e+06
2 4.29761e+06
Now with regularization...
Begin: 3.84222e+06
0 3.84197e+06
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
26-Sep-2024 17:20:25 CEST Closed output file "np04hd_raw_run029218_0097_dataflow4_datawriter_0_20240917T165137_reco_stage1_reco_stage2_20240926T152025_keepup.root"
26-Sep-2024 17:20:25 CEST Closed input file "np04hd_raw_run029218_0097_dataflow4_datawriter_0_20240917T165137_reco_stage1.root"
========================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
========================================================================================================================================
Full event 102.041 137.946 183.591 135.555 22.83 31
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.000597967 0.00308351 0.0141947 0.00289247 0.0022309 31
produce:fembfilter:PDHDFEMBFilter 0.713772 0.992237 1.17912 1.08149 0.154961 31
produce:wclsdatahd:WireCellToolkit 47.2094 62.2246 75.0538 64.6589 7.118 31
produce:gaushit:GausHitFinder 0.497667 1.06379 1.8445 1.10127 0.333946 31
produce:nhitsfilter:NumberOfHitsFilter 0.000240899 0.000438077 0.000883849 0.00040377 0.00012496 31
produce:reco3d:SpacePointSolver 0.229802 0.88253 3.09863 0.673256 0.694846 31
produce:hitpdune:DisambigFromSpacePoints 0.237489 0.76805 2.6702 0.534714 0.505721 31
produce:pandora:StandardPandora 7.79122 17.645 33.1506 16.4628 7.18958 31
produce:pandoraWriter:StandardPandora 0.109561 0.204481 0.352234 0.193096 0.0649112 31
produce:pandoraTrack:LArPandoraTrackCreation 0.598101 1.9319 3.77898 1.68196 0.96946 31
produce:pandoraShower:LArPandoraShowerCreation 0.502049 2.33947 4.91981 1.89028 1.3622 31
produce:pandoracalo:Calorimetry 0.312039 0.887398 1.63605 0.782867 0.385597 31
produce:pandoracalonosce:Calorimetry 0.275431 0.797473 1.57482 0.730379 0.380539 31
produce:pandorapid:Chi2ParticleID 0.00122949 0.00372591 0.0499749 0.00204842 0.00846739 31
produce:pandoraShowercalo:ShowerCalorimetry 0.476382 1.94426 4.54377 1.83211 1.07127 31
produce:pandoraShowercalonosce:ShowerCalorimetry 0.635284 2.01662 4.41662 1.9268 1.08634 31
produce:pandoraGnocchiCalo:GnocchiCalorimetry 0.0858216 0.200932 0.476748 0.187674 0.0820297 31
produce:pandoraGnocchiCalonosce:GnocchiCalorimetry 0.0309812 0.0712254 0.121141 0.0656847 0.0258216 31
produce:emtrkmichelid:EmTrackMichelId 28.9243 41.6135 60.402 43.0576 9.47409 31
[art]:TriggerResults:TriggerResultInserter 3.4798e-05 5.97485e-05 0.000114503 5.3903e-05 2.02315e-05 31
end_path:out1:RootOutput 9.956e-06 1.61761e-05 3.436e-05 1.3307e-05 6.64982e-06 31
end_path:michelremoving:michelremoving 0.0342501 0.115158 0.566021 0.0626729 0.135746 31
end_path:michelremovinggnocchi:michelremoving 0.0233629 0.105498 0.535829 0.0451853 0.128348 31
end_path:calibanagnocchi:CalibAnaTree 0.246453 0.802742 1.52034 0.744931 0.349993 31
end_path:out1:RootOutput(write) 0.628769 1.21479 1.84344 1.18813 0.325228 31
========================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 4899.63 MB
Peak resident set size usage (VmHWM): 2559.79 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np04hd_raw_run029218_0097_dataflow4_datawriter_0_20240917T165137_reco_stage1_reco_stage2_20240926T152025_keepup.root
\tHists: np04hd_raw_run029218_0097_dataflow4_datawriter_0_20240917T165137_reco_stage1_reco_stage2_20240926T152025_keepup_hists.root
Forming reco metadata
/cvmfs/larsoft.opensciencegrid.org/products/python/v3_9_15/Linux64bit+3.10-2.17/bin/python: No module named meta_maker
Error in reco metadata
