Jobsub ID 78692.63@justin-prod-sched02.dune.hep.ac.uk
| Jobsub ID | 78692.63@justin-prod-sched02.dune.hep.ac.uk |
| Workflow ID | 3447 |
| Stage ID | 1 |
| User name | calcuttj@fnal.gov |
| HTCondor Group | group_dune.prod_mcsim |
| Requested | Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 18000 (5 hours) |
| Submitted time | 2024-09-26 11:28:48 |
| Site | IT_CNAF |
| Entry | CMSHTPC_T1_IT_CNAF_condor_ce02 |
| Last heartbeat | 2024-09-26 14:45:18 |
| From worker node | Hostname | wn-204-11-13-02-a |
| cpuinfo | Intel(R) Xeon(R) CPU E5-2618L v4 @ 2.20GHz |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 216000 (60 hours) |
| Inner Apptainer? | True |
| Job state | jobscript_error |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk |
| Started | 2024-09-26 12:17:55 |
| Input files | hd-protodune:np04hd_raw_run029081_0685_dataflow7_datawriter_0_20240905T035522.hdf5
|
| Jobscript | Exit code | 1 |
| Real time | 0m (0s) |
| CPU time | 0m (0s = 0%) |
| Outputting started | |
| Output files | |
| Finished | 2024-09-26 14:45:18 |
| Saved logs | justin-logs:78692.63-justin-prod-sched02.dune.hep.ac.uk.logs.tgz |
| List job events Wrapper job log |
Jobscript log (last 10,000 characters)
wclsFrameSaver: saving 42180 traces tagged "gauss"
FrameSaver: q=882146 n=776339 tag=gauss
wclsFrameSaver: saving 56211 traces tagged "wiener"
FrameSaver: q=946550 n=701986 tag=wiener
2078 hits in TPC 0
3187 hits in TPC 1
5304 hits in TPC 2
3596 hits in TPC 3
2402 hits in TPC 4
5205 hits in TPC 5
3665 hits in TPC 6
2695 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 117 XUs and 152 XVs -> 0 XUVs
C:0 T:2 11005 XUs and 11720 XVs -> 545 XUVs
C:0 T:3 206 XUs and 141 XVs -> 5 XUVs
C:0 T:4 64 XUs and 102 XVs -> 2 XUVs
C:0 T:5 16384 XUs and 15707 XVs -> 308 XUVs
C:0 T:6 11082 XUs and 11556 XVs -> 1147 XUVs
C:0 T:7 151 XUs and 172 XVs -> 22 XUVs
2029 XUVs total
814 collection wire objects
2029 potential space points
Neighbour search...
54407 tests to find 29996 neighbours
Iterating with no regularization...
Begin: 3.96626e+06
0 3.70439e+06
1 3.67451e+06
2 3.66812e+06
3 3.66615e+06
Now with regularization...
Begin: 3.38999e+06
0 3.38858e+06
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Begin processing the 29th record. run: 29081 subRun: 1 event: 164639 at 26-Sep-2024 16:37:48 CEST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 58923 traces, 58923 summary
Retagger: tagging trace set: gauss with 44672 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 44672 traces tagged "gauss"
FrameSaver: q=920296 n=821191 tag=gauss
wclsFrameSaver: saving 58923 traces tagged "wiener"
FrameSaver: q=985580 n=740739 tag=wiener
3165 hits in TPC 0
4711 hits in TPC 1
5937 hits in TPC 2
3487 hits in TPC 3
1915 hits in TPC 4
3275 hits in TPC 5
5426 hits in TPC 6
3157 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 271 XUs and 701 XVs -> 14 XUVs
C:0 T:2 38354 XUs and 41541 XVs -> 1690 XUVs
C:0 T:3 167 XUs and 262 XVs -> 5 XUVs
C:0 T:4 47 XUs and 31 XVs -> 3 XUVs
C:0 T:5 8301 XUs and 4958 XVs -> 309 XUVs
C:0 T:6 43349 XUs and 42253 XVs -> 1769 XUVs
C:0 T:7 33 XUs and 23 XVs -> 1 XUVs
3791 XUVs total
1151 collection wire objects
3791 potential space points
Neighbour search...
355213 tests to find 160550 neighbours
Iterating with no regularization...
Begin: 4.97782e+06
0 4.75867e+06
1 4.74088e+06
2 4.73724e+06
Now with regularization...
Begin: 4.46724e+06
0 4.46583e+06
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Begin processing the 30th record. run: 29081 subRun: 1 event: 164647 at 26-Sep-2024 16:41:03 CEST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 61271 traces, 61271 summary
Retagger: tagging trace set: gauss with 44275 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 44275 traces tagged "gauss"
FrameSaver: q=991122 n=815298 tag=gauss
wclsFrameSaver: saving 61271 traces tagged "wiener"
FrameSaver: q=1.07065e+06 n=736055 tag=wiener
2433 hits in TPC 0
1647 hits in TPC 1
5601 hits in TPC 2
4444 hits in TPC 3
2661 hits in TPC 4
6213 hits in TPC 5
5881 hits in TPC 6
2512 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 107 XUs and 107 XVs -> 17 XUVs
C:0 T:2 27366 XUs and 23784 XVs -> 1113 XUVs
C:0 T:3 444 XUs and 406 XVs -> 45 XUVs
C:0 T:4 160 XUs and 130 XVs -> 8 XUVs
C:0 T:5 17843 XUs and 9734 XVs -> 387 XUVs
C:0 T:6 24717 XUs and 23683 XVs -> 455 XUVs
C:0 T:7 55 XUs and 20 XVs -> 0 XUVs
2025 XUVs total
1183 collection wire objects
2025 potential space points
Neighbour search...
38069 tests to find 22362 neighbours
Iterating with no regularization...
Begin: 5.88375e+06
0 5.48544e+06
1 5.45694e+06
2 5.45392e+06
Now with regularization...
Begin: 5.06993e+06
0 5.06904e+06
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
26-Sep-2024 16:44:37 CEST Closed output file "np04hd_raw_run029081_0685_dataflow7_datawriter_0_20240905T035522_reco_stage1_reco_stage2_20240926T144437_keepup.root"
26-Sep-2024 16:44:37 CEST Closed input file "np04hd_raw_run029081_0685_dataflow7_datawriter_0_20240905T035522_reco_stage1.root"
========================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
========================================================================================================================================
Full event 138.282 220.678 280.385 221.896 35.3248 30
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.000706174 0.00381183 0.00603972 0.00446351 0.00172368 30
produce:fembfilter:PDHDFEMBFilter 1.30609 1.3509 1.43981 1.34553 0.0250617 30
produce:wclsdatahd:WireCellToolkit 89.9092 92.4551 96.2662 92.3238 1.57008 30
produce:gaushit:GausHitFinder 0.850073 1.5928 2.33399 1.55736 0.384077 30
produce:nhitsfilter:NumberOfHitsFilter 0.000367073 0.000638344 0.000982007 0.000597619 0.000153713 30
produce:reco3d:SpacePointSolver 0.295908 2.18553 6.59539 1.74213 1.53787 30
produce:hitpdune:DisambigFromSpacePoints 0.285051 1.5192 3.55818 1.20247 0.778853 30
produce:pandora:StandardPandora 8.97322 30.4123 53.4307 29.7942 11.3316 30
produce:pandoraWriter:StandardPandora 0.14504 0.308155 0.404609 0.319455 0.0617903 30
produce:pandoraTrack:LArPandoraTrackCreation 0.984642 4.09795 7.16268 3.8534 1.35556 30
produce:pandoraShower:LArPandoraShowerCreation 0.952242 5.2947 8.87769 5.38357 2.13276 30
produce:pandoracalo:Calorimetry 0.512236 1.88362 2.87418 1.79281 0.575684 30
produce:pandoracalonosce:Calorimetry 0.432944 1.76965 2.94535 1.71621 0.597294 30
produce:pandorapid:Chi2ParticleID 0.00211671 0.0039908 0.0075171 0.00394431 0.000888537 30
produce:pandoraShowercalo:ShowerCalorimetry 0.868104 4.53964 7.87788 4.44104 1.87607 30
produce:pandoraShowercalonosce:ShowerCalorimetry 0.843587 4.43853 7.54152 4.46247 1.79498 30
produce:pandoraGnocchiCalo:GnocchiCalorimetry 0.150825 0.317713 0.434265 0.3131 0.0701689 30
produce:pandoraGnocchiCalonosce:GnocchiCalorimetry 0.0580946 0.124956 0.16871 0.12279 0.025527 30
produce:emtrkmichelid:EmTrackMichelId 30.3246 64.7169 83.3608 65.7531 13.0394 30
[art]:TriggerResults:TriggerResultInserter 6.7679e-05 8.64318e-05 0.000149847 7.82175e-05 2.13558e-05 30
end_path:out1:RootOutput 1.2425e-05 2.2548e-05 4.6124e-05 2.0998e-05 7.68355e-06 30
end_path:michelremoving:michelremoving 0.0362286 0.169245 0.657101 0.095771 0.148842 30
end_path:michelremovinggnocchi:michelremoving 0.03286 0.166979 0.682989 0.0936442 0.151588 30
end_path:calibanagnocchi:CalibAnaTree 0.306099 1.41941 2.73093 1.36263 0.465275 30
end_path:out1:RootOutput(write) 0.826991 1.69002 2.19094 1.75592 0.343051 30
========================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 4910.64 MB
Peak resident set size usage (VmHWM): 2332.24 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np04hd_raw_run029081_0685_dataflow7_datawriter_0_20240905T035522_reco_stage1_reco_stage2_20240926T144437_keepup.root
\tHists: np04hd_raw_run029081_0685_dataflow7_datawriter_0_20240905T035522_reco_stage1_reco_stage2_20240926T144437_keepup_hists.root
Forming reco metadata
/cvmfs/larsoft.opensciencegrid.org/products/python/v3_9_15/Linux64bit+3.10-2.17/bin/python: No module named meta_maker
Error in reco metadata
