Jobsub ID 263872.13@justin-prod-sched01.dune.hep.ac.uk
| Jobsub ID | 263872.13@justin-prod-sched01.dune.hep.ac.uk |
| Workflow ID | 3448 |
| Stage ID | 1 |
| User name | calcuttj@fnal.gov |
| HTCondor Group | group_dune.prod_mcsim |
| Requested | Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 18000 (5 hours) |
| Submitted time | 2024-09-26 13:58:59 |
| Site | IT_CNAF |
| Entry | CMSHTPC_T1_IT_CNAF_condor_ce02 |
| Last heartbeat | 2024-09-26 15:51:55 |
| From worker node | Hostname | wn-204-11-27-03-a |
| cpuinfo | Intel(R) Xeon(R) CPU E5-2618L v4 @ 2.20GHz |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 216000 (60 hours) |
| Inner Apptainer? | True |
| Job state | jobscript_error |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk |
| Started | 2024-09-26 14:01:09 |
| Input files | hd-protodune:np04hd_raw_run029097_0221_dataflow2_datawriter_0_20240906T150039.hdf5
|
| Jobscript | Exit code | 1 |
| Real time | 0m (0s) |
| CPU time | 0m (0s = 0%) |
| Outputting started | |
| Output files | |
| Finished | 2024-09-26 15:51:55 |
| Saved logs | justin-logs:263872.13-justin-prod-sched01.dune.hep.ac.uk.logs.tgz |
| List job events Wrapper job log |
Jobscript log (last 10,000 characters)
nd 74728 neighbours
Iterating with no regularization...
Begin: 4.5498e+06
0 4.40787e+06
1 4.40294e+06
2 4.40244e+06
Now with regularization...
Begin: 4.17674e+06
0 4.1765e+06
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
no of hits closeby 2 event 53258 TrkackID 6 startx, y, z 169.273 567.937 244.035 wireno, peakt tpcno 6 2103.88 6 dist 108.569min T, max_T6.5316 2256
no of hits closeby 2 event 53258 TrkackID 6 startx, y, z 169.273 567.937 244.035 wireno, peakt tpcno 6 2103.88 6 dist 108.569min T, max_T6.5316 2256
Begin processing the 29th record. run: 29097 subRun: 1 event: 53266 at 26-Sep-2024 17:46:10 CEST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 77263 traces, 77263 summary
Retagger: tagging trace set: gauss with 56333 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 56333 traces tagged "gauss"
FrameSaver: q=1.22982e+06 n=1091803 tag=gauss
wclsFrameSaver: saving 77263 traces tagged "wiener"
FrameSaver: q=1.34323e+06 n=1002058 tag=wiener
4141 hits in TPC 0
3744 hits in TPC 1
7723 hits in TPC 2
4999 hits in TPC 3
3328 hits in TPC 4
5653 hits in TPC 5
8110 hits in TPC 6
4497 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 1287 XUs and 1491 XVs -> 95 XUVs
C:0 T:2 45588 XUs and 70050 XVs -> 1310 XUVs
C:0 T:3 881 XUs and 1206 XVs -> 19 XUVs
C:0 T:4 155 XUs and 195 XVs -> 4 XUVs
C:0 T:5 16822 XUs and 25595 XVs -> 4490 XUVs
C:0 T:6 45429 XUs and 66056 XVs -> 5400 XUVs
C:0 T:7 1782 XUs and 1658 XVs -> 126 XUVs
11444 XUVs total
1866 collection wire objects
11444 potential space points
Neighbour search...
1397258 tests to find 794028 neighbours
Iterating with no regularization...
Begin: 9.57751e+06
0 7.75065e+06
1 7.62238e+06
2 7.60382e+06
3 7.59797e+06
Now with regularization...
Begin: 7.01458e+06
0 7.00901e+06
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
no of hits closeby 0 event 53266 TrkackID 4 startx, y, z 108.197 596.268 330.895 wireno, peakt tpcno 0 3149 5 dist 13.1719min T, max_T2402.46 5748.43
no of hits closeby 0 event 53266 TrkackID 4 startx, y, z 108.197 596.268 330.895 wireno, peakt tpcno 0 3149 5 dist 13.1719min T, max_T2402.46 5748.43
Begin processing the 30th record. run: 29097 subRun: 1 event: 53274 at 26-Sep-2024 17:49:22 CEST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 50272 traces, 50272 summary
Retagger: tagging trace set: gauss with 38347 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 38347 traces tagged "gauss"
FrameSaver: q=941269 n=765227 tag=gauss
wclsFrameSaver: saving 50272 traces tagged "wiener"
FrameSaver: q=986849 n=697074 tag=wiener
1487 hits in TPC 0
1416 hits in TPC 1
6300 hits in TPC 2
4148 hits in TPC 3
1449 hits in TPC 4
3774 hits in TPC 5
5283 hits in TPC 6
3814 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 43 XUs and 52 XVs -> 1 XUVs
C:0 T:2 20091 XUs and 19840 XVs -> 431 XUVs
C:0 T:3 60 XUs and 67 XVs -> 0 XUVs
C:0 T:4 571 XUs and 894 XVs -> 61 XUVs
C:0 T:5 3773 XUs and 3788 XVs -> 178 XUVs
C:0 T:6 10758 XUs and 16831 XVs -> 1088 XUVs
C:0 T:7 263 XUs and 269 XVs -> 23 XUVs
1782 XUVs total
1072 collection wire objects
1782 potential space points
Neighbour search...
24926 tests to find 14604 neighbours
Iterating with no regularization...
Begin: 8.02413e+06
0 7.67847e+06
1 7.65692e+06
2 7.6547e+06
Now with regularization...
Begin: 7.10244e+06
0 7.10201e+06
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
26-Sep-2024 17:51:26 CEST Closed output file "np04hd_raw_run029097_0221_dataflow2_datawriter_0_20240906T150039_reco_stage1_reco_stage2_20240926T155126_keepup.root"
26-Sep-2024 17:51:26 CEST Closed input file "np04hd_raw_run029097_0221_dataflow2_datawriter_0_20240906T150039_reco_stage1.root"
========================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
========================================================================================================================================
Full event 110.197 154.679 216.121 149.446 26.4873 30
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.000596792 0.00273847 0.00500414 0.0026272 0.00121081 30
produce:fembfilter:PDHDFEMBFilter 0.862474 1.02827 1.37029 0.884639 0.20879 30
produce:wclsdatahd:WireCellToolkit 64.7048 73.4109 87.4476 71.7557 6.5865 30
produce:gaushit:GausHitFinder 0.660486 1.05732 2.09157 0.965825 0.334761 30
produce:nhitsfilter:NumberOfHitsFilter 0.000257803 0.000430636 0.000650254 0.000426463 0.000102867 30
produce:reco3d:SpacePointSolver 0.200035 1.58295 12.2368 1.04378 2.20167 30
produce:hitpdune:DisambigFromSpacePoints 0.235445 0.84215 2.1643 0.663631 0.490877 30
produce:pandora:StandardPandora 8.93343 19.1289 54.6906 16.1566 10.1058 30
produce:pandoraWriter:StandardPandora 0.108824 0.210737 0.387555 0.19478 0.0645112 30
produce:pandoraTrack:LArPandoraTrackCreation 1.0198 2.16999 4.70104 1.81642 1.02305 30
produce:pandoraShower:LArPandoraShowerCreation 0.692969 2.48329 6.28834 2.15415 1.42091 30
produce:pandoracalo:Calorimetry 0.482118 1.08517 2.17377 0.973453 0.435518 30
produce:pandoracalonosce:Calorimetry 0.43644 0.983195 2.02201 0.898401 0.413876 30
produce:pandorapid:Chi2ParticleID 0.00175593 0.00267623 0.00442806 0.00234328 0.000828947 30
produce:pandoraShowercalo:ShowerCalorimetry 0.614488 2.20281 4.74311 2.07773 1.00628 30
produce:pandoraShowercalonosce:ShowerCalorimetry 0.598716 2.22606 4.71011 2.1856 0.991938 30
produce:pandoraGnocchiCalo:GnocchiCalorimetry 0.0819809 0.207876 0.431026 0.190865 0.0717604 30
produce:pandoraGnocchiCalonosce:GnocchiCalorimetry 0.0326249 0.0777104 0.163903 0.0698206 0.0277801 30
produce:emtrkmichelid:EmTrackMichelId 26.3115 43.6076 59.6471 42.5718 9.84018 30
[art]:TriggerResults:TriggerResultInserter 2.5055e-05 4.17119e-05 0.000158641 3.7271e-05 2.39351e-05 30
end_path:out1:RootOutput 8.453e-06 0.00099842 0.0296579 9.753e-06 0.00532193 30
end_path:michelremoving:michelremoving 0.0253792 0.229234 4.19883 0.0557921 0.74251 30
end_path:michelremovinggnocchi:michelremoving 0.0236564 0.0911835 0.431076 0.049382 0.0891299 30
end_path:calibanagnocchi:CalibAnaTree 0.263495 0.77698 1.48742 0.745396 0.322656 30
end_path:out1:RootOutput(write) 0.566446 1.15371 1.79666 1.13157 0.342849 30
========================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 4907.82 MB
Peak resident set size usage (VmHWM): 2099 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np04hd_raw_run029097_0221_dataflow2_datawriter_0_20240906T150039_reco_stage1_reco_stage2_20240926T155126_keepup.root
\tHists: np04hd_raw_run029097_0221_dataflow2_datawriter_0_20240906T150039_reco_stage1_reco_stage2_20240926T155126_keepup_hists.root
Forming reco metadata
/cvmfs/larsoft.opensciencegrid.org/products/python/v3_9_15/Linux64bit+3.10-2.17/bin/python: No module named meta_maker
Error in reco metadata
