Jobsub ID 263851.10@justin-prod-sched01.dune.hep.ac.uk
| Jobsub ID | 263851.10@justin-prod-sched01.dune.hep.ac.uk |
| Workflow ID | 3449 |
| Stage ID | 1 |
| User name | calcuttj@fnal.gov |
| HTCondor Group | group_dune.prod_mcsim |
| Requested | Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 18000 (5 hours) |
| Submitted time | 2024-09-26 13:52:03 |
| Site | IT_CNAF |
| Entry | CMSHTPC_T1_IT_CNAF_condor_ce02 |
| Last heartbeat | 2024-09-26 15:51:08 |
| From worker node | Hostname | cn-315-04-05.cr.cnaf.infn.it |
| cpuinfo | Intel(R) Xeon(R) CPU E5-2697 v4 @ 2.30GHz |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 216000 (60 hours) |
| Inner Apptainer? | True |
| Job state | jobscript_error |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk |
| Started | 2024-09-26 13:53:02 |
| Input files | hd-protodune:np04hd_raw_run029107_0869_dataflow3_datawriter_0_20240907T045124.hdf5
|
| Jobscript | Exit code | 1 |
| Real time | 0m (0s) |
| CPU time | 0m (0s = 0%) |
| Outputting started | |
| Output files | |
| Finished | 2024-09-26 15:51:08 |
| Saved logs | justin-logs:263851.10-justin-prod-sched01.dune.hep.ac.uk.logs.tgz |
| List job events Wrapper job log |
Jobscript log (last 10,000 characters)
cidences...
C:0 T:1 24 XUs and 46 XVs -> 0 XUVs
C:0 T:2 40447 XUs and 63691 XVs -> 1539 XUVs
C:0 T:3 183 XUs and 119 XVs -> 3 XUVs
C:0 T:4 74 XUs and 94 XVs -> 2 XUVs
C:0 T:5 20583 XUs and 29206 XVs -> 1189 XUVs
C:0 T:6 24195 XUs and 31366 XVs -> 1529 XUVs
C:0 T:7 128 XUs and 160 XVs -> 5 XUVs
4267 XUVs total
1832 collection wire objects
4267 potential space points
Neighbour search...
254587 tests to find 131056 neighbours
Iterating with no regularization...
Begin: 1.12926e+07
0 1.07226e+07
1 1.0685e+07
2 1.06818e+07
Now with regularization...
Begin: 1.00214e+07
0 1.00199e+07
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Begin processing the 29th record. run: 29107 subRun: 1 event: 208835 at 26-Sep-2024 17:44:46 CEST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 69359 traces, 69359 summary
Retagger: tagging trace set: gauss with 51457 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 51457 traces tagged "gauss"
FrameSaver: q=1.08927e+06 n=932663 tag=gauss
wclsFrameSaver: saving 69359 traces tagged "wiener"
FrameSaver: q=1.17003e+06 n=844074 tag=wiener
2945 hits in TPC 0
2571 hits in TPC 1
7922 hits in TPC 2
4181 hits in TPC 3
3377 hits in TPC 4
4744 hits in TPC 5
7716 hits in TPC 6
3277 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 793 XUs and 323 XVs -> 1 XUVs
C:0 T:2 59867 XUs and 97462 XVs -> 1705 XUVs
C:0 T:3 159 XUs and 229 XVs -> 7 XUVs
C:0 T:4 44 XUs and 40 XVs -> 1 XUVs
C:0 T:5 22692 XUs and 25223 XVs -> 549 XUVs
C:0 T:6 43422 XUs and 43514 XVs -> 565 XUVs
C:0 T:7 90 XUs and 147 XVs -> 4 XUVs
2832 XUVs total
1413 collection wire objects
2832 potential space points
Neighbour search...
77644 tests to find 45816 neighbours
Iterating with no regularization...
Begin: 1.26224e+07
0 1.14269e+07
1 1.13784e+07
2 1.13733e+07
Now with regularization...
Begin: 1.09526e+07
0 1.09502e+07
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Begin processing the 30th record. run: 29107 subRun: 1 event: 208843 at 26-Sep-2024 17:47:55 CEST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 60310 traces, 60310 summary
Retagger: tagging trace set: gauss with 45877 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 45877 traces tagged "gauss"
FrameSaver: q=1.01361e+06 n=864204 tag=gauss
wclsFrameSaver: saving 60310 traces tagged "wiener"
FrameSaver: q=1.07247e+06 n=779783 tag=wiener
3250 hits in TPC 0
2791 hits in TPC 1
4691 hits in TPC 2
2952 hits in TPC 3
4171 hits in TPC 4
5998 hits in TPC 5
4487 hits in TPC 6
2538 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 1188 XUs and 856 XVs -> 6 XUVs
C:0 T:2 10957 XUs and 10570 XVs -> 387 XUVs
C:0 T:3 149 XUs and 207 XVs -> 4 XUVs
C:0 T:4 90 XUs and 145 XVs -> 2 XUVs
C:0 T:5 14799 XUs and 17738 XVs -> 806 XUVs
C:0 T:6 16770 XUs and 9553 XVs -> 539 XUVs
C:0 T:7 8 XUs and 11 XVs -> 0 XUVs
1744 XUVs total
1041 collection wire objects
1744 potential space points
Neighbour search...
22764 tests to find 13224 neighbours
Iterating with no regularization...
Begin: 1.12285e+07
0 1.09088e+07
1 1.089e+07
2 1.08869e+07
Now with regularization...
Begin: 1.03385e+07
0 1.03372e+07
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
no of hits closeby 2 event 208843 TrkackID 0 startx, y, z -237.544 396.114 3.52496 wireno, peakt tpcno 474 4123.7 6 dist 0min T, max_T2510.31 5579.08
no of hits closeby 2 event 208843 TrkackID 0 startx, y, z -237.544 396.114 3.52496 wireno, peakt tpcno 474 4123.7 6 dist 0min T, max_T2510.31 5579.08
26-Sep-2024 17:50:39 CEST Closed output file "np04hd_raw_run029107_0869_dataflow3_datawriter_0_20240907T045124_reco_stage1_reco_stage2_20240926T155039_keepup.root"
26-Sep-2024 17:50:39 CEST Closed input file "np04hd_raw_run029107_0869_dataflow3_datawriter_0_20240907T045124_reco_stage1.root"
========================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
========================================================================================================================================
Full event 113.168 178.509 249.671 174.133 31.8405 30
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.000516684 0.00302078 0.00485337 0.00297969 0.00142679 30
produce:fembfilter:PDHDFEMBFilter 1.03215 1.06227 1.10505 1.06176 0.0113009 30
produce:wclsdatahd:WireCellToolkit 72.9665 74.9598 77.0241 74.9845 1.02679 30
produce:gaushit:GausHitFinder 0.653791 1.17981 1.82144 1.17979 0.260937 30
produce:nhitsfilter:NumberOfHitsFilter 0.000342979 0.000530292 0.000714035 0.000525534 8.93999e-05 30
produce:reco3d:SpacePointSolver 0.219099 1.90566 4.29903 1.75884 1.06615 30
produce:hitpdune:DisambigFromSpacePoints 0.20008 1.30916 3.68381 1.21519 0.792684 30
produce:pandora:StandardPandora 7.62357 23.6258 59.3395 20.1091 10.6733 30
produce:pandoraWriter:StandardPandora 0.122745 0.257952 0.393363 0.262358 0.0603857 30
produce:pandoraTrack:LArPandoraTrackCreation 0.858876 3.61952 8.4896 3.59131 1.65702 30
produce:pandoraShower:LArPandoraShowerCreation 0.72796 4.3834 8.92453 3.96209 2.17761 30
produce:pandoracalo:Calorimetry 0.434028 1.60705 3.24104 1.54437 0.642862 30
produce:pandoracalonosce:Calorimetry 0.409641 1.50248 3.08942 1.48013 0.622058 30
produce:pandorapid:Chi2ParticleID 0.00218131 0.00335337 0.00554756 0.00331137 0.000812578 30
produce:pandoraShowercalo:ShowerCalorimetry 0.654227 3.73337 8.72661 3.47179 1.78 30
produce:pandoraShowercalonosce:ShowerCalorimetry 0.681755 3.6731 8.86514 3.43406 1.7844 30
produce:pandoraGnocchiCalo:GnocchiCalorimetry 0.100777 0.238934 0.344891 0.240837 0.0570682 30
produce:pandoraGnocchiCalonosce:GnocchiCalorimetry 0.041614 0.0988117 0.147225 0.098957 0.0242587 30
produce:emtrkmichelid:EmTrackMichelId 24.73 52.3539 73.764 54.295 11.2751 30
[art]:TriggerResults:TriggerResultInserter 3.8144e-05 7.27665e-05 0.000115017 7.2531e-05 1.26966e-05 30
end_path:out1:RootOutput 1.141e-05 1.55857e-05 3.5997e-05 1.38825e-05 5.07998e-06 30
end_path:michelremoving:michelremoving 0.0328811 0.145093 0.45465 0.0849828 0.108151 30
end_path:michelremovinggnocchi:michelremoving 0.0309772 0.140434 0.454283 0.0795762 0.109725 30
end_path:calibanagnocchi:CalibAnaTree 0.356174 1.21385 2.46384 1.13704 0.523867 30
end_path:out1:RootOutput(write) 0.630713 1.31836 1.99186 1.33337 0.30541 30
========================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 4951.2 MB
Peak resident set size usage (VmHWM): 2305.82 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np04hd_raw_run029107_0869_dataflow3_datawriter_0_20240907T045124_reco_stage1_reco_stage2_20240926T155039_keepup.root
\tHists: np04hd_raw_run029107_0869_dataflow3_datawriter_0_20240907T045124_reco_stage1_reco_stage2_20240926T155039_keepup_hists.root
Forming reco metadata
/cvmfs/larsoft.opensciencegrid.org/products/python/v3_9_15/Linux64bit+3.10-2.17/bin/python: No module named meta_maker
Error in reco metadata
