Jobsub ID 263882.4@justin-prod-sched01.dune.hep.ac.uk
| Jobsub ID | 263882.4@justin-prod-sched01.dune.hep.ac.uk |
| Workflow ID | 3461 |
| Stage ID | 1 |
| User name | calcuttj@fnal.gov |
| HTCondor Group | group_dune.prod_mcsim |
| Requested | Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 18000 (5 hours) |
| Submitted time | 2024-09-26 14:01:22 |
| Site | IT_CNAF |
| Entry | CMSHTPC_T1_IT_CNAF_condor_ce02 |
| Last heartbeat | 2024-09-26 17:14:57 |
| From worker node | Hostname | wn-204-11-05-02-a |
| cpuinfo | Intel(R) Xeon(R) CPU E5-2618L v4 @ 2.20GHz |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 216000 (60 hours) |
| Inner Apptainer? | True |
| Job state | jobscript_error |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk |
| Started | 2024-09-26 14:03:10 |
| Input files | hd-protodune:np04hd_raw_run027298_0215_dataflow3_datawriter_0_20240619T150312.hdf5
|
| Jobscript | Exit code | 1 |
| Real time | 0m (0s) |
| CPU time | 0m (0s = 0%) |
| Outputting started | |
| Output files | |
| Finished | 2024-09-26 17:14:57 |
| Saved logs | justin-logs:263882.4-justin-prod-sched01.dune.hep.ac.uk.logs.tgz |
| List job events Wrapper job log |
Jobscript log (last 10,000 characters)
and 28148 XVs -> 793 XUVs
C:0 T:7 281 XUs and 149 XVs -> 3 XUVs
2041 XUVs total
1058 collection wire objects
2041 potential space points
Neighbour search...
42943 tests to find 27154 neighbours
Iterating with no regularization...
Begin: 2.02401e+07
0 1.95322e+07
1 1.94627e+07
2 1.94574e+07
Now with regularization...
Begin: 1.8596e+07
0 1.85946e+07
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Begin processing the 34th record. run: 27298 subRun: 1 event: 30235 at 26-Sep-2024 19:03:34 CEST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 103132 traces, 103132 summary
Retagger: tagging trace set: gauss with 74925 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 74925 traces tagged "gauss"
FrameSaver: q=3.73092e+06 n=1369292 tag=gauss
wclsFrameSaver: saving 103132 traces tagged "wiener"
FrameSaver: q=3.95246e+06 n=1222544 tag=wiener
3454 hits in TPC 0
3228 hits in TPC 1
10989 hits in TPC 2
6874 hits in TPC 3
6141 hits in TPC 4
8648 hits in TPC 5
12879 hits in TPC 6
5265 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:0 419 XUs and 162 XVs -> 1 XUVs
C:0 T:1 1583 XUs and 2887 XVs -> 15 XUVs
C:0 T:2 55670 XUs and 76821 XVs -> 3108 XUVs
C:0 T:3 496 XUs and 611 XVs -> 36 XUVs
C:0 T:4 255 XUs and 377 XVs -> 5 XUVs
C:0 T:5 28621 XUs and 57328 XVs -> 849 XUVs
C:0 T:6 49444 XUs and 76987 XVs -> 2462 XUVs
C:0 T:7 266 XUs and 274 XVs -> 8 XUVs
6484 XUVs total
2861 collection wire objects
6484 potential space points
Neighbour search...
327322 tests to find 149202 neighbours
Iterating with no regularization...
Begin: 5.8176e+08
0 5.30579e+08
1 5.27863e+08
2 5.27694e+08
Now with regularization...
Begin: 5.22794e+08
0 5.22704e+08
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
no of hits closeby 1 event 30235 TrkackID 4 startx, y, z 117.341 593.879 241.718 wireno, peakt tpcno 237 23.8101 5 dist 3.03144min T, max_T5.90098 1757.99
no of hits closeby 1 event 30235 TrkackID 4 startx, y, z 117.341 593.879 241.718 wireno, peakt tpcno 237 23.8101 5 dist 3.03144min T, max_T5.90098 1757.99
Begin processing the 35th record. run: 27298 subRun: 1 event: 30239 at 26-Sep-2024 19:10:04 CEST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 71132 traces, 71132 summary
Retagger: tagging trace set: gauss with 51845 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 51845 traces tagged "gauss"
FrameSaver: q=1.81921e+06 n=908351 tag=gauss
wclsFrameSaver: saving 71132 traces tagged "wiener"
FrameSaver: q=1.97415e+06 n=810137 tag=wiener
3420 hits in TPC 0
2539 hits in TPC 1
9520 hits in TPC 2
6040 hits in TPC 3
2426 hits in TPC 4
4481 hits in TPC 5
5847 hits in TPC 6
4393 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 1213 XUs and 1291 XVs -> 14 XUVs
C:0 T:2 89824 XUs and 46455 XVs -> 2474 XUVs
C:0 T:3 2228 XUs and 2822 XVs -> 86 XUVs
C:0 T:4 187 XUs and 90 XVs -> 10 XUVs
C:0 T:5 11516 XUs and 9228 XVs -> 93 XUVs
C:0 T:6 25216 XUs and 18230 XVs -> 696 XUVs
C:0 T:7 179 XUs and 218 XVs -> 14 XUVs
3387 XUVs total
1642 collection wire objects
3387 potential space points
Neighbour search...
127731 tests to find 69962 neighbours
Iterating with no regularization...
Begin: 4.31052e+07
0 3.83471e+07
1 3.78034e+07
2 3.77706e+07
Now with regularization...
Begin: 3.65913e+07
0 3.65802e+07
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
no of hits closeby 5 event 30239 TrkackID 3 startx, y, z 44.92 512.48 7.81607 wireno, peakt tpcno 173 7.39328 5 dist 0.0886949min T, max_T5.43427 657.01
no of hits closeby 5 event 30239 TrkackID 3 startx, y, z 44.92 512.48 7.81607 wireno, peakt tpcno 173 7.39328 5 dist 0.0886949min T, max_T5.43427 657.01
26-Sep-2024 19:14:18 CEST Closed output file "np04hd_raw_run027298_0215_dataflow3_datawriter_0_20240619T150312_reco_stage1_reco_stage2_20240926T171418_keepup.root"
26-Sep-2024 19:14:19 CEST Closed input file "np04hd_raw_run027298_0215_dataflow3_datawriter_0_20240619T150312_reco_stage1.root"
========================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
========================================================================================================================================
Full event 104.908 245.35 395.094 239.983 60.5473 35
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.000709304 0.00411663 0.00779182 0.00446508 0.0021313 35
produce:fembfilter:PDHDFEMBFilter 1.23906 1.46842 1.52819 1.46691 0.0489828 35
produce:wclsdatahd:WireCellToolkit 90.1934 95.0289 103.558 94.1346 3.59167 35
produce:gaushit:GausHitFinder 0.887673 1.84227 3.83575 1.74218 0.582031 35
produce:nhitsfilter:NumberOfHitsFilter 0.000462543 0.00106291 0.0114069 0.000704821 0.00178965 35
produce:reco3d:SpacePointSolver 0.384716 2.04109 7.98672 1.6166 1.5217 34
produce:hitpdune:DisambigFromSpacePoints 0.323409 1.72454 4.60762 1.49209 0.975612 34
produce:pandora:StandardPandora 16.1733 39.256 80.8737 36.2301 16.2275 34
produce:pandoraWriter:StandardPandora 0.208881 0.354701 0.586916 0.343204 0.0873873 34
produce:pandoraTrack:LArPandoraTrackCreation 2.95107 6.44922 14.8042 5.91286 2.55247 34
produce:pandoraShower:LArPandoraShowerCreation 3.07236 8.4188 21.0934 7.28711 4.49667 34
produce:pandoracalo:Calorimetry 1.34901 2.86575 6.87733 2.60464 1.25407 34
produce:pandoracalonosce:Calorimetry 1.2372 2.67812 6.50149 2.43178 1.20955 34
produce:pandorapid:Chi2ParticleID 0.00332181 0.0047685 0.00893727 0.00448711 0.00109745 34
produce:pandoraShowercalo:ShowerCalorimetry 2.2467 7.3527 20.4562 6.40135 4.47038 34
produce:pandoraShowercalonosce:ShowerCalorimetry 2.33593 7.01297 18.6221 6.11832 4.07929 34
produce:pandoraGnocchiCalo:GnocchiCalorimetry 0.183431 0.325215 0.594209 0.318182 0.101036 34
produce:pandoraGnocchiCalonosce:GnocchiCalorimetry 0.0777348 0.128143 0.229056 0.125261 0.0355989 34
produce:emtrkmichelid:EmTrackMichelId 41.6338 67.9891 109.94 65.3024 17.5578 34
[art]:TriggerResults:TriggerResultInserter 5.7787e-05 9.78357e-05 0.000205633 8.6834e-05 2.61736e-05 35
end_path:out1:RootOutput 1.4749e-05 3.33723e-05 0.00033839 2.2983e-05 5.27837e-05 35
end_path:michelremoving:michelremoving 6.8519e-05 0.192078 0.792701 0.111436 0.188228 35
end_path:michelremovinggnocchi:michelremoving 5.0904e-05 0.186564 0.711636 0.105254 0.179946 35
end_path:calibanagnocchi:CalibAnaTree 5.0339e-05 2.18161 5.61382 2.0768 1.06893 35
end_path:out1:RootOutput(write) 1.1132 1.76462 2.86371 1.71545 0.47231 35
========================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 5041.71 MB
Peak resident set size usage (VmHWM): 2421.22 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np04hd_raw_run027298_0215_dataflow3_datawriter_0_20240619T150312_reco_stage1_reco_stage2_20240926T171418_keepup.root
\tHists: np04hd_raw_run027298_0215_dataflow3_datawriter_0_20240619T150312_reco_stage1_reco_stage2_20240926T171418_keepup_hists.root
Forming reco metadata
/cvmfs/larsoft.opensciencegrid.org/products/python/v3_9_15/Linux64bit+3.10-2.17/bin/python: No module named meta_maker
Error in reco metadata
