Jobsub ID 78993.17@justin-prod-sched02.dune.hep.ac.uk
| Jobsub ID | 78993.17@justin-prod-sched02.dune.hep.ac.uk |
| Workflow ID | 3448 |
| Stage ID | 1 |
| User name | calcuttj@fnal.gov |
| HTCondor Group | group_dune.prod_mcsim |
| Requested | Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 18000 (5 hours) |
| Submitted time | 2024-09-26 13:52:03 |
| Site | IT_CNAF |
| Entry | CMSHTPC_T1_IT_CNAF_condor_ce02 |
| Last heartbeat | 2024-09-26 15:02:39 |
| From worker node | Hostname | cn-leo-118.cr.cnaf.infn.it |
| cpuinfo | Intel(R) Xeon(R) Platinum 8480+ |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 216000 (60 hours) |
| Inner Apptainer? | True |
| Job state | jobscript_error |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk |
| Started | 2024-09-26 13:53:19 |
| Input files | hd-protodune:np04hd_raw_run029097_0277_dataflow0_datawriter_0_20240906T160350.hdf5
|
| Jobscript | Exit code | 1 |
| Real time | 0m (0s) |
| CPU time | 0m (0s = 0%) |
| Outputting started | |
| Output files | |
| Finished | 2024-09-26 15:02:39 |
| Saved logs | justin-logs:78993.17-justin-prod-sched02.dune.hep.ac.uk.logs.tgz |
| List job events Wrapper job log |
Jobscript log (last 10,000 characters)
87 0.00738573 0.00116181 0.00197625
----view 1: 7.89762e-05 7.89165e-05 7.89762e-05 7.89762e-05 7.89165e-05 7.89165e-05 7.89762e-05 7.89762e-05 7.89165e-05 7.89165e-05 7.89762e-05
TwoViewTransverseTracksAlgorithm: failed to calculate correlation coefficient p-value for these numbers
----view 0: 0.0180588 0.0156631 0.0188599 0.0200405 0.0149345 0.00423622 0.00661087 0.00738573 0.00116181 0.00197625 0.00257254
----view 1: 7.89165e-05 7.89762e-05 7.89762e-05 7.89165e-05 7.89165e-05 7.89762e-05 7.89762e-05 7.89165e-05 7.89165e-05 7.89762e-05 7.89762e-05
TwoViewTransverseTracksAlgorithm: failed to calculate correlation coefficient p-value for these numbers
----view 0: 0.0156631 0.0188599 0.0200405 0.0149345 0.00423622 0.00661087 0.00738573 0.00116181 0.00197625 0.00257254 0.00209475
----view 1: 7.89762e-05 7.89762e-05 7.89165e-05 7.89165e-05 7.89762e-05 7.89762e-05 7.89165e-05 7.89165e-05 7.89762e-05 7.89762e-05 7.89165e-05
BdtBeamParticleIdTool::SliceFeatures::GetLArTPCIntercepts - inconsistent number of intercepts between a line and the LArTPC
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Begin processing the 29th record. run: 29097 subRun: 1 event: 66720 at 26-Sep-2024 16:59:12 CEST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 43471 traces, 43471 summary
Retagger: tagging trace set: gauss with 33602 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 33602 traces tagged "gauss"
FrameSaver: q=748260 n=632426 tag=gauss
wclsFrameSaver: saving 43471 traces tagged "wiener"
FrameSaver: q=805143 n=576744 tag=wiener
1978 hits in TPC 0
2265 hits in TPC 1
2347 hits in TPC 2
1803 hits in TPC 3
4316 hits in TPC 4
4263 hits in TPC 5
3661 hits in TPC 6
2660 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 64 XUs and 83 XVs -> 10 XUVs
C:0 T:2 30402 XUs and 25209 XVs -> 801 XUVs
C:0 T:3 22 XUs and 26 XVs -> 1 XUVs
C:0 T:4 326 XUs and 427 XVs -> 40 XUVs
C:0 T:5 25930 XUs and 12494 XVs -> 528 XUVs
C:0 T:6 14144 XUs and 16021 XVs -> 696 XUVs
C:0 T:7 315 XUs and 199 XVs -> 0 XUVs
2076 XUVs total
847 collection wire objects
2076 potential space points
Neighbour search...
67772 tests to find 41926 neighbours
Iterating with no regularization...
Begin: 6.19769e+06
0 5.78678e+06
1 5.75818e+06
2 5.75278e+06
Now with regularization...
Begin: 5.4294e+06
0 5.42736e+06
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Begin processing the 30th record. run: 29097 subRun: 1 event: 66728 at 26-Sep-2024 17:00:38 CEST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 59081 traces, 59081 summary
Retagger: tagging trace set: gauss with 44169 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 44169 traces tagged "gauss"
FrameSaver: q=989128 n=828856 tag=gauss
wclsFrameSaver: saving 59081 traces tagged "wiener"
FrameSaver: q=1.05179e+06 n=749151 tag=wiener
3286 hits in TPC 0
2023 hits in TPC 1
4557 hits in TPC 2
2144 hits in TPC 3
2218 hits in TPC 4
2497 hits in TPC 5
8270 hits in TPC 6
4550 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 3229 XUs and 1233 XVs -> 39 XUVs
C:0 T:2 11177 XUs and 8303 XVs -> 597 XUVs
C:0 T:3 36 XUs and 38 XVs -> 0 XUVs
C:0 T:4 12 XUs and 19 XVs -> 3 XUVs
C:0 T:5 3992 XUs and 4889 XVs -> 325 XUVs
C:0 T:6 29657 XUs and 24939 XVs -> 1184 XUVs
C:0 T:7 858 XUs and 610 XVs -> 16 XUVs
2164 XUVs total
1341 collection wire objects
2164 potential space points
Neighbour search...
39444 tests to find 22368 neighbours
Iterating with no regularization...
Begin: 7.17422e+06
0 6.94385e+06
1 6.93456e+06
2 6.93353e+06
Now with regularization...
Begin: 6.32159e+06
0 6.32119e+06
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
26-Sep-2024 17:02:16 CEST Closed output file "np04hd_raw_run029097_0277_dataflow0_datawriter_0_20240906T160350_reco_stage1_reco_stage2_20240926T150216_keepup.root"
26-Sep-2024 17:02:16 CEST Closed input file "np04hd_raw_run029097_0277_dataflow0_datawriter_0_20240906T160350_reco_stage1.root"
========================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
========================================================================================================================================
Full event 63.1922 92.9721 115.745 92.4884 10.4529 30
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.000504459 0.00228088 0.00641603 0.00220271 0.0016043 30
produce:fembfilter:PDHDFEMBFilter 0.675689 0.708964 0.924244 0.69792 0.04242 30
produce:wclsdatahd:WireCellToolkit 57.9686 60.2985 63.5925 59.8675 1.41877 30
produce:gaushit:GausHitFinder 0.333818 0.52482 0.751106 0.530825 0.109299 30
produce:nhitsfilter:NumberOfHitsFilter 0.000367053 0.000516521 0.000732965 0.00051013 8.97641e-05 30
produce:reco3d:SpacePointSolver 0.191703 0.758016 2.7356 0.640463 0.503627 29
produce:hitpdune:DisambigFromSpacePoints 0.152363 0.297921 0.576922 0.293868 0.109391 29
produce:pandora:StandardPandora 2.92003 7.7632 12.6576 7.81746 2.35687 29
produce:pandoraWriter:StandardPandora 0.0961438 0.14042 0.21921 0.14306 0.0287857 29
produce:pandoraTrack:LArPandoraTrackCreation 0.611552 2.05683 3.97414 1.98467 0.800478 29
produce:pandoraShower:LArPandoraShowerCreation 0.580557 2.37351 4.50163 2.44207 0.983329 29
produce:pandoracalo:Calorimetry 0.32945 0.841426 1.53119 0.758892 0.306525 29
produce:pandoracalonosce:Calorimetry 0.254465 0.783076 1.46246 0.723132 0.289318 29
produce:pandorapid:Chi2ParticleID 0.00108748 0.0228015 0.618481 0.00150186 0.112573 29
produce:pandoraShowercalo:ShowerCalorimetry 0.511235 1.85352 3.71877 1.79912 0.795228 29
produce:pandoraShowercalonosce:ShowerCalorimetry 0.474059 1.81974 3.71935 1.71477 0.785398 29
produce:pandoraGnocchiCalo:GnocchiCalorimetry 0.0596109 0.0887375 0.136885 0.0830518 0.0187032 29
produce:pandoraGnocchiCalonosce:GnocchiCalorimetry 0.025882 0.0410843 0.0696968 0.0400085 0.00962878 29
produce:emtrkmichelid:EmTrackMichelId 7.95017 12.0167 16.0763 12.3984 1.96458 29
[art]:TriggerResults:TriggerResultInserter 4.3914e-05 6.85642e-05 9.4939e-05 7.0265e-05 1.25935e-05 30
end_path:out1:RootOutput 1.3356e-05 2.30039e-05 6.1455e-05 1.99325e-05 9.34436e-06 30
end_path:michelremoving:michelremoving 5.997e-05 0.0794929 0.344611 0.0567186 0.0619497 30
end_path:michelremovinggnocchi:michelremoving 3.7769e-05 0.0771719 0.351793 0.0505293 0.0634707 30
end_path:calibanagnocchi:CalibAnaTree 3.6071e-05 0.585499 1.19535 0.560433 0.256602 30
end_path:out1:RootOutput(write) 0.25364 0.549764 0.800424 0.549821 0.113021 30
========================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 5740.15 MB
Peak resident set size usage (VmHWM): 2548.41 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np04hd_raw_run029097_0277_dataflow0_datawriter_0_20240906T160350_reco_stage1_reco_stage2_20240926T150216_keepup.root
\tHists: np04hd_raw_run029097_0277_dataflow0_datawriter_0_20240906T160350_reco_stage1_reco_stage2_20240926T150216_keepup_hists.root
Forming reco metadata
/cvmfs/larsoft.opensciencegrid.org/products/python/v3_9_15/Linux64bit+3.10-2.17/bin/python: No module named meta_maker
Error in reco metadata
