Jobsub ID 263758.6@justin-prod-sched01.dune.hep.ac.uk
| Jobsub ID | 263758.6@justin-prod-sched01.dune.hep.ac.uk |
| Workflow ID | 3449 |
| Stage ID | 1 |
| User name | calcuttj@fnal.gov |
| HTCondor Group | group_dune.prod_mcsim |
| Requested | Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 18000 (5 hours) |
| Submitted time | 2024-09-26 13:23:34 |
| Site | UK_Durham |
| Entry | DUNE_UK_SGridDurham_ce4 |
| Last heartbeat | 2024-09-26 15:09:33 |
| From worker node | Hostname | n224.dur.scotgrid.ac.uk |
| cpuinfo | Intel(R) Xeon(R) Gold 5220 CPU @ 2.20GHz |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 171000 (47 hours) |
| Inner Apptainer? | True |
| Job state | jobscript_error |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk |
| Started | 2024-09-26 13:25:17 |
| Input files | hd-protodune:np04hd_raw_run029107_0837_dataflow5_datawriter_0_20240907T042622.hdf5
|
| Jobscript | Exit code | 1 |
| Real time | 0m (0s) |
| CPU time | 0m (0s = 0%) |
| Outputting started | |
| Output files | |
| Finished | 2024-09-26 15:09:33 |
| Saved logs | justin-logs:263758.6-justin-prod-sched01.dune.hep.ac.uk.logs.tgz |
| List job events Wrapper job log |
Jobscript log (last 10,000 characters)
2135 XUVs
C:0 T:3 96 XUs and 115 XVs -> 5 XUVs
C:0 T:4 25 XUs and 8 XVs -> 0 XUVs
C:0 T:5 62786 XUs and 52134 XVs -> 10797 XUVs
C:0 T:6 45695 XUs and 70540 XVs -> 2867 XUVs
C:0 T:7 273 XUs and 252 XVs -> 2 XUVs
15827 XUVs total
2802 collection wire objects
15827 potential space points
Neighbour search...
2305837 tests to find 995916 neighbours
Iterating with no regularization...
Begin: 3.27951e+07
0 2.66324e+07
1 2.5892e+07
2 2.57921e+07
3 2.57687e+07
Now with regularization...
Begin: 2.31389e+07
0 2.30473e+07
1 2.30289e+07
Failure in algorithm Alg0065, LArTwoViewTransverseTracks, STATUS_CODE_INVALID_PARAMETER
Failure in algorithm Alg0051, LArTwoViewTransverseTracks, STATUS_CODE_INVALID_PARAMETER
Failure in algorithm Alg0060, LArTwoViewTransverseTracks, STATUS_CODE_INVALID_PARAMETER
Failure in algorithm Alg0065, LArTwoViewTransverseTracks, STATUS_CODE_INVALID_PARAMETER
Failure in algorithm Alg0056, LArTwoViewTransverseTracks, STATUS_CODE_INVALID_PARAMETER
Failure in algorithm Alg0064, LArTwoViewTransverseTracks, STATUS_CODE_INVALID_PARAMETER
Failure in algorithm Alg0069, LArTwoViewTransverseTracks, STATUS_CODE_INVALID_PARAMETER
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
no of hits closeby 0 event 201133 TrkackID 4 startx, y, z 302.178 235.863 147.21 wireno, peakt tpcno 189 3842.5 2 dist 83.0477min T, max_T6.34974 3865
no of hits closeby 0 event 201133 TrkackID 20 startx, y, z -31.0147 588.605 338.767 wireno, peakt tpcno 449 926.9 6 dist 0min T, max_T695.808 1097.79
no of hits closeby 0 event 201133 TrkackID 4 startx, y, z 302.178 235.863 147.21 wireno, peakt tpcno 189 3842.5 2 dist 83.0477min T, max_T6.34974 3865
no of hits closeby 0 event 201133 TrkackID 20 startx, y, z -31.0147 588.605 338.767 wireno, peakt tpcno 449 926.9 6 dist 0min T, max_T695.808 1097.79
Begin processing the 30th record. run: 29107 subRun: 1 event: 201141 at 26-Sep-2024 16:07:46 BST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 51583 traces, 51583 summary
Retagger: tagging trace set: gauss with 38153 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 38153 traces tagged "gauss"
FrameSaver: q=816925 n=708915 tag=gauss
wclsFrameSaver: saving 51583 traces tagged "wiener"
FrameSaver: q=881584 n=645276 tag=wiener
2783 hits in TPC 0
2571 hits in TPC 1
5382 hits in TPC 2
3892 hits in TPC 3
1459 hits in TPC 4
3563 hits in TPC 5
4472 hits in TPC 6
2305 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 432 XUs and 415 XVs -> 10 XUVs
C:0 T:2 28097 XUs and 21393 XVs -> 705 XUVs
C:0 T:3 46 XUs and 44 XVs -> 0 XUVs
C:0 T:4 11 XUs and 12 XVs -> 1 XUVs
C:0 T:5 4657 XUs and 3719 XVs -> 103 XUVs
C:0 T:6 8470 XUs and 8530 XVs -> 169 XUVs
C:0 T:7 195 XUs and 239 XVs -> 8 XUVs
996 XUVs total
578 collection wire objects
996 potential space points
Neighbour search...
13032 tests to find 8716 neighbours
Iterating with no regularization...
Begin: 4.27867e+06
0 4.06676e+06
1 4.05495e+06
2 4.05339e+06
Now with regularization...
Begin: 3.82226e+06
0 3.82191e+06
TwoViewTransverseTracksAlgorithm: failed to calculate correlation coefficient p-value for these numbers
----view 0: 0.00598012 0.000458598 0.00134902 0.00129309 0.00238496 0.00483218 0.00141892 0.00203578 0.0022232 0.00343748 0.00122508 0.00119624 0.00437985
----view 1: 0 0 0 0 0 0 0 0 0 0 0 0 0
TwoViewTransverseTracksAlgorithm: failed to calculate correlation coefficient p-value for these numbers
----view 0: 0.00598012 0.000458598 0.00134902 0.00129309 0.00238496 0.00483218 0.00141892 0.00203578 0.0022232 0.00343748 0.00122508 0.00119624 0.00437985
----view 1: 0 0 0 0 0 0 0 0 0 0 0 0 0
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
no of hits closeby 0 event 201141 TrkackID 4 startx, y, z 150.458 597.113 273.632 wireno, peakt tpcno 452 5103.29 2 dist 222.314min T, max_T3264.72 5167.96
no of hits closeby 0 event 201141 TrkackID 4 startx, y, z 150.458 597.113 273.632 wireno, peakt tpcno 452 5103.29 2 dist 222.314min T, max_T3264.72 5167.96
26-Sep-2024 16:09:03 BST Closed output file "np04hd_raw_run029107_0837_dataflow5_datawriter_0_20240907T042622_reco_stage1_reco_stage2_20240926T150903_keepup.root"
26-Sep-2024 16:09:03 BST Closed input file "np04hd_raw_run029107_0837_dataflow5_datawriter_0_20240907T042622_reco_stage1.root"
========================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
========================================================================================================================================
Full event 73.825 98.1729 146.921 93.5451 17.7215 30
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.000415638 0.00141678 0.00445309 0.00114467 0.000897014 30
produce:fembfilter:PDHDFEMBFilter 0.703008 0.731222 1.04903 0.712605 0.0635127 30
produce:wclsdatahd:WireCellToolkit 44.7393 48.1037 65.2109 47.1713 3.58401 30
produce:gaushit:GausHitFinder 0.489727 0.787214 1.17872 0.757509 0.1818 30
produce:nhitsfilter:NumberOfHitsFilter 0.00026885 0.000376344 0.000567413 0.000383012 6.9904e-05 30
produce:reco3d:SpacePointSolver 0.267012 1.26743 3.15293 1.01491 0.812584 30
produce:hitpdune:DisambigFromSpacePoints 0.241123 0.631722 1.9683 0.564222 0.330422 30
produce:pandora:StandardPandora 6.70763 13.65 30.2287 12.0797 5.93249 30
produce:pandoraWriter:StandardPandora 0.115415 0.172248 0.256418 0.158518 0.0390616 30
produce:pandoraTrack:LArPandoraTrackCreation 0.979873 2.23626 6.54698 1.86555 1.17506 30
produce:pandoraShower:LArPandoraShowerCreation 0.840289 2.40906 7.67599 1.81589 1.47094 30
produce:pandoracalo:Calorimetry 0.434625 0.902603 2.32055 0.80075 0.389186 30
produce:pandoracalonosce:Calorimetry 0.38304 0.833653 2.30028 0.734466 0.387871 30
produce:pandorapid:Chi2ParticleID 0.00143615 0.00361235 0.0509778 0.00191896 0.00880793 30
produce:pandoraShowercalo:ShowerCalorimetry 0.624173 2.03628 5.17555 1.65028 1.12875 30
produce:pandoraShowercalonosce:ShowerCalorimetry 0.620101 1.99535 4.87716 1.60196 1.08256 30
produce:pandoraGnocchiCalo:GnocchiCalorimetry 0.0994387 0.1397 0.194661 0.143878 0.0284368 30
produce:pandoraGnocchiCalonosce:GnocchiCalorimetry 0.0374126 0.0550548 0.0788652 0.0555022 0.0117373 30
produce:emtrkmichelid:EmTrackMichelId 13.6118 20.2545 31.6495 18.9256 4.853 30
[art]:TriggerResults:TriggerResultInserter 2.8539e-05 4.29787e-05 7.856e-05 3.9716e-05 1.17072e-05 30
end_path:out1:RootOutput 7.294e-06 1.01713e-05 2.3479e-05 8.9005e-06 3.26749e-06 30
end_path:michelremoving:michelremoving 0.0285859 0.109756 0.400231 0.0992002 0.0818677 30
end_path:michelremovinggnocchi:michelremoving 0.0264319 0.107432 0.397204 0.0948282 0.084381 30
end_path:calibanagnocchi:CalibAnaTree 0.28976 0.693415 2.87338 0.575508 0.459484 30
end_path:out1:RootOutput(write) 0.551039 0.858363 1.54497 0.808544 0.225277 30
========================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 6089.53 MB
Peak resident set size usage (VmHWM): 2962.01 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np04hd_raw_run029107_0837_dataflow5_datawriter_0_20240907T042622_reco_stage1_reco_stage2_20240926T150903_keepup.root
\tHists: np04hd_raw_run029107_0837_dataflow5_datawriter_0_20240907T042622_reco_stage1_reco_stage2_20240926T150903_keepup_hists.root
Forming reco metadata
/cvmfs/larsoft.opensciencegrid.org/products/python/v3_9_15/Linux64bit+3.10-2.17/bin/python: No module named meta_maker
Error in reco metadata
