Jobsub ID 79070.1@justin-prod-sched02.dune.hep.ac.uk
| Jobsub ID | 79070.1@justin-prod-sched02.dune.hep.ac.uk |
| Workflow ID | 3460 |
| Stage ID | 1 |
| User name | calcuttj@fnal.gov |
| HTCondor Group | group_dune.prod_mcsim |
| Requested | Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 18000 (5 hours) |
| Submitted time | 2024-09-26 14:20:22 |
| Site | UK_RAL-Tier1 |
| Entry | LIGO_UK_RAL_arc_ce02 |
| Last heartbeat | 2024-09-26 15:29:50 |
| From worker node | Hostname | dune001-2654103.0-lcg2597.gridpp.rl.ac.uk |
| cpuinfo | AMD EPYC 7763 64-Core Processor |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 216000 (60 hours) |
| Inner Apptainer? | True |
| Job state | jobscript_error |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk |
| Started | 2024-09-26 14:20:57 |
| Input files | hd-protodune:np04hd_raw_run029218_0090_dataflow2_datawriter_0_20240917T164733.hdf5
|
| Jobscript | Exit code | 1 |
| Real time | 0m (0s) |
| CPU time | 0m (0s = 0%) |
| Outputting started | |
| Output files | |
| Finished | 2024-09-26 15:29:50 |
| Saved logs | justin-logs:79070.1-justin-prod-sched02.dune.hep.ac.uk.logs.tgz |
| List job events Wrapper job log |
Jobscript log (last 10,000 characters)
hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 2955 XUs and 2084 XVs -> 42 XUVs
C:0 T:2 13909 XUs and 15034 XVs -> 318 XUVs
C:0 T:3 49 XUs and 54 XVs -> 0 XUVs
C:0 T:4 288 XUs and 305 XVs -> 8 XUVs
C:0 T:5 44193 XUs and 27920 XVs -> 2507 XUVs
C:0 T:6 16319 XUs and 52921 XVs -> 1866 XUVs
C:0 T:7 320 XUs and 371 XVs -> 43 XUVs
4784 XUVs total
1529 collection wire objects
4784 potential space points
Neighbour search...
279944 tests to find 163692 neighbours
Iterating with no regularization...
Begin: 1.93303e+07
0 1.75883e+07
1 1.74316e+07
2 1.74136e+07
3 1.74045e+07
Now with regularization...
Begin: 1.67186e+07
0 1.6714e+07
BdtBeamParticleIdTool::SliceFeatures::GetLArTPCIntercepts - inconsistent number of intercepts between a line and the LArTPC
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Begin processing the 30th record. run: 29218 subRun: 1 event: 22570 at 26-Sep-2024 15:26:07 UTC
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 67610 traces, 67610 summary
Retagger: tagging trace set: gauss with 49205 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 49205 traces tagged "gauss"
FrameSaver: q=1.13781e+06 n=895397 tag=gauss
wclsFrameSaver: saving 67610 traces tagged "wiener"
FrameSaver: q=1.24631e+06 n=816513 tag=wiener
3691 hits in TPC 0
3153 hits in TPC 1
5177 hits in TPC 2
3084 hits in TPC 3
5165 hits in TPC 4
8507 hits in TPC 5
3847 hits in TPC 6
2975 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 1027 XUs and 2294 XVs -> 411 XUVs
C:0 T:2 18461 XUs and 13350 XVs -> 1191 XUVs
C:0 T:3 394 XUs and 417 XVs -> 22 XUVs
C:0 T:4 602 XUs and 478 XVs -> 19 XUVs
C:0 T:5 48018 XUs and 42616 XVs -> 2672 XUVs
C:0 T:6 6937 XUs and 13575 XVs -> 1402 XUVs
C:0 T:7 72 XUs and 134 XVs -> 2 XUVs
5719 XUVs total
1821 collection wire objects
5719 potential space points
Neighbour search...
228843 tests to find 142740 neighbours
Iterating with no regularization...
Begin: 1.42017e+07
0 1.19189e+07
1 1.17771e+07
2 1.17569e+07
3 1.1752e+07
Now with regularization...
Begin: 1.08257e+07
0 1.08228e+07
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Begin processing the 31st record. run: 29218 subRun: 1 event: 22578 at 26-Sep-2024 15:27:49 UTC
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 69328 traces, 69328 summary
Retagger: tagging trace set: gauss with 52997 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 52997 traces tagged "gauss"
FrameSaver: q=1.25657e+06 n=967307 tag=gauss
wclsFrameSaver: saving 69328 traces tagged "wiener"
FrameSaver: q=1.37024e+06 n=884753 tag=wiener
2980 hits in TPC 0
3738 hits in TPC 1
7210 hits in TPC 2
5790 hits in TPC 3
3467 hits in TPC 4
5996 hits in TPC 5
4382 hits in TPC 6
2599 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 4274 XUs and 4640 XVs -> 201 XUVs
C:0 T:2 44420 XUs and 29395 XVs -> 1467 XUVs
C:0 T:3 68 XUs and 107 XVs -> 6 XUVs
C:0 T:4 216 XUs and 270 XVs -> 32 XUVs
C:0 T:5 28978 XUs and 15099 XVs -> 875 XUVs
C:0 T:6 17835 XUs and 8065 XVs -> 301 XUVs
C:0 T:7 405 XUs and 378 XVs -> 22 XUVs
2904 XUVs total
1760 collection wire objects
2904 potential space points
Neighbour search...
55854 tests to find 31800 neighbours
Iterating with no regularization...
Begin: 1.72949e+07
0 1.6586e+07
1 1.65497e+07
2 1.65468e+07
Now with regularization...
Begin: 1.54656e+07
0 1.54644e+07
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
26-Sep-2024 15:29:25 UTC Closed output file "np04hd_raw_run029218_0090_dataflow2_datawriter_0_20240917T164733_reco_stage1_reco_stage2_20240926T152925_keepup.root"
26-Sep-2024 15:29:25 UTC Closed input file "np04hd_raw_run029218_0090_dataflow2_datawriter_0_20240917T164733_reco_stage1.root"
========================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
========================================================================================================================================
Full event 57.4453 87.1506 118.466 86.7693 12.9777 31
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.000560382 0.00235848 0.00427756 0.0021845 0.00109638 31
produce:fembfilter:PDHDFEMBFilter 0.527851 0.759245 0.832586 0.780798 0.074702 31
produce:wclsdatahd:WireCellToolkit 32.5974 38.1941 43.0721 38.2241 2.39412 31
produce:gaushit:GausHitFinder 0.33823 0.73587 1.00938 0.77932 0.181526 31
produce:nhitsfilter:NumberOfHitsFilter 0.000218165 0.000349451 0.000541882 0.000345208 7.29112e-05 31
produce:reco3d:SpacePointSolver 0.0968887 0.73924 2.2818 0.631178 0.476079 31
produce:hitpdune:DisambigFromSpacePoints 0.0943075 0.555635 0.951974 0.577722 0.233353 31
produce:pandora:StandardPandora 4.08763 11.45 20.913 11.4255 4.07173 31
produce:pandoraWriter:StandardPandora 0.0994633 0.157167 0.247919 0.156536 0.0377462 31
produce:pandoraTrack:LArPandoraTrackCreation 0.466836 1.05233 1.89564 0.997157 0.394729 31
produce:pandoraShower:LArPandoraShowerCreation 0.497908 1.22406 2.26147 1.17675 0.45301 31
produce:pandoracalo:Calorimetry 0.287642 0.572775 0.973549 0.548468 0.180407 31
produce:pandoracalonosce:Calorimetry 0.241591 0.522706 0.865717 0.487616 0.170975 31
produce:pandorapid:Chi2ParticleID 0.0012883 0.00218002 0.00377631 0.00217414 0.000502682 31
produce:pandoraShowercalo:ShowerCalorimetry 0.451207 1.13925 2.14241 1.09747 0.432134 31
produce:pandoraShowercalonosce:ShowerCalorimetry 0.416165 1.11321 2.19615 1.04487 0.419509 31
produce:pandoraGnocchiCalo:GnocchiCalorimetry 0.0851562 0.159207 0.258635 0.167166 0.0463048 31
produce:pandoraGnocchiCalonosce:GnocchiCalorimetry 0.0276163 0.0534503 0.0890979 0.0515905 0.0158578 31
produce:emtrkmichelid:EmTrackMichelId 15.9179 27.1157 38.9721 28.1599 5.02467 31
[art]:TriggerResults:TriggerResultInserter 2.0521e-05 4.59881e-05 0.000110132 4.2831e-05 1.73994e-05 31
end_path:out1:RootOutput 4.51e-06 9.49235e-06 3.0111e-05 8.83e-06 4.40078e-06 31
end_path:michelremoving:michelremoving 0.0143364 0.109914 0.416499 0.0367278 0.0998176 31
end_path:michelremovinggnocchi:michelremoving 0.0136394 0.108219 0.413656 0.0316112 0.100033 31
end_path:calibanagnocchi:CalibAnaTree 0.171282 0.414638 0.775084 0.421573 0.148268 31
end_path:out1:RootOutput(write) 0.412052 0.838945 1.29624 0.870602 0.208896 31
========================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 4932.52 MB
Peak resident set size usage (VmHWM): 2579.12 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np04hd_raw_run029218_0090_dataflow2_datawriter_0_20240917T164733_reco_stage1_reco_stage2_20240926T152925_keepup.root
\tHists: np04hd_raw_run029218_0090_dataflow2_datawriter_0_20240917T164733_reco_stage1_reco_stage2_20240926T152925_keepup_hists.root
Forming reco metadata
/cvmfs/larsoft.opensciencegrid.org/products/python/v3_9_15/Linux64bit+3.10-2.17/bin/python: No module named meta_maker
Error in reco metadata
