Jobsub ID 263300.168@justin-prod-sched01.dune.hep.ac.uk
| Jobsub ID | 263300.168@justin-prod-sched01.dune.hep.ac.uk |
| Workflow ID | 3453 |
| Stage ID | 1 |
| User name | calcuttj@fnal.gov |
| HTCondor Group | group_dune.prod_mcsim |
| Requested | Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 18000 (5 hours) |
| Submitted time | 2024-09-26 10:00:00 |
| Site | FR_CCIN2P3 |
| Entry | DUNE_FR_CCIN2P3_cccondorce01 |
| Last heartbeat | 2024-09-26 11:29:01 |
| From worker node | Hostname | ccwcondor0532 |
| cpuinfo | AMD EPYC 7302 16-Core Processor |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 106200 (29 hours) |
| Inner Apptainer? | True |
| Job state | jobscript_error |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk |
| Started | 2024-09-26 10:09:50 |
| Input files | hd-protodune:np04hd_raw_run029107_3652_dataflow0_datawriter_0_20240909T044316.hdf5
|
| Jobscript | Exit code | 1 |
| Real time | 0m (0s) |
| CPU time | 0m (0s = 0%) |
| Outputting started | |
| Output files | |
| Finished | 2024-09-26 11:29:01 |
| Saved logs | justin-logs:263300.168-justin-prod-sched01.dune.hep.ac.uk.logs.tgz |
| List job events Wrapper job log |
Jobscript log (last 10,000 characters)
"
FrameSaver: q=1.19113e+06 n=865321 tag=wiener
3947 hits in TPC 0
2118 hits in TPC 1
5300 hits in TPC 2
3067 hits in TPC 3
4983 hits in TPC 4
8863 hits in TPC 5
6327 hits in TPC 6
3029 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 429 XUs and 410 XVs -> 42 XUVs
C:0 T:2 27358 XUs and 25576 XVs -> 1305 XUVs
C:0 T:3 198 XUs and 193 XVs -> 19 XUVs
C:0 T:4 15 XUs and 15 XVs -> 0 XUVs
C:0 T:5 35571 XUs and 25973 XVs -> 522 XUVs
C:0 T:6 30285 XUs and 35604 XVs -> 1154 XUVs
C:0 T:7 98 XUs and 110 XVs -> 6 XUVs
3048 XUVs total
1367 collection wire objects
3048 potential space points
Neighbour search...
151974 tests to find 77256 neighbours
Iterating with no regularization...
Begin: 7.55847e+06
0 6.87794e+06
1 6.83253e+06
2 6.82753e+06
Now with regularization...
Begin: 6.48972e+06
0 6.48818e+06
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Begin processing the 29th record. run: 29107 subRun: 1 event: 876736 at 26-Sep-2024 13:16:58 CEST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 49719 traces, 49719 summary
Retagger: tagging trace set: gauss with 37283 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 37283 traces tagged "gauss"
FrameSaver: q=818781 n=710694 tag=gauss
wclsFrameSaver: saving 49719 traces tagged "wiener"
FrameSaver: q=870496 n=644993 tag=wiener
2507 hits in TPC 0
1018 hits in TPC 1
6394 hits in TPC 2
3476 hits in TPC 3
2605 hits in TPC 4
3700 hits in TPC 5
4103 hits in TPC 6
3097 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 8 XUs and 4 XVs -> 1 XUVs
C:0 T:2 28875 XUs and 51682 XVs -> 928 XUVs
C:0 T:3 171 XUs and 269 XVs -> 4 XUVs
C:0 T:4 370 XUs and 531 XVs -> 9 XUVs
C:0 T:5 10718 XUs and 13139 XVs -> 292 XUVs
C:0 T:6 16214 XUs and 40340 XVs -> 1297 XUVs
C:0 T:7 26 XUs and 70 XVs -> 2 XUVs
2533 XUVs total
1206 collection wire objects
2533 potential space points
Neighbour search...
111473 tests to find 57994 neighbours
Iterating with no regularization...
Begin: 6.8462e+06
0 6.55716e+06
1 6.53823e+06
2 6.53453e+06
Now with regularization...
Begin: 6.14243e+06
0 6.14103e+06
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Begin processing the 30th record. run: 29107 subRun: 1 event: 876744 at 26-Sep-2024 13:18:15 CEST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 58850 traces, 58850 summary
Retagger: tagging trace set: gauss with 43548 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 43548 traces tagged "gauss"
FrameSaver: q=899291 n=832311 tag=gauss
wclsFrameSaver: saving 58850 traces tagged "wiener"
FrameSaver: q=965153 n=750003 tag=wiener
2783 hits in TPC 0
3326 hits in TPC 1
7309 hits in TPC 2
3523 hits in TPC 3
1417 hits in TPC 4
4659 hits in TPC 5
5453 hits in TPC 6
3014 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 1993 XUs and 1540 XVs -> 26 XUVs
C:0 T:2 20715 XUs and 30372 XVs -> 857 XUVs
C:0 T:3 979 XUs and 1227 XVs -> 14 XUVs
C:0 T:4 78 XUs and 303 XVs -> 3 XUVs
C:0 T:5 6798 XUs and 13544 XVs -> 476 XUVs
C:0 T:6 23774 XUs and 26961 XVs -> 917 XUVs
C:0 T:7 8 XUs and 14 XVs -> 0 XUVs
2293 XUVs total
1100 collection wire objects
2293 potential space points
Neighbour search...
110411 tests to find 59824 neighbours
Iterating with no regularization...
Begin: 5.05694e+06
0 4.53346e+06
1 4.5027e+06
2 4.49839e+06
Now with regularization...
Begin: 4.25433e+06
0 4.25277e+06
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
26-Sep-2024 13:19:37 CEST Closed output file "np04hd_raw_run029107_3652_dataflow0_datawriter_0_20240909T044316_reco_stage1_reco_stage2_20240926T111937_keepup.root"
26-Sep-2024 13:19:38 CEST Closed input file "np04hd_raw_run029107_3652_dataflow0_datawriter_0_20240909T044316_reco_stage1.root"
========================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
========================================================================================================================================
Full event 69.8891 98.7868 139.917 96.019 19.178 30
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.000479143 0.00336866 0.0115177 0.00287923 0.00239613 30
produce:fembfilter:PDHDFEMBFilter 0.551682 0.648383 0.790793 0.630394 0.0704606 30
produce:wclsdatahd:WireCellToolkit 36.7183 42.2263 46.4768 42.233 2.58585 30
produce:gaushit:GausHitFinder 0.388922 0.71425 1.043 0.67562 0.169521 30
produce:nhitsfilter:NumberOfHitsFilter 0.000280999 0.000505208 0.00105905 0.000478572 0.000176619 30
produce:reco3d:SpacePointSolver 0.203077 1.19622 6.94856 0.911266 1.2339 30
produce:hitpdune:DisambigFromSpacePoints 0.199966 0.508595 1.14299 0.453253 0.251522 30
produce:pandora:StandardPandora 5.67468 13.8746 33.4836 12.1813 6.82888 30
produce:pandoraWriter:StandardPandora 0.0784921 0.168048 0.264585 0.160113 0.0548521 30
produce:pandoraTrack:LArPandoraTrackCreation 0.474307 1.11261 2.03458 1.04741 0.398779 30
produce:pandoraShower:LArPandoraShowerCreation 0.515495 1.46812 3.44114 1.32388 0.728595 30
produce:pandoracalo:Calorimetry 0.249188 0.597827 1.13538 0.560221 0.205279 30
produce:pandoracalonosce:Calorimetry 0.219834 0.539826 1.02339 0.532092 0.199362 30
produce:pandorapid:Chi2ParticleID 0.0011973 0.00198066 0.00484387 0.00189449 0.000676254 30
produce:pandoraShowercalo:ShowerCalorimetry 0.429183 1.30535 3.04926 1.1755 0.609419 30
produce:pandoraShowercalonosce:ShowerCalorimetry 0.420488 1.24311 2.89628 1.14295 0.556082 30
produce:pandoraGnocchiCalo:GnocchiCalorimetry 0.0747974 0.140416 0.28012 0.130077 0.0454198 30
produce:pandoraGnocchiCalonosce:GnocchiCalorimetry 0.0271611 0.0493813 0.0869048 0.0459451 0.0161241 30
produce:emtrkmichelid:EmTrackMichelId 20.0135 31.3589 49.8199 30.8444 7.39742 30
[art]:TriggerResults:TriggerResultInserter 2.0919e-05 4.09187e-05 0.00010105 3.1309e-05 2.05826e-05 30
end_path:out1:RootOutput 6.953e-06 1.02881e-05 2.8544e-05 8.782e-06 4.63333e-06 30
end_path:michelremoving:michelremoving 0.0203283 0.113916 0.496768 0.0620978 0.11954 30
end_path:michelremovinggnocchi:michelremoving 0.0184625 0.106994 0.458603 0.0547771 0.120102 30
end_path:calibanagnocchi:CalibAnaTree 0.21778 0.428062 0.791532 0.385549 0.143095 30
end_path:out1:RootOutput(write) 0.509622 0.826349 1.31377 0.793047 0.226987 30
========================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 4919.62 MB
Peak resident set size usage (VmHWM): 2345.69 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np04hd_raw_run029107_3652_dataflow0_datawriter_0_20240909T044316_reco_stage1_reco_stage2_20240926T111937_keepup.root
\tHists: np04hd_raw_run029107_3652_dataflow0_datawriter_0_20240909T044316_reco_stage1_reco_stage2_20240926T111937_keepup_hists.root
Forming reco metadata
/cvmfs/larsoft.opensciencegrid.org/products/python/v3_9_15/Linux64bit+3.10-2.17/bin/python: No module named meta_maker
Error in reco metadata
