Jobsub ID 263283.122@justin-prod-sched01.dune.hep.ac.uk
| Jobsub ID | 263283.122@justin-prod-sched01.dune.hep.ac.uk |
| Workflow ID | 3447 |
| Stage ID | 1 |
| User name | calcuttj@fnal.gov |
| HTCondor Group | group_dune.prod_mcsim |
| Requested | Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 18000 (5 hours) |
| Submitted time | 2024-09-26 09:56:54 |
| Site | FR_CCIN2P3 |
| Entry | DUNE_FR_CCIN2P3_cccondorce02 |
| Last heartbeat | 2024-09-26 11:16:00 |
| From worker node | Hostname | ccwcondor0636 |
| cpuinfo | AMD EPYC 7453 28-Core Processor |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 106200 (29 hours) |
| Inner Apptainer? | True |
| Job state | jobscript_error |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk |
| Started | 2024-09-26 09:59:13 |
| Input files | hd-protodune:np04hd_raw_run029081_0885_dataflow2_datawriter_0_20240905T064140.hdf5
|
| Jobscript | Exit code | 1 |
| Real time | 0m (0s) |
| CPU time | 0m (0s = 0%) |
| Outputting started | |
| Output files | |
| Finished | 2024-09-26 11:16:00 |
| Saved logs | justin-logs:263283.122-justin-prod-sched01.dune.hep.ac.uk.logs.tgz |
| List job events Wrapper job log |
Jobscript log (last 10,000 characters)
dences...
C:0 T:0 8 XUs and 2 XVs -> 0 XUVs
C:0 T:1 98 XUs and 60 XVs -> 2 XUVs
C:0 T:2 31328 XUs and 29399 XVs -> 932 XUVs
C:0 T:3 126 XUs and 92 XVs -> 3 XUVs
C:0 T:4 24 XUs and 33 XVs -> 0 XUVs
C:0 T:5 5811 XUs and 4774 XVs -> 444 XUVs
C:0 T:6 102893 XUs and 354723 XVs -> 23536 XUVs
C:0 T:7 311 XUs and 531 XVs -> 22 XUVs
24939 XUVs total
1570 collection wire objects
24939 potential space points
Neighbour search...
4187309 tests to find 1617566 neighbours
Iterating with no regularization...
Begin: 2.42686e+07
0 2.04669e+07
1 2.03169e+07
2 2.02887e+07
3 2.02794e+07
Now with regularization...
Begin: 1.9668e+07
0 1.96614e+07
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
no of hits closeby 0 event 212618 TrkackID 11 startx, y, z 187.976 229.303 55.8971 wireno, peakt tpcno 88 2395.93 2 dist 32.0237min T, max_T4.29562 2411.87
no of hits closeby 0 event 212618 TrkackID 11 startx, y, z 187.976 229.303 55.8971 wireno, peakt tpcno 88 2395.93 2 dist 32.0237min T, max_T4.29562 2411.87
Begin processing the 29th record. run: 29081 subRun: 1 event: 212626 at 26-Sep-2024 13:02:20 CEST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 61733 traces, 61733 summary
Retagger: tagging trace set: gauss with 44879 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 44879 traces tagged "gauss"
FrameSaver: q=992535 n=837269 tag=gauss
wclsFrameSaver: saving 61733 traces tagged "wiener"
FrameSaver: q=1.06279e+06 n=758084 tag=wiener
1789 hits in TPC 0
2175 hits in TPC 1
7950 hits in TPC 2
4555 hits in TPC 3
1689 hits in TPC 4
3658 hits in TPC 5
6770 hits in TPC 6
3444 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:0 17 XUs and 15 XVs -> 0 XUVs
C:0 T:1 11 XUs and 9 XVs -> 1 XUVs
C:0 T:2 39082 XUs and 51936 XVs -> 2979 XUVs
C:0 T:3 624 XUs and 711 XVs -> 66 XUVs
C:0 T:4 6 XUs and 21 XVs -> 0 XUVs
C:0 T:5 7019 XUs and 5926 XVs -> 710 XUVs
C:0 T:6 48959 XUs and 64845 XVs -> 1858 XUVs
C:0 T:7 388 XUs and 438 XVs -> 8 XUVs
5622 XUVs total
1406 collection wire objects
5622 potential space points
Neighbour search...
484166 tests to find 256448 neighbours
Iterating with no regularization...
Begin: 5.83712e+06
0 5.07628e+06
1 5.00738e+06
2 4.99579e+06
3 4.99206e+06
Now with regularization...
Begin: 4.57787e+06
0 4.57531e+06
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Begin processing the 30th record. run: 29081 subRun: 1 event: 212634 at 26-Sep-2024 13:03:27 CEST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 65446 traces, 65446 summary
Retagger: tagging trace set: gauss with 46871 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 46871 traces tagged "gauss"
FrameSaver: q=1.05444e+06 n=858685 tag=gauss
wclsFrameSaver: saving 65446 traces tagged "wiener"
FrameSaver: q=1.15288e+06 n=780636 tag=wiener
3027 hits in TPC 0
2454 hits in TPC 1
6610 hits in TPC 2
4206 hits in TPC 3
3175 hits in TPC 4
5260 hits in TPC 5
6173 hits in TPC 6
2807 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 110 XUs and 180 XVs -> 8 XUVs
C:0 T:2 26284 XUs and 39003 XVs -> 2032 XUVs
C:0 T:3 198 XUs and 281 XVs -> 10 XUVs
C:0 T:4 27 XUs and 24 XVs -> 2 XUVs
C:0 T:5 13220 XUs and 22651 XVs -> 494 XUVs
C:0 T:6 91142 XUs and 95108 XVs -> 11011 XUVs
C:0 T:7 62 XUs and 35 XVs -> 1 XUVs
13558 XUVs total
2221 collection wire objects
13558 potential space points
Neighbour search...
1434424 tests to find 693006 neighbours
Iterating with no regularization...
Begin: 1.17347e+07
0 9.27004e+06
1 9.03095e+06
2 8.98386e+06
3 8.96563e+06
4 8.9558e+06
5 8.94996e+06
Now with regularization...
Begin: 8.21165e+06
0 8.20583e+06
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
26-Sep-2024 13:04:40 CEST Closed output file "np04hd_raw_run029081_0885_dataflow2_datawriter_0_20240905T064140_reco_stage1_reco_stage2_20240926T110440_keepup.root"
26-Sep-2024 13:04:40 CEST Closed input file "np04hd_raw_run029081_0885_dataflow2_datawriter_0_20240905T064140_reco_stage1.root"
========================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
========================================================================================================================================
Full event 61.1991 95.3787 233.563 92.9887 31.2985 30
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.000589393 0.00272611 0.00490744 0.00271393 0.00107468 30
produce:fembfilter:PDHDFEMBFilter 0.489719 0.620076 0.849037 0.613394 0.103961 30
produce:wclsdatahd:WireCellToolkit 30.663 37.3629 44.889 37.1943 4.67105 30
produce:gaushit:GausHitFinder 0.371065 0.664689 1.2975 0.649638 0.199942 30
produce:nhitsfilter:NumberOfHitsFilter 0.00020299 0.000330053 0.000515826 0.000320289 7.39753e-05 30
produce:reco3d:SpacePointSolver 0.20365 1.4108 10.7574 0.859719 2.02523 30
produce:hitpdune:DisambigFromSpacePoints 0.23849 0.578033 2.79927 0.467651 0.449089 30
produce:pandora:StandardPandora 5.16852 14.7608 87.0507 11.3385 14.4549 30
produce:pandoraWriter:StandardPandora 0.0763106 0.1529 0.298085 0.145167 0.050712 30
produce:pandoraTrack:LArPandoraTrackCreation 0.55166 1.5711 6.45156 1.34619 1.06462 30
produce:pandoraShower:LArPandoraShowerCreation 0.477773 2.02879 6.33046 1.84912 1.2424 30
produce:pandoracalo:Calorimetry 0.274943 0.809194 2.88543 0.674989 0.539763 30
produce:pandoracalonosce:Calorimetry 0.26782 0.716711 2.7389 0.644287 0.446711 30
produce:pandorapid:Chi2ParticleID 0.000955658 0.00182037 0.00395036 0.00166843 0.00064335 30
produce:pandoraShowercalo:ShowerCalorimetry 0.405932 1.74701 5.20768 1.57676 0.975762 30
produce:pandoraShowercalonosce:ShowerCalorimetry 0.452635 1.67728 5.13563 1.64572 0.951872 30
produce:pandoraGnocchiCalo:GnocchiCalorimetry 0.0530426 0.13591 0.225401 0.115618 0.0477375 30
produce:pandoraGnocchiCalonosce:GnocchiCalorimetry 0.0240436 0.047506 0.0789778 0.0450141 0.0155127 30
produce:emtrkmichelid:EmTrackMichelId 17.2271 29.5175 56.7593 29.191 8.42612 30
[art]:TriggerResults:TriggerResultInserter 2.7471e-05 4.86165e-05 8.1192e-05 4.60415e-05 1.4161e-05 30
end_path:out1:RootOutput 6.943e-06 1.33413e-05 2.614e-05 1.2453e-05 4.20117e-06 30
end_path:michelremoving:michelremoving 0.0160289 0.0722542 0.265962 0.0361622 0.0625298 30
end_path:michelremovinggnocchi:michelremoving 0.0133579 0.0664516 0.236552 0.0367075 0.0566286 30
end_path:calibanagnocchi:CalibAnaTree 0.21833 0.545739 2.26832 0.467625 0.363244 30
end_path:out1:RootOutput(write) 0.387112 0.696473 1.39647 0.687302 0.200635 30
========================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 4958.95 MB
Peak resident set size usage (VmHWM): 2312.61 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np04hd_raw_run029081_0885_dataflow2_datawriter_0_20240905T064140_reco_stage1_reco_stage2_20240926T110440_keepup.root
\tHists: np04hd_raw_run029081_0885_dataflow2_datawriter_0_20240905T064140_reco_stage1_reco_stage2_20240926T110440_keepup_hists.root
Forming reco metadata
/cvmfs/larsoft.opensciencegrid.org/products/python/v3_9_15/Linux64bit+3.10-2.17/bin/python: No module named meta_maker
Error in reco metadata
