Jobsub ID 263681.153@justin-prod-sched01.dune.hep.ac.uk
| Jobsub ID | 263681.153@justin-prod-sched01.dune.hep.ac.uk |
| Workflow ID | 3451 |
| Stage ID | 1 |
| User name | calcuttj@fnal.gov |
| HTCondor Group | group_dune.prod_mcsim |
| Requested | Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 18000 (5 hours) |
| Submitted time | 2024-09-26 13:07:22 |
| Site | ES_PIC |
| Entry | DUNE_T1_ES_PIC_ce14-multicore |
| Last heartbeat | 2024-09-26 14:30:51 |
| From worker node | Hostname | tds207.pic.es |
| cpuinfo | Intel(R) Xeon(R) CPU E5-2640 v3 @ 2.60GHz |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 216000 (60 hours) |
| Inner Apptainer? | True |
| Job state | jobscript_error |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk |
| Started | 2024-09-26 13:09:00 |
| Input files | hd-protodune:np04hd_raw_run029107_2174_dataflow1_datawriter_0_20240908T025850.hdf5
|
| Jobscript | Exit code | 1 |
| Real time | 0m (0s) |
| CPU time | 0m (0s = 0%) |
| Outputting started | |
| Output files | |
| Finished | 2024-09-26 14:30:51 |
| Saved logs | justin-logs:263681.153-justin-prod-sched01.dune.hep.ac.uk.logs.tgz |
| List job events Wrapper job log |
Jobscript log (last 10,000 characters)
06
0 9.13227e+06
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Begin processing the 29th record. run: 29107 subRun: 1 event: 522009 at 26-Sep-2024 16:26:05 CEST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 99182 traces, 99182 summary
Retagger: tagging trace set: gauss with 75600 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 75600 traces tagged "gauss"
FrameSaver: q=1.63698e+06 n=1426912 tag=gauss
wclsFrameSaver: saving 99182 traces tagged "wiener"
FrameSaver: q=1.75971e+06 n=1294373 tag=wiener
5473 hits in TPC 0
4573 hits in TPC 1
6791 hits in TPC 2
4659 hits in TPC 3
9507 hits in TPC 4
12062 hits in TPC 5
7165 hits in TPC 6
4510 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 2117 XUs and 1940 XVs -> 34 XUVs
C:0 T:2 26879 XUs and 26659 XVs -> 508 XUVs
C:0 T:3 41 XUs and 66 XVs -> 0 XUVs
C:0 T:4 517 XUs and 495 XVs -> 8 XUVs
C:0 T:5 75497 XUs and 104520 XVs -> 4490 XUVs
C:0 T:6 30814 XUs and 20331 XVs -> 491 XUVs
C:0 T:7 30 XUs and 48 XVs -> 0 XUVs
5531 XUVs total
2352 collection wire objects
5531 potential space points
Neighbour search...
279969 tests to find 142286 neighbours
Iterating with no regularization...
Begin: 1.30395e+07
0 1.14512e+07
1 1.13154e+07
2 1.13003e+07
3 1.12963e+07
Now with regularization...
Begin: 1.06436e+07
0 1.06407e+07
BdtBeamParticleIdTool::SliceFeatures::GetLArTPCIntercepts - inconsistent number of intercepts between a line and the LArTPC
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
no of hits closeby 0 event 522009 TrkackID 8 startx, y, z 128.75 486.026 2.93755 wireno, peakt tpcno 192 3560.07 2 dist 373min T, max_T5.57986 3598.06
no of hits closeby 0 event 522009 TrkackID 8 startx, y, z 128.75 486.026 2.93755 wireno, peakt tpcno 192 3560.07 2 dist 373min T, max_T5.57986 3598.06
Begin processing the 30th record. run: 29107 subRun: 1 event: 522017 at 26-Sep-2024 16:28:21 CEST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 60301 traces, 60301 summary
Retagger: tagging trace set: gauss with 43272 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 43272 traces tagged "gauss"
FrameSaver: q=930144 n=775273 tag=gauss
wclsFrameSaver: saving 60301 traces tagged "wiener"
FrameSaver: q=1.0051e+06 n=698997 tag=wiener
4323 hits in TPC 0
3497 hits in TPC 1
2779 hits in TPC 2
1762 hits in TPC 3
4526 hits in TPC 4
7464 hits in TPC 5
3523 hits in TPC 6
1506 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 3383 XUs and 5197 XVs -> 711 XUVs
C:0 T:2 5709 XUs and 4296 XVs -> 86 XUVs
C:0 T:3 60 XUs and 87 XVs -> 4 XUVs
C:0 T:4 94 XUs and 105 XVs -> 4 XUVs
C:0 T:5 20749 XUs and 29308 XVs -> 3788 XUVs
C:0 T:6 23011 XUs and 9724 XVs -> 228 XUVs
C:0 T:7 32 XUs and 82 XVs -> 3 XUVs
4824 XUVs total
1225 collection wire objects
4824 potential space points
Neighbour search...
222730 tests to find 140282 neighbours
Iterating with no regularization...
Begin: 4.23122e+06
0 3.73426e+06
1 3.66553e+06
2 3.65214e+06
3 3.64759e+06
4 3.64541e+06
Now with regularization...
Begin: 3.07956e+06
0 3.07787e+06
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
no of hits closeby 0 event 522017 TrkackID 12 startx, y, z -68.1304 292.46 374.129 wireno, peakt tpcno 325 122.232 6 dist 0.423538min T, max_T5 905.927
no of hits closeby 0 event 522017 TrkackID 12 startx, y, z -68.1304 292.46 374.129 wireno, peakt tpcno 325 122.232 6 dist 0.423538min T, max_T5 905.927
26-Sep-2024 16:29:36 CEST Closed output file "np04hd_raw_run029107_2174_dataflow1_datawriter_0_20240908T025850_reco_stage1_reco_stage2_20240926T142936_keepup.root"
26-Sep-2024 16:29:36 CEST Closed input file "np04hd_raw_run029107_2174_dataflow1_datawriter_0_20240908T025850_reco_stage1.root"
========================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
========================================================================================================================================
Full event 68.4675 92.7899 135.765 91.687 15.1543 30
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.000411551 0.00167089 0.0026974 0.00192575 0.00068438 30
produce:fembfilter:PDHDFEMBFilter 0.606775 0.627034 0.655651 0.628559 0.0126376 30
produce:wclsdatahd:WireCellToolkit 40.0369 45.274 48.865 44.6619 2.06765 30
produce:gaushit:GausHitFinder 0.424637 0.753874 1.25309 0.731296 0.179178 30
produce:nhitsfilter:NumberOfHitsFilter 0.000182222 0.000323262 0.000635311 0.000302705 0.000110756 30
produce:reco3d:SpacePointSolver 0.12393 0.611932 1.77237 0.531619 0.368571 30
produce:hitpdune:DisambigFromSpacePoints 0.142562 0.48542 1.41075 0.461227 0.254687 30
produce:pandora:StandardPandora 3.66802 13.0685 32.1581 11.3565 5.89336 30
produce:pandoraWriter:StandardPandora 0.073218 0.143048 0.236907 0.13951 0.0370799 30
produce:pandoraTrack:LArPandoraTrackCreation 0.426806 1.20301 2.73394 1.13234 0.486629 30
produce:pandoraShower:LArPandoraShowerCreation 0.33732 1.41179 3.59604 1.28229 0.701625 30
produce:pandoracalo:Calorimetry 0.289578 0.626535 1.33571 0.582579 0.231977 30
produce:pandoracalonosce:Calorimetry 0.259916 0.572943 1.24243 0.52014 0.215757 30
produce:pandorapid:Chi2ParticleID 0.00102329 0.00174794 0.00322016 0.00165612 0.000423099 30
produce:pandoraShowercalo:ShowerCalorimetry 0.429921 1.36442 3.67313 1.24942 0.726971 30
produce:pandoraShowercalonosce:ShowerCalorimetry 0.305721 1.31669 3.55657 1.21026 0.708348 30
produce:pandoraGnocchiCalo:GnocchiCalorimetry 0.0734239 0.129365 0.19115 0.130431 0.0276975 30
produce:pandoraGnocchiCalonosce:GnocchiCalorimetry 0.0243632 0.044739 0.0679925 0.0452484 0.0101448 30
produce:emtrkmichelid:EmTrackMichelId 12.348 23.6875 36.4075 23.1307 5.67207 30
[art]:TriggerResults:TriggerResultInserter 2.0513e-05 2.84884e-05 7.4462e-05 2.29575e-05 1.36585e-05 30
end_path:out1:RootOutput 3.991e-06 1.00369e-05 2.5386e-05 9.1875e-06 4.19294e-06 30
end_path:michelremoving:michelremoving 0.0170611 0.0920871 0.260421 0.0432826 0.0718012 30
end_path:michelremovinggnocchi:michelremoving 0.0162285 0.0902036 0.249503 0.040862 0.0705038 30
end_path:calibanagnocchi:CalibAnaTree 0.156726 0.47605 1.04908 0.430647 0.186692 30
end_path:out1:RootOutput(write) 0.397591 0.734227 1.20724 0.732633 0.165063 30
========================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 4942.42 MB
Peak resident set size usage (VmHWM): 2603.45 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np04hd_raw_run029107_2174_dataflow1_datawriter_0_20240908T025850_reco_stage1_reco_stage2_20240926T142936_keepup.root
\tHists: np04hd_raw_run029107_2174_dataflow1_datawriter_0_20240908T025850_reco_stage1_reco_stage2_20240926T142936_keepup_hists.root
Forming reco metadata
/cvmfs/larsoft.opensciencegrid.org/products/python/v3_9_15/Linux64bit+3.10-2.17/bin/python: No module named meta_maker
Error in reco metadata
