Jobsub ID 261577.185@justin-prod-sched01.dune.hep.ac.uk
| Jobsub ID | 261577.185@justin-prod-sched01.dune.hep.ac.uk |
| Workflow ID | 3390 |
| Stage ID | 1 |
| User name | calcuttj@fnal.gov |
| HTCondor Group | group_dune.prod_mcsim |
| Requested | Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 18000 (5 hours) |
| Submitted time | 2024-09-24 19:57:05 |
| Site | FR_CCIN2P3 |
| Entry | DUNE_FR_CCIN2P3_cccondorce04 |
| Last heartbeat | 2024-09-24 23:20:08 |
| From worker node | Hostname | ccwcondor0567 |
| cpuinfo | AMD EPYC 7453 28-Core Processor |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 106200 (29 hours) |
| Inner Apptainer? | True |
| Job state | stalled |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk |
| Started | 2024-09-24 22:10:06 |
| Input files | hd-protodune:np04hd_raw_run029006_2993_dataflow1_datawriter_0_20240901T174515.hdf5
|
| Outputting started | 2024-09-24 23:18:48 |
| Output files | |
| Finished | 2024-09-24 23:48:27 |
| List job events Wrapper job log |
Jobscript log (last 10,000 characters)
auss with 51502 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 51502 traces tagged "gauss"
FrameSaver: q=1.13957e+06 n=942433 tag=gauss
wclsFrameSaver: saving 69053 traces tagged "wiener"
FrameSaver: q=1.22205e+06 n=852847 tag=wiener
1662 hits in TPC 0
1584 hits in TPC 1
8088 hits in TPC 2
5956 hits in TPC 3
4183 hits in TPC 4
6030 hits in TPC 5
5787 hits in TPC 6
2944 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 21 XUs and 13 XVs -> 0 XUVs
C:0 T:2 40533 XUs and 55840 XVs -> 871 XUVs
C:0 T:3 828 XUs and 874 XVs -> 20 XUVs
C:0 T:4 651 XUs and 1234 XVs -> 40 XUVs
C:0 T:5 18720 XUs and 33049 XVs -> 3396 XUVs
C:0 T:6 19931 XUs and 25282 XVs -> 530 XUVs
C:0 T:7 56 XUs and 67 XVs -> 0 XUVs
4857 XUVs total
1227 collection wire objects
4857 potential space points
Neighbour search...
727015 tests to find 347668 neighbours
Iterating with no regularization...
Begin: 1.20543e+07
0 1.03801e+07
1 1.01996e+07
2 1.01854e+07
3 1.01819e+07
Now with regularization...
Begin: 9.60097e+06
0 9.59696e+06
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
25-Sep-2024 01:18:36 CEST Closed output file "np04hd_raw_run029006_2993_dataflow1_datawriter_0_20240901T174515_reco_stage1_reco_stage2_20240924T231836_keepup.root"
25-Sep-2024 01:18:36 CEST Closed input file "np04hd_raw_run029006_2993_dataflow1_datawriter_0_20240901T174515_reco_stage1.root"
========================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
========================================================================================================================================
Full event 64.3941 102.641 164.599 100.245 22.9276 30
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.000522228 0.00296601 0.00530718 0.00314479 0.0012805 30
produce:fembfilter:PDHDFEMBFilter 0.589911 0.675665 0.765258 0.680424 0.044034 30
produce:wclsdatahd:WireCellToolkit 35.654 40.0293 46.0428 39.9308 1.95601 30
produce:gaushit:GausHitFinder 0.395101 0.769863 1.26996 0.73755 0.229221 30
produce:nhitsfilter:NumberOfHitsFilter 0.000213109 0.000357334 0.000751317 0.000343934 9.72608e-05 30
produce:reco3d:SpacePointSolver 0.152579 1.36743 4.18506 1.03662 0.98665 30
produce:hitpdune:DisambigFromSpacePoints 0.174268 0.600058 1.5645 0.542933 0.330202 30
produce:pandora:StandardPandora 4.07105 15.072 32.4726 13.1457 7.21301 30
produce:pandoraWriter:StandardPandora 0.0703773 0.163856 0.253894 0.17076 0.0468178 30
produce:pandoraTrack:LArPandoraTrackCreation 0.665842 1.92946 4.19891 1.80362 0.828343 30
produce:pandoraShower:LArPandoraShowerCreation 0.395829 2.47247 6.30279 2.17376 1.50023 30
produce:pandoracalo:Calorimetry 0.293123 0.977068 2.58918 0.914044 0.4952 30
produce:pandoracalonosce:Calorimetry 0.241406 0.862706 2.34388 0.801045 0.44398 30
produce:pandorapid:Chi2ParticleID 0.00113834 0.0022943 0.00516503 0.00222844 0.000853777 30
produce:pandoraShowercalo:ShowerCalorimetry 0.329224 2.09292 5.67934 1.74189 1.29837 30
produce:pandoraShowercalonosce:ShowerCalorimetry 0.315038 2.05698 5.78379 1.71266 1.36116 30
produce:pandoraGnocchiCalo:GnocchiCalorimetry 0.0667528 0.143583 0.239851 0.142896 0.0443976 30
produce:pandoraGnocchiCalonosce:GnocchiCalorimetry 0.0236929 0.0524037 0.0863116 0.0487112 0.016358 30
produce:emtrkmichelid:EmTrackMichelId 17.623 31.4928 50.9571 31.4059 7.72966 30
[art]:TriggerResults:TriggerResultInserter 2.688e-05 6.6494e-05 0.000140093 6.5107e-05 2.20256e-05 30
end_path:out1:RootOutput 9.528e-06 1.76667e-05 4.4543e-05 1.6947e-05 6.33697e-06 30
end_path:michelremoving:michelremoving 0.0202629 0.102327 0.487163 0.0516332 0.107913 30
end_path:michelremovinggnocchi:michelremoving 0.019458 0.105942 0.500877 0.0511775 0.113332 30
end_path:calibanagnocchi:CalibAnaTree 0.208419 0.663155 1.34914 0.558955 0.308745 30
end_path:out1:RootOutput(write) 0.381256 0.791071 1.23157 0.801619 0.209632 30
========================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 4962.62 MB
Peak resident set size usage (VmHWM): 2384.74 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np04hd_raw_run029006_2993_dataflow1_datawriter_0_20240901T174515_reco_stage1_reco_stage2_20240924T231836_keepup.root
\tHists: np04hd_raw_run029006_2993_dataflow1_datawriter_0_20240901T174515_reco_stage1_reco_stage2_20240924T231836_keepup_hists.root
Forming reco metadata
Successfully opened file np04hd_raw_run029006_2993_dataflow1_datawriter_0_20240901T174515_reco_stage1_reco_stage2_20240924T231836_keepup.root
Ran successfully
{
"name": "np04hd_raw_run029006_2993_dataflow1_datawriter_0_20240901T174515_reco_stage1_reco_stage2_20240924T231836_keepup.root",
"namespace": "hd-protodune-det-reco",
"metadata": {
"core.file_format": "artroot",
"core.application.name": "reco",
"core.application.family": "dunesw",
"core.application.version": "v09_91_02d01",
"core.data_tier": "full-reconstructed",
"dune.config_file": "standard_reco_stage2_calibration_protodunehd_keepup.fcl",
"dune.campaign": "hd-protodune-reco-keepup-v0",
"core.start_time": 1727216783.0,
"core.end_time": 1727219917.0,
"core.events": [
718321,
718329,
718337,
718345,
718353,
718361,
718369,
718377,
718385,
718393,
718401,
718409,
718417,
718425,
718433,
718441,
718449,
718457,
718465,
718473,
718481,
718489,
718497,
718505,
718513,
718521,
718529,
718537,
718545,
718553
],
"core.event_count": 30,
"core.first_event_number": 718321,
"core.last_event_number": 718553,
"core.data_stream": "physics",
"core.file_content_status": "good",
"core.file_type": "detector",
"core.run_type": "hd-protodune",
"core.runs": [
29006
],
"core.runs_subruns": [
2900600001
],
"dune.daq_test": false,
"retention.status": "active",
"retention.class": "physics",
"origin.applications.config_files": {
"reco1": "standard_reco_stage1_protodunehd_keepup.fcl"
},
"origin.applications.versions": {
"reco1": "v09_91_02d01"
},
"origin.applications.names": [
"reco1"
]
},
"parents": [
{
"did": "hd-protodune:np04hd_raw_run029006_2993_dataflow1_datawriter_0_20240901T174515.hdf5"
}
]
}Forming hist metadata
formed
{
"name": "np04hd_raw_run029006_2993_dataflow1_datawriter_0_20240901T174515_reco_stage1_reco_stage2_20240924T231836_keepup_hists.root",
"namespace": "hd-protodune-det-reco",
"metadata": {
"core.file_format": "root",
"core.application.name": "reco",
"core.application.family": "dunesw",
"core.application.version": "v09_91_02d01",
"core.data_tier": "root-tuple-virtual",
"dune.config_file": "standard_reco_stage2_calibration_protodunehd_keepup.fcl",
"dune.campaign": "hd-protodune-reco-keepup-v0",
"core.start_time": 1727216783.0,
"core.end_time": 1727219917.0,
"core.data_stream": "physics",
"core.file_content_status": "good",
"core.file_type": "detector",
"core.run_type": "hd-protodune",
"core.runs": [
29006
],
"core.runs_subruns": [
2900600001
],
"dune.daq_test": false,
"retention.status": "active",
"retention.class": "physics",
"origin.applications.config_files": {
"reco1": "standard_reco_stage1_protodunehd_keepup.fcl"
},
"origin.applications.versions": {
"reco1": "v09_91_02d01"
},
"origin.applications.names": [
"reco1"
]
},
"parents": [
{
"did": "hd-protodune:np04hd_raw_run029006_2993_dataflow1_datawriter_0_20240901T174515.hdf5"
}
]
} 