Jobsub ID 276366.50@justin-prod-sched01.dune.hep.ac.uk
| Jobsub ID | 276366.50@justin-prod-sched01.dune.hep.ac.uk |
| Workflow ID | 3697 |
| Stage ID | 1 |
| User name | calcuttj@fnal.gov |
| HTCondor Group | group_dune.prod_mcsim |
| Requested | Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 18000 (5 hours) |
| Submitted time | 2024-10-17 10:23:43 |
| Site | US_BNL |
| Entry | DUNE_US_BNL_sp02 |
| Last heartbeat | 2024-10-17 13:33:20 |
| From worker node | Hostname | spool0727.sdcc.bnl.gov |
| cpuinfo | Intel(R) Xeon(R) CPU E5-2660 v3 @ 2.60GHz |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 171000 (47 hours) |
| Inner Apptainer? | False |
| Job state | stalled |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk |
| Started | 2024-10-17 10:32:50 |
| Input files | hd-protodune:np04hd_raw_run027404_0203_dataflow3_datawriter_0_20240624T115808.hdf5
|
| Outputting started | 2024-10-17 13:12:46 |
| Output files | |
| Finished | 2024-10-17 14:25:27 |
| List job events Wrapper job log |
Jobscript log (last 10,000 characters)
s tagged "gauss"
FrameSaver: q=1.27838e+06 n=660668 tag=gauss
wclsFrameSaver: saving 51119 traces tagged "wiener"
FrameSaver: q=1.39628e+06 n=594760 tag=wiener
3064 hits in TPC 0
4398 hits in TPC 1
3019 hits in TPC 2
1981 hits in TPC 3
1388 hits in TPC 4
3177 hits in TPC 5
7208 hits in TPC 6
3635 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:0 17 XUs and 10 XVs -> 0 XUVs
C:0 T:1 21834 XUs and 10331 XVs -> 226 XUVs
C:0 T:2 6168 XUs and 5549 XVs -> 230 XUVs
C:0 T:3 39 XUs and 51 XVs -> 2 XUVs
C:0 T:4 31 XUs and 49 XVs -> 2 XUVs
C:0 T:5 10716 XUs and 5479 XVs -> 167 XUVs
C:0 T:6 24938 XUs and 36764 XVs -> 2795 XUVs
C:0 T:7 828 XUs and 799 XVs -> 20 XUVs
3442 XUVs total
1054 collection wire objects
3442 potential space points
Neighbour search...
285842 tests to find 124364 neighbours
Iterating with no regularization...
Begin: 1.81076e+07
0 1.6478e+07
1 1.63345e+07
2 1.63151e+07
3 1.6306e+07
Now with regularization...
Begin: 1.56986e+07
0 1.56924e+07
BdtBeamParticleIdTool::SliceFeatures::GetLArTPCIntercepts - inconsistent number of intercepts between a line and the LArTPC
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
17-Oct-2024 13:12:28 UTC Closed output file "np04hd_raw_run027404_0203_dataflow3_datawriter_0_20240624T115808_reco_stage1_reco_stage2_20241017T131228_keepup.root"
17-Oct-2024 13:12:28 UTC Closed input file "np04hd_raw_run027404_0203_dataflow3_datawriter_0_20240624T115808_reco_stage1.root"
========================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
========================================================================================================================================
Full event 92.142 148.968 233.57 141.121 31.2393 35
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.000633707 0.00259381 0.00576471 0.00236959 0.00140289 35
produce:fembfilter:PDHDFEMBFilter 0.726236 0.917411 1.93592 0.857382 0.216059 35
produce:wclsdatahd:WireCellToolkit 60.9595 64.9891 70.6206 64.6648 2.42992 35
produce:gaushit:GausHitFinder 0.443272 0.954231 1.60568 0.964331 0.238747 35
produce:nhitsfilter:NumberOfHitsFilter 0.000238062 0.000431549 0.000623923 0.000416099 9.59002e-05 35
produce:reco3d:SpacePointSolver 0.162047 1.43465 6.46675 1.20512 1.14841 35
produce:hitpdune:DisambigFromSpacePoints 0.214872 0.819041 1.48466 0.869505 0.34605 35
produce:pandora:StandardPandora 6.64269 29.8593 83.9522 26.0891 17.11 35
produce:pandoraWriter:StandardPandora 0.0831009 0.176313 0.278802 0.173649 0.0429949 35
produce:pandoraTrack:LArPandoraTrackCreation 0.645328 2.40534 4.81719 2.29989 0.966958 35
produce:pandoraShower:LArPandoraShowerCreation 0.526181 2.39476 5.8046 2.22739 1.1216 35
produce:pandoracalo:Calorimetry 0.353353 1.18754 2.54152 1.08834 0.471031 35
produce:pandoracalonosce:Calorimetry 0.308169 1.13576 2.35663 1.08005 0.466915 35
produce:pandorapid:Chi2ParticleID 0.00138457 0.00234182 0.00447507 0.00221241 0.000726538 35
produce:pandoraShowercalo:ShowerCalorimetry 0.484276 2.4409 6.35543 2.28354 1.17525 35
produce:pandoraShowercalonosce:ShowerCalorimetry 0.503094 2.3192 5.69601 2.1739 1.10177 35
produce:pandoraGnocchiCalo:GnocchiCalorimetry 0.0682219 0.136466 0.198314 0.141725 0.0321361 35
produce:pandoraGnocchiCalonosce:GnocchiCalorimetry 0.0259162 0.0464201 0.0739178 0.0452087 0.0117174 35
produce:emtrkmichelid:EmTrackMichelId 17.8499 35.7353 54.3758 34.95 8.43101 35
[art]:TriggerResults:TriggerResultInserter 2.3195e-05 4.02638e-05 7.9058e-05 3.5057e-05 1.36207e-05 35
end_path:out1:RootOutput 9.799e-06 1.26056e-05 3.7746e-05 1.0757e-05 5.22139e-06 35
end_path:michelremoving:michelremoving 0.0207806 0.0888044 0.545455 0.038337 0.104904 35
end_path:michelremovinggnocchi:michelremoving 0.0201384 0.0842989 0.61953 0.035107 0.112109 35
end_path:calibanagnocchi:CalibAnaTree 0.311013 0.869408 1.86809 0.802643 0.379006 35
end_path:out1:RootOutput(write) 0.438932 0.855037 1.22508 0.868115 0.190592 35
========================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 4945.69 MB
Peak resident set size usage (VmHWM): 2319.94 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np04hd_raw_run027404_0203_dataflow3_datawriter_0_20240624T115808_reco_stage1_reco_stage2_20241017T131228_keepup.root
\tHists: np04hd_raw_run027404_0203_dataflow3_datawriter_0_20240624T115808_reco_stage1_reco_stage2_20241017T131228_keepup_hists.root
Forming reco metadata
Successfully opened file np04hd_raw_run027404_0203_dataflow3_datawriter_0_20240624T115808_reco_stage1_reco_stage2_20241017T131228_keepup.root
Ran successfully
{
"name": "np04hd_raw_run027404_0203_dataflow3_datawriter_0_20240624T115808_reco_stage1_reco_stage2_20241017T131228_keepup.root",
"namespace": "hd-protodune-det-reco",
"metadata": {
"core.file_format": "artroot",
"core.application.name": "reco",
"core.application.family": "dunesw",
"core.application.version": "v09_91_02d01",
"core.data_tier": "full-reconstructed",
"dune.config_file": "standard_reco_stage2_calibration_protodunehd_keepup.fcl",
"dune.campaign": "hd-protodune-reco-keepup-v0",
"core.start_time": 1729165474.0,
"core.end_time": 1729170750.0,
"core.events": [
28423,
28427,
28431,
28435,
28439,
28443,
28447,
28451,
28455,
28459,
28463,
28467,
28471,
28475,
28479,
28483,
28487,
28491,
28495,
28499,
28503,
28507,
28511,
28515,
28519,
28523,
28527,
28531,
28535,
28539,
28543,
28547,
28551,
28555,
28559
],
"core.event_count": 35,
"core.first_event_number": 28423,
"core.last_event_number": 28559,
"core.data_stream": "physics",
"core.file_content_status": "good",
"core.file_type": "detector",
"core.run_type": "hd-protodune",
"core.runs": [
27404
],
"core.runs_subruns": [
2740400001
],
"dune.daq_test": false,
"retention.status": "active",
"retention.class": "physics",
"origin.applications.config_files": {
"reco1": "standard_reco_stage1_protodunehd_keepup.fcl"
},
"origin.applications.versions": {
"reco1": "v09_91_02d01"
},
"origin.applications.names": [
"reco1"
]
},
"parents": [
{
"did": "hd-protodune:np04hd_raw_run027404_0203_dataflow3_datawriter_0_20240624T115808.hdf5"
}
]
}Forming hist metadata
formed
{
"name": "np04hd_raw_run027404_0203_dataflow3_datawriter_0_20240624T115808_reco_stage1_reco_stage2_20241017T131228_keepup_hists.root",
"namespace": "hd-protodune-det-reco",
"metadata": {
"core.file_format": "root",
"core.application.name": "reco",
"core.application.family": "dunesw",
"core.application.version": "v09_91_02d01",
"core.data_tier": "root-tuple-virtual",
"dune.config_file": "standard_reco_stage2_calibration_protodunehd_keepup.fcl",
"dune.campaign": "hd-protodune-reco-keepup-v0",
"core.start_time": 1729165474.0,
"core.end_time": 1729170750.0,
"core.data_stream": "physics",
"core.file_content_status": "good",
"core.file_type": "detector",
"core.run_type": "hd-protodune",
"core.runs": [
27404
],
"core.runs_subruns": [
2740400001
],
"dune.daq_test": false,
"retention.status": "active",
"retention.class": "physics",
"origin.applications.config_files": {
"reco1": "standard_reco_stage1_protodunehd_keepup.fcl"
},
"origin.applications.versions": {
"reco1": "v09_91_02d01"
},
"origin.applications.names": [
"reco1"
]
},
"parents": [
{
"did": "hd-protodune:np04hd_raw_run027404_0203_dataflow3_datawriter_0_20240624T115808.hdf5"
}
]
} 