Jobsub ID 79974.141@justin-prod-sched02.dune.hep.ac.uk
| Jobsub ID | 79974.141@justin-prod-sched02.dune.hep.ac.uk |
| Workflow ID | 3511 |
| Stage ID | 1 |
| User name | calcuttj@fnal.gov |
| HTCondor Group | group_dune.prod_mcsim |
| Requested | Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 18000 (5 hours) |
| Submitted time | 2024-09-27 08:27:59 |
| Site | US_UCSD |
| Entry | CMSHTPC_T2_US_UCSD_gw7 |
| Last heartbeat | 2024-09-27 16:33:08 |
| From worker node | Hostname | mh-7662-15.t2.ucsd.edu |
| cpuinfo | AMD EPYC 7662 64-Core Processor |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 171000 (47 hours) |
| Inner Apptainer? | True |
| Job state | outputting_failed |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk |
| Started | 2024-09-27 13:49:15 |
| Input files | hd-protodune:np04hd_raw_run027334_0114_dataflow1_datawriter_0_20240621T041250.hdf5
|
| Jobscript | Exit code | 0 |
| Real time | 2h (7802s) |
| CPU time | 1h (5789s = 74%) |
| Outputting started | 2024-09-27 15:59:18 |
| Output files | |
| Finished | 2024-09-27 16:33:08 |
| List job events Wrapper job log |
Jobscript log (last 10,000 characters)
539 traces, 66539 summary
Retagger: tagging trace set: gauss with 48579 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 48579 traces tagged "gauss"
FrameSaver: q=1.84788e+06 n=823400 tag=gauss
wclsFrameSaver: saving 66539 traces tagged "wiener"
FrameSaver: q=2.01107e+06 n=734676 tag=wiener
2666 hits in TPC 0
1748 hits in TPC 1
6837 hits in TPC 2
4385 hits in TPC 3
3228 hits in TPC 4
3950 hits in TPC 5
7336 hits in TPC 6
5398 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 6172 XUs and 4308 XVs -> 266 XUVs
C:0 T:2 39971 XUs and 17043 XVs -> 560 XUVs
C:0 T:3 581 XUs and 700 XVs -> 18 XUVs
C:0 T:4 228 XUs and 227 XVs -> 8 XUVs
C:0 T:5 47061 XUs and 23786 XVs -> 1066 XUVs
C:0 T:6 26825 XUs and 22584 XVs -> 1420 XUVs
C:0 T:7 667 XUs and 542 XVs -> 20 XUVs
3358 XUVs total
1430 collection wire objects
3358 potential space points
Neighbour search...
80600 tests to find 47032 neighbours
Iterating with no regularization...
Begin: 3.30539e+07
0 3.11534e+07
1 3.10012e+07
2 3.09798e+07
Now with regularization...
Begin: 2.89808e+07
0 2.89738e+07
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
27-Sep-2024 08:59:09 PDT Closed output file "np04hd_raw_run027334_0114_dataflow1_datawriter_0_20240621T041250_reco_stage1_reco_stage2_20240927T155909_keepup.root"
27-Sep-2024 08:59:09 PDT Closed input file "np04hd_raw_run027334_0114_dataflow1_datawriter_0_20240621T041250_reco_stage1.root"
========================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
========================================================================================================================================
Full event 39.1525 141.601 591.4 127.459 88.8132 35
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.000958777 0.00335876 0.00641335 0.00371236 0.00169417 35
produce:fembfilter:PDHDFEMBFilter 0.665992 0.73086 0.835257 0.734522 0.0307236 35
produce:wclsdatahd:WireCellToolkit 32.1115 36.4565 40.4135 36.2726 1.59629 35
produce:gaushit:GausHitFinder 0.472389 1.08226 1.6704 1.04577 0.280558 35
produce:nhitsfilter:NumberOfHitsFilter 0.000273295 0.00050672 0.000936365 0.000501806 0.000122167 35
produce:reco3d:SpacePointSolver 0.232029 2.24553 15.5079 1.60656 2.56858 33
produce:hitpdune:DisambigFromSpacePoints 0.193774 0.95765 2.51645 0.840007 0.491959 33
produce:pandora:StandardPandora 10.6707 54.6576 455.345 38.4494 75.464 33
produce:pandoraWriter:StandardPandora 0.102497 0.220539 0.318165 0.223118 0.0514038 33
produce:pandoraTrack:LArPandoraTrackCreation 0.873982 2.891 6.855 2.71216 1.428 33
produce:pandoraShower:LArPandoraShowerCreation 0.54882 3.08388 8.40408 2.98334 1.78146 33
produce:pandoracalo:Calorimetry 0.397407 1.29107 3.3556 1.28376 0.64096 33
produce:pandoracalonosce:Calorimetry 0.368641 1.21849 3.13613 1.2126 0.616306 33
produce:pandorapid:Chi2ParticleID 0.00146212 0.00281445 0.00627566 0.00248703 0.000998911 33
produce:pandoraShowercalo:ShowerCalorimetry 0.464909 2.5581 6.38542 2.25539 1.38911 33
produce:pandoraShowercalonosce:ShowerCalorimetry 0.445784 2.3871 5.587 2.18185 1.26545 33
produce:pandoraGnocchiCalo:GnocchiCalorimetry 0.0831557 0.166681 0.240687 0.171237 0.0395894 33
produce:pandoraGnocchiCalonosce:GnocchiCalorimetry 0.0295512 0.0582907 0.0811731 0.0587072 0.0136066 33
produce:emtrkmichelid:EmTrackMichelId 17.6258 35.1711 51.1218 34.8981 7.89763 33
[art]:TriggerResults:TriggerResultInserter 2.4105e-05 4.8155e-05 0.000126308 4.741e-05 2.16831e-05 35
end_path:out1:RootOutput 1.061e-05 1.59838e-05 8.473e-05 1.2434e-05 1.24087e-05 35
end_path:michelremoving:michelremoving 3.9504e-05 0.129472 0.820981 0.0376738 0.172348 35
end_path:michelremovinggnocchi:michelremoving 1.1963e-05 0.126257 0.807585 0.0337374 0.171063 35
end_path:calibanagnocchi:CalibAnaTree 1.3626e-05 1.00255 2.69263 0.804303 0.616972 35
end_path:out1:RootOutput(write) 0.478467 1.09532 1.70172 1.08717 0.294277 35
========================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 5097.68 MB
Peak resident set size usage (VmHWM): 2741.34 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np04hd_raw_run027334_0114_dataflow1_datawriter_0_20240621T041250_reco_stage1_reco_stage2_20240927T155909_keepup.root
\tHists: np04hd_raw_run027334_0114_dataflow1_datawriter_0_20240621T041250_reco_stage1_reco_stage2_20240927T155909_keepup_hists.root
Forming reco metadata
Successfully opened file np04hd_raw_run027334_0114_dataflow1_datawriter_0_20240621T041250_reco_stage1_reco_stage2_20240927T155909_keepup.root
Ran successfully
{
"name": "np04hd_raw_run027334_0114_dataflow1_datawriter_0_20240621T041250_reco_stage1_reco_stage2_20240927T155909_keepup.root",
"namespace": "hd-protodune-det-reco",
"metadata": {
"core.file_format": "artroot",
"core.application.name": "reco",
"core.application.family": "dunesw",
"core.application.version": "v09_91_02d01",
"core.data_tier": "full-reconstructed",
"dune.config_file": "standard_reco_stage2_calibration_protodunehd_keepup.fcl",
"dune.campaign": "hd-protodune-reco-keepup-v0",
"core.start_time": 1727447750.0,
"core.end_time": 1727452750.0,
"core.events": [
15961,
15965,
15969,
15973,
15977,
15981,
15985,
15989,
15993,
15997,
16001,
16005,
16009,
16013,
16017,
16021,
16025,
16029,
16033,
16037,
16041,
16045,
16049,
16053,
16057,
16061,
16065,
16069,
16073,
16077,
16081,
16085,
16089,
16093,
16097
],
"core.event_count": 35,
"core.first_event_number": 15961,
"core.last_event_number": 16097,
"core.data_stream": "physics",
"core.file_content_status": "good",
"core.file_type": "detector",
"core.run_type": "hd-protodune",
"core.runs": [
27334
],
"core.runs_subruns": [
2733400001
],
"dune.daq_test": false,
"retention.status": "active",
"retention.class": "physics",
"origin.applications.config_files": {
"reco1": "standard_reco_stage1_protodunehd_keepup.fcl"
},
"origin.applications.versions": {
"reco1": "v09_91_02d01"
},
"origin.applications.names": [
"reco1"
]
},
"parents": [
{
"did": "hd-protodune:np04hd_raw_run027334_0114_dataflow1_datawriter_0_20240621T041250.hdf5"
}
]
}Forming hist metadata
formed
{
"name": "np04hd_raw_run027334_0114_dataflow1_datawriter_0_20240621T041250_reco_stage1_reco_stage2_20240927T155909_keepup_hists.root",
"namespace": "hd-protodune-det-reco",
"metadata": {
"core.file_format": "root",
"core.application.name": "reco",
"core.application.family": "dunesw",
"core.application.version": "v09_91_02d01",
"core.data_tier": "root-tuple-virtual",
"dune.config_file": "standard_reco_stage2_calibration_protodunehd_keepup.fcl",
"dune.campaign": "hd-protodune-reco-keepup-v0",
"core.start_time": 1727447750.0,
"core.end_time": 1727452750.0,
"core.data_stream": "physics",
"core.file_content_status": "good",
"core.file_type": "detector",
"core.run_type": "hd-protodune",
"core.runs": [
27334
],
"core.runs_subruns": [
2733400001
],
"dune.daq_test": false,
"retention.status": "active",
"retention.class": "physics",
"origin.applications.config_files": {
"reco1": "standard_reco_stage1_protodunehd_keepup.fcl"
},
"origin.applications.versions": {
"reco1": "v09_91_02d01"
},
"origin.applications.names": [
"reco1"
]
},
"parents": [
{
"did": "hd-protodune:np04hd_raw_run027334_0114_dataflow1_datawriter_0_20240621T041250.hdf5"
}
]
} 