Jobsub ID 80022.187@justin-prod-sched02.dune.hep.ac.uk
| Jobsub ID | 80022.187@justin-prod-sched02.dune.hep.ac.uk |
| Workflow ID | 3513 |
| Stage ID | 1 |
| User name | calcuttj@fnal.gov |
| HTCondor Group | group_dune.prod_mcsim |
| Requested | Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 18000 (5 hours) |
| Submitted time | 2024-09-27 09:07:03 |
| Site | US_UCSD |
| Entry | CMSHTPC_T2_US_UCSD_gw7 |
| Last heartbeat | 2024-09-27 16:50:43 |
| From worker node | Hostname | mh-7662-2.t2.ucsd.edu |
| cpuinfo | AMD EPYC 7662 64-Core Processor |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 171000 (47 hours) |
| Inner Apptainer? | True |
| Job state | outputting_failed |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk |
| Started | 2024-09-27 14:53:20 |
| Input files | hd-protodune:np04hd_raw_run027365_0063_dataflow0_datawriter_0_20240622T024418.hdf5
|
| Jobscript | Exit code | 0 |
| Real time | 1h (5130s) |
| CPU time | 1h (3830s = 74%) |
| Outputting started | 2024-09-27 16:18:51 |
| Output files | |
| Finished | 2024-09-27 16:50:43 |
| List job events Wrapper job log |
Jobscript log (last 10,000 characters)
es, 54806 summary
Retagger: tagging trace set: gauss with 40791 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 40791 traces tagged "gauss"
FrameSaver: q=1.46929e+06 n=718311 tag=gauss
wclsFrameSaver: saving 54806 traces tagged "wiener"
FrameSaver: q=1.57985e+06 n=639297 tag=wiener
2287 hits in TPC 0
1892 hits in TPC 1
6489 hits in TPC 2
2292 hits in TPC 3
2965 hits in TPC 4
4167 hits in TPC 5
7669 hits in TPC 6
3534 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:0 29 XUs and 23 XVs -> 0 XUVs
C:0 T:1 1550 XUs and 1104 XVs -> 20 XUVs
C:0 T:2 42699 XUs and 65094 XVs -> 2426 XUVs
C:0 T:3 692 XUs and 561 XVs -> 8 XUVs
C:0 T:4 96 XUs and 90 XVs -> 7 XUVs
C:0 T:5 8060 XUs and 7886 XVs -> 404 XUVs
C:0 T:6 35115 XUs and 46361 XVs -> 1629 XUVs
C:0 T:7 439 XUs and 393 XVs -> 6 XUVs
4500 XUVs total
1593 collection wire objects
4500 potential space points
Neighbour search...
202964 tests to find 104822 neighbours
Iterating with no regularization...
Begin: 2.37395e+07
0 2.03275e+07
1 2.00939e+07
2 2.00606e+07
3 2.00496e+07
Now with regularization...
Begin: 1.88112e+07
0 1.88027e+07
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
27-Sep-2024 09:18:43 PDT Closed output file "np04hd_raw_run027365_0063_dataflow0_datawriter_0_20240622T024418_reco_stage1_reco_stage2_20240927T161843_keepup.root"
27-Sep-2024 09:18:43 PDT Closed input file "np04hd_raw_run027365_0063_dataflow0_datawriter_0_20240622T024418_reco_stage1.root"
========================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
========================================================================================================================================
Full event 65.951 87.4064 140.874 86.8207 16.2426 35
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.000532332 0.00226683 0.00350116 0.00240397 0.000858057 35
produce:fembfilter:PDHDFEMBFilter 0.590303 0.66224 0.746611 0.669453 0.032404 35
produce:wclsdatahd:WireCellToolkit 29.2927 32.8133 36.5547 32.9713 1.62532 35
produce:gaushit:GausHitFinder 0.4688 0.68476 1.07035 0.695467 0.129414 35
produce:nhitsfilter:NumberOfHitsFilter 0.000283377 0.000390198 0.000733717 0.000369289 8.58108e-05 35
produce:reco3d:SpacePointSolver 0.285059 1.50038 16.4529 0.843677 2.62885 35
produce:hitpdune:DisambigFromSpacePoints 0.316739 0.56743 1.47536 0.500118 0.261962 35
produce:pandora:StandardPandora 9.06223 16.8156 37.9736 15.0449 7.00538 35
produce:pandoraWriter:StandardPandora 0.110279 0.158282 0.220944 0.155664 0.0298175 35
produce:pandoraTrack:LArPandoraTrackCreation 0.743496 1.39894 2.64438 1.25316 0.455298 35
produce:pandoraShower:LArPandoraShowerCreation 0.744542 1.56734 2.95548 1.43558 0.660502 35
produce:pandoracalo:Calorimetry 0.399504 0.673375 1.09673 0.648118 0.178458 35
produce:pandoracalonosce:Calorimetry 0.369864 0.626917 1.07757 0.58426 0.180359 35
produce:pandorapid:Chi2ParticleID 0.00129692 0.00194931 0.00371169 0.0018728 0.000447412 35
produce:pandoraShowercalo:ShowerCalorimetry 0.686103 1.3683 2.54141 1.24789 0.537967 35
produce:pandoraShowercalonosce:ShowerCalorimetry 0.670475 1.32775 3.22099 1.19476 0.575226 35
produce:pandoraGnocchiCalo:GnocchiCalorimetry 0.0836182 0.130134 0.189815 0.126115 0.0256311 35
produce:pandoraGnocchiCalonosce:GnocchiCalorimetry 0.0295429 0.0454861 0.0672112 0.0440912 0.00894646 35
produce:emtrkmichelid:EmTrackMichelId 18.9586 25.4593 35.3436 25.5874 4.3016 35
[art]:TriggerResults:TriggerResultInserter 1.8091e-05 3.00551e-05 9.0762e-05 2.585e-05 1.40142e-05 35
end_path:out1:RootOutput 9.18e-06 1.2832e-05 5.3371e-05 1.037e-05 8.11225e-06 35
end_path:michelremoving:michelremoving 0.0145652 0.074524 0.346088 0.0266224 0.0800654 35
end_path:michelremovinggnocchi:michelremoving 0.014286 0.0719831 0.333534 0.0235363 0.078793 35
end_path:calibanagnocchi:CalibAnaTree 0.303176 0.505733 0.914051 0.446598 0.164451 35
end_path:out1:RootOutput(write) 0.588889 0.840243 1.24827 0.860424 0.166211 35
========================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 5013.05 MB
Peak resident set size usage (VmHWM): 2680.02 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np04hd_raw_run027365_0063_dataflow0_datawriter_0_20240622T024418_reco_stage1_reco_stage2_20240927T161843_keepup.root
\tHists: np04hd_raw_run027365_0063_dataflow0_datawriter_0_20240622T024418_reco_stage1_reco_stage2_20240927T161843_keepup_hists.root
Forming reco metadata
Successfully opened file np04hd_raw_run027365_0063_dataflow0_datawriter_0_20240622T024418_reco_stage1_reco_stage2_20240927T161843_keepup.root
Ran successfully
{
"name": "np04hd_raw_run027365_0063_dataflow0_datawriter_0_20240622T024418_reco_stage1_reco_stage2_20240927T161843_keepup.root",
"namespace": "hd-protodune-det-reco",
"metadata": {
"core.file_format": "artroot",
"core.application.name": "reco",
"core.application.family": "dunesw",
"core.application.version": "v09_91_02d01",
"core.data_tier": "full-reconstructed",
"dune.config_file": "standard_reco_stage2_calibration_protodunehd_keepup.fcl",
"dune.campaign": "hd-protodune-reco-keepup-v0",
"core.start_time": 1727450824.0,
"core.end_time": 1727453923.0,
"core.events": [
8828,
8832,
8836,
8840,
8844,
8848,
8852,
8856,
8860,
8864,
8868,
8872,
8876,
8880,
8884,
8888,
8892,
8896,
8900,
8904,
8908,
8912,
8916,
8920,
8924,
8928,
8932,
8936,
8940,
8944,
8948,
8952,
8956,
8960,
8964
],
"core.event_count": 35,
"core.first_event_number": 8828,
"core.last_event_number": 8964,
"core.data_stream": "physics",
"core.file_content_status": "good",
"core.file_type": "detector",
"core.run_type": "hd-protodune",
"core.runs": [
27365
],
"core.runs_subruns": [
2736500001
],
"dune.daq_test": false,
"retention.status": "active",
"retention.class": "physics",
"origin.applications.config_files": {
"reco1": "standard_reco_stage1_protodunehd_keepup.fcl"
},
"origin.applications.versions": {
"reco1": "v09_91_02d01"
},
"origin.applications.names": [
"reco1"
]
},
"parents": [
{
"did": "hd-protodune:np04hd_raw_run027365_0063_dataflow0_datawriter_0_20240622T024418.hdf5"
}
]
}Forming hist metadata
formed
{
"name": "np04hd_raw_run027365_0063_dataflow0_datawriter_0_20240622T024418_reco_stage1_reco_stage2_20240927T161843_keepup_hists.root",
"namespace": "hd-protodune-det-reco",
"metadata": {
"core.file_format": "root",
"core.application.name": "reco",
"core.application.family": "dunesw",
"core.application.version": "v09_91_02d01",
"core.data_tier": "root-tuple-virtual",
"dune.config_file": "standard_reco_stage2_calibration_protodunehd_keepup.fcl",
"dune.campaign": "hd-protodune-reco-keepup-v0",
"core.start_time": 1727450824.0,
"core.end_time": 1727453923.0,
"core.data_stream": "physics",
"core.file_content_status": "good",
"core.file_type": "detector",
"core.run_type": "hd-protodune",
"core.runs": [
27365
],
"core.runs_subruns": [
2736500001
],
"dune.daq_test": false,
"retention.status": "active",
"retention.class": "physics",
"origin.applications.config_files": {
"reco1": "standard_reco_stage1_protodunehd_keepup.fcl"
},
"origin.applications.versions": {
"reco1": "v09_91_02d01"
},
"origin.applications.names": [
"reco1"
]
},
"parents": [
{
"did": "hd-protodune:np04hd_raw_run027365_0063_dataflow0_datawriter_0_20240622T024418.hdf5"
}
]
} 