Jobsub ID 264658.120@justin-prod-sched01.dune.hep.ac.uk
| Jobsub ID | 264658.120@justin-prod-sched01.dune.hep.ac.uk |
| Workflow ID | 3512 |
| Stage ID | 1 |
| User name | calcuttj@fnal.gov |
| HTCondor Group | group_dune.prod_mcsim |
| Requested | Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 18000 (5 hours) |
| Submitted time | 2024-09-27 04:55:25 |
| Site | US_UCSD |
| Entry | CMSHTPC_T2_US_UCSD_gw7 |
| Last heartbeat | 2024-09-27 16:38:01 |
| From worker node | Hostname | mh-7662-6.t2.ucsd.edu |
| cpuinfo | AMD EPYC 7662 64-Core Processor |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 171000 (47 hours) |
| Inner Apptainer? | True |
| Job state | outputting_failed |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk |
| Started | 2024-09-27 06:34:52 |
| Input files | hd-protodune:np04hd_raw_run027341_0009_dataflow3_datawriter_0_20240621T131136.hdf5
|
| Jobscript | Exit code | 0 |
| Real time | 9h (35227s) |
| CPU time | 1h (4394s = 12%) |
| Outputting started | 2024-09-27 16:22:00 |
| Output files | |
| Finished | 2024-09-27 16:38:01 |
| List job events Wrapper job log |
Jobscript log (last 10,000 characters)
Retagger: tagging trace set: gauss with 69802 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 69802 traces tagged "gauss"
FrameSaver: q=2.71086e+06 n=1274040 tag=gauss
wclsFrameSaver: saving 91438 traces tagged "wiener"
FrameSaver: q=2.92189e+06 n=1158895 tag=wiener
5217 hits in TPC 0
4728 hits in TPC 1
5574 hits in TPC 2
3245 hits in TPC 3
5576 hits in TPC 4
6606 hits in TPC 5
14641 hits in TPC 6
5160 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 12538 XUs and 6207 XVs -> 326 XUVs
C:0 T:2 12629 XUs and 14347 XVs -> 707 XUVs
C:0 T:3 109 XUs and 141 XVs -> 8 XUVs
C:0 T:4 233 XUs and 142 XVs -> 2 XUVs
C:0 T:5 31067 XUs and 15757 XVs -> 441 XUVs
C:0 T:6 56057 XUs and 56356 XVs -> 2949 XUVs
C:0 T:7 262 XUs and 266 XVs -> 2 XUVs
4435 XUVs total
1861 collection wire objects
4435 potential space points
Neighbour search...
178229 tests to find 74190 neighbours
Iterating with no regularization...
Begin: 7.67795e+07
0 7.27709e+07
1 7.24103e+07
2 7.23685e+07
Now with regularization...
Begin: 6.82692e+07
0 6.82499e+07
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
27-Sep-2024 09:21:50 PDT Closed output file "np04hd_raw_run027341_0009_dataflow3_datawriter_0_20240621T131136_reco_stage1_reco_stage2_20240927T162150_keepup.root"
27-Sep-2024 09:21:50 PDT Closed input file "np04hd_raw_run027341_0009_dataflow3_datawriter_0_20240621T131136_reco_stage1.root"
========================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
========================================================================================================================================
Full event 70.9681 88.8217 112.509 88.7038 11.335 40
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.000563246 0.00244211 0.00399339 0.00267678 0.000956585 40
produce:fembfilter:PDHDFEMBFilter 0.667395 0.747055 0.910944 0.718178 0.0682196 40
produce:wclsdatahd:WireCellToolkit 33.2207 38.0024 44.1112 37.4474 2.41238 40
produce:gaushit:GausHitFinder 0.479118 0.700656 0.970887 0.706481 0.122505 40
produce:nhitsfilter:NumberOfHitsFilter 0.000275738 0.00038566 0.000548425 0.000385581 7.06133e-05 40
produce:reco3d:SpacePointSolver 0.195237 0.917371 2.97894 0.756417 0.638293 40
produce:hitpdune:DisambigFromSpacePoints 0.235532 0.496354 1.01703 0.47064 0.168832 40
produce:pandora:StandardPandora 7.73037 14.458 25.1577 13.1117 4.79276 40
produce:pandoraWriter:StandardPandora 0.101109 0.156269 0.215405 0.154259 0.030155 40
produce:pandoraTrack:LArPandoraTrackCreation 0.531117 1.24609 2.40726 1.15503 0.402528 40
produce:pandoraShower:LArPandoraShowerCreation 0.397117 1.36052 2.95723 1.31869 0.490033 40
produce:pandoracalo:Calorimetry 0.274726 0.606179 1.06646 0.596436 0.159651 40
produce:pandoracalonosce:Calorimetry 0.251393 0.552602 0.987549 0.543436 0.153144 40
produce:pandorapid:Chi2ParticleID 0.00122204 0.00186514 0.00310499 0.00182493 0.000352975 40
produce:pandoraShowercalo:ShowerCalorimetry 0.342755 1.21333 2.90974 1.16024 0.449198 40
produce:pandoraShowercalonosce:ShowerCalorimetry 0.335139 1.16672 2.81653 1.11221 0.431676 40
produce:pandoraGnocchiCalo:GnocchiCalorimetry 0.0690101 0.13197 0.213239 0.132741 0.025884 40
produce:pandoraGnocchiCalonosce:GnocchiCalorimetry 0.0262698 0.0463735 0.0686426 0.0471959 0.00895441 40
produce:emtrkmichelid:EmTrackMichelId 17.8096 25.4728 33.5097 26.1141 4.13754 40
[art]:TriggerResults:TriggerResultInserter 1.67e-05 2.82221e-05 7.0672e-05 2.5471e-05 1.08424e-05 40
end_path:out1:RootOutput 3.65e-06 1.01025e-05 5.8311e-05 7.85e-06 1.07973e-05 40
end_path:michelremoving:michelremoving 0.0135261 0.058864 0.25065 0.0264268 0.0592597 40
end_path:michelremovinggnocchi:michelremoving 0.0129696 0.0561526 0.246552 0.0235173 0.0585256 40
end_path:calibanagnocchi:CalibAnaTree 0.181193 0.472548 0.970597 0.472752 0.149453 40
end_path:out1:RootOutput(write) 0.521292 0.845892 1.23151 0.858504 0.14688 40
========================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 5033.91 MB
Peak resident set size usage (VmHWM): 2681.94 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np04hd_raw_run027341_0009_dataflow3_datawriter_0_20240621T131136_reco_stage1_reco_stage2_20240927T162150_keepup.root
\tHists: np04hd_raw_run027341_0009_dataflow3_datawriter_0_20240621T131136_reco_stage1_reco_stage2_20240927T162150_keepup_hists.root
Forming reco metadata
Successfully opened file np04hd_raw_run027341_0009_dataflow3_datawriter_0_20240621T131136_reco_stage1_reco_stage2_20240927T162150_keepup.root
Ran successfully
{
"name": "np04hd_raw_run027341_0009_dataflow3_datawriter_0_20240621T131136_reco_stage1_reco_stage2_20240927T162150_keepup.root",
"namespace": "hd-protodune-det-reco",
"metadata": {
"core.file_format": "artroot",
"core.application.name": "reco",
"core.application.family": "dunesw",
"core.application.version": "v09_91_02d01",
"core.data_tier": "full-reconstructed",
"dune.config_file": "standard_reco_stage2_calibration_protodunehd_keepup.fcl",
"dune.campaign": "hd-protodune-reco-keepup-v0",
"core.start_time": 1727450517.0,
"core.end_time": 1727454111.0,
"core.events": [
1443,
1447,
1451,
1455,
1459,
1463,
1467,
1471,
1475,
1479,
1483,
1487,
1491,
1495,
1499,
1503,
1507,
1511,
1515,
1519,
1523,
1527,
1531,
1535,
1539,
1543,
1547,
1551,
1555,
1559,
1563,
1567,
1571,
1575,
1579,
1583,
1587,
1591,
1595,
1599
],
"core.event_count": 40,
"core.first_event_number": 1443,
"core.last_event_number": 1599,
"core.data_stream": "physics",
"core.file_content_status": "good",
"core.file_type": "detector",
"core.run_type": "hd-protodune",
"core.runs": [
27341
],
"core.runs_subruns": [
2734100001
],
"dune.daq_test": false,
"retention.status": "active",
"retention.class": "physics",
"origin.applications.config_files": {
"reco1": "standard_reco_stage1_protodunehd_keepup.fcl"
},
"origin.applications.versions": {
"reco1": "v09_91_02d01"
},
"origin.applications.names": [
"reco1"
]
},
"parents": [
{
"did": "hd-protodune:np04hd_raw_run027341_0009_dataflow3_datawriter_0_20240621T131136.hdf5"
}
]
}Forming hist metadata
formed
{
"name": "np04hd_raw_run027341_0009_dataflow3_datawriter_0_20240621T131136_reco_stage1_reco_stage2_20240927T162150_keepup_hists.root",
"namespace": "hd-protodune-det-reco",
"metadata": {
"core.file_format": "root",
"core.application.name": "reco",
"core.application.family": "dunesw",
"core.application.version": "v09_91_02d01",
"core.data_tier": "root-tuple-virtual",
"dune.config_file": "standard_reco_stage2_calibration_protodunehd_keepup.fcl",
"dune.campaign": "hd-protodune-reco-keepup-v0",
"core.start_time": 1727450517.0,
"core.end_time": 1727454111.0,
"core.data_stream": "physics",
"core.file_content_status": "good",
"core.file_type": "detector",
"core.run_type": "hd-protodune",
"core.runs": [
27341
],
"core.runs_subruns": [
2734100001
],
"dune.daq_test": false,
"retention.status": "active",
"retention.class": "physics",
"origin.applications.config_files": {
"reco1": "standard_reco_stage1_protodunehd_keepup.fcl"
},
"origin.applications.versions": {
"reco1": "v09_91_02d01"
},
"origin.applications.names": [
"reco1"
]
},
"parents": [
{
"did": "hd-protodune:np04hd_raw_run027341_0009_dataflow3_datawriter_0_20240621T131136.hdf5"
}
]
} 