Jobsub ID 106019.164@justin-prod-sched02.dune.hep.ac.uk
| Jobsub ID | 106019.164@justin-prod-sched02.dune.hep.ac.uk | |
| Workflow ID | 4206 | |
| Stage ID | 1 | |
| User name | calcuttj@fnal.gov | |
| HTCondor Group | group_dune | |
| Requested | Processors | 1 |
| RSS bytes | 2096103424 (1999 MiB) | |
| Wall seconds limit | 80000 (22 hours) | |
| Submitted time | 2024-11-21 14:19:52 | |
| Site | ES_CIEMAT | |
| Entry | DUNE_T2_ES_CIEMAT_condorce2 | |
| Last heartbeat | 2024-11-22 01:12:57 | |
| From worker node | Hostname | gaew0194.ciemat.es |
| cpuinfo | Intel(R) Xeon(R) CPU E5-2650 v4 @ 2.20GHz | |
| OS release | Scientific Linux release 7.9 (Nitrogen) | |
| Processors | 1 | |
| RSS bytes | 2096103424 (1999 MiB) | |
| Wall seconds limit | 232200 (64 hours) | |
| Inner Apptainer? | True | |
| Job state | jobscript_error | |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk | |
| Started | 2024-11-21 14:37:47 | |
| Input files | hd-protodune:np04hd_raw_run028109_0019_dataflow5_datawriter_0_20240723T160735.hdf5 hd-protodune:np04hd_raw_run028109_0018_dataflow3_datawriter_0_20240723T160617.hdf5 hd-protodune:np04hd_raw_run028109_0019_dataflow0_datawriter_0_20240723T160735.hdf5 hd-protodune:np04hd_raw_run028109_0021_dataflow7_datawriter_0_20240723T160927.hdf5 hd-protodune:np04hd_raw_run028109_0021_dataflow3_datawriter_0_20240723T160927.hdf5 | |
| Jobscript | Exit code | 139 |
| Real time | 0m (0s) | |
| CPU time | 0m (0s = 0%) | |
| Outputting started | ||
| Output files | ||
| Finished | 2024-11-22 01:12:57 | |
| Saved logs | justin-logs:106019.164-justin-prod-sched02.dune.hep.ac.uk.logs.tgz | |
| List job events Wrapper job log | ||
Jobscript log (last 10,000 characters)
n -3.26209e+07 %MSG %MSG-i BeamEvent: BeamEvent:beamevent@BeginModule 22-Nov-2024 02:12:25 CET run: 28109 subRun: 1 event: 5339 BeamEvent_module.cc:1641 Breaking 2A 1A -396.027 %MSG %MSG-i BeamEvent: BeamEvent:beamevent@BeginModule 22-Nov-2024 02:12:25 CET run: 28109 subRun: 1 event: 5339 BeamEvent_module.cc:1625 114 57 Breaking 2A Gen -3.60118e+07 %MSG %MSG-i BeamEvent: BeamEvent:beamevent@BeginModule 22-Nov-2024 02:12:25 CET run: 28109 subRun: 1 event: 5339 BeamEvent_module.cc:1625 115 57 Breaking 2A Gen -3.06925e+07 %MSG %MSG-i BeamEvent: BeamEvent:beamevent@BeginModule 22-Nov-2024 02:12:25 CET run: 28109 subRun: 1 event: 5339 BeamEvent_module.cc:1641 Breaking 2A 1A -4.42892e+07 %MSG %MSG-i BeamEvent: BeamEvent:beamevent@BeginModule 22-Nov-2024 02:12:25 CET run: 28109 subRun: 1 event: 5339 BeamEvent_module.cc:1625 116 58 Breaking 2A Gen -1.37723e+06 %MSG %MSG-i BeamEvent: BeamEvent:beamevent@BeginModule 22-Nov-2024 02:12:25 CET run: 28109 subRun: 1 event: 5339 BeamEvent_module.cc:1641 Breaking 2A 1A -4.29119e+07 %MSG %MSG-i BeamEvent: BeamEvent:beamevent@BeginModule 22-Nov-2024 02:12:25 CET run: 28109 subRun: 1 event: 5339 BeamEvent_module.cc:1625 117 59 Breaking 2A Gen -9.49462e+06 %MSG %MSG-i BeamEvent: BeamEvent:beamevent@BeginModule 22-Nov-2024 02:12:25 CET run: 28109 subRun: 1 event: 5339 BeamEvent_module.cc:1641 Breaking 2A 1A -3.34172e+07 %MSG %MSG-i BeamEvent: BeamEvent:beamevent@BeginModule 22-Nov-2024 02:12:25 CET run: 28109 subRun: 1 event: 5339 BeamEvent_module.cc:1625 118 60 Breaking 2A Gen -3.34174e+07 %MSG %MSG-i BeamEvent: BeamEvent:beamevent@BeginModule 22-Nov-2024 02:12:25 CET run: 28109 subRun: 1 event: 5339 BeamEvent_module.cc:1641 Breaking 2A 1A -3.30034e+06 %MSG %MSG-i BeamEvent: BeamEvent:beamevent@BeginModule 22-Nov-2024 02:12:25 CET run: 28109 subRun: 1 event: 5339 BeamEvent_module.cc:1625 119 61 Breaking 2A Gen -1.43001e+08 %MSG %MSG-i BeamEvent: BeamEvent:beamevent@BeginModule 22-Nov-2024 02:12:25 CET run: 28109 subRun: 1 event: 5339 BeamEvent_module.cc:1625 120 61 Breaking 2A Gen -1.39701e+08 %MSG %MSG-i BeamEvent: BeamEvent:beamevent@BeginModule 22-Nov-2024 02:12:25 CET run: 28109 subRun: 1 event: 5339 BeamEvent_module.cc:1625 121 61 Breaking 2A Gen -1.13661e+08 %MSG %MSG-i BeamEvent: BeamEvent:beamevent@BeginModule 22-Nov-2024 02:12:25 CET run: 28109 subRun: 1 event: 5339 BeamEvent_module.cc:1625 122 61 Breaking 2A Gen -9.90437e+06 %MSG %MSG-i BeamEvent: BeamEvent:beamevent@BeginModule 22-Nov-2024 02:12:25 CET run: 28109 subRun: 1 event: 5339 BeamEvent_module.cc:1641 Breaking 2A 1A -6.19266e+07 %MSG WARNING: NONZERO HALF WARNING: NONZERO HALF WARNING: NONZERO HALF WARNING: NONZERO HALF WARNING: NONZERO HALF WARNING: NONZERO HALF WARNING: NONZERO HALF WARNING: NONZERO HALF WARNING: NONZERO HALF WARNING: NONZERO HALF WARNING: NONZERO HALF WARNING: NONZERO HALF WARNING: NONZERO HALF WARNING: NONZERO HALF WARNING: NONZERO HALF WARNING: NONZERO HALF WARNING: NONZERO HALF WARNING: NONZERO HALF WARNING: NONZERO HALF WARNING: NONZERO HALF WARNING: NONZERO HALF WARNING: NONZERO HALF WARNING: NONZERO HALF WARNING: NONZERO HALF WARNING: NONZERO HALF WARNING: NONZERO HALF WARNING: NONZERO HALF WARNING: NONZERO HALF WARNING: NONZERO HALF WARNING: NONZERO HALF WARNING: NONZERO HALF WARNING: NONZERO HALF WARNING: NONZERO HALF WARNING: NONZERO HALF WARNING: NONZERO HALF WARNING: NONZERO HALF WARNING: NONZERO HALF WARNING: NONZERO HALF WARNING: NONZERO HALF WARNING: NONZERO HALF WARNING: NONZERO HALF WARNING: NONZERO HALF WARNING: NONZERO HALF WARNING: NONZERO HALF WARNING: NONZERO HALF WARNING: NONZERO HALF WARNING: NONZERO HALF WARNING: NONZERO HALF %MSG-w BeamEvent: BeamEvent:beamevent@BeginModule 22-Nov-2024 02:12:25 CET run: 28109 subRun: 1 event: 5339 BeamEvent_module.cc:1828 Could not get XCET1 info %MSG %MSG-w BeamEvent: BeamEvent:beamevent@BeginModule 22-Nov-2024 02:12:25 CET run: 28109 subRun: 1 event: 5339 BeamEvent_module.cc:1846 Could not get XCET2 info %MSG Timing trigger: 12 Matched: 1 CKovs: 0 0 TOF, P: 96.3769 0.975296 Timing trigger: 12 Matched: 1 CKovs: 1 1 TOF, P: 94.3535 1.05599 Timing trigger: 12 Matched: 1 CKovs: 0 0 TOF, P: 97.7167 0.947863 Timing trigger: 12 Matched: 1 CKovs: 0 0 TOF, P: 95.8233 1.11366 Timing trigger: 12 Matched: 1 CKovs: 1 0 TOF, P: 96.0976 -1 Timing trigger: 12 Matched: 1 CKovs: 1 1 TOF, P: 93.9453 1.0013 Timing trigger: 12 Matched: 1 CKovs: 1 1 TOF, P: 96.9453 -1 Timing trigger: 12 Matched: 1 CKovs: 1 1 TOF, P: 95.7451 -1 Timing trigger: 12 Matched: 1 CKovs: 1 1 TOF, P: 95.2569 0.872772 Timing trigger: 12 Matched: 1 CKovs: 0 0 TOF, P: 95.4866 0.918427 Timing trigger: 12 Matched: 1 CKovs: 0 0 TOF, P: 95.1601 0.936304 Timing trigger: 12 Matched: 1 CKovs: 0 0 TOF, P: 97.1035 1.00485 Timing trigger: 8 Matched: 0 CKovs: 0 0 TOF, P: 0 -1 Timing trigger: 8 Matched: 0 CKovs: 0 0 TOF, P: 0 -1 Timing trigger: 8 Matched: 0 CKovs: 0 0 TOF, P: 0 -1 22-Nov-2024 02:12:39 CET Closed output file "np04hd_raw_run028109_0021_dataflow3_datawriter_0_20240723T160927_beam.root" 22-Nov-2024 02:12:39 CET Closed output file "np04hd_raw_run028109_0021_dataflow3_datawriter_0_20240723T160927_beam.root" =================================================================================================================================== TimeTracker printout (sec) Min Avg Max Median RMS nEvts =================================================================================================================================== Full event 0.45655 243.553 8276.81 0.584042 1212.73 155 ----------------------------------------------------------------------------------------------------------------------------------- source:HDF5RawInput3(read) 1.4278e-05 1.93621e-05 8.6858e-05 1.75e-05 7.38279e-06 155 produce:triggerrawdecoder:PDHDTriggerReader3 0.323627 0.405509 0.734708 0.396299 0.0598565 155 produce:ctbrawdecoder:PDHDCTBRawDecoder 0.0431511 0.0622164 0.102599 0.0616741 0.0137381 155 produce:timingrawdecoder:PDHDTimingRawDecoder 0.00869284 0.00926525 0.0131049 0.00896086 0.00061579 155 produce:beamevent:BeamEvent 0.000167787 242.963 8276.23 0.000877038 1212.73 155 [art]:TriggerResults:TriggerResultInserter 1.2392e-05 2.03045e-05 0.000556969 1.4606e-05 4.39481e-05 155 end_path:out1:RootOutput 2.825e-06 4.29476e-06 5.6825e-05 3.606e-06 4.49681e-06 155 end_path:out1:RootOutput(write) 0.0580163 0.113136 0.265959 0.11164 0.0277059 155 =================================================================================================================================== =================================================================================================================================== TimeTracker printout (sec) Min Avg Max Median RMS nEvts =================================================================================================================================== Full event 0.45655 243.553 8276.81 0.584042 1212.73 155 ----------------------------------------------------------------------------------------------------------------------------------- source:HDF5RawInput3(read) 1.4278e-05 1.93621e-05 8.6858e-05 1.75e-05 7.38279e-06 155 produce:triggerrawdecoder:PDHDTriggerReader3 0.323627 0.405509 0.734708 0.396299 0.0598565 155 produce:ctbrawdecoder:PDHDCTBRawDecoder 0.0431511 0.0622164 0.102599 0.0616741 0.0137381 155 produce:timingrawdecoder:PDHDTimingRawDecoder 0.00869284 0.00926525 0.0131049 0.00896086 0.00061579 155 produce:beamevent:BeamEvent 0.000167787 242.963 8276.23 0.000877038 1212.73 155 [art]:TriggerResults:TriggerResultInserter 1.2392e-05 2.03045e-05 0.000556969 1.4606e-05 4.39481e-05 155 end_path:out1:RootOutput 2.825e-06 4.29476e-06 5.6825e-05 3.606e-06 4.49681e-06 155 end_path:out1:RootOutput(write) 0.0580163 0.113136 0.265959 0.11164 0.0277059 155 =================================================================================================================================== ==================================================================================================== MemoryTracker summary (base-10 MB units used) Peak virtual memory usage (VmPeak) : 1463.9 MB Peak resident set size usage (VmHWM): 633.868 MB ==================================================================================================== ==================================================================================================== MemoryTracker summary (base-10 MB units used) Peak virtual memory usage (VmPeak) : 1463.9 MB Peak resident set size usage (VmHWM): 633.868 MB ==================================================================================================== TrigReport ---------- Event summary ------------- TrigReport Events total = 155 passed = 155 failed = 0 TrigReport ---------- Modules in End-path ---------- TrigReport Run Success Error Name TrigReport 155 155 0 out1 TimeReport ---------- Time summary [sec] ------- TimeReport CPU = 30.404116 Real = 38084.981135 MemReport ---------- Memory summary [base-10 MB] ------ MemReport VmPeak = 1463.9 VmHWM = 633.868