Jobsub ID 99963.17@justin-prod-sched02.dune.hep.ac.uk
| Jobsub ID | 99963.17@justin-prod-sched02.dune.hep.ac.uk |
| Workflow ID | 4121 |
| Stage ID | 1 |
| User name | imawby@fnal.gov |
| HTCondor Group | group_dune |
| Requested | Processors | 1 |
| RSS bytes | 4194304000 (4000 MiB) |
| Wall seconds limit | 43200 (12 hours) |
| Submitted time | 2024-11-16 20:55:48 |
| Site | UK_Edinburgh |
| Entry | DUNE_UK_SGridECDF_ce1_multicore |
| Last heartbeat | 2024-11-16 23:19:42 |
| From worker node | Hostname | node3a01.ecdf.ed.ac.uk |
| cpuinfo | Intel(R) Xeon(R) Gold 6242 CPU @ 2.80GHz |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4194304000 (4000 MiB) |
| Wall seconds limit | 171000 (47 hours) |
| Inner Apptainer? | True |
| Job state | outputting_failed |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk |
| Started | 2024-11-16 20:57:36 |
| Input files | fardet-hd:nutau_dune10kt_1x2x6_1116_102_20230827T173330Z_gen_g4_detsim_hitreco__20240220T194204Z_reco2.root
|
| Jobscript | Exit code | 0 |
| Real time | 2h (8512s) |
| CPU time | 2h (8460s = 99%) |
| Outputting started | 2024-11-16 23:19:29 |
| Output files | |
| Finished | 2024-11-16 23:19:42 |
| List job events Wrapper job log |
Jobscript log (last 10,000 characters)
lns148.pp.rl.ac.uk:51996
[2024-11-16 23:19:26.553369 +0000][Debug ][ExDbgMsg ][ 1405] [heplns148.pp.rl.ac.uk:51996] MsgHandler created: 0x293a9120 (message: kXR_read (handle: 0x00000000, offset: 1715544740, size: 102) ).
[2024-11-16 23:19:26.553533 +0000][Debug ][ExDbgMsg ][ 1405] [heplns148.pp.rl.ac.uk:51996] Moving MsgHandler: 0x293a9120 (message: kXR_read (handle: 0x00000000, offset: 1715544740, size: 102) ) from out-queu to in-queue.
[2024-11-16 23:19:26.582887 +0000][Debug ][ExDbgMsg ][ 1405] [msg: 0x2c3d67e0] Assigned MsgHandler: 0x293a9120.
[2024-11-16 23:19:26.582957 +0000][Debug ][ExDbgMsg ][ 1405] [handler: 0x293a9120] Removed MsgHandler: 0x293a9120 from the in-queue.
[2024-11-16 23:19:26.583114 +0000][Debug ][ExDbgMsg ][ 1405] [heplns148.pp.rl.ac.uk:51996] Calling MsgHandler: 0x293a9120 (message: kXR_read (handle: 0x00000000, offset: 1715544740, size: 102) ) with status: [SUCCESS] .
[2024-11-16 23:19:26.583185 +0000][Debug ][ExDbgMsg ][ 1405] [heplns148.pp.rl.ac.uk:51996] Destroying MsgHandler: 0x293a9120.
[2024-11-16 23:19:26.583309 +0000][Debug ][File ][ 1405] [0x114821e0@root://mover.pp.rl.ac.uk:1094/pnfs/pp.rl.ac.uk/data/dune/fardet-hd/d0/5f/nutau_dune10kt_1x2x6_1116_102_20230827T173330Z_gen_g4_detsim_hitreco__20240220T194204Z_reco2.root?xrdcl.requuid=bf508eca-de0a-431c-ab0c-98af794b67a6] Sending a read command for handle 0x0 to heplns148.pp.rl.ac.uk:51996
[2024-11-16 23:19:26.583392 +0000][Debug ][ExDbgMsg ][ 1405] [heplns148.pp.rl.ac.uk:51996] MsgHandler created: 0x293a9120 (message: kXR_read (handle: 0x00000000, offset: 1715544842, size: 108) ).
[2024-11-16 23:19:26.583493 +0000][Debug ][ExDbgMsg ][ 1405] [heplns148.pp.rl.ac.uk:51996] Moving MsgHandler: 0x293a9120 (message: kXR_read (handle: 0x00000000, offset: 1715544842, size: 108) ) from out-queu to in-queue.
[2024-11-16 23:19:26.597654 +0000][Debug ][ExDbgMsg ][ 1405] [msg: 0x2684c330] Assigned MsgHandler: 0x293a9120.
[2024-11-16 23:19:26.597724 +0000][Debug ][ExDbgMsg ][ 1405] [handler: 0x293a9120] Removed MsgHandler: 0x293a9120 from the in-queue.
[2024-11-16 23:19:26.597870 +0000][Debug ][ExDbgMsg ][ 1405] [heplns148.pp.rl.ac.uk:51996] Calling MsgHandler: 0x293a9120 (message: kXR_read (handle: 0x00000000, offset: 1715544842, size: 108) ) with status: [SUCCESS] .
[2024-11-16 23:19:26.597938 +0000][Debug ][ExDbgMsg ][ 1405] [heplns148.pp.rl.ac.uk:51996] Destroying MsgHandler: 0x293a9120.
[2024-11-16 23:19:26.598031 +0000][Debug ][File ][ 1405] [0x114821e0@root://mover.pp.rl.ac.uk:1094/pnfs/pp.rl.ac.uk/data/dune/fardet-hd/d0/5f/nutau_dune10kt_1x2x6_1116_102_20230827T173330Z_gen_g4_detsim_hitreco__20240220T194204Z_reco2.root?xrdcl.requuid=bf508eca-de0a-431c-ab0c-98af794b67a6] Sending a read command for handle 0x0 to heplns148.pp.rl.ac.uk:51996
[2024-11-16 23:19:26.598125 +0000][Debug ][ExDbgMsg ][ 1405] [heplns148.pp.rl.ac.uk:51996] MsgHandler created: 0x293a9120 (message: kXR_read (handle: 0x00000000, offset: 1715537505, size: 7235) ).
[2024-11-16 23:19:26.598228 +0000][Debug ][ExDbgMsg ][ 1405] [heplns148.pp.rl.ac.uk:51996] Moving MsgHandler: 0x293a9120 (message: kXR_read (handle: 0x00000000, offset: 1715537505, size: 7235) ) from out-queu to in-queue.
[2024-11-16 23:19:26.612208 +0000][Debug ][ExDbgMsg ][ 1405] [msg: 0x258a9ab0] Assigned MsgHandler: 0x293a9120.
[2024-11-16 23:19:26.612270 +0000][Debug ][ExDbgMsg ][ 1405] [handler: 0x293a9120] Removed MsgHandler: 0x293a9120 from the in-queue.
[2024-11-16 23:19:26.612365 +0000][Debug ][ExDbgMsg ][ 1405] [heplns148.pp.rl.ac.uk:51996] Calling MsgHandler: 0x293a9120 (message: kXR_read (handle: 0x00000000, offset: 1715537505, size: 7235) ) with status: [SUCCESS] .
[2024-11-16 23:19:26.612432 +0000][Debug ][ExDbgMsg ][ 1405] [heplns148.pp.rl.ac.uk:51996] Destroying MsgHandler: 0x293a9120.
Boundary wire vector sizes: 31, 70, 53
minwire 0: 1971
minwire 1: 566
minwire 2: 2223
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max wires due to vertex determination failure: 2379, 2878
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary:
Output 0: 0.0860747,
Output 1: 0.00018369, 6.8292e-06, 4.54802e-05, 0.999764,
Output 2: 0.979497, 0.001246, 0.000379703, 0.0188777,
Output 3: 0.0322885, 0.967665, 3.15493e-05, 1.53427e-05,
Output 4: 0.999943, 5.71701e-05, 2.49071e-07, 2.30205e-08,
Output 5: 0.999968, 2.92444e-05, 1.65597e-06, 8.94039e-07,
Output 6: 0.991008, 0.00855666, 0.000388261, 4.65371e-05,
[2024-11-16 23:19:28.199848 +0000][Debug ][File ][ 1405] [0x114821e0@root://mover.pp.rl.ac.uk:1094/pnfs/pp.rl.ac.uk/data/dune/fardet-hd/d0/5f/nutau_dune10kt_1x2x6_1116_102_20230827T173330Z_gen_g4_detsim_hitreco__20240220T194204Z_reco2.root?xrdcl.requuid=bf508eca-de0a-431c-ab0c-98af794b67a6] Sending a read command for handle 0x0 to heplns148.pp.rl.ac.uk:51996
[2024-11-16 23:19:28.199915 +0000][Debug ][ExDbgMsg ][ 1405] [heplns148.pp.rl.ac.uk:51996] MsgHandler created: 0x25d046e0 (message: kXR_read (handle: 0x00000000, offset: 1279726713, size: 65763) ).
[2024-11-16 23:19:28.200063 +0000][Debug ][ExDbgMsg ][ 1405] [heplns148.pp.rl.ac.uk:51996] Moving MsgHandler: 0x25d046e0 (message: kXR_read (handle: 0x00000000, offset: 1279726713, size: 65763) ) from out-queu to in-queue.
[2024-11-16 23:19:28.214142 +0000][Debug ][ExDbgMsg ][ 1405] [msg: 0x16d0f9d0] Assigned MsgHandler: 0x25d046e0.
[2024-11-16 23:19:28.214209 +0000][Debug ][ExDbgMsg ][ 1405] [handler: 0x25d046e0] Removed MsgHandler: 0x25d046e0 from the in-queue.
[2024-11-16 23:19:28.241824 +0000][Debug ][ExDbgMsg ][ 1405] [heplns148.pp.rl.ac.uk:51996] Calling MsgHandler: 0x25d046e0 (message: kXR_read (handle: 0x00000000, offset: 1279726713, size: 65763) ) with status: [SUCCESS] .
[2024-11-16 23:19:28.241905 +0000][Debug ][ExDbgMsg ][ 1405] [heplns148.pp.rl.ac.uk:51996] Destroying MsgHandler: 0x25d046e0.
[2024-11-16 23:19:28.345013 +0000][Debug ][File ][ 1405] [0x114821e0@root://mover.pp.rl.ac.uk:1094/pnfs/pp.rl.ac.uk/data/dune/fardet-hd/d0/5f/nutau_dune10kt_1x2x6_1116_102_20230827T173330Z_gen_g4_detsim_hitreco__20240220T194204Z_reco2.root?xrdcl.requuid=bf508eca-de0a-431c-ab0c-98af794b67a6] Sending a close command for handle 0x0 to heplns148.pp.rl.ac.uk:51996
[2024-11-16 23:19:28.345081 +0000][Debug ][ExDbgMsg ][ 1405] [heplns148.pp.rl.ac.uk:51996] MsgHandler created: 0x23b0ac50 (message: kXR_close (handle: 0x00000000) ).
[2024-11-16 23:19:28.345217 +0000][Debug ][ExDbgMsg ][ 1405] [heplns148.pp.rl.ac.uk:51996] Moving MsgHandler: 0x23b0ac50 (message: kXR_close (handle: 0x00000000) ) from out-queu to in-queue.
[2024-11-16 23:19:28.359886 +0000][Debug ][ExDbgMsg ][ 1405] [msg: 0x1bb427d0] Assigned MsgHandler: 0x23b0ac50.
[2024-11-16 23:19:28.359946 +0000][Debug ][ExDbgMsg ][ 1405] [handler: 0x23b0ac50] Removed MsgHandler: 0x23b0ac50 from the in-queue.
[2024-11-16 23:19:28.360061 +0000][Debug ][ExDbgMsg ][ 1405] [heplns148.pp.rl.ac.uk:51996] Calling MsgHandler: 0x23b0ac50 (message: kXR_close (handle: 0x00000000) ) with status: [SUCCESS] .
[2024-11-16 23:19:28.360121 +0000][Debug ][File ][ 1405] [0x114821e0@root://mover.pp.rl.ac.uk:1094/pnfs/pp.rl.ac.uk/data/dune/fardet-hd/d0/5f/nutau_dune10kt_1x2x6_1116_102_20230827T173330Z_gen_g4_detsim_hitreco__20240220T194204Z_reco2.root?xrdcl.requuid=bf508eca-de0a-431c-ab0c-98af794b67a6] Close returned from heplns148.pp.rl.ac.uk:51996 with: [SUCCESS]
[2024-11-16 23:19:28.360152 +0000][Debug ][ExDbgMsg ][ 1405] [heplns148.pp.rl.ac.uk:51996] Destroying MsgHandler: 0x23b0ac50.
16-Nov-2024 23:19:28 GMT Closed input file "root://mover.pp.rl.ac.uk:1094/pnfs/pp.rl.ac.uk/data/dune/fardet-hd/d0/5f/nutau_dune10kt_1x2x6_1116_102_20230827T173330Z_gen_g4_detsim_hitreco__20240220T194204Z_reco2.root"
16-Nov-2024 23:19:28 GMT Closed input file "root://mover.pp.rl.ac.uk:1094/pnfs/pp.rl.ac.uk/data/dune/fardet-hd/d0/5f/nutau_dune10kt_1x2x6_1116_102_20230827T173330Z_gen_g4_detsim_hitreco__20240220T194204Z_reco2.root"
TrigReport ---------- Event summary -------------
TrigReport Events total = 100 passed = 100 failed = 0
TrigReport ---------- Modules in End-path ----------
TrigReport Run Success Error Name
TrigReport 100 100 0 ccnuselection
TimeReport ---------- Time summary [sec] -------
TimeReport CPU = 8434.255401 Real = 8482.460964
MemReport ---------- Memory summary [base-10 MB] ------
MemReport VmPeak = 4097.84 VmHWM = 2572.24
Art has completed and will exit with status 0.
[2024-11-16 23:19:29.015416 +0000][Debug ][JobMgr ][ 1405] Stopping the job manager...
[2024-11-16 23:19:29.016105 +0000][Debug ][JobMgr ][ 1405] Job manager stopped
[2024-11-16 23:19:29.016162 +0000][Debug ][TaskMgr ][ 1405] Stopping the task manager...
[2024-11-16 23:19:29.016425 +0000][Debug ][TaskMgr ][ 1405] Task manager stopped
[2024-11-16 23:19:29.016492 +0000][Debug ][Poller ][ 1405] Stopping the poller...
[2024-11-16 23:19:29.016645 +0000][Debug ][AsyncSock ][ 1405] [heplns148.pp.rl.ac.uk:51996.0] Closing the socket
[2024-11-16 23:19:29.016689 +0000][Debug ][Poller ][ 1405] <[::ffff:192.41.104.55]:39178><--><[::ffff:130.246.47.148]:51996> Removing socket from the poller
[2024-11-16 23:19:29.016738 +0000][Debug ][PostMaster ][ 1405] [heplns148.pp.rl.ac.uk:51996] Destroying stream
[2024-11-16 23:19:29.016753 +0000][Debug ][AsyncSock ][ 1405] [heplns148.pp.rl.ac.uk:51996.0] Closing the socket
=== End last 100 lines of lar log file ===
