justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

21 July 2025: This instance at RAL is read-only. Please do not try submitting new workflows for now.

Jobsub ID 423656.24@justin-prod-sched01.dune.hep.ac.uk

Jobsub ID423656.24@justin-prod-sched01.dune.hep.ac.uk
Workflow ID7981
Stage ID1
User nameimawby@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
GPUNo
RSS bytes4194304000 (4000 MiB)
Wall seconds limit43200 (12 hours)
Submitted time2025-06-27 12:34:58
SiteUK_Edinburgh
EntryDUNE_UK_SGridECDF_ce1
Last heartbeat2025-06-27 12:50:25
From worker nodeHostnamenode2b05.ecdf.ed.ac.uk
cpuinfoIntel(R) Xeon(R) Gold 6338 CPU @ 2.00GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit171000 (47 hours)
GPU
Inner Apptainer?True
Job statefinished
Allocator namejustin-allocator-pro.dune.hep.ac.uk
Started2025-06-27 12:36:15
Input filesfardet-hd:nu_dune10kt_1x2x6_1111_861_20230827T034045Z_gen_g4_detsim_hitreco__20240229T185234Z_reco2.root
JobscriptExit code0
Real time13m (832s)
CPU time8m (539s = 64%)
Max RSS bytes1431584768 (1365 MiB)
Outputting started2025-06-27 12:50:08
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/imawby/splitting_nu_0/07981/1/001/HierarchyRecoPerformance_nu_dune10kt_1x2x6_1111_861_20230827T034045Z_gen_g4_detsim_hitreco__20240229T185234Z_reco2.root
Finished2025-06-27 12:50:25
Saved logsjustin-logs:423656.24-justin-prod-sched01.dune.hep.ac.uk.logs.tgz
List job events     Wrapper job log

Jobscript log (last 10,000 characters)

43899 +0100][Debug  ][ExDbgMsg          ][ 1304] [ds26.farm.particle.cz:20925] Calling MsgHandler: 0x199165a0 (message: kXR_read (handle: 0x00000000, offset: 1281950540, size: 124028) ) with status: [SUCCESS] .
[2025-06-27 13:50:04.543913 +0100][Debug  ][ExDbgMsg          ][ 1304] [ds26.farm.particle.cz:20925] Destroying MsgHandler: 0x199165a0.
[2025-06-27 13:50:04.561172 +0100][Debug  ][File              ][ 1304] [0xa6b5dd0@root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/5a/fc/nu_dune10kt_1x2x6_1111_861_20230827T034045Z_gen_g4_detsim_hitreco__20240229T185234Z_reco2.root?xrdcl.requuid=81604f1b-e822-4b5b-81e1-dd0f58457637] Sending a read command for handle 0x0 to ds26.farm.particle.cz:20925
[2025-06-27 13:50:04.561228 +0100][Debug  ][ExDbgMsg          ][ 1304] [ds26.farm.particle.cz:20925] MsgHandler created: 0x2689c500 (message: kXR_read (handle: 0x00000000, offset: 166695, size: 132) ).
[2025-06-27 13:50:04.561299 +0100][Debug  ][ExDbgMsg          ][ 1304] [ds26.farm.particle.cz:20925] Moving MsgHandler: 0x2689c500 (message: kXR_read (handle: 0x00000000, offset: 166695, size: 132) ) from out-queu to in-queue.
[2025-06-27 13:50:04.592634 +0100][Debug  ][ExDbgMsg          ][ 1304] [msg: 0x1b64e580] Assigned MsgHandler: 0x2689c500.
[2025-06-27 13:50:04.592678 +0100][Debug  ][ExDbgMsg          ][ 1304] [handler: 0x2689c500] Removed MsgHandler: 0x2689c500 from the in-queue.
[2025-06-27 13:50:04.592719 +0100][Debug  ][ExDbgMsg          ][ 1304] [ds26.farm.particle.cz:20925] Calling MsgHandler: 0x2689c500 (message: kXR_read (handle: 0x00000000, offset: 166695, size: 132) ) with status: [SUCCESS] .
[2025-06-27 13:50:04.592739 +0100][Debug  ][ExDbgMsg          ][ 1304] [ds26.farm.particle.cz:20925] Destroying MsgHandler: 0x2689c500.
[2025-06-27 13:50:04.592799 +0100][Debug  ][File              ][ 1304] [0xa6b5dd0@root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/5a/fc/nu_dune10kt_1x2x6_1111_861_20230827T034045Z_gen_g4_detsim_hitreco__20240229T185234Z_reco2.root?xrdcl.requuid=81604f1b-e822-4b5b-81e1-dd0f58457637] Sending a read command for handle 0x0 to ds26.farm.particle.cz:20925
[2025-06-27 13:50:04.592823 +0100][Debug  ][ExDbgMsg          ][ 1304] [ds26.farm.particle.cz:20925] MsgHandler created: 0x2689c500 (message: kXR_read (handle: 0x00000000, offset: 166827, size: 138) ).
[2025-06-27 13:50:04.592873 +0100][Debug  ][ExDbgMsg          ][ 1304] [ds26.farm.particle.cz:20925] Moving MsgHandler: 0x2689c500 (message: kXR_read (handle: 0x00000000, offset: 166827, size: 138) ) from out-queu to in-queue.
[2025-06-27 13:50:04.624097 +0100][Debug  ][ExDbgMsg          ][ 1304] [msg: 0x1b64e7a0] Assigned MsgHandler: 0x2689c500.
[2025-06-27 13:50:04.624114 +0100][Debug  ][ExDbgMsg          ][ 1304] [handler: 0x2689c500] Removed MsgHandler: 0x2689c500 from the in-queue.
[2025-06-27 13:50:04.624137 +0100][Debug  ][ExDbgMsg          ][ 1304] [ds26.farm.particle.cz:20925] Calling MsgHandler: 0x2689c500 (message: kXR_read (handle: 0x00000000, offset: 166827, size: 138) ) with status: [SUCCESS] .
[2025-06-27 13:50:04.624150 +0100][Debug  ][ExDbgMsg          ][ 1304] [ds26.farm.particle.cz:20925] Destroying MsgHandler: 0x2689c500.
[2025-06-27 13:50:04.624193 +0100][Debug  ][File              ][ 1304] [0xa6b5dd0@root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/5a/fc/nu_dune10kt_1x2x6_1111_861_20230827T034045Z_gen_g4_detsim_hitreco__20240229T185234Z_reco2.root?xrdcl.requuid=81604f1b-e822-4b5b-81e1-dd0f58457637] Sending a read command for handle 0x0 to ds26.farm.particle.cz:20925
[2025-06-27 13:50:04.624210 +0100][Debug  ][ExDbgMsg          ][ 1304] [ds26.farm.particle.cz:20925] MsgHandler created: 0x1b64df90 (message: kXR_read (handle: 0x00000000, offset: 157909, size: 2206) ).
[2025-06-27 13:50:04.624262 +0100][Debug  ][ExDbgMsg          ][ 1304] [ds26.farm.particle.cz:20925] Moving MsgHandler: 0x1b64df90 (message: kXR_read (handle: 0x00000000, offset: 157909, size: 2206) ) from out-queu to in-queue.
[2025-06-27 13:50:04.666446 +0100][Debug  ][ExDbgMsg          ][ 1304] [msg: 0x1b64e920] Assigned MsgHandler: 0x1b64df90.
[2025-06-27 13:50:04.666463 +0100][Debug  ][ExDbgMsg          ][ 1304] [handler: 0x1b64df90] Removed MsgHandler: 0x1b64df90 from the in-queue.
[2025-06-27 13:50:04.666483 +0100][Debug  ][ExDbgMsg          ][ 1304] [ds26.farm.particle.cz:20925] Calling MsgHandler: 0x1b64df90 (message: kXR_read (handle: 0x00000000, offset: 157909, size: 2206) ) with status: [SUCCESS] .
[2025-06-27 13:50:04.666495 +0100][Debug  ][ExDbgMsg          ][ 1304] [ds26.farm.particle.cz:20925] Destroying MsgHandler: 0x1b64df90.
27-Jun-2025 13:50:07 BST  Closed output file "nu_dune10kt_1x2x6_1111_861_20230827T034045Z_gen_g4_detsim_hitreco__20240229T185234Z_reco2_reco2.root"
[2025-06-27 13:50:07.468491 +0100][Debug  ][File              ][ 1304] [0xa6b5dd0@root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/5a/fc/nu_dune10kt_1x2x6_1111_861_20230827T034045Z_gen_g4_detsim_hitreco__20240229T185234Z_reco2.root?xrdcl.requuid=81604f1b-e822-4b5b-81e1-dd0f58457637] Sending a close command for handle 0x0 to ds26.farm.particle.cz:20925
[2025-06-27 13:50:07.468625 +0100][Debug  ][ExDbgMsg          ][ 1304] [ds26.farm.particle.cz:20925] MsgHandler created: 0xa781420 (message: kXR_close (handle: 0x00000000) ).
[2025-06-27 13:50:07.468696 +0100][Debug  ][ExDbgMsg          ][ 1304] [ds26.farm.particle.cz:20925] Moving MsgHandler: 0xa781420 (message: kXR_close (handle: 0x00000000) ) from out-queu to in-queue.
[2025-06-27 13:50:07.500399 +0100][Debug  ][ExDbgMsg          ][ 1304] [msg: 0x142e8140] Assigned MsgHandler: 0xa781420.
[2025-06-27 13:50:07.500433 +0100][Debug  ][ExDbgMsg          ][ 1304] [handler: 0xa781420] Removed MsgHandler: 0xa781420 from the in-queue.
[2025-06-27 13:50:07.500467 +0100][Debug  ][ExDbgMsg          ][ 1304] [ds26.farm.particle.cz:20925] Calling MsgHandler: 0xa781420 (message: kXR_close (handle: 0x00000000) ) with status: [SUCCESS] .
[2025-06-27 13:50:07.500494 +0100][Debug  ][File              ][ 1304] [0xa6b5dd0@root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/5a/fc/nu_dune10kt_1x2x6_1111_861_20230827T034045Z_gen_g4_detsim_hitreco__20240229T185234Z_reco2.root?xrdcl.requuid=81604f1b-e822-4b5b-81e1-dd0f58457637] Close returned from ds26.farm.particle.cz:20925 with: [SUCCESS] 
[2025-06-27 13:50:07.500521 +0100][Debug  ][ExDbgMsg          ][ 1304] [ds26.farm.particle.cz:20925] Destroying MsgHandler: 0xa781420.
27-Jun-2025 13:50:07 BST  Closed input file "root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/5a/fc/nu_dune10kt_1x2x6_1111_861_20230827T034045Z_gen_g4_detsim_hitreco__20240229T185234Z_reco2.root"

================================================================================================================================
TimeTracker printout (sec)                        Min           Avg           Max         Median          RMS         nEvts   
================================================================================================================================
Full event                                     0.266338       5.27483       170.778       2.80144       16.9293        100    
--------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                         0.0408662     0.0721019     0.201002      0.0662836     0.0156574       100    
reco:pandora:StandardPandora                   0.201333       5.19094       170.607        2.724        16.9191        100    
[art]:TriggerResults:TriggerResultInserter    1.1705e-05    3.19094e-05   0.000519085   1.89115e-05   6.01402e-05      100    
end_path:out1:RootOutput                       2.039e-06    2.91588e-06   1.0362e-05     2.643e-06    9.9496e-07       100    
end_path:out1:RootOutput(write)               0.000760898    0.0115939      0.11508     0.00630251     0.0173696       100    
================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 2408.77 MB
  Peak resident set size usage (VmHWM): 1431.58 MB
====================================================================================================
Art has completed and will exit with status 0.
[2025-06-27 13:50:07.903911 +0100][Debug  ][JobMgr            ][ 1304] Stopping the job manager...
[2025-06-27 13:50:07.904829 +0100][Debug  ][JobMgr            ][ 1304] Job manager stopped
[2025-06-27 13:50:07.905284 +0100][Debug  ][TaskMgr           ][ 1304] Stopping the task manager...
[2025-06-27 13:50:07.905412 +0100][Debug  ][TaskMgr           ][ 1304] Task manager stopped
[2025-06-27 13:50:07.905486 +0100][Debug  ][Poller            ][ 1304] Stopping the poller...
[2025-06-27 13:50:07.905876 +0100][Debug  ][AsyncSock         ][ 1304] [ds26.farm.particle.cz:20925.0] Closing the socket
[2025-06-27 13:50:07.905912 +0100][Debug  ][Poller            ][ 1304] <[::ffff:192.41.105.38]:41592><--><[::ffff:147.231.25.164]:20925> Removing socket from the poller
[2025-06-27 13:50:07.905973 +0100][Debug  ][PostMaster        ][ 1304] [ds26.farm.particle.cz:20925] Destroying stream
[2025-06-27 13:50:07.906107 +0100][Debug  ][AsyncSock         ][ 1304] [ds26.farm.particle.cz:20925.0] Closing the socket
[2025-06-27 13:50:07.906135 +0100][Debug  ][AsyncSock         ][ 1304] [se1.farm.particle.cz:1094.0] Closing the socket
[2025-06-27 13:50:07.906186 +0100][Debug  ][Poller            ][ 1304] <[::ffff:192.41.105.38]:39288><--><[::ffff:147.231.25.100]:1094> Removing socket from the poller
[2025-06-27 13:50:07.906216 +0100][Debug  ][PostMaster        ][ 1304] [se1.farm.particle.cz:1094] Destroying stream
[2025-06-27 13:50:07.906330 +0100][Debug  ][AsyncSock         ][ 1304] [se1.farm.particle.cz:1094.0] Closing the socket
=== End last 100 lines of lar log file ===
justIN time: 2025-08-14 16:39:34 UTC       justIN version: 01.03.02