21 July 2025: This instance at RAL is read-only. Please do not try submitting new workflows for now.
Jobsub ID 221482.78@justin-prod-sched02.dune.hep.ac.uk
| Jobsub ID | 221482.78@justin-prod-sched02.dune.hep.ac.uk | |
| Workflow ID | 7650 | |
| Stage ID | 1 | |
| User name | hsouza@fnal.gov | |
| HTCondor Group | group_dune | |
| Requested | Processors | 1 |
| GPU | No | |
| RSS bytes | 3145728000 (3000 MiB) | |
| Wall seconds limit | 80000 (22 hours) | |
| Submitted time | 2025-06-14 22:05:29 | |
| Site | UK_Edinburgh | |
| Entry | DUNE_UK_SGridECDF_ce1_multicore | |
| Last heartbeat | 2025-06-14 22:31:51 | |
| From worker node | Hostname | node2b23.ecdf.ed.ac.uk |
| cpuinfo | Intel(R) Xeon(R) Gold 6338 CPU @ 2.00GHz | |
| OS release | Scientific Linux release 7.9 (Nitrogen) | |
| Processors | 1 | |
| RSS bytes | 3145728000 (3000 MiB) | |
| Wall seconds limit | 171000 (47 hours) | |
| GPU | ||
| Inner Apptainer? | True | |
| Job state | outputting_failed | |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk | |
| Started | 2025-06-14 22:07:18 | |
| Input files | fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6220954_385_20231122T092634Z_gen_g4_detsim_hitreco__20240503T065700Z_reco2.root | |
| Jobscript | Exit code | 0 |
| Real time | 2m (149s) | |
| CPU time | 0m (41s = 27%) | |
| Max RSS bytes | 1440215040 (1373 MiB) | |
| Outputting started | 2025-06-14 22:09:47 | |
| Output files | ||
| Finished | 2025-06-14 22:31:51 | |
| List job events Wrapper job log | ||
Jobscript log (last 10,000 characters)
0kt_1x2x6_6220954_385_20231122T092634Z_gen_g4_detsim_hitreco__20240503T065700Z_reco2.root?xrdcl.requuid=f943764e-83af-468a-94c2-b3d915d647f2] Sending a read command for handle 0x0 to xrootd1.esc.qmul.ac.uk:1094 [2025-06-14 23:09:46.982852 +0100][Debug ][ExDbgMsg ][ 1315] [xrootd1.esc.qmul.ac.uk:1094] MsgHandler created: 0x11d83f70 (message: kXR_read (handle: 0x00000000, offset: 1409428835, size: 107) ). [2025-06-14 23:09:46.982905 +0100][Debug ][ExDbgMsg ][ 1315] [xrootd1.esc.qmul.ac.uk:1094] Moving MsgHandler: 0x11d83f70 (message: kXR_read (handle: 0x00000000, offset: 1409428835, size: 107) ) from out-queu to in-queue. [2025-06-14 23:09:46.995884 +0100][Debug ][ExDbgMsg ][ 1315] [msg: 0xc1574e0] Assigned MsgHandler: 0x11d83f70. [2025-06-14 23:09:46.995893 +0100][Debug ][ExDbgMsg ][ 1315] [handler: 0x11d83f70] Removed MsgHandler: 0x11d83f70 from the in-queue. [2025-06-14 23:09:46.995925 +0100][Debug ][ExDbgMsg ][ 1315] [xrootd1.esc.qmul.ac.uk:1094] Calling MsgHandler: 0x11d83f70 (message: kXR_read (handle: 0x00000000, offset: 1409428835, size: 107) ) with status: [SUCCESS] . [2025-06-14 23:09:46.995936 +0100][Debug ][ExDbgMsg ][ 1315] [xrootd1.esc.qmul.ac.uk:1094] Destroying MsgHandler: 0x11d83f70. [2025-06-14 23:09:46.995955 +0100][Debug ][File ][ 1315] [0x81c22f0@root://xrootd1.esc.qmul.ac.uk:1094//dune/RSE/fardet-hd/40/16/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6220954_385_20231122T092634Z_gen_g4_detsim_hitreco__20240503T065700Z_reco2.root?xrdcl.requuid=f943764e-83af-468a-94c2-b3d915d647f2] Sending a read command for handle 0x0 to xrootd1.esc.qmul.ac.uk:1094 [2025-06-14 23:09:46.995972 +0100][Debug ][ExDbgMsg ][ 1315] [xrootd1.esc.qmul.ac.uk:1094] MsgHandler created: 0x11d83f70 (message: kXR_read (handle: 0x00000000, offset: 1409428942, size: 113) ). [2025-06-14 23:09:46.996003 +0100][Debug ][ExDbgMsg ][ 1315] [xrootd1.esc.qmul.ac.uk:1094] Moving MsgHandler: 0x11d83f70 (message: kXR_read (handle: 0x00000000, offset: 1409428942, size: 113) ) from out-queu to in-queue. [2025-06-14 23:09:47.009094 +0100][Debug ][ExDbgMsg ][ 1315] [msg: 0x11d86f50] Assigned MsgHandler: 0x11d83f70. [2025-06-14 23:09:47.009126 +0100][Debug ][ExDbgMsg ][ 1315] [handler: 0x11d83f70] Removed MsgHandler: 0x11d83f70 from the in-queue. [2025-06-14 23:09:47.009189 +0100][Debug ][ExDbgMsg ][ 1315] [xrootd1.esc.qmul.ac.uk:1094] Calling MsgHandler: 0x11d83f70 (message: kXR_read (handle: 0x00000000, offset: 1409428942, size: 113) ) with status: [SUCCESS] . [2025-06-14 23:09:47.009204 +0100][Debug ][ExDbgMsg ][ 1315] [xrootd1.esc.qmul.ac.uk:1094] Destroying MsgHandler: 0x11d83f70. [2025-06-14 23:09:47.009232 +0100][Debug ][File ][ 1315] [0x81c22f0@root://xrootd1.esc.qmul.ac.uk:1094//dune/RSE/fardet-hd/40/16/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6220954_385_20231122T092634Z_gen_g4_detsim_hitreco__20240503T065700Z_reco2.root?xrdcl.requuid=f943764e-83af-468a-94c2-b3d915d647f2] Sending a read command for handle 0x0 to xrootd1.esc.qmul.ac.uk:1094 [2025-06-14 23:09:47.009250 +0100][Debug ][ExDbgMsg ][ 1315] [xrootd1.esc.qmul.ac.uk:1094] MsgHandler created: 0x11d83f70 (message: kXR_read (handle: 0x00000000, offset: 1409429055, size: 238) ). [2025-06-14 23:09:47.009291 +0100][Debug ][ExDbgMsg ][ 1315] [xrootd1.esc.qmul.ac.uk:1094] Moving MsgHandler: 0x11d83f70 (message: kXR_read (handle: 0x00000000, offset: 1409429055, size: 238) ) from out-queu to in-queue. [2025-06-14 23:09:47.022442 +0100][Debug ][ExDbgMsg ][ 1315] [msg: 0xc1574e0] Assigned MsgHandler: 0x11d83f70. [2025-06-14 23:09:47.022467 +0100][Debug ][ExDbgMsg ][ 1315] [handler: 0x11d83f70] Removed MsgHandler: 0x11d83f70 from the in-queue. [2025-06-14 23:09:47.022514 +0100][Debug ][ExDbgMsg ][ 1315] [xrootd1.esc.qmul.ac.uk:1094] Calling MsgHandler: 0x11d83f70 (message: kXR_read (handle: 0x00000000, offset: 1409429055, size: 238) ) with status: [SUCCESS] . [2025-06-14 23:09:47.022529 +0100][Debug ][ExDbgMsg ][ 1315] [xrootd1.esc.qmul.ac.uk:1094] Destroying MsgHandler: 0x11d83f70. [2025-06-14 23:09:47.028387 +0100][Debug ][File ][ 1315] [0x81c22f0@root://xrootd1.esc.qmul.ac.uk:1094//dune/RSE/fardet-hd/40/16/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6220954_385_20231122T092634Z_gen_g4_detsim_hitreco__20240503T065700Z_reco2.root?xrdcl.requuid=f943764e-83af-468a-94c2-b3d915d647f2] Sending a close command for handle 0x0 to xrootd1.esc.qmul.ac.uk:1094 [2025-06-14 23:09:47.028498 +0100][Debug ][ExDbgMsg ][ 1315] [xrootd1.esc.qmul.ac.uk:1094] MsgHandler created: 0xba13500 (message: kXR_close (handle: 0x00000000) ). [2025-06-14 23:09:47.028563 +0100][Debug ][ExDbgMsg ][ 1315] [xrootd1.esc.qmul.ac.uk:1094] Moving MsgHandler: 0xba13500 (message: kXR_close (handle: 0x00000000) ) from out-queu to in-queue. [2025-06-14 23:09:47.044030 +0100][Debug ][ExDbgMsg ][ 1315] [msg: 0xb37e240] Assigned MsgHandler: 0xba13500. [2025-06-14 23:09:47.044078 +0100][Debug ][ExDbgMsg ][ 1315] [handler: 0xba13500] Removed MsgHandler: 0xba13500 from the in-queue. [2025-06-14 23:09:47.044112 +0100][Debug ][ExDbgMsg ][ 1315] [xrootd1.esc.qmul.ac.uk:1094] Calling MsgHandler: 0xba13500 (message: kXR_close (handle: 0x00000000) ) with status: [SUCCESS] . [2025-06-14 23:09:47.044247 +0100][Debug ][File ][ 1315] [0x81c22f0@root://xrootd1.esc.qmul.ac.uk:1094//dune/RSE/fardet-hd/40/16/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6220954_385_20231122T092634Z_gen_g4_detsim_hitreco__20240503T065700Z_reco2.root?xrdcl.requuid=f943764e-83af-468a-94c2-b3d915d647f2] Close returned from xrootd1.esc.qmul.ac.uk:1094 with: [SUCCESS] [2025-06-14 23:09:47.044267 +0100][Debug ][ExDbgMsg ][ 1315] [xrootd1.esc.qmul.ac.uk:1094] Destroying MsgHandler: 0xba13500. 14-Jun-2025 23:09:47 BST Closed input file "root://xrootd1.esc.qmul.ac.uk:1094//dune/RSE/fardet-hd/40/16/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6220954_385_20231122T092634Z_gen_g4_detsim_hitreco__20240503T065700Z_reco2.root" ================================================================================================================================ TimeTracker printout (sec) Min Avg Max Median RMS nEvts ================================================================================================================================ Full event 0.265261 1.00185 4.31445 0.911956 0.5327 100 -------------------------------------------------------------------------------------------------------------------------------- source:RootInput(read) 0.0134131 0.0234259 0.0560989 0.0268435 0.00857464 100 prod:pandorapid:Chi2ParticleID 0.0765605 0.13388 0.234372 0.121374 0.0372722 100 prod:energyreconue:EnergyReco 0.102266 0.312665 0.812005 0.287961 0.105175 100 prod:energyreconumu:EnergyReco 0.00127264 0.11794 1.48915 0.0817312 0.224806 100 prod:energyreconc:EnergyReco 0.00287423 0.0386376 0.417455 0.0309093 0.0522321 100 [art]:TriggerResults:TriggerResultInserter 9.881e-06 1.19214e-05 5.1887e-05 1.11515e-05 4.21198e-06 100 end_path:ana:PIDASelectionAna 0.0585553 0.374966 1.65271 0.335466 0.212765 100 ================================================================================================================================ ==================================================================================================== MemoryTracker summary (base-10 MB units used) Peak virtual memory usage (VmPeak) : 11220.3 MB Peak resident set size usage (VmHWM): 1440.22 MB ==================================================================================================== Art has completed and will exit with status 0. [2025-06-14 23:09:47.241507 +0100][Debug ][JobMgr ][ 1315] Stopping the job manager... [2025-06-14 23:09:47.242268 +0100][Debug ][JobMgr ][ 1315] Job manager stopped [2025-06-14 23:09:47.242343 +0100][Debug ][TaskMgr ][ 1315] Stopping the task manager... [2025-06-14 23:09:47.242476 +0100][Debug ][TaskMgr ][ 1315] Task manager stopped [2025-06-14 23:09:47.242489 +0100][Debug ][Poller ][ 1315] Stopping the poller... [2025-06-14 23:09:47.242718 +0100][Debug ][AsyncSock ][ 1315] [xrootd1.esc.qmul.ac.uk:1094.0] Closing the socket [2025-06-14 23:09:47.242762 +0100][Debug ][Poller ][ 1315] <[::ffff:192.41.105.56]:59788><--><[::ffff:194.36.11.135]:1094> Removing socket from the poller [2025-06-14 23:09:47.243025 +0100][Debug ][PostMaster ][ 1315] [xrootd1.esc.qmul.ac.uk:1094] Destroying stream [2025-06-14 23:09:47.243246 +0100][Debug ][AsyncSock ][ 1315] [xrootd1.esc.qmul.ac.uk:1094.0] Closing the socket === End last 100 lines of lar log file === .: total 9624 -rw-r--r-- 1 gl05pi6 eddie_users 9749096 Jun 14 23:09 atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6220954_385_20231122T092634Z_gen_g4_detsim_hitreco__20240503T065700Z_reco2_reco_2025-06-14T_220722Z.log -rw-r--r-- 1 gl05pi6 eddie_users 72924 Jun 14 23:09 atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6220954_385_20231122T092634Z_gen_g4_detsim_hitreco__20240503T065700Z_reco2_pida_2025-06-14T_220722Z.root -rw-r--r-- 1 gl05pi6 eddie_users 19362 Jun 14 23:09 jobscript.log -rw-r--r-- 1 gl05pi6 eddie_users 188 Jun 14 23:09 justin-processed-pfns.txt -rw-r--r-- 1 gl05pi6 eddie_users 137 Jun 14 23:07 all-input-dids.txt -rw-r--r-- 1 gl05pi6 eddie_users 0 Jun 14 23:07 debugprod.log