justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

21 July 2025: This instance at RAL is read-only. Please do not try submitting new workflows for now.

Jobsub ID 413077.137@justin-prod-sched01.dune.hep.ac.uk

Jobsub ID413077.137@justin-prod-sched01.dune.hep.ac.uk
Workflow ID7645
Stage ID1
User namehsouza@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
GPUNo
RSS bytes3145728000 (3000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2025-06-14 05:46:40
SiteUK_Durham
EntryDUNE_UK_SGridDurham_ce3
Last heartbeat2025-06-14 06:09:02
From worker nodeHostnamen235.dur.scotgrid.ac.uk
cpuinfoAMD EPYC 7352 24-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes3145728000 (3000 MiB)
Wall seconds limit171000 (47 hours)
GPU
Inner Apptainer?True
Job stateoutputting_failed
Allocator namejustin-allocator-pro.dune.hep.ac.uk
Started2025-06-14 05:48:25
Input filesfardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50515283_178_20231201T103739Z_gen_g4_detsim_hitreco__20240507T184937Z_reco2.root
JobscriptExit code0
Real time5m (301s)
CPU time0m (30s = 9%)
Max RSS bytes916213760 (873 MiB)
Outputting started2025-06-14 05:53:26
Output files
Finished2025-06-14 06:09:02
List job events     Wrapper job log

Jobscript log (last 10,000 characters)

. run: 50515283 subRun: 1 event: 17830 at 14-Jun-2025 06:50:12 BST
Begin processing the 31st record. run: 50515283 subRun: 1 event: 17831 at 14-Jun-2025 06:50:15 BST
Begin processing the 32nd record. run: 50515283 subRun: 1 event: 17832 at 14-Jun-2025 06:50:26 BST
Begin processing the 33rd record. run: 50515283 subRun: 1 event: 17833 at 14-Jun-2025 06:50:29 BST
Begin processing the 34th record. run: 50515283 subRun: 1 event: 17834 at 14-Jun-2025 06:50:31 BST
Begin processing the 35th record. run: 50515283 subRun: 1 event: 17835 at 14-Jun-2025 06:50:34 BST
Begin processing the 36th record. run: 50515283 subRun: 1 event: 17836 at 14-Jun-2025 06:50:36 BST
Begin processing the 37th record. run: 50515283 subRun: 1 event: 17837 at 14-Jun-2025 06:50:39 BST
Begin processing the 38th record. run: 50515283 subRun: 1 event: 17838 at 14-Jun-2025 06:50:43 BST
Begin processing the 39th record. run: 50515283 subRun: 1 event: 17839 at 14-Jun-2025 06:50:46 BST
Begin processing the 40th record. run: 50515283 subRun: 1 event: 17840 at 14-Jun-2025 06:50:49 BST
Begin processing the 41st record. run: 50515283 subRun: 1 event: 17841 at 14-Jun-2025 06:50:50 BST
Begin processing the 42nd record. run: 50515283 subRun: 1 event: 17842 at 14-Jun-2025 06:50:53 BST
Begin processing the 43rd record. run: 50515283 subRun: 1 event: 17843 at 14-Jun-2025 06:50:56 BST
Begin processing the 44th record. run: 50515283 subRun: 1 event: 17844 at 14-Jun-2025 06:50:58 BST
Begin processing the 45th record. run: 50515283 subRun: 1 event: 17845 at 14-Jun-2025 06:51:01 BST
Begin processing the 46th record. run: 50515283 subRun: 1 event: 17846 at 14-Jun-2025 06:51:03 BST
Begin processing the 47th record. run: 50515283 subRun: 1 event: 17847 at 14-Jun-2025 06:51:07 BST
Begin processing the 48th record. run: 50515283 subRun: 1 event: 17848 at 14-Jun-2025 06:51:09 BST
Begin processing the 49th record. run: 50515283 subRun: 1 event: 17849 at 14-Jun-2025 06:51:12 BST
Begin processing the 50th record. run: 50515283 subRun: 1 event: 17850 at 14-Jun-2025 06:51:15 BST
Begin processing the 51st record. run: 50515283 subRun: 1 event: 17851 at 14-Jun-2025 06:51:18 BST
Begin processing the 52nd record. run: 50515283 subRun: 1 event: 17852 at 14-Jun-2025 06:51:21 BST
Begin processing the 53rd record. run: 50515283 subRun: 1 event: 17853 at 14-Jun-2025 06:51:22 BST
Begin processing the 54th record. run: 50515283 subRun: 1 event: 17854 at 14-Jun-2025 06:51:25 BST
Begin processing the 55th record. run: 50515283 subRun: 1 event: 17855 at 14-Jun-2025 06:51:26 BST
Begin processing the 56th record. run: 50515283 subRun: 1 event: 17856 at 14-Jun-2025 06:51:28 BST
Begin processing the 57th record. run: 50515283 subRun: 1 event: 17857 at 14-Jun-2025 06:51:31 BST
Begin processing the 58th record. run: 50515283 subRun: 1 event: 17858 at 14-Jun-2025 06:51:34 BST
Begin processing the 59th record. run: 50515283 subRun: 1 event: 17859 at 14-Jun-2025 06:51:37 BST
Begin processing the 60th record. run: 50515283 subRun: 1 event: 17860 at 14-Jun-2025 06:51:38 BST
Begin processing the 61st record. run: 50515283 subRun: 1 event: 17861 at 14-Jun-2025 06:51:39 BST
Begin processing the 62nd record. run: 50515283 subRun: 1 event: 17862 at 14-Jun-2025 06:51:42 BST
Begin processing the 63rd record. run: 50515283 subRun: 1 event: 17863 at 14-Jun-2025 06:51:45 BST
Begin processing the 64th record. run: 50515283 subRun: 1 event: 17864 at 14-Jun-2025 06:51:47 BST
Begin processing the 65th record. run: 50515283 subRun: 1 event: 17865 at 14-Jun-2025 06:51:50 BST
Begin processing the 66th record. run: 50515283 subRun: 1 event: 17866 at 14-Jun-2025 06:51:52 BST
Begin processing the 67th record. run: 50515283 subRun: 1 event: 17867 at 14-Jun-2025 06:51:56 BST
Begin processing the 68th record. run: 50515283 subRun: 1 event: 17868 at 14-Jun-2025 06:51:58 BST
Begin processing the 69th record. run: 50515283 subRun: 1 event: 17869 at 14-Jun-2025 06:52:01 BST
Begin processing the 70th record. run: 50515283 subRun: 1 event: 17870 at 14-Jun-2025 06:52:04 BST
Begin processing the 71st record. run: 50515283 subRun: 1 event: 17871 at 14-Jun-2025 06:52:08 BST
Begin processing the 72nd record. run: 50515283 subRun: 1 event: 17872 at 14-Jun-2025 06:52:11 BST
Begin processing the 73rd record. run: 50515283 subRun: 1 event: 17873 at 14-Jun-2025 06:52:14 BST
Begin processing the 74th record. run: 50515283 subRun: 1 event: 17874 at 14-Jun-2025 06:52:16 BST
Begin processing the 75th record. run: 50515283 subRun: 1 event: 17875 at 14-Jun-2025 06:52:20 BST
Begin processing the 76th record. run: 50515283 subRun: 1 event: 17876 at 14-Jun-2025 06:52:22 BST
Begin processing the 77th record. run: 50515283 subRun: 1 event: 17877 at 14-Jun-2025 06:52:25 BST
Begin processing the 78th record. run: 50515283 subRun: 1 event: 17878 at 14-Jun-2025 06:52:27 BST
Begin processing the 79th record. run: 50515283 subRun: 1 event: 17879 at 14-Jun-2025 06:52:30 BST
Begin processing the 80th record. run: 50515283 subRun: 1 event: 17880 at 14-Jun-2025 06:52:32 BST
Begin processing the 81st record. run: 50515283 subRun: 1 event: 17881 at 14-Jun-2025 06:52:35 BST
Begin processing the 82nd record. run: 50515283 subRun: 1 event: 17882 at 14-Jun-2025 06:52:37 BST
Begin processing the 83rd record. run: 50515283 subRun: 1 event: 17883 at 14-Jun-2025 06:52:41 BST
Begin processing the 84th record. run: 50515283 subRun: 1 event: 17884 at 14-Jun-2025 06:52:42 BST
Begin processing the 85th record. run: 50515283 subRun: 1 event: 17885 at 14-Jun-2025 06:52:46 BST
Begin processing the 86th record. run: 50515283 subRun: 1 event: 17886 at 14-Jun-2025 06:52:48 BST
Begin processing the 87th record. run: 50515283 subRun: 1 event: 17887 at 14-Jun-2025 06:52:51 BST
Begin processing the 88th record. run: 50515283 subRun: 1 event: 17888 at 14-Jun-2025 06:52:52 BST
Begin processing the 89th record. run: 50515283 subRun: 1 event: 17889 at 14-Jun-2025 06:52:55 BST
Begin processing the 90th record. run: 50515283 subRun: 1 event: 17890 at 14-Jun-2025 06:52:57 BST
Begin processing the 91st record. run: 50515283 subRun: 1 event: 17891 at 14-Jun-2025 06:52:58 BST
Begin processing the 92nd record. run: 50515283 subRun: 1 event: 17892 at 14-Jun-2025 06:53:01 BST
Begin processing the 93rd record. run: 50515283 subRun: 1 event: 17893 at 14-Jun-2025 06:53:04 BST
Begin processing the 94th record. run: 50515283 subRun: 1 event: 17894 at 14-Jun-2025 06:53:06 BST
Begin processing the 95th record. run: 50515283 subRun: 1 event: 17895 at 14-Jun-2025 06:53:09 BST
Begin processing the 96th record. run: 50515283 subRun: 1 event: 17896 at 14-Jun-2025 06:53:13 BST
Begin processing the 97th record. run: 50515283 subRun: 1 event: 17897 at 14-Jun-2025 06:53:16 BST
Begin processing the 98th record. run: 50515283 subRun: 1 event: 17898 at 14-Jun-2025 06:53:18 BST
Begin processing the 99th record. run: 50515283 subRun: 1 event: 17899 at 14-Jun-2025 06:53:20 BST
Begin processing the 100th record. run: 50515283 subRun: 1 event: 17900 at 14-Jun-2025 06:53:23 BST
14-Jun-2025 06:53:26 BST  Closed input file "root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/79/47/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50515283_178_20231201T103739Z_gen_g4_detsim_hitreco__20240507T184937Z_reco2.root"

================================================================================================================================
TimeTracker printout (sec)                        Min           Avg           Max         Median          RMS         nEvts   
================================================================================================================================
Full event                                     0.678177       2.16716       6.01233       2.19762      0.625337        100    
--------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                         0.0343811     0.0627175     0.125102      0.0684711     0.0219907       100    
prod:pandorapid:Chi2ParticleID                 0.203658      0.303221      0.419667      0.317959      0.0519371       100    
prod:energyreconue:EnergyReco                   0.24834       0.71318       1.14894      0.724047      0.186004        100    
prod:energyreconumu:EnergyReco                0.00175882     0.173362       1.15707      0.166391      0.158125        100    
prod:energyreconc:EnergyReco                  0.00243849     0.0498334     0.469179      0.0497322     0.0582142       100    
[art]:TriggerResults:TriggerResultInserter     8.526e-06    1.26898e-05   7.2416e-05    1.1622e-05    6.5126e-06       100    
end_path:ana:PIDASelectionAna                  0.139096      0.864375       2.74855      0.847485      0.310988        100    
================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 14968.9 MB
  Peak resident set size usage (VmHWM): 904.217 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
.:
total 104
-rw-r--r--. 1 dune001 dune 66162 Jun 14 06:53 atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50515283_178_20231201T103739Z_gen_g4_detsim_hitreco__20240507T184937Z_reco2_pida_2025-06-14T_054830Z.root
-rw-r--r--. 1 dune001 dune 14218 Jun 14 06:53 atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50515283_178_20231201T103739Z_gen_g4_detsim_hitreco__20240507T184937Z_reco2_reco_2025-06-14T_054830Z.log
-rw-r--r--. 1 dune001 dune 11973 Jun 14 06:53 jobscript.log
-rw-r--r--. 1 dune001 dune   187 Jun 14 06:53 justin-processed-pfns.txt
-rw-r--r--. 1 dune001 dune   138 Jun 14 06:48 all-input-dids.txt
-rw-r--r--. 1 dune001 dune     0 Jun 14 06:48 debugprod.log
justIN time: 2025-08-14 22:17:51 UTC       justIN version: 01.03.02