justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

21 July 2025: This instance at RAL is read-only. Please do not try submitting new workflows for now.

Jobsub ID 419106.8@justin-prod-sched01.dune.hep.ac.uk

Jobsub ID419106.8@justin-prod-sched01.dune.hep.ac.uk
Workflow ID7805
Stage ID1
User namehsouza@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
GPUNo
RSS bytes3145728000 (3000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2025-06-20 19:23:31
SiteFR_CCIN2P3
EntryDUNE_FR_CCIN2P3_cccondorce01
Last heartbeat2025-06-20 20:01:14
From worker nodeHostnameccwcondor0718
cpuinfoAMD EPYC 7302 16-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes3145728000 (3000 MiB)
Wall seconds limit106200 (29 hours)
GPU
Inner Apptainer?True
Job stateoutputting_failed
Allocator namejustin-allocator-pro.dune.hep.ac.uk
Started2025-06-20 19:43:33
Input filesfardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6409602_319_20231202T150414Z_gen_g4_detsim_hitreco__20240508T023733Z_reco2.root
JobscriptExit code0
Real time2m (177s)
CPU time0m (47s = 26%)
Max RSS bytes1402834944 (1337 MiB)
Outputting started2025-06-20 19:46:31
Output files
Finished2025-06-20 20:01:14
List job events     Wrapper job log

Jobscript log (last 10,000 characters)

th record. run: 6409602 subRun: 1 event: 31930 at 20-Jun-2025 21:44:34 CEST
Begin processing the 31st record. run: 6409602 subRun: 1 event: 31931 at 20-Jun-2025 21:44:35 CEST
Begin processing the 32nd record. run: 6409602 subRun: 1 event: 31932 at 20-Jun-2025 21:44:36 CEST
Begin processing the 33rd record. run: 6409602 subRun: 1 event: 31933 at 20-Jun-2025 21:44:36 CEST
Begin processing the 34th record. run: 6409602 subRun: 1 event: 31934 at 20-Jun-2025 21:44:37 CEST
Begin processing the 35th record. run: 6409602 subRun: 1 event: 31935 at 20-Jun-2025 21:44:39 CEST
Begin processing the 36th record. run: 6409602 subRun: 1 event: 31936 at 20-Jun-2025 21:44:40 CEST
Begin processing the 37th record. run: 6409602 subRun: 1 event: 31937 at 20-Jun-2025 21:44:41 CEST
Begin processing the 38th record. run: 6409602 subRun: 1 event: 31938 at 20-Jun-2025 21:44:45 CEST
Begin processing the 39th record. run: 6409602 subRun: 1 event: 31939 at 20-Jun-2025 21:44:47 CEST
Begin processing the 40th record. run: 6409602 subRun: 1 event: 31940 at 20-Jun-2025 21:44:49 CEST
Begin processing the 41st record. run: 6409602 subRun: 1 event: 31941 at 20-Jun-2025 21:44:50 CEST
Begin processing the 42nd record. run: 6409602 subRun: 1 event: 31942 at 20-Jun-2025 21:44:53 CEST
Begin processing the 43rd record. run: 6409602 subRun: 1 event: 31943 at 20-Jun-2025 21:44:54 CEST
Begin processing the 44th record. run: 6409602 subRun: 1 event: 31944 at 20-Jun-2025 21:45:03 CEST
Begin processing the 45th record. run: 6409602 subRun: 1 event: 31945 at 20-Jun-2025 21:45:04 CEST
Begin processing the 46th record. run: 6409602 subRun: 1 event: 31946 at 20-Jun-2025 21:45:05 CEST
Begin processing the 47th record. run: 6409602 subRun: 1 event: 31947 at 20-Jun-2025 21:45:06 CEST
Begin processing the 48th record. run: 6409602 subRun: 1 event: 31948 at 20-Jun-2025 21:45:07 CEST
Begin processing the 49th record. run: 6409602 subRun: 1 event: 31949 at 20-Jun-2025 21:45:08 CEST
Begin processing the 50th record. run: 6409602 subRun: 1 event: 31950 at 20-Jun-2025 21:45:09 CEST
Begin processing the 51st record. run: 6409602 subRun: 1 event: 31951 at 20-Jun-2025 21:45:10 CEST
Begin processing the 52nd record. run: 6409602 subRun: 1 event: 31952 at 20-Jun-2025 21:45:12 CEST
Begin processing the 53rd record. run: 6409602 subRun: 1 event: 31953 at 20-Jun-2025 21:45:14 CEST
Begin processing the 54th record. run: 6409602 subRun: 1 event: 31954 at 20-Jun-2025 21:45:17 CEST
Begin processing the 55th record. run: 6409602 subRun: 1 event: 31955 at 20-Jun-2025 21:45:19 CEST
Begin processing the 56th record. run: 6409602 subRun: 1 event: 31956 at 20-Jun-2025 21:45:21 CEST
Begin processing the 57th record. run: 6409602 subRun: 1 event: 31957 at 20-Jun-2025 21:45:22 CEST
Begin processing the 58th record. run: 6409602 subRun: 1 event: 31958 at 20-Jun-2025 21:45:24 CEST
Begin processing the 59th record. run: 6409602 subRun: 1 event: 31959 at 20-Jun-2025 21:45:26 CEST
Begin processing the 60th record. run: 6409602 subRun: 1 event: 31960 at 20-Jun-2025 21:45:28 CEST
Begin processing the 61st record. run: 6409602 subRun: 1 event: 31961 at 20-Jun-2025 21:45:30 CEST
Begin processing the 62nd record. run: 6409602 subRun: 1 event: 31962 at 20-Jun-2025 21:45:31 CEST
Begin processing the 63rd record. run: 6409602 subRun: 1 event: 31963 at 20-Jun-2025 21:45:32 CEST
Begin processing the 64th record. run: 6409602 subRun: 1 event: 31964 at 20-Jun-2025 21:45:39 CEST
Begin processing the 65th record. run: 6409602 subRun: 1 event: 31965 at 20-Jun-2025 21:45:40 CEST
Begin processing the 66th record. run: 6409602 subRun: 1 event: 31966 at 20-Jun-2025 21:45:42 CEST
Begin processing the 67th record. run: 6409602 subRun: 1 event: 31967 at 20-Jun-2025 21:45:43 CEST
Begin processing the 68th record. run: 6409602 subRun: 1 event: 31968 at 20-Jun-2025 21:45:44 CEST
Begin processing the 69th record. run: 6409602 subRun: 1 event: 31969 at 20-Jun-2025 21:45:46 CEST
Begin processing the 70th record. run: 6409602 subRun: 1 event: 31970 at 20-Jun-2025 21:45:47 CEST
Begin processing the 71st record. run: 6409602 subRun: 1 event: 31971 at 20-Jun-2025 21:45:47 CEST
Begin processing the 72nd record. run: 6409602 subRun: 1 event: 31972 at 20-Jun-2025 21:45:48 CEST
Begin processing the 73rd record. run: 6409602 subRun: 1 event: 31973 at 20-Jun-2025 21:45:49 CEST
Begin processing the 74th record. run: 6409602 subRun: 1 event: 31974 at 20-Jun-2025 21:45:51 CEST
Begin processing the 75th record. run: 6409602 subRun: 1 event: 31975 at 20-Jun-2025 21:45:52 CEST
Begin processing the 76th record. run: 6409602 subRun: 1 event: 31976 at 20-Jun-2025 21:45:53 CEST
Begin processing the 77th record. run: 6409602 subRun: 1 event: 31977 at 20-Jun-2025 21:45:55 CEST
Begin processing the 78th record. run: 6409602 subRun: 1 event: 31978 at 20-Jun-2025 21:45:58 CEST
Begin processing the 79th record. run: 6409602 subRun: 1 event: 31979 at 20-Jun-2025 21:45:58 CEST
Begin processing the 80th record. run: 6409602 subRun: 1 event: 31980 at 20-Jun-2025 21:46:00 CEST
Begin processing the 81st record. run: 6409602 subRun: 1 event: 31981 at 20-Jun-2025 21:46:01 CEST
Begin processing the 82nd record. run: 6409602 subRun: 1 event: 31982 at 20-Jun-2025 21:46:02 CEST
Begin processing the 83rd record. run: 6409602 subRun: 1 event: 31983 at 20-Jun-2025 21:46:03 CEST
Begin processing the 84th record. run: 6409602 subRun: 1 event: 31984 at 20-Jun-2025 21:46:04 CEST
Begin processing the 85th record. run: 6409602 subRun: 1 event: 31985 at 20-Jun-2025 21:46:05 CEST
Begin processing the 86th record. run: 6409602 subRun: 1 event: 31986 at 20-Jun-2025 21:46:06 CEST
Begin processing the 87th record. run: 6409602 subRun: 1 event: 31987 at 20-Jun-2025 21:46:07 CEST
Begin processing the 88th record. run: 6409602 subRun: 1 event: 31988 at 20-Jun-2025 21:46:08 CEST
Begin processing the 89th record. run: 6409602 subRun: 1 event: 31989 at 20-Jun-2025 21:46:10 CEST
Begin processing the 90th record. run: 6409602 subRun: 1 event: 31990 at 20-Jun-2025 21:46:11 CEST
Begin processing the 91st record. run: 6409602 subRun: 1 event: 31991 at 20-Jun-2025 21:46:13 CEST
Begin processing the 92nd record. run: 6409602 subRun: 1 event: 31992 at 20-Jun-2025 21:46:21 CEST
Begin processing the 93rd record. run: 6409602 subRun: 1 event: 31993 at 20-Jun-2025 21:46:22 CEST
Begin processing the 94th record. run: 6409602 subRun: 1 event: 31994 at 20-Jun-2025 21:46:23 CEST
Begin processing the 95th record. run: 6409602 subRun: 1 event: 31995 at 20-Jun-2025 21:46:24 CEST
Begin processing the 96th record. run: 6409602 subRun: 1 event: 31996 at 20-Jun-2025 21:46:25 CEST
Begin processing the 97th record. run: 6409602 subRun: 1 event: 31997 at 20-Jun-2025 21:46:26 CEST
Begin processing the 98th record. run: 6409602 subRun: 1 event: 31998 at 20-Jun-2025 21:46:27 CEST
Begin processing the 99th record. run: 6409602 subRun: 1 event: 31999 at 20-Jun-2025 21:46:28 CEST
Begin processing the 100th record. run: 6409602 subRun: 1 event: 32000 at 20-Jun-2025 21:46:29 CEST
20-Jun-2025 21:46:30 CEST  Closed input file "root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/b9/c4/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6409602_319_20231202T150414Z_gen_g4_detsim_hitreco__20240508T023733Z_reco2.root"

================================================================================================================================
TimeTracker printout (sec)                        Min           Avg           Max         Median          RMS         nEvts   
================================================================================================================================
Full event                                     0.242759       1.06558       4.68729       0.85973      0.641694        100    
--------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                         0.0182783     0.0411188     0.0858321     0.0430101     0.0138026       100    
prod:pandorapid:Chi2ParticleID                 0.106585       0.19047      0.657117      0.185278      0.0614023       100    
prod:energyreconue:EnergyReco                  0.0835162     0.346686      0.673644       0.34624      0.0791931       100    
prod:energyreconumu:EnergyReco                0.00363455     0.181668       1.3828       0.165723      0.181326        100    
prod:energyreconc:EnergyReco                  0.00343355     0.0500833     0.403123      0.0440662     0.0586262       100    
[art]:TriggerResults:TriggerResultInserter    1.1141e-05    1.76782e-05   0.000112802   1.5945e-05    1.15956e-05      100    
end_path:ana:PIDASelectionAna                 0.00320607     0.254974       2.11021      0.0718353     0.356179        100    
================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 11192.5 MB
  Peak resident set size usage (VmHWM): 1402.83 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
.:
total 64
-rw-r--r-- 1 dune001 lbno 27062 Jun 20 21:46 atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6409602_319_20231202T150414Z_gen_g4_detsim_hitreco__20240508T023733Z_reco2_pida_2025-06-20T_194338Z.root
-rw-r--r-- 1 dune001 lbno 14218 Jun 20 21:46 atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6409602_319_20231202T150414Z_gen_g4_detsim_hitreco__20240508T023733Z_reco2_reco_2025-06-20T_194338Z.log
-rw-r--r-- 1 dune001 lbno 11972 Jun 20 21:46 jobscript.log
-rw-r--r-- 1 dune001 lbno   186 Jun 20 21:46 justin-processed-pfns.txt
-rw-r--r-- 1 dune001 lbno   137 Jun 20 21:43 all-input-dids.txt
-rw-r--r-- 1 dune001 lbno     0 Jun 20 21:43 debugprod.log
justIN time: 2025-08-15 00:23:52 UTC       justIN version: 01.03.02