21 July 2025: This instance at RAL is read-only. Please do not try submitting new workflows for now.

Jobsub ID 209654.144@justin-prod-sched02.dune.hep.ac.uk

Jobsub ID	209654.144@justin-prod-sched02.dune.hep.ac.uk
Workflow ID	7131
Stage ID	1
User name	smanthey@fnal.gov
HTCondor Group	group_dune
Requested	Processors	1
	GPU	No
	RSS bytes	12582912000 (12000 MiB)
	Wall seconds limit	80000 (22 hours)
Submitted time	2025-05-26 09:06:17
Site	US_FNAL-T1
Entry	CMSHTPC_T1_US_FNAL_condce_opp1_whole
Last heartbeat	2025-05-27 09:47:31
From worker node	Hostname	dunegli-42167-0-cmswn2411.fnal.gov
	cpuinfo	AMD EPYC 7551 32-Core Processor
	OS release	Scientific Linux release 7.9 (Nitrogen)
	Processors	1
	RSS bytes	12582912000 (12000 MiB)
	Wall seconds limit	171000 (47 hours)
	GPU
	Inner Apptainer?	True
Job state	outputting_failed
Allocator name	justin-allocator-pro.dune.hep.ac.uk
Started	2025-05-27 07:47:37
Input files	usertests:fdvd_lowBg_reco2_2025-05-12T_031719Z.root
Jobscript	Exit code	0
	Real time	1h (7178s)
	CPU time	1h (7114s = 99%)
	Max RSS bytes	1382662144 (1318 MiB)
Outputting started	2025-05-27 09:47:16
Output files
Finished	2025-05-27 09:47:31
List job events Wrapper job log

Jobscript log (last 10,000 characters)

uana@BeginModule 27-May-2025 09:30:27 UTC  run: 20000047 subRun: 0 event: 9
[32m*** Matched preselection cluster: 
 - MainTrackID 76084
 - Purity 1.00 Hits 2
 - #AdjCl 18 ( 0 signal )
 - Gen 1 -> marley
 - Reco Time,Y,Z (2482.01, -14.64, 1500.26)
[0m
%MSG
%MSG-i SolarNuAna:  SolarNuAna:solarnuana@BeginModule 27-May-2025 09:30:27 UTC  run: 20000047 subRun: 0 event: 9
[32m*** Matched preselection cluster: 
 - MainTrackID 76081
 - Purity 1.00 Hits 5
 - #AdjCl 11 ( 0 signal )
 - Gen 1 -> marley
 - Reco Time,Y,Z (2487.23, -53.00, 1540.10)
[0m
%MSG
%MSG-i SolarNuAna:  SolarNuAna:solarnuana@BeginModule 27-May-2025 09:30:27 UTC  run: 20000047 subRun: 0 event: 9
[32m*** Matched preselection cluster: 
 - MainTrackID 76081
 - Purity 1.00 Hits 2
 - #AdjCl 12 ( 0 signal )
 - Gen 1 -> marley
 - Reco Time,Y,Z (2046.63, -93.84, 1552.87)
[0m
%MSG
%MSG-i SolarNuAna:  SolarNuAna:solarnuana@BeginModule 27-May-2025 09:30:27 UTC  run: 20000047 subRun: 0 event: 9
[32m*** Matched preselection cluster: 
 - MainTrackID 76081
 - Purity 1.00 Hits 3
 - #AdjCl 13 ( 0 signal )
 - Gen 1 -> marley
 - Reco Time,Y,Z (2424.70, -52.55, 1559.67)
[0m
%MSG
%MSG-i SolarNuAna:  SolarNuAna:solarnuana@BeginModule 27-May-2025 09:30:27 UTC  run: 20000047 subRun: 0 event: 9
[32m*** Matched preselection cluster: 
 - MainTrackID 76081
 - Purity 1.00 Hits 2
 - #AdjCl 12 ( 0 signal )
 - Gen 1 -> marley
 - Reco Time,Y,Z (1907.08, -98.34, 1564.12)
[0m
%MSG
%MSG-i SolarNuAna:  SolarNuAna:solarnuana@BeginModule 27-May-2025 09:30:27 UTC  run: 20000047 subRun: 0 event: 9
[32m*** Matched preselection cluster: 
 - MainTrackID 76081
 - Purity 1.00 Hits 2
 - #AdjCl 14 ( 0 signal )
 - Gen 1 -> marley
 - Reco Time,Y,Z (2432.04, -51.12, 1571.64)
[0m
%MSG
Begin processing the 10th record. run: 20000047 subRun: 0 event: 10 at 27-May-2025 09:30:27 UTC
%MSG-i SolarNuAna:  SolarNuAna:solarnuana@BeginModule 27-May-2025 09:30:30 UTC  run: 20000047 subRun: 0 event: 10
[35m
TPC Frequency in [MHz]: 2.00
TPC Tick in [us]: 0.50
Event Flag: 411617132
Succesfull reset of variables for evt 10
#########################################[0m
%MSG
%MSG-i SolarNuAna:  SolarNuAna:solarnuana@BeginModule 27-May-2025 09:30:30 UTC  run: 20000047 subRun: 0 event: 10
[91m
There are a total of 76816 Particles in the event

# of particles 47	from gen 1 marley
# of particles 31189	from gen 2 Ar39GenInLAr
# of particles 3583	from gen 3 Kr85GenInLAr
# of particles 1	from gen 4 Ar42GenInLAr
# of particles 0	from gen 5 K42From42ArGenInLAr
# of particles 32	from gen 6 Rn222ChainRn222GenInLAr
# of particles 28	from gen 7 Rn222ChainPo218GenInLAr
# of particles 63	from gen 8 Rn222ChainPb214GenInLAr
# of particles 29	from gen 9 Rn222ChainBi214GenInLAr
# of particles 10	from gen 10 Rn222ChainPb210GenInLAr
# of particles 28	from gen 11 Rn220ChainPb212GenInLAr
# of particles 234	from gen 12 K40GenInCathode
# of particles 3087	from gen 13 U238ChainGenInCathode
# of particles 3629	from gen 14 Th232ChainGenInCathode
# of particles 107	from gen 15 K40GenInAnode
# of particles 1318	from gen 16 U238ChainGenInAnode
# of particles 1628	from gen 17 Th232ChainGenInAnode
# of particles 0	from gen 18 Rn222ChainGenInPDS
# of particles 5	from gen 19 K42From42ArGenInUpperMesh
# of particles 3	from gen 20 Rn222ChainFromPo218GenInUpperMesh
# of particles 26	from gen 21 Rn222ChainFromPb214GenInUpperMesh
# of particles 74	from gen 22 Rn222ChainFromBi214GenInUpperMesh
# of particles 38	from gen 23 Rn222ChainFromPb210GenInUpperMesh
# of particles 40	from gen 24 Rn222ChainFromBi210GenInUpperMesh
# of particles 63	from gen 25 Rn220ChainFromPb212GenInUpperMesh
# of particles 28057	from gen 26 CavernwallGammasAtLAr
# of particles 3497	from gen 27 foamGammasAtLAr
# of particles 0	from gen 28 CavernwallNeutronsAtLAr
# of particles 0	from gen 29 CryostatNGammasAtLAr
# of particles 0	from gen 30 CavernNGammasAtLAr[0m
%MSG
%MSG-i SolarNuAna:  SolarNuAna:solarnuana@BeginModule 27-May-2025 09:30:31 UTC  run: 20000047 subRun: 0 event: 10
[33m
Number of Signal Particles: 5
Neutrino Interaction: -1
Neutrino Energy: 42.59 MeV
Position (128.11, 232.54, -45.80) cm
	Gen.	PdgCode		Energy		EndPosition		Mother
------------------------------------------------------------------------
marley	22		1.29	 (-16.69, 223.56, -90.52)	76890
marley	1000190410	38146.00 (-32.43, 220.17, -95.66)	76890
marley	11		1.54	 (-32.31, 220.34, -95.58)	76889
marley	-12		0.16	 (-1691.79, 3757.53, -5433.06)	76889
marley	1000190419	38147.30 (-32.43, 220.17, -95.66)	76889
marley	1000180410	38149.01 (-32.43, 220.17, -95.66)	76772
marley	22		3.48	 (-4.22, 138.86, -106.63)	76772
marley	22		3.31	 (21.42, 239.68, -104.88)	76772
marley	22		2.96	 (-11.83, 266.91, -188.20)	76772
marley	22		2.87	 (-15.66, 302.56, -85.70)	76772
marley	22		1.34	 (-5.49, 313.27, -107.07)	76772
marley	1000070140	13040.20 (-22.70, 244.31, -189.75)	76772
marley	1000070140	13040.20 (-23.04, 244.29, -189.28)	76772
marley	1000070140	13040.20 (-22.77, 244.49, -189.19)	76772
marley	1000070140	13040.20 (-22.77, 244.49, -189.20)	76772
marley	1000260560	52089.81 (-22.41, 245.80, -188.68)	76772
marley	1000070140	13040.20 (-22.62, 245.31, -188.52)	76772
marley	1000260560	52089.81 (-22.55, 245.75, -189.28)	76772
marley	1000070140	13040.20 (-24.62, 246.27, -187.66)	76772
marley	1000070140	13040.20 (-25.06, 246.40, -188.52)	76772
marley	1000070140	13040.20 (-24.51, 246.70, -188.61)	76772
marley	1000070140	13040.20 (-24.45, 246.45, -188.46)	76772
marley	1000070140	13040.20 (-24.61, 246.87, -188.69)	76772
marley	2212		938.27	 (-27.52, 238.35, -169.55)	76772
marley	2212		938.27	 (-28.58, 238.89, -160.27)	76772
marley	2212		938.27	 (-34.59, 235.26, -153.71)	76772
marley	2212		938.28	 (-44.41, 229.50, -139.64)	76772
marley	1000060120	11174.86 (-49.19, 231.91, -137.93)	76772
marley	2212		938.29	 (-68.01, 228.71, -108.08)	76772
marley	2212		938.31	 (-68.05, 229.50, -107.51)	76772
marley	1000180400	37215.54 (-26.50, 254.50, 88.93)	76772
marley	1000180400	37215.54 (-26.70, 245.32, 94.16)	76772
marley	1000180400	37215.54 (65.75, 137.28, 41.52)	76772
marley	1000180400	37215.54 (66.89, 136.99, 40.52)	76772
marley	1000180400	37215.54 (72.93, 131.69, 43.08)	76772
marley	1000180400	37215.54 (74.77, 138.38, 55.50)	76772
marley	1000180400	37215.54 (75.55, 138.84, 54.61)	76772
marley	1000180400	37215.54 (75.77, 138.59, 53.83)	76772
marley	1000180400	37215.55 (75.92, 138.83, 53.67)	76772
marley	1000180400	37215.55 (67.61, 132.61, 56.01)	76772
marley	1000180400	37215.55 (83.78, 145.95, 45.81)	76772
marley	1000180400	37215.55 (84.77, 135.00, 52.94)	76772
marley	1000180400	37215.55 (81.11, 138.14, 55.30)	76772
marley	1000180400	37215.54 (121.93, 149.26, 46.65)	76772
marley	2112		939.69	 (-32.43, 220.17, -95.66)	0
marley	1000190390	36284.86 (128.11, 232.54, -45.80)	0
marley	11		33.58	 (122.11, 231.80, -40.97)	0
Signal Track IDs: 76770; 76771; 76772; 76773; 76774; 76775; 76776; 76777; 76778; 76779; 76780; 76781; 76782; 76783; 76784; 76785; 76786; 76787; 76788; 76789; 76790; 76791; 76792; 76793; 76794; 76795; 76796; 76797; 76798; 76799; 76800; 76801; 76802; 76803; 76805; 76806; 76884; 76885; 76886; 76887; 76888; 76889; 76890; 76891; 76892; 76893; 76894; [0m
%MSG
%MSG-i SolarNuAna:  SolarNuAna:solarnuana@BeginModule 27-May-2025 09:30:31 UTC  run: 20000047 subRun: 0 event: 10
[34m[0m
%MSG
%MSG-i SolarNuAna:  SolarNuAna:solarnuana@BeginModule 27-May-2025 09:30:31 UTC  run: 20000047 subRun: 0 event: 10
[36m
# OpHits (ophit10ppm) in full geometry: 25194
# OpFlashes (solarflash) in full geometry: 0
# Hits (gaushit) in each view: 3131, 3078, 4034, 0
# Cluster from the hits: 1411, 1417, 3456, 0
# Tracks (pmtracktc) in full geometry: 196[0m
%MSG
27-May-2025 09:47:15 UTC  Closed input file "root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/usertests/c6/0a/fdvd_lowBg_reco2_2025-05-12T_031719Z.root"

====================================================================================================================
TimeTracker printout (sec)            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================
Full event                          528.416       712.748       1144.21       637.095       196.592        10     
--------------------------------------------------------------------------------------------------------------------
source:RootInput(read)            0.000730448    0.0105597     0.0269664    0.00589929     0.0105145       10     
end_path:solarnuana:SolarNuAna      528.415       712.737       1144.19       637.081       196.589        10     
====================================================================================================================
%MSG-i NuRandomService:  SolarNuAna:solarnuana@EndJob 27-May-2025 09:47:15 UTC  ModuleEndJob

Summary of seeds computed by the NuRandomService
Random policy: 'perEvent'
  algorithm version: EventTimestamp_v1

%MSG

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 2011.39 MB
  Peak resident set size usage (VmHWM): 1382.66 MB
====================================================================================================

TrigReport ---------- Event summary -------------
TrigReport Events total = 10 passed = 10 failed = 0

TrigReport ---------- Modules in End-path ----------
TrigReport        Run    Success      Error Name
TrigReport         10         10          0 solarnuana

TimeReport ---------- Time summary [sec] -------
TimeReport CPU = 7094.677721 Real = 7157.192052

MemReport  ---------- Memory summary [base-10 MB] ------
MemReport  VmPeak = 2011.39 VmHWM = 1382.66

Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
Output file: fdhd_ana_2025-05-27T_074747Z.root
Output hist file: fdhd_ana_2025-05-27T_074747Z_hist.root
lar exit code 0