Jobsub ID 348040.107@justin-prod-sched01.dune.hep.ac.uk

Jobsub ID	348040.107@justin-prod-sched01.dune.hep.ac.uk
Workflow ID	5470
Stage ID	1
User name	higuera@fnal.gov
HTCondor Group	group_dune.prod_mcsim
Requested	Processors	1
	GPU	No
	RSS bytes	4193255424 (3999 MiB)
	Wall seconds limit	80000 (22 hours)
Submitted time	2025-03-07 00:44:39
Site	US_UCSD
Entry	CMSHTPC_T2_US_UCSD_gw6
Last heartbeat	2025-03-07 07:12:50
From worker node	Hostname	mh-7662-7.t2.ucsd.edu
	cpuinfo	AMD EPYC 7662 64-Core Processor
	OS release	Scientific Linux release 7.9 (Nitrogen)
	Processors	1
	RSS bytes	4193255424 (3999 MiB)
	Wall seconds limit	171000 (47 hours)
	GPU
	Inner Apptainer?	True
Job state	finished
Allocator name	justin-allocator-pro.dune.hep.ac.uk
Started	2025-03-07 02:55:14
Input files	monte-carlo-005470-004271
Jobscript	Exit code	0
	Real time	4h (15277s)
	CPU time	3h (14340s = 93%)
	Max RSS bytes	1911414784 (1822 MiB)
Outputting started	2025-03-07 07:09:53
Output files	fardet-hd:prodbackground_radiological_decay0_dune10kt_1x2x6_lateralAPA_20250307T025521Z_gen_004271_supernova_g4_halfActiveVol_detsim_reco.root
Finished	2025-03-07 07:12:50
Saved logs	justin-logs:348040.107-justin-prod-sched01.dune.hep.ac.uk.logs.tgz
List job events Wrapper job log

Jobscript log (last 10,000 characters)

Vs
C:0 T:17 7 XUs and 8 XVs -> 3 XUVs
C:0 T:19 27 XUs and 23 XVs -> 11 XUVs
C:0 T:21 7 XUs and 8 XVs -> 3 XUVs
C:0 T:23 14 XUs and 14 XVs -> 8 XUVs
80 XUVs total
63 collection wire objects
80 potential space points
Neighbour search...
160 tests to find 80 neighbours
Iterating with no regularization...
Begin: 338447
0 304886
1 304876
Now with regularization...
Begin: 241138
0 241133
Begin processing the 44th record. run: 5470 subRun: 0 event: 213544 at 06-Mar-2025 23:09:45 PST
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 14 XUs and 19 XVs -> 9 XUVs
C:0 T:3 24 XUs and 23 XVs -> 14 XUVs
C:0 T:5 5 XUs and 9 XVs -> 3 XUVs
C:0 T:7 3 XUs and 3 XVs -> 0 XUVs
C:0 T:9 10 XUs and 13 XVs -> 5 XUVs
C:0 T:11 16 XUs and 17 XVs -> 11 XUVs
C:0 T:13 5 XUs and 5 XVs -> 5 XUVs
C:0 T:15 6 XUs and 7 XVs -> 4 XUVs
C:0 T:17 2 XUs and 3 XVs -> 1 XUVs
C:0 T:19 5 XUs and 5 XVs -> 3 XUVs
C:0 T:21 16 XUs and 18 XVs -> 9 XUVs
C:0 T:23 16 XUs and 18 XVs -> 10 XUVs
74 XUVs total
55 collection wire objects
74 potential space points
Neighbour search...
148 tests to find 74 neighbours
Iterating with no regularization...
Begin: 221127
0 174823
1 174215
2 174193
Now with regularization...
Begin: 120816
0 120807
Begin processing the 45th record. run: 5470 subRun: 0 event: 213545 at 06-Mar-2025 23:09:45 PST
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 15 XUs and 16 XVs -> 9 XUVs
C:0 T:3 55 XUs and 53 XVs -> 33 XUVs
C:0 T:5 16 XUs and 14 XVs -> 11 XUVs
C:0 T:7 4 XUs and 3 XVs -> 3 XUVs
C:0 T:9 7 XUs and 7 XVs -> 3 XUVs
C:0 T:11 6 XUs and 8 XVs -> 4 XUVs
C:0 T:13 11 XUs and 14 XVs -> 7 XUVs
C:0 T:15 10 XUs and 9 XVs -> 7 XUVs
C:0 T:17 2 XUs and 3 XVs -> 2 XUVs
C:0 T:19 20 XUs and 19 XVs -> 12 XUVs
C:0 T:21 20 XUs and 22 XVs -> 11 XUVs
C:0 T:23 22 XUs and 21 XVs -> 13 XUVs
115 XUVs total
92 collection wire objects
115 potential space points
Neighbour search...
221 tests to find 106 neighbours
Iterating with no regularization...
Begin: 345312
0 315615
1 315615
Now with regularization...
Begin: 229680
0 229677
Begin processing the 46th record. run: 5470 subRun: 0 event: 213546 at 06-Mar-2025 23:09:46 PST
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 23 XUs and 22 XVs -> 12 XUVs
C:0 T:3 33 XUs and 41 XVs -> 21 XUVs
C:0 T:5 15 XUs and 12 XVs -> 6 XUVs
C:0 T:7 4 XUs and 7 XVs -> 3 XUVs
C:0 T:9 21 XUs and 22 XVs -> 16 XUVs
C:0 T:11 11 XUs and 14 XVs -> 5 XUVs
C:0 T:13 10 XUs and 7 XVs -> 6 XUVs
C:0 T:15 15 XUs and 18 XVs -> 8 XUVs
C:0 T:17 8 XUs and 8 XVs -> 6 XUVs
C:0 T:19 2 XUs and 2 XVs -> 0 XUVs
C:0 T:21 21 XUs and 26 XVs -> 9 XUVs
C:0 T:23 21 XUs and 15 XVs -> 10 XUVs
102 XUVs total
80 collection wire objects
102 potential space points
Neighbour search...
206 tests to find 102 neighbours
Iterating with no regularization...
Begin: 300653
0 265612
1 265574
Now with regularization...
Begin: 185414
0 185408
Begin processing the 47th record. run: 5470 subRun: 0 event: 213547 at 06-Mar-2025 23:09:46 PST
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 31 XUs and 26 XVs -> 15 XUVs
C:0 T:3 15 XUs and 18 XVs -> 11 XUVs
C:0 T:5 13 XUs and 14 XVs -> 6 XUVs
C:0 T:7 4 XUs and 6 XVs -> 3 XUVs
C:0 T:9 11 XUs and 10 XVs -> 6 XUVs
C:0 T:11 10 XUs and 10 XVs -> 5 XUVs
C:0 T:13 10 XUs and 6 XVs -> 6 XUVs
C:0 T:15 20 XUs and 16 XVs -> 12 XUVs
C:0 T:17 4 XUs and 4 XVs -> 2 XUVs
C:0 T:19 5 XUs and 7 XVs -> 4 XUVs
C:0 T:21 35 XUs and 40 XVs -> 21 XUVs
C:0 T:23 20 XUs and 19 XVs -> 13 XUVs
104 XUVs total
76 collection wire objects
104 potential space points
Neighbour search...
196 tests to find 88 neighbours
Iterating with no regularization...
Begin: 374361
0 325258
1 324611
2 324601
Now with regularization...
Begin: 248186
0 248182
Begin processing the 48th record. run: 5470 subRun: 0 event: 213548 at 06-Mar-2025 23:09:47 PST
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 48 XUs and 43 XVs -> 29 XUVs
C:0 T:3 23 XUs and 23 XVs -> 15 XUVs
C:0 T:5 15 XUs and 14 XVs -> 8 XUVs
C:0 T:7 11 XUs and 10 XVs -> 5 XUVs
C:0 T:9 11 XUs and 11 XVs -> 8 XUVs
C:0 T:11 3 XUs and 4 XVs -> 3 XUVs
C:0 T:13 14 XUs and 16 XVs -> 9 XUVs
C:0 T:15 10 XUs and 7 XVs -> 5 XUVs
C:0 T:17 10 XUs and 11 XVs -> 8 XUVs
C:0 T:19 15 XUs and 14 XVs -> 10 XUVs
C:0 T:21 17 XUs and 18 XVs -> 10 XUVs
C:0 T:23 28 XUs and 23 XVs -> 14 XUVs
124 XUVs total
93 collection wire objects
124 potential space points
Neighbour search...
238 tests to find 108 neighbours
Iterating with no regularization...
Begin: 281735
0 263773
1 263696
Now with regularization...
Begin: 190546
0 190545
Begin processing the 49th record. run: 5470 subRun: 0 event: 213549 at 06-Mar-2025 23:09:47 PST
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 9 XUs and 12 XVs -> 7 XUVs
C:0 T:3 30 XUs and 32 XVs -> 15 XUVs
C:0 T:5 11 XUs and 11 XVs -> 8 XUVs
C:0 T:7 7 XUs and 10 XVs -> 5 XUVs
C:0 T:9 11 XUs and 12 XVs -> 8 XUVs
C:0 T:11 23 XUs and 21 XVs -> 17 XUVs
C:0 T:13 9 XUs and 12 XVs -> 7 XUVs
C:0 T:15 12 XUs and 10 XVs -> 8 XUVs
C:0 T:17 5 XUs and 7 XVs -> 3 XUVs
C:0 T:19 0 XUs and 2 XVs -> 0 XUVs
C:0 T:21 21 XUs and 23 XVs -> 16 XUVs
C:0 T:23 25 XUs and 24 XVs -> 17 XUVs
111 XUVs total
79 collection wire objects
111 potential space points
Neighbour search...
239 tests to find 128 neighbours
Iterating with no regularization...
Begin: 327061
0 293557
1 293253
2 293244
Now with regularization...
Begin: 211206
0 211199
Begin processing the 50th record. run: 5470 subRun: 0 event: 213550 at 06-Mar-2025 23:09:48 PST
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 17 XUs and 16 XVs -> 9 XUVs
C:0 T:3 27 XUs and 28 XVs -> 13 XUVs
C:0 T:5 7 XUs and 10 XVs -> 4 XUVs
C:0 T:7 27 XUs and 27 XVs -> 20 XUVs
C:0 T:9 14 XUs and 15 XVs -> 7 XUVs
C:0 T:11 10 XUs and 7 XVs -> 5 XUVs
C:0 T:13 6 XUs and 4 XVs -> 3 XUVs
C:0 T:15 7 XUs and 8 XVs -> 5 XUVs
C:0 T:17 15 XUs and 14 XVs -> 8 XUVs
C:0 T:19 30 XUs and 28 XVs -> 17 XUVs
C:0 T:21 19 XUs and 22 XVs -> 13 XUVs
C:0 T:23 21 XUs and 19 XVs -> 14 XUVs
118 XUVs total
87 collection wire objects
118 potential space points
Neighbour search...
228 tests to find 110 neighbours
Iterating with no regularization...
Begin: 322735
0 284192
1 284176
Now with regularization...
Begin: 198164
0 198157
06-Mar-2025 23:09:49 PST  Closed output file "prodbackground_radiological_decay0_dune10kt_1x2x6_lateralAPA_20250307T025521Z_gen_004271_supernova_g4_halfActiveVol_detsim_reco.root"
06-Mar-2025 23:09:49 PST  Closed input file "prodbackground_radiological_decay0_dune10kt_1x2x6_lateralAPA_20250307T025521Z_gen_004271_supernova_g4_halfActiveVol_detsim.root"

================================================================================================================================
TimeTracker printout (sec)                        Min           Avg           Max         Median          RMS         nEvts   
================================================================================================================================
Full event                                      0.04264      0.0488419     0.0847429     0.0471006    0.00713577       50     
--------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                        0.000291286   0.000432685   0.00141695    0.000396603   0.000155965      50     
reco:gaushit:GausHitFinder                     0.0343641     0.0387157     0.0593369     0.037371     0.00499721       50     
reco:spsolve:SpacePointSolver                 0.00207194    0.00286811    0.00900514    0.00275382    0.000958775      50     
reco:hitfd:DisambigFromSpacePoints            0.000732925   0.000995535   0.00381478    0.000939025   0.000420231      50     
reco:rns:RandomNumberSaver                     1.553e-05    2.15453e-05   0.000203124   1.76055e-05   2.59603e-05      50     
[art]:TriggerResults:TriggerResultInserter     9.07e-06     1.05821e-05   4.5191e-05     9.751e-06    4.9899e-06       50     
end_path:out1:RootOutput                       3.12e-06     3.98488e-06   1.9011e-05     3.63e-06     2.16042e-06      50     
end_path:out1:RootOutput(write)               0.00444087    0.00556754     0.0101663    0.00539806    0.000865512      50     
================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 1773.72 MB
  Peak resident set size usage (VmHWM): 864.58 MB
====================================================================================================
Art has completed and will exit with status 0.
written output file: prodbackground_radiological_decay0_dune10kt_1x2x6_lateralAPA_20250307T025521Z_gen_004271.root 109961042
written output file: prodbackground_radiological_decay0_dune10kt_1x2x6_lateralAPA_20250307T025521Z_gen_004271_supernova_g4_halfActiveVol.root 721422725
written output file: prodbackground_radiological_decay0_dune10kt_1x2x6_lateralAPA_20250307T025521Z_gen_004271_supernova_g4_halfActiveVol_detsim.root 1835889372
written output file: prodbackground_radiological_decay0_dune10kt_1x2x6_lateralAPA_20250307T025521Z_gen_004271_supernova_g4_halfActiveVol_detsim_hist.root 12750
written output file: prodbackground_radiological_decay0_dune10kt_1x2x6_lateralAPA_20250307T025521Z_gen_004271_supernova_g4_halfActiveVol_detsim_reco.root 1839363094
written output file: prodbackground_radiological_decay0_dune10kt_1x2x6_lateralAPA_20250307T025521Z_gen_004271_supernova_g4_halfActiveVol_detsim_reco.root.ext.json 1607
written output file: prodbackground_radiological_decay0_dune10kt_1x2x6_lateralAPA_20250307T025521Z_gen_004271_supernova_g4_halfActiveVol_detsim_reco.root.json 1837
written output file: prodbackground_radiological_decay0_dune10kt_1x2x6_lateralAPA_20250307T025521Z_gen_004271_supernova_g4_halfActiveVol_detsim_reco.root.temp.json 1837