21 July 2025: This instance at RAL is read-only. Please do not try submitting new workflows for now.
Jobsub ID 212554.83@justin-prod-sched02.dune.hep.ac.uk
| Jobsub ID | 212554.83@justin-prod-sched02.dune.hep.ac.uk | |
| Workflow ID | 7257 | |
| Stage ID | 1 | |
| User name | smanthey@fnal.gov | |
| HTCondor Group | group_dune | |
| Requested | Processors | 1 |
| GPU | No | |
| RSS bytes | 4194304000 (4000 MiB) | |
| Wall seconds limit | 80000 (22 hours) | |
| Submitted time | 2025-05-31 16:43:25 | |
| Site | US_FNAL-T1 | |
| Entry | CMSHTPC_T1_US_FNAL_condce_opp1_whole | |
| Last heartbeat | 2025-06-01 15:50:25 | |
| From worker node | Hostname | dunegli-42342-0-cmswn2385.fnal.gov |
| cpuinfo | Intel(R) Xeon(R) CPU E5-2670 v3 @ 2.30GHz | |
| OS release | Scientific Linux release 7.9 (Nitrogen) | |
| Processors | 1 | |
| RSS bytes | 4194304000 (4000 MiB) | |
| Wall seconds limit | 171000 (47 hours) | |
| GPU | ||
| Inner Apptainer? | True | |
| Job state | outputting_failed | |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk | |
| Started | 2025-06-01 04:29:51 | |
| Input files | fardet-vd:prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250130T213848Z_gen_004818_supernova_g4stage1_g4stage2_detsim_reco.root | |
| Jobscript | Exit code | 0 |
| Real time | 11h (40820s) | |
| CPU time | 11h (40648s = 99%) | |
| Max RSS bytes | 1765769216 (1683 MiB) | |
| Outputting started | 2025-06-01 15:50:15 | |
| Output files | ||
| Finished | 2025-06-01 15:50:25 | |
| List job events Wrapper job log | ||
Jobscript log (last 10,000 characters)
ey -12 1.07 (-4655.16, -2865.39, 3888.80) 133031 marley 1000190419 38147.30 (260.90, 230.52, 1081.95) 133031 marley 1000180410 38149.00 (260.90, 230.52, 1081.95) 132984 marley 22 2.48 (261.56, 239.74, 1069.41) 132984 marley 22 3.83 (280.92, 233.06, 1087.15) 132984 marley 22 2.53 (247.95, 233.06, 1091.97) 132984 marley 22 2.06 (264.85, 219.32, 1095.20) 132984 marley 2212 938.27 (462.89, -43.85, 1214.07) 132984 marley 2212 938.27 (455.68, -48.51, 1195.38) 132984 marley 1000060120 11174.86 (455.63, -54.08, 1194.41) 132984 marley 1000060130 12109.48 (453.84, -59.41, 1212.90) 132984 marley 2212 938.32 (463.25, -56.55, 1209.20) 132984 marley 2212 938.27 (463.85, -60.79, 1205.52) 132984 marley 2212 938.31 (464.25, -84.43, 1191.95) 132984 marley 1000060120 11174.89 (466.00, -87.71, 1183.79) 132984 marley 1000060120 11174.89 (473.14, -77.01, 1185.52) 132984 marley 2212 938.29 (474.94, -80.07, 1187.97) 132984 marley 2212 938.40 (480.57, -86.85, 1199.67) 132984 marley 1000070140 13040.30 (505.25, -104.96, 1206.22) 132984 marley 2212 938.27 (482.83, -90.01, 1206.39) 132984 marley 1000180400 37215.56 (272.08, 62.20, 1224.80) 132984 marley 1000180400 37215.58 (277.96, 71.84, 1212.51) 132984 marley 1000180400 37215.56 (279.33, 60.91, 1232.76) 132984 marley 1000180400 37215.55 (271.99, 61.46, 1233.37) 132984 marley 1000180400 37215.54 (267.91, 67.05, 1227.69) 132984 marley 1000180400 37215.58 (242.45, 69.81, 1215.82) 132984 marley 1000180400 37215.56 (247.62, 79.31, 1224.66) 132984 marley 1000180400 37215.54 (146.35, 52.20, 1402.29) 132984 marley 1000180400 37215.59 (114.99, 59.63, 1424.84) 132984 marley 1000180400 37215.55 (116.44, 60.03, 1422.38) 132984 marley 1000180400 37215.58 (150.38, 90.81, 1402.56) 132984 marley 1000180400 37215.56 (139.88, 96.93, 1409.38) 132984 marley 1000180400 37215.57 (177.82, 217.50, 1466.89) 132984 marley 1000180400 37215.60 (176.60, 217.59, 1468.26) 132984 marley 1000180400 37215.54 (175.67, 213.12, 1461.55) 132984 marley 1000180400 37215.54 (175.33, 212.61, 1460.96) 132984 marley 1000180400 37215.56 (174.67, 212.31, 1459.70) 132984 marley 1000180400 37215.54 (161.89, 216.51, 1458.62) 132984 marley 1000180400 37215.62 (136.65, 217.85, 1455.75) 132984 marley 1000180400 37215.63 (225.64, 186.48, 1433.62) 132984 marley 1000180400 37215.55 (205.80, 202.86, 1434.08) 132984 marley 1000180400 37215.57 (204.99, 205.11, 1434.88) 132984 marley 1000180400 37215.54 (189.93, 203.78, 1438.55) 132984 marley 1000180400 37215.62 (147.84, 177.03, 1445.43) 132984 marley 2112 940.67 (260.90, 230.52, 1081.95) 0 marley 1000020040 3732.03 (149.57, 184.18, 1451.55) 0 marley 1000170350 32564.82 (149.57, 184.18, 1451.55) 0 marley 11 40.97 (144.93, 179.43, 1441.16) 0 Signal Track IDs: 132981; 132982; 132983; 132984; 132985; 132986; 132987; 132988; 132989; 132990; 132991; 132992; 132993; 132994; 132995; 132996; 132997; 132998; 132999; 133000; 133001; 133002; 133003; 133004; 133005; 133006; 133007; 133008; 133009; 133010; 133011; 133012; 133013; 133014; 133015; 133016; 133017; 133018; 133019; 133020; 133021; 133027; 133028; 133029; 133030; 133031; 133032; 133033; 133034; 133035; 133036; [0m %MSG %MSG-i SolarNuAna: SolarNuAna:solarnuana@BeginModule 01-Jun-2025 14:51:02 UTC run: 4992 subRun: 0 event: 48180 [34mOpFlash PE 35.05 with purity 0.66 time 0.70 plane 1 - Vertex (213.34, 743.30, 1641.62) *** 1st Sanity check: Ratio 0.36 <= 1.00 *** 2nd Sanity check: #OpHits 3 >= 3 OpFlash PE 166.03 with purity 0.51 time 0.55 plane 0 - Vertex (-327.50, 471.15, 1437.90) *** 1st Sanity check: Ratio 0.14 <= 1.00 *** 2nd Sanity check: #OpHits 13 >= 13 OpFlash PE 39.20 with purity 0.45 time 0.81 plane 0 - Vertex (-327.50, 385.97, 1113.98) *** 1st Sanity check: Ratio 0.33 <= 1.00 *** 2nd Sanity check: #OpHits 4 >= 4 OpFlash PE 22.87 with purity 0.64 time 1.00 plane 0 - Vertex (-327.50, 604.28, 1751.44) *** 1st Sanity check: Ratio 0.37 <= 1.00 *** 2nd Sanity check: #OpHits 3 >= 3 OpFlash PE 85.47 with purity 0.46 time 0.90 plane 0 - Vertex (-327.50, -280.22, 1659.28) *** 1st Sanity check: Ratio 0.19 <= 1.00 *** 2nd Sanity check: #OpHits 8 >= 8 OpFlash PE 45.18 with purity 0.86 time 1.05 plane 0 - Vertex (-327.50, 443.78, 983.01) *** 1st Sanity check: Ratio 0.21 <= 1.00 *** 2nd Sanity check: #OpHits 5 >= 5 OpFlash PE 126.87 with purity 0.32 time 1.22 plane 0 - Vertex (-327.50, -40.47, 1807.54) *** 1st Sanity check: Ratio 0.12 <= 1.00 *** 2nd Sanity check: #OpHits 12 >= 12 OpFlash PE 90.50 with purity 0.58 time 1.39 plane 1 - Vertex (169.97, 743.30, 1355.55) *** 1st Sanity check: Ratio 0.22 <= 1.00 *** 2nd Sanity check: #OpHits 7 >= 7 OpFlash PE 25.81 with purity 0.00 time 1.35 plane 1 - Vertex (285.07, 743.30, 1704.55) *** 1st Sanity check: Ratio 0.46 <= 1.00 *** 2nd Sanity check: #OpHits 3 >= 3 OpFlash PE 157.60 with purity 0.53 time 1.71 plane 0 - Vertex (-327.50, 413.02, 1465.39) *** 1st Sanity check: Ratio 0.21 <= 1.00 *** 2nd Sanity check: #OpHits 10 >= 10 OpFlash PE 59.46 with purity 0.63 time 1.85 plane 0 - Vertex (-327.50, 172.39, 1669.98) *** 1st Sanity check: Ratio 0.20 <= 1.00 *** 2nd Sanity check: #OpHits 7 >= 7 OpFlash PE 32.18 with purity 0.67 time 1.98 plane 0 - Vertex (-327.50, 561.83, 1794.27) *** 1st Sanity check: Ratio 0.41 <= 1.00 *** 2nd Sanity check: #OpHits 3 >= 3 OpFlash PE 39.63 with purity 0.26 time 1.96 plane 0 - Vertex (-327.50, 373.40, 1377.78) *** 1st Sanity check: Ratio 0.68 <= 1.00 *** 2nd Sanity check: #OpHits 3 >= 3 OpFlash PE 81.26 with purity 0.50 time 2.46 plane 0 - Vertex (-327.50, 5.03, 1537.34) *** 1st Sanity check: Ratio 0.37 <= 1.00 *** 2nd Sanity check: #OpHits 7 >= 7 OpFlash PE 56.16 with purity 0.04 time 2.56 plane 0 - Vertex (-327.50, -206.70, 1231.78) *** 1st Sanity check: Ratio 0.78 <= 1.00 *** 2nd Sanity check: #OpHits 3 >= 3 OpFlash PE 30.86 with purity 0.00 time 2.80 plane 0 - Vertex (-327.50, -328.38, 666.57) *** 1st Sanity check: Ratio 0.39 <= 1.00 *** 2nd Sanity check: #OpHits 4 >= 4 [0m %MSG %MSG-i SolarNuAna: SolarNuAna:solarnuana@BeginModule 01-Jun-2025 14:51:03 UTC run: 4992 subRun: 0 event: 48180 [36m # OpHits (ophit10ppm) in full geometry: 52486 # OpFlashes (solarflash) in full geometry: 4193 # Hits (gaushit) in each view: 6825, 6793, 6678, 0 # Cluster from the hits: 2847, 2895, 5656, 0 # Tracks (pmtracktc) in full geometry: 0[0m %MSG %MSG-i SolarNuAna: SolarNuAna:solarnuana@BeginModule 01-Jun-2025 15:50:11 UTC run: 4992 subRun: 0 event: 48180 [32m*** Matched preselection cluster: - MainTrackID 133029 - Purity 1.00 Hits 2 - Gen 1 -> marley - TPC 61 - #AdjCl 17 ( 5 signal ) - Reco Time,Y,Z ( 421.18, 218.74, 1077.76 ) *** Matched flash: - Purity 0.00 Plane 3 #Hits 4 PE 2298.25 MaxPE 587.63 - Time -348.10 Fast 0.77 Residual 655.07 - Reco Time,Y,Z ( -348.10, 220.00, 2188.38 ) *** Reconstructed Interaction Vertex: - True X,Y,Z ( 149.57, 184.18, 1451.55 ) - Reco X,Y,Z ( 201.52, 218.74, 1077.76 )[0m %MSG %MSG-i SolarNuAna: SolarNuAna:solarnuana@BeginModule 01-Jun-2025 15:50:11 UTC run: 4992 subRun: 0 event: 48180 [32m*** Matched preselection cluster: - MainTrackID 132981 - Purity 1.00 Hits 22 - Gen 1 -> marley - TPC 77 - #AdjCl 19 ( 0 signal ) - Reco Time,Y,Z ( 1121.88, 179.82, 1447.07 ) *** Matched flash: - Purity 0.00 Plane 4 #Hits 4 PE 2594.38 MaxPE 802.28 - Time 589.41 Fast 0.48 Residual 40307.79 - Reco Time,Y,Z ( 589.41, 220.00, -96.50 ) *** Reconstructed Interaction Vertex: - True X,Y,Z ( 149.57, 184.18, 1451.55 ) - Reco X,Y,Z ( 239.54, 179.82, 1447.07 )[0m %MSG 01-Jun-2025 15:50:11 UTC Closed input file "root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-vd/ba/2d/prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250130T213848Z_gen_004818_supernova_g4stage1_g4stage2_detsim_reco.root" ==================================================================================================================== TimeTracker printout (sec) Min Avg Max Median RMS nEvts ==================================================================================================================== Full event 3512.41 4071.43 4466.33 4151.86 333.315 10 -------------------------------------------------------------------------------------------------------------------- source:RootInput(read) 0.000569522 0.00283715 0.0140774 0.000754252 0.00443702 10 end_path:solarnuana:SolarNuAna 3512.41 4071.42 4466.33 4151.86 333.314 10 ==================================================================================================================== %MSG-i NuRandomService: SolarNuAna:solarnuana@EndJob 01-Jun-2025 15:50:11 UTC ModuleEndJob Summary of seeds computed by the NuRandomService Random policy: 'perEvent' algorithm version: EventTimestamp_v1 %MSG ==================================================================================================== MemoryTracker summary (base-10 MB units used) Peak virtual memory usage (VmPeak) : 2492.72 MB Peak resident set size usage (VmHWM): 1765.77 MB ==================================================================================================== TrigReport ---------- Event summary ------------- TrigReport Events total = 10 passed = 10 failed = 0 TrigReport ---------- Modules in End-path ---------- TrigReport Run Success Error Name TrigReport 10 10 0 solarnuana TimeReport ---------- Time summary [sec] ------- TimeReport CPU = 40612.453217 Real = 40781.500661 MemReport ---------- Memory summary [base-10 MB] ------ MemReport VmPeak = 2492.72 VmHWM = 1765.77 Art has completed and will exit with status 0. === End last 100 lines of lar log file === Output file: fdhd_ana_2025-06-01T_043002Z.root Output hist file: fdhd_ana_2025-06-01T_043002Z_hist.root lar exit code 0