21 July 2025: This instance at RAL is read-only. Please do not try submitting new workflows for now.
Jobsub ID 209655.46@justin-prod-sched02.dune.hep.ac.uk
| Jobsub ID | 209655.46@justin-prod-sched02.dune.hep.ac.uk | |
| Workflow ID | 7131 | |
| Stage ID | 1 | |
| User name | smanthey@fnal.gov | |
| HTCondor Group | group_dune | |
| Requested | Processors | 1 |
| GPU | No | |
| RSS bytes | 12582912000 (12000 MiB) | |
| Wall seconds limit | 80000 (22 hours) | |
| Submitted time | 2025-05-26 14:31:38 | |
| Site | US_FNAL-T1 | |
| Entry | CMSHTPC_T1_US_FNAL_condce_opp1_whole | |
| Last heartbeat | 2025-05-27 13:15:18 | |
| From worker node | Hostname | dunegli-42167-0-cmswn2411.fnal.gov |
| cpuinfo | AMD EPYC 7551 32-Core Processor | |
| OS release | Scientific Linux release 7.9 (Nitrogen) | |
| Processors | 1 | |
| RSS bytes | 12582912000 (12000 MiB) | |
| Wall seconds limit | 171000 (47 hours) | |
| GPU | ||
| Inner Apptainer? | True | |
| Job state | outputting_failed | |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk | |
| Started | 2025-05-27 10:23:52 | |
| Input files | usertests:fdvd_lowBg_reco2_2025-05-12T_112532Z.root | |
| Jobscript | Exit code | 0 |
| Real time | 2h (10274s) | |
| CPU time | 2h (10187s = 99%) | |
| Max RSS bytes | 1376870400 (1313 MiB) | |
| Outputting started | 2025-05-27 13:15:07 | |
| Output files | ||
| Finished | 2025-05-27 13:15:18 | |
| List job events Wrapper job log | ||
Jobscript log (last 10,000 characters)
y 1000190400 37216.66 (-309.53, 425.01, 1500.43) 0 marley 11 52.47 (-309.80, 412.07, 1505.98) 0 Signal Track IDs: 76605; 76606; 76607; 76608; 76609; 76670; 76671; 76672; [0m %MSG %MSG-i SolarNuAna: SolarNuAna:solarnuana@BeginModule 27-May-2025 12:42:37 UTC run: 20000047 subRun: 0 event: 9 [34m[0m %MSG %MSG-i SolarNuAna: SolarNuAna:solarnuana@BeginModule 27-May-2025 12:42:37 UTC run: 20000047 subRun: 0 event: 9 [36m # OpHits (ophit10ppm) in full geometry: 25582 # OpFlashes (solarflash) in full geometry: 0 # Hits (gaushit) in each view: 3384, 3315, 4176, 0 # Cluster from the hits: 1507, 1521, 3548, 0 # Tracks (pmtracktc) in full geometry: 188[0m %MSG %MSG-w SolarNuAna: SolarNuAna:solarnuana@BeginModule 27-May-2025 12:58:57 UTC run: 20000047 subRun: 0 event: 9 Signal cluster reconstructed outside of detector volume! RecoY = -966.21 %MSG %MSG-i SolarNuAna: SolarNuAna:solarnuana@BeginModule 27-May-2025 12:58:58 UTC run: 20000047 subRun: 0 event: 9 [32m*** Matched preselection cluster: - MainTrackID 76607 - Purity 1.00 Hits 3 - #AdjCl 2 ( 0 signal ) - Gen 1 -> marley - Reco Time,Y,Z (7996.67, 446.01, 1498.99) [0m %MSG %MSG-i SolarNuAna: SolarNuAna:solarnuana@BeginModule 27-May-2025 12:58:58 UTC run: 20000047 subRun: 0 event: 9 [32m*** Matched preselection cluster: - MainTrackID 76605 - Purity 1.00 Hits 19 - #AdjCl 4 ( 0 signal ) - Gen 1 -> marley - Reco Time,Y,Z (7904.18, 415.98, 1503.75) [0m %MSG %MSG-i SolarNuAna: SolarNuAna:solarnuana@BeginModule 27-May-2025 12:58:58 UTC run: 20000047 subRun: 0 event: 9 [32m*** Matched preselection cluster: - MainTrackID 76605 - Purity 1.00 Hits 2 - #AdjCl 6 ( 0 signal ) - Gen 1 -> marley - Reco Time,Y,Z (8048.70, 400.88, 1511.54) [0m %MSG %MSG-i SolarNuAna: SolarNuAna:solarnuana@BeginModule 27-May-2025 12:58:58 UTC run: 20000047 subRun: 0 event: 9 [32m*** Matched preselection cluster: - MainTrackID 76605 - Purity 1.00 Hits 2 - #AdjCl 0 ( 0 signal ) - Gen 1 -> marley - Reco Time,Y,Z (7937.11, -966.21, 1519.76) [0m %MSG Begin processing the 10th record. run: 20000047 subRun: 0 event: 10 at 27-May-2025 12:58:58 UTC %MSG-i SolarNuAna: SolarNuAna:solarnuana@BeginModule 27-May-2025 12:59:01 UTC run: 20000047 subRun: 0 event: 10 [35m TPC Frequency in [MHz]: 2.00 TPC Tick in [us]: 0.50 Event Flag: 1470462713 Succesfull reset of variables for evt 10 #########################################[0m %MSG %MSG-i SolarNuAna: SolarNuAna:solarnuana@BeginModule 27-May-2025 12:59:02 UTC run: 20000047 subRun: 0 event: 10 [91m There are a total of 76473 Particles in the event # of particles 10 from gen 1 marley # of particles 31100 from gen 2 Ar39GenInLAr # of particles 3600 from gen 3 Kr85GenInLAr # of particles 1 from gen 4 Ar42GenInLAr # of particles 1 from gen 5 K42From42ArGenInLAr # of particles 37 from gen 6 Rn222ChainRn222GenInLAr # of particles 32 from gen 7 Rn222ChainPo218GenInLAr # of particles 74 from gen 8 Rn222ChainPb214GenInLAr # of particles 39 from gen 9 Rn222ChainBi214GenInLAr # of particles 35 from gen 10 Rn222ChainPb210GenInLAr # of particles 41 from gen 11 Rn220ChainPb212GenInLAr # of particles 208 from gen 12 K40GenInCathode # of particles 2975 from gen 13 U238ChainGenInCathode # of particles 3592 from gen 14 Th232ChainGenInCathode # of particles 114 from gen 15 K40GenInAnode # of particles 1335 from gen 16 U238ChainGenInAnode # of particles 1560 from gen 17 Th232ChainGenInAnode # of particles 2 from gen 18 Rn222ChainGenInPDS # of particles 0 from gen 19 K42From42ArGenInUpperMesh # of particles 0 from gen 20 Rn222ChainFromPo218GenInUpperMesh # of particles 25 from gen 21 Rn222ChainFromPb214GenInUpperMesh # of particles 25 from gen 22 Rn222ChainFromBi214GenInUpperMesh # of particles 38 from gen 23 Rn222ChainFromPb210GenInUpperMesh # of particles 33 from gen 24 Rn222ChainFromBi210GenInUpperMesh # of particles 42 from gen 25 Rn220ChainFromPb212GenInUpperMesh # of particles 28057 from gen 26 CavernwallGammasAtLAr # of particles 3497 from gen 27 foamGammasAtLAr # of particles 0 from gen 28 CavernwallNeutronsAtLAr # of particles 0 from gen 29 CryostatNGammasAtLAr # of particles 0 from gen 30 CavernNGammasAtLAr[0m %MSG %MSG-i SolarNuAna: SolarNuAna:solarnuana@BeginModule 27-May-2025 12:59:02 UTC run: 20000047 subRun: 0 event: 10 [33m Number of Signal Particles: 9 Neutrino Interaction: -1 Neutrino Energy: 52.30 MeV Position (-46.46, 610.75, 364.51) cm Gen. PdgCode Energy EndPosition Mother ------------------------------------------------------------------------ marley 11 1.17 (-46.55, 610.73, 364.59) 76465 marley -12 0.65 (-4655.16, 2555.76, 1088.89) 76465 marley 1000200400 37214.71 (-46.46, 610.75, 364.51) 76465 marley 22 0.03 (-46.88, 610.50, 364.46) 0 marley 22 0.77 (-39.09, 603.57, 343.19) 0 marley 22 0.84 (-44.58, 596.01, 327.64) 0 marley 22 0.65 (-52.77, 582.11, 352.33) 0 marley 22 2.09 (-45.58, 617.41, 338.37) 0 marley 1000190400 37216.54 (-46.46, 610.75, 364.51) 0 marley 11 46.92 (-58.90, 612.57, 363.56) 0 Signal Track IDs: 76464; 76465; 76466; 76467; 76468; 76469; 76470; 76545; 76546; 76547; [0m %MSG %MSG-i SolarNuAna: SolarNuAna:solarnuana@BeginModule 27-May-2025 12:59:02 UTC run: 20000047 subRun: 0 event: 10 [34m[0m %MSG %MSG-i SolarNuAna: SolarNuAna:solarnuana@BeginModule 27-May-2025 12:59:02 UTC run: 20000047 subRun: 0 event: 10 [36m # OpHits (ophit10ppm) in full geometry: 24790 # OpFlashes (solarflash) in full geometry: 0 # Hits (gaushit) in each view: 3357, 3279, 4171, 0 # Cluster from the hits: 1532, 1504, 3565, 0 # Tracks (pmtracktc) in full geometry: 196[0m %MSG %MSG-i SolarNuAna: SolarNuAna:solarnuana@BeginModule 27-May-2025 13:15:06 UTC run: 20000047 subRun: 0 event: 10 [32m*** Matched preselection cluster: - MainTrackID 76464 - Purity 1.00 Hits 2 - #AdjCl 3 ( 0 signal ) - Gen 1 -> marley - Reco Time,Y,Z (5330.53, 613.65, 352.83) [0m %MSG %MSG-i SolarNuAna: SolarNuAna:solarnuana@BeginModule 27-May-2025 13:15:06 UTC run: 20000047 subRun: 0 event: 10 [32m*** Matched preselection cluster: - MainTrackID 76464 - Purity 1.00 Hits 2 - #AdjCl 3 ( 0 signal ) - Gen 1 -> marley - Reco Time,Y,Z (5230.60, 611.18, 356.38) [0m %MSG %MSG-i SolarNuAna: SolarNuAna:solarnuana@BeginModule 27-May-2025 13:15:06 UTC run: 20000047 subRun: 0 event: 10 [32m*** Matched preselection cluster: - MainTrackID 76466 - Purity 1.00 Hits 2 - #AdjCl 5 ( 0 signal ) - Gen 1 -> marley - Reco Time,Y,Z (4417.79, 590.08, 357.40) [0m %MSG %MSG-i SolarNuAna: SolarNuAna:solarnuana@BeginModule 27-May-2025 13:15:06 UTC run: 20000047 subRun: 0 event: 10 [32m*** Matched preselection cluster: - MainTrackID 76464 - Purity 1.00 Hits 2 - #AdjCl 3 ( 0 signal ) - Gen 1 -> marley - Reco Time,Y,Z (4899.63, 607.80, 358.53) [0m %MSG %MSG-i SolarNuAna: SolarNuAna:solarnuana@BeginModule 27-May-2025 13:15:06 UTC run: 20000047 subRun: 0 event: 10 [32m*** Matched preselection cluster: - MainTrackID 76464 - Purity 1.00 Hits 2 - #AdjCl 3 ( 0 signal ) - Gen 1 -> marley - Reco Time,Y,Z (5025.46, 617.67, 364.10) [0m %MSG %MSG-i SolarNuAna: SolarNuAna:solarnuana@BeginModule 27-May-2025 13:15:06 UTC run: 20000047 subRun: 0 event: 10 [32m*** Matched preselection cluster: - MainTrackID 76464 - Purity 1.00 Hits 1 - #AdjCl 3 ( 0 signal ) - Gen 1 -> marley - Reco Time,Y,Z (5570.31, 622.62, 369.98) [0m %MSG %MSG-i SolarNuAna: SolarNuAna:solarnuana@BeginModule 27-May-2025 13:15:06 UTC run: 20000047 subRun: 0 event: 10 [32m*** Matched preselection cluster: - MainTrackID 76464 - Purity 1.00 Hits 3 - #AdjCl 4 ( 0 signal ) - Gen 1 -> marley - Reco Time,Y,Z (5488.61, 620.84, 373.56) [0m %MSG 27-May-2025 13:15:06 UTC Closed input file "root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/usertests/98/20/fdvd_lowBg_reco2_2025-05-12T_112532Z.root" ==================================================================================================================== TimeTracker printout (sec) Min Avg Max Median RMS nEvts ==================================================================================================================== Full event 904.331 1021.32 1118.44 1009.35 69.1424 10 -------------------------------------------------------------------------------------------------------------------- source:RootInput(read) 0.000703438 0.0222027 0.0724963 0.0156247 0.0235418 10 end_path:solarnuana:SolarNuAna 904.33 1021.3 1118.44 1009.34 69.1383 10 ==================================================================================================================== %MSG-i NuRandomService: SolarNuAna:solarnuana@EndJob 27-May-2025 13:15:06 UTC ModuleEndJob Summary of seeds computed by the NuRandomService Random policy: 'perEvent' algorithm version: EventTimestamp_v1 %MSG ==================================================================================================== MemoryTracker summary (base-10 MB units used) Peak virtual memory usage (VmPeak) : 2005.89 MB Peak resident set size usage (VmHWM): 1376.87 MB ==================================================================================================== TrigReport ---------- Event summary ------------- TrigReport Events total = 10 passed = 10 failed = 0 TrigReport ---------- Modules in End-path ---------- TrigReport Run Success Error Name TrigReport 10 10 0 solarnuana TimeReport ---------- Time summary [sec] ------- TimeReport CPU = 10167.283230 Real = 10249.986374 MemReport ---------- Memory summary [base-10 MB] ------ MemReport VmPeak = 2005.89 VmHWM = 1376.87 Art has completed and will exit with status 0. === End last 100 lines of lar log file === Output file: fdhd_ana_2025-05-27T_102403Z.root Output hist file: fdhd_ana_2025-05-27T_102403Z_hist.root lar exit code 0