21 July 2025: This instance at RAL is read-only. Please do not try submitting new workflows for now.
Jobsub ID 209655.32@justin-prod-sched02.dune.hep.ac.uk
| Jobsub ID | 209655.32@justin-prod-sched02.dune.hep.ac.uk | |
| Workflow ID | 7131 | |
| Stage ID | 1 | |
| User name | smanthey@fnal.gov | |
| HTCondor Group | group_dune | |
| Requested | Processors | 1 |
| GPU | No | |
| RSS bytes | 12582912000 (12000 MiB) | |
| Wall seconds limit | 80000 (22 hours) | |
| Submitted time | 2025-05-26 14:31:38 | |
| Site | US_FNAL-T1 | |
| Entry | CMSHTPC_T1_US_FNAL_condce_opp1_whole | |
| Last heartbeat | 2025-05-27 12:37:47 | |
| From worker node | Hostname | dunegli-42167-0-cmswn2411.fnal.gov |
| cpuinfo | AMD EPYC 7551 32-Core Processor | |
| OS release | Scientific Linux release 7.9 (Nitrogen) | |
| Processors | 1 | |
| RSS bytes | 12582912000 (12000 MiB) | |
| Wall seconds limit | 171000 (47 hours) | |
| GPU | ||
| Inner Apptainer? | True | |
| Job state | outputting_failed | |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk | |
| Started | 2025-05-27 10:05:28 | |
| Input files | usertests:fdvd_lowBg_reco2_2025-05-12T_012106Z.root | |
| Jobscript | Exit code | 0 |
| Real time | 2h (9125s) | |
| CPU time | 2h (9049s = 99%) | |
| Max RSS bytes | 1372532736 (1308 MiB) | |
| Outputting started | 2025-05-27 12:37:35 | |
| Output files | ||
| Finished | 2025-05-27 12:37:47 | |
| List job events Wrapper job log | ||
Jobscript log (last 10,000 characters)
-i SolarNuAna: SolarNuAna:solarnuana@BeginModule 27-May-2025 12:11:13 UTC run: 20000047 subRun: 0 event: 9 [91m There are a total of 76376 Particles in the event # of particles 7 from gen 1 marley # of particles 31257 from gen 2 Ar39GenInLAr # of particles 3523 from gen 3 Kr85GenInLAr # of particles 7 from gen 4 Ar42GenInLAr # of particles 2 from gen 5 K42From42ArGenInLAr # of particles 28 from gen 6 Rn222ChainRn222GenInLAr # of particles 19 from gen 7 Rn222ChainPo218GenInLAr # of particles 60 from gen 8 Rn222ChainPb214GenInLAr # of particles 30 from gen 9 Rn222ChainBi214GenInLAr # of particles 23 from gen 10 Rn222ChainPb210GenInLAr # of particles 35 from gen 11 Rn220ChainPb212GenInLAr # of particles 234 from gen 12 K40GenInCathode # of particles 2897 from gen 13 U238ChainGenInCathode # of particles 3663 from gen 14 Th232ChainGenInCathode # of particles 92 from gen 15 K40GenInAnode # of particles 1330 from gen 16 U238ChainGenInAnode # of particles 1580 from gen 17 Th232ChainGenInAnode # of particles 0 from gen 18 Rn222ChainGenInPDS # of particles 2 from gen 19 K42From42ArGenInUpperMesh # of particles 5 from gen 20 Rn222ChainFromPo218GenInUpperMesh # of particles 21 from gen 21 Rn222ChainFromPb214GenInUpperMesh # of particles 51 from gen 22 Rn222ChainFromBi214GenInUpperMesh # of particles 67 from gen 23 Rn222ChainFromPb210GenInUpperMesh # of particles 41 from gen 24 Rn222ChainFromBi210GenInUpperMesh # of particles 28 from gen 25 Rn220ChainFromPb212GenInUpperMesh # of particles 27779 from gen 26 CavernwallGammasAtLAr # of particles 3595 from gen 27 foamGammasAtLAr # of particles 0 from gen 28 CavernwallNeutronsAtLAr # of particles 0 from gen 29 CryostatNGammasAtLAr # of particles 0 from gen 30 CavernNGammasAtLAr[0m %MSG %MSG-i SolarNuAna: SolarNuAna:solarnuana@BeginModule 27-May-2025 12:11:13 UTC run: 20000047 subRun: 0 event: 9 [33m Number of Signal Particles: 7 Neutrino Interaction: -1 Neutrino Energy: 42.28 MeV Position (-94.85, 288.79, 903.68) cm Gen. PdgCode Energy EndPosition Mother ------------------------------------------------------------------------ marley 11 0.56 (-94.86, 288.79, 903.68) 76371 marley -12 0.52 (-4655.16, -3838.41, -2873.12) 76371 marley 22 2.09 (-119.85, 295.05, 948.19) 0 marley 22 2.62 (-40.03, 275.65, 921.22) 0 marley 2212 940.45 (-94.86, 288.79, 903.69) 0 marley 1000180390 36285.89 (-94.85, 288.79, 903.68) 0 marley 11 26.76 (-104.31, 291.93, 899.79) 0 Signal Track IDs: 76370; 76371; 76372; 76373; 76374; 76418; 76419; [0m %MSG %MSG-i SolarNuAna: SolarNuAna:solarnuana@BeginModule 27-May-2025 12:11:13 UTC run: 20000047 subRun: 0 event: 9 [34m[0m %MSG %MSG-i SolarNuAna: SolarNuAna:solarnuana@BeginModule 27-May-2025 12:11:13 UTC run: 20000047 subRun: 0 event: 9 [36m # OpHits (ophit10ppm) in full geometry: 24648 # OpFlashes (solarflash) in full geometry: 0 # Hits (gaushit) in each view: 3271, 3237, 4020, 0 # Cluster from the hits: 1474, 1499, 3465, 0 # Tracks (pmtracktc) in full geometry: 208[0m %MSG %MSG-i SolarNuAna: SolarNuAna:solarnuana@BeginModule 27-May-2025 12:22:42 UTC run: 20000047 subRun: 0 event: 9 [32m*** Matched preselection cluster: - MainTrackID 76370 - Purity 1.00 Hits 10 - #AdjCl 6 ( 0 signal ) - Gen 1 -> marley - Reco Time,Y,Z (5294.07, 290.49, 901.51) [0m %MSG %MSG-i SolarNuAna: SolarNuAna:solarnuana@BeginModule 27-May-2025 12:22:42 UTC run: 20000047 subRun: 0 event: 9 [32m*** Matched preselection cluster: - MainTrackID 76373 - Purity 1.00 Hits 2 - #AdjCl 5 ( 0 signal ) - Gen 1 -> marley - Reco Time,Y,Z (4636.18, 285.24, 919.94) [0m %MSG %MSG-i SolarNuAna: SolarNuAna:solarnuana@BeginModule 27-May-2025 12:22:42 UTC run: 20000047 subRun: 0 event: 9 [32m*** Matched preselection cluster: - MainTrackID 76374 - Purity 1.00 Hits 2 - #AdjCl 4 ( 0 signal ) - Gen 1 -> marley - Reco Time,Y,Z (5543.63, 289.22, 948.12) [0m %MSG Begin processing the 10th record. run: 20000047 subRun: 0 event: 10 at 27-May-2025 12:22:43 UTC %MSG-i SolarNuAna: SolarNuAna:solarnuana@BeginModule 27-May-2025 12:22:45 UTC run: 20000047 subRun: 0 event: 10 [35m TPC Frequency in [MHz]: 2.00 TPC Tick in [us]: 0.50 Event Flag: 539969191 Succesfull reset of variables for evt 10 #########################################[0m %MSG %MSG-i SolarNuAna: SolarNuAna:solarnuana@BeginModule 27-May-2025 12:22:45 UTC run: 20000047 subRun: 0 event: 10 [91m There are a total of 76371 Particles in the event # of particles 8 from gen 1 marley # of particles 30705 from gen 2 Ar39GenInLAr # of particles 3393 from gen 3 Kr85GenInLAr # of particles 1 from gen 4 Ar42GenInLAr # of particles 0 from gen 5 K42From42ArGenInLAr # of particles 25 from gen 6 Rn222ChainRn222GenInLAr # of particles 27 from gen 7 Rn222ChainPo218GenInLAr # of particles 49 from gen 8 Rn222ChainPb214GenInLAr # of particles 88 from gen 9 Rn222ChainBi214GenInLAr # of particles 28 from gen 10 Rn222ChainPb210GenInLAr # of particles 28 from gen 11 Rn220ChainPb212GenInLAr # of particles 263 from gen 12 K40GenInCathode # of particles 3043 from gen 13 U238ChainGenInCathode # of particles 3792 from gen 14 Th232ChainGenInCathode # of particles 97 from gen 15 K40GenInAnode # of particles 1341 from gen 16 U238ChainGenInAnode # of particles 1736 from gen 17 Th232ChainGenInAnode # of particles 1 from gen 18 Rn222ChainGenInPDS # of particles 3 from gen 19 K42From42ArGenInUpperMesh # of particles 0 from gen 20 Rn222ChainFromPo218GenInUpperMesh # of particles 25 from gen 21 Rn222ChainFromPb214GenInUpperMesh # of particles 40 from gen 22 Rn222ChainFromBi214GenInUpperMesh # of particles 36 from gen 23 Rn222ChainFromPb210GenInUpperMesh # of particles 37 from gen 24 Rn222ChainFromBi210GenInUpperMesh # of particles 51 from gen 25 Rn220ChainFromPb212GenInUpperMesh # of particles 28057 from gen 26 CavernwallGammasAtLAr # of particles 3497 from gen 27 foamGammasAtLAr # of particles 0 from gen 28 CavernwallNeutronsAtLAr # of particles 0 from gen 29 CryostatNGammasAtLAr # of particles 0 from gen 30 CavernNGammasAtLAr[0m %MSG %MSG-i SolarNuAna: SolarNuAna:solarnuana@BeginModule 27-May-2025 12:22:46 UTC run: 20000047 subRun: 0 event: 10 [33m Number of Signal Particles: 7 Neutrino Interaction: -1 Neutrino Energy: 69.28 MeV Position (308.41, -665.52, 1770.91) cm Gen. PdgCode Energy EndPosition Mother ------------------------------------------------------------------------ marley 11 0.75 (308.42, -665.51, 1770.93) 76366 marley -12 1.08 (2804.21, -5033.92, 3713.80) 76366 marley 1000200400 37214.71 (308.41, -665.52, 1770.91) 76366 marley 22 0.03 (308.12, -665.78, 1770.84) 0 marley 22 0.77 (322.61, -684.10, 1771.65) 0 marley 22 4.39 (365.09, -834.43, 1910.63) 0 marley 1000190400 37216.63 (308.41, -665.52, 1770.91) 0 marley 11 63.00 (301.75, -671.75, 1765.23) 0 Signal Track IDs: 76365; 76366; 76367; 76368; 76369; 76429; 76430; 76431; [0m %MSG %MSG-i SolarNuAna: SolarNuAna:solarnuana@BeginModule 27-May-2025 12:22:46 UTC run: 20000047 subRun: 0 event: 10 [34m[0m %MSG %MSG-i SolarNuAna: SolarNuAna:solarnuana@BeginModule 27-May-2025 12:22:46 UTC run: 20000047 subRun: 0 event: 10 [36m # OpHits (ophit10ppm) in full geometry: 24831 # OpFlashes (solarflash) in full geometry: 0 # Hits (gaushit) in each view: 3094, 3021, 4035, 0 # Cluster from the hits: 1413, 1398, 3457, 0 # Tracks (pmtracktc) in full geometry: 189[0m %MSG %MSG-i SolarNuAna: SolarNuAna:solarnuana@BeginModule 27-May-2025 12:37:33 UTC run: 20000047 subRun: 0 event: 10 [32m*** Matched preselection cluster: - MainTrackID 76365 - Purity 1.00 Hits 14 - #AdjCl 5 ( 0 signal ) - Gen 1 -> marley - Reco Time,Y,Z (255.11, -668.66, 1767.33) [0m %MSG 27-May-2025 12:37:33 UTC Closed input file "root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/usertests/2c/12/fdvd_lowBg_reco2_2025-05-12T_012106Z.root" ==================================================================================================================== TimeTracker printout (sec) Min Avg Max Median RMS nEvts ==================================================================================================================== Full event 612.246 907.388 1129.07 941.625 171.393 10 -------------------------------------------------------------------------------------------------------------------- source:RootInput(read) 0.000633127 0.00529225 0.0128077 0.00425069 0.00476868 10 end_path:solarnuana:SolarNuAna 612.245 907.383 1129.06 941.618 171.393 10 ==================================================================================================================== %MSG-i NuRandomService: SolarNuAna:solarnuana@EndJob 27-May-2025 12:37:33 UTC ModuleEndJob Summary of seeds computed by the NuRandomService Random policy: 'perEvent' algorithm version: EventTimestamp_v1 %MSG ==================================================================================================== MemoryTracker summary (base-10 MB units used) Peak virtual memory usage (VmPeak) : 2004.85 MB Peak resident set size usage (VmHWM): 1372.53 MB ==================================================================================================== TrigReport ---------- Event summary ------------- TrigReport Events total = 10 passed = 10 failed = 0 TrigReport ---------- Modules in End-path ---------- TrigReport Run Success Error Name TrigReport 10 10 0 solarnuana TimeReport ---------- Time summary [sec] ------- TimeReport CPU = 9029.730631 Real = 9100.743708 MemReport ---------- Memory summary [base-10 MB] ------ MemReport VmPeak = 2004.85 VmHWM = 1372.53 Art has completed and will exit with status 0. === End last 100 lines of lar log file === Output file: fdhd_ana_2025-05-27T_100539Z.root Output hist file: fdhd_ana_2025-05-27T_100539Z_hist.root lar exit code 0