21 July 2025: This instance at RAL is read-only. Please do not try submitting new workflows for now.
Jobsub ID 209655.36@justin-prod-sched02.dune.hep.ac.uk
| Jobsub ID | 209655.36@justin-prod-sched02.dune.hep.ac.uk | |
| Workflow ID | 7131 | |
| Stage ID | 1 | |
| User name | smanthey@fnal.gov | |
| HTCondor Group | group_dune | |
| Requested | Processors | 1 |
| GPU | No | |
| RSS bytes | 12582912000 (12000 MiB) | |
| Wall seconds limit | 80000 (22 hours) | |
| Submitted time | 2025-05-26 14:31:38 | |
| Site | US_FNAL-T1 | |
| Entry | CMSHTPC_T1_US_FNAL_condce_opp1_whole | |
| Last heartbeat | 2025-05-27 13:02:09 | |
| From worker node | Hostname | dunegli-42167-0-cmswn2411.fnal.gov |
| cpuinfo | AMD EPYC 7551 32-Core Processor | |
| OS release | Scientific Linux release 7.9 (Nitrogen) | |
| Processors | 1 | |
| RSS bytes | 12582912000 (12000 MiB) | |
| Wall seconds limit | 171000 (47 hours) | |
| GPU | ||
| Inner Apptainer? | True | |
| Job state | outputting_failed | |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk | |
| Started | 2025-05-27 10:08:16 | |
| Input files | usertests:fdvd_lowBg_reco2_2025-05-12T_114553Z.root | |
| Jobscript | Exit code | 0 |
| Real time | 2h (10420s) | |
| CPU time | 2h (10334s = 99%) | |
| Max RSS bytes | 1366446080 (1303 MiB) | |
| Outputting started | 2025-05-27 13:01:57 | |
| Output files | ||
| Finished | 2025-05-27 13:02:09 | |
| List job events Wrapper job log | ||
Jobscript log (last 10,000 characters)
marley 22 2.97 (-276.63, 836.03, 1203.97) 76555 marley 22 1.71 (-162.46, 715.55, 1497.52) 76555 marley 2212 938.27 (-295.69, 757.46, 1587.55) 76555 marley 2212 938.27 (-293.86, 758.13, 1603.75) 76555 marley 2212 938.29 (-292.64, 757.36, 1603.97) 76555 marley 2212 938.37 (-292.11, 755.79, 1599.47) 76555 marley 1000180400 37215.54 (-229.62, 504.95, 1576.86) 76555 marley 1000180400 37215.54 (-216.59, 510.18, 1557.73) 76555 marley 1000180400 37215.54 (-236.28, 497.06, 1539.52) 76555 marley 1000180400 37215.55 (-253.54, 499.01, 1553.96) 76555 marley 1000180400 37215.55 (-220.57, 473.05, 1534.60) 76555 marley 1000180400 37215.54 (-368.42, 483.14, 1562.34) 76555 marley 1000180400 37215.54 (-369.84, 476.11, 1570.48) 76555 marley 1000180360 33494.37 (-352.77, 448.79, 1611.14) 76555 marley 1000180400 37215.54 (-348.11, 444.59, 1618.87) 76555 marley 1000180400 37215.55 (-348.40, 439.45, 1620.54) 76555 marley 1000180400 37215.54 (-348.49, 444.51, 1598.86) 76555 marley 1000180400 37215.54 (-348.93, 444.08, 1596.47) 76555 marley 1000180400 37215.55 (-218.71, 355.66, 1530.12) 76555 marley 1000180400 37215.55 (-227.85, 370.33, 1507.66) 76555 marley 1000180400 37215.54 (-195.48, 362.51, 1508.98) 76555 marley 1000180400 37215.55 (-194.95, 362.86, 1509.17) 76555 marley 1000180400 37215.56 (-178.87, 347.32, 1414.19) 76555 marley 1000180400 37215.56 (-177.39, 347.36, 1417.43) 76555 marley 1000180400 37215.55 (-189.23, 341.26, 1381.22) 76555 marley 1000180400 37215.54 (-236.93, 399.43, 1356.48) 76555 marley 1000180400 37215.54 (-238.82, 418.79, 1349.72) 76555 marley 1000180400 37215.57 (-243.58, 446.48, 1352.59) 76555 marley 1000180360 33494.41 (-243.01, 445.19, 1351.76) 76555 marley 1000180400 37215.56 (-270.14, 472.56, 1367.58) 76555 marley 1000180400 37215.58 (-268.69, 477.06, 1346.02) 76555 marley 22 1.46 (-236.62, 481.15, 1378.24) 76535 marley 1000180400 37215.56 (-268.06, 476.95, 1355.34) 76535 marley 2112 940.00 (-161.50, 719.67, 1479.85) 76535 marley 1000180400 37215.72 (-267.59, 474.36, 1359.02) 76535 marley 1000180400 37215.56 (-269.79, 472.92, 1345.23) 76535 marley 22 1.46 (-316.36, 502.11, 1273.34) 76534 marley 1000180400 37215.62 (-267.49, 476.70, 1336.78) 76534 marley 2112 941.68 (-268.06, 476.95, 1355.34) 76534 marley 2112 943.22 (-267.49, 476.70, 1336.78) 0 marley 1000190390 36284.93 (-284.24, 474.43, 1317.89) 0 marley 11 29.71 (-293.23, 479.30, 1316.06) 0 Signal Track IDs: 76532; 76533; 76534; 76535; 76536; 76537; 76553; 76554; 76555; 76556; 76557; 76579; 76580; 76581; 76582; 76583; 76584; 76585; 76586; 76587; 76588; 76589; 76590; 76591; 76592; 76593; 76594; 76595; 76596; 76597; 76598; 76599; 76600; 76601; 76602; 76603; 76604; 76605; 76606; 76608; 76610; 76611; 76612; 76613; 76614; 76615; 76616; 76617; 76618; 76619; 76620; [0m %MSG %MSG-i SolarNuAna: SolarNuAna:solarnuana@BeginModule 27-May-2025 12:28:28 UTC run: 20000047 subRun: 0 event: 9 [34m[0m %MSG %MSG-i SolarNuAna: SolarNuAna:solarnuana@BeginModule 27-May-2025 12:28:28 UTC run: 20000047 subRun: 0 event: 9 [36m # OpHits (ophit10ppm) in full geometry: 25557 # OpFlashes (solarflash) in full geometry: 0 # Hits (gaushit) in each view: 3324, 3283, 4219, 0 # Cluster from the hits: 1501, 1522, 3600, 0 # Tracks (pmtracktc) in full geometry: 209[0m %MSG %MSG-i SolarNuAna: SolarNuAna:solarnuana@BeginModule 27-May-2025 12:45:40 UTC run: 20000047 subRun: 0 event: 9 [32m*** Matched preselection cluster: - MainTrackID 76532 - Purity 1.00 Hits 14 - #AdjCl 2 ( 0 signal ) - Gen 1 -> marley - Reco Time,Y,Z (7651.36, 477.19, 1317.50) [0m %MSG %MSG-i SolarNuAna: SolarNuAna:solarnuana@BeginModule 27-May-2025 12:45:40 UTC run: 20000047 subRun: 0 event: 9 [32m*** Matched preselection cluster: - MainTrackID 76537 - Purity 1.00 Hits 2 - #AdjCl 0 ( 0 signal ) - Gen 1 -> marley - Reco Time,Y,Z (7976.69, 516.78, 1298.48) [0m %MSG Begin processing the 10th record. run: 20000047 subRun: 0 event: 10 at 27-May-2025 12:45:40 UTC %MSG-i SolarNuAna: SolarNuAna:solarnuana@BeginModule 27-May-2025 12:45:43 UTC run: 20000047 subRun: 0 event: 10 [35m TPC Frequency in [MHz]: 2.00 TPC Tick in [us]: 0.50 Event Flag: 1384172758 Succesfull reset of variables for evt 10 #########################################[0m %MSG %MSG-i SolarNuAna: SolarNuAna:solarnuana@BeginModule 27-May-2025 12:45:43 UTC run: 20000047 subRun: 0 event: 10 [91m There are a total of 76780 Particles in the event # of particles 8 from gen 1 marley # of particles 31228 from gen 2 Ar39GenInLAr # of particles 3580 from gen 3 Kr85GenInLAr # of particles 6 from gen 4 Ar42GenInLAr # of particles 0 from gen 5 K42From42ArGenInLAr # of particles 43 from gen 6 Rn222ChainRn222GenInLAr # of particles 34 from gen 7 Rn222ChainPo218GenInLAr # of particles 69 from gen 8 Rn222ChainPb214GenInLAr # of particles 53 from gen 9 Rn222ChainBi214GenInLAr # of particles 16 from gen 10 Rn222ChainPb210GenInLAr # of particles 18 from gen 11 Rn220ChainPb212GenInLAr # of particles 213 from gen 12 K40GenInCathode # of particles 3072 from gen 13 U238ChainGenInCathode # of particles 3554 from gen 14 Th232ChainGenInCathode # of particles 106 from gen 15 K40GenInAnode # of particles 1418 from gen 16 U238ChainGenInAnode # of particles 1598 from gen 17 Th232ChainGenInAnode # of particles 0 from gen 18 Rn222ChainGenInPDS # of particles 0 from gen 19 K42From42ArGenInUpperMesh # of particles 2 from gen 20 Rn222ChainFromPo218GenInUpperMesh # of particles 23 from gen 21 Rn222ChainFromPb214GenInUpperMesh # of particles 46 from gen 22 Rn222ChainFromBi214GenInUpperMesh # of particles 50 from gen 23 Rn222ChainFromPb210GenInUpperMesh # of particles 35 from gen 24 Rn222ChainFromBi210GenInUpperMesh # of particles 54 from gen 25 Rn220ChainFromPb212GenInUpperMesh # of particles 28057 from gen 26 CavernwallGammasAtLAr # of particles 3497 from gen 27 foamGammasAtLAr # of particles 0 from gen 28 CavernwallNeutronsAtLAr # of particles 0 from gen 29 CryostatNGammasAtLAr # of particles 0 from gen 30 CavernNGammasAtLAr[0m %MSG %MSG-i SolarNuAna: SolarNuAna:solarnuana@BeginModule 27-May-2025 12:45:43 UTC run: 20000047 subRun: 0 event: 10 [33m Number of Signal Particles: 7 Neutrino Interaction: -1 Neutrino Energy: 48.25 MeV Position (-236.01, -463.66, 1710.33) cm Gen. PdgCode Energy EndPosition Mother ------------------------------------------------------------------------ marley 11 0.99 (-236.06, -463.65, 1710.37) 76774 marley -12 0.83 (-2762.78, -5033.92, 1184.24) 76774 marley 1000200400 37214.71 (-236.01, -463.66, 1710.33) 76774 marley 22 0.03 (-236.08, -463.33, 1710.35) 0 marley 22 1.61 (-215.30, -423.44, 1737.74) 0 marley 22 1.09 (-230.73, -458.53, 1726.18) 0 marley 1000190400 37216.58 (-236.01, -463.66, 1710.33) 0 marley 11 44.48 (-234.06, -456.91, 1708.84) 0 Signal Track IDs: 76773; 76774; 76775; 76776; 76777; 76819; 76820; 76821; [0m %MSG %MSG-i SolarNuAna: SolarNuAna:solarnuana@BeginModule 27-May-2025 12:45:43 UTC run: 20000047 subRun: 0 event: 10 [34m[0m %MSG %MSG-i SolarNuAna: SolarNuAna:solarnuana@BeginModule 27-May-2025 12:45:44 UTC run: 20000047 subRun: 0 event: 10 [36m # OpHits (ophit10ppm) in full geometry: 25240 # OpFlashes (solarflash) in full geometry: 0 # Hits (gaushit) in each view: 3172, 3120, 4071, 0 # Cluster from the hits: 1392, 1417, 3454, 0 # Tracks (pmtracktc) in full geometry: 193[0m %MSG %MSG-i SolarNuAna: SolarNuAna:solarnuana@BeginModule 27-May-2025 13:01:55 UTC run: 20000047 subRun: 0 event: 10 [32m*** Matched preselection cluster: - MainTrackID 76773 - Purity 1.00 Hits 11 - #AdjCl 9 ( 0 signal ) - Gen 1 -> marley - Reco Time,Y,Z (6966.82, -458.94, 1709.57) [0m %MSG 27-May-2025 13:01:56 UTC Closed input file "root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/usertests/84/00/fdvd_lowBg_reco2_2025-05-12T_114553Z.root" ==================================================================================================================== TimeTracker printout (sec) Min Avg Max Median RMS nEvts ==================================================================================================================== Full event 972.597 1036.77 1092.38 1033.27 37.088 10 -------------------------------------------------------------------------------------------------------------------- source:RootInput(read) 0.000729288 0.00937813 0.0311389 0.00588564 0.00991594 10 end_path:solarnuana:SolarNuAna 972.596 1036.76 1092.37 1033.25 37.0863 10 ==================================================================================================================== %MSG-i NuRandomService: SolarNuAna:solarnuana@EndJob 27-May-2025 13:01:56 UTC ModuleEndJob Summary of seeds computed by the NuRandomService Random policy: 'perEvent' algorithm version: EventTimestamp_v1 %MSG ==================================================================================================== MemoryTracker summary (base-10 MB units used) Peak virtual memory usage (VmPeak) : 2006.3 MB Peak resident set size usage (VmHWM): 1366.45 MB ==================================================================================================== TrigReport ---------- Event summary ------------- TrigReport Events total = 10 passed = 10 failed = 0 TrigReport ---------- Modules in End-path ---------- TrigReport Run Success Error Name TrigReport 10 10 0 solarnuana TimeReport ---------- Time summary [sec] ------- TimeReport CPU = 10314.619841 Real = 10395.580711 MemReport ---------- Memory summary [base-10 MB] ------ MemReport VmPeak = 2006.3 VmHWM = 1366.45 Art has completed and will exit with status 0. === End last 100 lines of lar log file === Output file: fdhd_ana_2025-05-27T_100827Z.root Output hist file: fdhd_ana_2025-05-27T_100827Z_hist.root lar exit code 0