21 July 2025: This instance at RAL is read-only. Please do not try submitting new workflows for now.
Jobsub ID 228444.119@justin-prod-sched02.dune.hep.ac.uk
| Jobsub ID | 228444.119@justin-prod-sched02.dune.hep.ac.uk | |
| Workflow ID | 7875 | |
| Stage ID | 1 | |
| User name | higuera@fnal.gov | |
| HTCondor Group | group_dune | |
| Requested | Processors | 1 |
| GPU | No | |
| RSS bytes | 4194304000 (4000 MiB) | |
| Wall seconds limit | 80000 (22 hours) | |
| Submitted time | 2025-06-24 11:25:08 | |
| Site | UK_Manchester | |
| Entry | UBoone_T2_UK_Manchester_ce02 | |
| Last heartbeat | 2025-06-24 19:25:02 | |
| From worker node | Hostname | wn1208080.tier2.hep.manchester.ac.uk |
| cpuinfo | AMD EPYC 7702P 64-Core Processor | |
| OS release | Scientific Linux release 7.9 (Nitrogen) | |
| Processors | 1 | |
| RSS bytes | 4194304000 (4000 MiB) | |
| Wall seconds limit | 257400 (71 hours) | |
| GPU | ||
| Inner Apptainer? | True | |
| Job state | stalled | |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk | |
| Started | 2025-06-24 12:03:43 | |
| Input files | fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50571149_545_20231202T190906Z_gen_g4_detsim_hitreco.root | |
| Outputting started | 2025-06-24 12:17:53 | |
| Output files | ||
| Finished | 2025-06-24 20:26:26 | |
| List job events Wrapper job log | ||
Jobscript log (last 10,000 characters)
localProducts_larsoft_v09_89_01_e26_prof/' is not a directory WARNING: '/_reco/localProducts_larsoft_v09_89_01_e26_prof/' is not a directory WARNING: '/_reco/localProducts_larsoft_v09_89_01_e26_prof/' is not a directory WARNING: '/_reco/localProducts_larsoft_v09_89_01_e26_prof/' is not a directory WARNING: '/_reco/localProducts_larsoft_v09_89_01_e26_prof/' is not a directory WARNING: '/_reco/localProducts_larsoft_v09_89_01_e26_prof/' is not a directory WARNING: '/_reco/localProducts_larsoft_v09_89_01_e26_prof/' is not a directory WARNING: '/_reco/localProducts_larsoft_v09_89_01_e26_prof/' is not a directory WARNING: '/_reco/localProducts_larsoft_v09_89_01_e26_prof/' is not a directory WARNING: '/_reco/localProducts_larsoft_v09_89_01_e26_prof/' is not a directory WARNING: '/_reco/localProducts_larsoft_v09_89_01_e26_prof/' is not a directory WARNING: '/_reco/localProducts_larsoft_v09_89_01_e26_prof/' is not a directory WARNING: '/_reco/localProducts_larsoft_v09_89_01_e26_prof/' is not a directory WARNING: '/_reco/localProducts_larsoft_v09_89_01_e26_prof/' is not a directory WARNING: '/_reco/localProducts_larsoft_v09_89_01_e26_prof/' is not a directory WARNING: '/_reco/localProducts_larsoft_v09_89_01_e26_prof/' is not a directory WARNING: '/_reco/localProducts_larsoft_v09_89_01_e26_prof/' is not a directory WARNING: '/_reco/localProducts_larsoft_v09_89_01_e26_prof/' is not a directory WARNING: '/_reco/localProducts_larsoft_v09_89_01_e26_prof/' is not a directory WARNING: '/_reco/localProducts_larsoft_v09_89_01_e26_prof/' is not a directory WARNING: '/_reco/localProducts_larsoft_v09_89_01_e26_prof/' is not a directory WARNING: '/_reco/localProducts_larsoft_v09_89_01_e26_prof/' is not a directory WARNING: '/_reco/localProducts_larsoft_v09_89_01_e26_prof/' is not a directory WARNING: '/_reco/localProducts_larsoft_v09_89_01_e26_prof/' is not a directory Justin processors: 1 /cvmfs/fifeuser2.opensciencegrid.org/sw/dune/cdba6e7604dd8b049ed4282aee20b939a4b77fad/atmo_reco2_lbl_model.fcl Input PFN = root://ccxrootdegee.in2p3.fr:1094/pnfs/in2p3.fr/data/dune/disk/fardet-hd/39/8f/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50571149_545_20231202T190906Z_gen_g4_detsim_hitreco.root /cvmfs/larsoft.opensciencegrid.org/products/xrootd/v5_5_5a/Linux64bit+3.10-2.17-e26-p3915-prof/lib/libXrdPosixPreload.so === Start last 100 lines of lar log file === Running Ophitfinder with InputDigiType = 'recob' Found hits: 1753! Begin processing the 98th record. run: 50571149 subRun: 1 event: 54598 at 24-Jun-2025 13:17:35 BST Boundary wire vector sizes: 1724, 1597, 1338 minwire 0: 1064 minwire 1: 1104 minwire 2: 921 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299 Classifier summary: Output 0: 0.0655896, 0.0338143, 0.900596, Output 1: 0.677096, 0.313784, 0.00733881, 0.00178149, Output 2: 0.0849859, 0.199312, 0.424412, 0.29129, 780 ***Wiener Filter*******ApplyPostfilter*** Running Ophitfinder with InputDigiType = 'recob' Found hits: 3144! Begin processing the 99th record. run: 50571149 subRun: 1 event: 54599 at 24-Jun-2025 13:17:42 BST Boundary wire vector sizes: 48, 48, 37 minwire 0: 1147 minwire 1: 2217 minwire 2: 712 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299 Classifier summary: Output 0: 0.642968, 0.266075, 0.0909564, Output 1: 0.800929, 0.192964, 0.00587248, 0.00023428, Output 2: 0.905063, 0.0934434, 0.00145382, 3.95208e-05, %MSG-e DUNEAna: EnergyReco:energyrecnumumcs@BeginModule 24-Jun-2025 13:17:45 BST run: 50571149 subRun: 1 event: 54599 Failed to find product with label wclsmcnfsp:gauss ... returning empty vector %MSG 436 ***Wiener Filter*******ApplyPostfilter*** Running Ophitfinder with InputDigiType = 'recob' Found hits: 1228! Begin processing the 100th record. run: 50571149 subRun: 1 event: 54600 at 24-Jun-2025 13:17:47 BST Boundary wire vector sizes: 271, 251, 258 minwire 0: 682 minwire 1: 1805 minwire 2: 37 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299 Classifier summary: Output 0: 0.00993687, 0.10819, 0.881874, Output 1: 0.30143, 0.570632, 0.110922, 0.0170166, Output 2: 0.182234, 0.691455, 0.117054, 0.00925699, %MSG-e DUNEAna: EnergyReco:energyrecnumumcs@BeginModule 24-Jun-2025 13:17:49 BST run: 50571149 subRun: 1 event: 54600 Failed to find product with label wclsmcnfsp:gauss ... returning empty vector %MSG 609 ***Wiener Filter*******ApplyPostfilter*** Running Ophitfinder with InputDigiType = 'recob' Found hits: 2072! 24-Jun-2025 13:17:52 BST Closed output file "atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50571149_545_20231202T190906Z_gen_g4_detsim_hitreco_reco2.root" 24-Jun-2025 13:17:52 BST Closed input file "root://ccxrootdegee.in2p3.fr:1094/pnfs/in2p3.fr/data/dune/disk/fardet-hd/39/8f/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50571149_545_20231202T190906Z_gen_g4_detsim_hitreco.root" ======================================================================================================================================== TimeTracker printout (sec) Min Avg Max Median RMS nEvts ======================================================================================================================================== Full event 0.526513 6.06386 91.6527 4.07879 10.1332 100 ---------------------------------------------------------------------------------------------------------------------------------------- source:RootInput(read) 0.0226881 0.0281573 0.0487407 0.024609 0.00820288 100 reco:pandora:StandardPandora 0.142308 3.04267 77.2627 1.30918 8.55426 100 reco:pandoraTrack:LArPandoraTrackCreation 0.00010085 0.00725311 0.11099 0.00108294 0.0180646 100 reco:pandoraShower:LArPandoraModularShowerCreation 0.000157206 0.0172324 0.383595 0.00235851 0.0563007 100 reco:pandoracalo:Calorimetry 8.3738e-05 0.0248144 0.539151 0.000685842 0.0833906 100 reco:pandorapid:Chi2ParticleID 3.168e-05 0.000137224 0.00158358 9.31155e-05 0.000175362 100 reco:cvnmap:CVNMapper 1.6902e-05 0.00745892 0.0225596 0.00939398 0.00586216 100 reco:cvneva:CVNEvaluator 1.5098e-05 0.25728 1.86509 0.348654 0.238549 100 reco:energyrecnumu:EnergyReco 0.0669142 0.221919 4.08029 0.106435 0.521795 100 reco:energyrecnue:EnergyReco 0.000167195 0.0127117 0.0758898 0.000936779 0.0242913 100 reco:energyrecnc:EnergyReco 0.00194845 0.102745 0.29569 0.122539 0.0624435 100 reco:energyrecnumurange:EnergyReco 0.000137709 0.0986932 3.5347 0.00151667 0.451739 100 reco:energyrecnumumcs:EnergyReco 0.000101441 0.0995074 3.58929 0.00145828 0.457653 100 reco:opdec:Deconvolution 0.0716099 0.24987 0.48584 0.252216 0.081026 100 reco:ophitspe:OpHitFinderDeco 0.202699 1.67046 3.36723 1.68161 0.66595 100 reco:opflash:OpFlashFinder 0.000142699 0.00115044 0.00422928 0.00112031 0.000573538 100 reco:rns:RandomNumberSaver 1.593e-05 2.1159e-05 0.000221467 1.852e-05 2.03308e-05 100 reco:anglereconue:NuAngularReco 0.000129644 0.00155493 0.0259365 0.000226988 0.00416218 100 reco:anglereconumu:NuAngularReco 6.5784e-05 0.00145296 0.0255311 0.000138812 0.00413794 100 reco:anglereconuepfps:NuAngularReco 0.000101821 0.00156091 0.0255214 0.000219994 0.00416366 100 reco:anglereconumupfps:NuAngularReco 8.9749e-05 0.00150696 0.0253774 0.000189241 0.00413679 100 [art]:TriggerResults:TriggerResultInserter 8.405e-06 1.03266e-05 5.1117e-05 9.7385e-06 4.21686e-06 100 end_path:out1:RootOutput 2.474e-06 3.34863e-06 1.5349e-05 3.256e-06 1.26107e-06 100 end_path:out1:RootOutput(write) 0.00373383 0.214781 0.535125 0.263549 0.132368 100 ======================================================================================================================================== ==================================================================================================== MemoryTracker summary (base-10 MB units used) Peak virtual memory usage (VmPeak) : 3631.42 MB Peak resident set size usage (VmHWM): 1868.43 MB ==================================================================================================== Art has completed and will exit with status 0. === End last 100 lines of lar log file === lar exit code 0