21 July 2025: This instance at RAL is read-only. Please do not try submitting new workflows for now.
Jobsub ID 425428.47@justin-prod-sched01.dune.hep.ac.uk
| Jobsub ID | 425428.47@justin-prod-sched01.dune.hep.ac.uk | |
| Workflow ID | 8030 | |
| Stage ID | 1 | |
| User name | epennacc@fnal.gov | |
| HTCondor Group | group_dune.prod_mcsim | |
| Requested | Processors | 1 |
| GPU | No | |
| RSS bytes | 4194304000 (4000 MiB) | |
| Wall seconds limit | 80000 (22 hours) | |
| Submitted time | 2025-07-01 06:24:32 | |
| Site | US_Wisconsin | |
| Entry | HCCHTPC_US_Wisconsin_osg01_rhel7 | |
| Last heartbeat | 2025-07-01 08:35:19 | |
| From worker node | Hostname | e2619 |
| cpuinfo | AMD EPYC 7763 64-Core Processor | |
| OS release | Scientific Linux release 7.9 (Nitrogen) | |
| Processors | 1 | |
| RSS bytes | 4194304000 (4000 MiB) | |
| Wall seconds limit | 82800 (23 hours) | |
| GPU | ||
| Inner Apptainer? | True | |
| Job state | jobscript_error | |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk | |
| Started | 2025-07-01 08:34:30 | |
| Input files | fardet-vd:prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250128T140343Z_gen_002658_supernova_g4stage1_g4stage2_detsim_reco.root | |
| Jobscript | Exit code | 20 |
| Real time | 0m (0s) | |
| CPU time | 0m (0s = 0%) | |
| Max RSS bytes | 0 (0 MiB) | |
| Outputting started | ||
| Output files | ||
| Finished | 2025-07-01 08:35:19 | |
| Saved logs | justin-logs:425428.47-justin-prod-sched01.dune.hep.ac.uk.logs.tgz | |
| List job events Wrapper job log | ||
Jobscript log (last 10,000 characters)
Group 73 (apa073) has 864 channels from 3/3 readout planes. GeoApaChannelGroupService::ctor: Group 74 (apa074) has 864 channels from 3/3 readout planes. GeoApaChannelGroupService::ctor: Group 75 (apa075) has 864 channels from 3/3 readout planes. GeoApaChannelGroupService::ctor: Group 76 (apa076) has 864 channels from 3/3 readout planes. GeoApaChannelGroupService::ctor: Group 77 (apa077) has 864 channels from 3/3 readout planes. GeoApaChannelGroupService::ctor: Group 78 (apa078) has 864 channels from 3/3 readout planes. GeoApaChannelGroupService::ctor: Group 79 (apa079) has 864 channels from 3/3 readout planes. GeoApaChannelGroupService::ctor: Group 80 (apa080) has 864 channels from 3/3 readout planes. GeoApaChannelGroupService::ctor: Group 81 (apa081) has 864 channels from 3/3 readout planes. GeoApaChannelGroupService::ctor: Group 82 (apa082) has 864 channels from 3/3 readout planes. GeoApaChannelGroupService::ctor: Group 83 (apa083) has 864 channels from 3/3 readout planes. GeoApaChannelGroupService::ctor: Group 84 (apa084) has 864 channels from 3/3 readout planes. GeoApaChannelGroupService::ctor: Group 85 (apa085) has 864 channels from 3/3 readout planes. GeoApaChannelGroupService::ctor: Group 86 (apa086) has 864 channels from 3/3 readout planes. GeoApaChannelGroupService::ctor: Group 87 (apa087) has 864 channels from 3/3 readout planes. GeoApaChannelGroupService::ctor: Group 88 (apa088) has 864 channels from 3/3 readout planes. GeoApaChannelGroupService::ctor: Group 89 (apa089) has 864 channels from 3/3 readout planes. GeoApaChannelGroupService::ctor: Group 90 (apa090) has 864 channels from 3/3 readout planes. GeoApaChannelGroupService::ctor: Group 91 (apa091) has 864 channels from 3/3 readout planes. GeoApaChannelGroupService::ctor: Group 92 (apa092) has 864 channels from 3/3 readout planes. GeoApaChannelGroupService::ctor: Group 93 (apa093) has 864 channels from 3/3 readout planes. GeoApaChannelGroupService::ctor: Group 94 (apa094) has 864 channels from 3/3 readout planes. GeoApaChannelGroupService::ctor: Group 95 (apa095) has 864 channels from 3/3 readout planes. GeoApaChannelGroupService::ctor: Group 96 (apa096) has 864 channels from 3/3 readout planes. GeoApaChannelGroupService::ctor: Group 97 (apa097) has 864 channels from 3/3 readout planes. GeoApaChannelGroupService::ctor: Group 98 (apa098) has 864 channels from 3/3 readout planes. GeoApaChannelGroupService::ctor: Group 99 (apa099) has 864 channels from 3/3 readout planes. GeoApaChannelGroupService::ctor: Group 100 (apa100) has 864 channels from 3/3 readout planes. GeoApaChannelGroupService::ctor: Group 101 (apa101) has 864 channels from 3/3 readout planes. GeoApaChannelGroupService::ctor: Group 102 (apa102) has 864 channels from 3/3 readout planes. GeoApaChannelGroupService::ctor: Group 103 (apa103) has 864 channels from 3/3 readout planes. GeoApaChannelGroupService::ctor: Group 104 (apa104) has 864 channels from 3/3 readout planes. GeoApaChannelGroupService::ctor: Group 105 (apa105) has 864 channels from 3/3 readout planes. GeoApaChannelGroupService::ctor: Group 106 (apa106) has 864 channels from 3/3 readout planes. GeoApaChannelGroupService::ctor: Group 107 (apa107) has 864 channels from 3/3 readout planes. GeoApaChannelGroupService::ctor: Group 108 (apa108) has 864 channels from 3/3 readout planes. GeoApaChannelGroupService::ctor: Group 109 (apa109) has 864 channels from 3/3 readout planes. GeoApaChannelGroupService::ctor: Group 110 (apa110) has 864 channels from 3/3 readout planes. GeoApaChannelGroupService::ctor: Group 111 (apa111) has 864 channels from 3/3 readout planes. DuneToolManager::fclFilename: Taking fcl name from command line: lar -c reco2_supernova_dunevd10kt_1x8x14_3view_30deg_prod2024.fcl root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-vd/46/d8/prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250128T140343Z_gen_002658_supernova_g4stage1_g4stage2_detsim_reco.root -o prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250128T140343Z_gen_002658_supernova_g4stage1_g4stage2_detsim_reco_20250701T083433Z_reco2.root -n -1 AcdDigitReader::ctor: LogLevel: 1 DuneToolManager::getPrivate: ERROR: Tool name is blank StandardRawDigitExtractService::ctor: Retrieved digit read tool digitReader StandardRawDigitExtractService::ctor: StandardRawDigitExtractService: StandardRawDigitExtractService::ctor: LogLevel: 1 StandardRawDigitExtractService::ctor: DigitReadTool: digitReader StandardRawDigitExtractService::ctor: PedestalOption: 1 StandardRawDigitExtractService::ctor: FlagStuckOff: 0 StandardRawDigitExtractService::ctor: FlagStuckOn: 0 StandardRawDigitPrepService::ctor: Fetching extract service. StandardRawDigitPrepService::ctor: Fetching channel status provider. StandardRawDigitPrepService::ctor: Channel status provider: @0xc51cb90 StandardRawDigitPrepService::ctor: Extract service: @0x5dfb4f0 StandardRawDigitPrepService::ctor: Fetching deconvolution service. StandardRawDigitPrepService::ctor: Deconvolution service: @0xc517f40 StandardRawDigitPrepService::ctor: Fetching ROI building service. StandardRawDigitPrepService::ctor: ROI building service: @0xc51bd70 StandardRawDigitPrepService::ctor: Fetching wire building service. StandardRawDigitPrepService::ctor: Wire building service: @0x42bffc0 StandardRawDigitPrepService::ctor: StandardRawDigitPrepService: StandardRawDigitPrepService::ctor: LogLevel: 1 StandardRawDigitPrepService::ctor: SkipBad: 1 StandardRawDigitPrepService::ctor: SkipNoisy: 0 StandardRawDigitPrepService::ctor: ChannelStatusOnline: 0 StandardRawDigitPrepService::ctor: DoMitigation: 0 StandardRawDigitPrepService::ctor: DoEarlySignalFinding: 0 StandardRawDigitPrepService::ctor: DoNoiseRemoval: 0 StandardRawDigitPrepService::ctor: DoDeconvolution: 1 StandardRawDigitPrepService::ctor: DoPedestalAdjustment: 0 StandardRawDigitPrepService::ctor: DoROI: 1 StandardRawDigitPrepService::ctor: DoWires: 1 StandardRawDigitPrepService::ctor: DoDump: 0 StandardRawDigitPrepService::ctor: DoIntermediateStates: 0 StandardRawDigitPrepService::ctor: No display tools. Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_U (Potential memory leak). Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_V (Potential memory leak). Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_Y (Potential memory leak). Reading model from /cvmfs/dune.opensciencegrid.org/products/dune/dune_pardata/v01_84_00/CnnModels/cnn_ndkemtrk_pitch_5_wire_44_drift_48_down_6_mean_notes_AtmAndNdk.nnet Layers 12 Layer 0 Convolution2D LayerConv2D 48x1x5x5 border_mode valid Layer 1 Activation Activation type relu Layer 2 Dropout Layer 3 Flatten Layer 4 Dense weights 84480 bias 128 Layer 5 Activation Activation type tanh Layer 6 Dropout Layer 7 Dense weights 128 bias 32 Layer 8 Activation Activation type tanh Layer 9 Dropout Layer 10 Dense weights 32 bias 4 Layer 11 Activation Activation type sigmoid ************************************************************************************************************************************* Unique Ptrs that are added to the event ************************************************************************************************************************************* * Data Product Name: InitialTrack * Instance Name: * Type: std::vector<recob::Track, std::allocator<recob::Track> >* * * Data Product Name: ShowerPCA * Instance Name: * Type: std::vector<recob::PCAxis, std::allocator<recob::PCAxis> >* * * Data Product Name: shower * Instance Name: * Type: std::vector<recob::Shower, std::allocator<recob::Shower> >* * * Association Name: PFParticlePCAxisAssn * Instance Name: * Type: art::Assns<recob::PFParticle, recob::PCAxis, void>* * * Association Name: ShowerPCAxisAssn * Instance Name: * Type: art::Assns<recob::Shower, recob::PCAxis, void>* * * Association Name: ShowerTrackAssn * Instance Name: * Type: art::Assns<recob::Shower, recob::Track, void>* * * Association Name: ShowerTrackHitAssn * Instance Name: * Type: art::Assns<recob::Track, recob::Hit, void>* * * Association Name: clusterAssociationsbase * Instance Name: * Type: art::Assns<recob::Shower, recob::Cluster, void>* * * Association Name: hitAssociationsbase * Instance Name: * Type: art::Assns<recob::Shower, recob::Hit, void>* * * Association Name: pfShowerAssociationsbase * Instance Name: * Type: art::Assns<recob::Shower, recob::PFParticle, void>* * * Association Name: spShowerAssociationsbase * Instance Name: * Type: art::Assns<recob::Shower, recob::SpacePoint, void>* * ************************************************************************************************************************************* 01-Jul-2025 03:35:06 CDT Initiating request to open input file "root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-vd/46/d8/prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250128T140343Z_gen_002658_supernova_g4stage1_g4stage2_detsim_reco.root" MultiPandoraApiImpl::DeletePandoraInstances - unable to find daughter instances associated with primary 0 %MSG-s ArtException: TriggerResultInserter:TriggerResults@Construction 01-Jul-2025 03:35:07 CDT ModuleConstruction cet::exception caught in art ---- FileOpenError BEGIN RootInputFileSequence::initFile(): Input file root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-vd/46/d8/prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250128T140343Z_gen_002658_supernova_g4stage1_g4stage2_detsim_reco.root was not found or could not be opened. ---- FileOpenError END %MSG Art has completed and will exit with status 20.