Jobsub ID 70395.128@justin-prod-sched02.dune.hep.ac.uk
| Jobsub ID | 70395.128@justin-prod-sched02.dune.hep.ac.uk |
| Workflow ID | 3207 |
| Stage ID | 1 |
| User name | imawby@fnal.gov |
| HTCondor Group | group_dune |
| Requested | Processors | 1 |
| RSS bytes | 4194304000 (4000 MiB) |
| Wall seconds limit | 43200 (12 hours) |
| Submitted time | 2024-09-12 15:58:31 |
| Site | US_FNAL-FermiGrid |
| Entry | FNAL_GPGrid_ce03_mcore_op_duneonly |
| Last heartbeat | 2024-09-12 20:41:01 |
| From worker node | Hostname | dunegli-3943196-0-fnpc23008.fnal.gov |
| cpuinfo | AMD EPYC 7543 32-Core Processor |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4194304000 (4000 MiB) |
| Wall seconds limit | 172800 (48 hours) |
| Inner Apptainer? | True |
| Job state | jobscript_error |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk |
| Started | 2024-09-12 16:42:07 |
| Input files | fardet-hd:nutau_dune10kt_1x2x6_1099_906_20230826T025903Z_gen_g4_detsim_hitreco__20240219T201943Z_reco2.root
|
| Jobscript | Exit code | 1 |
| Real time | 0m (0s) |
| CPU time | 0m (0s = 0%) |
| Outputting started | |
| Output files | |
| Finished | 2024-09-12 20:41:01 |
| Saved logs | justin-logs:70395.128-justin-prod-sched02.dune.hep.ac.uk.logs.tgz |
| List job events Wrapper job log |
Jobscript log (last 10,000 characters)
gov/data/dune/RSE/fardet-hd/cf/d4/nutau_dune10kt_1x2x6_1099_906_20230826T025903Z_gen_g4_detsim_hitreco__20240219T201943Z_reco2.root
lar exit code 20
=== Start last 100 lines of lar log file ===
Info in <TGeoManager::CheckGeometry>: ...Nothing to fix
Info in <TGeoManager::CloseGeometry>: Counting nodes...
Info in <TGeoManager::Voxelize>: Voxelizing...
Info in <TGeoManager::CloseGeometry>: Building cache...
Info in <TGeoManager::CountLevels>: max level = 5, max placements = 1149
Info in <TGeoManager::CloseGeometry>: 67545 nodes/ 3831 volume UID's in Geometry imported from GDML
Info in <TGeoManager::CloseGeometry>: ----------------modeler ready----------------
GeoApaChannelGroupService::ctor: Group 0 (apa00) has 2560 channels from 4/4 readout planes.
GeoApaChannelGroupService::ctor: Group 1 (apa01) has 2560 channels from 4/4 readout planes.
GeoApaChannelGroupService::ctor: Group 2 (apa02) has 2560 channels from 4/4 readout planes.
GeoApaChannelGroupService::ctor: Group 3 (apa03) has 2560 channels from 4/4 readout planes.
GeoApaChannelGroupService::ctor: Group 4 (apa04) has 2560 channels from 4/4 readout planes.
GeoApaChannelGroupService::ctor: Group 5 (apa05) has 2560 channels from 4/4 readout planes.
GeoApaChannelGroupService::ctor: Group 6 (apa06) has 2560 channels from 4/4 readout planes.
GeoApaChannelGroupService::ctor: Group 7 (apa07) has 2560 channels from 4/4 readout planes.
GeoApaChannelGroupService::ctor: Group 8 (apa08) has 2560 channels from 4/4 readout planes.
GeoApaChannelGroupService::ctor: Group 9 (apa09) has 2560 channels from 4/4 readout planes.
GeoApaChannelGroupService::ctor: Group 10 (apa10) has 2560 channels from 4/4 readout planes.
GeoApaChannelGroupService::ctor: Group 11 (apa11) has 2560 channels from 4/4 readout planes.
Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_U (Potential memory leak).
Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_V (Potential memory leak).
Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_Y (Potential memory leak).
: Booking "BDTG" of type "BDT" from /cvmfs/fifeuser1.opensciencegrid.org/sw/dune/8bf29f4f0035d8327219b7de405f437121e006bc/MCC11_FHC_Pandizzle_NewProduction.weights.xml.
: Reading weight file: /cvmfs/fifeuser1.opensciencegrid.org/sw/dune/8bf29f4f0035d8327219b7de405f437121e006bc/MCC11_FHC_Pandizzle_NewProduction.weights.xml
<HEADER> DataSetInfo : [Default] : Added class "Signal"
<HEADER> DataSetInfo : [Default] : Added class "Background"
: Booked classifier "BDTG" of type: "BDT"
: Booking "BDTG" of type "BDT" from /cvmfs/fifeuser1.opensciencegrid.org/sw/dune/8bf29f4f0035d8327219b7de405f437121e006bc/MCC11_FHC_Pandrizzle_Backup_NewProduction.weights.xml.
: Reading weight file: /cvmfs/fifeuser1.opensciencegrid.org/sw/dune/8bf29f4f0035d8327219b7de405f437121e006bc/MCC11_FHC_Pandrizzle_Backup_NewProduction.weights.xml
<HEADER> DataSetInfo : [Default] : Added class "Signal"
<HEADER> DataSetInfo : [Default] : Added class "Background"
: Booked classifier "BDTG" of type: "BDT"
: Booking "BDTG" of type "BDT" from /cvmfs/fifeuser1.opensciencegrid.org/sw/dune/8bf29f4f0035d8327219b7de405f437121e006bc/MCC11_FHC_Pandrizzle_Enhanced_NewProduction.weights.xml.
: Reading weight file: /cvmfs/fifeuser1.opensciencegrid.org/sw/dune/8bf29f4f0035d8327219b7de405f437121e006bc/MCC11_FHC_Pandrizzle_Enhanced_NewProduction.weights.xml
<HEADER> DataSetInfo : [Default] : Added class "Signal"
<HEADER> DataSetInfo : [Default] : Added class "Background"
: Booked classifier "BDTG" of type: "BDT"
HELLLLO
Directory path: /cvmfs/fifeuser1.opensciencegrid.org/sw/dune/8bf29f4f0035d8327219b7de405f437121e006bc/my_model_calo_VGG_BN_VARS/
GOOOODBYE
*************************************************************************************************************************************
Unique Ptrs that are added to the event
*************************************************************************************************************************************
* Data Product Name: InitialTrack * Instance Name: * Type: std::vector<recob::Track, std::allocator<recob::Track> >* *
* Data Product Name: ShowerPCA * Instance Name: * Type: std::vector<recob::PCAxis, std::allocator<recob::PCAxis> >* *
* Data Product Name: shower * Instance Name: * Type: std::vector<recob::Shower, std::allocator<recob::Shower> >* *
* Association Name: PFParticlePCAxisAssn * Instance Name: * Type: art::Assns<recob::PFParticle, recob::PCAxis, void>* *
* Association Name: ShowerPCAxisAssn * Instance Name: * Type: art::Assns<recob::Shower, recob::PCAxis, void>* *
* Association Name: ShowerTrackAssn * Instance Name: * Type: art::Assns<recob::Shower, recob::Track, void>* *
* Association Name: ShowerTrackHitAssn * Instance Name: * Type: art::Assns<recob::Track, recob::Hit, void>* *
* Association Name: clusterAssociationsbase * Instance Name: * Type: art::Assns<recob::Shower, recob::Cluster, void>* *
* Association Name: hitAssociationsbase * Instance Name: * Type: art::Assns<recob::Shower, recob::Hit, void>* *
* Association Name: pfShowerAssociationsbase * Instance Name: * Type: art::Assns<recob::Shower, recob::PFParticle, void>* *
* Association Name: spShowerAssociationsbase * Instance Name: * Type: art::Assns<recob::Shower, recob::SpacePoint, void>* *
*************************************************************************************************************************************
: Booking "electron" of type "BDT" from /cvmfs/dune.opensciencegrid.org/products/dune/dune_pardata/v01_84_00/mvapid_weights/electron_all_BDT.weights.xml.
: Reading weight file: /cvmfs/dune.opensciencegrid.org/products/dune/dune_pardata/v01_84_00/mvapid_weights/electron_all_BDT.weights.xml
<HEADER> DataSetInfo : [Default] : Added class "Signal"
<HEADER> DataSetInfo : [Default] : Added class "Background"
: Booked classifier "BDT" of type: "BDT"
: Booking "muon" of type "BDT" from /cvmfs/dune.opensciencegrid.org/products/dune/dune_pardata/v01_84_00/mvapid_weights/muon_all_BDT.weights.xml.
: Reading weight file: /cvmfs/dune.opensciencegrid.org/products/dune/dune_pardata/v01_84_00/mvapid_weights/muon_all_BDT.weights.xml
: Booked classifier "BDT" of type: "BDT"
: Booking "photon" of type "BDT" from /cvmfs/dune.opensciencegrid.org/products/dune/dune_pardata/v01_84_00/mvapid_weights/photon_all_BDT.weights.xml.
: Reading weight file: /cvmfs/dune.opensciencegrid.org/products/dune/dune_pardata/v01_84_00/mvapid_weights/photon_all_BDT.weights.xml
: Booked classifier "BDT" of type: "BDT"
: Booking "pich" of type "BDT" from /cvmfs/dune.opensciencegrid.org/products/dune/dune_pardata/v01_84_00/mvapid_weights/pich_all_BDT.weights.xml.
: Reading weight file: /cvmfs/dune.opensciencegrid.org/products/dune/dune_pardata/v01_84_00/mvapid_weights/pich_all_BDT.weights.xml
: Booked classifier "BDT" of type: "BDT"
: Booking "proton" of type "BDT" from /cvmfs/dune.opensciencegrid.org/products/dune/dune_pardata/v01_84_00/mvapid_weights/proton_all_BDT.weights.xml.
: Reading weight file: /cvmfs/dune.opensciencegrid.org/products/dune/dune_pardata/v01_84_00/mvapid_weights/proton_all_BDT.weights.xml
: Booked classifier "BDT" of type: "BDT"
12-Sep-2024 16:42:30 UTC Initiating request to open input file "root://dcdndoor.sdcc.bnl.gov:1094//pnfs/sdcc.bnl.gov/data/dune/RSE/fardet-hd/cf/d4/nutau_dune10kt_1x2x6_1099_906_20230826T025903Z_gen_g4_detsim_hitreco__20240219T201943Z_reco2.root"
12-Sep-2024 16:42:30 UTC Initiating request to open input file "root://dcdndoor.sdcc.bnl.gov:1094//pnfs/sdcc.bnl.gov/data/dune/RSE/fardet-hd/cf/d4/nutau_dune10kt_1x2x6_1099_906_20230826T025903Z_gen_g4_detsim_hitreco__20240219T201943Z_reco2.root"
MultiPandoraApiImpl::DeletePandoraInstances - unable to find daughter instances associated with primary 0
%MSG-s ArtException: TriggerResultInserter:TriggerResults@Construction 12-Sep-2024 20:40:53 UTC ModuleConstruction
cet::exception caught in art
---- FileOpenError BEGIN
---- FatalRootError BEGIN
Fatal Root Error: TNetXNGFile::Open
[FATAL] Auth failed: No protocols left to try
ROOT severity: 3000
---- FatalRootError END
RootInputFileSequence::initFile(): Input file root://dcdndoor.sdcc.bnl.gov:1094//pnfs/sdcc.bnl.gov/data/dune/RSE/fardet-hd/cf/d4/nutau_dune10kt_1x2x6_1099_906_20230826T025903Z_gen_g4_detsim_hitreco__20240219T201943Z_reco2.root was not found or could not be opened.
---- FileOpenError END
%MSG
%MSG-s ArtException: TriggerResultInserter:TriggerResults@Construction 12-Sep-2024 20:40:53 UTC ModuleConstruction
cet::exception caught in art
---- FileOpenError BEGIN
---- FatalRootError BEGIN
Fatal Root Error: TNetXNGFile::Open
[FATAL] Auth failed: No protocols left to try
ROOT severity: 3000
---- FatalRootError END
RootInputFileSequence::initFile(): Input file root://dcdndoor.sdcc.bnl.gov:1094//pnfs/sdcc.bnl.gov/data/dune/RSE/fardet-hd/cf/d4/nutau_dune10kt_1x2x6_1099_906_20230826T025903Z_gen_g4_detsim_hitreco__20240219T201943Z_reco2.root was not found or could not be opened.
---- FileOpenError END
%MSG
Art has completed and will exit with status 20.
=== End last 100 lines of lar log file === 