Jobsub ID 104649.128@justin-prod-sched02.dune.hep.ac.uk
| Jobsub ID | 104649.128@justin-prod-sched02.dune.hep.ac.uk |
| Workflow ID | 4147 |
| Stage ID | 1 |
| User name | osbiston@fnal.gov |
| HTCondor Group | group_dune |
| Requested | Processors | 1 |
| RSS bytes | 2097152000 (2000 MiB) |
| Wall seconds limit | 80000 (22 hours) |
| Submitted time | 2024-11-19 21:43:53 |
| Site | US_FNAL-T1 |
| Entry | CMSHTPC_T1_US_FNAL_condce_opp1_whole |
| Last heartbeat | 2024-11-19 22:21:04 |
| From worker node | Hostname | dunegli-36912-0-cmswn2289.fnal.gov |
| cpuinfo | Intel(R) Xeon(R) CPU E5-2670 v3 @ 2.30GHz |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 2097152000 (2000 MiB) |
| Wall seconds limit | 171000 (47 hours) |
| Inner Apptainer? | True |
| Job state | outputting_failed |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk |
| Started | 2024-11-19 21:52:40 |
| Input files | fardet-hd:prodmarley_nue_cc_flat_dune10kt_1x2x6_20241023T132916Z_gen_000327_supernova_g4_detsim_reco.root
|
| Jobscript | Exit code | 0 |
| Real time | 27m (1666s) |
| CPU time | 2m (129s = 7%) |
| Outputting started | 2024-11-19 22:20:28 |
| Output files | |
| Finished | 2024-11-19 22:21:04 |
| List job events Wrapper job log |
Jobscript log (last 10,000 characters)
Input PFN = root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-hd/16/68/prodmarley_nue_cc_flat_dune10kt_1x2x6_20241023T132916Z_gen_000327_supernova_g4_detsim_reco.root
Setting up larsoft UPS area... /cvmfs/larsoft.opensciencegrid.org
Setting up DUNE UPS area... /cvmfs/dune.opensciencegrid.org/products/dune/
/cvmfs/larsoft.opensciencegrid.org/products/xrootd/v5_5_5a/Linux64bit+3.10-2.17-e26-p3915-prof/lib/libXrdPosixPreload.so
=== Start last 100 lines of lar log file ===
StandardRawDigitPrepService::ctor: ROI building service: @0xd9c3050
StandardRawDigitPrepService::ctor: Fetching wire building service.
StandardRawDigitPrepService::ctor: Wire building service: @0x4f70250
StandardRawDigitPrepService::ctor: StandardRawDigitPrepService:
StandardRawDigitPrepService::ctor: LogLevel: 1
StandardRawDigitPrepService::ctor: SkipBad: 1
StandardRawDigitPrepService::ctor: SkipNoisy: 0
StandardRawDigitPrepService::ctor: ChannelStatusOnline: 0
StandardRawDigitPrepService::ctor: DoMitigation: 0
StandardRawDigitPrepService::ctor: DoEarlySignalFinding: 0
StandardRawDigitPrepService::ctor: DoNoiseRemoval: 0
StandardRawDigitPrepService::ctor: DoDeconvolution: 1
StandardRawDigitPrepService::ctor: DoPedestalAdjustment: 0
StandardRawDigitPrepService::ctor: DoROI: 1
StandardRawDigitPrepService::ctor: DoWires: 1
StandardRawDigitPrepService::ctor: DoDump: 0
StandardRawDigitPrepService::ctor: DoIntermediateStates: 0
StandardRawDigitPrepService::ctor: No display tools.
Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_U (Potential memory leak).
Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_V (Potential memory leak).
Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_Y (Potential memory leak).
*************************************************************************************************************************************
Unique Ptrs that are added to the event
*************************************************************************************************************************************
* Data Product Name: InitialTrack * Instance Name: * Type: std::vector<recob::Track, std::allocator<recob::Track> >* *
* Data Product Name: ShowerPCA * Instance Name: * Type: std::vector<recob::PCAxis, std::allocator<recob::PCAxis> >* *
* Data Product Name: shower * Instance Name: * Type: std::vector<recob::Shower, std::allocator<recob::Shower> >* *
* Association Name: PFParticlePCAxisAssn * Instance Name: * Type: art::Assns<recob::PFParticle, recob::PCAxis, void>* *
* Association Name: ShowerPCAxisAssn * Instance Name: * Type: art::Assns<recob::Shower, recob::PCAxis, void>* *
* Association Name: ShowerTrackAssn * Instance Name: * Type: art::Assns<recob::Shower, recob::Track, void>* *
* Association Name: ShowerTrackHitAssn * Instance Name: * Type: art::Assns<recob::Track, recob::Hit, void>* *
* Association Name: clusterAssociationsbase * Instance Name: * Type: art::Assns<recob::Shower, recob::Cluster, void>* *
* Association Name: hitAssociationsbase * Instance Name: * Type: art::Assns<recob::Shower, recob::Hit, void>* *
* Association Name: pfShowerAssociationsbase * Instance Name: * Type: art::Assns<recob::Shower, recob::PFParticle, void>* *
* Association Name: spShowerAssociationsbase * Instance Name: * Type: art::Assns<recob::Shower, recob::SpacePoint, void>* *
*************************************************************************************************************************************
19-Nov-2024 21:55:34 UTC Initiating request to open input file "root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-hd/16/68/prodmarley_nue_cc_flat_dune10kt_1x2x6_20241023T132916Z_gen_000327_supernova_g4_detsim_reco.root"
19-Nov-2024 21:55:39 UTC Opened input file "root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-hd/16/68/prodmarley_nue_cc_flat_dune10kt_1x2x6_20241023T132916Z_gen_000327_supernova_g4_detsim_reco.root"
19-Nov-2024 21:55:40 UTC Opened output file with pattern "prodmarley_nue_cc_flat_dune10kt_1x2x6_20241023T132916Z_gen_000327_supernova_g4_detsim_reco_matt_2024-11-19T_215303Z.root"
19-Nov-2024 21:55:55 UTC Closed input file "root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-hd/16/68/prodmarley_nue_cc_flat_dune10kt_1x2x6_20241023T132916Z_gen_000327_supernova_g4_detsim_reco.root"
====================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
====================================================================================================================
[ No processed events ]
====================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 9121.6 MB
Peak resident set size usage (VmHWM): 1118.71 MB
====================================================================================================
%MSG-s ArtException: PostEndJob 19-Nov-2024 21:56:10 UTC ModuleEndJob
---- EventProcessorFailure BEGIN
EventProcessor: an exception occurred during current event processing
---- Geometry BEGIN
Geometry used for run run: 3786 is incompatible with the one configured in the job!
=== job configuration ==================================================
Geometry information version: 2
Detector name: 'dunevd10kt_3view_30deg_v5_refactored_1x8x14ref'
Full configuration:
--------------------------------------------------------------------------------
DisableWiresInG4: true
GDML: "dunevd10kt_3view_30deg_v5_refactored_1x8x14ref.gdml"
Name: "dunevd10kt_3view_30deg_v5_refactored_1x8x14ref"
ROOT: "dunevd10kt_3view_30deg_v5_refactored_1x8x14ref.gdml"
SortingParameters: {
ChannelsPerOpDet: 1
}
SurfaceY: 147828
service_type: "Geometry"
--------------------------------------------------------------------------------
=== run configuration ==================================================
Geometry information version: 2
Detector name: 'dune10kt_v6_1x2x6'
Full configuration:
--------------------------------------------------------------------------------
DisableWiresInG4: true
GDML: "dune10kt_v6_refactored_1x2x6.gdml"
Name: "dune10kt_v6_1x2x6"
ROOT: "dune10kt_v6_refactored_1x2x6.gdml"
SortingParameters: {
ChannelsPerOpDet: 1
}
SurfaceY: 147828
service_type: "Geometry"
--------------------------------------------------------------------------------
========================================================================
---- Geometry END
---- EventProcessorFailure END
---- FatalRootError BEGIN
Fatal Root Error: TTree::SetEntries
Tree branches have different numbers of entries, eg EventAuxiliary has 0 entries while sim::OpDetDivRecs_opdigi__detsim. has 100 entries.
ROOT severity: 2000
---- FatalRootError END
%MSG
Art has completed and will exit with status 1.
=== End last 100 lines of lar log file ===
lar exit code 0 