Jobsub ID 296526.0@justin-prod-sched01.dune.hep.ac.uk
| Jobsub ID | 296526.0@justin-prod-sched01.dune.hep.ac.uk |
| Workflow ID | 4128 |
| Stage ID | 1 |
| User name | imawby@fnal.gov |
| HTCondor Group | group_dune |
| Requested | Processors | 1 |
| RSS bytes | 4194304000 (4000 MiB) |
| Wall seconds limit | 43200 (12 hours) |
| Submitted time | 2024-11-19 09:01:19 |
| Site | US_PuertoRico |
| Entry | CMSHTPC_T3_US_PuertoRico_UPRM |
| Last heartbeat | 2024-11-19 11:15:11 |
| From worker node | Hostname | compute-1-4.local |
| cpuinfo | Intel(R) Xeon(R) CPU W3550 @ 3.07GHz |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4194304000 (4000 MiB) |
| Wall seconds limit | 171000 (47 hours) |
| Inner Apptainer? | True |
| Job state | outputting_failed |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk |
| Started | 2024-11-19 09:02:44 |
| Input files | fardet-hd:nue_dune10kt_1x2x6_1435_504_20230828T103555Z_gen_g4_detsim_hitreco__20240221T034825Z_reco2.root
|
| Jobscript | Exit code | 0 |
| Real time | 2h (7931s) |
| CPU time | 2h (8211s = 103%) |
| Outputting started | 2024-11-19 11:14:57 |
| Output files | |
| Finished | 2024-11-19 11:15:11 |
| List job events Wrapper job log |
Jobscript log (last 10,000 characters)
Justin processors: 1
Setting up larsoft UPS area... /cvmfs/larsoft.opensciencegrid.org
Setting up DUNE UPS area... /cvmfs/dune.opensciencegrid.org/products/dune/
MRB_PROJECT=larsoft
MRB_PROJECT_VERSION=v09_91_02
MRB_QUALS=e26:prof
MRB_TOP=/cvmfs/fifeuser3.opensciencegrid.org/sw/dune/baeee8261a966c7bf48ffdcd205bcc1e0b341bf3
MRB_SOURCE=/cvmfs/fifeuser3.opensciencegrid.org/sw/dune/baeee8261a966c7bf48ffdcd205bcc1e0b341bf3/srcs
MRB_BUILDDIR=/cvmfs/fifeuser3.opensciencegrid.org/sw/dune/baeee8261a966c7bf48ffdcd205bcc1e0b341bf3/build_slf7.x86_64
MRB_INSTALL=/cvmfs/fifeuser3.opensciencegrid.org/sw/dune/baeee8261a966c7bf48ffdcd205bcc1e0b341bf3/localProducts_larsoft_v09_91_02_e26_prof
PRODUCTS=/cvmfs/fifeuser3.opensciencegrid.org/sw/dune/baeee8261a966c7bf48ffdcd205bcc1e0b341bf3/localProducts_larsoft_v09_91_02_e26_prof:/cvmfs/dune.opensciencegrid.org/products/dune:/cvmfs/larsoft.opensciencegrid.org/products:/cvmfs/larsoft.opensciencegrid.org/packages:/cvmfs/fermilab.opensciencegrid.org/products/common/db/
CETPKG_INSTALL=/cvmfs/fifeuser3.opensciencegrid.org/sw/dune/baeee8261a966c7bf48ffdcd205bcc1e0b341bf3/localProducts_larsoft_v09_91_02_e26_prof
local product directory is /cvmfs/fifeuser3.opensciencegrid.org/sw/dune/baeee8261a966c7bf48ffdcd205bcc1e0b341bf3/localProducts_larsoft_v09_91_02_e26_prof
----------- this block should be empty ------------------
---------------------------------------------------------
Input PFN = root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-hd/8b/0e/nue_dune10kt_1x2x6_1435_504_20230828T103555Z_gen_g4_detsim_hitreco__20240221T034825Z_reco2.root
lar exit code 0
=== Start last 100 lines of lar log file ===
%MSG
%MSG-w LArPandora: StandardPandora:pandoraRedo@BeginModule 19-Nov-2024 07:14:26 AST run: 1435 subRun: 1 event: 50498
CreatePandoraMCParticles - found an unknown process
%MSG
%MSG-w LArPandora: StandardPandora:pandoraRedo@BeginModule 19-Nov-2024 07:14:26 AST run: 1435 subRun: 1 event: 50498
CreatePandoraMCParticles - found an unknown process
%MSG
%MSG-w LArPandora: StandardPandora:pandoraRedo@BeginModule 19-Nov-2024 07:14:26 AST run: 1435 subRun: 1 event: 50498
CreatePandoraMCParticles - found an unknown process
%MSG
%MSG-w LArPandora: StandardPandora:pandoraRedo@BeginModule 19-Nov-2024 07:14:26 AST run: 1435 subRun: 1 event: 50498
CreatePandoraMCParticles - found an unknown process
%MSG
%MSG-w LArPandora: StandardPandora:pandoraRedo@BeginModule 19-Nov-2024 07:14:26 AST run: 1435 subRun: 1 event: 50498
CreatePandoraMCParticles - found an unknown process
%MSG
%MSG-w LArPandora: StandardPandora:pandoraRedo@BeginModule 19-Nov-2024 07:14:26 AST run: 1435 subRun: 1 event: 50498
CreatePandoraMCParticles - found an unknown process
%MSG
%MSG-w LArPandora: StandardPandora:pandoraRedo@BeginModule 19-Nov-2024 07:14:26 AST run: 1435 subRun: 1 event: 50498
CreatePandoraMCParticles - found an unknown process
%MSG
%MSG-e ShowerProducedPtrsHolder: LArPandoraModularShowerCreation:pandoraShowerRedo@BeginModule 19-Nov-2024 07:14:30 AST run: 1435 subRun: 1 event: 50498
Trying to add data product: InitialTrack. This element does not exist in the element holder
%MSG
%MSG-e ShowerProducedPtrsHolder: LArPandoraModularShowerCreation:pandoraShowerRedo@BeginModule 19-Nov-2024 07:14:30 AST run: 1435 subRun: 1 event: 50498
Trying to add data product: InitialTrack. This element does not exist in the element holder
%MSG
Boundary wire vector sizes: 811, 1007, 822
minwire 0: 1822
minwire 1: 412
minwire 2: 1827
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary:
Output 0: 0.0991449,
Output 1: 0.00231878, 0.862777, 0.011935, 0.122969,
Output 2: 0.0325543, 0.161214, 0.804948, 0.00128411,
Output 3: 0.188442, 0.43542, 0.218218, 0.15792,
Output 4: 0.747821, 0.221775, 0.0251794, 0.00522411,
Output 5: 0.780936, 0.213088, 0.00566166, 0.000313963,
Output 6: 0.999163, 0.000752672, 5.45876e-05, 2.97848e-05,
%MSG-e ShowerProducedPtrsHolder: LArPandoraModularShowerCreation:pandoraShowerRedo@BeginModule 19-Nov-2024 07:14:49 AST run: 1435 subRun: 1 event: 50499
Trying to add data product: InitialTrack. This element does not exist in the element holder
%MSG
%MSG-e ShowerProducedPtrsHolder: LArPandoraModularShowerCreation:pandoraShowerRedo@BeginModule 19-Nov-2024 07:14:49 AST run: 1435 subRun: 1 event: 50499
Trying to add data product: InitialTrack. This element does not exist in the element holder
%MSG
Boundary wire vector sizes: 225, 289, 246
minwire 0: 2169
minwire 1: 205
minwire 2: 2529
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max wires due to vertex determination failure: 2379, 2878
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary:
Output 0: 0.0216984,
Output 1: 0.161696, 0.023428, 0.00674072, 0.808135,
Output 2: 0.00606029, 0.712308, 0.234561, 0.0470699,
Output 3: 0.00633672, 0.0320464, 0.186403, 0.775214,
Output 4: 0.827966, 0.16109, 0.0108432, 0.000100677,
Output 5: 0.0543616, 0.945518, 0.000117041, 3.73325e-06,
Output 6: 0.999182, 0.000769877, 3.778e-05, 1.02913e-05,
%MSG-e DUNEAna: dunereco/FDSelections/CCNuSelection:ccnuselection@BeginModule 19-Nov-2024 07:14:52 AST run: 1435 subRun: 1 event: 50499
Failed to find product with label caldata ... returning empty vector
%MSG
%MSG-e DUNEAna: dunereco/FDSelections/CCNuSelection:ccnuselection@BeginModule 19-Nov-2024 07:14:52 AST run: 1435 subRun: 1 event: 50499
Failed to find product with label caldata ... returning empty vector
%MSG
PandoraContentApi::GetList(*this, m_inputHitListName, pCaloHitList) return STATUS_CODE_NOT_INITIALIZED
in function: GetVolumeIdToHitListMap
in file: /exp/dune/app/users/imawby/dunesw_v09_91_02d00/srcs/larpandoracontent/larpandoracontent/LArControlFlow/MasterAlgorithm.cc line#: 271
this->GetVolumeIdToHitListMap(volumeIdToHitListMap) return STATUS_CODE_NOT_INITIALIZED
in function: Run
in file: /exp/dune/app/users/imawby/dunesw_v09_91_02d00/srcs/larpandoracontent/larpandoracontent/LArControlFlow/MasterAlgorithm.cc line#: 165
iter->second->Run() throw STATUS_CODE_NOT_INITIALIZED
in function: RunAlgorithm
in file: /scratch/workspace/build-larbase/BUILDTYPE/prof/QUAL/s131-e26/label1/swarm/label2/SLF7/build/pandora/v03_16_00l/src/pandora-v03-16-00/PandoraSDK-v03-04-01/src/Api/PandoraContentApiImpl.cc line#: 235
Failure in algorithm Alg0002, LArDLMaster, STATUS_CODE_NOT_INITIALIZED
19-Nov-2024 07:14:55 AST Closed input file "root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-hd/8b/0e/nue_dune10kt_1x2x6_1435_504_20230828T103555Z_gen_g4_detsim_hitreco__20240221T034825Z_reco2.root"
19-Nov-2024 07:14:55 AST Closed input file "root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-hd/8b/0e/nue_dune10kt_1x2x6_1435_504_20230828T103555Z_gen_g4_detsim_hitreco__20240221T034825Z_reco2.root"
TrigReport ---------- Event summary -------------
TrigReport Events total = 100 passed = 100 failed = 0
TrigReport ---------- Modules in End-path ----------
TrigReport Run Success Error Name
TrigReport 100 100 0 ccnuselection
TimeReport ---------- Time summary [sec] -------
TimeReport CPU = 8167.418326 Real = 7880.983994
MemReport ---------- Memory summary [base-10 MB] ------
MemReport VmPeak = 4008.35 VmHWM = 2281.59
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file === 