21 July 2025: This instance at RAL is read-only. Please do not try submitting new workflows for now.
Jobsub ID 230421.1@justin-prod-sched02.dune.hep.ac.uk
| Jobsub ID | 230421.1@justin-prod-sched02.dune.hep.ac.uk | |
| Workflow ID | 7956 | |
| Stage ID | 1 | |
| User name | gabrielb@fnal.gov | |
| HTCondor Group | group_dune | |
| Requested | Processors | 1 |
| GPU | No | |
| RSS bytes | 16777216000 (16000 MiB) | |
| Wall seconds limit | 80000 (22 hours) | |
| Submitted time | 2025-06-26 18:58:22 | |
| Site | FR_CCIN2P3 | |
| Entry | DUNE_FR_CCIN2P3_cccondorce04 | |
| Last heartbeat | 2025-06-26 19:14:48 | |
| From worker node | Hostname | ccwcondor0597 |
| cpuinfo | AMD EPYC 7453 28-Core Processor | |
| OS release | Scientific Linux release 7.9 (Nitrogen) | |
| Processors | 1 | |
| RSS bytes | 16777216000 (16000 MiB) | |
| Wall seconds limit | 106200 (29 hours) | |
| GPU | ||
| Inner Apptainer? | True | |
| Job state | jobscript_error | |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk | |
| Started | 2025-06-26 18:59:33 | |
| Input files | ||
| Jobscript | Exit code | 1 |
| Real time | 0m (0s) | |
| CPU time | 0m (0s = 0%) | |
| Max RSS bytes | 0 (0 MiB) | |
| Outputting started | ||
| Output files | ||
| Finished | 2025-06-26 19:14:48 | |
| Saved logs | justin-logs:230421.1-justin-prod-sched02.dune.hep.ac.uk.logs.tgz | |
| List job events Wrapper job log | ||
Jobscript log (last 10,000 characters)
ce: PDFastSimPVS:PDFastSimExternal@BeginModule 26-Jun-2025 21:08:25 CEST run: 1 subRun: 0 event: 1
Random seed for this event, engine 'PDFastSimExternal.photon': 105221609
%MSG
%MSG-i NuRandomService: PDFastSimPVS:PDFastSimExternal@BeginModule 26-Jun-2025 21:08:25 CEST run: 1 subRun: 0 event: 1
Random seed for this event, engine 'PDFastSimExternal.scinttime': 867117347
%MSG
%MSG-i PhotonVisibilityService: PDFastSimPVS:PDFastSimExternal@BeginModule 26-Jun-2025 21:08:27 CEST run: 1 subRun: 0 event: 1
PhotonVisibilityService Loading photon library from file /cvmfs/dune.osgstorage.org/pnfs/fnal.gov/usr/dune/persistent/stash/PhotonPropagation/LibraryData/Photon_library_protoDUNEhd_library_protoDUNEhd_v2_refactored_nonActive.root for 760182 voxels and 160 optical detectors.
%MSG
%MSG-i PhotonLibrary: PDFastSimPVS:PDFastSimExternal@BeginModule 26-Jun-2025 21:08:27 CEST run: 1 subRun: 0 event: 1
Reading photon library from input file: /cvmfs/dune.osgstorage.org/pnfs/fnal.gov/usr/dune/persistent/stash/PhotonPropagation/LibraryData/Photon_library_protoDUNEhd_library_protoDUNEhd_v2_refactored_nonActive.root
%MSG
%MSG-i PhotonLibrary: PDFastSimPVS:PDFastSimExternal@BeginModule 26-Jun-2025 21:08:34 CEST run: 1 subRun: 0 event: 1
Photon lookup table size : 760182 voxels, 160 channels (no voxel geometry included)
%MSG
26-Jun-2025 21:08:34 CEST Opened output file with pattern "cosmics_g4_.root"
%MSG-i Root_Information: PostProcessEvent TClass::Init() 26-Jun-2025 21:08:34 CEST run: 1 subRun: 0 event: 1
no dictionary for class sim::IDE is available
ROOT severity: 2000
%MSG
%MSG-i FastCloning: PostProcessEvent 26-Jun-2025 21:08:34 CEST run: 1 subRun: 0 event: 1
Fast cloning event data products from input file.
%MSG
Begin processing the 2nd record. run: 1 subRun: 0 event: 2 at 26-Jun-2025 21:09:07 CEST
%MSG-w ParticleInventory: ProcessEvent 26-Jun-2025 21:09:07 CEST run: 1 subRun: 0 event: 2
Rebuild failed to get the MCParticles. This is expected when running on a generation or simulation step.
%MSG
%MSG-w ParticleInventory: ProcessEvent 26-Jun-2025 21:09:07 CEST run: 1 subRun: 0 event: 2
Rebuild failed to get the MCParticles. This is expected when running on a generation or simulation step.
%MSG
%MSG-w ParticleInventory: ProcessEvent 26-Jun-2025 21:09:07 CEST run: 1 subRun: 0 event: 2
Rebuild failed to get the MCParticles. This is expected when running on a generation or simulation step.
%MSG
%MSG-w PhotonBackTrackerService: ProcessEvent 26-Jun-2025 21:09:07 CEST run: 1 subRun: 0 event: 2
Rebuild failed to get the OpDetBTRs. This is expected when running on a generation or simulation step.
%MSG
%MSG-i NuRandomService: larg4Main:largeant@BeginModule 26-Jun-2025 21:09:07 CEST run: 1 subRun: 0 event: 2
Random seed for this event, engine 'largeant': 201819427
%MSG
%MSG-i generatePrimaries: larg4Main:largeant@BeginModule 26-Jun-2025 21:09:07 CEST run: 1 subRun: 0 event: 2 MCTruthEventAction.cc:112
Generating 845 particles
%MSG
%MSG-w beginOfEventAction::keepableGenerators: larg4Main:largeant@BeginModule 26-Jun-2025 21:09:07 CEST run: 1 subRun: 0 event: 2
storeTrajectories set to true and a non-empty keepGenTrajectories list provided in configuration file, but none of the generators in this list are present in the event! Double check list or don't provide keepGenTrajectories in the configuration to keep all trajectories from all generator labels. This may be expected for generators that have a nonzero probability of producing no particles (e.g. some radiologicals)
%MSG
%MSG-i endOfEventAction: larg4Main:largeant@BeginModule 26-Jun-2025 21:09:22 CEST run: 1 subRun: 0 event: 2 ParticleListAction.cc:817
MCTruth Handles Size: 1
%MSG
%MSG-i endOfEventAction: larg4Main:largeant@BeginModule 26-Jun-2025 21:09:22 CEST run: 1 subRun: 0 event: 2 ParticleListAction.cc:827
mclistHandle Size: 1
%MSG
%MSG-i endOfEventAction: larg4Main:largeant@BeginModule 26-Jun-2025 21:09:22 CEST run: 1 subRun: 0 event: 2 ParticleListAction.cc:830
Found 845 particles
%MSG
%MSG-i NuRandomService: IonAndScint:IonAndScint@BeginModule 26-Jun-2025 21:09:24 CEST run: 1 subRun: 0 event: 2
Random seed for this event, engine 'IonAndScint.ISCalcAlg': 57473524
%MSG
IonAndScint Module Producer
SimEnergyDeposit input module: largeant, instance name: LArG4DetectorServicevolTPCActiveInner
Skip SimEnergyDeposit in: LArG4DetectorServicevolTPCActiveOuter
%MSG-i NuRandomService: IonAndScint:IonAndScintExternal@BeginModule 26-Jun-2025 21:09:25 CEST run: 1 subRun: 0 event: 2
Random seed for this event, engine 'IonAndScintExternal.ISCalcAlg': 266590569
%MSG
IonAndScint Module Producer
Skip SimEnergyDeposit in:
Skip SimEnergyDeposit in: LArG4DetectorServicevolTPCActiveInner
Skip SimEnergyDeposit in: priorSCE
SimEnergyDeposit input module: largeant, instance name: LArG4DetectorServicevolTPCActiveOuter
%MSG-i NuRandomService: PDFastSimPAR:PDFastSim@BeginModule 26-Jun-2025 21:09:25 CEST run: 1 subRun: 0 event: 2
Random seed for this event, engine 'PDFastSim.photon': 734915769
%MSG
%MSG-i NuRandomService: PDFastSimPAR:PDFastSim@BeginModule 26-Jun-2025 21:09:25 CEST run: 1 subRun: 0 event: 2
Random seed for this event, engine 'PDFastSim.scinttime': 211901621
%MSG
%MSG-i NuRandomService: PDFastSimPVS:PDFastSimExternal@BeginModule 26-Jun-2025 21:13:26 CEST run: 1 subRun: 0 event: 2
Random seed for this event, engine 'PDFastSimExternal.photon': 68129217
%MSG
%MSG-i NuRandomService: PDFastSimPVS:PDFastSimExternal@BeginModule 26-Jun-2025 21:13:26 CEST run: 1 subRun: 0 event: 2
Random seed for this event, engine 'PDFastSimExternal.scinttime': 820997796
%MSG
IonAndScint endJob.
IonAndScint endJob.
26-Jun-2025 21:13:40 CEST Closed input file "data_gen.root"
================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
================================================================================================================================
Full event 135.08 201.055 267.029 201.055 65.9748 2
--------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.000854704 0.00452054 0.00818638 0.00452054 0.00366584 2
simulate:rns:RandomNumberSaver 0.000296707 0.00564127 0.0109858 0.00564127 0.00534456 2
simulate:largeant:larg4Main 7.4211 12.0913 16.7616 12.0913 4.67023 2
simulate:IonAndScint:IonAndScint 0.367152 0.608532 0.849911 0.608532 0.241379 2
simulate:IonAndScintExternal:IonAndScint 0.00871062 0.0343305 0.0599505 0.0343305 0.0256199 2
simulate:PDFastSim:PDFastSimPAR 91.0248 166.053 241.081 166.053 75.0284 2
simulate:PDFastSimExternal:PDFastSimPVS 8.24169 8.43147 8.62125 8.43147 0.189779 2
[art]:TriggerResults:TriggerResultInserter 2.3003e-05 4.2535e-05 6.2067e-05 4.2535e-05 1.9532e-05 2
end_path:out1:RootOutput 8.415e-06 1.5043e-05 2.1671e-05 1.5043e-05 6.628e-06 2
end_path:out1:RootOutput(write) 27.6221 27.6221 27.6221 27.6221 0 1
================================================================================================================================
%MSG-i NuRandomService: RootOutput:out1@EndJob 26-Jun-2025 21:13:40 CEST ModuleEndJob
Summary of seeds computed by the NuRandomService
Random policy: 'perEvent'
algorithm version: EventTimestamp_v1
Configured value Last value ModuleLabel.InstanceName
(per event) 57473524 IonAndScint.ISCalcAlg
(per event) 266590569 IonAndScintExternal.ISCalcAlg
(per event) 734915769 PDFastSim.photon
(per event) 211901621 PDFastSim.scinttime
(per event) 68129217 PDFastSimExternal.photon
(per event) 820997796 PDFastSimExternal.scinttime
(per event) 201819427 largeant
%MSG
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 13682.5 MB
Peak resident set size usage (VmHWM): 12397.2 MB
====================================================================================================
TrigReport ---------- Event summary -------------
TrigReport Events total = 2 passed = 2 failed = 0
TrigReport ---------- Modules in End-path ----------
TrigReport Run Success Error Name
TrigReport 2 2 0 out1
TimeReport ---------- Time summary [sec] -------
TimeReport CPU = 418.106028 Real = 427.434211
MemReport ---------- Memory summary [base-10 MB] ------
MemReport VmPeak = 13682.5 VmHWM = 12397.2
%MSG-s ArtException: PostEndJob 26-Jun-2025 21:14:25 CEST ModuleEndJob
---- EventProcessorFailure BEGIN
EventProcessor: an exception occurred during current event processing
---- FatalRootError BEGIN
Fatal Root Error: TBufferFile::WriteByteCount
bytecount too large (more than 1073741822)
ROOT severity: 3000
---- FatalRootError END
---- EventProcessorFailure END
---- FatalRootError BEGIN
Fatal Root Error: TTree::SetEntries
Tree branches have different numbers of entries, eg EventAuxiliary has 1 entries while simb::MCTruths_cosmicgenerator__SinglesGen. has 10 entries.
ROOT severity: 2000
---- FatalRootError END
%MSG
Art has completed and will exit with status 1.
lar exit code 1