justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

21 July 2025: This instance at RAL is read-only. Please do not try submitting new workflows for now.

Jobsub ID 423274.4@justin-prod-sched01.dune.hep.ac.uk

Jobsub ID423274.4@justin-prod-sched01.dune.hep.ac.uk
Workflow ID7956
Stage ID1
User namegabrielb@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
GPUNo
RSS bytes16777216000 (16000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2025-06-27 03:26:59
SiteFR_CCIN2P3
EntryDUNE_FR_CCIN2P3_cccondorce02
Last heartbeat2025-06-27 06:05:50
From worker nodeHostnameccwcondor0583
cpuinfoAMD EPYC 7453 28-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes16777216000 (16000 MiB)
Wall seconds limit106200 (29 hours)
GPU
Inner Apptainer?True
Job statejobscript_error
Allocator namejustin-allocator-pro.dune.hep.ac.uk
Started2025-06-27 03:28:01
Input files
JobscriptExit code1
Real time0m (0s)
CPU time0m (0s = 0%)
Max RSS bytes0 (0 MiB)
Outputting started 
Output files
Finished2025-06-27 06:05:50
Saved logsjustin-logs:423274.4-justin-prod-sched01.dune.hep.ac.uk.logs.tgz
List job events     Wrapper job log

Jobscript log (last 10,000 characters)

stAction.cc:817
MCTruth Handles Size: 1
%MSG
%MSG-i endOfEventAction:  larg4Main:largeant@BeginModule  27-Jun-2025 05:55:37 CEST run: 1 subRun: 0 event: 8 ParticleListAction.cc:827
mclistHandle Size: 1
%MSG
%MSG-i endOfEventAction:  larg4Main:largeant@BeginModule  27-Jun-2025 05:55:37 CEST run: 1 subRun: 0 event: 8 ParticleListAction.cc:830
Found 901 particles
%MSG
%MSG-i NuRandomService:  IonAndScint:IonAndScint@BeginModule  27-Jun-2025 05:55:38 CEST run: 1 subRun: 0 event: 8
Random seed for this event, engine 'IonAndScint.ISCalcAlg': 367924990
%MSG
IonAndScint Module Producer
SimEnergyDeposit input module: largeant, instance name: LArG4DetectorServicevolTPCActiveInner
Skip SimEnergyDeposit in: LArG4DetectorServicevolTPCActiveOuter
%MSG-i NuRandomService:  IonAndScint:IonAndScintExternal@BeginModule  27-Jun-2025 05:55:39 CEST run: 1 subRun: 0 event: 8
Random seed for this event, engine 'IonAndScintExternal.ISCalcAlg': 680046877
%MSG
IonAndScint Module Producer
Skip SimEnergyDeposit in: 
Skip SimEnergyDeposit in: LArG4DetectorServicevolTPCActiveInner
Skip SimEnergyDeposit in: priorSCE
SimEnergyDeposit input module: largeant, instance name: LArG4DetectorServicevolTPCActiveOuter
%MSG-i NuRandomService:  PDFastSimPAR:PDFastSim@BeginModule  27-Jun-2025 05:55:39 CEST run: 1 subRun: 0 event: 8
Random seed for this event, engine 'PDFastSim.photon': 105328127
%MSG
%MSG-i NuRandomService:  PDFastSimPAR:PDFastSim@BeginModule  27-Jun-2025 05:55:39 CEST run: 1 subRun: 0 event: 8
Random seed for this event, engine 'PDFastSim.scinttime': 218563371
%MSG
%MSG-i NuRandomService:  PDFastSimPVS:PDFastSimExternal@BeginModule  27-Jun-2025 05:58:13 CEST run: 1 subRun: 0 event: 8
Random seed for this event, engine 'PDFastSimExternal.photon': 245479188
%MSG
%MSG-i NuRandomService:  PDFastSimPVS:PDFastSimExternal@BeginModule  27-Jun-2025 05:58:13 CEST run: 1 subRun: 0 event: 8
Random seed for this event, engine 'PDFastSimExternal.scinttime': 710413138
%MSG
Begin processing the 9th record. run: 1 subRun: 0 event: 9 at 27-Jun-2025 05:59:06 CEST
%MSG-w ParticleInventory:  ProcessEvent 27-Jun-2025 05:59:06 CEST  run: 1 subRun: 0 event: 9
Rebuild failed to get the MCParticles. This is expected when running on a generation or simulation step.
%MSG
%MSG-w ParticleInventory:  ProcessEvent 27-Jun-2025 05:59:06 CEST  run: 1 subRun: 0 event: 9
Rebuild failed to get the MCParticles. This is expected when running on a generation or simulation step.
%MSG
%MSG-w ParticleInventory:  ProcessEvent 27-Jun-2025 05:59:06 CEST  run: 1 subRun: 0 event: 9
Rebuild failed to get the MCParticles. This is expected when running on a generation or simulation step.
%MSG
%MSG-w PhotonBackTrackerService:  ProcessEvent 27-Jun-2025 05:59:06 CEST  run: 1 subRun: 0 event: 9
Rebuild failed to get the OpDetBTRs. This is expected when running on a generation or simulation step.
%MSG
%MSG-i NuRandomService:  larg4Main:largeant@BeginModule  27-Jun-2025 05:59:06 CEST run: 1 subRun: 0 event: 9
Random seed for this event, engine 'largeant': 535289496
%MSG
%MSG-i generatePrimaries:  larg4Main:largeant@BeginModule  27-Jun-2025 05:59:06 CEST run: 1 subRun: 0 event: 9 MCTruthEventAction.cc:112
Generating 770 particles
%MSG
%MSG-w beginOfEventAction::keepableGenerators:  larg4Main:largeant@BeginModule  27-Jun-2025 05:59:07 CEST run: 1 subRun: 0 event: 9
storeTrajectories set to true and a non-empty keepGenTrajectories list provided in configuration file, but none of the generators in this list are present in the event! Double check list or don't provide keepGenTrajectories in the configuration to keep all trajectories from all generator labels. This may be expected for generators that have a nonzero probability of producing no particles (e.g. some radiologicals)
%MSG
%MSG-i endOfEventAction:  larg4Main:largeant@BeginModule  27-Jun-2025 05:59:31 CEST run: 1 subRun: 0 event: 9 ParticleListAction.cc:817
MCTruth Handles Size: 1
%MSG
%MSG-i endOfEventAction:  larg4Main:largeant@BeginModule  27-Jun-2025 05:59:31 CEST run: 1 subRun: 0 event: 9 ParticleListAction.cc:827
mclistHandle Size: 1
%MSG
%MSG-i endOfEventAction:  larg4Main:largeant@BeginModule  27-Jun-2025 05:59:31 CEST run: 1 subRun: 0 event: 9 ParticleListAction.cc:830
Found 770 particles
%MSG
%MSG-i NuRandomService:  IonAndScint:IonAndScint@BeginModule  27-Jun-2025 05:59:35 CEST run: 1 subRun: 0 event: 9
Random seed for this event, engine 'IonAndScint.ISCalcAlg': 425609593
%MSG
IonAndScint Module Producer
SimEnergyDeposit input module: largeant, instance name: LArG4DetectorServicevolTPCActiveInner
Skip SimEnergyDeposit in: LArG4DetectorServicevolTPCActiveOuter
%MSG-i NuRandomService:  IonAndScint:IonAndScintExternal@BeginModule  27-Jun-2025 05:59:39 CEST run: 1 subRun: 0 event: 9
Random seed for this event, engine 'IonAndScintExternal.ISCalcAlg': 419151965
%MSG
IonAndScint Module Producer
Skip SimEnergyDeposit in: 
Skip SimEnergyDeposit in: LArG4DetectorServicevolTPCActiveInner
Skip SimEnergyDeposit in: priorSCE
SimEnergyDeposit input module: largeant, instance name: LArG4DetectorServicevolTPCActiveOuter
%MSG-i NuRandomService:  PDFastSimPAR:PDFastSim@BeginModule  27-Jun-2025 05:59:39 CEST run: 1 subRun: 0 event: 9
Random seed for this event, engine 'PDFastSim.photon': 175391771
%MSG
%MSG-i NuRandomService:  PDFastSimPAR:PDFastSim@BeginModule  27-Jun-2025 05:59:39 CEST run: 1 subRun: 0 event: 9
Random seed for this event, engine 'PDFastSim.scinttime': 150631952
%MSG
%MSG-i NuRandomService:  PDFastSimPVS:PDFastSimExternal@BeginModule  27-Jun-2025 08:05:10 CEST run: 1 subRun: 0 event: 9
Random seed for this event, engine 'PDFastSimExternal.photon': 773473513
%MSG
%MSG-i NuRandomService:  PDFastSimPVS:PDFastSimExternal@BeginModule  27-Jun-2025 08:05:10 CEST run: 1 subRun: 0 event: 9
Random seed for this event, engine 'PDFastSimExternal.scinttime': 248218366
%MSG
IonAndScint endJob.
IonAndScint endJob.
27-Jun-2025 08:05:19 CEST  Closed input file "data_gen.root"

================================================================================================================================
TimeTracker printout (sec)                        Min           Avg           Max         Median          RMS         nEvts   
================================================================================================================================
Full event                                      127.858       990.743       7569.77       186.354       2326.21         9     
--------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                        0.00118939    0.00308949    0.00588322    0.00269743    0.00126073        9     
simulate:rns:RandomNumberSaver                0.000495797   0.00838309     0.011793     0.00922601    0.00317072        9     
simulate:largeant:larg4Main                     7.62849       12.3165       28.3048       9.66618       6.14602         9     
simulate:IonAndScint:IonAndScint               0.345717       1.02566       4.76227      0.654297       1.3295          9     
simulate:IonAndScintExternal:IonAndScint       0.0217926     0.0362846     0.0550745     0.0354715    0.00886255        9     
simulate:PDFastSim:PDFastSimPAR                 91.8791        944.6        7531.08       127.428       2328.76         9     
simulate:PDFastSimExternal:PDFastSimPVS         2.93717       5.09482       10.5645       4.95648       2.19726         9     
[art]:TriggerResults:TriggerResultInserter    1.4537e-05    2.71987e-05   9.7262e-05    1.8485e-05    2.49366e-05       9     
end_path:out1:RootOutput                       5.41e-06     9.06689e-06   3.1248e-05     6.532e-06    7.86849e-06       9     
end_path:out1:RootOutput(write)                 23.996        31.1001       39.3594       30.9015       5.32453         8     
================================================================================================================================
%MSG-i NuRandomService:  RootOutput:out1@EndJob 27-Jun-2025 08:05:19 CEST  ModuleEndJob

Summary of seeds computed by the NuRandomService
Random policy: 'perEvent'
  algorithm version: EventTimestamp_v1
   Configured value          Last value   ModuleLabel.InstanceName
        (per event)           425609593   IonAndScint.ISCalcAlg
        (per event)           419151965   IonAndScintExternal.ISCalcAlg
        (per event)           175391771   PDFastSim.photon
        (per event)           150631952   PDFastSim.scinttime
        (per event)           773473513   PDFastSimExternal.photon
        (per event)           248218366   PDFastSimExternal.scinttime
        (per event)           535289496   largeant

%MSG

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 27340 MB
  Peak resident set size usage (VmHWM): 23605.2 MB
====================================================================================================

TrigReport ---------- Event summary -------------
TrigReport Events total = 9 passed = 9 failed = 0

TrigReport ---------- Modules in End-path ----------
TrigReport        Run    Success      Error Name
TrigReport          9          9          0 out1

TimeReport ---------- Time summary [sec] -------
TimeReport CPU = 8789.806404 Real = 8989.985952

MemReport  ---------- Memory summary [base-10 MB] ------
MemReport  VmPeak = 27340 VmHWM = 23605.2

terminate called after throwing an instance of 'cet::coded_exception<art::errors::ErrorCodes, &art::ExceptionDetail::translate[abi:cxx11]>'
  what():  ---- FatalRootError BEGIN
  Fatal Root Error: TBufferFile::Expand
  Requested size (2147483647) is too large (max is 2147483646).
  ROOT severity: 6000
---- FatalRootError END

../justin-jobscript: line 82:  1198 Aborted                 (core dumped) lar -c standard_g4_protodunehd.fcl -s data_gen.root -o "$outFile"
lar exit code 134
justIN time: 2025-08-14 18:48:37 UTC       justIN version: 01.03.02