justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

21 July 2025: This instance at RAL is read-only. Please do not try submitting new workflows for now.

Jobsub ID 222819.21@justin-prod-sched02.dune.hep.ac.uk

Jobsub ID222819.21@justin-prod-sched02.dune.hep.ac.uk
Workflow ID7662
Stage ID1
User namewwang@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
GPUNo
RSS bytes2097152000 (2000 MiB)
Wall seconds limit7200 (2 hours)
Submitted time2025-06-16 12:10:06
SiteUS_Wisconsin
EntryHCC_US_Wisconsin_osg01_rhel7
Last heartbeat2025-06-16 14:15:10
From worker nodeHostnamee2472
cpuinfoAMD EPYC 7452 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes2097152000 (2000 MiB)
Wall seconds limit82800 (23 hours)
GPU
Inner Apptainer?True
Job stateoutputting_failed
Allocator namejustin-allocator-pro.dune.hep.ac.uk
Started2025-06-16 13:40:52
Input files
JobscriptExit code0
Real time4m (240s)
CPU time2m (169s = 70%)
Max RSS bytes783626240 (747 MiB)
Outputting started2025-06-16 13:44:52
Output files
Finished2025-06-16 14:15:10
List job events     Wrapper job log

Jobscript log (last 10,000 characters)

Setting up larsoft UPS area... /cvmfs/larsoft.opensciencegrid.org
Setting up DUNE UPS area... /cvmfs/dune.opensciencegrid.org/products/dune/
/cvmfs/larsoft.opensciencegrid.org/products/xrootd/v5_5_5a/Linux64bit+3.10-2.17-e26-p3915-prof/lib/libXrdPosixPreload.so
=== Start last 100 lines of lar log file ===
5980  -727.102  -31.1352  83.9188
5981  -727.102  -93.4056  83.9188
5982  -727.102  -155.676  83.9188
5983  -727.102  -217.946  83.9188
5984  -727.102  -280.217  83.9188
5985  -727.102  -342.487  83.9188
5986  -727.102  -404.758  83.9188
5987  -727.102  -467.028  83.9188
5988  -727.102  -529.298  83.9188
5989  -727.102  -591.569  83.9188
5990  -727.102  -31.1352  35.1188
5991  -727.102  -93.4056  35.1188
5992  -727.102  -155.676  35.1188
5993  -727.102  -217.946  35.1188
5994  -727.102  -280.217  35.1188
5995  -727.102  -342.487  35.1188
5996  -727.102  -404.758  35.1188
5997  -727.102  -467.028  35.1188
5998  -727.102  -529.298  35.1188
5999  -727.102  -591.569  35.1188
Cryo Boundaries
Xmin: -747.298 Xmax: 747.298 Ymin: -658.099 Ymax: 699.599 Zmin: -302.946 Zmax: 5858.94
Begin processing the 1st record. run: 1 subRun: 0 event: 1 at 16-Jun-2025 08:42:50 CDT
%MSG-w ParticleInventory:  ProcessEvent 16-Jun-2025 08:42:50 CDT  run: 1 subRun: 0 event: 1
Rebuild failed to get the MCParticles. This is expected when running on a generation or simulation step.
%MSG
%MSG-w ParticleInventory:  ProcessEvent 16-Jun-2025 08:42:50 CDT  run: 1 subRun: 0 event: 1
Rebuild failed to get the MCParticles. This is expected when running on a generation or simulation step.
%MSG
%MSG-w ParticleInventory:  ProcessEvent 16-Jun-2025 08:42:50 CDT  run: 1 subRun: 0 event: 1
Rebuild failed to get the MCParticles. This is expected when running on a generation or simulation step.
%MSG
%MSG-w BackTrackerService:  ProcessEvent 16-Jun-2025 08:42:50 CDT  run: 1 subRun: 0 event: 1
Rebuild failed to get the SimChannels. This is expected when running on a generation or simulation step.
%MSG
%MSG-w PhotonBackTrackerService:  ProcessEvent 16-Jun-2025 08:42:50 CDT  run: 1 subRun: 0 event: 1
Rebuild failed to get the OpDetBTRs. This is expected when running on a generation or simulation step.
%MSG
%MSG-w LightSource:  LightSource:generator@BeginModule 16-Jun-2025 08:42:50 CDT  run: 1 subRun: 0 event: 1
EVGEN Light Source : Warning, reached end of file, looping back to beginning
%MSG
%MSG-i PhotonVisibilityService:  SimPhotonCounter:pmtresponse@EndJob  16-Jun-2025 08:42:50 CDT ModuleEndJob
 Vis service running library build job.  Please ensure  job contains LightSource, LArG4, SimPhotonCounter
%MSG
%MSG-i PhotonVisibilityService:  SimPhotonCounter:pmtresponse@EndJob  16-Jun-2025 08:42:50 CDT ModuleEndJob
 Vis service  Storing Library entries to file...
%MSG
%MSG-i PhotonLibrary:  SimPhotonCounter:pmtresponse@EndJob  16-Jun-2025 08:42:50 CDT ModuleEndJob
Writing photon library to file
%MSG

====================================================================================================================
TimeTracker printout (sec)            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================
[ No processed events ]
====================================================================================================================
%MSG-i NuRandomService:  SimPhotonCounter:pmtresponse@EndJob  16-Jun-2025 08:44:51 CDT ModuleEndJob

Summary of seeds computed by the NuRandomService
Random policy: 'random'
  master seed: 694104684
  seed within: [ 1 ; 900000000 ]
   Configured value          Last value   ModuleLabel.InstanceName
          343126742              (same)   generator
          896742218              (same)   largeant.GEANT
          419727531              (same)   largeant.propagation

%MSG

TrigReport ---------- Event summary -------------
TrigReport Events total = 1 passed = 0 failed = 1

TrigReport ---------- Modules in End-path ----------
TrigReport        Run    Success      Error Name
TrigReport          0          0          0 pmtresponse

TimeReport ---------- Time summary [sec] -------
TimeReport CPU = 125.504239 Real = 129.513260

MemReport  ---------- Memory summary [base-10 MB] ------
MemReport  VmPeak = 1643.01 VmHWM = 765.792

Graphics systems deleted.
Visualization Manager deleting...
%MSG-s ArtException:  PostEndJob 16-Jun-2025 08:44:51 CDT ModuleEndJob
---- EventProcessorFailure BEGIN
  EventProcessor: an exception occurred during current event processing
  ---- ScheduleExecutionFailure BEGIN
    Path: ProcessingStopped.
    ---- OtherArt BEGIN
      ---- LightSource BEGIN
        EVGEN Light Source : File error in $INPUT_TAR_DIR_LOCAL/myLightSourceSteering.txt
        The above exception was thrown while processing module LightSource/generator run: 1 subRun: 0 event: 1
      ---- LightSource END
    ---- OtherArt END
    Exception going through path simulate
  ---- ScheduleExecutionFailure END
---- EventProcessorFailure END
%MSG
Art has completed and will exit with status 1.
=== End last 100 lines of lar log file ===
lar exit code 0
2025-06-16T_134124Z
This job is done with output file 2025-06-16T_134124Z_reco_2025-06-16T_134124Z.root created at 2025-06-16T_134124Z
justIN time: 2025-08-14 16:40:32 UTC       justIN version: 01.03.02