21 July 2025: This instance at RAL is read-only. Please do not try submitting new workflows for now.
Jobsub ID 423891.14@justin-prod-sched01.dune.hep.ac.uk
| Jobsub ID | 423891.14@justin-prod-sched01.dune.hep.ac.uk | |
| Workflow ID | 8014 | |
| Stage ID | 1 | |
| User name | gabrielb@fnal.gov | |
| HTCondor Group | group_dune | |
| Requested | Processors | 1 |
| GPU | No | |
| RSS bytes | 41943040000 (40000 MiB) | |
| Wall seconds limit | 80000 (22 hours) | |
| Submitted time | 2025-06-27 18:43:38 | |
| Site | CERN | |
| Entry | CMSHTPC_T2_CH_CERN_ce507 | |
| Last heartbeat | 2025-06-27 19:43:21 | |
| From worker node | Hostname | b9p11p3792.cern.ch |
| cpuinfo | AMD EPYC 7543 32-Core Processor | |
| OS release | Scientific Linux release 7.9 (Nitrogen) | |
| Processors | 1 | |
| RSS bytes | 41943040000 (40000 MiB) | |
| Wall seconds limit | 343800 (95 hours) | |
| GPU | ||
| Inner Apptainer? | True | |
| Job state | jobscript_error | |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk | |
| Started | 2025-06-27 19:10:40 | |
| Input files | ||
| Jobscript | Exit code | 1 |
| Real time | 0m (0s) | |
| CPU time | 0m (0s = 0%) | |
| Max RSS bytes | 0 (0 MiB) | |
| Outputting started | ||
| Output files | ||
| Finished | 2025-06-27 19:43:21 | |
| Saved logs | justin-logs:423891.14-justin-prod-sched01.dune.hep.ac.uk.logs.tgz | |
| List job events Wrapper job log | ||
Jobscript log (last 10,000 characters)
===================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
================================================================================================================================
Full event 141.904 190.9 304.79 176.412 52.4846 7
--------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.000698344 0.00247263 0.0050205 0.00198813 0.00125944 7
simulate:rns:RandomNumberSaver 0.000229525 0.00673443 0.0106824 0.00805583 0.00315851 7
simulate:largeant:larg4Main 8.58617 10.8879 12.8524 11.0483 1.77861 7
simulate:IonAndScint:IonAndScint 0.461743 0.579954 0.971505 0.500514 0.168723 7
simulate:IonAndScintExternal:IonAndScint 0.0131349 0.0329833 0.0503343 0.0350173 0.0106446 7
simulate:PDFastSim:PDFastSimPAR 100.021 126.027 203.595 116.923 32.857 7
simulate:PDFastSimExternal:PDFastSimPVS 3.21332 25.2748 152.8 4.22709 52.065 7
[art]:TriggerResults:TriggerResultInserter 1.629e-05 2.93336e-05 8.8202e-05 2.0401e-05 2.41676e-05 7
end_path:out1:RootOutput 4.95e-06 8.41e-06 2.379e-05 6.12e-06 6.2954e-06 7
end_path:out1:RootOutput(write) 29.1942 32.7557 36.167 33.3744 2.62714 6
================================================================================================================================
%MSG-i NuRandomService: RootOutput:out1@EndJob 27-Jun-2025 21:41:50 CEST ModuleEndJob
Summary of seeds computed by the NuRandomService
Random policy: 'perEvent'
algorithm version: EventTimestamp_v1
Configured value Last value ModuleLabel.InstanceName
(per event) 105889038 IonAndScint.ISCalcAlg
(per event) 885979438 IonAndScintExternal.ISCalcAlg
(per event) 149195452 PDFastSim.photon
(per event) 268840968 PDFastSim.scinttime
(per event) 699059928 PDFastSimExternal.photon
(per event) 450260862 PDFastSimExternal.scinttime
(per event) 481047338 largeant
%MSG
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 12664.5 MB
Peak resident set size usage (VmHWM): 11463.7 MB
====================================================================================================
TrigReport ---------- Event summary -------------
TrigReport Events total = 7 passed = 7 failed = 0
TrigReport ---------- Modules in End-path ----------
TrigReport Run Success Error Name
TrigReport 7 7 0 out1
TimeReport ---------- Time summary [sec] -------
TimeReport CPU = 1221.126907 Real = 1388.011194
MemReport ---------- Memory summary [base-10 MB] ------
MemReport VmPeak = 12664.5 VmHWM = 11463.7
%MSG-s ArtException: PostEndJob 27-Jun-2025 21:42:30 CEST ModuleEndJob
---- EventProcessorFailure BEGIN
EventProcessor: an exception occurred during current event processing
---- FatalRootError BEGIN
Fatal Root Error: TBufferFile::WriteByteCount
bytecount too large (more than 1073741822)
ROOT severity: 3000
---- FatalRootError END
---- EventProcessorFailure END
---- FatalRootError BEGIN
Fatal Root Error: TTree::SetEntries
Tree branches have different numbers of entries, eg EventAuxiliary has 6 entries while simb::MCTruths_cosmicgenerator__SinglesGen. has 10 entries.
ROOT severity: 2000
---- FatalRootError END
%MSG
Art has completed and will exit with status 1.
%MSG-i MF_INIT_OK: Early 27-Jun-2025 21:42:43 CEST JobSetup
Messagelogger initialization complete.
%MSG
Info in <TGeoManager::Import>: Reading geometry from file: /cvmfs/dune.opensciencegrid.org/products/dune/dunecore/v10_08_00d00/gdml/protodunehd_v6_refactored.gdml
Info in <TGeoManager::TGeoManager>: Geometry GDMLImport, Geometry imported from GDML created
Info in <TGeoManager::SetTopVolume>: Top volume is volWorld. Master volume is volWorld
Info in <TGeoNavigator::BuildCache>: --- Maximum geometry depth set to 100
Info in <TGeoManager::CheckGeometry>: Fixing runtime shapes...
Info in <TGeoManager::CheckGeometry>: ...Nothing to fix
Info in <TGeoManager::CloseGeometry>: Counting nodes...
Info in <TGeoManager::Voxelize>: Voxelizing...
Info in <TGeoManager::CloseGeometry>: Building cache...
Info in <TGeoManager::CountLevels>: max level = 5, max placements = 1148
Info in <TGeoManager::CloseGeometry>: 25610 nodes/ 5537 volume UID's in Geometry imported from GDML
Info in <TGeoManager::CloseGeometry>: ----------------modeler ready----------------
%MSG-i AuxDetGeometryCore: Early 27-Jun-2025 21:42:47 CEST JobSetup
New detector geometry loaded from
/cvmfs/dune.opensciencegrid.org/products/dune/dunecore/v10_08_00d00/gdml/protodunehd_v6_refactored.gdml
%MSG
%MSG-i GeometryCore: Early 27-Jun-2025 21:42:47 CEST JobSetup
Sorting volumes...
%MSG
%MSG-i GeometryCore: Early 27-Jun-2025 21:42:47 CEST JobSetup
New detector geometry loaded from
/cvmfs/dune.opensciencegrid.org/products/dune/dunecore/v10_08_00d00/gdml/protodunehd_v6_refactored.gdml
%MSG
%MSG-i DuneApaWireReadoutGeom: Early 27-Jun-2025 21:42:47 CEST JobSetup
Initializing wire readout...
%MSG
Cryostat 0:
10240 total channels
4 APAs
For all identical APA:
Number of channels per APA = 2560
U channels per APA = 800
V channels per APA = 800
Z channels per APA = 960
Pitch in U Plane = 0.4669
Pitch in V Plane = 0.4669
Pitch in Z Plane = 0.4792
%MSG-i SimpleChannelStatusService: Early 27-Jun-2025 21:42:47 CEST JobSetup
Loaded from configuration:
- 0 bad channels
- 0 noisy channels
- largest channel ID: 10239, largest present: 10239
%MSG
%MSG-i setupProvider<DetectorPropertiesStandard>: Early 27-Jun-2025 21:42:47 CEST JobSetup
Asked to ignore 1 keys: 'InheritNumberTimeSamples'
%MSG
%MSG-i LArG4DetectorService::Ctr: Early 27-Jun-2025 21:42:47 CEST JobSetup
Reading stepLimit(s) from the configuration file, for volume(s):
%MSG
%MSG-i LArG4DetectorService::Ctr: Early 27-Jun-2025 21:42:47 CEST JobSetup
Volume: volTPCActiveInner, stepLimit: 0.3
%MSG
%MSG-i LArG4DetectorService::Ctr: Early 27-Jun-2025 21:42:47 CEST JobSetup
Volume: volTPCActiveOuter, stepLimit: 0.3
%MSG
registering to primaryGeneratorActionsMap_
registering to eventActionsMap_
registering to trackingActionsMap_
registering to steppingActionsMap_
%MSG-i ParticleListActionService: Early 27-Jun-2025 21:42:47 CEST JobSetup
The full tracking information will not be stored for particles resulting from the following processes:
{ "conv" "LowEnConversion" "Pair" "compt" "Compt" "Brem" "phot" "Photo" "Ion" "annihil" }
%MSG
%MSG-i ParticleListActionService: Early 27-Jun-2025 21:42:47 CEST JobSetup
Trajectory sparsification enabled with SparsifyMargin : 0.015
%MSG
%MSG-i PhotonVisibilityService: Early 27-Jun-2025 21:42:47 CEST JobSetup
PhotonVisbilityService initializing
%MSG
%MSG-i SignalShapingServiceDUNE: Early 27-Jun-2025 21:42:47 CEST JobSetup
Getting Filter from .fcl file
%MSG
%MSG-i SignalShapingServiceDUNE: Early 27-Jun-2025 21:42:47 CEST JobSetup
using the field response provided from a .root file
%MSG
Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_U (Potential memory leak).
Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_V (Potential memory leak).
Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_Y (Potential memory leak).
Using custom SPE response taken from SPE_NP04_FBK_2024_without_pretrigger.dat
Using custom SPE response taken from SPE_NP04_HPK_2024_without_pretrigger.dat
%MSG-i RANDOM: OpDetDigitizerProtoDUNEHD:opdigi@Construction 27-Jun-2025 21:42:48 CEST ModuleConstruction
Instantiated HepJamesRandom engine "opdigi:0:" with seed 834182586.
%MSG
Generating waveforms of 5327.74us = 332983 Samples
TimeBegin: -2255.74
fTimeEnd: 3072
SampleFreq: 62.5 MHz
ReadoutWindow: 332983 ticks
PreTrigger: 0 ticks
SampleSize: 16 ns
Full streaming channels: 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159
SPE Templates:
SPE_NP04_FBK_2024_without_pretrigger.dat 4 14 24 34 40 42 45 46 47 49 50 52 55 56 57 59 60 62 65 66 67 69 70 72 75 76 77 79 84 85 86 87 94 95 96 97 104 105 106 107 114 115 116 117 120 121 124 125 127 129 130 131 134 135 137 139 140 141 144 145 147 149 150 151 154 155 157 159
SPE_NP04_HPK_2024_without_pretrigger.dat 0 1 2 3 5 6 7 8 9 10 11 12 13 15 16 17 18 19 20 21 22 23 25 26 27 28 29 30 31 32 33 35 36 37 38 39 41 43 44 48 51 53 54 58 61 63 64 68 71 73 74 78 80 81 82 83 88 89 90 91 92 93 98 99 100 101 102 103 108 109 110 111 112 113 118 119 122 123 126 128 132 133 136 138 142 143 146 148 152 153 156 158
Inputer: "wclsSimDepoSource:"
Outputer: "wclsSimChannelSink:postdrift"
Outputer: "wclsFrameSaver:simdigits"
wclsFrameSaver: promising to produce raw::RawDigits named "daq"
%MSG-s ArtException: TriggerResultInserter:TriggerResults@Construction 27-Jun-2025 21:43:04 CEST ModuleConstruction
cet::exception caught in art
---- CatalogServiceError BEGIN
Input file not found: cosmics_g4_.root.
---- CatalogServiceError END
%MSG
Art has completed and will exit with status 24.
lar exit code 24
pfn
outfile cosmics_detsim_.root