justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 103569.1@justin-prod-sched02.dune.hep.ac.uk

Jobsub ID103569.1@justin-prod-sched02.dune.hep.ac.uk
Workflow ID4144
Stage ID1
User namegvittist@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit18000 (5 hours)
Submitted time2024-11-19 07:54:55
SiteFR_CCIN2P3
EntryDUNE_FR_CCIN2P3_cccondorce01
Last heartbeat2024-11-19 08:10:12
From worker nodeHostnameccwcondor0362
cpuinfoAMD EPYC 9334 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit106200 (29 hours)
Inner Apptainer?True
Job stateoutputting_failed
Allocator namejustin-allocator-pro.dune.hep.ac.uk
Started2024-11-19 07:55:58
Input fileshd-protodune:np04hd_raw_run029233_0109_dataflow1_datawriter_0_20240920T164530.hdf5
JobscriptExit code0
Real time13m (836s)
CPU time12m (754s = 90%)
Outputting started2024-11-19 08:09:54
Output files
Finished2024-11-19 08:10:12
List job events     Wrapper job log

Jobscript log (last 10,000 characters)

Input PFN = root://eospublic.cern.ch:1094//eos/experiment/neutplatform/protodune/dune/hd-protodune/86/da/np04hd_raw_run029233_0109_dataflow1_datawriter_0_20240920T164530.hdf5
Setting up larsoft UPS area... /cvmfs/larsoft.opensciencegrid.org
Setting up DUNE UPS area... /cvmfs/dune.opensciencegrid.org/products/dune/
/cvmfs/larsoft.opensciencegrid.org/products/xrootd/v5_5_5a/Linux64bit+3.10-2.17-e26-p3915-prof/lib/libXrdPosixPreload.so
=== Start last 100 lines of lar log file ===
RawFrameSource: got 10240 raw::RawDigit objects
	input nticks=5859 keeping as is
Retagger: tagging trace set: raw with 10240 traces, 0 summary
wclsFrameSaver: saving raw::RawDigits tagged "raw"
wclsFrameSaver: found empty channel masks for "bad"
WARNING: Could not find offline channel for 9 0 32
WARNING: Could not find offline channel for 9 0 32
WARNING: Could not find offline channel for 9 0 32
WARNING: Could not find offline channel for 9 0 32
WARNING: Could not find offline channel for 9 0 32
WARNING: Could not find offline channel for 9 0 32
WARNING: Could not find offline channel for 9 0 32
WARNING: Could not find offline channel for 9 0 32
WARNING: Could not find offline channel for 9 0 32
WARNING: Could not find offline channel for 9 0 32
WARNING: Could not find offline channel for 9 0 32
WARNING: Could not find offline channel for 9 0 32
WARNING: Could not find offline channel for 9 0 32
WARNING: Could not find offline channel for 9 0 32
RawFrameSource: got 10240 raw::RawDigit objects
	input nticks=5859 keeping as is
Retagger: tagging trace set: raw with 10240 traces, 0 summary
wclsFrameSaver: saving raw::RawDigits tagged "raw"
wclsFrameSaver: found empty channel masks for "bad"
WARNING: Could not find offline channel for 9 0 32
WARNING: Could not find offline channel for 9 0 32
WARNING: Could not find offline channel for 9 0 32
WARNING: Could not find offline channel for 9 0 32
WARNING: Could not find offline channel for 9 0 32
WARNING: Could not find offline channel for 9 0 32
WARNING: Could not find offline channel for 9 0 32
WARNING: Could not find offline channel for 9 0 32
WARNING: Could not find offline channel for 9 0 32
WARNING: Could not find offline channel for 9 0 32
WARNING: Could not find offline channel for 9 0 32
WARNING: Could not find offline channel for 9 0 32
WARNING: Could not find offline channel for 9 0 32
WARNING: Could not find offline channel for 9 0 32
WARNING: Could not find offline channel for 9 0 32
WARNING: Could not find offline channel for 9 0 32
RawFrameSource: got 10240 raw::RawDigit objects
	input nticks=5859 keeping as is
Retagger: tagging trace set: raw with 10240 traces, 0 summary
wclsFrameSaver: saving raw::RawDigits tagged "raw"
wclsFrameSaver: found empty channel masks for "bad"
WARNING: Could not find offline channel for 9 0 32
WARNING: Could not find offline channel for 9 0 32
WARNING: Could not find offline channel for 9 0 32
WARNING: Could not find offline channel for 9 0 32
WARNING: Could not find offline channel for 9 0 32
WARNING: Could not find offline channel for 9 0 32
WARNING: Could not find offline channel for 9 0 32
WARNING: Could not find offline channel for 9 0 32
WARNING: Could not find offline channel for 9 0 32
WARNING: Could not find offline channel for 9 0 32
WARNING: Could not find offline channel for 9 0 32
WARNING: Could not find offline channel for 9 0 32
WARNING: Could not find offline channel for 9 0 32
WARNING: Could not find offline channel for 9 0 32
WARNING: Could not find offline channel for 9 0 32
WARNING: Could not find offline channel for 9 0 32
WARNING: Could not find offline channel for 9 0 32
WARNING: Could not find offline channel for 9 0 32
WARNING: Could not find offline channel for 9 0 32
WARNING: Could not find offline channel for 9 0 32
WARNING: Could not find offline channel for 9 0 32
WARNING: Could not find offline channel for 9 0 32
WARNING: Could not find offline channel for 9 0 32
RawFrameSource: got 10240 raw::RawDigit objects
	input nticks=5859 keeping as is
Retagger: tagging trace set: raw with 10240 traces, 0 summary
wclsFrameSaver: saving raw::RawDigits tagged "raw"
wclsFrameSaver: found empty channel masks for "bad"
19-Nov-2024 09:09:53 CET  Closed output file "np04hd_raw_run029233_0109_dataflow1_datawriter_0_20240920T164530_reco_2024-10-30T_075601Z.root"

===================================================================================================================================
TimeTracker printout (sec)                           Min           Avg           Max         Median          RMS         nEvts   
===================================================================================================================================
Full event                                         22.7958       25.9664       28.1901       26.3168       1.41237        31     
-----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read)                       2.5859e-05    4.9087e-05    0.000223759   4.4097e-05    3.29674e-05      31     
produce:tpcrawdecoder:PDHDTPCReader                3.15165       3.43502       3.93695       3.4201       0.192071        31     
produce:triggerrawdecoder:PDHDTriggerReader3      0.0751456     0.0804687     0.0894459     0.0800512    0.00316336       31     
produce:timingrawdecoder:PDHDTimingRawDecoder    0.00285756    0.00299082    0.00332246     0.002993     0.000103995      31     
produce:pdhddaphne:DAPHNEReaderPDHD                4.08224       5.07836       5.50355       5.20803       0.3665         31     
produce:fembfilter:PDHDFEMBFilter                8.8104e-05    0.000246778   0.000580518   0.000311369   0.000121434      31     
produce:wclsdatahdfilter:WireCellToolkit           10.7327       12.8477       14.2332       13.1592      0.973155        31     
[art]:TriggerResults:TriggerResultInserter        2.615e-05    3.9634e-05    9.6586e-05    4.0592e-05    1.2191e-05       31     
end_path:out1:RootOutput                          3.816e-06    8.72635e-06   2.4327e-05     8.603e-06    3.12935e-06      31     
end_path:out1:RootOutput(write)                    3.65513       4.52015       4.98498       4.7037       0.427945        31     
===================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 3285.56 MB
  Peak resident set size usage (VmHWM): 1866.34 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
lar exit code 0
justIN time: 2024-11-24 04:30:44 UTC       justIN version: 01.01.09