justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 272741.0@justin-prod-sched01.dune.hep.ac.uk

Jobsub ID272741.0@justin-prod-sched01.dune.hep.ac.uk
Workflow ID3662
Stage ID1
User namegvittist@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit18000 (5 hours)
Submitted time2024-10-10 05:58:17
SiteUK_RAL-Tier1
EntryLIGO_UK_RAL_arc_ce03
Last heartbeat2024-10-10 07:53:55
From worker nodeHostnamedune001-2627870.0-lcg2574.gridpp.rl.ac.uk
cpuinfoAMD EPYC 7452 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit216000 (60 hours)
Inner Apptainer?True
Job stateoutputting_failed
Allocator namejustin-allocator-pro.dune.hep.ac.uk
Started2024-10-10 06:00:37
Input fileshd-protodune:np04hd_raw_run029230_0031_dataflow1_datawriter_0_20240919T210548.hdf5
JobscriptExit code0
Real time22m (1377s)
CPU time16m (994s = 72%)
Outputting started2024-10-10 06:23:35
Output files
Finished2024-10-10 07:53:55
List job events     Wrapper job log

Jobscript log (last 10,000 characters)

Input PFN = root://eospublic.cern.ch:1094//eos/experiment/neutplatform/protodune/dune/hd-protodune/ae/a2/np04hd_raw_run029230_0031_dataflow1_datawriter_0_20240919T210548.hdf5
Setting up larsoft UPS area... /cvmfs/larsoft.opensciencegrid.org
Setting up DUNE UPS area... /cvmfs/dune.opensciencegrid.org/products/dune/
/cvmfs/larsoft.opensciencegrid.org/products/xrootd/v5_5_5a/Linux64bit+3.10-2.17-e26-p3915-prof/lib/libXrdPosixPreload.so
=== Start last 100 lines of lar log file ===
WARNING: Could not find offline channel for 9 0 32
WARNING: Could not find offline channel for 9 0 32
WARNING: Could not find offline channel for 9 0 32
WARNING: Could not find offline channel for 9 0 32
RawFrameSource: got 10240 raw::RawDigit objects
	input nticks=5859 keeping as is
Retagger: tagging trace set: raw with 10240 traces, 0 summary
wclsFrameSaver: saving raw::RawDigits tagged "raw"
wclsFrameSaver: found empty channel masks for "bad"
WARNING: Could not find offline channel for 9 0 32
WARNING: Could not find offline channel for 9 0 32
WARNING: Could not find offline channel for 9 0 32
WARNING: Could not find offline channel for 9 0 32
WARNING: Could not find offline channel for 9 0 32
WARNING: Could not find offline channel for 9 0 32
WARNING: Could not find offline channel for 9 0 32
WARNING: Could not find offline channel for 9 0 32
WARNING: Could not find offline channel for 9 0 32
WARNING: Could not find offline channel for 9 0 32
WARNING: Could not find offline channel for 9 0 32
WARNING: Could not find offline channel for 9 0 32
WARNING: Could not find offline channel for 9 0 32
WARNING: Could not find offline channel for 9 0 32
WARNING: Could not find offline channel for 9 0 32
WARNING: Could not find offline channel for 9 0 32
RawFrameSource: got 10240 raw::RawDigit objects
	input nticks=5859 keeping as is
Retagger: tagging trace set: raw with 10240 traces, 0 summary
wclsFrameSaver: saving raw::RawDigits tagged "raw"
wclsFrameSaver: found empty channel masks for "bad"
WARNING: Could not find offline channel for 9 0 32
WARNING: Could not find offline channel for 9 0 32
WARNING: Could not find offline channel for 9 0 32
WARNING: Could not find offline channel for 9 0 32
WARNING: Could not find offline channel for 9 0 32
WARNING: Could not find offline channel for 9 0 32
WARNING: Could not find offline channel for 9 0 32
WARNING: Could not find offline channel for 9 0 32
WARNING: Could not find offline channel for 9 0 32
WARNING: Could not find offline channel for 9 0 32
WARNING: Could not find offline channel for 9 0 32
WARNING: Could not find offline channel for 9 0 32
WARNING: Could not find offline channel for 9 0 32
WARNING: Could not find offline channel for 9 0 32
WARNING: Could not find offline channel for 9 0 32
WARNING: Could not find offline channel for 9 0 32
RawFrameSource: got 10240 raw::RawDigit objects
	input nticks=5859 keeping as is
Retagger: tagging trace set: raw with 10240 traces, 0 summary
wclsFrameSaver: saving raw::RawDigits tagged "raw"
wclsFrameSaver: found empty channel masks for "bad"
WARNING: Could not find offline channel for 9 0 32
WARNING: Could not find offline channel for 9 0 32
WARNING: Could not find offline channel for 9 0 32
WARNING: Could not find offline channel for 9 0 32
WARNING: Could not find offline channel for 9 0 32
WARNING: Could not find offline channel for 9 0 32
WARNING: Could not find offline channel for 9 0 32
WARNING: Could not find offline channel for 9 0 32
WARNING: Could not find offline channel for 9 0 32
WARNING: Could not find offline channel for 9 0 32
WARNING: Could not find offline channel for 9 0 32
WARNING: Could not find offline channel for 9 0 32
WARNING: Could not find offline channel for 9 0 32
WARNING: Could not find offline channel for 9 0 32
WARNING: Could not find offline channel for 9 0 32
WARNING: Could not find offline channel for 9 0 32
WARNING: Could not find offline channel for 9 0 32
RawFrameSource: got 10240 raw::RawDigit objects
	input nticks=5859 keeping as is
Retagger: tagging trace set: raw with 10240 traces, 0 summary
wclsFrameSaver: saving raw::RawDigits tagged "raw"
wclsFrameSaver: found empty channel masks for "bad"
10-Oct-2024 06:23:34 UTC  Closed output file "np04hd_raw_run029230_0031_dataflow1_datawriter_0_20240919T210548_reco_2024-10-10T_060044Z.root"

===================================================================================================================================
TimeTracker printout (sec)                           Min           Avg           Max         Median          RMS         nEvts   
===================================================================================================================================
Full event                                         37.5588       42.4845       46.0072       43.1503       2.30989        31     
-----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read)                       3.0951e-05    5.49238e-05   0.000253412    4.737e-05    3.70918e-05      31     
produce:tpcrawdecoder:PDHDTPCReader                11.1244       11.646        12.2799       11.6319      0.308626        31     
produce:triggerrawdecoder:PDHDTriggerReader3      0.618349      0.637785       0.6784        0.63918      0.015596        31     
produce:timingrawdecoder:PDHDTimingRawDecoder     0.0230142     0.0298861     0.0622688     0.023396     0.00901483       31     
produce:pdhddaphne:DAPHNEReaderPDHD                7.09031       7.69157       8.13765       7.72484      0.273179        31     
produce:fembfilter:PDHDFEMBFilter                0.000112892   0.000382496   0.000566905   0.000438534   0.000142395      31     
produce:wclsdatahdfilter:WireCellToolkit           13.6586       16.9199       18.7163       17.6407       1.53524        31     
[art]:TriggerResults:TriggerResultInserter        1.92e-05     4.64093e-05   0.000154961   4.4511e-05    2.18294e-05      31     
end_path:out1:RootOutput                          2.95e-06     4.75452e-06    3.104e-05     3.89e-06     4.81779e-06      31     
end_path:out1:RootOutput(write)                    4.40483       5.55717       6.48447       6.05835      0.762112        31     
===================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 3267.68 MB
  Peak resident set size usage (VmHWM): 1746.61 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
lar exit code 0
justIN time: 2024-11-17 10:39:25 UTC       justIN version: 01.01.09