justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 272773.26@justin-prod-sched01.dune.hep.ac.uk

Jobsub ID272773.26@justin-prod-sched01.dune.hep.ac.uk
Workflow ID3662
Stage ID1
User namegvittist@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit18000 (5 hours)
Submitted time2024-10-10 07:40:44
SiteCERN
EntryCMSHTPC_T2_CH_CERN_ce508
Last heartbeat2024-10-10 09:31:35
From worker nodeHostnameb9p19p6242.cern.ch
cpuinfoIntel(R) Xeon(R) Silver 4216 CPU @ 2.10GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit171000 (47 hours)
Inner Apptainer?True
Job stateoutputting_failed
Allocator namejustin-allocator-pro.dune.hep.ac.uk
Started2024-10-10 07:42:08
Input fileshd-protodune:np04hd_raw_run029233_0015_dataflow1_datawriter_0_20240920T003408.hdf5
JobscriptExit code0
Real time19m (1150s)
CPU time17m (1077s = 93%)
Outputting started2024-10-10 08:01:18
Output files
Finished2024-10-10 09:31:35
List job events     Wrapper job log

Jobscript log (last 10,000 characters)

Input PFN = root://eospublic.cern.ch:1094//eos/experiment/neutplatform/protodune/dune/hd-protodune/c0/23/np04hd_raw_run029233_0015_dataflow1_datawriter_0_20240920T003408.hdf5
Setting up larsoft UPS area... /cvmfs/larsoft.opensciencegrid.org
Setting up DUNE UPS area... /cvmfs/dune.opensciencegrid.org/products/dune/
/cvmfs/larsoft.opensciencegrid.org/products/xrootd/v5_5_5a/Linux64bit+3.10-2.17-e26-p3915-prof/lib/libXrdPosixPreload.so
=== Start last 100 lines of lar log file ===
WARNING: Could not find offline channel for 9 0 32
WARNING: Could not find offline channel for 9 0 32
WARNING: Could not find offline channel for 9 0 32
WARNING: Could not find offline channel for 9 0 32
WARNING: Could not find offline channel for 9 0 32
WARNING: Could not find offline channel for 9 0 32
RawFrameSource: got 10240 raw::RawDigit objects
	input nticks=5859 keeping as is
Retagger: tagging trace set: raw with 10240 traces, 0 summary
wclsFrameSaver: saving raw::RawDigits tagged "raw"
wclsFrameSaver: found empty channel masks for "bad"
WARNING: Could not find offline channel for 9 0 32
WARNING: Could not find offline channel for 9 0 32
WARNING: Could not find offline channel for 9 0 32
WARNING: Could not find offline channel for 9 0 32
WARNING: Could not find offline channel for 9 0 32
WARNING: Could not find offline channel for 9 0 32
WARNING: Could not find offline channel for 9 0 32
WARNING: Could not find offline channel for 9 0 32
WARNING: Could not find offline channel for 9 0 32
WARNING: Could not find offline channel for 9 0 32
WARNING: Could not find offline channel for 9 0 32
WARNING: Could not find offline channel for 9 0 32
WARNING: Could not find offline channel for 9 0 32
WARNING: Could not find offline channel for 9 0 32
WARNING: Could not find offline channel for 9 0 32
WARNING: Could not find offline channel for 9 0 32
WARNING: Could not find offline channel for 9 0 32
WARNING: Could not find offline channel for 9 0 32
WARNING: Could not find offline channel for 9 0 32
RawFrameSource: got 10240 raw::RawDigit objects
	input nticks=5859 keeping as is
Retagger: tagging trace set: raw with 10240 traces, 0 summary
wclsFrameSaver: saving raw::RawDigits tagged "raw"
wclsFrameSaver: found empty channel masks for "bad"
WARNING: Could not find offline channel for 9 0 32
WARNING: Could not find offline channel for 9 0 32
WARNING: Could not find offline channel for 9 0 32
WARNING: Could not find offline channel for 9 0 32
WARNING: Could not find offline channel for 9 0 32
WARNING: Could not find offline channel for 9 0 32
WARNING: Could not find offline channel for 9 0 32
WARNING: Could not find offline channel for 9 0 32
WARNING: Could not find offline channel for 9 0 32
WARNING: Could not find offline channel for 9 0 32
WARNING: Could not find offline channel for 9 0 32
WARNING: Could not find offline channel for 9 0 32
WARNING: Could not find offline channel for 9 0 32
RawFrameSource: got 10240 raw::RawDigit objects
	input nticks=5859 keeping as is
Retagger: tagging trace set: raw with 10240 traces, 0 summary
wclsFrameSaver: saving raw::RawDigits tagged "raw"
wclsFrameSaver: found empty channel masks for "bad"
WARNING: Could not find offline channel for 9 0 32
WARNING: Could not find offline channel for 9 0 32
WARNING: Could not find offline channel for 9 0 32
WARNING: Could not find offline channel for 9 0 32
WARNING: Could not find offline channel for 9 0 32
WARNING: Could not find offline channel for 9 0 32
WARNING: Could not find offline channel for 9 0 32
WARNING: Could not find offline channel for 9 0 32
WARNING: Could not find offline channel for 9 0 32
WARNING: Could not find offline channel for 9 0 32
WARNING: Could not find offline channel for 9 0 32
WARNING: Could not find offline channel for 9 0 32
WARNING: Could not find offline channel for 9 0 32
WARNING: Could not find offline channel for 9 0 32
WARNING: Could not find offline channel for 9 0 32
RawFrameSource: got 10240 raw::RawDigit objects
	input nticks=5859 keeping as is
Retagger: tagging trace set: raw with 10240 traces, 0 summary
wclsFrameSaver: saving raw::RawDigits tagged "raw"
wclsFrameSaver: found empty channel masks for "bad"
10-Oct-2024 10:01:18 CEST  Closed output file "np04hd_raw_run029233_0015_dataflow1_datawriter_0_20240920T003408_reco_2024-10-10T_074213Z.root"

===================================================================================================================================
TimeTracker printout (sec)                           Min           Avg           Max         Median          RMS         nEvts   
===================================================================================================================================
Full event                                         31.9995       35.6825       42.6093       34.945        2.84017        31     
-----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read)                       2.2495e-05    3.59216e-05   0.000154847   2.9231e-05    2.30737e-05      31     
produce:tpcrawdecoder:PDHDTPCReader                3.0649        3.6864        4.46257       3.64523      0.384066        31     
produce:triggerrawdecoder:PDHDTriggerReader3     0.00929593     0.0156084     0.048299      0.0120483    0.00914747       31     
produce:timingrawdecoder:PDHDTimingRawDecoder    0.000357121   0.000520994   0.000759452   0.00050291    9.30789e-05      31     
produce:pdhddaphne:DAPHNEReaderPDHD                9.49837       10.8809       13.1456       10.4531       1.03635        31     
produce:fembfilter:PDHDFEMBFilter                 0.0001016    0.000191922   0.000946589   0.000123885   0.000158474      31     
produce:wclsdatahdfilter:WireCellToolkit           14.3059       15.8157       19.3921       15.1525       1.54435        31     
[art]:TriggerResults:TriggerResultInserter       1.8876e-05     3.189e-05    0.000105501   2.9689e-05    1.44754e-05      31     
end_path:out1:RootOutput                          3.215e-06    5.95774e-06   2.4841e-05     5.647e-06    3.64576e-06      31     
end_path:out1:RootOutput(write)                    4.78001       5.28199       6.67176       5.15691      0.397448        31     
===================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 3289.04 MB
  Peak resident set size usage (VmHWM): 1811.35 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
lar exit code 0
justIN time: 2024-11-17 10:40:38 UTC       justIN version: 01.01.09