justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 363010.45@justin-prod-sched01.dune.hep.ac.uk

Jobsub ID363010.45@justin-prod-sched01.dune.hep.ac.uk
Workflow ID6004
Stage ID1
User nameamoor@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
GPUNo
RSS bytes6291456000 (6000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2025-04-07 10:54:28
SiteUS_FNAL-FermiGrid
EntryFNAL_GPGrid_ce04_mcore_op_duneonly
Last heartbeat2025-04-07 11:50:37
From worker nodeHostnamedunegli-4620850-0-fnpc22036.fnal.gov
cpuinfoAMD EPYC 7543 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes6291456000 (6000 MiB)
Wall seconds limit172800 (48 hours)
GPU
Inner Apptainer?True
Job statefinished
Allocator namejustin-allocator-pro.dune.hep.ac.uk
Started2025-04-07 11:37:44
Input filesusertests:000493_reco_data_2025-04-02T_083324Z_reco_data_2025-04-02T_132810Z_reco_data_2025-04-04T_113506Z.root
JobscriptExit code0
Real time11m (701s)
CPU time9m (598s = 85%)
Max RSS bytes1200312320 (1144 MiB)
Outputting started2025-04-07 11:49:26
Output filesusertests:000493_reco_data_2025-04-02T_083324Z_reco_data_2025-04-02T_132810Z_reco_data_2025-04-04T_113506Z_reco_data_2025-04-07T_113747Z.root
Finished2025-04-07 11:50:37
Saved logsjustin-logs:363010.45-justin-prod-sched01.dune.hep.ac.uk.logs.tgz
List job events     Wrapper job log

Jobscript log (last 10,000 characters)

AFM reco1 jobscript.
Input PFN = root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/usertests/58/6a/000493_reco_data_2025-04-02T_083324Z_reco_data_2025-04-02T_132810Z_reco_data_2025-04-04T_113506Z.root
Setting up larsoft UPS area... /cvmfs/larsoft.opensciencegrid.org
Setting up DUNE UPS area... /cvmfs/dune.opensciencegrid.org/products/dune/
/cvmfs/larsoft.opensciencegrid.org/products/xrootd/v5_5_5a/Linux64bit+3.10-2.17-e26-p3915-prof/lib/libXrdPosixPreload.so
=== Start last 50 lines of lar log file ===
41 -1.19351e+07
42 -1.19517e+07
43 -1.19677e+07
44 -1.19834e+07
45 -1.19989e+07
46 -1.2014e+07
47 -1.20288e+07
48 -1.2043e+07
49 -1.20569e+07
50 -1.20702e+07
51 -1.20832e+07
52 -1.2096e+07
53 -1.21085e+07
54 -1.2121e+07
55 -1.21332e+07
56 -1.21452e+07
Begin processing the 992nd record. run: 20000031 subRun: 0 event: 992 at 07-Apr-2025 11:49:22 UTC
Begin processing the 993rd record. run: 20000031 subRun: 0 event: 993 at 07-Apr-2025 11:49:22 UTC
Begin processing the 994th record. run: 20000031 subRun: 0 event: 994 at 07-Apr-2025 11:49:23 UTC
Begin processing the 995th record. run: 20000031 subRun: 0 event: 995 at 07-Apr-2025 11:49:23 UTC
Begin processing the 996th record. run: 20000031 subRun: 0 event: 996 at 07-Apr-2025 11:49:23 UTC
Begin processing the 997th record. run: 20000031 subRun: 0 event: 997 at 07-Apr-2025 11:49:23 UTC
Begin processing the 998th record. run: 20000031 subRun: 0 event: 998 at 07-Apr-2025 11:49:23 UTC
Begin processing the 999th record. run: 20000031 subRun: 0 event: 999 at 07-Apr-2025 11:49:24 UTC
Begin processing the 1000th record. run: 20000031 subRun: 0 event: 1000 at 07-Apr-2025 11:49:24 UTC
07-Apr-2025 11:49:25 UTC  Closed output file "000493_reco_data_2025-04-02T_083324Z_reco_data_2025-04-02T_132810Z_reco_data_2025-04-04T_113506Z_reco_data_2025-04-07T_113747Z.root"
07-Apr-2025 11:49:25 UTC  Closed input file "root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/usertests/58/6a/000493_reco_data_2025-04-02T_083324Z_reco_data_2025-04-02T_132810Z_reco_data_2025-04-04T_113506Z.root"

================================================================================================================================
TimeTracker printout (sec)                        Min           Avg           Max         Median          RMS         nEvts   
================================================================================================================================
Full event                                     0.147541      0.519859        198.5       0.150947       6.61885       1000    
--------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                        0.000110328   0.00131649      0.28698     0.000187239    0.0102219      1000    
reco:gaushit:GausHitFinder                     0.146287      0.179952       1.90972      0.148989      0.0901521      1000    
reco:spsolve:SpacePointSolver                 5.6647e-05     0.325728       198.185     7.33535e-05     6.61253       1000    
reco:hitfd:DisambigFromSpacePoints            0.000230896    0.0044262     0.134879     0.000322605    0.0115091      1000    
reco:rns:RandomNumberSaver                    1.0359e-05    1.54267e-05   0.000147239   1.4958e-05    4.48456e-06     1000    
[art]:TriggerResults:TriggerResultInserter     5.58e-06     7.79049e-06    3.732e-05     7.645e-06     1.35e-06       1000    
end_path:out1:RootOutput                       1.984e-06    3.38453e-06   1.7262e-05     3.276e-06     8.954e-07      1000    
end_path:out1:RootOutput(write)               0.000499053   0.00821784     0.0989553    0.000695214    0.0148345      1000    
================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 2192.81 MB
  Peak resident set size usage (VmHWM): 1200.31 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 50 lines of lar log file ===
lar exit code 0
extractor_prod.py exit code 0
pdjson2metadata exit code 0
Metadata extraction succeeds
===Metadata JSON===
{
    "metadata": {
        "DUNE.campaign": "justIN.w6004s1",
        "art.file_format_era": "ART_2011a",
        "art.file_format_version": 15,
        "art.first_event": 1,
        "art.last_event": 1000,
        "art.process_name": "Reco1",
        "art.run_type": "physics",
        "core.application": "art.reco",
        "core.application.family": "art",
        "core.application.name": "reco",
        "core.application.version": "v10_04_06d00",
        "core.data_stream": "out1",
        "core.data_tier": "hit-reconstructed",
        "core.end_time": 1744026564.0,
        "core.event_count": 1000,
        "core.file_format": "artroot",
        "core.file_type": "mc",
        "core.first_event_number": 1,
        "core.last_event_number": 1000,
        "core.run_type": "physics",
        "core.runs": [
            20000031
        ],
        "core.runs_subruns": [
            2000003100000
        ],
        "core.start_time": 1744025888.0
    },
    "parents": [
        {
            "did": "usertests:000493_reco_data_2025-04-02T_083324Z_reco_data_2025-04-02T_132810Z_reco_data_2025-04-04T_113506Z.root"
        }
    ]
}
===================
.:
total 7280592
-rw-r--r-- 1 dunegli fnalgrid 7454975172 Apr  7 11:49 000493_reco_data_2025-04-02T_083324Z_reco_data_2025-04-02T_132810Z_reco_data_2025-04-04T_113506Z_reco_data_2025-04-07T_113747Z.root
-rw-r--r-- 1 dunegli fnalgrid     317195 Apr  7 11:49 000493_reco_data_2025-04-02T_083324Z_reco_data_2025-04-02T_132810Z_reco_data_2025-04-04T_113506Z_reco_2025-04-07T_113747Z.log
-rw-r--r-- 1 dunegli fnalgrid       5594 Apr  7 11:49 jobscript.log
-rw-r--r-- 1 dunegli fnalgrid       1143 Apr  7 11:49 000493_reco_data_2025-04-02T_083324Z_reco_data_2025-04-02T_132810Z_reco_data_2025-04-04T_113506Z_reco_data_2025-04-07T_113747Z.root.json
-rw-r--r-- 1 dunegli fnalgrid        988 Apr  7 11:49 000493_reco_data_2025-04-02T_083324Z_reco_data_2025-04-02T_132810Z_reco_data_2025-04-04T_113506Z_reco_data_2025-04-07T_113747Z.root.ext.json
-rw-r--r-- 1 dunegli fnalgrid        519 Apr  7 11:49 reco1_hist.root
-rw-r--r-- 1 dunegli fnalgrid        224 Apr  7 11:37 all-input-dids.txt
-rw-r--r-- 1 dunegli fnalgrid        188 Apr  7 11:49 justin-processed-pfns.txt
-rw-r--r-- 1 dunegli fnalgrid          0 Apr  7 11:37 debugprod.log
justIN time: 2025-05-15 04:24:21 UTC       justIN version: 01.03.00