justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 257502.125@justin-prod-sched01.dune.hep.ac.uk

Jobsub ID257502.125@justin-prod-sched01.dune.hep.ac.uk
Workflow ID3262
Stage ID1
User nameamoor@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
RSS bytes5242880000 (5000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2024-09-16 13:43:02
SiteUS_FNAL-T1
EntryCMSHTPC_T1_US_FNAL_condce_opp1_whole
Last heartbeat2024-09-16 19:55:38
From worker nodeHostnamedunegli-34037-0-cmswn2301.fnal.gov
cpuinfoIntel(R) Xeon(R) CPU E5-2670 v3 @ 2.30GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes5242880000 (5000 MiB)
Wall seconds limit171000 (47 hours)
Inner Apptainer?True
Job statefinished
Allocator namejustin-allocator-pro.dune.hep.ac.uk
Started2024-09-16 14:00:45
Input filesusertests:000987_reco_data_2024-08-16T_163132Z_reco_data_2024-08-19T_111142Z_reco_data_2024-08-20T_124301Z_reco_data_2024-08-22T_092944Z.root
JobscriptExit code0
Real time5h (21260s)
CPU time5h (21103s = 99%)
Outputting started2024-09-16 19:55:07
Output filesusertests:000987_reco_data_2024-08-16T_163132Z_reco_data_2024-08-19T_111142Z_reco_data_2024-08-20T_124301Z_reco_data_2024-08-22T_092944Z_reco_data_2024-09-16T_140053Z.root
Finished2024-09-16 19:55:38
Saved logsjustin-logs:257502.125-justin-prod-sched01.dune.hep.ac.uk.logs.tgz
List job events     Wrapper job log

Jobscript log (last 10,000 characters)

AFM reco2 jobscript.
Input PFN = root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/usertests/d7/e0/000987_reco_data_2024-08-16T_163132Z_reco_data_2024-08-19T_111142Z_reco_data_2024-08-20T_124301Z_reco_data_2024-08-22T_092944Z.root
Setting up larsoft UPS area... /cvmfs/larsoft.opensciencegrid.org
Setting up DUNE UPS area... /cvmfs/dune.opensciencegrid.org/products/dune/
/cvmfs/larsoft.opensciencegrid.org/products/xrootd/v5_4_3b/Linux64bit+3.10-2.17-e20-p3913-prof/lib/libXrdPosixPreload.so
=== Start last 50 lines of lar log file ===
%MSG-e ShowerProducedPtrsHolder:   LArPandoraModularShowerCreation:pandoraShower@BeginModule  16-Sep-2024 19:55:01 UTC run: 20000031 subRun: 0 event: 909
Trying to add data product: InitialTrack. This element does not exist in the element holder
%MSG
%MSG-e ShowerProducedPtrsHolder:   LArPandoraModularShowerCreation:pandoraShower@BeginModule  16-Sep-2024 19:55:01 UTC run: 20000031 subRun: 0 event: 909
Trying to add data product: InitialTrack. This element does not exist in the element holder
%MSG
%MSG-e ShowerProducedPtrsHolder:   LArPandoraModularShowerCreation:pandoraShower@BeginModule  16-Sep-2024 19:55:01 UTC run: 20000031 subRun: 0 event: 909
Trying to add data product: InitialTrack. This element does not exist in the element holder
%MSG
%MSG-e ShowerProducedPtrsHolder:   LArPandoraModularShowerCreation:pandoraShower@BeginModule  16-Sep-2024 19:55:01 UTC run: 20000031 subRun: 0 event: 909
Trying to add data product: InitialTrack. This element does not exist in the element holder
%MSG
%MSG-e ShowerProducedPtrsHolder:   LArPandoraModularShowerCreation:pandoraShower@BeginModule  16-Sep-2024 19:55:01 UTC run: 20000031 subRun: 0 event: 909
Trying to add data product: InitialTrack. This element does not exist in the element holder
%MSG
%MSG-e ShowerProducedPtrsHolder:   LArPandoraModularShowerCreation:pandoraShower@BeginModule  16-Sep-2024 19:55:01 UTC run: 20000031 subRun: 0 event: 909
Trying to add data product: InitialTrack. This element does not exist in the element holder
%MSG
16-Sep-2024 19:55:02 UTC  Closed output file "000987_reco_data_2024-08-16T_163132Z_reco_data_2024-08-19T_111142Z_reco_data_2024-08-20T_124301Z_reco_data_2024-08-22T_092944Z_reco_data_2024-09-16T_140053Z.root"
16-Sep-2024 19:55:02 UTC  Closed input file "root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/usertests/d7/e0/000987_reco_data_2024-08-16T_163132Z_reco_data_2024-08-19T_111142Z_reco_data_2024-08-20T_124301Z_reco_data_2024-08-22T_092944Z.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                             0.0109889      1763.83       21144.2       1.19833       5843.4         12     
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.000702996   0.00224248     0.0150913    0.00103244    0.00388379       12     
reco:linecluster:LineCluster                          0.00200873      206.706       2479.3       0.0705076      685.213        12     
reco:trajcluster:TrajCluster                          0.000954997     243.256       2911.54      0.441309       804.52         12     
reco:pandora:StandardPandora                          0.000814229     1305.22       15653.5      0.260817       4326.18        12     
reco:pandoraTrack:LArPandoraTrackCreation             0.000108775     1.45068       16.288       0.0452284      4.4752         12     
reco:pandoraShower:LArPandoraModularShowerCreation    0.000154948    0.149418      0.500872      0.0722927     0.157523        12     
reco:pandoracalo:Calorimetry                          8.6913e-05     0.566838       6.16217      0.0340887      1.68811        12     
reco:pandorapid:Chi2ParticleID                        3.3807e-05    0.00144372     0.0109741    0.000499146   0.00291862       12     
reco:trkshowersplit:TrackShowerHits                   0.000136958    0.977967       11.464       0.0158788      3.1618         12     
reco:ophit:OpHitFinder                                0.00231378      5.36755       64.2972      0.0120233      17.768         12     
reco:opflash:OpFlashFinder                            0.000146258   0.00062296    0.00263468    0.000458487   0.000632858      12     
reco:rns:RandomNumberSaver                            1.5056e-05    2.79879e-05   0.000142601   1.70995e-05   3.46479e-05      12     
[art]:TriggerResults:TriggerResultInserter             8.727e-06    1.15362e-05    3.536e-05     9.283e-06    7.19511e-06      12     
end_path:out1:RootOutput                               2.478e-06    4.15742e-06   1.5298e-05     3.053e-06    3.42327e-06      12     
end_path:out1:RootOutput(write)                        0.0028185     0.138451       1.05898      0.0488563     0.280968        12     
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 3573.78 MB
  Peak resident set size usage (VmHWM): 2623.56 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 50 lines of lar log file ===
lar exit code 0
extractor_prod.py exit code 0
pdjson2metadata exit code 0
Metadata extraction succeeds
===Metadata JSON===
{
    "metadata": {
        "DUNE.campaign": "justIN.w3262s1",
        "art.file_format_era": "ART_2011a",
        "art.file_format_version": 15,
        "art.first_event": 56,
        "art.last_event": 909,
        "art.process_name": "Reco2",
        "art.run_type": "physics",
        "core.application": "art.reco",
        "core.application.family": "art",
        "core.application.name": "reco",
        "core.application.version": "v09_75_03d00",
        "core.data_stream": "out1",
        "core.data_tier": "full-reconstructed",
        "core.end_time": 1726516502.0,
        "core.event_count": 12,
        "core.file_format": "artroot",
        "core.file_type": "mc",
        "core.first_event_number": 56,
        "core.last_event_number": 909,
        "core.run_type": "physics",
        "core.runs": [
            20000031
        ],
        "core.runs_subruns": [
            2000003100000
        ],
        "core.start_time": 1726516458.0
    },
    "parents": [
        {
            "did": "usertests:000987_reco_data_2024-08-16T_163132Z_reco_data_2024-08-19T_111142Z_reco_data_2024-08-20T_124301Z_reco_data_2024-08-22T_092944Z.root"
        }
    ]
}
===================
.:
total 1168608
-rw-r--r-- 1 dunegli fnalgrid 1196331473 Sep 16 19:55 000987_reco_data_2024-08-16T_163132Z_reco_data_2024-08-19T_111142Z_reco_data_2024-08-20T_124301Z_reco_data_2024-08-22T_092944Z_reco_data_2024-09-16T_140053Z.root
-rw-r--r-- 1 dunegli fnalgrid     286062 Sep 16 19:55 000987_reco_data_2024-08-16T_163132Z_reco_data_2024-08-19T_111142Z_reco_data_2024-08-20T_124301Z_reco_data_2024-08-22T_092944Z_reco_2024-09-16T_140053Z.log
-rw-r--r-- 1 dunegli fnalgrid       7560 Sep 16 19:55 reco2_hist.root
-rw-r--r-- 1 dunegli fnalgrid       7140 Sep 16 19:55 jobscript.log
-rw-r--r-- 1 dunegli fnalgrid       1172 Sep 16 19:55 000987_reco_data_2024-08-16T_163132Z_reco_data_2024-08-19T_111142Z_reco_data_2024-08-20T_124301Z_reco_data_2024-08-22T_092944Z_reco_data_2024-09-16T_140053Z.root.json
-rw-r--r-- 1 dunegli fnalgrid       1047 Sep 16 19:55 000987_reco_data_2024-08-16T_163132Z_reco_data_2024-08-19T_111142Z_reco_data_2024-08-20T_124301Z_reco_data_2024-08-22T_092944Z_reco_data_2024-09-16T_140053Z.root.ext.json
-rw-r--r-- 1 dunegli fnalgrid        284 Sep 16 14:00 all-input-dids.txt
-rw-r--r-- 1 dunegli fnalgrid        218 Sep 16 19:55 justin-processed-pfns.txt
-rw-r--r-- 1 dunegli fnalgrid          0 Sep 16 14:01 debugprod.log
justIN time: 2024-11-17 07:13:16 UTC       justIN version: 01.01.09