justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 75257.0@justin-prod-sched02.dune.hep.ac.uk

Jobsub ID75257.0@justin-prod-sched02.dune.hep.ac.uk
Workflow ID3317
Stage ID1
User nameamoor@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2024-09-20 12:15:50
SiteUS_UConn-HPC
EntryGLUEX_US_UConn-HPC_osgce
Last heartbeat2024-09-20 12:19:23
From worker nodeHostnamecn411
cpuinfoAMD EPYC 7452 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit172800 (48 hours)
Inner Apptainer?True
Job stateaborted
Allocator namejustin-allocator-pro.dune.hep.ac.uk
Started2024-09-20 12:17:18
Input filesusertests:000396_reco_data_2024-08-16T_162233Z_reco_data_2024-08-19T_140214Z_reco_data_2024-08-20T_115338Z_reco_data_2024-08-22T_092928Z_reco_data_2024-08-23T_100636Z.root
Outputting started2024-09-20 12:18:57
Output files
Finished2024-09-20 12:19:23
List job events     Wrapper job log

Jobscript log (last 10,000 characters)

<TFile::Append>: Replacing existing TH1: FieldResponse_U (Potential memory leak).
Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_V (Potential memory leak).
Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_Y (Potential memory leak).
Active Boundaries: 
	x: -363.376 to 363.376
	y: -607.829 to 607.829
	z: -0.87625 to 1393.46
20-Sep-2024 08:17:57 EDT  Initiating request to open input file "root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/usertests/54/0d/000396_reco_data_2024-08-16T_162233Z_reco_data_2024-08-19T_140214Z_reco_data_2024-08-20T_115338Z_reco_data_2024-08-22T_092928Z_reco_data_2024-08-23T_100636Z.root"
Plugin version SecClnt v5.4.3 is incompatible with secztn v5.6.8 (must be <= 5.4.x) in sec.protocol libXrdSecztn-5.so
20-Sep-2024 08:18:02 EDT  Opened input file "root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/usertests/54/0d/000396_reco_data_2024-08-16T_162233Z_reco_data_2024-08-19T_140214Z_reco_data_2024-08-20T_115338Z_reco_data_2024-08-22T_092928Z_reco_data_2024-08-23T_100636Z.root"
job begin...
Fiducial volume:
-353.376	< x <	353.376
-597.829	< y <	597.829
9.12375	< z <	1383.46
job begin...
Fiducial volume:
-353.376	< x <	353.376
-597.829	< y <	597.829
9.12375	< z <	1383.46
Begin processing the 1st record. run: 20000031 subRun: 0 event: 74 at 20-Sep-2024 08:18:04 EDT
Analysing.

Warning: there was no track found for track-like PFParticle with ID 0
Warning: there was no track found for track-like PFParticle with ID 1
Warning: there was no track found for track-like PFParticle with ID 2
Warning: there was no shower found for shower-like PFParticle with ID 3
Warning: there was no track found for track-like PFParticle with ID 4
Warning: there was no track found for track-like PFParticle with ID 5
Warning: there was no shower found for shower-like PFParticle with ID 6
20-Sep-2024 08:18:08 EDT  Opened output file with pattern "000396_reco_data_2024-08-16T_162233Z_reco_data_2024-08-19T_140214Z_reco_data_2024-08-20T_115338Z_reco_data_2024-08-22T_092928Z_reco_data_2024-08-23T_100636Z_reco_data_2024-09-20T_121730Z.root"
Begin processing the 2nd record. run: 20000031 subRun: 0 event: 110 at 20-Sep-2024 08:18:24 EDT
Analysing.

Warning: there was no track found for track-like PFParticle with ID 0
Warning: there was no shower found for shower-like PFParticle with ID 1
Warning: there was no track found for track-like PFParticle with ID 2
Warning: there was no shower found for shower-like PFParticle with ID 3
Warning: there was no shower found for shower-like PFParticle with ID 4
Warning: there was no track found for track-like PFParticle with ID 5
Warning: there was no shower found for shower-like PFParticle with ID 6
Warning: there was no track found for track-like PFParticle with ID 7
Warning: there was no track found for track-like PFParticle with ID 8
Warning: there was no shower found for shower-like PFParticle with ID 9
Warning: there was no track found for track-like PFParticle with ID 10
Warning: there was no shower found for shower-like PFParticle with ID 11
Warning: there was no track found for track-like PFParticle with ID 12
Warning: there was no shower found for shower-like PFParticle with ID 13
Warning: there was no track found for track-like PFParticle with ID 14
Warning: there was no shower found for shower-like PFParticle with ID 15
Warning: there was no track found for track-like PFParticle with ID 16
Warning: there was no track found for track-like PFParticle with ID 17
Warning: there was no track found for track-like PFParticle with ID 18
Warning: there was no shower found for shower-like PFParticle with ID 19
Warning: there was no track found for track-like PFParticle with ID 20
Warning: there was no track found for track-like PFParticle with ID 21
Warning: there was no track found for track-like PFParticle with ID 22
Warning: there was no track found for track-like PFParticle with ID 23
Warning: there was no vertex found for PFParticle with ID 24
Warning: there was no shower found for shower-like PFParticle with ID 24
Warning: there was no shower found for shower-like PFParticle with ID 25
Warning: there was no shower found for shower-like PFParticle with ID 26
Warning: there was no shower found for shower-like PFParticle with ID 27
Warning: there was no shower found for shower-like PFParticle with ID 28
Warning: there was no shower found for shower-like PFParticle with ID 29
Warning: there was no shower found for shower-like PFParticle with ID 30
20-Sep-2024 08:18:47 EDT  Closed output file "000396_reco_data_2024-08-16T_162233Z_reco_data_2024-08-19T_140214Z_reco_data_2024-08-20T_115338Z_reco_data_2024-08-22T_092928Z_reco_data_2024-08-23T_100636Z_reco_data_2024-09-20T_121730Z.root"
20-Sep-2024 08:18:47 EDT  Closed input file "root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/usertests/54/0d/000396_reco_data_2024-08-16T_162233Z_reco_data_2024-08-19T_140214Z_reco_data_2024-08-20T_115338Z_reco_data_2024-08-22T_092928Z_reco_data_2024-08-23T_100636Z.root"

==============================================================================================================================
TimeTracker printout (sec)                      Min           Avg           Max         Median          RMS         nEvts   
==============================================================================================================================
Full event                                    17.7298       19.8104       21.8909       19.8104       2.08052         2     
------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                       0.0267236     0.0449408     0.0631581     0.0449408     0.0182173        2     
end_path:out1:RootOutput                    1.0139e-05    7.5828e-05    0.000141517   7.5828e-05    6.5689e-05        2     
end_path:analysistree:AnalysisTree            1.68378       1.91135       2.13892       1.91135       0.22757         2     
end_path:pandoraeff:NeutrinoTrackingEff     0.000419041   0.00182798    0.00323691    0.00182798    0.00140894        2     
end_path:pandorashweff:NeutrinoShowerEff    6.4582e-05    0.00224195    0.00441932    0.00224195    0.00217737        2     
end_path:runcheckereco:CheckRecoEnergy      1.7653e-05    0.000321401   0.00062515    0.000321401   0.000303749       2     
end_path:flashmatchAna:FlashMatchAna         0.189351      0.276809      0.364268      0.276809      0.0874583        2     
end_path:out1:RootOutput(write)               15.8203       17.572        19.3236       17.572        1.75166         2     
==============================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 2698.87 MB
  Peak resident set size usage (VmHWM): 1514.74 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 200 lines of lar log file ===
lar exit code 0
extractor_prod.py exit code 0
pdjson2metadata exit code 0
Metadata extraction succeeds
===Metadata JSON===
{
    "metadata": {
        "DUNE.campaign": "justIN.w3317s1",
        "art.file_format_era": "ART_2011a",
        "art.file_format_version": 15,
        "art.first_event": 74,
        "art.last_event": 110,
        "art.process_name": "Mergeana",
        "art.run_type": "physics",
        "core.application": "art.reco",
        "core.application.family": "art",
        "core.application.name": "reco",
        "core.application.version": "v09_75_03d00",
        "core.data_stream": "out1",
        "core.data_tier": "full-reconstructed",
        "core.end_time": 1726834727.0,
        "core.event_count": 2,
        "core.file_format": "artroot",
        "core.file_type": "mc",
        "core.first_event_number": 74,
        "core.last_event_number": 110,
        "core.run_type": "physics",
        "core.runs": [
            20000031
        ],
        "core.runs_subruns": [
            2000003100000
        ],
        "core.start_time": 1726834688.0
    },
    "parents": [
        {
            "did": "usertests:000396_reco_data_2024-08-16T_162233Z_reco_data_2024-08-19T_140214Z_reco_data_2024-08-20T_115338Z_reco_data_2024-08-22T_092928Z_reco_data_2024-08-23T_100636Z.root"
        }
    ]
}
===================
.:
total 23440
-rw-r--r-- 1 osgusers domain users 22613589 Sep 20 08:18 000396_reco_data_2024-08-16T_162233Z_reco_data_2024-08-19T_140214Z_reco_data_2024-08-20T_115338Z_reco_data_2024-08-22T_092928Z_reco_data_2024-08-23T_100636Z_reco_data_2024-09-20T_121730Z.root
-rw-r--r-- 1 osgusers domain users  1343439 Sep 20 08:18 ana_hist.root
-rw-r--r-- 1 osgusers domain users    13321 Sep 20 08:18 jobscript.log
-rw-r--r-- 1 osgusers domain users     8793 Sep 20 08:18 000396_reco_data_2024-08-16T_162233Z_reco_data_2024-08-19T_140214Z_reco_data_2024-08-20T_115338Z_reco_data_2024-08-22T_092928Z_reco_data_2024-08-23T_100636Z_reco_2024-09-20T_121730Z.log
-rw-r--r-- 1 osgusers domain users     1204 Sep 20 08:18 000396_reco_data_2024-08-16T_162233Z_reco_data_2024-08-19T_140214Z_reco_data_2024-08-20T_115338Z_reco_data_2024-08-22T_092928Z_reco_data_2024-08-23T_100636Z_reco_data_2024-09-20T_121730Z.root.json
-rw-r--r-- 1 osgusers domain users     1107 Sep 20 08:18 000396_reco_data_2024-08-16T_162233Z_reco_data_2024-08-19T_140214Z_reco_data_2024-08-20T_115338Z_reco_data_2024-08-22T_092928Z_reco_data_2024-08-23T_100636Z_reco_data_2024-09-20T_121730Z.root.ext.json
-rw-r--r-- 1 osgusers domain users      344 Sep 20 08:17 all-input-dids.txt
-rw-r--r-- 1 osgusers domain users      248 Sep 20 08:18 justin-processed-pfns.txt
-rw-r--r-- 1 osgusers domain users        0 Sep 20 08:17 debugprod.log
justIN time: 2024-09-29 09:25:17 UTC       justIN version: 01.01.08