justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 270076.68@justin-prod-sched01.dune.hep.ac.uk

Jobsub ID270076.68@justin-prod-sched01.dune.hep.ac.uk
Workflow ID3593
Stage ID1
User namecalcuttj@fnal.gov
HTCondor Groupgroup_dune.prod_mcsim
RequestedProcessors1
RSS bytes4193255424 (3999 MiB)
Wall seconds limit18000 (5 hours)
Submitted time2024-10-05 11:23:21
SiteUS_BNL
EntryDUNE_US_BNL_sp02
Last heartbeat2024-10-05 11:56:19
From worker nodeHostnamespool1487.sdcc.bnl.gov
cpuinfoIntel(R) Xeon(R) Gold 6336Y CPU @ 2.40GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4193255424 (3999 MiB)
Wall seconds limit171000 (47 hours)
Inner Apptainer?False
Job stateaborted
Allocator namejustin-allocator-pro.dune.hep.ac.uk
Started2024-10-05 11:41:11
Input fileshd-protodune:np04hd_raw_run027927_0431_dataflow2_datawriter_0_20240709T154435.hdf5
Outputting started2024-10-05 11:54:43
Output files
Finished2024-10-05 11:56:19
List job events     Wrapper job log

Jobscript log (last 10,000 characters)

ow2_datawriter_0_20240709T154435_reco_stage1.root"
job begin...
job begin...
Scale HitLimit based on readout window size 5859
HitLimit = 19530
Begin processing the 1st record. run: 27927 subRun: 1 event: 111586 at 05-Oct-2024 11:54:13 UTC
05-Oct-2024 11:54:14 UTC  Opened output file with pattern "%ifb_reco_stage2_%tc_keepup.root"
Begin processing the 2nd record. run: 27927 subRun: 1 event: 111594 at 05-Oct-2024 11:54:15 UTC
Begin processing the 3rd record. run: 27927 subRun: 1 event: 111602 at 05-Oct-2024 11:54:16 UTC
Begin processing the 4th record. run: 27927 subRun: 1 event: 111610 at 05-Oct-2024 11:54:16 UTC
Begin processing the 5th record. run: 27927 subRun: 1 event: 111618 at 05-Oct-2024 11:54:17 UTC
Begin processing the 6th record. run: 27927 subRun: 1 event: 111626 at 05-Oct-2024 11:54:17 UTC
Begin processing the 7th record. run: 27927 subRun: 1 event: 111634 at 05-Oct-2024 11:54:18 UTC
Begin processing the 8th record. run: 27927 subRun: 1 event: 111642 at 05-Oct-2024 11:54:18 UTC
Begin processing the 9th record. run: 27927 subRun: 1 event: 111650 at 05-Oct-2024 11:54:19 UTC
Begin processing the 10th record. run: 27927 subRun: 1 event: 111658 at 05-Oct-2024 11:54:19 UTC
Begin processing the 11th record. run: 27927 subRun: 1 event: 111666 at 05-Oct-2024 11:54:20 UTC
Begin processing the 12th record. run: 27927 subRun: 1 event: 111674 at 05-Oct-2024 11:54:21 UTC
Begin processing the 13th record. run: 27927 subRun: 1 event: 111682 at 05-Oct-2024 11:54:21 UTC
Begin processing the 14th record. run: 27927 subRun: 1 event: 111690 at 05-Oct-2024 11:54:22 UTC
Begin processing the 15th record. run: 27927 subRun: 1 event: 111698 at 05-Oct-2024 11:54:22 UTC
Begin processing the 16th record. run: 27927 subRun: 1 event: 111706 at 05-Oct-2024 11:54:23 UTC
Begin processing the 17th record. run: 27927 subRun: 1 event: 111714 at 05-Oct-2024 11:54:23 UTC
Begin processing the 18th record. run: 27927 subRun: 1 event: 111722 at 05-Oct-2024 11:54:24 UTC
Begin processing the 19th record. run: 27927 subRun: 1 event: 111730 at 05-Oct-2024 11:54:24 UTC
Begin processing the 20th record. run: 27927 subRun: 1 event: 111738 at 05-Oct-2024 11:54:25 UTC
Begin processing the 21st record. run: 27927 subRun: 1 event: 111746 at 05-Oct-2024 11:54:25 UTC
Begin processing the 22nd record. run: 27927 subRun: 1 event: 111754 at 05-Oct-2024 11:54:26 UTC
Begin processing the 23rd record. run: 27927 subRun: 1 event: 111762 at 05-Oct-2024 11:54:26 UTC
Begin processing the 24th record. run: 27927 subRun: 1 event: 111770 at 05-Oct-2024 11:54:27 UTC
Begin processing the 25th record. run: 27927 subRun: 1 event: 111778 at 05-Oct-2024 11:54:27 UTC
Begin processing the 26th record. run: 27927 subRun: 1 event: 111786 at 05-Oct-2024 11:54:28 UTC
Begin processing the 27th record. run: 27927 subRun: 1 event: 111794 at 05-Oct-2024 11:54:28 UTC
Begin processing the 28th record. run: 27927 subRun: 1 event: 111802 at 05-Oct-2024 11:54:29 UTC
Begin processing the 29th record. run: 27927 subRun: 1 event: 111810 at 05-Oct-2024 11:54:29 UTC
Begin processing the 30th record. run: 27927 subRun: 1 event: 111818 at 05-Oct-2024 11:54:30 UTC
Begin processing the 31st record. run: 27927 subRun: 1 event: 111826 at 05-Oct-2024 11:54:30 UTC
Begin processing the 32nd record. run: 27927 subRun: 1 event: 111834 at 05-Oct-2024 11:54:31 UTC
Begin processing the 33rd record. run: 27927 subRun: 1 event: 111842 at 05-Oct-2024 11:54:31 UTC
Begin processing the 34th record. run: 27927 subRun: 1 event: 111850 at 05-Oct-2024 11:54:32 UTC
Begin processing the 35th record. run: 27927 subRun: 1 event: 111858 at 05-Oct-2024 11:54:32 UTC
Begin processing the 36th record. run: 27927 subRun: 1 event: 111866 at 05-Oct-2024 11:54:33 UTC
05-Oct-2024 11:54:34 UTC  Closed output file "np04hd_raw_run027927_0431_dataflow2_datawriter_0_20240709T154435_reco_stage1_reco_stage2_20241005T115434_keepup.root"
05-Oct-2024 11:54:34 UTC  Closed input file "np04hd_raw_run027927_0431_dataflow2_datawriter_0_20240709T154435_reco_stage1.root"

===================================================================================================================================
TimeTracker printout (sec)                           Min           Avg           Max         Median          RMS         nEvts   
===================================================================================================================================
Full event                                        0.470153      0.526196      0.713475      0.517848      0.0534644       36     
-----------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                           0.000167195   0.000234782   0.000907826   0.000192354   0.000138052      36     
produce:fembfilter:PDHDFEMBFilter                 0.469029      0.524914      0.706911      0.516796      0.0528838       36     
[art]:TriggerResults:TriggerResultInserter       3.5625e-05    4.28281e-05   0.000106898   3.8363e-05    1.37127e-05      36     
end_path:out1:RootOutput                          2.772e-06     3.899e-06    1.5616e-05    3.4245e-06    2.12708e-06      36     
end_path:michelremoving:michelremoving           1.2331e-05    2.05676e-05   0.000230696   1.38575e-05   3.56199e-05      36     
end_path:michelremovinggnocchi:michelremoving     2.66e-06     5.14367e-06   6.7068e-05     3.065e-06    1.05693e-05      36     
end_path:calibanagnocchi:CalibAnaTree             2.336e-06    4.59978e-06   7.0489e-05     2.574e-06    1.11514e-05      36     
end_path:out1:RootOutput(write)                  0.00044526    0.000643524    0.0040339    0.000479722   0.000591684      36     
===================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 3704.61 MB
  Peak resident set size usage (VmHWM): 1090.91 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np04hd_raw_run027927_0431_dataflow2_datawriter_0_20240709T154435_reco_stage1_reco_stage2_20241005T115434_keepup.root
\tHists: np04hd_raw_run027927_0431_dataflow2_datawriter_0_20240709T154435_reco_stage1_reco_stage2_20241005T115434_keepup_hists.root
Forming reco metadata
Successfully opened file np04hd_raw_run027927_0431_dataflow2_datawriter_0_20240709T154435_reco_stage1_reco_stage2_20241005T115434_keepup.root
Ran successfully
{
  "name": "np04hd_raw_run027927_0431_dataflow2_datawriter_0_20240709T154435_reco_stage1_reco_stage2_20241005T115434_keepup.root",
  "namespace": "hd-protodune-det-reco",
  "metadata": {
    "core.file_format": "artroot",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v09_91_02d01",
    "core.data_tier": "full-reconstructed",
    "dune.config_file": "standard_reco_stage2_calibration_protodunehd_keepup.fcl",
    "dune.campaign": "hd-protodune-reco-keepup-v0",
    "core.start_time": 1728129220.0,
    "core.end_time": 1728129274.0,
    "core.events": [
      111586,
      111594,
      111602,
      111610,
      111618,
      111626,
      111634,
      111642,
      111650,
      111658,
      111666,
      111674,
      111682,
      111690,
      111698,
      111706,
      111714,
      111722,
      111730,
      111738,
      111746,
      111754,
      111762,
      111770,
      111778,
      111786,
      111794,
      111802,
      111810,
      111818,
      111826,
      111834,
      111842,
      111850,
      111858,
      111866
    ],
    "core.event_count": 36,
    "core.first_event_number": 111586,
    "core.last_event_number": 111866,
    "core.data_stream": "cosmics",
    "core.file_content_status": "good",
    "core.file_type": "detector",
    "core.run_type": "hd-protodune",
    "core.runs": [
      27927
    ],
    "core.runs_subruns": [
      2792700001
    ],
    "dune.daq_test": false,
    "retention.status": "active",
    "retention.class": "physics",
    "origin.applications.config_files": {
      "reco1": "standard_reco_stage1_protodunehd_keepup.fcl"
    },
    "origin.applications.versions": {
      "reco1": "v09_91_02d01"
    },
    "origin.applications.names": [
      "reco1"
    ]
  },
  "parents": [
    {
      "did": "hd-protodune:np04hd_raw_run027927_0431_dataflow2_datawriter_0_20240709T154435.hdf5"
    }
  ]
}Forming hist metadata
formed
{
  "name": "np04hd_raw_run027927_0431_dataflow2_datawriter_0_20240709T154435_reco_stage1_reco_stage2_20241005T115434_keepup_hists.root",
  "namespace": "hd-protodune-det-reco",
  "metadata": {
    "core.file_format": "root",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v09_91_02d01",
    "core.data_tier": "root-tuple-virtual",
    "dune.config_file": "standard_reco_stage2_calibration_protodunehd_keepup.fcl",
    "dune.campaign": "hd-protodune-reco-keepup-v0",
    "core.start_time": 1728129220.0,
    "core.end_time": 1728129274.0,
    "core.data_stream": "cosmics",
    "core.file_content_status": "good",
    "core.file_type": "detector",
    "core.run_type": "hd-protodune",
    "core.runs": [
      27927
    ],
    "core.runs_subruns": [
      2792700001
    ],
    "dune.daq_test": false,
    "retention.status": "active",
    "retention.class": "physics",
    "origin.applications.config_files": {
      "reco1": "standard_reco_stage1_protodunehd_keepup.fcl"
    },
    "origin.applications.versions": {
      "reco1": "v09_91_02d01"
    },
    "origin.applications.names": [
      "reco1"
    ]
  },
  "parents": [
    {
      "did": "hd-protodune:np04hd_raw_run027927_0431_dataflow2_datawriter_0_20240709T154435.hdf5"
    }
  ]
}
justIN time: 2024-11-21 19:13:47 UTC       justIN version: 01.01.09