justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 88973.7@justin-prod-sched02.dune.hep.ac.uk

Jobsub ID88973.7@justin-prod-sched02.dune.hep.ac.uk
Workflow ID3877
Stage ID1
User namecalcuttj@fnal.gov
HTCondor Groupgroup_dune.prod_mcsim
RequestedProcessors1
RSS bytes4193255424 (3999 MiB)
Wall seconds limit18000 (5 hours)
Submitted time2024-10-30 23:16:47
SiteCERN
EntryCMSHTPC_T2_CH_CERN_ce513
Last heartbeat2024-10-30 23:44:53
From worker nodeHostnameb9p05p5577.cern.ch
cpuinfoAMD EPYC 7543 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4193255424 (3999 MiB)
Wall seconds limit171000 (47 hours)
Inner Apptainer?True
Job stateaborted
Allocator namejustin-allocator-pro.dune.hep.ac.uk
Started2024-10-30 23:18:27
Input fileshd-protodune:np04hd_raw_run032513_0000_dataflow0_datawriter_0_20241030T221147.hdf5
Outputting started2024-10-30 23:43:35
Output files
Finished2024-10-30 23:44:53
List job events     Wrapper job log

Jobscript log (last 10,000 characters)

17 0
18 0
19 0
20 0
21 0
22 0
23 0
24 0
25 0
26 0
27 0
28 0
29 0
30 0
31 0
32 0
33 0
34 0
35 0
36 0
37 0
38 0
39 0
40 0
41 0
42 0
43 0
44 0
45 0
46 0
47 0
48 0
49 0
50 0
51 0
52 0
53 0
54 0
55 0
56 0
57 0
58 0
59 0
60 0
61 0
62 0
63 0
64 0
65 0
66 0
67 0
68 0
69 0
70 0
71 0
72 0
73 0
74 0
75 0
76 0
77 0
78 0
79 0
80 0
81 0
82 0
83 0
84 0
85 0
86 0
87 0
88 0
89 0
90 0
91 0
92 0
93 0
94 0
95 0
96 0
97 0
98 0
99 0
Failure in algorithm Alg0002, LArMaster, STATUS_CODE_NOT_FOUND
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------

Begin processing the 22nd record. run: 32513 subRun: 1 event: 44 at 31-Oct-2024 00:42:43 CET
RawFrameSource: got 10240 raw::RawDigit objects
	input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 2 traces, 2 summary
Retagger: tagging trace set: gauss with 1 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 1 traces tagged "gauss"
FrameSaver: q=181.203 n=311 tag=gauss
wclsFrameSaver: saving 2 traces tagged "wiener"
FrameSaver: q=178.347 n=310 tag=wiener
17 hits in TPC 6
Failure in algorithm Alg0002, LArMaster, STATUS_CODE_NOT_FOUND
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------

31-Oct-2024 00:43:21 CET  Closed output file "np04hd_raw_run032513_0000_dataflow0_datawriter_0_20241030T221147_reco_stage1_reco_stage2_20241030T234321_keepup.root"
31-Oct-2024 00:43:21 CET  Closed input file "np04hd_raw_run032513_0000_dataflow0_datawriter_0_20241030T221147_reco_stage1.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              36.3859       39.2422       58.7529       38.518        4.38811        22     
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.000276964   0.000425495   0.000622368   0.000425586   8.69121e-05      22     
produce:fembfilter:PDHDFEMBFilter                      0.594912       0.73963      0.829675       0.74871      0.0560461       22     
produce:wclsdatahd:WireCellToolkit                      35.444        37.1841       39.0204       37.2174       1.0318         22     
produce:gaushit:GausHitFinder                         0.00355022    0.00467105    0.00720502    0.00446626    0.000926865      22     
produce:nhitsfilter:NumberOfHitsFilter                4.4041e-05    7.28742e-05   0.000212863   6.5856e-05    3.2792e-05       22     
produce:reco3d:SpacePointSolver                       8.9431e-05     0.0010011    0.00323479    0.00112966    0.000693664      22     
produce:hitpdune:DisambigFromSpacePoints              0.000214293   0.000390082   0.00198812    0.000285659   0.000359453      22     
produce:pandora:StandardPandora                       0.00303451     0.782864       16.8839      0.0100879      3.51358        22     
produce:pandoraWriter:StandardPandora                 0.000172412   0.000834195   0.00314539    0.000540047   0.000785498      22     
produce:pandoraTrack:LArPandoraTrackCreation          0.000121392   0.000299043   0.00315151    0.000166127   0.000622672      22     
produce:pandoraShower:LArPandoraShowerCreation        0.000111122   0.000180426   0.000830651   0.000153562   0.000142985      22     
produce:pandoracalo:Calorimetry                       0.000103291   0.000193904   0.00114739    0.000146582   0.000209029      22     
produce:pandoracalonosce:Calorimetry                  7.2241e-05    8.7968e-05    0.000112311   8.3681e-05    1.1376e-05       22     
produce:pandorapid:Chi2ParticleID                     4.4701e-05    8.27275e-05   0.000526967   5.9756e-05    9.73007e-05      22     
produce:pandoraShowercalo:ShowerCalorimetry           7.3471e-05    0.000114308   0.000694139   8.5771e-05    0.00012673       22     
produce:pandoraShowercalonosce:ShowerCalorimetry      4.5361e-05    7.23769e-05   9.7301e-05    7.1071e-05    1.04479e-05      22     
produce:pandoraGnocchiCalo:GnocchiCalorimetry         6.8601e-05    0.000145775   0.00106343    0.000102031   0.000200602      22     
produce:pandoraGnocchiCalonosce:GnocchiCalorimetry    6.1811e-05    9.41917e-05   0.000109811   9.4021e-05    1.04537e-05      22     
produce:emtrkmichelid:EmTrackMichelId                  0.132891      0.505266       2.68877       0.3694       0.516979        22     
[art]:TriggerResults:TriggerResultInserter             1.522e-05    2.49794e-05   7.8811e-05    2.05455e-05   1.31113e-05      22     
end_path:out1:RootOutput                                4.6e-06     1.00288e-05   2.4591e-05    9.3155e-06    3.58783e-06      22     
end_path:michelremoving:michelremoving                0.00074523    0.00131107    0.00438586    0.00114379    0.000726304      22     
end_path:michelremovinggnocchi:michelremoving         0.000472936   0.00160302      0.00297     0.00156678    0.000518822      22     
end_path:calibanagnocchi:CalibAnaTree                 0.00550508    0.00821255     0.010252     0.00818351    0.000860571      22     
end_path:out1:RootOutput(write)                       0.00414036    0.00737214    0.00938612    0.00730126    0.00123289       22     
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4707.99 MB
  Peak resident set size usage (VmHWM): 2363.2 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np04hd_raw_run032513_0000_dataflow0_datawriter_0_20241030T221147_reco_stage1_reco_stage2_20241030T234321_keepup.root
\tHists: np04hd_raw_run032513_0000_dataflow0_datawriter_0_20241030T221147_reco_stage1_reco_stage2_20241030T234321_keepup_hists.root
Forming reco metadata
Successfully opened file np04hd_raw_run032513_0000_dataflow0_datawriter_0_20241030T221147_reco_stage1_reco_stage2_20241030T234321_keepup.root
Ran successfully
{
  "name": "np04hd_raw_run032513_0000_dataflow0_datawriter_0_20241030T221147_reco_stage1_reco_stage2_20241030T234321_keepup.root",
  "namespace": "hd-protodune-det-reco",
  "metadata": {
    "core.file_format": "artroot",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v09_91_02d01",
    "core.data_tier": "full-reconstructed",
    "dune.config_file": "standard_reco_stage2_calibration_protodunehd_keepup.fcl",
    "dune.campaign": "hd-protodune-reco-keepup-v0",
    "core.start_time": 1730330893.0,
    "core.end_time": 1730331802.0,
    "core.events": [
      2,
      4,
      6,
      8,
      10,
      12,
      14,
      16,
      18,
      20,
      22,
      24,
      26,
      28,
      30,
      32,
      34,
      36,
      38,
      40,
      42,
      44
    ],
    "core.event_count": 22,
    "core.first_event_number": 2,
    "core.last_event_number": 44,
    "core.data_stream": "cosmics",
    "core.file_content_status": "good",
    "core.file_type": "detector",
    "core.run_type": "hd-protodune",
    "core.runs": [
      32513
    ],
    "core.runs_subruns": [
      3251300001
    ],
    "dune.daq_test": false,
    "retention.status": "active",
    "retention.class": "physics",
    "origin.applications.config_files": {
      "reco1": "standard_reco_stage1_protodunehd_keepup.fcl"
    },
    "origin.applications.versions": {
      "reco1": "v09_91_02d01"
    },
    "origin.applications.names": [
      "reco1"
    ]
  },
  "parents": [
    {
      "did": "hd-protodune:np04hd_raw_run032513_0000_dataflow0_datawriter_0_20241030T221147.hdf5"
    }
  ]
}Forming hist metadata
formed
{
  "name": "np04hd_raw_run032513_0000_dataflow0_datawriter_0_20241030T221147_reco_stage1_reco_stage2_20241030T234321_keepup_hists.root",
  "namespace": "hd-protodune-det-reco",
  "metadata": {
    "core.file_format": "root",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v09_91_02d01",
    "core.data_tier": "root-tuple-virtual",
    "dune.config_file": "standard_reco_stage2_calibration_protodunehd_keepup.fcl",
    "dune.campaign": "hd-protodune-reco-keepup-v0",
    "core.start_time": 1730330893.0,
    "core.end_time": 1730331802.0,
    "core.data_stream": "cosmics",
    "core.file_content_status": "good",
    "core.file_type": "detector",
    "core.run_type": "hd-protodune",
    "core.runs": [
      32513
    ],
    "core.runs_subruns": [
      3251300001
    ],
    "dune.daq_test": false,
    "retention.status": "active",
    "retention.class": "physics",
    "origin.applications.config_files": {
      "reco1": "standard_reco_stage1_protodunehd_keepup.fcl"
    },
    "origin.applications.versions": {
      "reco1": "v09_91_02d01"
    },
    "origin.applications.names": [
      "reco1"
    ]
  },
  "parents": [
    {
      "did": "hd-protodune:np04hd_raw_run032513_0000_dataflow0_datawriter_0_20241030T221147.hdf5"
    }
  ]
}
justIN time: 2024-11-17 05:51:09 UTC       justIN version: 01.01.09