justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

21 July 2025: This instance at RAL is read-only. Please do not try submitting new workflows for now.

Jobsub ID 419105.33@justin-prod-sched01.dune.hep.ac.uk

Jobsub ID419105.33@justin-prod-sched01.dune.hep.ac.uk
Workflow ID7805
Stage ID1
User namehsouza@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
GPUNo
RSS bytes3145728000 (3000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2025-06-20 19:21:02
SiteFR_CCIN2P3
EntryDUNE_FR_CCIN2P3_cccondorce01
Last heartbeat2025-06-20 20:02:43
From worker nodeHostnameccwcondor0510
cpuinfoAMD EPYC 7302 16-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes3145728000 (3000 MiB)
Wall seconds limit106200 (29 hours)
GPU
Inner Apptainer?True
Job stateoutputting_failed
Allocator namejustin-allocator-pro.dune.hep.ac.uk
Started2025-06-20 19:39:08
Input filesfardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6413332_723_20231203T043942Z_gen_g4_detsim_hitreco__20240508T064347Z_reco2.root
JobscriptExit code0
Real time2m (160s)
CPU time0m (45s = 28%)
Max RSS bytes1337229312 (1275 MiB)
Outputting started2025-06-20 19:41:49
Output files
Finished2025-06-20 20:02:43
List job events     Wrapper job log

Jobscript log (last 10,000 characters)

th record. run: 6413332 subRun: 1 event: 72330 at 20-Jun-2025 21:40:06 CEST
Begin processing the 31st record. run: 6413332 subRun: 1 event: 72331 at 20-Jun-2025 21:40:07 CEST
Begin processing the 32nd record. run: 6413332 subRun: 1 event: 72332 at 20-Jun-2025 21:40:08 CEST
Begin processing the 33rd record. run: 6413332 subRun: 1 event: 72333 at 20-Jun-2025 21:40:09 CEST
Begin processing the 34th record. run: 6413332 subRun: 1 event: 72334 at 20-Jun-2025 21:40:10 CEST
Begin processing the 35th record. run: 6413332 subRun: 1 event: 72335 at 20-Jun-2025 21:40:12 CEST
Begin processing the 36th record. run: 6413332 subRun: 1 event: 72336 at 20-Jun-2025 21:40:12 CEST
Begin processing the 37th record. run: 6413332 subRun: 1 event: 72337 at 20-Jun-2025 21:40:14 CEST
Begin processing the 38th record. run: 6413332 subRun: 1 event: 72338 at 20-Jun-2025 21:40:15 CEST
Begin processing the 39th record. run: 6413332 subRun: 1 event: 72339 at 20-Jun-2025 21:40:16 CEST
Begin processing the 40th record. run: 6413332 subRun: 1 event: 72340 at 20-Jun-2025 21:40:17 CEST
Begin processing the 41st record. run: 6413332 subRun: 1 event: 72341 at 20-Jun-2025 21:40:18 CEST
Begin processing the 42nd record. run: 6413332 subRun: 1 event: 72342 at 20-Jun-2025 21:40:19 CEST
Begin processing the 43rd record. run: 6413332 subRun: 1 event: 72343 at 20-Jun-2025 21:40:20 CEST
Begin processing the 44th record. run: 6413332 subRun: 1 event: 72344 at 20-Jun-2025 21:40:21 CEST
Begin processing the 45th record. run: 6413332 subRun: 1 event: 72345 at 20-Jun-2025 21:40:22 CEST
Begin processing the 46th record. run: 6413332 subRun: 1 event: 72346 at 20-Jun-2025 21:40:23 CEST
Begin processing the 47th record. run: 6413332 subRun: 1 event: 72347 at 20-Jun-2025 21:40:24 CEST
Begin processing the 48th record. run: 6413332 subRun: 1 event: 72348 at 20-Jun-2025 21:40:25 CEST
Begin processing the 49th record. run: 6413332 subRun: 1 event: 72349 at 20-Jun-2025 21:40:26 CEST
Begin processing the 50th record. run: 6413332 subRun: 1 event: 72350 at 20-Jun-2025 21:40:28 CEST
Begin processing the 51st record. run: 6413332 subRun: 1 event: 72351 at 20-Jun-2025 21:40:29 CEST
Begin processing the 52nd record. run: 6413332 subRun: 1 event: 72352 at 20-Jun-2025 21:40:30 CEST
Begin processing the 53rd record. run: 6413332 subRun: 1 event: 72353 at 20-Jun-2025 21:40:31 CEST
Begin processing the 54th record. run: 6413332 subRun: 1 event: 72354 at 20-Jun-2025 21:40:31 CEST
Begin processing the 55th record. run: 6413332 subRun: 1 event: 72355 at 20-Jun-2025 21:40:33 CEST
Begin processing the 56th record. run: 6413332 subRun: 1 event: 72356 at 20-Jun-2025 21:40:35 CEST
Begin processing the 57th record. run: 6413332 subRun: 1 event: 72357 at 20-Jun-2025 21:40:36 CEST
Begin processing the 58th record. run: 6413332 subRun: 1 event: 72358 at 20-Jun-2025 21:40:40 CEST
Begin processing the 59th record. run: 6413332 subRun: 1 event: 72359 at 20-Jun-2025 21:40:40 CEST
Begin processing the 60th record. run: 6413332 subRun: 1 event: 72360 at 20-Jun-2025 21:40:41 CEST
Begin processing the 61st record. run: 6413332 subRun: 1 event: 72361 at 20-Jun-2025 21:40:42 CEST
Begin processing the 62nd record. run: 6413332 subRun: 1 event: 72362 at 20-Jun-2025 21:40:53 CEST
Begin processing the 63rd record. run: 6413332 subRun: 1 event: 72363 at 20-Jun-2025 21:40:55 CEST
Begin processing the 64th record. run: 6413332 subRun: 1 event: 72364 at 20-Jun-2025 21:40:56 CEST
Begin processing the 65th record. run: 6413332 subRun: 1 event: 72365 at 20-Jun-2025 21:40:58 CEST
Begin processing the 66th record. run: 6413332 subRun: 1 event: 72366 at 20-Jun-2025 21:40:59 CEST
Begin processing the 67th record. run: 6413332 subRun: 1 event: 72367 at 20-Jun-2025 21:40:59 CEST
Begin processing the 68th record. run: 6413332 subRun: 1 event: 72368 at 20-Jun-2025 21:41:00 CEST
Begin processing the 69th record. run: 6413332 subRun: 1 event: 72369 at 20-Jun-2025 21:41:01 CEST
Begin processing the 70th record. run: 6413332 subRun: 1 event: 72370 at 20-Jun-2025 21:41:03 CEST
Begin processing the 71st record. run: 6413332 subRun: 1 event: 72371 at 20-Jun-2025 21:41:03 CEST
Begin processing the 72nd record. run: 6413332 subRun: 1 event: 72372 at 20-Jun-2025 21:41:05 CEST
Begin processing the 73rd record. run: 6413332 subRun: 1 event: 72373 at 20-Jun-2025 21:41:06 CEST
Begin processing the 74th record. run: 6413332 subRun: 1 event: 72374 at 20-Jun-2025 21:41:07 CEST
Begin processing the 75th record. run: 6413332 subRun: 1 event: 72375 at 20-Jun-2025 21:41:08 CEST
Begin processing the 76th record. run: 6413332 subRun: 1 event: 72376 at 20-Jun-2025 21:41:11 CEST
Begin processing the 77th record. run: 6413332 subRun: 1 event: 72377 at 20-Jun-2025 21:41:13 CEST
Begin processing the 78th record. run: 6413332 subRun: 1 event: 72378 at 20-Jun-2025 21:41:14 CEST
Begin processing the 79th record. run: 6413332 subRun: 1 event: 72379 at 20-Jun-2025 21:41:15 CEST
Begin processing the 80th record. run: 6413332 subRun: 1 event: 72380 at 20-Jun-2025 21:41:16 CEST
Begin processing the 81st record. run: 6413332 subRun: 1 event: 72381 at 20-Jun-2025 21:41:18 CEST
Begin processing the 82nd record. run: 6413332 subRun: 1 event: 72382 at 20-Jun-2025 21:41:19 CEST
Begin processing the 83rd record. run: 6413332 subRun: 1 event: 72383 at 20-Jun-2025 21:41:20 CEST
Begin processing the 84th record. run: 6413332 subRun: 1 event: 72384 at 20-Jun-2025 21:41:21 CEST
Begin processing the 85th record. run: 6413332 subRun: 1 event: 72385 at 20-Jun-2025 21:41:22 CEST
Begin processing the 86th record. run: 6413332 subRun: 1 event: 72386 at 20-Jun-2025 21:41:25 CEST
Begin processing the 87th record. run: 6413332 subRun: 1 event: 72387 at 20-Jun-2025 21:41:26 CEST
Begin processing the 88th record. run: 6413332 subRun: 1 event: 72388 at 20-Jun-2025 21:41:27 CEST
Begin processing the 89th record. run: 6413332 subRun: 1 event: 72389 at 20-Jun-2025 21:41:28 CEST
Begin processing the 90th record. run: 6413332 subRun: 1 event: 72390 at 20-Jun-2025 21:41:28 CEST
Begin processing the 91st record. run: 6413332 subRun: 1 event: 72391 at 20-Jun-2025 21:41:31 CEST
Begin processing the 92nd record. run: 6413332 subRun: 1 event: 72392 at 20-Jun-2025 21:41:32 CEST
Begin processing the 93rd record. run: 6413332 subRun: 1 event: 72393 at 20-Jun-2025 21:41:33 CEST
Begin processing the 94th record. run: 6413332 subRun: 1 event: 72394 at 20-Jun-2025 21:41:39 CEST
Begin processing the 95th record. run: 6413332 subRun: 1 event: 72395 at 20-Jun-2025 21:41:40 CEST
Begin processing the 96th record. run: 6413332 subRun: 1 event: 72396 at 20-Jun-2025 21:41:41 CEST
Begin processing the 97th record. run: 6413332 subRun: 1 event: 72397 at 20-Jun-2025 21:41:42 CEST
Begin processing the 98th record. run: 6413332 subRun: 1 event: 72398 at 20-Jun-2025 21:41:43 CEST
Begin processing the 99th record. run: 6413332 subRun: 1 event: 72399 at 20-Jun-2025 21:41:46 CEST
Begin processing the 100th record. run: 6413332 subRun: 1 event: 72400 at 20-Jun-2025 21:41:47 CEST
20-Jun-2025 21:41:48 CEST  Closed input file "root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/c7/c8/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6413332_723_20231203T043942Z_gen_g4_detsim_hitreco__20240508T064347Z_reco2.root"

================================================================================================================================
TimeTracker printout (sec)                        Min           Avg           Max         Median          RMS         nEvts   
================================================================================================================================
Full event                                      0.24061      0.950297       7.14451      0.783714      0.784746        100    
--------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                         0.0182028     0.0302348     0.0564539     0.0358793     0.0105509       100    
prod:pandorapid:Chi2ParticleID                 0.106568      0.156848       0.2399        0.16284      0.027778        100    
prod:energyreconue:EnergyReco                  0.0826713     0.307627      0.498781      0.329092      0.0897311       100    
prod:energyreconumu:EnergyReco                0.00356813     0.156633      0.583263      0.166402      0.109271        100    
prod:energyreconc:EnergyReco                  0.00337119     0.039052      0.272938      0.0323823     0.044023        100    
[art]:TriggerResults:TriggerResultInserter    1.0901e-05    1.83915e-05   7.4791e-05    1.6196e-05    9.36112e-06      100    
end_path:ana:PIDASelectionAna                 0.00311682     0.259286       5.76857      0.0469622     0.634647        100    
================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 11111.4 MB
  Peak resident set size usage (VmHWM): 1337.23 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
.:
total 64
-rw-r--r-- 1 dune001 lbno 25026 Jun 20 21:41 atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6413332_723_20231203T043942Z_gen_g4_detsim_hitreco__20240508T064347Z_reco2_pida_2025-06-20T_193913Z.root
-rw-r--r-- 1 dune001 lbno 14218 Jun 20 21:41 atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6413332_723_20231203T043942Z_gen_g4_detsim_hitreco__20240508T064347Z_reco2_reco_2025-06-20T_193913Z.log
-rw-r--r-- 1 dune001 lbno 11972 Jun 20 21:41 jobscript.log
-rw-r--r-- 1 dune001 lbno   186 Jun 20 21:41 justin-processed-pfns.txt
-rw-r--r-- 1 dune001 lbno   137 Jun 20 21:39 all-input-dids.txt
-rw-r--r-- 1 dune001 lbno     0 Jun 20 21:39 debugprod.log
justIN time: 2025-08-15 00:07:05 UTC       justIN version: 01.03.02