21 July 2025: This instance at RAL is read-only. Please do not try submitting new workflows for now.
Jobsub ID 229180.1@justin-prod-sched02.dune.hep.ac.uk
| Jobsub ID | 229180.1@justin-prod-sched02.dune.hep.ac.uk | |
| Workflow ID | 7908 | |
| Stage ID | 1 | |
| User name | epennacc@fnal.gov | |
| HTCondor Group | group_dune.prod_mcsim | |
| Requested | Processors | 1 |
| GPU | No | |
| RSS bytes | 4194304000 (4000 MiB) | |
| Wall seconds limit | 80000 (22 hours) | |
| Submitted time | 2025-06-25 08:18:01 | |
| Site | UK_Edinburgh | |
| Entry | DUNE_UK_SGridECDF_ce1_multicore | |
| Last heartbeat | 2025-06-25 13:07:10 | |
| From worker node | Hostname | node2b08.ecdf.ed.ac.uk |
| cpuinfo | Intel(R) Xeon(R) Gold 6338 CPU @ 2.00GHz | |
| OS release | Scientific Linux release 7.9 (Nitrogen) | |
| Processors | 1 | |
| RSS bytes | 4194304000 (4000 MiB) | |
| Wall seconds limit | 171000 (47 hours) | |
| GPU | ||
| Inner Apptainer? | True | |
| Job state | finished | |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk | |
| Started | 2025-06-25 08:18:46 | |
| Input files | fardet-vd:prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250201T004205Z_gen_004267_supernova_g4stage1_g4stage2_detsim_reco.root | |
| Jobscript | Exit code | 0 |
| Real time | 4h (17148s) | |
| CPU time | 4h (16873s = 98%) | |
| Max RSS bytes | 2539094016 (2421 MiB) | |
| Outputting started | 2025-06-25 13:04:34 | |
| Output files | fardet-vd:prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250201T004205Z_gen_004267_supernova_g4stage1_g4stage2_detsim_reco_20250625T081851Z_reco2.root | |
| Finished | 2025-06-25 13:07:10 | |
| Saved logs | justin-logs:229180.1-justin-prod-sched02.dune.hep.ac.uk.logs.tgz | |
| List job events Wrapper job log | ||
Jobscript log (last 10,000 characters)
%MSG %MSG-e WireIDIntersectionCheck: PMAlgTrackMaker:pmtracktc@BeginModule 25-Jun-2025 14:01:53 BST run: 5013 subRun: 0 event: 42670 Comparing two wires in the same plane: return failure %MSG %MSG-e WireIDIntersectionCheck: PMAlgTrackMaker:pmtracktc@BeginModule 25-Jun-2025 14:01:53 BST run: 5013 subRun: 0 event: 42670 Comparing two wires in the same plane: return failure %MSG %MSG-e WireIDIntersectionCheck: PMAlgTrackMaker:pmtracktc@BeginModule 25-Jun-2025 14:01:53 BST run: 5013 subRun: 0 event: 42670 Comparing two wires in the same plane: return failure %MSG %MSG-e WireIDIntersectionCheck: PMAlgTrackMaker:pmtracktc@BeginModule 25-Jun-2025 14:01:53 BST run: 5013 subRun: 0 event: 42670 Comparing two wires in the same plane: return failure %MSG %MSG-e WireIDIntersectionCheck: PMAlgTrackMaker:pmtracktc@BeginModule 25-Jun-2025 14:01:53 BST run: 5013 subRun: 0 event: 42670 Comparing two wires in the same plane: return failure %MSG %MSG-e WireIDIntersectionCheck: PMAlgTrackMaker:pmtracktc@BeginModule 25-Jun-2025 14:01:54 BST run: 5013 subRun: 0 event: 42670 Comparing two wires in the same plane: return failure %MSG [2025-06-25 14:04:28.032578 +0100][Debug ][File ] [0xdecb850@root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-vd/da/7d/prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250201T004205Z_gen_004267_supernova_g4stage1_g4stage2_detsim_reco.root?xrdcl.requuid=5ad54842-ff5c-4e7c-a8f2-f54ad4793837] Sending a read command for handle 0x0 to pubstor2307.fnal.gov:22428 [2025-06-25 14:04:28.032658 +0100][Debug ][ExDbgMsg ] [pubstor2307.fnal.gov:22428] MsgHandler created: 0x14b305d0 (message: kXR_read (handle: 0x00000000, offset: 2966623616, size: 1396978) ). [2025-06-25 14:04:28.032742 +0100][Debug ][ExDbgMsg ] [pubstor2307.fnal.gov:22428] Moving MsgHandler: 0x14b305d0 (message: kXR_read (handle: 0x00000000, offset: 2966623616, size: 1396978) ) from out-queu to in-queue. [2025-06-25 14:04:28.170175 +0100][Debug ][ExDbgMsg ] [msg: 0x22b0b850] Assigned MsgHandler: 0x14b305d0. [2025-06-25 14:04:28.170220 +0100][Debug ][ExDbgMsg ] [handler: 0x14b305d0] Removed MsgHandler: 0x14b305d0 from the in-queue. [2025-06-25 14:04:28.787931 +0100][Debug ][ExDbgMsg ] [pubstor2307.fnal.gov:22428] Calling MsgHandler: 0x14b305d0 (message: kXR_read (handle: 0x00000000, offset: 2966623616, size: 1396978) ) with status: [SUCCESS] . [2025-06-25 14:04:28.787981 +0100][Debug ][ExDbgMsg ] [pubstor2307.fnal.gov:22428] Destroying MsgHandler: 0x14b305d0. 25-Jun-2025 14:04:30 BST Closed output file "prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250201T004205Z_gen_004267_supernova_g4stage1_g4stage2_detsim_reco_20250625T081851Z_reco2.root" [2025-06-25 14:04:30.064322 +0100][Debug ][File ] [0xdecb850@root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-vd/da/7d/prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250201T004205Z_gen_004267_supernova_g4stage1_g4stage2_detsim_reco.root?xrdcl.requuid=5ad54842-ff5c-4e7c-a8f2-f54ad4793837] Sending a close command for handle 0x0 to pubstor2307.fnal.gov:22428 [2025-06-25 14:04:30.064397 +0100][Debug ][ExDbgMsg ] [pubstor2307.fnal.gov:22428] MsgHandler created: 0x32ca99f0 (message: kXR_close (handle: 0x00000000) ). [2025-06-25 14:04:30.064506 +0100][Debug ][ExDbgMsg ] [pubstor2307.fnal.gov:22428] Moving MsgHandler: 0x32ca99f0 (message: kXR_close (handle: 0x00000000) ) from out-queu to in-queue. [2025-06-25 14:04:30.169367 +0100][Debug ][ExDbgMsg ] [msg: 0x32a3ef00] Assigned MsgHandler: 0x32ca99f0. [2025-06-25 14:04:30.169411 +0100][Debug ][ExDbgMsg ] [handler: 0x32ca99f0] Removed MsgHandler: 0x32ca99f0 from the in-queue. [2025-06-25 14:04:30.169477 +0100][Debug ][ExDbgMsg ] [pubstor2307.fnal.gov:22428] Calling MsgHandler: 0x32ca99f0 (message: kXR_close (handle: 0x00000000) ) with status: [SUCCESS] . [2025-06-25 14:04:30.169606 +0100][Debug ][File ] [0xdecb850@root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-vd/da/7d/prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250201T004205Z_gen_004267_supernova_g4stage1_g4stage2_detsim_reco.root?xrdcl.requuid=5ad54842-ff5c-4e7c-a8f2-f54ad4793837] Close returned from pubstor2307.fnal.gov:22428 with: [SUCCESS] [2025-06-25 14:04:30.169668 +0100][Debug ][ExDbgMsg ] [pubstor2307.fnal.gov:22428] Destroying MsgHandler: 0x32ca99f0. 25-Jun-2025 14:04:30 BST Closed input file "root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-vd/da/7d/prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250201T004205Z_gen_004267_supernova_g4stage1_g4stage2_detsim_reco.root" ======================================================================================================================================== TimeTracker printout (sec) Min Avg Max Median RMS nEvts ======================================================================================================================================== Full event 1414.41 1690.88 1927.97 1725.7 188.229 10 ---------------------------------------------------------------------------------------------------------------------------------------- source:RootInput(read) 0.00235223 0.0289675 0.12597 0.00655932 0.0471671 10 reco:pandora:StandardPandora 1260.21 1536.42 1776.82 1574.09 189.549 10 reco:pandoraTrack:LArPandoraTrackCreation 0.000188475 0.00507751 0.0360696 0.00249878 0.0103943 10 reco:pandoraShower:LArPandoraModularShowerCreation 0.000208375 0.00111884 0.00562912 0.000685931 0.00152656 10 reco:pandoracalo:Calorimetry 0.000119053 0.000837266 0.00259333 0.000281435 0.000972179 10 reco:pandorapid:Chi2ParticleID 3.9825e-05 0.000116534 0.000613163 7.1414e-05 0.000166269 10 reco:linecluster:LineCluster 0.96932 1.03528 1.40708 0.978529 0.129606 10 reco:trajcluster:TrajCluster 0.221279 0.238831 0.258341 0.236412 0.0100138 10 reco:pmtracktc:PMAlgTrackMaker 3.94277 4.36434 4.90865 4.40707 0.305018 10 reco:emtrkmichelid:EmTrackMichelId 143.483 147.472 153.168 146.194 3.45787 10 reco:solarflash:SolarOpFlash 1.08095 1.12916 1.30368 1.09487 0.0688692 10 [art]:TriggerResults:TriggerResultInserter 1.8387e-05 2.26059e-05 3.8692e-05 2.1237e-05 5.6394e-06 10 end_path:out1:RootOutput 6.339e-06 8.0674e-06 1.5631e-05 7.3055e-06 2.56095e-06 10 end_path:out1:RootOutput(write) 0.168718 0.180971 0.205397 0.179613 0.00929275 10 ======================================================================================================================================== ==================================================================================================== MemoryTracker summary (base-10 MB units used) Peak virtual memory usage (VmPeak) : 4007.22 MB Peak resident set size usage (VmHWM): 2539.09 MB ==================================================================================================== [2025-06-25 14:04:30.546532 +0100][Debug ][JobMgr ] Stopping the job manager... [2025-06-25 14:04:30.546919 +0100][Debug ][JobMgr ] Job manager stopped [2025-06-25 14:04:30.546976 +0100][Debug ][TaskMgr ] Stopping the task manager... [2025-06-25 14:04:30.547114 +0100][Debug ][TaskMgr ] Task manager stopped [2025-06-25 14:04:30.547120 +0100][Debug ][Poller ] Stopping the poller... [2025-06-25 14:04:30.547316 +0100][Debug ][AsyncSock ] [pubstor2307.fnal.gov:22428.0] Closing the socket [2025-06-25 14:04:30.547338 +0100][Debug ][Poller ] <[::ffff:192.41.105.41]:59668><--><[::ffff:131.225.69.93]:22428> Removing socket from the poller [2025-06-25 14:04:30.547386 +0100][Debug ][PostMaster ] [pubstor2307.fnal.gov:22428] Destroying stream [2025-06-25 14:04:30.547391 +0100][Debug ][AsyncSock ] [pubstor2307.fnal.gov:22428.0] Closing the socket Art has completed and will exit with status 0. Moving on to metadata extractor ============OUTPUT FILE: prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250201T004205Z_gen_004267_supernova_g4stage1_g4stage2_detsim_reco_20250625T081851Z_reco2.root Extracted metadata checking file====== fardet-vd:prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250201T004205Z_gen_004267_supernova_g4stage1_g4stage2_detsim_reco.root MD conversion to MetaCat OK all-input-dids.txt debugprod.log jobscript.log justin-processed-pfns.txt old_md.json prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250201T004205Z_gen_004267_supernova_g4stage1_g4stage2_detsim_reco_20250625T081851Z_reco2.root prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250201T004205Z_gen_004267_supernova_g4stage1_g4stage2_detsim_reco_20250625T081851Z_reco2.root.ext.json prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250201T004205Z_gen_004267_supernova_g4stage1_g4stage2_detsim_reco_20250625T081851Z_reco2.root.json reco2_hist.root