Jobsub ID 108550.47@justin-prod-sched02.dune.hep.ac.uk
| Jobsub ID | 108550.47@justin-prod-sched02.dune.hep.ac.uk |
| Workflow ID | 4213 |
| Stage ID | 1 |
| User name | ismerio@fnal.gov |
| HTCondor Group | group_dune |
| Requested | Processors | 1 |
| RSS bytes | 4194304000 (4000 MiB) |
| Wall seconds limit | 80000 (22 hours) |
| Submitted time | 2024-11-22 23:15:23 |
| Site | UK_RAL-Tier1 |
| Entry | LIGO_UK_RAL_arc_ce04 |
| Last heartbeat | 2024-11-23 00:21:45 |
| From worker node | Hostname | dune001-2994290.0-lcg2699.gridpp.rl.ac.uk |
| cpuinfo | AMD EPYC 9654 96-Core Processor |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4194304000 (4000 MiB) |
| Wall seconds limit | 216000 (60 hours) |
| Inner Apptainer? | True |
| Job state | finished |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk |
| Started | 2024-11-22 23:33:25 |
| Input files | fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6390196_832_20231201T210816Z_gen_g4_detsim_hitreco__20240507T223649Z_reco2.root
|
| Jobscript | Exit code | 0 |
| Real time | 36m (2203s) |
| CPU time | 38m (2314s = 105%) |
| Outputting started | 2024-11-23 00:10:09 |
| Output files | https://fndcadoor.fnal.gov:2880/dune/scratch/users/ismerio/04213/1/Validation_ccnc_atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6390196_832_20231201T210816Z_gen_g4_detsim_hitreco__20240507T223649Z_reco2.root
|
| Finished | 2024-11-23 00:21:45 |
| Saved logs | justin-logs:108550.47-justin-prod-sched02.dune.hep.ac.uk.logs.tgz |
| List job events Wrapper job log |
Jobscript log (last 10,000 characters)
plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck: PMAlgTrajFitter:pmtrajfittc@BeginModule 23-Nov-2024 00:10:01 UTC run: 6390196 subRun: 1 event: 83300
1st wire C:0 T:2 P:2 W:789 does not exist (max wire number: 480)
%MSG
%MSG-e WireIDIntersectionCheck: PMAlgTrajFitter:pmtrajfittc@BeginModule 23-Nov-2024 00:10:01 UTC run: 6390196 subRun: 1 event: 83300
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck: PMAlgTrajFitter:pmtrajfittc@BeginModule 23-Nov-2024 00:10:01 UTC run: 6390196 subRun: 1 event: 83300
Comparing two wires in the same plane: return failure
%MSG
%MSG-e pma::Track3D: PMAlgTrajFitter:pmtrajfittc@BeginModule 23-Nov-2024 00:10:01 UTC run: 6390196 subRun: 1 event: 83300
0 enabled hits in AverageDist2 calculation.
%MSG
%MSG-e pma::Track3D: PMAlgTrajFitter:pmtrajfittc@BeginModule 23-Nov-2024 00:10:01 UTC run: 6390196 subRun: 1 event: 83300
Track empty.
%MSG
%MSG-e pma::Track3D: PMAlgTrajFitter:pmtrajfittc@BeginModule 23-Nov-2024 00:10:01 UTC run: 6390196 subRun: 1 event: 83300
TuneFullTree failed.
%MSG
%MSG-e WireIDIntersectionCheck: EMShower:emshower@BeginModule 23-Nov-2024 00:10:05 UTC run: 6390196 subRun: 1 event: 83300
Comparing two wires in the same plane: return failure
%MSG
Boundary wire vector sizes: 188, 236, 176
minwire 0: 538
minwire 1: 2517
minwire 2: 167
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 199
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 199
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 199
Classifier summary:
Output 0: 0.392119, 0.484854, 0.123028,
Output 1: 0.271976, 0.612528, 0.107297, 0.00819819,
Output 2: 0.093917, 0.906083,
473
***Wiener Filter*******ApplyPostfilter***
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 1590!
23-Nov-2024 00:10:08 UTC Closed output file "atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6390196_832_20231201T210816Z_gen_g4_detsim_hitreco__20240507T223649Z_reco2_reco_data_2024-11-22T_233329Z.root"
23-Nov-2024 00:10:08 UTC Closed input file "root://xrootd.echo.stfc.ac.uk:1094/dune:/protodune/RSE/fardet-hd/5e/f8/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6390196_832_20231201T210816Z_gen_g4_detsim_hitreco__20240507T223649Z_reco2.root"
========================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
========================================================================================================================================
Full event 1.49198 20.6667 320.775 7.82054 48.8461 100
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.00153569 0.110093 0.450075 0.0865797 0.0886098 100
reco:linecluster:LineCluster 0.0328812 0.487434 13.3137 0.21102 1.42295 100
reco:trajcluster:TrajCluster 0.00727216 0.670884 21.5675 0.0872075 2.60119 100
reco:pandora:StandardPandora 0.880213 2.20368 50.1471 1.18371 5.49111 100
reco:pandoraTrack:LArPandoraTrackCreation 0.000115644 0.00509645 0.133492 0.00131087 0.0150096 100
reco:pandoraShower:LArPandoraModularShowerCreation 0.000147942 0.00780177 0.0721879 0.0024647 0.0140955 100
reco:pandoracalo:Calorimetry 9.1478e-05 0.017055 0.536877 0.00078736 0.0690474 100
reco:pandorapid:Chi2ParticleID 3.341e-05 0.000144742 0.00110329 0.000113947 0.000137867 100
reco:trkshowersplit:TrackShowerHits 0.000105649 0.0220256 0.483617 0.000625011 0.0855313 100
reco:pmtrack:PMAlgTrackMaker 0.000926845 1.88128 47.9003 0.133895 6.5358 100
reco:pmtrackcalo:Calorimetry 5.0546e-05 0.00640661 0.226988 0.000799163 0.0249117 100
reco:pmtrackpid:Chi2ParticleID 2.0781e-05 0.000216548 0.0020833 0.000134788 0.000322376 100
reco:pmtrajfit:PMAlgTrajFitter 0.00013956 1.51134 29.1976 0.253512 4.63276 100
reco:pmtrajfitcalo:Calorimetry 3.7075e-05 0.0159311 0.426223 0.00114869 0.0619538 100
reco:pmtrajfitpid:Chi2ParticleID 1.8297e-05 0.00016346 0.00117088 0.000130251 0.000153994 100
reco:pmtracktc:PMAlgTrackMaker 0.000794636 2.36841 70.8615 0.144858 9.25323 100
reco:pmtracktccalo:Calorimetry 4.5708e-05 0.00372131 0.105502 0.000848777 0.0115093 100
reco:pmtracktcpid:Chi2ParticleID 1.9199e-05 0.000193931 0.00143043 0.000133461 0.000235066 100
reco:pmtrajfittc:PMAlgTrajFitter 0.000173911 1.10937 18.7755 0.203223 3.02891 100
reco:pmtrajfittccalo:Calorimetry 5.287e-05 0.0107159 0.265934 0.000975689 0.0374783 100
reco:pmtrajfittcpid:Chi2ParticleID 1.978e-05 0.000267499 0.00230518 0.000145639 0.000428488 100
reco:blurredcluster:BlurredClustering 0.00392042 0.773903 7.42053 0.370957 1.09294 100
reco:emtrkmichelid:EmTrackMichelId 0.124297 6.94446 106.898 2.03257 16.1094 100
reco:emshower:EMShower 0.000244808 0.0223296 0.219112 0.00427087 0.0367175 100
reco:cvnmap:CVNMapper 2.2323e-05 0.00524779 0.0333367 0.00399968 0.00617662 100
reco:cvneva:CVNEvaluator 1.6174e-05 0.487698 1.03942 0.57473 0.330879 100
reco:energyrecnumu:EnergyReco 0.000290207 0.0263049 0.663629 0.00521257 0.0809429 100
reco:energyrecnue:EnergyReco 9.6104e-05 0.0015715 0.0297857 0.000422561 0.00395522 100
reco:energyrecnc:EnergyReco 0.0013329 0.00319016 0.0310912 0.0022236 0.00417264 100
reco:energyrecnumurange:EnergyReco 0.000124307 0.0223896 0.6638 0.00181804 0.0796582 100
reco:energyrecnumumcs:EnergyReco 0.0012758 0.0236931 0.679076 0.00255557 0.0814304 100
reco:opdec:Deconvolution 0.0647712 0.296919 0.765588 0.279936 0.139407 100
reco:ophitspe:OpHitFinderDeco 0.111757 1.42781 2.86172 1.40064 0.62363 100
reco:opflash:OpFlashFinder 0.000197777 0.0012665 0.00704127 0.0010232 0.00088842 100
reco:rns:RandomNumberSaver 1.9049e-05 3.09233e-05 0.000454564 2.50775e-05 4.30175e-05 100
reco:anglereconue:NuAngularReco 0.000148754 0.00128686 0.026738 0.000340978 0.00326386 100
reco:anglereconumu:NuAngularReco 5.4873e-05 0.00102639 0.0254322 0.000177371 0.00311535 100
reco:anglereconuepfps:NuAngularReco 0.000102274 0.00119608 0.0280501 0.000298375 0.00333598 100
reco:anglereconumupfps:NuAngularReco 7.5744e-05 0.00115429 0.03281 0.000230557 0.00374342 100
[art]:TriggerResults:TriggerResultInserter 7.952e-06 1.1497e-05 6.612e-05 1.0862e-05 5.80238e-06 100
end_path:out1:RootOutput 3.125e-06 4.40446e-06 2.1763e-05 3.971e-06 1.91989e-06 100
end_path:out1:RootOutput(write) 0.0131588 0.19123 0.768384 0.184524 0.138988 100
========================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 40377.9 MB
Peak resident set size usage (VmHWM): 2585.99 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
Validation_ccnc_atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6390196_832_20231201T210816Z_gen_g4_detsim_hitreco__20240507T223649Z_reco2.root
all-input-dids.txt
atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6390196_832_20231201T210816Z_gen_g4_detsim_hitreco__20240507T223649Z_reco2_reco_2024-11-22T_233329Z.log
atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6390196_832_20231201T210816Z_gen_g4_detsim_hitreco__20240507T223649Z_reco2_reco_data_2024-11-22T_233329Z.root
debugprod.log
jobscript.log
justin-processed-pfns.txt
reco2_hist.root
MyPandoraSettings_Master_Atmos_DUNEFD.xml
MyPandoraSettings_Master_DUNEFD.xml
build_slf7.x86_64
localProducts_larsoft_v09_91_02_e26_prof
setup_env-testreco.sh
srcs
temp.txt
temp2.txt
test
work
