justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 301694.137@justin-prod-sched01.dune.hep.ac.uk

Jobsub ID301694.137@justin-prod-sched01.dune.hep.ac.uk
Workflow ID4213
Stage ID1
User nameismerio@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2024-11-23 04:07:23
SiteUK_Durham
EntryDUNE_UK_SGridDurham_ce3
Last heartbeat2024-11-23 04:57:21
From worker nodeHostnamen236.dur.scotgrid.ac.uk
cpuinfoAMD EPYC 7352 24-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit171000 (47 hours)
Inner Apptainer?True
Job stateoutputting_failed
Allocator namejustin-allocator-pro.dune.hep.ac.uk
Started2024-11-23 04:08:24
Input filesfardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6494524_52_20231208T063508Z_gen_g4_detsim_hitreco__20240510T055640Z_reco2.root
JobscriptExit code0
Real time48m (2920s)
CPU time38m (2284s = 78%)
Outputting started2024-11-23 04:57:05
Output files
Finished2024-11-23 04:57:21
List job events     Wrapper job log

Jobscript log (last 10,000 characters)

ubRun: 1 event: 5300
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrajFitter:pmtrajfittc@BeginModule  23-Nov-2024 04:56:53 GMT run: 6494524 subRun: 1 event: 5300
1st wire C:0 T:12 P:2 W:835 does not exist (max wire number: 480)
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrajFitter:pmtrajfittc@BeginModule  23-Nov-2024 04:56:53 GMT run: 6494524 subRun: 1 event: 5300
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrajFitter:pmtrajfittc@BeginModule  23-Nov-2024 04:56:53 GMT run: 6494524 subRun: 1 event: 5300
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrajFitter:pmtrajfittc@BeginModule  23-Nov-2024 04:56:53 GMT run: 6494524 subRun: 1 event: 5300
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrajFitter:pmtrajfittc@BeginModule  23-Nov-2024 04:56:53 GMT run: 6494524 subRun: 1 event: 5300
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrajFitter:pmtrajfittc@BeginModule  23-Nov-2024 04:56:53 GMT run: 6494524 subRun: 1 event: 5300
Comparing two wires in the same plane: return failure
%MSG
Boundary wire vector sizes: 436, 312, 251
minwire 0: 1817
minwire 1: 1369
minwire 2: 1423
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 199
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 199
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 199
Classifier summary: 
Output 0: 0.0107277, 0.00458162, 0.984691, 
Output 1: 0.622853, 0.335047, 0.0333011, 0.00879893, 
Output 2: 0.538755, 0.461245, 

647
***Wiener Filter*******ApplyPostfilter***
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 2618!
23-Nov-2024 04:57:04 GMT  Closed output file "atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6494524_52_20231208T063508Z_gen_g4_detsim_hitreco__20240510T055640Z_reco2_reco_data_2024-11-23T_040829Z.root"
23-Nov-2024 04:57:04 GMT  Closed input file "root://golias100.farm.particle.cz:1094/dpm/farm.particle.cz/home/dune/RSE/fardet-hd/c7/34/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6494524_52_20231208T063508Z_gen_g4_detsim_hitreco__20240510T055640Z_reco2.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              1.12955       23.0446       328.236       10.4994       43.5961        100    
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                 0.0343459     0.0690691     0.112513      0.0782084     0.0208076       100    
reco:linecluster:LineCluster                           0.200459      0.532726       11.0154      0.368295       1.0721         100    
reco:trajcluster:TrajCluster                           0.100495      0.673962       23.4855      0.110165       2.83171        100    
reco:pandora:StandardPandora                           0.108148       1.75899       12.972        1.34489       1.57959        100    
reco:pandoraTrack:LArPandoraTrackCreation             0.000128402   0.00409692     0.0430867    0.00114659    0.00781127       100    
reco:pandoraShower:LArPandoraModularShowerCreation    0.000176152    0.0065664     0.0760298    0.00226914     0.0112852       100    
reco:pandoracalo:Calorimetry                          9.6492e-05     0.0103224     0.251531     0.000793178    0.0323194       100    
reco:pandorapid:Chi2ParticleID                        3.8903e-05    0.000201944   0.00340538    0.000124464   0.000404898      100    
reco:trkshowersplit:TrackShowerHits                   6.1335e-05     0.0146469     0.513202     0.000697232    0.0597701       100    
reco:pmtrack:PMAlgTrackMaker                          0.00131749      1.80797       53.6491       0.1601        6.44931        100    
reco:pmtrackcalo:Calorimetry                          6.3079e-05    0.00662177     0.162267     0.000807645    0.0210543       100    
reco:pmtrackpid:Chi2ParticleID                         2.624e-05    0.000280651   0.00284419    0.000129122   0.000444915      100    
reco:pmtrajfit:PMAlgTrajFitter                        0.000192652     1.63827       33.0408      0.181918       4.31991        100    
reco:pmtrajfitcalo:Calorimetry                        5.6607e-05     0.010929      0.271096     0.00122663     0.0340121       100    
reco:pmtrajfitpid:Chi2ParticleID                      2.4957e-05    0.000183352   0.00107047    0.000137388   0.000163922      100    
reco:pmtracktc:PMAlgTrackMaker                        0.00104607      2.25787       59.1243       0.19181       7.90708        100    
reco:pmtracktccalo:Calorimetry                        6.2538e-05    0.00382345     0.0496447    0.000786521   0.00808645       100    
reco:pmtracktcpid:Chi2ParticleID                      2.5969e-05    0.000242496    0.001286     0.000139167   0.000277891      100    
reco:pmtrajfittc:PMAlgTrajFitter                      0.000153158     1.10321       16.378       0.240005       2.3867         100    
reco:pmtrajfittccalo:Calorimetry                      6.1105e-05    0.00905475     0.208855     0.000973201    0.0269877       100    
reco:pmtrajfittcpid:Chi2ParticleID                    2.5448e-05    0.000337956   0.00313197    0.000154997   0.000491354      100    
reco:blurredcluster:BlurredClustering                  6.89e-05      0.975415       5.56742      0.529631       1.17034        100    
reco:emtrkmichelid:EmTrackMichelId                    0.000101822     9.03545       107.003       2.90957       16.7298        100    
reco:emshower:EMShower                                0.000158648    0.0244912     0.323761     0.00505371     0.0458704       100    
reco:cvnmap:CVNMapper                                 1.8545e-05     0.0129751     0.0687519     0.015514      0.0115527       100    
reco:cvneva:CVNEvaluator                              1.4517e-05     0.615989       1.39888      0.766163      0.442408        100    
reco:energyrecnumu:EnergyReco                         0.000362632    0.0277143      0.60645     0.00397046     0.0874263       100    
reco:energyrecnue:EnergyReco                          0.000133371   0.00135807     0.0114535    0.00051534    0.00204915       100    
reco:energyrecnc:EnergyReco                            0.0017556    0.00350802     0.0158391    0.00297058    0.00242248       100    
reco:energyrecnumurange:EnergyReco                    0.000188836    0.0259277      0.60293      0.0023527     0.0866738       100    
reco:energyrecnumumcs:EnergyReco                      0.00195001     0.0271619     0.602636     0.00329418     0.0866569       100    
reco:opdec:Deconvolution                                0.10783      0.414947      0.701245      0.409873      0.131302        100    
reco:ophitspe:OpHitFinderDeco                           0.30045       1.73851       3.66571       1.69651      0.799803        100    
reco:opflash:OpFlashFinder                            0.000143601    0.0014721    0.00756159    0.00135449    0.000873063      100    
reco:rns:RandomNumberSaver                            1.8244e-05     4.871e-05    0.000463222   4.2901e-05    4.44999e-05      100    
reco:anglereconue:NuAngularReco                       0.00015434    0.000927019   0.00939096    0.000391421   0.00159797       100    
reco:anglereconumu:NuAngularReco                      6.4722e-05    0.000685868   0.00925891    0.000186486   0.00150718       100    
reco:anglereconuepfps:NuAngularReco                   0.000120777   0.000886632   0.00951325    0.000317573   0.00164749       100    
reco:anglereconumupfps:NuAngularReco                  9.8155e-05    0.000748035    0.0095331    0.000243333   0.00151408       100    
[art]:TriggerResults:TriggerResultInserter             8.486e-06    1.36222e-05   7.0863e-05    1.2549e-05    6.79068e-06      100    
end_path:out1:RootOutput                               2.936e-06    8.66838e-06   2.8824e-05    7.7195e-06    4.05761e-06      100    
end_path:out1:RootOutput(write)                       0.00817967     0.224827      0.818066      0.233739      0.156551        100    
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 16562.2 MB
  Peak resident set size usage (VmHWM): 1979.03 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
Validation_ccnc_atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6494524_52_20231208T063508Z_gen_g4_detsim_hitreco__20240510T055640Z_reco2.root
all-input-dids.txt
atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6494524_52_20231208T063508Z_gen_g4_detsim_hitreco__20240510T055640Z_reco2_reco_2024-11-23T_040829Z.log
atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6494524_52_20231208T063508Z_gen_g4_detsim_hitreco__20240510T055640Z_reco2_reco_data_2024-11-23T_040829Z.root
debugprod.log
jobscript.log
justin-processed-pfns.txt
reco2_hist.root
MyPandoraSettings_Master_Atmos_DUNEFD.xml
MyPandoraSettings_Master_DUNEFD.xml
build_slf7.x86_64
localProducts_larsoft_v09_91_02_e26_prof
setup_env-testreco.sh
srcs
temp.txt
temp2.txt
test
work
justIN time: 2024-11-23 12:17:39 UTC       justIN version: 01.01.09