justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 108735.16@justin-prod-sched02.dune.hep.ac.uk

Jobsub ID108735.16@justin-prod-sched02.dune.hep.ac.uk
Workflow ID4213
Stage ID1
User nameismerio@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2024-11-23 04:18:14
SiteUK_Brunel
EntryCMSHTPC_T2_UK_London_Brunel_dc2_22
Last heartbeat2024-11-23 05:20:22
From worker nodeHostnamewn-b2-15-00.brunel.ac.uk
cpuinfoIntel(R) Xeon(R) CPU E5-2630 v3 @ 2.40GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit171000 (47 hours)
Inner Apptainer?True
Job stateoutputting_failed
Allocator namejustin-allocator-pro.dune.hep.ac.uk
Started2024-11-23 04:26:57
Input filesfardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74494002_956_20231201T170222Z_gen_g4_detsim_hitreco__20240507T200432Z_reco2.root
JobscriptExit code0
Real time53m (3191s)
CPU time47m (2852s = 89%)
Outputting started2024-11-23 05:20:09
Output files
Finished2024-11-23 05:20:22
List job events     Wrapper job log

Jobscript log (last 10,000 characters)

5700
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrajFitter:pmtrajfittc@BeginModule  23-Nov-2024 05:19:58 GMT run: 74494002 subRun: 1 event: 95700
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrajFitter:pmtrajfittc@BeginModule  23-Nov-2024 05:19:58 GMT run: 74494002 subRun: 1 event: 95700
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrajFitter:pmtrajfittc@BeginModule  23-Nov-2024 05:19:58 GMT run: 74494002 subRun: 1 event: 95700
1st wire C:0 T:7 P:2 W:629 does not exist (max wire number: 480)
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrajFitter:pmtrajfittc@BeginModule  23-Nov-2024 05:19:58 GMT run: 74494002 subRun: 1 event: 95700
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrajFitter:pmtrajfittc@BeginModule  23-Nov-2024 05:19:58 GMT run: 74494002 subRun: 1 event: 95700
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrajFitter:pmtrajfittc@BeginModule  23-Nov-2024 05:19:58 GMT run: 74494002 subRun: 1 event: 95700
Comparing two wires in the same plane: return failure
%MSG
Boundary wire vector sizes: 369, 193, 192
minwire 0: 836
minwire 1: 2200
minwire 2: 265
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 199
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 199
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 199
Classifier summary: 
Output 0: 0.264726, 0.00631104, 0.728963, 
Output 1: 0.330033, 0.558109, 0.10382, 0.00803756, 
Output 2: 0.324588, 0.675412, 

628
***Wiener Filter*******ApplyPostfilter***
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 2295!
23-Nov-2024 05:20:08 GMT  Closed output file "atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74494002_956_20231201T170222Z_gen_g4_detsim_hitreco__20240507T200432Z_reco2_reco_data_2024-11-23T_042700Z.root"
23-Nov-2024 05:20:08 GMT  Closed input file "root://golias100.farm.particle.cz:1094/dpm/farm.particle.cz/home/dune/RSE/fardet-hd/de/3e/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74494002_956_20231201T170222Z_gen_g4_detsim_hitreco__20240507T200432Z_reco2.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                             0.764118       27.6747       516.006       10.3501       70.2162        100    
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                 0.0217695     0.0501198     0.473275      0.0438546     0.0458607       100    
reco:linecluster:LineCluster                           0.198495      0.737489       41.797       0.236478       4.15883        100    
reco:trajcluster:TrajCluster                           0.0640791      1.28345       59.7431      0.0791968      7.00753        100    
reco:pandora:StandardPandora                           0.0752764      2.16032       42.485        1.24827       5.28011        100    
reco:pandoraTrack:LArPandoraTrackCreation             0.000138529   0.00582814     0.143039     0.00154684     0.0163413       100    
reco:pandoraShower:LArPandoraModularShowerCreation    0.000173692    0.0100633     0.277708     0.00320334     0.0297567       100    
reco:pandoracalo:Calorimetry                          9.7839e-05     0.0228182      1.10306     0.00122822     0.119698        100    
reco:pandorapid:Chi2ParticleID                         3.979e-05    0.000142913   0.00113196    0.000111042   0.000129276      100    
reco:trkshowersplit:TrackShowerHits                   0.000105296    0.023045      0.872741     0.000821709     0.11636        100    
reco:pmtrack:PMAlgTrackMaker                          0.00115739      2.57534       71.6945      0.273509       10.137         100    
reco:pmtrackcalo:Calorimetry                          7.8878e-05     0.0109225     0.431325     0.000922153    0.0481294       100    
reco:pmtrackpid:Chi2ParticleID                        3.3009e-05    0.000252972   0.00334634    0.000130695   0.000432309      100    
reco:pmtrajfit:PMAlgTrajFitter                        0.000190409     1.9802        45.8638      0.416146       5.71298        100    
reco:pmtrajfitcalo:Calorimetry                        7.5678e-05     0.0231196      1.07369     0.00144737     0.116937        100    
reco:pmtrajfitpid:Chi2ParticleID                      3.2811e-05    0.000155187   0.000885084   0.000115531   0.000128248      100    
reco:pmtracktc:PMAlgTrackMaker                        0.00101074      3.11936       97.1564      0.244367       12.7543        100    
reco:pmtracktccalo:Calorimetry                        7.8532e-05    0.00738167     0.250583     0.000935451    0.0282125       100    
reco:pmtracktcpid:Chi2ParticleID                      3.3078e-05    0.000238997    0.0027346    0.000135587   0.000366314      100    
reco:pmtrajfittc:PMAlgTrajFitter                      0.000174561     1.3097        23.7862      0.322251       3.43105        100    
reco:pmtrajfittccalo:Calorimetry                      7.5597e-05     0.0176483     0.587437     0.00119872     0.0782657       100    
reco:pmtrajfittcpid:Chi2ParticleID                    3.3445e-05    0.000328991    0.0041466    0.000148058   0.000587492      100    
reco:blurredcluster:BlurredClustering                 8.5112e-05      1.03524       8.59776      0.548927       1.45953        100    
reco:emtrkmichelid:EmTrackMichelId                     0.0267671      10.7616       142.115       3.66164       21.9422        100    
reco:emshower:EMShower                                0.00023481     0.0297859     0.252575     0.00848271     0.0470675       100    
reco:cvnmap:CVNMapper                                 2.4313e-05     0.0105216     0.0389913    0.00934327    0.00843699       100    
reco:cvneva:CVNEvaluator                              1.8652e-05     0.514435      0.984136      0.658984      0.291773        100    
reco:energyrecnumu:EnergyReco                         0.000280283    0.0176651     0.193416     0.00361063     0.035005        100    
reco:energyrecnue:EnergyReco                          0.000154954    0.0019159     0.038107     0.000551384   0.00463381       100    
reco:energyrecnc:EnergyReco                           0.00201046     0.0038704     0.0421049    0.00243903    0.00495029       100    
reco:energyrecnumurange:EnergyReco                    0.000160837    0.016087      0.190289      0.0022973     0.034498        100    
reco:energyrecnumumcs:EnergyReco                      0.00155134     0.0173742     0.198297     0.00332347     0.0346389       100    
reco:opdec:Deconvolution                               0.0768876      0.32625      0.578201      0.340912      0.0990873       100    
reco:ophitspe:OpHitFinderDeco                          0.247566       1.37542       2.44219       1.42072      0.506992        100    
reco:opflash:OpFlashFinder                            0.000210159   0.00120051    0.00377559    0.00115244    0.000624145      100    
reco:rns:RandomNumberSaver                            1.5971e-05    2.61335e-05   0.000256036   2.16005e-05   2.47376e-05      100    
reco:anglereconue:NuAngularReco                       0.000104843   0.00142109     0.0353736    0.000292261   0.00415445       100    
reco:anglereconumu:NuAngularReco                      8.2828e-05    0.00134759     0.0348081    0.000225883   0.00411034       100    
reco:anglereconuepfps:NuAngularReco                   0.000125832   0.00146835     0.0350382    0.000317354   0.00418834       100    
reco:anglereconumupfps:NuAngularReco                  0.000119315   0.00142707     0.0349842    0.000295711   0.00414226       100    
[art]:TriggerResults:TriggerResultInserter             9.841e-06    1.23602e-05   3.8132e-05    1.0958e-05    4.27895e-06      100    
end_path:out1:RootOutput                               2.395e-06    4.48311e-06   2.1085e-05     3.34e-06     2.40116e-06      100    
end_path:out1:RootOutput(write)                        0.0136766     0.218288      0.994735      0.220112      0.150791        100    
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 8465.99 MB
  Peak resident set size usage (VmHWM): 2366.28 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
Validation_ccnc_atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74494002_956_20231201T170222Z_gen_g4_detsim_hitreco__20240507T200432Z_reco2.root
all-input-dids.txt
atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74494002_956_20231201T170222Z_gen_g4_detsim_hitreco__20240507T200432Z_reco2_reco_2024-11-23T_042700Z.log
atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74494002_956_20231201T170222Z_gen_g4_detsim_hitreco__20240507T200432Z_reco2_reco_data_2024-11-23T_042700Z.root
debugprod.log
jobscript.log
justin-processed-pfns.txt
reco2_hist.root
MyPandoraSettings_Master_Atmos_DUNEFD.xml
MyPandoraSettings_Master_DUNEFD.xml
build_slf7.x86_64
localProducts_larsoft_v09_91_02_e26_prof
setup_env-testreco.sh
srcs
temp.txt
temp2.txt
test
work
justIN time: 2024-11-23 11:59:41 UTC       justIN version: 01.01.09