justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 301749.13@justin-prod-sched01.dune.hep.ac.uk

Jobsub ID301749.13@justin-prod-sched01.dune.hep.ac.uk
Workflow ID4213
Stage ID1
User nameismerio@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2024-11-23 05:12:39
SiteUK_Lancaster
EntryUBoone_UK_Lancaster_HEC_grendel_ce02
Last heartbeat2024-11-23 06:03:31
From worker nodeHostnamecomp20-05
cpuinfoIntel(R) Xeon(R) Gold 6148 CPU @ 2.40GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit257400 (71 hours)
Inner Apptainer?True
Job stateoutputting_failed
Allocator namejustin-allocator-pro.dune.hep.ac.uk
Started2024-11-23 05:14:00
Input filesfardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_66049673_417_20231207T154254Z_gen_g4_detsim_hitreco__20240510T044212Z_reco2.root
JobscriptExit code0
Real time49m (2957s)
CPU time45m (2709s = 91%)
Outputting started2024-11-23 06:03:18
Output files
Finished2024-11-23 06:03:31
List job events     Wrapper job log

Jobscript log (last 10,000 characters)

WireIDIntersectionCheck:  PMAlgTrajFitter:pmtrajfittc@BeginModule  23-Nov-2024 06:03:08 GMT run: 66049673 subRun: 1 event: 41800
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrajFitter:pmtrajfittc@BeginModule  23-Nov-2024 06:03:08 GMT run: 66049673 subRun: 1 event: 41800
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrajFitter:pmtrajfittc@BeginModule  23-Nov-2024 06:03:08 GMT run: 66049673 subRun: 1 event: 41800
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrajFitter:pmtrajfittc@BeginModule  23-Nov-2024 06:03:08 GMT run: 66049673 subRun: 1 event: 41800
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrajFitter:pmtrajfittc@BeginModule  23-Nov-2024 06:03:08 GMT run: 66049673 subRun: 1 event: 41800
1st wire C:0 T:10 P:2 W:604 does not exist (max wire number: 480)
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrajFitter:pmtrajfittc@BeginModule  23-Nov-2024 06:03:08 GMT run: 66049673 subRun: 1 event: 41800
Comparing two wires in the same plane: return failure
%MSG
Boundary wire vector sizes: 95, 95, 85
minwire 0: 1277
minwire 1: 1689
minwire 2: 803
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 199
Used alternate method to get min and max wires due to vertex determination failure: 93, 292
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 199
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 199
Classifier summary: 
Output 0: 0.99905, 7.27493e-05, 0.000877452, 
Output 1: 0.189351, 0.795708, 0.0141384, 0.000802651, 
Output 2: 0.996715, 0.00328511, 

604
***Wiener Filter*******ApplyPostfilter***
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 2164!
23-Nov-2024 06:03:17 GMT  Closed output file "atmnu_max_weighted_randompolicy_dune10kt_1x2x6_66049673_417_20231207T154254Z_gen_g4_detsim_hitreco__20240510T044212Z_reco2_reco_data_2024-11-23T_051403Z.root"
23-Nov-2024 06:03:17 GMT  Closed input file "root://xrootd1.esc.qmul.ac.uk:1094//dune/RSE/fardet-hd/e9/3e/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_66049673_417_20231207T154254Z_gen_g4_detsim_hitreco__20240510T044212Z_reco2.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              1.89558       27.5293       681.512       10.0075       74.3163        100    
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                 0.0102152     0.119732      0.595224      0.0397372     0.141743        100    
reco:linecluster:LineCluster                           0.0892735     0.508983       8.57257       0.12054       1.19166        100    
reco:trajcluster:TrajCluster                           0.0279427     0.719579       35.2992      0.038007       3.83036        100    
reco:pandora:StandardPandora                            1.13741       2.39921       75.2449       1.26528       7.50281        100    
reco:pandoraTrack:LArPandoraTrackCreation             0.000149216   0.00599484     0.220462     0.00124989     0.0224277       100    
reco:pandoraShower:LArPandoraModularShowerCreation    0.000180848    0.0148824     0.450738     0.00231384     0.0535657       100    
reco:pandoracalo:Calorimetry                          0.000108121    0.038048       2.71517     0.000790613     0.27412        100    
reco:pandorapid:Chi2ParticleID                        3.9631e-05    0.00015307    0.00126452    0.000117257   0.000150302      100    
reco:trkshowersplit:TrackShowerHits                    9.156e-05     0.0148535     0.766462     0.000544273    0.081504        100    
reco:pmtrack:PMAlgTrackMaker                          0.00116072      1.79194       64.4964      0.137938       7.27879        100    
reco:pmtrackcalo:Calorimetry                          8.4499e-05     0.0115701     0.622171     0.000718847    0.0633006       100    
reco:pmtrackpid:Chi2ParticleID                         3.358e-05    0.00024337    0.00382848    0.000133785   0.000445511      100    
reco:pmtrajfit:PMAlgTrajFitter                        0.000197037     1.85488       81.5612       0.2085        8.41817        100    
reco:pmtrajfitcalo:Calorimetry                        6.5418e-05     0.0373688      2.62466      0.0011398     0.264898        100    
reco:pmtrajfitpid:Chi2ParticleID                       2.899e-05    0.000164163   0.00089588    0.000135722   0.000122034      100    
reco:pmtracktc:PMAlgTrackMaker                        0.00108506      2.11031       79.3843      0.154932       8.82005        100    
reco:pmtracktccalo:Calorimetry                         7.923e-05    0.00793649     0.314805     0.000826563    0.0333437       100    
reco:pmtracktcpid:Chi2ParticleID                       3.19e-05     0.00022712     0.0027888    0.00013163    0.000335435      100    
reco:pmtrajfittc:PMAlgTrajFitter                      0.000187712     1.27436       43.2861      0.243466       4.73875        100    
reco:pmtrajfittccalo:Calorimetry                      6.7487e-05     0.0178459     0.978595     0.000889582     0.10085        100    
reco:pmtrajfittcpid:Chi2ParticleID                    2.8143e-05    0.000304698    0.0048744    0.000142471   0.000580917      100    
reco:blurredcluster:BlurredClustering                 0.00822723      1.06961       12.9232      0.434972       1.6771         100    
reco:emtrkmichelid:EmTrackMichelId                     0.144324       12.0048       260.467       2.89725       30.2875        100    
reco:emshower:EMShower                                0.000256268    0.026252      0.466001     0.00396951      0.05642        100    
reco:cvnmap:CVNMapper                                 2.1532e-05     0.0159557     0.0689652     0.013612      0.0133512       100    
reco:cvneva:CVNEvaluator                              1.7917e-05     0.517811      0.809263      0.697027      0.315444        100    
reco:energyrecnumu:EnergyReco                         0.000285141    0.0381862      1.90153     0.00415231     0.194115        100    
reco:energyrecnue:EnergyReco                          0.000143383   0.00189988     0.0639436    0.000458672    0.0065031       100    
reco:energyrecnc:EnergyReco                           0.00200732    0.00369848     0.0683107    0.00232024    0.00675761       100    
reco:energyrecnumurange:EnergyReco                    0.000223281    0.0361941      1.92344     0.00208252     0.196204        100    
reco:energyrecnumumcs:EnergyReco                      0.00158572     0.0374261      1.91538      0.0028248     0.195302        100    
reco:opdec:Deconvolution                               0.0646275     0.420221       3.38996      0.262899      0.512822        100    
reco:ophitspe:OpHitFinderDeco                          0.251047       2.2017        4.36369       2.16882      0.907176        100    
reco:opflash:OpFlashFinder                            0.000259942   0.00130461    0.00404267    0.00119357    0.000614005      100    
reco:rns:RandomNumberSaver                             1.973e-05    3.43552e-05   0.000252918   2.84665e-05   2.47153e-05      100    
reco:anglereconue:NuAngularReco                       0.000154828   0.00148016     0.0560453    0.000350115   0.00566922       100    
reco:anglereconumu:NuAngularReco                      6.9196e-05    0.00125084     0.0539043    0.000186094   0.00545412       100    
reco:anglereconuepfps:NuAngularReco                   0.000117954   0.00137933     0.0546292    0.000300203   0.00551678       100    
reco:anglereconumupfps:NuAngularReco                  0.000101513   0.00131808     0.054475     0.000255518   0.00550363       100    
[art]:TriggerResults:TriggerResultInserter             8.959e-06    1.22267e-05   4.4315e-05    1.0817e-05    4.01876e-06      100    
end_path:out1:RootOutput                               2.796e-06    5.2322e-06    1.6457e-05    3.6705e-06    2.69918e-06      100    
end_path:out1:RootOutput(write)                        0.0142076      0.21839       1.30217      0.219733      0.165407        100    
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 9663.73 MB
  Peak resident set size usage (VmHWM): 2582.27 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
Validation_ccnc_atmnu_max_weighted_randompolicy_dune10kt_1x2x6_66049673_417_20231207T154254Z_gen_g4_detsim_hitreco__20240510T044212Z_reco2.root
all-input-dids.txt
atmnu_max_weighted_randompolicy_dune10kt_1x2x6_66049673_417_20231207T154254Z_gen_g4_detsim_hitreco__20240510T044212Z_reco2_reco_2024-11-23T_051403Z.log
atmnu_max_weighted_randompolicy_dune10kt_1x2x6_66049673_417_20231207T154254Z_gen_g4_detsim_hitreco__20240510T044212Z_reco2_reco_data_2024-11-23T_051403Z.root
debugprod.log
jobscript.log
justin-processed-pfns.txt
reco2_hist.root
MyPandoraSettings_Master_Atmos_DUNEFD.xml
MyPandoraSettings_Master_DUNEFD.xml
build_slf7.x86_64
localProducts_larsoft_v09_91_02_e26_prof
setup_env-testreco.sh
srcs
temp.txt
temp2.txt
test
work
justIN time: 2024-11-23 11:58:59 UTC       justIN version: 01.01.09