justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 108744.82@justin-prod-sched02.dune.hep.ac.uk

Jobsub ID108744.82@justin-prod-sched02.dune.hep.ac.uk
Workflow ID4213
Stage ID1
User nameismerio@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2024-11-23 04:38:45
SiteUK_Lancaster
EntryUBoone_UK_Lancaster_HEC_grendel_ce02
Last heartbeat2024-11-23 06:18:53
From worker nodeHostnamecomp23-06
cpuinfoIntel(R) Xeon(R) Gold 6248 CPU @ 2.50GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit257400 (71 hours)
Inner Apptainer?True
Job stateoutputting_failed
Allocator namejustin-allocator-pro.dune.hep.ac.uk
Started2024-11-23 04:44:57
Input filesfardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50604274_641_20231207T030000Z_gen_g4_detsim_hitreco__20240510T050837Z_reco2.root
JobscriptExit code0
Real time1h (5620s)
CPU time1h (5385s = 95%)
Outputting started2024-11-23 06:18:38
Output files
Finished2024-11-23 06:18:53
List job events     Wrapper job log

Jobscript log (last 10,000 characters)

n the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrajFitter:pmtrajfittc@BeginModule  23-Nov-2024 06:18:30 GMT run: 50604274 subRun: 1 event: 64200
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrajFitter:pmtrajfittc@BeginModule  23-Nov-2024 06:18:30 GMT run: 50604274 subRun: 1 event: 64200
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  EMShower:emshower@BeginModule  23-Nov-2024 06:18:32 GMT run: 50604274 subRun: 1 event: 64200
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  EMShower:emshower@BeginModule  23-Nov-2024 06:18:32 GMT run: 50604274 subRun: 1 event: 64200
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  EMShower:emshower@BeginModule  23-Nov-2024 06:18:32 GMT run: 50604274 subRun: 1 event: 64200
Comparing two wires in the same plane: return failure
%MSG
Boundary wire vector sizes: 59, 61, 39
minwire 0: 1215
minwire 1: 1684
minwire 2: 917
Used alternate method to get min and max wires due to vertex determination failure: 168, 367
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 199
Used alternate method to get min and max wires due to vertex determination failure: 313, 512
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 199
Used alternate method to get min and max wires due to vertex determination failure: 86, 285
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 199
Classifier summary: 
Output 0: 0.99842, 0.000158417, 0.00142106, 
Output 1: 0.17502, 0.747947, 0.0758643, 0.00116883, 
Output 2: 0.996345, 0.00365469, 

592
***Wiener Filter*******ApplyPostfilter***
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 2222!
23-Nov-2024 06:18:37 GMT  Closed output file "atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50604274_641_20231207T030000Z_gen_g4_detsim_hitreco__20240510T050837Z_reco2_reco_data_2024-11-23T_044502Z.root"
23-Nov-2024 06:18:37 GMT  Closed input file "root://xrootd.echo.stfc.ac.uk:1094/dune:/protodune/RSE/fardet-hd/89/90/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50604274_641_20231207T030000Z_gen_g4_detsim_hitreco__20240510T050837Z_reco2.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              1.1583        54.1924       1661.79       9.75896       192.064        100    
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                 0.0224306     0.104746       0.37175      0.0795731     0.0752334       100    
reco:linecluster:LineCluster                           0.0828044      1.55654       53.3199      0.289092       6.66378        100    
reco:trajcluster:TrajCluster                           0.0312384      4.14283       129.821      0.117345       19.8848        100    
reco:pandora:StandardPandora                           0.0675125      11.2063       756.258       1.20898       76.5266        100    
reco:pandoraTrack:LArPandoraTrackCreation             0.00013309     0.0116247     0.396009     0.00165825     0.0447274       100    
reco:pandoraShower:LArPandoraModularShowerCreation    0.000181752    0.0125295     0.136925     0.00257886     0.0244346       100    
reco:pandoracalo:Calorimetry                          9.7572e-05     0.0825128      4.08056     0.000865275    0.443249        100    
reco:pandorapid:Chi2ParticleID                        3.8301e-05    0.000160722   0.00173315    0.000110454   0.000202419      100    
reco:trkshowersplit:TrackShowerHits                   9.6086e-05     0.0506702      1.99136     0.000578778    0.237678        100    
reco:pmtrack:PMAlgTrackMaker                           0.001049       4.30253       98.0093      0.117472       15.9166        100    
reco:pmtrackcalo:Calorimetry                          7.8407e-05     0.0235281      1.03375     0.000739651     0.11463        100    
reco:pmtrackpid:Chi2ParticleID                        2.9701e-05    0.000314753    0.0043602    0.000118603   0.000619967      100    
reco:pmtrajfit:PMAlgTrajFitter                        0.000185089     3.44229       96.645       0.159499       13.2193        100    
reco:pmtrajfitcalo:Calorimetry                         6.094e-05     0.0825787      4.03082     0.00111747     0.440134        100    
reco:pmtrajfitpid:Chi2ParticleID                      2.5314e-05    0.000177407   0.00161996    0.000130466   0.00019575       100    
reco:pmtracktc:PMAlgTrackMaker                        0.000987957     4.90104       119.225      0.143891       17.5679        100    
reco:pmtracktccalo:Calorimetry                        7.1646e-05     0.0111937     0.392601     0.000821401    0.0444021       100    
reco:pmtracktcpid:Chi2ParticleID                      2.8993e-05    0.000261296   0.00297227    0.000124175   0.000418997      100    
reco:pmtrajfittc:PMAlgTrajFitter                      0.000158906     2.31566       63.9736      0.201277       7.95629        100    
reco:pmtrajfittccalo:Calorimetry                       6.149e-05     0.0375112      1.49424     0.00086973     0.169941        100    
reco:pmtrajfittcpid:Chi2ParticleID                    2.5595e-05    0.000391985   0.00511628    0.000127646   0.000761385      100    
reco:blurredcluster:BlurredClustering                 6.0403e-05      1.14551       13.0429      0.616677       1.81583        100    
reco:emtrkmichelid:EmTrackMichelId                    9.7006e-05      16.9614       299.952       3.12588       41.3918        100    
reco:emshower:EMShower                                0.00016316     0.0250304     0.529887     0.00287412     0.0663842       100    
reco:cvnmap:CVNMapper                                  1.712e-05     0.0140073     0.0444881     0.0126457     0.0106229       100    
reco:cvneva:CVNEvaluator                              1.4618e-05     0.603137      0.890743      0.798276      0.358938        100    
reco:energyrecnumu:EnergyReco                         0.000262216    0.0916075      3.84251     0.00562005     0.410933        100    
reco:energyrecnue:EnergyReco                          0.000134344   0.00355053      0.11583     0.000489796    0.0132969       100    
reco:energyrecnc:EnergyReco                           0.00173282    0.00515947     0.116129     0.00217504     0.0132554       100    
reco:energyrecnumurange:EnergyReco                    0.000223625    0.0886662      3.8439      0.00203169     0.410862        100    
reco:energyrecnumumcs:EnergyReco                      0.000972294    0.0914311      3.99218     0.00275544     0.424591        100    
reco:opdec:Deconvolution                               0.0857619     0.393332      0.947743      0.374745       0.13887        100    
reco:ophitspe:OpHitFinderDeco                          0.307077       2.24275       4.27373       2.15173      0.832095        100    
reco:opflash:OpFlashFinder                            0.000304215   0.00137574    0.00440369    0.00127259    0.000595946      100    
reco:rns:RandomNumberSaver                             1.994e-05    3.27854e-05   0.000294615   2.6725e-05    2.77813e-05      100    
reco:anglereconue:NuAngularReco                       0.000152234   0.00286629      0.10528     0.000348948    0.0118787       100    
reco:anglereconumu:NuAngularReco                      6.3955e-05     0.0026181     0.103671     0.000196924    0.0116712       100    
reco:anglereconuepfps:NuAngularReco                   0.000113423   0.00275691     0.103785     0.000302569    0.0117214       100    
reco:anglereconumupfps:NuAngularReco                  9.1065e-05    0.00269927     0.104077     0.00027603     0.0117188       100    
[art]:TriggerResults:TriggerResultInserter             8.896e-06    1.19992e-05   4.5762e-05    1.0704e-05    4.14578e-06      100    
end_path:out1:RootOutput                               2.672e-06    5.00046e-06   1.7381e-05    3.8905e-06    2.48592e-06      100    
end_path:out1:RootOutput(write)                        0.0139698     0.227281       1.35275      0.204654      0.194713        100    
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 9991.19 MB
  Peak resident set size usage (VmHWM): 2824.35 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
Validation_ccnc_atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50604274_641_20231207T030000Z_gen_g4_detsim_hitreco__20240510T050837Z_reco2.root
all-input-dids.txt
atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50604274_641_20231207T030000Z_gen_g4_detsim_hitreco__20240510T050837Z_reco2_reco_2024-11-23T_044502Z.log
atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50604274_641_20231207T030000Z_gen_g4_detsim_hitreco__20240510T050837Z_reco2_reco_data_2024-11-23T_044502Z.root
debugprod.log
jobscript.log
justin-processed-pfns.txt
reco2_hist.root
MyPandoraSettings_Master_Atmos_DUNEFD.xml
MyPandoraSettings_Master_DUNEFD.xml
build_slf7.x86_64
localProducts_larsoft_v09_91_02_e26_prof
setup_env-testreco.sh
srcs
temp.txt
temp2.txt
test
work
justIN time: 2024-11-23 11:09:11 UTC       justIN version: 01.01.09