justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 91714.103@justin-prod-sched02.dune.hep.ac.uk

Jobsub ID91714.103@justin-prod-sched02.dune.hep.ac.uk
Workflow ID3932
Stage ID1
User nameismerio@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2024-11-06 03:49:39
SiteUK_Manchester
EntryUBoone_T2_UK_Manchester_ce01
Last heartbeat2024-11-06 07:36:55
From worker nodeHostnamewn6219301.tier2.hep.manchester.ac.uk
cpuinfoAMD EPYC 7702P 64-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit257400 (71 hours)
Inner Apptainer?True
Job statefinished
Allocator namejustin-allocator-pro.dune.hep.ac.uk
Started2024-11-06 03:52:11
Input filesfardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6419482_675_20231203T221640Z_gen_g4_detsim_hitreco__20240509T221336Z_reco2.root
JobscriptExit code0
Real time3h (13452s)
CPU time3h (12747s = 94%)
Outputting started2024-11-06 07:36:23
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/ismerio/03932/1/Validation_ccnc_atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6419482_675_20231203T221640Z_gen_g4_detsim_hitreco__20240509T221336Z_reco2.root
Finished2024-11-06 07:36:55
Saved logsjustin-logs:91714.103-justin-prod-sched02.dune.hep.ac.uk.logs.tgz
List job events     Wrapper job log

Jobscript log (last 10,000 characters)

pmtrajfittc@BeginModule  06-Nov-2024 07:35:58 GMT run: 6419482 subRun: 1 event: 67600
0 enabled hits in AverageDist2 calculation.
%MSG
%MSG-e pma::Track3D:  PMAlgTrajFitter:pmtrajfittc@BeginModule  06-Nov-2024 07:35:58 GMT run: 6419482 subRun: 1 event: 67600
Track empty.
%MSG
%MSG-e pma::Track3D:  PMAlgTrajFitter:pmtrajfittc@BeginModule  06-Nov-2024 07:35:58 GMT run: 6419482 subRun: 1 event: 67600
TuneFullTree failed.
%MSG
%MSG-e pma::Track3D:  PMAlgTrajFitter:pmtrajfittc@BeginModule  06-Nov-2024 07:35:58 GMT run: 6419482 subRun: 1 event: 67600
0 enabled hits in AverageDist2 calculation.
%MSG
%MSG-e pma::Track3D:  PMAlgTrajFitter:pmtrajfittc@BeginModule  06-Nov-2024 07:35:58 GMT run: 6419482 subRun: 1 event: 67600
Track empty.
%MSG
%MSG-e pma::Track3D:  PMAlgTrajFitter:pmtrajfittc@BeginModule  06-Nov-2024 07:35:58 GMT run: 6419482 subRun: 1 event: 67600
TuneFullTree failed.
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrajFitter:pmtrajfittc@BeginModule  06-Nov-2024 07:35:58 GMT run: 6419482 subRun: 1 event: 67600
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  EMShower:emshower@BeginModule  06-Nov-2024 07:36:15 GMT run: 6419482 subRun: 1 event: 67600
Comparing two wires in the same plane: return failure
%MSG
Boundary wire vector sizes: 518, 680, 470
minwire 0: 599
minwire 1: 2736
minwire 2: 5
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 199
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 199
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 199
Classifier summary: 
Output 0: 0.000163227, 0.99979, 4.64798e-05, 
Output 1: 0.0213634, 0.784941, 0.192237, 0.00145918, 
Output 2: 0.989278, 0.0107215, 

531
***Wiener Filter*******ApplyPostfilter***
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 1831!
06-Nov-2024 07:36:21 GMT  Closed output file "atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6419482_675_20231203T221640Z_gen_g4_detsim_hitreco__20240509T221336Z_reco2_reco_data_2024-11-06T_035222Z.root"
06-Nov-2024 07:36:21 GMT  Closed input file "root://golias100.farm.particle.cz:1094/dpm/farm.particle.cz/home/dune/RSE/fardet-hd/6f/0f/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6419482_675_20231203T221640Z_gen_g4_detsim_hitreco__20240509T221336Z_reco2.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              3.58119       130.852       9167.95       16.263        911.979        100    
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                 0.0331762     0.0587606     0.105811      0.0632823     0.0147194       100    
reco:linecluster:LineCluster                           0.248825      0.707697       14.8976      0.276896       2.0998         100    
reco:trajcluster:TrajCluster                           0.0792279      1.48944       44.7068      0.101211       6.42817        100    
reco:pandora:StandardPandora                            1.61403       8.21913       465.771       2.17488       46.7216        100    
reco:pandoraTrack:LArPandoraTrackCreation             0.00018041     0.0118562     0.356427     0.00242404     0.0392688       100    
reco:pandoraShower:LArPandoraModularShowerCreation    0.000224444    0.0178835     0.231177     0.00470083     0.0365586       100    
reco:pandoracalo:Calorimetry                          0.000138311     0.05144       2.8985      0.00189962     0.298882        100    
reco:pandorapid:Chi2ParticleID                        5.1968e-05    0.000270211   0.00300313    0.000189237   0.000344318      100    
reco:trkshowersplit:TrackShowerHits                   0.000186762    0.0427789      1.54142     0.00142839     0.186932        100    
reco:pmtrack:PMAlgTrackMaker                          0.00159821      4.87571       141.481      0.293391       18.4459        100    
reco:pmtrackcalo:Calorimetry                          9.8987e-05     0.0223853      1.08389     0.00130338      0.11324        100    
reco:pmtrackpid:Chi2ParticleID                        4.5366e-05    0.000496419   0.00750952    0.000218041   0.000973691      100    
reco:pmtrajfit:PMAlgTrajFitter                        0.000315234     80.1924       7788.61      0.491882       774.747        100    
reco:pmtrajfitcalo:Calorimetry                        0.000112412    0.0564391      3.28259     0.00277313     0.335966        100    
reco:pmtrajfitpid:Chi2ParticleID                      4.6157e-05    0.000350134    0.0065156    0.000237944   0.000662766      100    
reco:pmtracktc:PMAlgTrackMaker                         0.0019246      5.83046       154.161      0.433039       21.7765        100    
reco:pmtracktccalo:Calorimetry                        0.000121219    0.0133353     0.598562     0.00148797     0.0617477       100    
reco:pmtracktcpid:Chi2ParticleID                      5.0556e-05    0.000447803   0.00587526    0.000248548   0.000742505      100    
reco:pmtrajfittc:PMAlgTrajFitter                      0.000313061     3.60398       133.509      0.592649       14.2095        100    
reco:pmtrajfittccalo:Calorimetry                      0.000113354    0.0375528      1.88325     0.00211314     0.196904        100    
reco:pmtrajfittcpid:Chi2ParticleID                    4.7069e-05    0.000665022    0.0117598    0.000268171   0.00144622       100    
reco:blurredcluster:BlurredClustering                 0.00976108      1.6748        17.3616      0.710173       2.48473        100    
reco:emtrkmichelid:EmTrackMichelId                     0.372026       18.6947       367.945       5.26503       45.9642        100    
reco:emshower:EMShower                                0.000413991    0.0380632     0.384856     0.00548925     0.0696342       100    
reco:cvnmap:CVNMapper                                 3.4685e-05     0.0105831     0.107412     0.00956078     0.0132617       100    
reco:cvneva:CVNEvaluator                              2.6279e-05      1.30638       2.40561       1.61568      0.740406        100    
reco:energyrecnumu:EnergyReco                         0.000417347    0.108648       5.63793     0.00794924     0.578541        100    
reco:energyrecnue:EnergyReco                          0.000185901   0.00387743     0.101768     0.000846051    0.0114792       100    
reco:energyrecnc:EnergyReco                           0.00295942    0.00811693     0.109906     0.00447007     0.0122708       100    
reco:energyrecnumurange:EnergyReco                    0.000285017     0.10371       5.51719     0.00437386     0.568517        100    
reco:energyrecnumumcs:EnergyReco                      0.00308502     0.105531       5.26914     0.00659478     0.547028        100    
reco:opdec:Deconvolution                               0.126803      0.443112      0.758965      0.439409      0.123354        100    
reco:ophitspe:OpHitFinderDeco                           0.45464       2.71565       5.39817       2.71612       1.04777        100    
reco:opflash:OpFlashFinder                            0.000401076   0.00205555    0.00795643     0.0018931    0.00108709       100    
reco:rns:RandomNumberSaver                            2.3824e-05     3.71e-05     0.000494303   3.1204e-05    4.62887e-05      100    
reco:anglereconue:NuAngularReco                       0.000245312   0.00267392     0.0828818    0.000510002   0.00921341       100    
reco:anglereconumu:NuAngularReco                      0.000116459   0.00246587     0.0810947    0.000326266   0.00906027       100    
reco:anglereconuepfps:NuAngularReco                   0.000201059   0.00295089     0.082542     0.000470804   0.00966237       100    
reco:anglereconumupfps:NuAngularReco                  0.000166885   0.00259881     0.0823291    0.000437109   0.00920254       100    
[art]:TriggerResults:TriggerResultInserter            1.2162e-05    1.82094e-05   8.2736e-05    1.6637e-05    7.66957e-06      100    
end_path:out1:RootOutput                               3.296e-06    4.55226e-06   2.6139e-05     4.348e-06    2.20139e-06      100    
end_path:out1:RootOutput(write)                        0.032196      0.389823       2.36118      0.375076      0.304764        100    
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 21927.5 MB
  Peak resident set size usage (VmHWM): 2460.45 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
Validation_ccnc_atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6419482_675_20231203T221640Z_gen_g4_detsim_hitreco__20240509T221336Z_reco2.root
all-input-dids.txt
atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6419482_675_20231203T221640Z_gen_g4_detsim_hitreco__20240509T221336Z_reco2_reco_2024-11-06T_035222Z.log
atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6419482_675_20231203T221640Z_gen_g4_detsim_hitreco__20240509T221336Z_reco2_reco_data_2024-11-06T_035222Z.root
debugprod.log
jobscript.log
justin-processed-pfns.txt
reco2_hist.root
MyPandoraSettings_Master_Atmos_DUNEFD.xml
MyPandoraSettings_Master_DUNEFD.xml
build_slf7.x86_64
localProducts_larsoft_v09_91_02_e26_prof
setup_env-testreco.sh
srcs
work
justIN time: 2024-11-17 10:55:11 UTC       justIN version: 01.01.09