justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 108510.117@justin-prod-sched02.dune.hep.ac.uk

Jobsub ID108510.117@justin-prod-sched02.dune.hep.ac.uk
Workflow ID4213
Stage ID1
User nameismerio@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2024-11-22 22:08:08
SiteES_PIC
EntryDUNE_T1_ES_PIC_ce13-multicore
Last heartbeat2024-11-22 23:54:19
From worker nodeHostnametds218.pic.es
cpuinfoIntel(R) Xeon(R) CPU E5-2640 v3 @ 2.60GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit216000 (60 hours)
Inner Apptainer?True
Job statefinished
Allocator namejustin-allocator-pro.dune.hep.ac.uk
Started2024-11-22 22:25:28
Input filesfardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74510902_439_20231203T101845Z_gen_g4_detsim_hitreco__20240508T075358Z_reco2.root
JobscriptExit code0
Real time1h (5205s)
CPU time1h (5186s = 99%)
Outputting started2024-11-22 23:52:14
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/ismerio/04213/1/Validation_ccnc_atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74510902_439_20231203T101845Z_gen_g4_detsim_hitreco__20240508T075358Z_reco2.root
Finished2024-11-22 23:54:19
Saved logsjustin-logs:108510.117-justin-prod-sched02.dune.hep.ac.uk.logs.tgz
List job events     Wrapper job log

Jobscript log (last 10,000 characters)

his element does not exist in the element holder
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtrack@BeginModule  23-Nov-2024 00:52:09 CET run: 74510902 subRun: 1 event: 44000
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtrack@BeginModule  23-Nov-2024 00:52:09 CET run: 74510902 subRun: 1 event: 44000
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtrack@BeginModule  23-Nov-2024 00:52:09 CET run: 74510902 subRun: 1 event: 44000
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtrack@BeginModule  23-Nov-2024 00:52:09 CET run: 74510902 subRun: 1 event: 44000
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtrack@BeginModule  23-Nov-2024 00:52:09 CET run: 74510902 subRun: 1 event: 44000
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrajFitter:pmtrajfit@BeginModule  23-Nov-2024 00:52:09 CET run: 74510902 subRun: 1 event: 44000
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  23-Nov-2024 00:52:09 CET run: 74510902 subRun: 1 event: 44000
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrajFitter:pmtrajfittc@BeginModule  23-Nov-2024 00:52:09 CET run: 74510902 subRun: 1 event: 44000
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  EMShower:emshower@BeginModule  23-Nov-2024 00:52:10 CET run: 74510902 subRun: 1 event: 44000
Comparing two wires in the same plane: return failure
%MSG
212
***Wiener Filter*******ApplyPostfilter***
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 306!
23-Nov-2024 00:52:12 CET  Closed output file "atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74510902_439_20231203T101845Z_gen_g4_detsim_hitreco__20240508T075358Z_reco2_reco_data_2024-11-22T_222533Z.root"
23-Nov-2024 00:52:12 CET  Closed input file "root://xrootd.pic.es:1094/pnfs/pic.es/data/dune/RSE/fardet-hd/7c/1c/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74510902_439_20231203T101845Z_gen_g4_detsim_hitreco__20240508T075358Z_reco2.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                             0.236445       50.6935       960.144       14.5122       148.986        100    
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.00121556     0.016912      0.037098      0.0193875    0.00799118       100    
reco:linecluster:LineCluster                          0.00179809      1.50511       97.7081      0.030753       10.1747        100    
reco:trajcluster:TrajCluster                          0.00109467      3.17675       178.905      0.0237533      18.9659        100    
reco:pandora:StandardPandora                           0.0177434      7.72426       431.047       2.11195       43.3798        100    
reco:pandoraTrack:LArPandoraTrackCreation             0.000154538   0.00989801     0.168946     0.00266544     0.0248251       100    
reco:pandoraShower:LArPandoraModularShowerCreation    0.000203081    0.0137199     0.194836      0.0057339     0.0251176       100    
reco:pandoracalo:Calorimetry                          0.00012479     0.0519435      1.54069     0.00216446     0.234804        100    
reco:pandorapid:Chi2ParticleID                        4.7405e-05    0.000239838   0.00178031    0.000202698   0.000212318      100    
reco:trkshowersplit:TrackShowerHits                   9.5481e-05     0.0467887      1.31282     0.00112565     0.197637        100    
reco:pmtrack:PMAlgTrackMaker                          0.00144527      5.01173       142.761      0.410834       19.1145        100    
reco:pmtrackcalo:Calorimetry                          9.4538e-05     0.0206147     0.655904     0.00180152     0.0872068       100    
reco:pmtrackpid:Chi2ParticleID                        4.2281e-05    0.000480425    0.0061121    0.000235025   0.000846053      100    
reco:pmtrajfit:PMAlgTrajFitter                        0.00021863      3.50229       67.5034      0.498413       10.1144        100    
reco:pmtrajfitcalo:Calorimetry                        8.7433e-05     0.0518569      1.63435     0.00231159     0.228589        100    
reco:pmtrajfitpid:Chi2ParticleID                      3.9287e-05    0.000277462   0.00121485    0.000217432   0.000221798      100    
reco:pmtracktc:PMAlgTrackMaker                        0.00127884      6.03223       167.358      0.562965       22.4322        100    
reco:pmtracktccalo:Calorimetry                        9.9493e-05    0.00963961     0.204446     0.00196551     0.029302        100    
reco:pmtracktcpid:Chi2ParticleID                      4.1577e-05    0.000393937   0.00293186    0.000252164   0.000495247      100    
reco:pmtrajfittc:PMAlgTrajFitter                      0.000218911     2.80524       54.7795      0.635191       8.14685        100    
reco:pmtrajfittccalo:Calorimetry                      9.0209e-05     0.0295239     0.776987     0.00224056     0.116077        100    
reco:pmtrajfittcpid:Chi2ParticleID                    4.0304e-05    0.000537584   0.00461238    0.000243223   0.00084006       100    
reco:blurredcluster:BlurredClustering                 0.000132469     1.32082       8.37953      0.588781       1.6964         100    
reco:emtrkmichelid:EmTrackMichelId                    0.000199055     15.7919       198.572       5.29231       33.7632        100    
reco:emshower:EMShower                                0.000273441    0.0588047     0.619578      0.0184826     0.0974511       100    
reco:cvnmap:CVNMapper                                 2.6409e-05     0.0139991     0.0522519     0.0129869     0.0104093       100    
reco:cvneva:CVNEvaluator                               2.175e-05      0.88307       2.16234      0.953604      0.538584        100    
reco:energyrecnumu:EnergyReco                         0.000386092    0.0367376     0.455211     0.00526216     0.0905535       100    
reco:energyrecnue:EnergyReco                          0.000186605   0.00328415     0.052585     0.00091079    0.00757587       100    
reco:energyrecnc:EnergyReco                           0.00233304    0.00581301     0.0548183    0.00359128    0.00787168       100    
reco:energyrecnumurange:EnergyReco                    0.000306956    0.0348808      0.46021     0.00340206     0.0910268       100    
reco:energyrecnumumcs:EnergyReco                      0.00107248     0.0367771     0.454312     0.00484308     0.0903944       100    
reco:opdec:Deconvolution                               0.0174131     0.229739      0.481512      0.236913      0.0939927       100    
reco:ophitspe:OpHitFinderDeco                          0.103494       1.91084       4.08168       1.7878       0.915395        100    
reco:opflash:OpFlashFinder                            0.00032494    0.00192047    0.00627159    0.00186412    0.000961943      100    
reco:rns:RandomNumberSaver                            2.0238e-05    3.94732e-05   0.000444321   3.6128e-05    4.16266e-05      100    
reco:anglereconue:NuAngularReco                       0.000152743   0.00240524     0.0481405    0.000553852   0.00649063       100    
reco:anglereconumu:NuAngularReco                      8.9916e-05    0.00221754     0.0462011    0.000391403   0.00641915       100    
reco:anglereconuepfps:NuAngularReco                   0.000142639   0.00257649     0.0638692    0.000550738   0.00773956       100    
reco:anglereconumupfps:NuAngularReco                  0.000131704   0.00232771     0.0466088    0.000509258   0.00641289       100    
[art]:TriggerResults:TriggerResultInserter            1.0881e-05    1.83643e-05   7.8121e-05    1.81825e-05   7.17529e-06      100    
end_path:out1:RootOutput                               2.928e-06     5.892e-06     2.754e-05    5.7815e-06    2.91474e-06      100    
end_path:out1:RootOutput(write)                        0.0214975     0.342397       1.58268      0.334208      0.243661        100    
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 8547.3 MB
  Peak resident set size usage (VmHWM): 2452.56 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
Validation_ccnc_atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74510902_439_20231203T101845Z_gen_g4_detsim_hitreco__20240508T075358Z_reco2.root
all-input-dids.txt
atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74510902_439_20231203T101845Z_gen_g4_detsim_hitreco__20240508T075358Z_reco2_reco_2024-11-22T_222533Z.log
atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74510902_439_20231203T101845Z_gen_g4_detsim_hitreco__20240508T075358Z_reco2_reco_data_2024-11-22T_222533Z.root
debugprod.log
jobscript.log
justin-processed-pfns.txt
reco2_hist.root
MyPandoraSettings_Master_Atmos_DUNEFD.xml
MyPandoraSettings_Master_DUNEFD.xml
build_slf7.x86_64
localProducts_larsoft_v09_91_02_e26_prof
setup_env-testreco.sh
srcs
temp.txt
temp2.txt
test
work
justIN time: 2024-11-27 04:15:58 UTC       justIN version: 01.01.09