justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 301487.7@justin-prod-sched01.dune.hep.ac.uk

Jobsub ID301487.7@justin-prod-sched01.dune.hep.ac.uk
Workflow ID4213
Stage ID1
User nameismerio@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2024-11-22 22:57:53
SiteCERN
EntryCMSHTPC_T2_CH_CERN_ce511
Last heartbeat2024-11-23 00:16:23
From worker nodeHostnameb9p12p0328.cern.ch
cpuinfoAMD EPYC 7543 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit171000 (47 hours)
Inner Apptainer?True
Job statefinished
Allocator namejustin-allocator-pro.dune.hep.ac.uk
Started2024-11-22 23:11:43
Input filesfardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74518431_681_20231204T223741Z_gen_g4_detsim_hitreco__20240510T025612Z_reco2.root
JobscriptExit code0
Real time1h (3647s)
CPU time58m (3493s = 95%)
Outputting started2024-11-23 00:12:31
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/ismerio/04213/1/Validation_ccnc_atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74518431_681_20231204T223741Z_gen_g4_detsim_hitreco__20240510T025612Z_reco2.root
Finished2024-11-23 00:16:23
Saved logsjustin-logs:301487.7-justin-prod-sched01.dune.hep.ac.uk.logs.tgz
List job events     Wrapper job log

Jobscript log (last 10,000 characters)

llTree failed.
%MSG
%MSG-e pma::Track3D:  PMAlgTrajFitter:pmtrajfittc@BeginModule  23-Nov-2024 01:11:46 CET run: 74518431 subRun: 1 event: 68200
0 enabled hits in AverageDist2 calculation.
%MSG
%MSG-e pma::Track3D:  PMAlgTrajFitter:pmtrajfittc@BeginModule  23-Nov-2024 01:11:46 CET run: 74518431 subRun: 1 event: 68200
Track empty.
%MSG
%MSG-e pma::Track3D:  PMAlgTrajFitter:pmtrajfittc@BeginModule  23-Nov-2024 01:11:46 CET run: 74518431 subRun: 1 event: 68200
TuneFullTree failed.
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrajFitter:pmtrajfittc@BeginModule  23-Nov-2024 01:11:46 CET run: 74518431 subRun: 1 event: 68200
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  EMShower:emshower@BeginModule  23-Nov-2024 01:12:23 CET run: 74518431 subRun: 1 event: 68200
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  EMShower:emshower@BeginModule  23-Nov-2024 01:12:24 CET run: 74518431 subRun: 1 event: 68200
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  EMShower:emshower@BeginModule  23-Nov-2024 01:12:24 CET run: 74518431 subRun: 1 event: 68200
Comparing two wires in the same plane: return failure
%MSG
Boundary wire vector sizes: 2302, 2217, 1035
minwire 0: 1814
minwire 1: 646
minwire 2: 1488
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 199
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 199
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 199
Classifier summary: 
Output 0: 0.00153132, 0.997719, 0.000749801, 
Output 1: 0.0656753, 0.198648, 0.446241, 0.289436, 
Output 2: 0.457545, 0.542455, 

771
***Wiener Filter*******ApplyPostfilter***
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 2912!
23-Nov-2024 01:12:30 CET  Closed output file "atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74518431_681_20231204T223741Z_gen_g4_detsim_hitreco__20240510T025612Z_reco2_reco_data_2024-11-22T_231150Z.root"
23-Nov-2024 01:12:30 CET  Closed input file "root://golias100.farm.particle.cz:1094/dpm/farm.particle.cz/home/dune/RSE/fardet-hd/64/78/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74518431_681_20231204T223741Z_gen_g4_detsim_hitreco__20240510T025612Z_reco2.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              0.51353       34.0877       351.755       14.3936       57.1356        100    
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                 0.017354      0.0531607     0.837372      0.0438856     0.0835443       100    
reco:linecluster:LineCluster                           0.095761      0.371715         5.5         0.17364       0.79683        100    
reco:trajcluster:TrajCluster                           0.0485628     0.963886       24.3057      0.0704063      3.28385        100    
reco:pandora:StandardPandora                           0.0583707      2.52505       31.3359       1.4282        4.27688        100    
reco:pandoraTrack:LArPandoraTrackCreation             0.000175222   0.00711563     0.0904635    0.00230237     0.0124764       100    
reco:pandoraShower:LArPandoraModularShowerCreation    0.000244352    0.0141894     0.164126     0.00543809     0.0248379       100    
reco:pandoracalo:Calorimetry                          0.000145192    0.0240646      0.54659     0.00206059     0.070528        100    
reco:pandorapid:Chi2ParticleID                         4.79e-05     0.000237046   0.00330054    0.000186837   0.000332745      100    
reco:trkshowersplit:TrackShowerHits                   7.4311e-05     0.0271261     0.356425     0.00145341     0.0706693       100    
reco:pmtrack:PMAlgTrackMaker                          0.00116958      3.39181       63.7325      0.480697       8.99572        100    
reco:pmtrackcalo:Calorimetry                           6.965e-05     0.0106144     0.231269     0.00174411     0.0303998       100    
reco:pmtrackpid:Chi2ParticleID                         2.897e-05    0.000374214   0.00335072    0.000210882   0.000512953      100    
reco:pmtrajfit:PMAlgTrajFitter                        0.000248932     2.67474       34.7915      0.677746       5.92669        100    
reco:pmtrajfitcalo:Calorimetry                         9.14e-05      0.0232662     0.607709     0.00282877     0.070699        100    
reco:pmtrajfitpid:Chi2ParticleID                       3.388e-05    0.000401701    0.0149303    0.000192407   0.00147959       100    
reco:pmtracktc:PMAlgTrackMaker                        0.00127499      3.94473       70.5609      0.569551       10.4803        100    
reco:pmtracktccalo:Calorimetry                        7.5691e-05    0.00715743     0.100631     0.00176992     0.0160479       100    
reco:pmtracktcpid:Chi2ParticleID                       3.386e-05    0.000341117   0.00255711    0.000229632   0.000383342      100    
reco:pmtrajfittc:PMAlgTrajFitter                      0.000213052     2.01137       23.9311      0.641475       3.85877        100    
reco:pmtrajfittccalo:Calorimetry                      8.8651e-05     0.0169484     0.399747     0.00230776     0.0483524       100    
reco:pmtrajfittcpid:Chi2ParticleID                     3.55e-05     0.000486483    0.0040458    0.000254623   0.000637329      100    
reco:blurredcluster:BlurredClustering                 0.000140451     1.13469       7.27371       0.6356        1.34358        100    
reco:emtrkmichelid:EmTrackMichelId                    0.000137211     12.7529       113.358       4.67671       20.4851        100    
reco:emshower:EMShower                                0.000212542    0.0328562     0.184634     0.00536001     0.0482943       100    
reco:cvnmap:CVNMapper                                 2.3171e-05     0.0133514     0.0447414     0.0154462     0.0094327       100    
reco:cvneva:CVNEvaluator                               1.911e-05     0.872659       1.50169       1.06663       0.47968        100    
reco:energyrecnumu:EnergyReco                         0.000383314    0.0575746      1.5182      0.00598007     0.187252        100    
reco:energyrecnue:EnergyReco                          0.000144731   0.00247831     0.0336294    0.000845838   0.00458508       100    
reco:energyrecnc:EnergyReco                           0.00182261    0.00561702     0.0393793    0.00376077     0.0049905       100    
reco:energyrecnumurange:EnergyReco                    0.000142971    0.0562821      1.5714      0.00350491     0.195088        100    
reco:energyrecnumumcs:EnergyReco                      0.00159606     0.0581765      1.68377      0.0058747     0.202498        100    
reco:opdec:Deconvolution                               0.0548526     0.310508      0.510766      0.312094      0.0907201       100    
reco:ophitspe:OpHitFinderDeco                          0.146638       2.40548       4.20823       2.47249       0.87058        100    
reco:opflash:OpFlashFinder                            0.000151021    0.0019577    0.00437228    0.00200226    0.000660244      100    
reco:rns:RandomNumberSaver                            2.3431e-05    5.59497e-05   0.000313713   5.0016e-05    3.41198e-05      100    
reco:anglereconue:NuAngularReco                       0.000205872   0.00185699     0.026915     0.000664102   0.00349666       100    
reco:anglereconumu:NuAngularReco                      8.6591e-05     0.0015074     0.0272326    0.000331283    0.0034644       100    
reco:anglereconuepfps:NuAngularReco                   0.000129911   0.00170913     0.0277995    0.00054572    0.00353775       100    
reco:anglereconumupfps:NuAngularReco                  9.9201e-05    0.00159321     0.0279163    0.000415088   0.00353276       100    
[art]:TriggerResults:TriggerResultInserter             9.981e-06    1.6247e-05    3.5641e-05     1.569e-05    3.96254e-06      100    
end_path:out1:RootOutput                               3.41e-06     7.66435e-06   6.6631e-05     6.73e-06     6.54467e-06      100    
end_path:out1:RootOutput(write)                        0.0125658     0.305064       1.05688      0.306712      0.188715        100    
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 21718.4 MB
  Peak resident set size usage (VmHWM): 2267.46 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
Validation_ccnc_atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74518431_681_20231204T223741Z_gen_g4_detsim_hitreco__20240510T025612Z_reco2.root
all-input-dids.txt
atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74518431_681_20231204T223741Z_gen_g4_detsim_hitreco__20240510T025612Z_reco2_reco_2024-11-22T_231150Z.log
atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74518431_681_20231204T223741Z_gen_g4_detsim_hitreco__20240510T025612Z_reco2_reco_data_2024-11-22T_231150Z.root
debugprod.log
jobscript.log
justin-processed-pfns.txt
reco2_hist.root
MyPandoraSettings_Master_Atmos_DUNEFD.xml
MyPandoraSettings_Master_DUNEFD.xml
build_slf7.x86_64
localProducts_larsoft_v09_91_02_e26_prof
setup_env-testreco.sh
srcs
temp.txt
temp2.txt
test
work
justIN time: 2024-11-23 14:18:25 UTC       justIN version: 01.01.09