justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 108551.124@justin-prod-sched02.dune.hep.ac.uk

Jobsub ID108551.124@justin-prod-sched02.dune.hep.ac.uk
Workflow ID4213
Stage ID1
User nameismerio@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2024-11-22 23:16:35
SiteUK_RAL-Tier1
EntryLIGO_UK_RAL_arc_ce02
Last heartbeat2024-11-23 00:08:16
From worker nodeHostnamedune001-3353702.0-lcg2706.gridpp.rl.ac.uk
cpuinfoAMD EPYC 9654 96-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit216000 (60 hours)
Inner Apptainer?True
Job statefinished
Allocator namejustin-allocator-pro.dune.hep.ac.uk
Started2024-11-22 23:33:26
Input filesfardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6390196_764_20231201T210707Z_gen_g4_detsim_hitreco__20240507T222624Z_reco2.root
JobscriptExit code0
Real time33m (2033s)
CPU time36m (2188s = 107%)
Outputting started2024-11-23 00:07:20
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/ismerio/04213/1/Validation_ccnc_atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6390196_764_20231201T210707Z_gen_g4_detsim_hitreco__20240507T222624Z_reco2.root
Finished2024-11-23 00:08:16
Saved logsjustin-logs:108551.124-justin-prod-sched02.dune.hep.ac.uk.logs.tgz
List job events     Wrapper job log

Jobscript log (last 10,000 characters)

Classifier summary: 
Output 0: 0.994124, 0.000776846, 0.00509919, 
Output 1: 0.504113, 0.478494, 0.0166411, 0.000751943, 
Output 2: 0.995017, 0.00498262, 

415
***Wiener Filter*******ApplyPostfilter***
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 1353!
Begin processing the 100th record. run: 6390196 subRun: 1 event: 76500 at 23-Nov-2024 00:07:17 UTC
> Running Algorithm: Alg0001, LArPreProcessing
> Running Algorithm: Alg0002, LArDLMaster
> Running Algorithm: Alg0003, LArNeutrinoEventValidation
---RAW-MATCHING-OUTPUT--------------------------------------------------------------------------
Failure in algorithm Alg0003, LArNeutrinoEventValidation, STATUS_CODE_NOT_INITIALIZED
%MSG-e ShowerProducedPtrsHolder:   LArPandoraModularShowerCreation:pandoraShower@BeginModule  23-Nov-2024 00:07:18 UTC run: 6390196 subRun: 1 event: 76500
Trying to add data product: InitialTrack. This element does not exist in the element holder
%MSG
%MSG-e ShowerProducedPtrsHolder:   LArPandoraModularShowerCreation:pandoraShower@BeginModule  23-Nov-2024 00:07:18 UTC run: 6390196 subRun: 1 event: 76500
Trying to add data product: ShowerPCA. This element does not exist in the element holder
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrajFitter:pmtrajfit@BeginModule  23-Nov-2024 00:07:18 UTC run: 6390196 subRun: 1 event: 76500
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  23-Nov-2024 00:07:18 UTC run: 6390196 subRun: 1 event: 76500
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrajFitter:pmtrajfittc@BeginModule  23-Nov-2024 00:07:18 UTC run: 6390196 subRun: 1 event: 76500
Comparing two wires in the same plane: return failure
%MSG
150
***Wiener Filter*******ApplyPostfilter***
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 304!
23-Nov-2024 00:07:20 UTC  Closed output file "atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6390196_764_20231201T210707Z_gen_g4_detsim_hitreco__20240507T222624Z_reco2_reco_data_2024-11-22T_233330Z.root"
23-Nov-2024 00:07:20 UTC  Closed input file "root://xrootd.echo.stfc.ac.uk:1094/dune:/protodune/RSE/fardet-hd/11/7f/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6390196_764_20231201T210707Z_gen_g4_detsim_hitreco__20240507T222624Z_reco2.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                             0.321218       19.0689       138.538       7.82019       29.9825        100    
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.00138323     0.108871       0.36584      0.0893041     0.0739801       100    
reco:linecluster:LineCluster                           0.0402514     0.291362       1.43466      0.208618      0.262297        100    
reco:trajcluster:TrajCluster                          0.000854458    0.404328       5.32991      0.0942731     0.955702        100    
reco:pandora:StandardPandora                           0.111019       1.66426       9.28104       1.23637       1.48652        100    
reco:pandoraTrack:LArPandoraTrackCreation             0.000128313   0.00558704     0.0601657    0.00134991     0.0108913       100    
reco:pandoraShower:LArPandoraModularShowerCreation    0.000169736   0.00856669     0.0643986    0.00210531     0.0142493       100    
reco:pandoracalo:Calorimetry                          8.7772e-05     0.0104635     0.159236     0.000744793    0.0275398       100    
reco:pandorapid:Chi2ParticleID                        3.2629e-05    0.000167664    0.0018919    0.000118874   0.000205426      100    
reco:trkshowersplit:TrackShowerHits                   6.3896e-05     0.0154127     0.314028     0.000552207    0.0453566       100    
reco:pmtrack:PMAlgTrackMaker                          0.00105458      1.59627       20.4477      0.169742       3.85255        100    
reco:pmtrackcalo:Calorimetry                          5.1528e-05     0.0063775     0.0805339    0.000735173    0.0159069       100    
reco:pmtrackpid:Chi2ParticleID                        2.0911e-05    0.000232965   0.00168107    0.000128614   0.000293377      100    
reco:pmtrajfit:PMAlgTrajFitter                        0.000174893     1.16906       10.2581      0.272083       2.25973        100    
reco:pmtrajfitcalo:Calorimetry                        4.7632e-05     0.0105889     0.255089      0.0010644     0.0303201       100    
reco:pmtrajfitpid:Chi2ParticleID                      2.0451e-05    0.000175425   0.000629379   0.000135634   0.00013418       100    
reco:pmtracktc:PMAlgTrackMaker                        0.00115181      1.78594       25.1802      0.154452       4.48747        100    
reco:pmtracktccalo:Calorimetry                        7.0356e-05    0.00470741     0.0563989    0.000806546    0.0108352       100    
reco:pmtracktcpid:Chi2ParticleID                      2.7431e-05    0.000211317   0.000986206   0.000125965   0.000226527      100    
reco:pmtrajfittc:PMAlgTrajFitter                      0.000161324     1.01849       13.0316      0.185397       2.1373         100    
reco:pmtrajfittccalo:Calorimetry                      6.3115e-05    0.00881893     0.134723     0.000921128    0.0225304       100    
reco:pmtrajfittcpid:Chi2ParticleID                    2.5679e-05    0.000286732   0.00222888    0.000138574   0.000402384      100    
reco:blurredcluster:BlurredClustering                 0.000207432    0.935082       7.06523       0.47527       1.33667        100    
reco:emtrkmichelid:EmTrackMichelId                    0.00011255      7.45864       55.7159       1.9642        13.185         100    
reco:emshower:EMShower                                0.000162495    0.0222303     0.187155     0.00279264     0.0386129       100    
reco:cvnmap:CVNMapper                                 1.8598e-05    0.00532774     0.036596     0.00417832    0.00632765       100    
reco:cvneva:CVNEvaluator                              1.5062e-05     0.512518      0.999962      0.586334      0.307723        100    
reco:energyrecnumu:EnergyReco                         0.000338079    0.027635      0.567493     0.00647798     0.083467        100    
reco:energyrecnue:EnergyReco                          0.000107532   0.00161543     0.0234602    0.000429882   0.00331806       100    
reco:energyrecnc:EnergyReco                           0.00129746    0.00324664     0.0277953    0.00227359    0.00352721       100    
reco:energyrecnumurange:EnergyReco                    0.000124147    0.0235474     0.559729     0.00157031     0.0825125       100    
reco:energyrecnumumcs:EnergyReco                      0.00126243     0.0245539     0.554355     0.00263837     0.0821687       100    
reco:opdec:Deconvolution                               0.0528865     0.291947      0.784934      0.273582      0.151181        100    
reco:ophitspe:OpHitFinderDeco                          0.0153351      1.44697       3.22032       1.32003      0.715574        100    
reco:opflash:OpFlashFinder                            0.000113591    0.0011945    0.00490197    0.00108299    0.000628763      100    
reco:rns:RandomNumberSaver                            1.7707e-05    2.8697e-05    0.000355446   2.33855e-05   3.37523e-05      100    
reco:anglereconue:NuAngularReco                       0.000156176   0.00151177     0.021527     0.000330363   0.00353294       100    
reco:anglereconumu:NuAngularReco                      6.4868e-05    0.00130694     0.0213356    0.000169916   0.00350691       100    
reco:anglereconuepfps:NuAngularReco                   0.000110326   0.00142091     0.021277     0.000264114   0.00348035       100    
reco:anglereconumupfps:NuAngularReco                   7.33e-05     0.00135074     0.0212169    0.000219806   0.00350059       100    
[art]:TriggerResults:TriggerResultInserter             8.032e-06    1.14123e-05   4.5789e-05    1.07765e-05   4.55182e-06      100    
end_path:out1:RootOutput                               2.765e-06    3.93017e-06   2.0901e-05    3.5205e-06    1.91635e-06      100    
end_path:out1:RootOutput(write)                        0.0125833     0.196926      0.629237      0.194021      0.131752        100    
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 40408.2 MB
  Peak resident set size usage (VmHWM): 2610.79 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
Validation_ccnc_atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6390196_764_20231201T210707Z_gen_g4_detsim_hitreco__20240507T222624Z_reco2.root
all-input-dids.txt
atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6390196_764_20231201T210707Z_gen_g4_detsim_hitreco__20240507T222624Z_reco2_reco_2024-11-22T_233330Z.log
atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6390196_764_20231201T210707Z_gen_g4_detsim_hitreco__20240507T222624Z_reco2_reco_data_2024-11-22T_233330Z.root
debugprod.log
jobscript.log
justin-processed-pfns.txt
reco2_hist.root
MyPandoraSettings_Master_Atmos_DUNEFD.xml
MyPandoraSettings_Master_DUNEFD.xml
build_slf7.x86_64
localProducts_larsoft_v09_91_02_e26_prof
setup_env-testreco.sh
srcs
temp.txt
temp2.txt
test
work
justIN time: 2024-11-24 16:02:50 UTC       justIN version: 01.01.09