Jobsub ID 97029.187@justin-prod-sched02.dune.hep.ac.uk
| Jobsub ID | 97029.187@justin-prod-sched02.dune.hep.ac.uk |
| Workflow ID | 4103 |
| Stage ID | 1 |
| User name | amoor@fnal.gov |
| HTCondor Group | group_dune |
| Requested | Processors | 1 |
| RSS bytes | 8388608000 (8000 MiB) |
| Wall seconds limit | 80000 (22 hours) |
| Submitted time | 2024-11-14 13:41:08 |
| Site | UK_Manchester |
| Entry | UBoone_T2_UK_Manchester_ce01 |
| Last heartbeat | 2024-11-14 14:35:07 |
| From worker node | Hostname | wn1904251.tier2.hep.manchester.ac.uk |
| cpuinfo | AMD EPYC 7513 32-Core Processor |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 8388608000 (8000 MiB) |
| Wall seconds limit | 257400 (71 hours) |
| Inner Apptainer? | True |
| Job state | jobscript_error |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk |
| Started | 2024-11-14 14:32:53 |
| Input files | usertests:000058_reco_data_2024-11-14T_092348Z.root
|
| Jobscript | Exit code | 1 |
| Real time | 0m (0s) |
| CPU time | 0m (0s = 0%) |
| Outputting started | |
| Output files | |
| Finished | 2024-11-14 14:35:07 |
| Saved logs | justin-logs:97029.187-justin-prod-sched02.dune.hep.ac.uk.logs.tgz |
| List job events Wrapper job log |
Jobscript log (last 10,000 characters)
AFM g4 jobscript.
Input PFN = root://cephc02.gla.scotgrid.ac.uk:1094//cephfs/dune/RSE/usertests/1c/b3/000058_reco_data_2024-11-14T_092348Z.root
Setting up larsoft UPS area... /cvmfs/larsoft.opensciencegrid.org
Setting up DUNE UPS area... /cvmfs/dune.opensciencegrid.org/products/dune/
/cvmfs/larsoft.opensciencegrid.org/products/xrootd/v5_4_3b/Linux64bit+3.10-2.17-e20-p3913-prof/lib/libXrdPosixPreload.so
=== Start last 100 lines of lar log file ===
%MSG
%MSG-i fStepLimit: larg4Main:largeant@Construction 14-Nov-2024 14:34:35 GMT ModuleConstruction
fStepLimit: 0.4 0.04 cm
%MSG
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
Attaching sensitive Detector: SimEnergyDeposit to Volume: volTPCInner
%MSG-i AuxUnit: larg4Main:largeant@Construction 14-Nov-2024 14:34:35 GMT ModuleConstruction
Unit parsed = mm from unit category: Length
%MSG
%MSG-i AuxUnit: larg4Main:largeant@Construction 14-Nov-2024 14:34:35 GMT ModuleConstruction
Valid StepLimit unit category obtained: Length
%MSG
%MSG-i fStepLimit: larg4Main:largeant@Construction 14-Nov-2024 14:34:35 GMT ModuleConstruction
fStepLimit: 0.4 0.04 cm
%MSG
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
Attaching sensitive Detector: SimEnergyDeposit to Volume: volTPCActiveOuter
%MSG-i AuxUnit: larg4Main:largeant@Construction 14-Nov-2024 14:34:35 GMT ModuleConstruction
Unit parsed = mm from unit category: Length
%MSG
%MSG-i AuxUnit: larg4Main:largeant@Construction 14-Nov-2024 14:34:35 GMT ModuleConstruction
Valid StepLimit unit category obtained: Length
%MSG
%MSG-i fStepLimit: larg4Main:largeant@Construction 14-Nov-2024 14:34:35 GMT ModuleConstruction
fStepLimit: 0.4 0.04 cm
%MSG
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
Attaching sensitive Detector: SimEnergyDeposit to Volume: volTPCPlaneZInner
%MSG-i AuxUnit: larg4Main:largeant@Construction 14-Nov-2024 14:34:35 GMT ModuleConstruction
Unit parsed = mm from unit category: Length
%MSG
%MSG-i AuxUnit: larg4Main:largeant@Construction 14-Nov-2024 14:34:35 GMT ModuleConstruction
Valid StepLimit unit category obtained: Length
%MSG
%MSG-i fStepLimit: larg4Main:largeant@Construction 14-Nov-2024 14:34:35 GMT ModuleConstruction
fStepLimit: 0.4 0.04 cm
%MSG
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%MSG-w LArG4DetectorService::setStepLimits: larg4Main:largeant@Construction 14-Nov-2024 14:34:35 GMT ModuleConstruction LArG4Detector.cc:322
Setting step limits from configuration file. This will OVERRIDE redundant stepLimit(s) set in the GDML file. Note that stepLimits are only active if enabled in the physicsListService via the appropriate parameter.
%MSG
%MSG-w LArG4DetectorService::setStepLimits: larg4Main:largeant@Construction 14-Nov-2024 14:34:35 GMT ModuleConstruction LArG4Detector.cc:358
New stepLimit matches previously set stepLimit from the GDML file for volume : volTPCActiveInner stepLimit : 0.4 mm. Nothing will be changed.
%MSG
%MSG-w LArG4DetectorService::setStepLimits: larg4Main:largeant@Construction 14-Nov-2024 14:34:35 GMT ModuleConstruction LArG4Detector.cc:358
New stepLimit matches previously set stepLimit from the GDML file for volume : volTPCActiveOuter stepLimit : 0.4 mm. Nothing will be changed.
%MSG
List SD Tree:
Volume volTPCActiveInner has the following list of auxiliary information:
--> Type: SensDet Value: SimEnergyDeposit
--> Type: StepLimit Value: 0.4
Volume volTPCPlaneUInner has the following list of auxiliary information:
--> Type: SensDet Value: SimEnergyDeposit
--> Type: StepLimit Value: 0.4
Volume volTPCPlaneVInner has the following list of auxiliary information:
--> Type: SensDet Value: SimEnergyDeposit
--> Type: StepLimit Value: 0.4
Volume volTPCInner has the following list of auxiliary information:
--> Type: SensDet Value: SimEnergyDeposit
--> Type: StepLimit Value: 0.4
Volume volTPCActiveOuter has the following list of auxiliary information:
--> Type: SensDet Value: SimEnergyDeposit
--> Type: StepLimit Value: 0.4
Volume volTPCPlaneZInner has the following list of auxiliary information:
--> Type: SensDet Value: SimEnergyDeposit
--> Type: StepLimit Value: 0.4
/
/volTPCActiveInner_SimEnergyDeposit *** Active
/volTPCPlaneUInner_SimEnergyDeposit *** Active
/volTPCPlaneVInner_SimEnergyDeposit *** Active
/volTPCInner_SimEnergyDeposit *** Active
/volTPCActiveOuter_SimEnergyDeposit *** Active
/volTPCPlaneZInner_SimEnergyDeposit *** Active
Collection Capacity: 0
==================================================
nr of LV ======================: 1
%MSG-i RANDOM: larg4Main:largeant@Construction 14-Nov-2024 14:34:35 GMT ModuleConstruction
Instantiated G4Engine engine "largeant:0:" with seed 0.
%MSG
%MSG-i NuRandomService: larg4Main:largeant@Construction 14-Nov-2024 14:34:35 GMT ModuleConstruction
Seeding G4Engine engine "largeant:" with seed 0.
%MSG
14-Nov-2024 14:34:35 GMT Initiating request to open input file "root://cephc02.gla.scotgrid.ac.uk:1094//cephfs/dune/RSE/usertests/1c/b3/000058_reco_data_2024-11-14T_092348Z.root"
Plugin version SecClnt v5.4.3 is incompatible with secztn v5.6.8 (must be <= 5.4.x) in sec.protocol libXrdSecztn-5.so
%MSG-i Root_Information: TriggerResultInserter:TriggerResults@Construction TNetXNGFile::Open() 14-Nov-2024 14:34:36 GMT ModuleConstruction
[ERROR] Server responded with an error: [3010] Unable to open /cephfs/dune/RSE/usertests/1c/b3/000058_reco_data_2024-11-14T_092348Z.root; permission denied
ROOT severity: 3000
%MSG
%MSG-s ArtException: TriggerResultInserter:TriggerResults@Construction 14-Nov-2024 14:34:36 GMT ModuleConstruction
cet::exception caught in art
---- FileOpenError BEGIN
RootInputFileSequence::initFile(): Input file root://cephc02.gla.scotgrid.ac.uk:1094//cephfs/dune/RSE/usertests/1c/b3/000058_reco_data_2024-11-14T_092348Z.root was not found or could not be opened.
---- FileOpenError END
%MSG
Art has completed and will exit with status 20.
=== End last 100 lines of lar log file ===
lar exit code 0
Traceback (most recent call last):
File "/cvmfs/dune.opensciencegrid.org/products/dune/duneutil/v09_75_03d00/bin/extractor_prod.py", line 434, in <module>
main()
File "/cvmfs/dune.opensciencegrid.org/products/dune/duneutil/v09_75_03d00/bin/extractor_prod.py", line 373, in main
mddict = expSpecificMetadata.getmetadata()
File "/cvmfs/dune.opensciencegrid.org/products/dune/duneutil/v09_75_03d00/bin/extractor_prod.py", line 344, in getmetadata
jobt = self.get_job(proc)
File "/cvmfs/dune.opensciencegrid.org/products/dune/duneutil/v09_75_03d00/bin/extractor_prod.py", line 69, in get_job
raise RuntimeError('sam_metadata_dumper returned nonzero exit status {}.'.format(rc))
RuntimeError: sam_metadata_dumper returned nonzero exit status 1.
extractor_prod.py exit code 1
Error reading metadata from file: Expecting value: line 1 column 1 (char 0)
pdjson2metadata exit code 1
.:
total 32
-rw-r--r-- 1 dune01 dune01 15827 Nov 14 14:34 000058_reco_data_2024-11-14T_092348Z_reco_2024-11-14T_143300Z.log
-rw-r--r-- 1 dune01 dune01 6976 Nov 14 14:34 jobscript.log
-rw-r--r-- 1 dune01 dune01 519 Nov 14 14:34 g4_hist.root
-rw-r--r-- 1 dune01 dune01 104 Nov 14 14:32 all-input-dids.txt
-rw-r--r-- 1 dune01 dune01 0 Nov 14 14:34 000058_reco_data_2024-11-14T_092348Z_reco_data_2024-11-14T_143300Z.root.ext.json
-rw-r--r-- 1 dune01 dune01 0 Nov 14 14:34 000058_reco_data_2024-11-14T_092348Z_reco_data_2024-11-14T_143300Z.root.json 