centos 部署 SparkR
---恢复内容开始---
环境配置——
- 操作系统:CentOS 6.5
- JDK版本:1.7.0_67
- Hadoop集群版本:CDH 5.3.0
安装过程——
1、(1)安装R
yum install -y R (2)出现问题:
yum 安装软件时,报错:No package XXX available.
[root@localhost ~]# yum -y install redis
Loaded plugins: fastestmirror
No package redis available.
Nothing to do
(3)解决方案
配置一台CentOS 7.x或RHEL 7.x的系统使用Fedora EPEL repos和第三方软件包安装各种软件包。请注意,这些软件包没有正式获得任何的CentOS或Red Hat的支持,但提供了许多流行的包和应用程序。
显示以上错误,有以下2种原因:
1、dns问题;
2、镜像连接错误;
DNS问题
首先测试下网络连接,如果ping不通,需要设置下DNS。
[root@localhost ~]# ping www.baidu.com
XXXXXXXXXXXXXXXXX
[root@localhost ~]# echo "nameserver 8.8.8.8">>/etc/resolv.conf
再次ping即可ping通了。
镜像连接错误问题
具体原因:使用yum搜索某些rpm包,找不到包是因为CentOS是RedHat企业版编译过来的,去掉了所有关于版权问题的东西。安装EPEL后可以很好的解决这个问题。EPEL(Extra Packages for Enterprise Linux )即企业版Linux的扩展包,提供了很多可共Centos使用的组件,安装完这个以后基本常用的rpm都可以找到。
Method #1 » 从企业版Linux库配置包中安装扩展包。(推荐 )
[root@localhost ~]# sudo yum install epel-release
Loaded plugins: fastestmirror, langpacks
Loading mirror speeds from cached hostfile
Resolving Dependencies
--> Running transaction check
---> Package epel-release.noarch 0:7-5 will be installed
--> Finished Dependency Resolution Dependencies Resolved ================================================================================
Package Arch Version Repository Size
================================================================================
Installing:
epel-release noarch 7-5 extras 14 k Transaction Summary
================================================================================
Install 1 Package Total download size: 14 k
Installed size: 24 k
Is this ok [y/d/N]: y
Downloading packages:
epel-release-7-5.noarch.rpm | 14 kB 00:01
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
Installing : epel-release-7-5.noarch 1/1
Verifying : epel-release-7-5.noarch 1/1 Installed:
epel-release.noarch 0:7-5 Complete!
(4)最后运行下列代码 即可完成安装
yum install -y R
2、安装curl-devel(非常重要!否则无法进行RCurl包的安装,进而无法安装devtools)
yum install -y curl-devel
3.在R shell中安装devtools
install.packages("RCurl")
install.packages("devtools")
install.packages("devtools")
出现错误: [plain] view plain copy
------------------------- ANTICONF ERROR ---------------------------
Configuration failed because libcurl was not found. Try installing:
* deb: libcurl4-openssl-dev (Debian, Ubuntu, etc)
* rpm: libcurl-devel (Fedora, CentOS, RHEL)
* csw: libcurl_dev (Solaris)
If libcurl is already installed, check that 'pkg-config' is in your
PATH and PKG_CONFIG_PATH contains a libcurl.pc file. If pkg-config
is unavailable you can set INCLUDE_DIR and LIB_DIR manually via:
R CMD INSTALL --configure-vars='INCLUDE_DIR=... LIB_DIR=...'
--------------------------------------------------------------------
ERROR: configuration failed for package ‘curl’
* removing ‘/home/jifeng/R/x86_64-unknown-linux-gnu-library/3.1/curl’
* installing *source* package ‘whisker’ ...
** package ‘whisker’ successfully unpacked and MD5 sums checked
** R
** inst
** preparing package for lazy loading
** help
*** installing help indices
** building package indices
** testing if installed package can be loaded
* DONE (whisker)
* installing *source* package ‘digest’ ...
** package ‘digest’ successfully unpacked and MD5 sums checked
** libs [plain] view plain copy
configure: error: in `/tmp/RtmpkQMD9t/R.INSTALL6c1166a0db0e/git2r':
configure: error:
---------------------------------------------
OpenSSL library required Please install:
libssl-dev (deb) or openssl-devel (rpm)
---------------------------------------------
See `config.log' for more details
ERROR: configuration failed for package ‘git2r’
* removing ‘/home/jifeng/R/x86_64-unknown-linux-gnu-library/3.1/git2r’
* installing *source* package ‘stringr’ ...
** package ‘stringr’ successfully unpacked and MD5 sums checked
** R
** inst
** preparing package for lazy loading
** help
*** installing help indices
** building package indices
** installing vignettes
** testing if installed package can be loaded
* DONE (stringr)
* installing *source* package ‘xml2’ ...
** package ‘xml2’ successfully unpacked and MD5 sums checked
Package libxml-2.0 was not found in the pkg-config search path.
Perhaps you should add the directory containing `libxml-2.0.pc'
to the PKG_CONFIG_PATH environment variable
No package 'libxml-2.0' found
Package libxml-2.0 was not found in the pkg-config search path.
Perhaps you should add the directory containing `libxml-2.0.pc'
to the PKG_CONFIG_PATH environment variable
No package 'libxml-2.0' found
Using PKG_CFLAGS=
Using PKG_LIBS=-lxml2
------------------------- ANTICONF ERROR ---------------------------
Configuration failed because libxml-2.0 was not found. Try installing:
* deb: libxml2-dev (Debian, Ubuntu, etc)
* rpm: libxml2-devel (Fedora, CentOS, RHEL)
* csw: libxml2_dev (Solaris)
If libxml-2.0 is already installed, check that 'pkg-config' is in your
PATH and PKG_CONFIG_PATH contains a libxml-2.0.pc file. If pkg-config
is unavailable you can set INCLUDE_DIR and LIB_DIR manually via:
R CMD INSTALL --configure-vars='INCLUDE_DIR=... LIB_DIR=...'
--------------------------------------------------------------------
ERROR: configuration failed for package ‘xml2’
* removing ‘/home/jifeng/R/x86_64-unknown-linux-gnu-library/3.1/xml2’
* installing *source* package ‘memoise’ ...
** package ‘memoise’ successfully unpacked and MD5 sums checked
** R
** preparing package for lazy loading
** help
*** installing help indices
** building package indices
** testing if installed package can be loaded
* DONE (memoise)
ERROR: dependency ‘curl’ is not available for package ‘httr’
* removing ‘/home/jifeng/R/x86_64-unknown-linux-gnu-library/3.1/httr’
* installing *source* package ‘evaluate’ ...
** package ‘evaluate’ successfully unpacked and MD5 sums checked
** R
** preparing package for lazy loading
** help
*** installing help indices
** building package indices
** testing if installed package can be loaded
* DONE (evaluate)
* installing *source* package ‘roxygen2’ ...
** package ‘roxygen2’ successfully unpacked and MD5 sums checked
** libs
g++ -I/usr/local/lib64/R/include -DNDEBUG -I/usr/local/include -I"/home/jifeng/R/x86_64-unknown-linux-gnu-library/3.1/Rcpp/include" -fpic -g -O2 -c RcppExports.cpp -o RcppExports.o
g++ -I/usr/local/lib64/R/include -DNDEBUG -I/usr/local/include -I"/home/jifeng/R/x86_64-unknown-linux-gnu-library/3.1/Rcpp/include" -fpic -g -O2 -c isComplete.cpp -o isComplete.o
g++ -I/usr/local/lib64/R/include -DNDEBUG -I/usr/local/include -I"/home/jifeng/R/x86_64-unknown-linux-gnu-library/3.1/Rcpp/include" -fpic -g -O2 -c leadingSpaces.cpp -o leadingSpaces.o
g++ -I/usr/local/lib64/R/include -DNDEBUG -I/usr/local/include -I"/home/jifeng/R/x86_64-unknown-linux-gnu-library/3.1/Rcpp/include" -fpic -g -O2 -c parser.cpp -o parser.o
g++ -I/usr/local/lib64/R/include -DNDEBUG -I/usr/local/include -I"/home/jifeng/R/x86_64-unknown-linux-gnu-library/3.1/Rcpp/include" -fpic -g -O2 -c wrapString.cpp -o wrapString.o
g++ -shared -L/usr/local/lib64 -o roxygen2.so RcppExports.o isComplete.o leadingSpaces.o parser.o wrapString.o -L/usr/local/lib64/R/lib -lR
installing to /home/jifeng/R/x86_64-unknown-linux-gnu-library/3.1/roxygen2/libs
** R
** inst
** preparing package for lazy loading
** help
*** installing help indices
** building package indices
** installing vignettes
** testing if installed package can be loaded
* DONE (roxygen2)
ERROR: dependencies ‘curl’, ‘xml2’ are not available for package ‘rversions’
* removing ‘/home/jifeng/R/x86_64-unknown-linux-gnu-library/3.1/rversions’
ERROR: dependencies ‘httr’, ‘curl’, ‘rversions’, ‘git2r’ are not available for package ‘devtools’
* removing ‘/home/jifeng/R/x86_64-unknown-linux-gnu-library/3.1/devtools’ The downloaded source packages are in
‘/tmp/Rtmp1A16li/downloaded_packages’
Warning messages:
1: In install.packages("devtools") :
installation of package ‘curl’ had non-zero exit status
2: In install.packages("devtools") :
installation of package ‘git2r’ had non-zero exit status
3: In install.packages("devtools") :
installation of package ‘xml2’ had non-zero exit status
4: In install.packages("devtools") :
installation of package ‘httr’ had non-zero exit status
5: In install.packages("devtools") :
installation of package ‘rversions’ had non-zero exit status
6: In install.packages("devtools") :
installation of package ‘devtools’ had non-zero exit status 根据错误安装,我这里安装了 [plain] view plain copy
[root@feng03 ~]# yum install libcurl-devel
[root@feng03 ~]# yum install openssl-devel
[root@feng03 ~]# yum install libxml2-devel 进入R shell
>install.packages("git2r")
>install.packages("xml2")
>install.packages("rversions")
步骤4: 安装好rJava及devtools,接下来安装SparkR
library(devtools)
install_github("amplab-extras/SparkR-pkg", subdir="pkg")
出现错误
解决:
/data1/bigdata/spark-1.6.1/R/lib/SparkR
---恢复内容结束---
centos 部署 SparkR的更多相关文章
- 跳过 centos部署 webpy的各种坑
用centos部署webpy发现的各种坑: 1.python 版本: 2.中文编码: 3.web模块路径: 在命令行里输入python,能import web,但是网站错误报告一直报告没有找到web模 ...
- 【Linux】Centos部署MySQL
将CentOS部署MySQL需要本地配置环境.本地编译MySQL,耗时较长的情况,优化为编译成型MySQL并打包,推送并按配置部署. 首先需要在一台机器配置好环境,搭个YUM源,并将所需要的包取出备用 ...
- Tigase XMPP Server在CentOS部署和配置
Tigase XMPP Server在CentOS部署与配置 作者:chszs,转载需注明.博客主页:http://blog.csdn.net/chszs 以下讲述Tigase XMPP Server ...
- SpringBoot入门教程(二)CentOS部署SpringBoot项目从0到1
在之前的博文<详解intellij idea搭建SpringBoot>介绍了idea搭建SpringBoot的详细过程, 并在<CentOS安装Tomcat>中介绍了Tomca ...
- 为什么使用centos部署服务器
这个是实验室同学面试的时候,面试官问的一个问题? 为什么选择centos系统,为什么centos系统用的比较多呢? 首先我们说下redhat红帽公司,它是全球最大的linux服务提供商,它的服务是最好 ...
- CentOS下SparkR安装部署:hadoop2.7.3+spark2.0.0+scale2.11.8+hive2.1.0
注:之前本人写了一篇SparkR的安装部署文章:SparkR安装部署及数据分析实例,当时SparkR项目还没正式入主Spark,需要自己下载SparkR安装包,但现在spark已经支持R接口,so更新 ...
- hadoop集群搭建--CentOS部署Hadoop服务
在了解了Hadoop的相关知识后,接下来就是Hadoop环境的搭建,搭建Hadoop环境是正式学习大数据的开始,接下来就开始搭建环境!我们用到环境为:VMware 12+CentOS6.4 hadoo ...
- CentOS部署yapi
转载自 https://www.linuxidc.com/Linux/2018-01/150513.htm 在mongoDB添加yum源时,源路径有修改,原文中的路径404不可用 一.准备工作 1.1 ...
- netcore项目在Centos部署:nohup和supervisor方式
Centos上部署netcore项目 1 准备工作 在Centos上部署netcore应用程序有两种常用方式:nohup和supervisord,这里简单演示一下这两种部署方式. 首先我们写一个简单的 ...
随机推荐
- Oracle游标解析
本节对Oracle中的游标进行详细讲解. 本节所举实例来源Oracle中scott用户下的emp表dept表: 一.游标: 1.概念: 游标的本质是一个结果集resultset,主要用来临时存储从数据 ...
- oc 异常处理
#import <UIKit/UIKit.h> #import "AppDelegate.h" int main(int argc, char * argv[]) { ...
- 从错误中学python(4)——最小公约数与辗转相除法
题目 给你两个正整数a和b, 输出它们的最大公约数 辗转相除法 辗转相除法的步骤 def gcd(b,a): b,a=a,b%a if a==0: return b else: return gcd( ...
- maven将镜像站点改为中国开源镜像点
在Apache官网上下载bin文件,解压到相应目录.然后配置/etc/profile即可,环境变量名为M2_HOME,如下:(配置完后注意source /etc/profile)#Mavenexpor ...
- 【转】在ASP.NET应用启动的时候初始化的几种方法
ASP.NET 4.0 之前,有两种方法:通过Global.asax 中的 Application_Start 事件启动,或者通过定义在 App_Code 文件夹中任意类中的AppInitialize ...
- C/C++ 数据结构之算法(面试)
数据结构中的排序算法. 排序算法的相关知识: (1)排序的概念:所谓排序就是要整理文件中的记录,使之按关键字递增(或递减)次序排列起来. (2)稳定的排序方法:在待排序的文件中,若存在多个关键字相同的 ...
- PHP函数之类
if(isset($_POST['dosubmit'])) { } 打散 $array_urls = explode("\r\n",$inurl_str); foreach ($a ...
- pannel加载窗体
public static void loadFillForm(Panel panel, System.Windows.Forms.Form frm) { if (frm != null && ...
- angular学习笔记(十)-src和href处理
本篇主要介绍angular中图片的src和链接的href的处理: 用到了以下两个属性: ng-src: 绑定了数据的路径表达式 ng-href: 绑定了数据的路径表达式 例如: <!DOCTYP ...
- Eclipse build error 解决方法The library '*.jar' contains native libraries that will not run on the dev
[2013-08-29 16:56:58 - jarsotest] The library 'wnp.jar' contains native libraries that will not run ...