保存文件时权限被拒绝 曾经踩过的坑: 保存结果到hdfs上没有写的权限 通过修改权限将文件写入到指定的目录下
*
*
* $HADOOP_HOME/bin/hdfs dfs -chmod 777 /user
*
*
* Exception in thread "main" org.apache.hadoop.security.AccessControlException:
* Permission denied: user=Mypc, access=WRITE,
* inode="/":fan:supergroup:drwxr-xr-x

package cn.spark.study.sql;

import org.apache.spark.SparkConf;
import org.apache.spark.api.java.JavaSparkContext;
import org.apache.spark.sql.DataFrame;
import org.apache.spark.sql.SQLContext;
import org.apache.spark.sql.SaveMode; /**
* SaveModel示例
*
* @author 张运涛
*
*/
public class SaveModeTest { @SuppressWarnings("deprecation")
public static void main(String[] args) {
System.setProperty("hadoop.home.dir", "E:\\hadoop-2.7.1"); SparkConf conf = new SparkConf().setMaster("local").setAppName("SaveModeTest");
JavaSparkContext sc = new JavaSparkContext(conf);
SQLContext sqlContext = new SQLContext(sc); DataFrame peopleDF = sqlContext.read().format("json").load("hdfs://zhang:9000/user/hadoop/spark/people.json");
/**
* 保存文件时权限被拒绝 曾经踩过的坑: 保存结果到hdfs上没有写的权限 通过修改权限将文件写入到指定的目录下
*
*
* $HADOOP_HOME/bin/hdfs dfs -chmod 777 /user
*
*
* Exception in thread "main" org.apache.hadoop.security.AccessControlException:
* Permission denied: user=Mypc, access=WRITE,
* inode="/":fan:supergroup:drwxr-xr-x
*
*/ peopleDF.save("hdfs://zhang:9000/user/people_savemode_test", "json", SaveMode.Append);
} }

Exception in thread "main" org.apache.hadoop.security.AccessControlException: Permission denied: user=lenovo, access=WRITE, inode="/user/hadoop/spark/people_savemode_test/_temporary/0":hadoop:supergro的更多相关文章

  1. Exception in thread "main" org.apache.hadoop.security.AccessControlException: Permission denied: user=Mypc, access=WRITE, inode="/":fan:supergroup:drwxr-xr-x

    在window上编程提示没有写Hadoop的权限 Exception in thread "main" org.apache.hadoop.security.AccessContr ...

  2. kylin cube测试时,报错:org.apache.hadoop.security.AccessControlException: Permission denied: user=root, access=WRITE, inode="/user":hdfs:supergroup:drwxr-xr-x

    异常: org.apache.hadoop.security.AccessControlException: Permission denied: user=root, access=WRITE, i ...

  3. 一脸懵逼加从入门到绝望学习hadoop之 org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.AccessControlException): Permission denied: user=Administrator, access=WRITE, inode="/":root:supergroup:drwxr-xr报错

    1:初学hadoop遇到各种错误,这里贴一下,方便以后脑补吧,报错如下: 主要是在window环境下面搞hadoop,而hadoop部署在linux操作系统上面:出现这个错误是权限的问题,操作hado ...

  4. 报错:HDFS IO error org.apache.hadoop.security.AccessControlException: Permission denied: user=root, access=WRITE, inode="/yk/dl/alarm_his":hdfs:supergroup:drwxr-xr-x

    报错背景: CDH集成了Flume服务,准备通过Flume将kafka中的数据放到HDFS中, 启动Flume的时候报错. 报错现象: // :: INFO hdfs.HDFSDataStream: ...

  5. 异常-Caused by: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.AccessControlException): Permission denied: user=hdfs, access=WRITE, inode="/hbase":root:supergroup:drwxr-xr-x

    1 详细异常 Caused by: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.AccessControlExce ...

  6. 【Hadoop & Ecilpse】Exception in thread "main" org.apache.hadoop.security.AccessControlException: Permission denied: user=bruce, access=WRITE, inode="/out2/_temporary/0":atguigu:supergroup:drwxr-xr-x

    问题再现: 使用本机 Ecilpse  (Windows环境) 去访问远程 hadoop 集群出现以下异常: 问题原因: 因为远程提交的情况下如果没有 hadoop 的系统环境变量,就会读取当前主机的 ...

  7. hadoop 3.x org.apache.hadoop.security.AccessControlException: Permission denied: user=Administrator, access=WRITE, inode="/":tele:supergroup:drwxr-xr-x

    权限不足,上传文件时应当使用启动hadoop的账户,即在获取FileSystem时就应当指定用户 修改后的代码 public class Demo1 { public static void main ...

  8. org.apache.hadoop.security.AccessControlException: org.apache.hadoop.security .AccessControlException: Permission denied: user=Administrator, access=WRITE, inode="hadoop": hadoop:supergroup:rwxr-xr-x

    这时windows远程调试hadoop集群出现的这里 做个记录   我用改变系统变量的方法 修正了错误 网上搜索出来大概有三种: 1.在系统的环境变量或java JVM变量里面添加HADOOP_USE ...

  9. 访问HDFS报错:org.apache.hadoop.security.AccessControlException: Permission denied

    import org.apache.hadoop.conf.Configuration; import org.apache.hadoop.fs.FileSystem; import org.apac ...

随机推荐

  1. Android、iOS、和Web如何做灰度发布?

        主要参考了: https://www.zhihu.com/question/21714205     https://www.zhihu.com/question/28296375  一.概述 ...

  2. Java RandomAccessFile与MappedByteBuffer

    Java RandomAccessFile与MappedByteBuffer https://www.cnblogs.com/guazi/p/6838915.html

  3. MySQL之You can't specify target table for update in FROM clause解决办法

    这篇文章主要介绍了mysql中You can't specify target table for update in FROM clause错误解决方法,需要的朋友可以参考下 MySQL中You c ...

  4. spring+springmvc+ibatis整合注解方式实例

    需求说明 实现用户通过数据库验证登录需求.採用 Myeclipse+Tomcat 6.0+Mysql 5.0+JDK 1.6 2.数据库表 开发所用是Mysql数据库,仅仅建立单张用户表T_USER, ...

  5. mysql distinct 用法详解及优化

    本事例实验用表task,结构如下 MySQL> desc task; +-------------+------------+------+-----+-------------------+- ...

  6. 分析轮子(八)- List.java 各种遍历方式及遍历时移除元素的方法

    注:玩的是JDK1.7版本 1:先尝栗子,再分析,代码简单,注释清晰,可自玩一下 /** * @description:测试集合遍历和移除元素的方式 * @author:godtrue * @crea ...

  7. Kafka Docker集群搭建

    1. Zookeeper下载 http://apache.org/dist/zookeeper/ http://mirrors.hust.edu.cn/apache/zookeeper/zookeep ...

  8. HLS playlist典型示例

    [时间:2018-06] [状态:Open] [关键词:流媒体,HLS,m3u8,playlist,variant, alternate] 0 引言 本文主要是对apple官网上的echnical N ...

  9. TensorFlow 图片resize方法

    参见这篇博客 tensorflow里面用于改变图像大小的函数是tf.image.resize_images(image, (w, h), method):image表示需要改变此存的图像,第二个参数改 ...

  10. centos6.5上安装ftp服务

    这是之前搭建过,但没记录,因为昨天使用人过来说使用有问题,突然发现没有记录,好心慌,现在的记忆真的只有1周而已,穷和老都是原罪啊!! 环境准备:centos6.5 vm 安装ftp: