从协议VersionedProtocol开始4——AdminOperationsProtocol、InterTrackerProtocol、JobSubmissionProtocol、TaskUmbilicalProtocol
1、package org.apache.hadoop.mapred这四个协议都在这个包下。
2、从最简单的AdminOperationsProtocol看,
void refreshQueues() throws IOException;
void refreshNodes() throws IOException;
boolean setSafeMode(JobTracker.SafeModeAction safeModeAction) throws IOException;
3、相对于hdfs里边那几个协议,这几个协议都经常修改,不是很稳定
InterTrackerProtocol
HeartbeatResponse heartbeat(TaskTrackerStatus status,
boolean restarted,
boolean initialContact,
boolean acceptNewTasks,
short responseId)
throws IOException;
public String getFilesystemName() throws IOException;
public void reportTaskTrackerError(String taskTracker,
String errorClass,
String errorMessage) throws IOException;
TaskCompletionEvent[] getTaskCompletionEvents(JobID jobid, int fromEventId
, int maxEvents) throws IOException;
public String getSystemDir();
public String getBuildVersion() throws IOException;
public String getVIVersion() throws IOException;
JobSubmissionProtocol
public JobID getNewJobId() throws IOException;
public JobStatus submitJob(JobID jobName, String jobSubmitDir, Credentials ts)
throws IOException;
public ClusterStatus getClusterStatus(boolean detailed) throws IOException;
public AccessControlList getQueueAdmins(String queueName) throws IOException;
public void killJob(JobID jobid) throws IOException;
public void setJobPriority(JobID jobid, String priority)
throws IOException;
public boolean killTask(TaskAttemptID taskId, boolean shouldFail) throws IOException;
public JobProfile getJobProfile(JobID jobid) throws IOException;
public JobStatus getJobStatus(JobID jobid) throws IOException;
public Counters getJobCounters(JobID jobid) throws IOException;
public TaskReport[] getMapTaskReports(JobID jobid) throws IOException;
public TaskReport[] getReduceTaskReports(JobID jobid) throws IOException;
public TaskReport[] getCleanupTaskReports(JobID jobid) throws IOException;
public TaskReport[] getSetupTaskReports(JobID jobid) throws IOException;
public String getFilesystemName() throws IOException;
public JobStatus[] jobsToComplete() throws IOException;
public JobStatus[] getAllJobs() throws IOException;
public TaskCompletionEvent[] getTaskCompletionEvents(JobID jobid
, int fromEventId, int maxEvents) throws IOException;
public String[] getTaskDiagnostics(TaskAttemptID taskId)
throws IOException;
public String getSystemDir();
public String getStagingAreaDir() throws IOException;
public JobQueueInfo[] getQueues() throws IOException;
public JobQueueInfo getQueueInfo(String queue) throws IOException;
public JobStatus[] getJobsFromQueue(String queue) throws IOException;
public QueueAclsInfo[] getQueueAclsForCurrentUser() throws IOException;
public
Token<DelegationTokenIdentifier> getDelegationToken(Text renewer
) throws IOException,
InterruptedException;
public long renewDelegationToken(Token<DelegationTokenIdentifier> token
) throws IOException,
InterruptedException;
public void cancelDelegationToken(Token<DelegationTokenIdentifier> token
) throws IOException,InterruptedException;
TaskUmbilicalProtocol
JvmTask getTask(JvmContext context) throws IOException;
boolean statusUpdate(TaskAttemptID taskId, TaskStatus taskStatus,
JvmContext jvmContext) throws IOException, InterruptedException;
void reportDiagnosticInfo(TaskAttemptID taskid, String trace,
JvmContext jvmContext) throws IOException;
void reportNextRecordRange(TaskAttemptID taskid, SortedRanges.Range range,
JvmContext jvmContext) throws IOException;
boolean ping(TaskAttemptID taskid, JvmContext jvmContext) throws IOException;
void done(TaskAttemptID taskid, JvmContext jvmContext) throws IOException;
void commitPending(TaskAttemptID taskId, TaskStatus taskStatus,
JvmContext jvmContext) throws IOException, InterruptedException;
boolean canCommit(TaskAttemptID taskid, JvmContext jvmContext) throws IOException;
void shuffleError(TaskAttemptID taskId, String message, JvmContext jvmContext)
throws IOException;
void fsError(TaskAttemptID taskId, String message, JvmContext jvmContext)
throws IOException;
void fatalError(TaskAttemptID taskId, String message, JvmContext jvmContext)
throws IOException;
MapTaskCompletionEventsUpdate getMapCompletionEvents(JobID jobId,
int fromIndex,
int maxLocs,
TaskAttemptID id,
JvmContext jvmContext)
throws IOException;
void updatePrivateDistributedCacheSizes(org.apache.hadoop.mapreduce.JobID jobId,
long[] sizes) throws IOException;
从协议VersionedProtocol开始4——AdminOperationsProtocol、InterTrackerProtocol、JobSubmissionProtocol、TaskUmbilicalProtocol的更多相关文章
- 从协议VersionedProtocol开始
VersionedProtocol协议是Hadoop的最顶层协议接口的抽象:5--3--3共11个协议,嘿嘿 1)HDFS相关 ClientDatanodeProtocol:client与datano ...
- 从协议VersionedProtocol开始3——ClientProtocol、DatanodeProtocol、NamenodeProtocol、RefreshAuthorizationPolicyProtocol、RefreshUserMappingsProtocol
1.ClientProtocol这个玩意的版本号是61L:DatanodeProtocol 是26L:NamenodeProtocol是 3L;RefreshAuthorizationPolicyPr ...
- 从协议VersionedProtocol开始2——ClientDatanodeProtocol和InterDatanodeProtocol
1.首先,我看的是hadoop1.2.1 这个里边,有点奇怪ClientDatanodeProtocol的versionID是4,但是InterDatanodeProtocol的versionID是3 ...
- 从协议VersionedProtocol开始1
Phase 0: Make a plan You must first decide what steps you're going to have in your process. It sound ...
- Hadoop系列番外篇之一文搞懂Hadoop RPC框架及细节实现
@ 目录 Hadoop RPC 框架解析 1.Hadoop RPC框架概述 1.1 RPC框架特点 1.2 Hadoop RPC框架 2.Java基础知识回顾 2.1 Java反射机制与动态代理 2. ...
- MapReduce剖析笔记之四:TaskTracker通过心跳机制获取任务的流程
上一节分析到了JobTracker把作业从队列里取出来并进行了初始化,所谓的初始化,主要是获取了Map.Reduce任务的数量,并统计了哪些DataNode所在的服务器可以处理哪些Split等等,将这 ...
- MapReduce剖析笔记之二:Job提交的过程
上一节以WordCount分析了MapReduce的基本执行流程,但并没有从框架上进行分析,这一部分工作在后续慢慢补充.这一节,先剖析一下作业提交过程. 在分析之前,我们先进行一下粗略的思考,如果要我 ...
- hadoop之JobTracker功能分析
JobTracker是整个MapReduce计算框架中的主服务,相当于集群的“管理者”,负责整个集群的作业控制和资源管理.本文对JobTracker的启动过程及心跳接收与应答两个主要功能进行分析. 1 ...
- 【Hadoop代码笔记】通过JobClient对Jobtracker的调用详细了解Hadoop RPC
Hadoop的各个服务间,客户端和服务间的交互采用RPC方式.关于这种机制介绍的资源很多,也不难理解,这里不做背景介绍.只是尝试从Jobclient向JobTracker提交作业这个最简单的客户端服务 ...
随机推荐
- golang json
1.Go语言的JSON 库 Go语言自带的JSON转换库为 encoding/json 1.1)其中把对象转换为JSON的方法(函数)为 json.Marshal(),其函数原型如下 func Mar ...
- 批处理+VBS+注册表实现开机自动启动EXE程序
批处理+VBS+注册表实现WINDOWS开机自动启动EXE程序 以下都是基于WINDOWS系统. 我们都知道当我们有想某个程序在开机时自动运行,只能有三个方式: 1.做成服务,然后对服务进行配置为自动 ...
- grep的用法
grep的用法首先创建我们练习grep命令时需要用到的demo文件demo_file. $ cat demo_file THIS LINE IS THE 1ST UPPER CASE LINE IN ...
- J2EE 第二阶段项目之JUnit4进行单元测试(五)
今天学习了JUnit4进行单元测试.这样就可以不写页面直接进行过功能模块测试.也不是很深入的了解. JUnit4和自己写的代码可以分割开来. 首先呢准备两个jar包: 可以对mapper进行测试,当然 ...
- 静态库冲突的解决办法:duplicate symbol
昨天在做微信sdk和xmpp的集成,发现各自单独集成没问题,一起集成却总报错,百度了好一会儿才知道,这应该是库冲突造成的问题 然后参考了很多文章,跟着敲敲一遍,却发现问题多多,最后主要综合结合了这两个 ...
- about opencl
Platform:LG G3,Adreno 330 1. 8M(3264x2448) memmap方式读入时间24ms,读出时间12ms,时间与内存大小基本成线性关系.使用memmap 与 memco ...
- form in drupal
qin_form_ajax_example_form($form, &$form_state)类似函数的参数永远都是一样的,最多把$form前面也加上& 当没有实现页面跳转时,$for ...
- js到处excel
<!DOCTYPE html> <html> <head> <meta http-equiv="Content-Type" content ...
- Timer与TimerTask的真正原理&使用介绍
转载: Timer与TimerTask的真正原理&使用介绍 其实就Timer来讲就是一个调度器,而TimerTask呢只是一个实现了run方法的一个类,而具体的TimerTask需要由你自己来 ...
- struts2 拦截器的注册在strut.xml中
根据需要:需要在struts.xml中注册拦截器为以下内容 根据自己的业务需要更改参数 <!DOCTYPE struts PUBLIC "-//Apache Software Fou ...