没有合适的资源?快使用搜索试试~ 我知道了~
oozie 提交任务参数传递到下一个任务节点 oozie 提交任务参数传递到下一个任务节点
资源推荐
资源详情
资源评论
Java Acon
目录
1.官网解释..................................................................................................................................... 1
2.JavaAcon 使用........................................................................................................................... 4
2.1 编写自定义功能的 main 函数.......................................................................................... 4
2.2 编写 workow.xml 文件.................................................................................................... 6
2.3 编写 job.properes 文件.................................................................................................. 7
2.4 部署.................................................................................................................................. 7
2.5 运行.................................................................................................................................. 8
1.官网解释
Thejavaaction will execute thepublic static void main(String[] args)method of the
specied main Java class.
Java applications are executed in the Hadoop cluster as map-reduce job with a single Mapper task.
The work&ow job will wait until the java application completes its execution before continuing to the next
action.
Thejavaaction has to be congured with the job-tracker, name-node, main Java class, JVM options and
arguments.
To indicate anokaction transition, the main Java class must complete gracefully themainmethod
invocation.
To indicate anerroraction transition, the main Java class must throw an exception.
The main Java class must not callSystem.exit(int n)as this will make thejavaaction to do
anerrortransition regardless of the used exit code.
Ajavaaction can be congured to perform HDFS les/directories cleanup before starting the Java
application. This capability enables Oozie to retry a Java application in the situation of a transient or non-
transient failure (This can be used to cleanup any temporary data which may have been created by the Java
application in case of failure).
Ajavaaction can create a Hadoop conguration. The Hadoop conguration is made available as a local le
to the Java application in its running directory, the le name isoozie-action.conf.xml. Similar
tomap-reduceandpigactions it is possible to refer ajob.xmlle and using inline conguration
properties. For repeated conguration properties later values override earlier ones.
Inline property values can be parameterized (templatized) using EL expressions.
The Hadoopmapred.job.tracker(=job-tracker=) andfs.default.name(=name-node=)
properties must not be present in thejob-xmland in the inline conguration.
As withmap-reduceandpigactions, it is possible to add les and archives to be available to the
Java application. Refer to section [#FilesAchives][Adding Files and Archives for the Job].
Thecapture-outputelement can be used to propagate values back into Oozie context, which can
then be accessed via EL-functions. This needs to be written out as a java properties format le. The lename
is obtained via a System property specied by the
constantJavaMainMapper.OOZIE_JAVA_MAIN_CAPTURE_OUTPUT_FILE
IMPORTANT:In order for a Java action to succeed on a secure cluster, it must propagate the Hadoop
delegation token like in the following code snippet (this is benign on non-secure clusters):
// propagate delegation related props from launcher job to MR job
if (System.getenv("HADOOP_TOKEN_FILE_LOCATION") != null) {
jobConf.set("mapreduce.job.credentials.binary",
System.getenv("HADOOP_TOKEN_FILE_LOCATION"));
}
IMPORTANT:Because the Java application is run from within a Map-Reduce job, from Hadoop 0.20. onwards
a queue must be assigned to it. The queue name must be specied as a conguration property.
Syntax:
<work&ow-app name="[WF-DEF-NAME]" xmlns="uri:oozie:work&ow:0.1">
...
<action name="[NODE-NAME]">
<java>
<job-tracker>[JOB-TRACKER]</job-tracker>
<name-node>[NAME-NODE]</name-node>
<prepare>
<delete path="[PATH]"/>
...
<mkdir path="[PATH]"/>
...
</prepare>
剩余7页未读,继续阅读
资源评论
wenxiaogen
- 粉丝: 21
- 资源: 14
上传资源 快速赚钱
- 我的内容管理 展开
- 我的资源 快来上传第一个资源
- 我的收益 登录查看自己的收益
- 我的积分 登录查看自己的积分
- 我的C币 登录后查看C币余额
- 我的收藏
- 我的下载
- 下载帮助
最新资源
- vlmcsd-1113-2020-03-28-Hotbird64(最新版本KMS)
- 433.基于SpringBoot的冷链物流系统(含报告).zip
- com.harmonyos4.exception.PowerFailureException(怎么解决).md
- 使用 Python 字典统计字符串中每个字符的出现次数.docx
- com.harmonyos4.exception.SystemBootFailureException(怎么解决).md
- 球队获胜数据集.zip
- ERR-NULL-POINTER(解决方案).md
- <项目代码>YOLOv8 航拍行人识别<目标检测>
- 计算机网络-socket-inet-master.zip
- Java编程学习路线:从基础到实战全攻略
资源上传下载、课程学习等过程中有任何疑问或建议,欢迎提出宝贵意见哦~我们会及时处理!
点击此处反馈
安全验证
文档复制为VIP权益,开通VIP直接复制
信息提交成功