After a few hours tweaking and googling, I managed to install apache oozie 3.3.2 on Hadoop 1.1.1.
The document provided in apache oozie 3.3.2 is not very clear. After some googling, I found this blog is a great guide for you install oozie 3.3.1 on Hadoop 0.23.0.
http://srikanthayalasomayajulu.blogspot.sg/2013/02/apache-oozie-331-installation-on-apache.html
If you want to install oozie on MRv2/YARN, please follow the blog post.
Here we go, we're going to install oozie 3.3.2 on Hadoop 1.1.1, which is offically supported version.
Apache oozie 3.3.2 official quick start page: http://oozie.apache.org/docs/3.3.2/DG_QuickStart.html
My local environment:
- Apache Hadoop 1.1.1 pseudo setup
- Apache oozie 3.3.2, the latest release
- Java 1.7 u25
- Apache Maven 3.0.4
Oozie server installation:
- Download apache oozie 3.3.2 via: http://www.apache.org/dyn/closer.cgi/oozie/3.3.2
- Unpack the oozie-3.3.2.tar.gz under development directory: ~/dev/oozie-3.3.2
- The following two properties are required in hadoop core-site.xml:
-
<!-- OOZIE --> <property> <name>hadoop.proxyuser.[OOZIE_SERVER_USER].hosts</name> <value>[OOZIE_SERVER_HOSTNAME]</value> </property> <property> <name>hadoop.proxyuser.[OOZIE_SERVER_USER].groups</name> <value>[USER_GROUPS_THAT_ALLOW_IMPERSONATION]</value> </property>
Replaced [OOZIE_SERVER_USER] with gsun , [OOZIE_SERVER_HOSTNAME] with * and [OOZIE_GROUPS_THAT_ALLOW_IMPERSONATION] with * . TIP: in this blog post, the $OOZIE_HOME refers to ~/dev/oozie-3.3.2 - Download extjs-2.2.zip, don't unzip it right now.
- Building a distro
-
cd $OOZIE_HOME ./bin/mkdistro.sh -DskipTests
It will take a while, go get a coffee, you'll be fine. - If the build successfully, we could go to the next step. In $OOZIE_HOME a directory called *hadooplibs/* directory will be created containing the Hadoop JARs for the versions of Hadoop that the Oozie distribution supports.
MacBookPro:oozie-3.3.2 gsun$ cd hadooplibs/ MacBookPro:hadooplibs gsun$ ls hadoop-1/ hadoop-3/ hadoop-distcp-2/ hadoop-test-1/ hadoop-test-3/ target/ hadoop-2/ hadoop-distcp-1/ hadoop-distcp-3/ hadoop-test-2/ pom.xml MacBookPro:hadooplibs gsun$ tree hadoop-1/ hadoop-1/ ├── pom.xml └── target ├── archive-tmp ├── hadooplibs │ └── hadooplib-1.1.1.oozie-3.3.2 │ ├── commons-beanutils-1.7.0.jar │ ├── commons-beanutils-core-1.8.0.jar │ ├── commons-codec-1.4.jar │ ├── commons-collections-3.2.1.jar │ ├── commons-configuration-1.6.jar │ ├── commons-digester-1.8.jar │ ├── commons-el-1.0.jar │ ├── commons-io-2.1.jar │ ├── commons-lang-2.4.jar │ ├── commons-logging-1.1.jar │ ├── commons-math-2.1.jar │ ├── commons-net-1.4.1.jar │ ├── hadoop-client-1.1.1.jar │ ├── hadoop-core-1.1.1.jar │ ├── hsqldb-1.8.0.7.jar │ ├── jackson-core-asl-1.8.8.jar │ ├── jackson-mapper-asl-1.8.8.jar │ ├── log4j-1.2.16.jar │ ├── oro-2.0.8.jar │ └── xmlenc-0.52.jar ├── maven-archiver │ └── pom.properties └── oozie-hadoop-1.1.1.oozie-3.3.2.jar 5 directories, 23 files MacBookPro:hadooplibs gsun$
- Create a directory named libext/ in $OOZIE_HOME, copy all of jars inside hadoop-1 into $OOZIE_HOME/libext/
- Building with ExtJS library for Oozie Web Console. Copy the extjs-2.2.zip into $OOZIE_HOME/webapp/src/main/webapp, then unzip it.
- Run oozie-setup.sh to create an oozie.war file.
> cd $OOZIE_HOME > ./distro/target/oozie-3.3.2-distro/oozie-3.3.2/bin/oozie-setup.sh -extjs $OOZIE_HOME/webapp/src/main/webapp/ext-2.2.zip
If the build success, go to the next step, or refer to apache Oozie official docs for oozie-setup.sh usage. - Copy the newly minted oozie.war file to your Tomcat deployment directory.
> cd $OOZIE_HOME > vi ./distro/target/oozie-3.3.2-distro/oozie-3.3.2/conf/oozie-site.xml
Change the property below to true as shown.<property> <name>oozie.service.JPAService.create.db.schema</name> <value>true</value> <description> Creates Oozie DB. If set to true, it creates the DB schema if it does not exist. If the DB schema exists is a NOP. If set to false, it does not create the DB schema. If the DB schema does not exist it fails start up. </description> </property>
- Run below command:
> cd $OOZIE_HOME > ./distro/target/oozie-3.3.2-distro/oozie-3.3.2/bin/ooziedb.sh create -sqlfile oozie.sql -run
if the command is successfully run, the SQL has been created successfully.
Start the oozie server.
> cd $OOZIE_HOME ./distro/target/oozie-3.3.2-distro/oozie-3.3.2/bin/oozied.sh run
- From quick start in Oozie official site, we know that all Oozie server scripts (=oozie-setup.sh=, oozied.sh , oozie-start.sh , oozie-run.sh and oozie-stop.sh ) run only under the Unix user that owns the Oozie installation directory, if necessary use sudo -u OOZIE_USER when invoking the scripts.As of Oozie 3.3.2, use of oozie-start.sh , oozie-run.sh , and oozie-stop.sh has been deprecated and will print a warning. The oozied.sh script should be used instead; passing it start , run , or stop as an argument will perform the behaviors of oozie-start.sh , oozie-run.sh , and oozie-stop.sh respectively.Check the Oozie log file ./distro/target/oozie-3.3.2-distro/oozie-3.3.2/logs/oozie.log to ensure Oozie started properly.
- Using the Oozie command line tool check the status of Oozie:
- $ ./distro/target/oozie-3.3.2-distro/oozie-3.3.2/bin/oozie admin -oozie http://[ip address of namenode]:11000/oozie -statusIn my case, the Oozie server can not start due to below exception:
java.lang.NoClassDefFoundError: org/apache/commons/configuration/Configuration java.lang.NoClassDefFoundError: org/apache/hadoop/util/ReflectionUtils
- You have to copy hadoop-core.jar and commons-configuration.jar under $HADOOP_HOME/lib into $OOZIE_HOME/distro/target/oozie-3.3.2-distro/oozie-3.3.2/oozie-server/webapps/oozie/WEB-INF/lib . You should see the oozie server up and running. Enjoy oozie!
相关推荐
oozie-4.1源码。github下载的。 oozie-4.1源码。github下载的。
Apache Oozie-5.2.1源码编译包
oozie-5.1.0.tar.gz 编译结果,受限上传大小,文件分3部分
oozie-5.1.0.tar.gz 编译结果,受限上传大小,文件分3部分
oozie-4.3.1.tar.gz 源码,可以利用该tar包进行编译安装oozie
linux.64 下的 oozie-4.3.0.tar.gz 源码包 解压后编译即可 注意对应的版本 cd bin ./mkdistro.sh -Phadoop-2 -Dhadoop.auth.version=2.8.2 -Ddistcp.version=2.8.2 -Dhadoop.version=2.8.2 -Dsqoop.version=1.4.6 -...
oozie 4.1.0 linux安装包
之前公司需要结合hadoop-2.7.2搭建oozie-4.2.0的时候,一直不知从何下手,官网下的包需要结合hadoop版本进行二次编译,手动编译很多次都一直中断, 这个包结合hadoop-2.7.2进行编译的,希望对需要搭建oozie-4.2.0的...
2019-06-25 最新oozie5.0.0.tar.gz基于工作流调度hadoop作业web工具
oozie-core
[atguigu@hadoop102 oozie-4.0.0-cdh5.3.6]$ tar -zxvf oozie-examples.tar.gz 2)创建工作目录 [atguigu@hadoop102 oozie-4.0.0-cdh5.3.6]$ mkdir oozie-apps/ 3)拷贝任务模板到oozie-apps/目录 [atguigu@hadoop...
oozie-5.1.0.tar.gz 编译结果,受限上传大小,文件分3部分
oozie 权威图书。pdf原版 2015-05-08: First Release Mohammad Kamrul Islam & Aravind Srinivasan
oozie-client安装npm安装oozie-client得到帮助节点app.js-帮助选项: -s,--save保存参数-undefined,--cluster hdinsight群集名称(期望值)-undefined,--user用户(期望值)-undefined,--pass password(期望值...
oozie-4.2.0
http://archive.cloudera.com/cdh5/cdh/5/oozie-4.1.0-cdh5.5.2.tar.gz 文件较大,1.6G 还需要下载ExjJS,这是扩展的JavaScript的UI桌面框架。 必须是2.2版本的,这是官网指定的,已经写死在oozied.sh中。下载地址:...
andlaz/hadoop-oozie su oozie -c 'oozie-setup.sh sharelib create -fs hdfs://namenode:8020' 启动Ooozie docker run -d --name oozie -p 0.0.0.0:11000 -p 0.0.0.0:11001:11001 \ andlaz/hadoop-oozie su oozie ...
oozie 提交任务参数传递到下一个任务节点 oozie 提交任务参数传递到下一个任务节点
oozie-graphite包含一些有用的粘合剂,用于将操作数据从 oozie 包/协调器和/或 oozie-internal 仪器推送到石墨中。 兼容性 版本 1.0 + 版本 1.1.0 + 如何构建 使用 ,只需使用捆绑和预配置的 gradlew 包装器。 ...
Oozie Impala 动作 这大量借鉴了。 使用说明 命令行界面 ...$ java -jar target/oozie-impala-action-1.0-jar-with-dependencies.jar sample.sql " $CONNECTION_URL " 乌兹 参考run-oozie/workflow.xml的例子