RHEL/CentOS 6/5-de CDH 4.X üçin Apache Oozie iş akymynyň meýilnamasyny guruň we sazlaň


Oozie Hadoop üçin açyk çeşme meýilleşdiriji, iş prosesini we işleriň arasyndaky utgaşdyrmagy aňsatlaşdyrýar. Giriş maglumatlary üçin işleriň arasyndaky garaşlylygy kesgitläp bileris we şonuň üçin iş meýilnamasyny ulanyp, işe garaşlylygy awtomatlaşdyryp bileris.

Bu gollanmada, Oozie-i baş düwünime (meselem, ussat host ady hökmünde we namenode/JT gurlan ýeri) gurdum, ýöne önümçilik ulgamynda oozie aýratyn Hadoop-da gurulmaly düwün.

Gurnama görkezmeleri iki bölege bölünýär, oňa A we B diýýäris.

  1. A. Oozi gurmak.
  2. B. Oozi konfigurasiýasy.

Ilki bilen “hostname” buýrugyny ulanyp, ulgam host adyny barlalyň.

 hostname

master

Usul A: RHEL/CentOS 6/5-de Oozie gurnama

CDH4 gurmak üçin buluteranyň sahypasyndan resmi CDH ammaryny ulanýarys. Resmi CDH göçürip almak bölümine giriň we CDH4 (ýagny 4.6) wersiýasyny göçürip alyň ýa-da ammary göçürip almak we gurmak üçin aşakdaky wget buýrugyny ulanyp bilersiňiz.

# wget http://archive.cloudera.com/cdh4/one-click-install/redhat/6/i386/cloudera-cdh-4-0.i386.rpm
# yum --nogpgcheck localinstall cloudera-cdh-4-0.i386.rpm

# wget http://archive.cloudera.com/cdh4/one-click-install/redhat/6/x86_64/cloudera-cdh-4-0.x86_64.rpm
# yum --nogpgcheck localinstall cloudera-cdh-4-0.x86_64.rpm
# wget http://archive.cloudera.com/cdh4/one-click-install/redhat/5/i386/cloudera-cdh-4-0.i386.rpm
# yum --nogpgcheck localinstall cloudera-cdh-4-0.i386.rpm

# wget http://archive.cloudera.com/cdh4/one-click-install/redhat/5/x86_64/cloudera-cdh-4-0.x86_64.rpm
# yum --nogpgcheck localinstall cloudera-cdh-4-0.x86_64.rpm

Bir gezek ulgamyňyzyň aşagynda CDH ammaryny goşduňyz, ulgamda Oozie gurmak üçin aşakdaky buýrugy ulanyp bilersiňiz.

 yum install oozie

Indi, oozie müşderisini guruň (ýokardaky buýruk müşderiniň gurnama bölegini öz içine almalydyr, ýöne buýrukdan aşakda synanyşmasaňyz).

 yum install oozie-client

Bellik: aboveokardaky gurnama ulgam işe başlanda oozie hyzmatyny hem düzýär. Gowy iş! Gurmagyň birinji bölümi bilen gutardyk, indi oozie sazlamak üçin ikinji bölüme geçeliň.

Usul B: RHEL/CentOS 6/5-de Oozie konfigurasiýasy

Oozie Hadoop bilen gönüden-göni täsirleşmeýänligi sebäpli, bu ýerde hiç hili kartalaşdyrylan konfigurasiýa gerek däl.

Üns beriň: oozie işlemeýän wagty ähli sazlamalary düzüň, bu oozie hyzmaty işlemeýän wagtynda aşakdaky ädimleri ýerine ýetirmeli diýmekdir.

Oozie-de DB-de gurlan “Derbi” bar, ýöne Mysql DB-ni ulanmagy maslahat berýärin. Geliň, indiki makalany ulanyp, MySQL maglumat bazasyny guralyň.

  1. MySQL maglumat bazasyny RHEL/CentOS 6/5
  2. guruň

Gurnama bölümi ýerine ýetirilenden soň, oozie DB döretmek we aşakda görkezilişi ýaly artykmaçlyklar bermek üçin indiki ädimiňize geçiň.

 mysql -uroot -p
Enter password:
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 3
Server version: 5.5.38 MySQL Community Server (GPL) by Remi

Copyright (c) 2000, 2014, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> create database oozie;
Query OK, 1 row affected (0.00 sec)

mysql> grant all privileges on oozie.* to 'oozie'@'localhost' identified by 'oozie';
Query OK, 0 rows affected (0.00 sec)

mysql> grant all privileges on oozie.* to 'oozie'@'%' identified by 'oozie';
Query OK, 0 rows affected (0.00 sec)

mysql> exit
Bye

Ondan soň, MySQL üçin Oozie häsiýetlerini düzüň. Oozie-site.xml faýly açyň we görkezilişi ýaly aşakdaky häsiýetleri redaktirläň.

 cd /etc/oozie/conf
 vi oozie-site.xml

Aşakdaky häsiýetleri giriziň (diňe ussany [meniň host adym] host adyňyz bilen çalşyň).

<property>
        <name>oozie.service.JPAService.jdbc.driver</name>
        <value>com.mysql.jdbc.Driver</value>
    </property>
    <property>
        <name>oozie.service.JPAService.jdbc.url</name>
        <value>jdbc:mysql://master:3306/oozie</value>
    </property>
    <property>
        <name>oozie.service.JPAService.jdbc.username</name>
        <value>oozie</value>
    </property>
    <property>
        <name>oozie.service.JPAService.jdbc.password</name>
        <value>oozie</value>
    </property>

MySQL JDBC birikdiriji sürüjisi JAR-ny Oozie lib katalogyna göçürip alyň. Munuň üçin terminalda aşakdaky çynlakaý buýrugy işlediň.

 cd /tmp/
 wget http://dev.mysql.com/get/Downloads/Connector-J/mysql-connector-java-5.1.31.tar.gz
 tar -zxf mysql-connector-java-5.1.31.tar.gz	
 cd mysql-connector-java-5.1.31
 cp mysql-connector-java-5.1.31-bin.jar /var/lib/oozie/

Aşakdaky buýruklary ýerine ýetirip, oozie maglumat bazasynyň shemasyny dörediň we munuň oozie ulanyjysy hökmünde işlemelidigini ýadyňyzdan çykarmaň.

 sudo -u oozie /usr/lib/oozie/bin/ooziedb.sh create -run
setting OOZIE_CONFIG=/etc/oozie/conf
setting OOZIE_DATA=/var/lib/oozie
setting OOZIE_LOG=/var/log/oozie
setting OOZIE_CATALINA_HOME=/usr/lib/bigtop-tomcat
setting CATALINA_TMPDIR=/var/lib/oozie
setting CATALINA_PID=/var/run/oozie/oozie.pid
setting CATALINA_BASE=/usr/lib/oozie/oozie-server-0.20
setting CATALINA_OPTS=-Xmx1024m
setting OOZIE_HTTPS_PORT=11443
...
DONE
Oozie DB has been created for Oozie version '3.3.2-cdh4.7.0'
The SQL commands have been written to: /tmp/ooziedb-8250405588513665350.sql

Oozie web konsolyny işletmek üçin ExtJS lib-i internetden göçürip almaly. Resmi CDH ExtJS sahypasyna giriň we ExtJS 2.2 wersiýa kitaphanalaryny göçürip alyň ýa-da bukjany aşakdaky buýruk bilen göçürip alyp bilersiňiz.

 cd /tmp/
 wget http://archive.cloudera.com/gplextras/misc/ext-2.2.zip
 unzip ext-2.2.zip
 mv ext-2.2 /var/lib/oozie/

Ahyrynda, aşakdaky buýruklary işledip, oozie serwerini işe giriziň.

 service oozie status
not running.

 service oozie start

 service oozie status
running

 oozie admin -oozie http://localhost:11000/oozie -status
System mode: NORMAL

Halaýan brauzeriňizi ulanyp, oozie UI açyň we IP adresiňizi görkeziň. Bu ýagdaýda meniň IP 192.168.1.129.

http://192.168.1.129:11000

Indi bu UI-ni görseňiz. Gutlaýarys !! Oozie-ni üstünlikli düzdüňiz.

Bu amal RHEL/CentOS 6/5-de üstünlikli synag edildi. Upcomingakynda çykjak makalalarymda oozie arkaly hasoop işlerini nädip düzmelidigini we tertipleşdirilmegini paýlaşmakçy. Has giňişleýin baglanyşykda boluň we pikirleriňizi düşündirmegi ýatdan çykarmaň.