安装hadoop下的sqoop1.99.3及配置问题全解决-创新互联-成都创新互联网站建设

关于创新互联

多方位宣传企业产品与服务 突出企业形象

公司简介 公司的服务 荣誉资质 新闻动态 联系我们

安装hadoop下的sqoop1.99.3及配置问题全解决-创新互联

2016年8月27日,解决sqoop

创新互联致力于互联网网站建设与网站营销,提供成都网站制作、成都做网站、网站开发、seo优化、网站排名、互联网营销、微信小程序定制开发、公众号商城、等建站开发,创新互联网站建设策划专家,为不同类型的客户提供良好的互联网应用定制解决方案,帮助客户在新的全球化互联网环境中保持优势。

先下载的是sqoop1.99.7,结果配置的时候,出现了没有server/conf目录,以及启动时出现无法配置错误

./bin/sqoop.sh server start

yq@ubuntu:/opt/sqoop-1.99.7$ ./bin/sqoop.sh server start

Setting conf dir: ./bin/../conf

Sqoop home directory: /opt/sqoop-1.99.7

Starting the Sqoop2 server...

Sqoop2 server started.

dyq@ubuntu:/opt/sqoop-1.99.7$ 0   [main] INFO  org.apache.sqoop.core.SqoopServer  - Initializing Sqoop server.

58  [main] INFO  org.apache.sqoop.core.PropertiesConfigurationProvider  - Starting config file poller thread

Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/commons/configuration/Configuration

at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.(DefaultMetricsSystem.java:38)

at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.(DefaultMetricsSystem.java:36)

at org.apache.hadoop.security.UserGroupInformation$UgiMetrics.create(UserGroupInformation.java:120)

at org.apache.hadoop.security.UserGroupInformation.(UserGroupInformation.java:236)

at org.apache.sqoop.security.authentication.SimpleAuthenticationHandler.secureLogin(SimpleAuthenticationHandler.java:39)

at org.apache.sqoop.security.AuthenticationManager.initialize(AuthenticationManager.java:98)

at org.apache.sqoop.core.SqoopServer.initialize(SqoopServer.java:57)

at org.apache.sqoop.server.SqoopJettyServer.(SqoopJettyServer.java:67)

at org.apache.sqoop.server.SqoopJettyServer.main(SqoopJettyServer.java:177)

Caused by: java.lang.ClassNotFoundException: org.apache.commons.configuration.Configuration

at java.net.URLClassLoader$1.run(URLClassLoader.java:366)

at java.net.URLClassLoader$1.run(URLClassLoader.java:355)

at java.security.AccessController.doPrivileged(Native Method)

at java.net.URLClassLoader.findClass(URLClassLoader.java:354)

at java.lang.ClassLoader.loadClass(ClassLoader.java:425)

at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)

at java.lang.ClassLoader.loadClass(ClassLoader.java:358)

... 9 more

解决办法是下载sqoop1.99.3,然后哪找如下配置,搞掂:

  1. 2.解压文件到工作目录:

  2. hadoop@hadoopMaster:$ sudo tar -xvf /opt/hn/hadoop_family/sqoop-1.99.3-bin-hadoop200.tar.gz

  3. hadoop@hadoopMaster:mv /opt/hn/hadoop_family/sqoop-1.99.3-bin-hadoop200 /usr/local/sqoop

  4. 3.修改环境变量:

  5. hadoop@hadoopMaster:~$ vim /etc/profile

  6. 添加如下内容:

  7. #sqoop

  8. export SQOOP_HOME=/usr/sqoop-1.99.3

  9. export PATH=$SQOOP_HOME/bin:$PATH

  10. export CATALINA_HOME=$SQOOP_HOME/server

  11. export LOGDIR=$SQOOP_HOME/logs

  12. 保存退出即时生效:

  13. source /etc/profile

  14. 4.修改sqoop配置:

  15. hadoop@hadoopMaster:~$ vim /usr/local/sqoop/server/conf/sqoop.properties

  16. #修改指向我的hadoop的配置文件目录

  17. org.apache.sqoop.submission.engine.mapreduce.configuration.directory=/usr/local/hadoop-2.6.2/etc/hadoop

  18. #把hadoop目录下的jar包都引进来

  19. hadoop@hadoopMaster:~$ vim /usr/local/sqoop/server/conf/catalina.properties

  20. common.loader=/usr/local/hadoop/share/hadoop/common/*.jar,/usr/local/hadoop/share/hadoop/common/lib/*.jar,/usr/local/hadoop/share/hadoop/hdfs/*.jar,/usr/local/hadoop/share/hadoop/hdfs/lib/*.jar,/usr/local/hadoop/share/hadoop/mapreduce/*.jar,/usr/local/hadoop/share/hadoop/mapreduce/lib/*.jar,/usr/local/hadoop/share/hadoop/tools/*.jar,/usr/local/hadoop/share/hadoop/tools/lib/*.jar,/usr/local/hadoop/share/hadoop/yarn/*.jar,/usr/local/hadoop/share/hadoop/yarn/lib/*.jar,/usr/local/hadoop/share/hadoop/httpfs/tomcat/lib/*.jar

  21. 5.下载mysql驱动包

  22. mysql-connector-java-5.1.16-bin.jar

  23. 6.启动/停止sqoop200

  24. hadoop@hadoopMaster:/usr/local/sqoop/bin$ ./sqoop.sh server start/stop

跑起来试试:

dyq@ubuntu:/opt/sqoop-1.99.3$  ./bin/sqoop.sh server start

Sqoop home directory: /opt/sqoop-1.99.3

Setting SQOOP_HTTP_PORT:   12000

Setting SQOOP_ADMIN_PORT:   12001

Using  CATALINA_OPTS:

Adding to CATALINA_OPTS:   -Dsqoop.http.port=12000 -Dsqoop.admin.port=12001

Using CATALINA_BASE:  /opt/sqoop-1.99.3/server

Using CATALINA_HOME:  /opt/sqoop-1.99.3/server

Using CATALINA_TMPDIR: /opt/sqoop-1.99.3/server/temp

Using JRE_HOME:     /usr/java/jdk1.7.0_79/jre

Using CLASSPATH:    /opt/sqoop-1.99.3/server/bin/bootstrap.jar

另外有需要云服务器可以了解下创新互联scvps.cn,海内外云服务器15元起步,三天无理由+7*72小时售后在线,公司持有idc许可证,提供“云服务器、裸金属服务器、高防服务器、香港服务器、美国服务器、虚拟主机、免备案服务器”等云主机租用服务以及企业上云的综合解决方案,具有“安全稳定、简单易用、服务可用性高、性价比高”等特点与优势,专为企业上云打造定制,能够满足用户丰富、多元化的应用场景需求。


标题名称:安装hadoop下的sqoop1.99.3及配置问题全解决-创新互联
网站URL:http://kswsj.cn/article/djsidj.html

其他资讯