我写的是maven项目,pom文件为:

  1. <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
  2. <modelVersion>4.0.0</modelVersion>
  3. <groupId>kafkaStreaming</groupId>
  4. <artifactId>kafkaStreaming</artifactId>
  5. <version>0.0.1-SNAPSHOT</version>
  6. <properties>
  7. <maven.compiler.source>1.7</maven.compiler.source>
  8. <maven.compiler.target>1.7</maven.compiler.target>
  9. <encoding>UTF-8</encoding>
  10. <scala.version>2.10.4</scala.version>
  11. <spark.version>1.6.3</spark.version>
  12. <hadoop.version>2.6.4</hadoop.version>
  13. <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
  14. </properties>
  15.  
  16. <dependencies>
  17. <dependency>
  18. <groupId>org.scala-lang</groupId>
  19. <artifactId>scala-library</artifactId>
  20. <version>${scala.version}</version>
  21. <scope>compile</scope>
  22. </dependency>
  23.  
  24. <dependency>
  25. <groupId>org.apache.spark</groupId>
  26. <artifactId>spark-core_2.10</artifactId>
  27. <version>${spark.version}</version>
  28. </dependency>
  29.  
  30. <dependency>
  31. <groupId>org.apache.hadoop</groupId>
  32. <artifactId>hadoop-client</artifactId>
  33. <version>${hadoop.version}</version>
  34. </dependency>
  35.  
  36. <dependency>
  37. <groupId>org.apache.spark</groupId>
  38. <artifactId>spark-sql_2.10</artifactId>
  39. <version>${spark.version}</version>
  40. </dependency>
  41.  
  42. <dependency>
  43. <groupId>org.apache.spark</groupId>
  44. <artifactId>spark-streaming_2.10</artifactId>
  45. <version>${spark.version}</version>
  46. </dependency>
  47.  
  48. <dependency>
  49. <groupId>dom4j</groupId>
  50. <artifactId>dom4j</artifactId>
  51. <version>1.6.1</version>
  52. </dependency>
  53.  
  54. <dependency>
  55. <groupId>org.apache.spark</groupId>
  56. <artifactId>spark-streaming-kafka_2.10</artifactId>
  57. <version>1.6.1</version>
  58. </dependency>
  59.  
  60. <dependency>
  61. <groupId>org.apache.spark</groupId>
  62. <artifactId>spark-streaming-flume_2.10</artifactId>
  63. <version>${spark.version}</version>
  64. </dependency>
  65. <dependency>
  66. <groupId>log4j</groupId>
  67. <artifactId>log4j</artifactId>
  68. <version>1.2.12</version>
  69. </dependency>
  70. <dependency>
  71. <groupId>com.google.code.gson</groupId>
  72. <artifactId>gson</artifactId>
  73. <version>2.5</version>
  74. </dependency>
  75. <dependency>
  76. <groupId>com.alibaba</groupId>
  77. <artifactId>fastjson</artifactId>
  78. <version>1.2.4</version>
  79. </dependency>
  80.  
  81. <dependency>
  82. <groupId>redis.clients</groupId>
  83. <artifactId>jedis</artifactId>
  84. <version>2.8.0</version>
  85. </dependency>
  86. <!-- 2017-5-6添加hbase -->
  87. <!-- hbase依赖 -->
  88. <dependency>
  89. <groupId>org.apache.hbase</groupId>
  90. <artifactId>hbase-common</artifactId>
  91. <version>1.1.2</version>
  92. </dependency>
  93. <dependency>
  94. <groupId>org.apache.hbase</groupId>
  95. <artifactId>hbase-client</artifactId>
  96. <version>1.1.2</version>
  97. </dependency>
  98. <dependency>
  99. <groupId>org.apache.hbase</groupId>
  100. <artifactId>hbase-server</artifactId>
  101. <version>1.1.2</version>
  102. </dependency>
  103.  
  104. <dependency>
  105. <groupId>org.apache.spark</groupId>
  106. <artifactId>spark-hive_2.10</artifactId>
  107. <version>${spark.version}</version>
  108. </dependency>
  109.  
  110. <!-- hive依赖 -->
  111. <dependency>
  112. <groupId>org.apache.hive</groupId>
  113. <artifactId>hive-exec</artifactId>
  114. <version>1.2.1</version>
  115. </dependency>
  116. <dependency>
  117. <groupId>org.apache.hive</groupId>
  118. <artifactId>hive-jdbc</artifactId>
  119. <version>1.2.1</version>
  120. </dependency>
  121. <dependency>
  122. <groupId>org.apache.hive</groupId>
  123. <artifactId>hive-service</artifactId>
  124. <version>1.2.1</version>
  125. </dependency>
  126. <!-- <dependency> <groupId>cn.itcast.spark</groupId> <artifactId>bigdata</artifactId>
  127. <version>2.0</version> </dependency> -->
  128. <dependency>
  129. <groupId>org.apache.zookeeper</groupId>
  130. <artifactId>zookeeper</artifactId>
  131. <version>3.4.6</version>
  132. </dependency>
  133. <!-- 添加oracle jdbc driver -->
  134. <dependency>
  135. <groupId>com.oracle</groupId>
  136. <artifactId>ojdbc14</artifactId>
  137. <version>10.2.0.5.0</version>
  138. </dependency>
  139. <dependency>
  140. <groupId>commons-collections</groupId>
  141. <artifactId>commons-collections</artifactId>
  142. <version>3.2.2</version>
  143. </dependency>
  144. <dependency>
  145. <groupId>commons-lang</groupId>
  146. <artifactId>commons-lang</artifactId>
  147. <version>2.6</version>
  148. </dependency>
  149. <dependency>
  150. <groupId>org.apache.commons</groupId>
  151. <artifactId>commons-lang3</artifactId>
  152. <version>3.5</version>
  153. </dependency>
  154.  
  155. <dependency>
  156. <groupId>com.google.guava</groupId>
  157. <artifactId>guava</artifactId>
  158. <version>18.0</version>
  159. </dependency>
  160.  
  161. <!-- kafka -->
  162. <!-- <dependency>
  163. <groupId>org.apache.kafka</groupId>
  164. <artifactId>kafka_2.11</artifactId>
  165. <version>0.11.0.0</version>
  166. </dependency -->
  167. </dependencies>
  168.  
  169. <build>
  170. <resources>
  171. <resource>
  172. <directory>src/main/resources</directory>
  173. </resource>
  174. </resources>
  175. <plugins>
  176. <plugin>
  177. <groupId>org.apache.maven.plugins</groupId>
  178. <artifactId>maven-resources-plugin</artifactId>
  179. <configuration>
  180. <encoding>${project.build.sourceEncoding}</encoding>
  181. </configuration>
  182. <executions>
  183. <execution>
  184. <goals>
  185. <goal>copy-resources</goal>
  186. </goals>
  187. </execution>
  188. </executions>
  189. </plugin>
  190. <plugin>
  191. <groupId>net.alchim31.maven</groupId>
  192. <artifactId>scala-maven-plugin</artifactId>
  193. <version>3.2.2</version>
  194. <configuration>
  195. <recompileMode>incremental</recompileMode>
  196. <args>
  197. <arg>-target:jvm-1.7</arg>
  198. </args>
  199. <javacArgs>
  200. <javacArg>-source</javacArg>
  201. <javacArg>1.7</javacArg>
  202. <javacArg>-target</javacArg>
  203. <javacArg>1.7</javacArg>
  204. </javacArgs>
  205. </configuration>
  206. <executions>
  207. <execution>
  208. <id>scala-compile-first</id>
  209. <phase>process-resources</phase>
  210. <goals>
  211. <goal>compile</goal>
  212. </goals>
  213. <configuration>
  214. <args>
  215. <arg>-make:transitive</arg>
  216. <arg>-dependencyfile</arg>
  217. <arg>${project.build.directory}/.scala_dependencies</arg>
  218. </args>
  219. </configuration>
  220. </execution>
  221. <execution>
  222. <id>scala-test-compile</id>
  223. <phase>process-test-resources</phase>
  224. <goals>
  225. <goal>testCompile</goal>
  226. </goals>
  227. <configuration>
  228. <args>
  229. <arg>-make:transitive</arg>
  230. <arg>-dependencyfile</arg>
  231. <arg>${project.build.directory}/.scala_dependencies</arg>
  232. </args>
  233. </configuration>
  234. </execution>
  235. </executions>
  236. </plugin>
  237. <plugin>
  238. <groupId>org.apache.maven.plugins</groupId>
  239. <artifactId>maven-compiler-plugin</artifactId>
  240. <configuration>
  241. <source>1.7</source>
  242. <target>1.7</target>
  243. </configuration>
  244. <executions>
  245. <execution>
  246. <phase>compile</phase>
  247. <goals>
  248. <goal>compile</goal>
  249. </goals>
  250. </execution>
  251. </executions>
  252. </plugin>
  253. <plugin>
  254. <groupId>org.apache.maven.plugins</groupId>
  255. <artifactId>maven-shade-plugin</artifactId>
  256. <version>2.4.3</version>
  257. <executions>
  258. <execution>
  259. <phase>package</phase>
  260. <goals>
  261. <goal>shade</goal>
  262. </goals>
  263. <configuration>
  264. <filters>
  265. <filter>
  266. <artifact>*:*</artifact>
  267. <excludes>
  268. <exclude>META-INF/*.SF</exclude>
  269. <exclude>META-INF/*.DSA</exclude>
  270. <exclude>META-INF/*.RSA</exclude>
  271. </excludes>
  272. </filter>
  273. </filters>
  274. </configuration>
  275. </execution>
  276. </executions>
  277. </plugin>
  278.  
  279. </plugins>
  280. <pluginManagement>
  281. <plugins>
  282. <!--This plugin's configuration is used to store Eclipse m2e settings
  283. only. It has no influence on the Maven build itself. -->
  284. <plugin>
  285. <groupId>org.eclipse.m2e</groupId>
  286. <artifactId>lifecycle-mapping</artifactId>
  287. <version>1.0.0</version>
  288. <configuration>
  289. <lifecycleMappingMetadata>
  290. <pluginExecutions>
  291. <pluginExecution>
  292. <pluginExecutionFilter>
  293. <groupId>
  294. net.alchim31.maven
  295. </groupId>
  296. <artifactId>
  297. scala-maven-plugin
  298. </artifactId>
  299. <versionRange>
  300. [3.1.6,)
  301. </versionRange>
  302. <goals>
  303. <goal>compile</goal>
  304. <goal>testCompile</goal>
  305. </goals>
  306. </pluginExecutionFilter>
  307. <action>
  308. <ignore></ignore>
  309. </action>
  310. </pluginExecution>
  311. </pluginExecutions>
  312. </lifecycleMappingMetadata>
  313. </configuration>
  314. </plugin>
  315. <!-- 可运行jar包 -->
  316. <plugin>
  317. <groupId>org.apache.maven.plugins</groupId>
  318. <artifactId>maven-shade-plugin</artifactId>
  319. <executions>
  320. <execution>
  321. <phase>package</phase>
  322. <goals>
  323. <goal>shade</goal>
  324. </goals>
  325. <configuration>
  326. <transformers>
  327. <transformer implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
  328. <mainClass>kafkaSparkStream.KafkaToSpark</mainClass>
  329. </transformer>
  330. </transformers>
  331. <filters>
  332. <filter>
  333. <artifact>*:*</artifact>
  334. <excludes>
  335. <exclude>META-INF/*.SF</exclude>
  336. <exclude>META-INF/*.DSA</exclude>
  337. <exclude>META-INF/*.RSA</exclude>
  338. </excludes>
  339. </filter>
  340. </filters>
  341. </configuration>
  342. </execution>
  343. </executions>
  344. </plugin>
  345. </plugins>
  346. </pluginManagement>
  347. </build>
  348.  
  349. </project>

打包步骤:

因为pom文件中已经写有全类名,所以打的包是可运行的jar包

《1》右键pom文件

《2》先clean一下

《3》、在install

《4》、打包好,选择jar,选择那个大的

eclipse和scala整合,打包配置文件及打包步骤的更多相关文章

  1. Eclipse+Maven+Scala Project+Spark | 编译并打包wordcount程序

    学习用Eclipse+Maven来构建并打包一个简单的单词统计的例程. 本项目源码已托管于Github –>[Spark-wordcount] 第一步 在EclipseIDE中安装Scala插件 ...

  2. springmvc 项目完整示例02 项目创建-eclipse创建动态web项目 配置文件 junit单元测试

    包结构 所需要的jar包直接拷贝到lib目录下 然后选定 build path 之后开始写项目代码 配置文件 ApplicationContext.xml <?xml version=" ...

  3. iOS开发之 xcode6 APP 打包提交审核详细步骤

    一. 在xcode6.1和ios10.10.1环境下实现app发布 http://blog.csdn.net/mad1989/article/details/8167529 http://jingya ...

  4. 给Eclipse中hibernate.cfg.xml配置文件加提示

    在hibernate框架需要的jar包中找到hibernate3.jar,并用压缩软件打开,如图: 2 选择org文件夹--打开下一级文件夹 3 点击类型,方便找到dtd文件,下拉查看dtd文件,有两 ...

  5. 基于Eclipse的scala应用开发

    原创文章,转载请注明: 转载自www.cnblogs.com/tovin/p/3823968.html 为了更好的学习scala语言,本文介绍如何基于Maven来构建scala项目 1.首先参照www ...

  6. eclipse的scala环境搭建

    两种方法使eclipse安装scala环境(eclipse luna) 1.下载eclipse for scala IDE http://scala-ide.org/download/sdk.html ...

  7. android-xBuild apk差分与合成,zip差分与合成,lua打包,apk打包,png/jpg图片压缩

    android-xBuild 是一项集成了apk差分与合成,zip差分与合成,lua打包.apk打包,png/jpg图片压缩五大功能的开源项目 (github地址:https://github.com ...

  8. [error] eclipse编写spring等xml配置文件时只有部分提示,tx无提示

    eclipse编写spring等xml配置文件时只有<bean>.<context>等有提示,其他标签都没有提示 这时就需要做以下两步操作(下面以事务管理标签为例) 1,添加命 ...

  9. Eclipse+maven+scala+spark环境搭建

    准备条件 我用的Eclipse版本 Eclipse Java EE IDE for Web Developers. Version: Luna Release (4.4.0) 我用的是Eclipse ...

随机推荐

  1. hugepage设置

    1.设置memlock 编辑/etc/security/limits.conf 增加: * Soft memlock 稍小于RAM值 * hard memlock 稍小于RAM值 Memlock,锁定 ...

  2. 两个gif图片动画效果

    <div className="uploading-animation-tip-wrap"> <img src={require('~/shared/assets ...

  3. HTTP无状态协议理解

    TTP协议是无状态协议. 无状态是指协议对于事务处理没有记忆能力.缺少状态意味着如果后续处理需要前面的信息,则它必须重传,这样可能导致每次连接传送的数据量增大.另一方面,在服务器不需要先前信息时它的应 ...

  4. Python——初识网络爬虫(网页爬取)

    网络爬虫(又称为网页蜘蛛,网络机器人,在FOAF社区中间,更经常的称为网页追逐者),是一种按照一定的规则,自动地抓取万维网信息的程序或者脚本.另外一些不常使用的名字还有蚂蚁.自动索引.模拟程序或者蠕虫 ...

  5. IOS开发之——绘图(CGContext)

    0 CGContextRef context = UIGraphicsGetCurrentContext(); 设置上下文 1 CGContextMoveToPoint 开始画线 2 CGContex ...

  6. bootstrap 模态窗口 多重/多个弹窗滚动条补丁

    由于bootstrap的模态窗口默认不支持多次弹出, 在关闭的时候会有滚动条消失的问题. 经过观察和查看源码, 发现在开启和关闭的时候会在body上增加/减少一个"modal-open&qu ...

  7. Android NDK 学习之接受Java传入的字符串

    本博客主要是在Ubuntu 下开发,且默认你已经安装了Eclipse,Android SDK, Android NDK, CDT插件. 在Eclipse中添加配置NDK,路径如下Eclipse-> ...

  8. 解决 google 浏览器记住密码导致输入框样式改变(变成淡黄色背景)

    直接在页面上使用css代码: input:-webkit-autofill , textarea:-webkit-autofill, select:-webkit-autofill { -webkit ...

  9. laravel管理模型插入

    post控制器public function comment(Post $post,Request $request){ try{ if(empty($request->content)){ E ...

  10. VUe键盘修饰符及自定义指令获取焦点

    首先需要在keyup事件之后. 修饰符 来绑定事件 <body> <div class="box"> <!-- 这里的 @keyup.enter=&q ...