A simple Spark test project

作者: NickYang 分类: 大数据,技术文章 发布时间: 2016-08-10 20:31

I start to learn Spark to process some log files, here is a simple example.

How to build Spark, please see http://spark.apache.org/docs/latest/building-spark.html

Scala file

sbt file(use sbt to build this example)

The result is  in result directory, two files, one is _SUCCESS that tells us the right result, the other one is “part-00000”, contains words and words’ count in this text file.

(package,1)
(For,2)
(Programs,1)
(processing.,1)
(Because, 1)
(The,1)
(cluster.,1)
(its,1)
([run,1)
(APIs,1)
(have,1)
(Try,1)

 

BTW. this article is written in Ubuntu, haven’t Chinese input method(English version).

如果觉得我的文章对您有用,请随意打赏。您的支持将鼓励我继续创作!

发表评论

电子邮件地址不会被公开。 必填项已用*标注