关注服务器架构设计
关注互联网

A simple Spark test project

I start to learn Spark to process some log files, here is a simple example.

How to build Spark, please see http://spark.apache.org/docs/latest/building-spark.html

Scala file

sbt file(use sbt to build this example)

The result is  in result directory, two files, one is _SUCCESS that tells us the right result, the other one is “part-00000”, contains words and words’ count in this text file.

(package,1)
(For,2)
(Programs,1)
(processing.,1)
(Because, 1)
(The,1)
(cluster.,1)
(its,1)
([run,1)
(APIs,1)
(have,1)
(Try,1)

 

BTW. this article is written in Ubuntu, haven’t Chinese input method(English version).

未经允许不得转载:C++爱好者博客 » A simple Spark test project

评论 抢沙发

  • 昵称 (必填)
  • 邮箱 (必填)
  • 网址

ArkGameFrame游戏服务器解决方案

Github地址