Ask Your Question
1

How can I use "Spark" package after installing it? Can't find it anywhere

asked 2015-09-15 08:51:29 -0600

paladini gravatar image

updated 2015-09-15 08:52:39 -0600

Hello guys,

I have to install Apache Spark on a Fedora 22 (actually, a Fedora 22 VM) and I discovered the package "Spark". So I've installed it with sudo dnf install spark (and it downloaded more than 400mb among A LOT of packages).

It's everything okay, except that I can't run spark or any other related command on my command line. This simply doesn't exist, can't find any link with "spark" in /usr/bin. After that, I have looked for Spark in my whole computer (with whereis and locate) and I didn't find anything useful.

Please, help me! How can I use Spark package after install it? Check this link about the Spark package.

edit retag flag offensive close merge delete

3 Answers

Sort by ยป oldest newest most voted
2

answered 2015-09-17 01:56:44 -0600

hedayat gravatar image

Let's see what is in spark package:

[root@hvlap hedayat]# dnf -C repoquery -l spark
Last metadata expiration check performed 6:53:39 ago on Thu Sep 17 04:24:25 2015.
/usr/share/doc/spark
/usr/share/doc/spark/LICENSE
/usr/share/doc/spark/README.md
/usr/share/java/spark
/usr/share/java/spark/spark-bagel.jar
/usr/share/java/spark/spark-core.jar
/usr/share/java/spark/spark-graphx.jar
/usr/share/java/spark/spark-mllib.jar
/usr/share/java/spark/spark-streaming.jar
/usr/share/maven-fragments/spark
/usr/share/maven-poms/JPP.spark-spark-bagel.pom
/usr/share/maven-poms/JPP.spark-spark-core.pom
/usr/share/maven-poms/JPP.spark-spark-graphx.pom
/usr/share/maven-poms/JPP.spark-spark-mllib.pom
/usr/share/maven-poms/JPP.spark-spark-streaming.pom

So, there are no executable files in the main spark package. And as I looked through the spark package dependencies, I feel that you won't find spark launch scripts in Fedora packages. While I think it is reasonable to expect Fedora to ship those scripts too (I recommend opening a feature request bug in Fedora bugzilla ( http://redhat.bugzilla.com ) against spark package; they are not shipped currently. You should be able to download the scripts manually and make them work with Fedora spark installation. However, it seems to me that Spark is mainly an API, and you are mainly expected to write programs using it.

Also, there is no spark binary at all. As can be seen in the link given by @Florian, there are a number of convenient scrips like spark-shell (which Fedora doesn't include), but no spark binary.

edit flag offensive delete link more
1

answered 2015-09-15 12:38:02 -0600

florian gravatar image

Just read this: http://spark.apache.org/docs/latest/

There is a section that describes how to Running the Examples and Shell, and much more.

edit flag offensive delete link more
1

answered 2015-09-15 10:18:15 -0600

Psyringe gravatar image

Go to /usr/bin/ using cd /usr/bin/.
Then use ls | grep spark command to see if there's any executable containing "spark".

edit flag offensive delete link more

Comments

Or use locate spark.

florian gravatar imageflorian ( 2015-09-15 12:39:09 -0600 )edit
1

whereis spark should also work.

sideburns gravatar imagesideburns ( 2015-09-15 14:04:44 -0600 )edit

Question Tools

1 follower

Stats

Asked: 2015-09-15 08:51:29 -0600

Seen: 2,583 times

Last updated: Sep 17 '15