Skip to content

Commit 17d9442

Browse files
finished ci workflow + added build badge on readme
1 parent 852d500 commit 17d9442

File tree

4 files changed

+14
-23
lines changed

4 files changed

+14
-23
lines changed

Diff for: .github/workflows/ci.yml

+2-4
Original file line numberDiff line numberDiff line change
@@ -1,11 +1,9 @@
11
name: Docker Hub Build
22

33
on:
4-
push:
5-
branches: [ feature/github-ci ]
64

7-
# schedule:
8-
# - cron: '0 0/6 * * *'
5+
schedule:
6+
- cron: '0 0/6 * * *'
97

108
jobs:
119

Diff for: .gitignore

-7
Original file line numberDiff line numberDiff line change
@@ -2,13 +2,6 @@
22
# Created by https://www.toptal.com/developers/gitignore/api/spark,jupyternotebooks,pycharm,pycharm+iml,pycharm+all
33
# Edit at https://www.toptal.com/developers/gitignore?templates=spark,jupyternotebooks,pycharm,pycharm+iml,pycharm+all
44

5-
### App defined ###
6-
7-
docker-compose.yml
8-
9-
build/build.yml
10-
build/docker-compose.yml
11-
125
### JupyterNotebooks ###
136
# gitignore template for Jupyter Notebooks
147
# website: http://jupyter.org/

Diff for: CONTRIBUTING.md

+8-8
Original file line numberDiff line numberDiff line change
@@ -14,11 +14,11 @@ and parallel computing through our projects. Please follow the template bellow t
1414

1515
### Contributions ideas
1616

17-
- Microsoft Windows build script;
18-
- DockerHub CI/CD integration;
19-
- Spark submit support;
20-
- JupyterLab Scala kernel;
21-
- Jupyter notebook with Apache Spark Scala API examples;
22-
- JupyterLab R kernel;
23-
- Jupyter notebook with Apache Spark R API examples;
24-
- Test coverage.
17+
- [] Microsoft Windows build script;
18+
- [x] DockerHub CI/CD integration;
19+
- [] Spark submit support;
20+
- [] JupyterLab Scala kernel;
21+
- [] Jupyter notebook with Apache Spark Scala API examples;
22+
- [] JupyterLab R kernel;
23+
- [] Jupyter notebook with Apache Spark R API examples;
24+
- [] Test coverage.

Diff for: README.md

+4-4
Original file line numberDiff line numberDiff line change
@@ -1,17 +1,17 @@
11
# Apache Spark Standalone Cluster on Docker
22
> Build your own Apache Spark Standalone cluster with a JupyterLab interface on Docker :zap:
33
4-
This project gives you an out-of-the-box **Apache Spark** cluster with a **JupyterLab** interface and a simulated **Apache Hadoop Distributed File System**, all built on top of **Docker**. Learn Apache Spark through its Python API, **PySpark**, by running the [Jupyter notebooks](build/workspace/pyspark.ipynb) with examples on how to read, process and write data.
4+
This project gives you an out-of-the-box **Apache Spark** cluster in standalone mode with a **JupyterLab** interface and a simulated **Apache Hadoop Distributed File System**, all built on top of **Docker**. Learn Apache Spark through its Python API, **PySpark**, by running the [Jupyter notebooks](build/workspace/pyspark.ipynb) with examples on how to read, process and write data.
5+
6+
> The project just got its [own article](https://towardsdatascience.com/apache-spark-cluster-on-docker-ft-a-juyterlab-interface-418383c95445) at Towards Data Science Medium blog! :sparkles:
57
68
<p align="center"><img src="docs/image/cluster-architecture.png"></p>
79

10+
![Docker Hub Build](https://github.com/andre-marcos-perez/spark-standalone-cluster-on-docker/workflows/Docker%20Hub%20Build/badge.svg?branch=master)
811
![jupyterlab-latest-version](https://img.shields.io/docker/v/andreper/jupyterlab/2.1.4-spark-3.0.0?color=brightgreen&label=jupyterlab-latest)
912
![spark-latest-version](https://img.shields.io/docker/v/andreper/spark-master/3.0.0-hadoop-2.7?color=yellow&label=spark-latest)
1013
![docker-version](https://img.shields.io/badge/docker-v1.13.0%2B-blue)
1114
![docker-compose-version](https://img.shields.io/badge/docker--compose-v3.0%2B-blue)
12-
![repository-license](https://img.shields.io/github/license/andre-marcos-perez/spark-cluster-on-docker?color=orange)
13-
14-
> The project just got its [own article](https://towardsdatascience.com/apache-spark-cluster-on-docker-ft-a-juyterlab-interface-418383c95445) at Towards Data Science Medium blog! :sparkles:
1515

1616
## Contents
1717

0 commit comments

Comments
 (0)