Added readme, added tests, and fixed up some more comments.

Author: Skarlso

README.rst

@@ -1,5 +1,5 @@
 .. raw:: html
-    
+
     <img src="https://cdn.rawgit.com/michelvocks/ef3894f63c3bb004bca1a2fd5f7eb644/raw/c36d614db8afe229b466b38de1636a82ad809f64/gaia-logo-text.png" width="650px">
 
 |build-status| |go-report| |go-doc| |apache2| |chat| |codecov|
@@ -13,8 +13,8 @@ Motivation
 
 .. begin-motivation
 
-*Automation Engineer*, *DevOps*, *SRE*, *Cloud Engineer*, 
-*Platform Engineer* - they all have one in common: 
+*Automation Engineer*, *DevOps*, *SRE*, *Cloud Engineer*,
+*Platform Engineer* - they all have one in common:
 The majority of tech people are not motivated to take up this work and they are hard to recruit.
 
 One of the main reasons for this is the abstraction and poor execution of many automation tools. They come with their own configuration (`YAML`_ syntax) specification or limit the user to one specific programming language. Testing is nearly impossible because most automation tools lack the ability to mock services and subsystems. Even tiny things, for example parsing a JSON file, are sometimes really painful because external, outdated libraries were used and not included in the standard framework.
@@ -26,11 +26,11 @@ How does it work?
 
 .. begin-architecture
 
-Gaia is based on `HashiCorp's go-plugin`_. It's a plugin system that uses `gRPC`_ to communicate over `HTTP/2`_. HashiCorp developed this tool initially for `Packer`_ but it's now heavily used by `Terraform`_, `Nomad`_, and `Vault`_ too. 
+Gaia is based on `HashiCorp's go-plugin`_. It's a plugin system that uses `gRPC`_ to communicate over `HTTP/2`_. HashiCorp developed this tool initially for `Packer`_ but it's now heavily used by `Terraform`_, `Nomad`_, and `Vault`_ too.
 
 Plugins, which we named pipelines, are applications which can be written in any programming language as long as `gRPC`_ is supported. All functions, which we call Jobs, are exposed to Gaia and can form up a dependency graph which describes the order of execution.
 
-Pipelines can be compiled locally or simply over the build system. Gaia clones the git repository and automatically builds the included pipeline. If a change (`git push`_) happened, Gaia will automatically rebuild the pipeline for you.  
+Pipelines can be compiled locally or simply over the build system. Gaia clones the git repository and automatically builds the included pipeline. If a change (`git push`_) happened, Gaia will automatically rebuild the pipeline for you.
 
 After a pipeline has been started, all log output are returned back to Gaia and displayed in a detailed overview with their final result status.
 
@@ -107,7 +107,7 @@ Here is an example:
         jobs := sdk.Jobs{
             sdk.Job{
                 Handler:     DoSomethingAwesome,
-	        Title:       "DoSomethingAwesome", 
+	        Title:       "DoSomethingAwesome",
 		Description: "This job does something awesome.",
 
                 // Increase the priority if this job should be executed later than other jobs.
@@ -123,7 +123,7 @@ Here is an example:
 
 Like you can see, pipelines are defined by jobs. Usually, a function represents a job. You can define as many jobs in your pipeline as you want.
 
-At the end, we define a jobs array that populates all jobs to gaia. We also add some information like a title, a description and the priority. 
+At the end, we define a jobs array that populates all jobs to gaia. We also add some information like a title, a description and the priority.
 
 The priority is really important and should always be used. If, for example, job A has a higher priority (decimal number) as job B, job A will be executed **after** job B. Priority defines therefore the order of execution. If two or more jobs have the same priority, those will be executed simultanously. You can compare it with the `Unix nice level`_.
 
@@ -132,6 +132,11 @@ Gaia will compile it and add it to it's store for later execution.
 
 Please find a bit more sophisticated example in our `go-example repo`_.
 
+Security
+--------
+
+See the Documentation located here: |security-docs|.
+
 Documentation and more
 ======================
 
@@ -144,11 +149,11 @@ What problem solves **Gaia**?
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 Literally every tool which were designed for automation, continuous integration (CI), and continuous deployment (CD) like Spinnaker, Jenkins, Gitlab CI/CD, TravisCI, CircleCI, Codeship, Bamboo and many more, introduced their own configuration format. Some of them don't even support *configuration/automation as code*. This works well for simple tasks like running a ``go install`` or ``mvn clean install`` but in the real world there is more to do.
 
-Gaia is the first platform which does not limit the user and provides full support for almost all common programming languages without losing the features offered by todays CI/CD tools.    
+Gaia is the first platform which does not limit the user and provides full support for almost all common programming languages without losing the features offered by todays CI/CD tools.
 
 What is a **pipeline**?
 ~~~~~~~~~~~~~~~~~~~~~~~
-A pipeline is a real application with at least one function (we call it Job). Every programming language can be used as long as gRPC is supported. We offer SDKs (currently only Go but others are already in development) to support the development. 
+A pipeline is a real application with at least one function (we call it Job). Every programming language can be used as long as gRPC is supported. We offer SDKs (currently only Go but others are already in development) to support the development.
 
 What is a **job**?
 ~~~~~~~~~~~~~~~~~~
@@ -169,12 +174,12 @@ Gaia is currently in alpha version available. We extremely recommend to not use
 
 One of the main issues currently is the lack of unit- and integration tests. This is on our to-do list and we are working on this topic with high priority.
 
-It is planned that other programming languages should be supported in the next few month. It is up to the community which languages will be supported next. 
+It is planned that other programming languages should be supported in the next few month. It is up to the community which languages will be supported next.
 
 Contributing
 ============
 
-Gaia can only evolve and become a great product with the help of contributors. If you like to contribute, please have a look at our `issues section`_. We do our best to mark issues for new contributors with the label *good first issue*. 
+Gaia can only evolve and become a great product with the help of contributors. If you like to contribute, please have a look at our `issues section`_. We do our best to mark issues for new contributors with the label *good first issue*.
 
 If you think you found a good first issue, please consider this list as a short guide:
 
@@ -225,7 +230,7 @@ If you have any questions feel free to contact us on `gitter`_.
     :alt: Apache licensed
     :target: https://github.com/gaia-pipeline/gaia/blob/master/LICENSE
 
-.. |chat| image:: https://badges.gitter.im/Join%20Chat.svg   
+.. |chat| image:: https://badges.gitter.im/Join%20Chat.svg
     :alt: Gitter
     :target: https://gitter.im/gaia-pipeline
 
@@ -247,15 +252,18 @@ If you have any questions feel free to contact us on `gitter`_.
 .. |sh-create-pipeline-history| image:: https://cdn.rawgit.com/michelvocks/6868118d0da06a422e69e453497eb30d/raw/142a2969c4d27d4135ef8f96213bb166009fda1e/create_pipeline_history.png
     :alt: gaia create pipeline history screenshot
     :width: 650px
-    
+
 .. |sh-pipeline-detailed| image:: https://cdn.rawgit.com/michelvocks/6868118d0da06a422e69e453497eb30d/raw/51b4d6cbc3d86b1fe9531250db5456595423d9ec/pipeline_detailed.png
     :alt: gaia pipeline detailed screenshot
     :width: 650px
-    
+
 .. |sh-pipeline-logs| image:: https://cdn.rawgit.com/michelvocks/6868118d0da06a422e69e453497eb30d/raw/51b4d6cbc3d86b1fe9531250db5456595423d9ec/pipeline_logs.png
     :alt: gaia pipeline logs screenshot
     :width: 650px
 
 .. |sh-settings| image:: https://cdn.rawgit.com/michelvocks/6868118d0da06a422e69e453497eb30d/raw/142a2969c4d27d4135ef8f96213bb166009fda1e/settings.png
     :alt: gaia settings screenshot
     :width: 650px
+.. |security-docs|
+    :alt: Security in Gaia
+    :target: https://github.com/gaia-pipeline/gaia/blob/master/security/README.md

handlers/vault.go

@@ -55,7 +55,7 @@ func SetSecret(c echo.Context) error {
 	if err != nil {
 		return c.String(http.StatusInternalServerError, err.Error())
 	}
-	return c.String(http.StatusOK, "secret successfully set")
+	return c.String(http.StatusCreated, "secret successfully set")
 }
 
 // ListSecrets retrieves all secrets from the vault.

handlers/vault_test.go

@@ -0,0 +1,125 @@
+package handlers
+
+import (
+	"bytes"
+	"encoding/json"
+	"io/ioutil"
+	"net/http"
+	"net/http/httptest"
+	"os"
+	"testing"
+
+	"github.com/gaia-pipeline/gaia"
+	"github.com/gaia-pipeline/gaia/store"
+	hclog "github.com/hashicorp/go-hclog"
+	"github.com/labstack/echo"
+)
+
+func TestVaultWorkflowAddListDelete(t *testing.T) {
+	dataDir, err := ioutil.TempDir("", "temp")
+	if err != nil {
+		t.Fatalf("error creating data dir %v", err.Error())
+	}
+
+	defer func() {
+		gaia.Cfg = nil
+		os.RemoveAll(dataDir)
+	}()
+
+	gaia.Cfg = &gaia.Config{
+		Logger:    hclog.NewNullLogger(),
+		DataPath:  dataDir,
+		CAPath:    dataDir,
+		VaultPath: dataDir,
+	}
+
+	dataStore := store.NewStore()
+	err = dataStore.Init()
+	if err != nil {
+		t.Fatalf("cannot initialize store: %v", err.Error())
+	}
+
+	e := echo.New()
+	InitHandlers(e, dataStore, nil)
+	t.Run("can add secret", func(t *testing.T) {
+		body := map[string]string{
+			"Key":   "Key",
+			"Value": "Value",
+		}
+		bodyBytes, _ := json.Marshal(body)
+		req := httptest.NewRequest(echo.POST, "/api/"+apiVersion+"/secret", bytes.NewBuffer(bodyBytes))
+		req.Header.Set("Content-Type", "application/json")
+		rec := httptest.NewRecorder()
+		c := e.NewContext(req, rec)
+
+		SetSecret(c)
+
+		if rec.Code != http.StatusCreated {
+			t.Fatalf("expected response code %v got %v", http.StatusCreated, rec.Code)
+		}
+	})
+
+	t.Run("can update secret", func(t *testing.T) {
+		body := map[string]string{
+			"Key":   "Key",
+			"Value": "Value",
+		}
+		bodyBytes, _ := json.Marshal(body)
+		req := httptest.NewRequest(echo.PUT, "/api/"+apiVersion+"/secret", bytes.NewBuffer(bodyBytes))
+		req.Header.Set("Content-Type", "application/json")
+		rec := httptest.NewRecorder()
+		c := e.NewContext(req, rec)
+
+		SetSecret(c)
+
+		if rec.Code != http.StatusCreated {
+			t.Fatalf("expected response code %v got %v", http.StatusCreated, rec.Code)
+		}
+	})
+
+	t.Run("can list secrets", func(t *testing.T) {
+		req := httptest.NewRequest(echo.GET, "/api/"+apiVersion+"/secrets", nil)
+		req.Header.Set("Content-Type", "application/json")
+		rec := httptest.NewRecorder()
+		c := e.NewContext(req, rec)
+
+		ListSecrets(c)
+
+		if rec.Code != http.StatusOK {
+			t.Fatalf("expected response code %v got %v", http.StatusCreated, rec.Code)
+		}
+		body, _ := ioutil.ReadAll(rec.Body)
+		expectedBody := "[{\"key\":\"Key\",\"value\":\"**********\"}]"
+		if string(body) != expectedBody {
+			t.Fatalf("response body did not equal expected body: expected: %s, actual: %s", expectedBody, string(body))
+		}
+	})
+
+	t.Run("can delete secrets", func(t *testing.T) {
+		req := httptest.NewRequest(echo.DELETE, "/api/"+apiVersion+"/secret/:key", nil)
+		req.Header.Set("Content-Type", "application/json")
+		rec := httptest.NewRecorder()
+		c := e.NewContext(req, rec)
+		c.SetParamNames("key")
+		c.SetParamValues("Key")
+
+		RemoveSecret(c)
+
+		if rec.Code != http.StatusOK {
+			t.Fatalf("expected response code %v got %v", http.StatusCreated, rec.Code)
+		}
+	})
+
+	t.Run("can delete fails if no secret is provided", func(t *testing.T) {
+		req := httptest.NewRequest(echo.DELETE, "/api/"+apiVersion+"/secret/:key", nil)
+		req.Header.Set("Content-Type", "application/json")
+		rec := httptest.NewRecorder()
+		c := e.NewContext(req, rec)
+
+		RemoveSecret(c)
+
+		if rec.Code != http.StatusBadRequest {
+			t.Fatalf("expected response code %v got %v", http.StatusCreated, rec.Code)
+		}
+	})
+}

security/README.md

@@ -0,0 +1,32 @@
+# Security
+
+## Certificates
+
+Gaia, when first started will create a signed certificate in a location
+defined by the user under `gaia.Cfg.CAPath` which can be set by the runtime flag
+`-capath=/etc/gaia/cert` for example. It is recommended that the certificate
+is kept separate from the main Gaia work folder and in a secure location.
+
+This certificate is used in two places. First, in the communication between the
+admin portal and the back-end. Second, by the Vault.
+
+## The Vault
+
+The Vault is a secure storage for secret values like, password, tokens and other
+things that the user would like to pass securly into a Pipeline. The Vault is
+encrypted using AES cipher technology where the key is derived from the above
+certificate and the IV is included in the encrypted content.
+
+The Vault file's location can be configured through the runtime variable called
+`VaultPath`. For maximum security it is recommended that this file is kept on an
+encrypted, mounted drive. In case there is a breach the drive can be quickly removed
+and the file deleted, thus rotating all of the secrets at once, under Gaia.
+
+To create an encrypted MacOSX image follow this guide: [Encrypted Secure Disk Image on Mac](https://www.howtogeek.com/183826/how-to-create-an-encrypted-file-container-disk-image-on-a-mac/).
+
+To create an encrypted disk on Linux follow this guide: [Encrypted Disk Image on Linux](http://freesoftwaremagazine.com/articles/create_encrypted_disk_image_gnulinux/).
+
+The admin will never see the secure values, not when editing, not when adding and not
+when looking at the list of secrets. Only the Key names are displayed at all times.
+
+It's possible to Add, Delete, Update and List secrets in the system.