feat(notification): add new notification type: task_validation (#233)

* feat(notification): add new notification type: task_validation

New notification type has been introduced: task_validation. It will be
fired every time a task is created and the task will require to be
validated by a human operator, aka, task will not be auto-runned.
This will concerns tasks that require resolver_inputs, task not declared
as auto_runnable, or task created by a requester which is not an
allowed_resolver.

* feat(notification): add task_step_update notification type

task_step_update notification will be fired every time a step is
changing its state.

This commit also modifies the behaviour of task_state_update to prevent
notifying when the state is not changing, but only the steps count is
evolving.

Signed-off-by: Romain Beuque <[email protected]>

Author: rbeuque74

Diff for: .golangci.yml

@@ -0,0 +1,62 @@
+# This file contains all available configuration options
+# with their default values.
+
+# options for analysis running
+run:
+  # include test files or not, default is true
+  tests: false
+  # timeout for running linters, default is 1m
+  timeout: 4m
+
+linters:
+  enable:
+    - govet
+    - megacheck
+    - staticcheck
+    - deadcode
+    - structcheck
+    - typecheck
+    - gosimple
+    - varcheck
+    - ineffassign
+    - bodyclose
+    - asciicheck
+    - exportloopref
+    - goconst
+    - misspell
+    - gofmt
+    - goimports
+    - predeclared
+    - gosec
+    - golint
+    - nolintlint
+    - unconvert
+    - errcheck
+    - scopelint
+    - unparam
+    - unused
+
+  disable:
+    - maligned
+
+    # cosmetic
+    - lll
+    - gochecknoglobals
+    - gochecknoinits
+
+linters-settings:
+  maligned:
+    # print struct with more effective memory layout or not, false by default
+    suggest-new: true
+
+issues:
+  # only reports issues that have been introduced in commits ahead of origin/master
+  new-from-rev: origin/master
+  # exclude some irrelevant issues for us
+  exclude:
+    - '^G401: Use of weak cryptographic primitive'
+    - '^G402: TLS InsecureSkipVerify'
+    - '^G404: Use of weak random number generator'
+    - '^G501: Blocklisted import crypto/md5: weak cryptographic primitive'
+    - '^G505: Blocklisted import crypto/sha1: weak cryptographic primitive'
+    - '^G601: Implicit memory aliasing in for loop'

Diff for: CHANGELOG.md

@@ -6,6 +6,11 @@ We will list in this document any breaking changes between versions, that requir
 
 ## Breaking changes
 
+### v1.13.0
+#### Notifications
+- Added a new `notification_type` : `task_validation` that fires every time a new task need a human validation. To integrate different `notification_strategy`, all `notification_strategy` are scopped to the `notification_type`. Then, `default_notification_strategy` is now an object containing the `notification_type` as key, and the `strategy` as value ; and `template_notification_strategies` is now an object containing the `notification_type` as key, and the strategies array as value.
+- Changed the configuration key `task_state_action` (in the `notify_action` section) to the new value `task_state_update` (to match the `notification_type` value).
+
 ### v1.10.0
 #### SQL
 - `005_resolution_creation_timestamp.sql` migration file should be applied while upgrading. It adds a column `created` in the `resolution` table to keep track of the resolution creation timestamp.

Diff for: README.md

@@ -156,10 +156,13 @@ Extending this basic authentication mechanism is possible by developing an "init
 Every task state change can be notified to a notification backend.
 µTask implements three differents notification backends: Slack, [TaT](https://github.com/ovh/tat), and generic webhooks.
 
-Default payload that will be sent for generic webhooks is :
+Default payload that will be sent for generic webhooks are:
+
+__task_state_update notifications:__
 ```json
 {
     "message": "string",
+    "notification_type": "task_state_update",
     "task_id": "public_task_uuid",
     "title": "task title string",
     "state": "current task state",
@@ -173,6 +176,40 @@ Default payload that will be sent for generic webhooks is :
 }
 ```
 
+__task_step_update notifications:__
+```json
+{
+    "message": "string",
+    "notification_type": "task_step_update",
+    "task_id": "public_task_uuid",
+    "title": "task title string",
+    "state": "current task state",
+    "template": "template_name",
+    "step_name": "string",
+    "step_state": "string",
+    "requester": "string",
+    "resolver": "string",
+    "steps": "14/20",
+    "resolution_id": "public_resolution_uuid",
+    "tags": "{\"tag1\":\"value1\"}"
+}
+```
+
+__task_validation notifications:__
+```json
+{
+    "message": "string",
+    "notification_type": "task_validation",
+    "task_id": "public_task_uuid",
+    "title": "task title string",
+    "state": "TODO",
+    "template": "template_name",
+    "requester": "optional",
+    "potential_resolvers": "user1,user2,admin",
+    "tags": "{\"tag1\":\"value1\"}"
+}
+```
+
 Notification backends can be configured in the global µTask configuration, as described [here](./config/README.md#utask-cfg).
 
 ## Authoring Task Templates <a name="templates"></a>
@@ -211,7 +248,7 @@ The following templating functions are available:
 | **`Golang`**       | Builtin functions from Golang text template                                                                                                                                                                                                                                                                                                                     | [Doc](https://golang.org/pkg/text/template/#hdr-Actions) |
 | **`Sprig`**        | Extended set of functions from the Sprig project                                                                                                                                                                                                                                                                                                                | [Doc](https://masterminds.github.io/sprig/)              |
 | **`field`**        | Equivalent to the dot notation, for entries with forbidden characters                                                                                                                                                                                                                                                                                           | ``{{field `config` `foo.bar`}}``                         |
-| **`fieldFrom`**    | Equivalent to the dot notation, for entries with forbidden characters. It takes the previous template expression as source for the templating values. Example: ```{{ `{"foo.foo":"bar"}` | fromJson | fieldFrom `foo.foo` }}```                                                                                                                                   | ```{{expr | fieldFrom `config` `foo.bar`}}```              |
+| **`fieldFrom`**    | Equivalent to the dot notation, for entries with forbidden characters. It takes the previous template expression as source for the templating values. Example: ```{{ `{"foo.foo":"bar"}` | fromJson | fieldFrom `foo.foo` }}```                                                                                                                                 | ```{{expr | fieldFrom `config` `foo.bar`}}```            |
 | **`eval`**         | Evaluates the value of a template variable                                                                                                                                                                                                                                                                                                                      | ``{{eval `var1`}}``                                      |
 | **`evalCache`**    | Evaluates the value of a template variable, and cache for future usage (to avoid further computation)                                                                                                                                                                                                                                                           | ``{{evalCache `var1`}}``                                 |
 | **`fromJson`**     | Decodes a JSON document into a structure. If the input cannot be decoded as JSON, the function will return an empty string                                                                                                                                                                                                                                      | ``{{fromJson `{"a":"b"}`}}``                             |

Diff for: config/README.md

@@ -61,18 +61,27 @@ postgres://user:pass@db/utask?sslmode=disable
                 "url": "http://localhost:9999/tat",
                 "topic": "utask.notifications"
             },
-            "default_notification_strategy":"silent",
-            "template_notification_strategies": [{
-                "templates": ["hello-world", "hello-world-2"],
-                "notification_strategy": "always"
-            }]
+            "default_notification_strategy": {
+                "task_state_update": "silent",
+                "task_validation": "always"
+            },
+            "template_notification_strategies": {
+                "task_state_update": [
+                    {
+                        "templates": ["hello-world", "hello-world-2"],
+                        "notification_strategy": "always"
+                    }
+                ]
+            }
         },
         "slack-webhook": {
             "type": "slack",
             "config": {
                 "webhook_url": "https://hooks.slack.com/services/T00000000/B00000000/XXXXXXXXXXXXXXXXXXXXXXXX"
             },
-            "default_notification_strategy":"failure_only"
+            "default_notification_strategy": {
+                "task_state_update": "failure_only"
+            },
         },
         "webhook-example.org": {
             "type": "webhook",
@@ -88,11 +97,20 @@ postgres://user:pass@db/utask?sslmode=disable
     },
     // notify_actions specifies a notification config for existing events in µTask
     // existing events are:
-    // - task_state_action (fired every time a task's state changes)
+    // - task_state_update: fired every time a task's state changes
+    // - task_validation: fired every time a new task is created and requires a human validation
+    // - task_step_update: fired every time a step's state changes
     "notify_actions": {
-        "task_state_action": {
+        "task_state_update": {
             "disabled": false, // set to true to avoid sending out notification
             "notify_backends": ["tat-internal", "slack-webhook"] // choose among the named configs in notify_config, leave empty to broadcast on any notification backend
+        },
+        "task_validation": {
+            "disabled": false, // set to true to avoid sending out notification
+            "notify_backends": ["slack-webhook"] // choose among the named configs in notify_config, leave empty to broadcast on any notification backend
+        },
+        "task_step_update": {
+            "disabled": true // set to true to avoid sending out notification
         }
     },
     // database_config holds configuration to fine-tune DB connection

Diff for: engine/engine.go

@@ -406,6 +406,11 @@ func resolve(dbp zesty.DBProvider, res *resolution.Resolution, t *task.Task, sm
 				}
 			}
 
+			var oldState string
+			if oldStep, ok := res.Steps[s.Name]; ok {
+				oldState = oldStep.State
+			}
+
 			// "commit" step back into resolution
 			res.SetStep(s.Name, s)
 			// consolidate its result into live values
@@ -429,6 +434,10 @@ func resolve(dbp zesty.DBProvider, res *resolution.Resolution, t *task.Task, sm
 
 			debugLogger.Debugf("Engine: resolve() %s loop, step %s (#%d) result: %s", res.PublicID, s.Name, s.TryCount, s.State)
 
+			if newStep, ok := res.Steps[s.Name]; ok && newStep.State != oldState {
+				t.NotifyStepState(s.Name, newStep.State)
+			}
+
 			// update done step count
 			// ignore foreach iterations for global done count
 			if s.IsFinal() && !s.IsChild() {

Diff for: engine/step/executor/executor.go

@@ -2,6 +2,7 @@ package executor
 
 import (
 	"encoding/json"
+
 	"github.com/ghodss/yaml"
 )
 

Diff for: models/resolution/resolution.go

@@ -489,7 +489,18 @@ func (r *Resolution) setSteps(st map[string]*step.Step) {
 }
 
 func (r *Resolution) SetStepState(stepName, state string) {
+	oldState := r.Steps[stepName].State
 	r.Steps[stepName].State = state
+
+	// notify about step state modification
+	if oldState != state {
+		if dbp, err := zesty.NewDBProvider(utask.DBName); err == nil {
+			if t, err := task.LoadFromID(dbp, r.TaskID); err == nil {
+				t.NotifyStepState(stepName, state)
+			}
+		}
+	}
+
 	if r.Values == nil {
 		return
 	}

Diff for: models/task/task.go

@@ -533,8 +533,16 @@ func applyTemplateToMap(m map[string]interface{}, values *values.Values) error {
 
 // SetState updates the task's state
 func (t *Task) SetState(s string) {
+	var notify bool
+	if t.State != s {
+		notify = true
+	}
+
 	t.State = s
-	t.notifyState(nil)
+
+	if notify {
+		t.notifyState(nil)
+	}
 }
 
 func (t *Task) SetTags(tags map[string]string, values *values.Values) error {
@@ -620,6 +628,58 @@ func (t *Task) notifyState(potentialResolvers []string) {
 
 	notify.Send(
 		notify.WrapTaskStateUpdate(tsu),
-		notify.ListActions().TaskStateAction,
+		notify.ListActions().TaskStateUpdateAction,
+	)
+}
+
+func (t *Task) NotifyValidationRequired(tt *tasktemplate.TaskTemplate) {
+	notificationAllowedResolverUsernames := []string{}
+	if tt != nil {
+		notificationAllowedResolverUsernames = append(notificationAllowedResolverUsernames, tt.AllowedResolverUsernames...)
+	}
+	if tt.AllowAllResolverUsernames {
+		notificationAllowedResolverUsernames = append(notificationAllowedResolverUsernames, t.RequesterUsername)
+	}
+
+	tv := &notify.TaskValidation{
+		Title:              t.Title,
+		PublicID:           t.PublicID,
+		State:              t.State,
+		TemplateName:       t.TemplateName,
+		PotentialResolvers: notificationAllowedResolverUsernames,
+		RequesterUsername:  t.RequesterUsername,
+		Tags:               t.Tags,
+	}
+
+	notify.Send(
+		notify.WrapTaskValidation(tv),
+		notify.ListActions().TaskValidationAction,
+	)
+}
+
+func (t *Task) NotifyStepState(stepName, stepState string) {
+	if t.Resolution == nil || t.ResolverUsername == nil {
+		// matches mainly the period where the task is getting created and all steps states are assigned to TODO
+		return
+	}
+
+	tsu := &notify.TaskStepUpdate{
+		Title:              t.Title,
+		PublicID:           t.PublicID,
+		State:              t.State,
+		TemplateName:       t.TemplateName,
+		RequesterUsername:  t.RequesterUsername,
+		ResolverUsername:   *t.ResolverUsername,
+		StepsDone:          t.StepsDone,
+		StepsTotal:         t.StepsTotal,
+		Tags:               t.Tags,
+		StepName:           stepName,
+		StepState:          stepState,
+		ResolutionPublicID: *t.Resolution,
+	}
+
+	notify.Send(
+		notify.WrapTaskStepUpdate(tsu),
+		notify.ListActions().TaskStepUpdateAction,
 	)
 }