e2e: updating PR pipeline

Author: churik

doc/pipeline_process.md

@@ -2,7 +2,7 @@
 
 [Pipeline for QA](https://github.com/status-im/status-mobile/projects/7) is a project board for developers and testers used to track the status of a pull request, get reviews and manual testing, and run autotests.
 
-The generally accepted recommendations for its use are described below:
+The generally accepted requirements for its use are described below:
 
 ## Opening a PR
 - Once a PR is created, it moves to the ```REVIEW``` column where a review will be requested automatically.
@@ -13,35 +13,70 @@ The generally accepted recommendations for its use are described below:
 ### What if the work is still in progress?
 
 - If PR work is not finished yet, please mark it as a draft or add [WIP] to the title and keep it in the `CONTRIBUTOR` column until it's ready to be reviewed/tested.
+- Alternatively, you can open a `draft`.
 
+### When is a PR considered ready to move forward?
 
-### When is a PR considered to be Ready for testing by QA team? 


-Ready for testing PR should meet the following criteria: 
+Ready for testing, a PR should meet the following criteria:
 
-1. Reviewed and has at least 1 approval
-2. Rebased to `develop` branch (both `status-mobile` and `status-go` if needed, depending on what part has changes) 
-3. All possible conflicts have been resolved 
-4. Has the label: `request-manual-qa`
+1. Reviewed and has at least 1 approval.
+2. Rebased to the `develop` branch (both `status-mobile` and `status-go` if needed, depending on what part has changes).
+   - **NOTE:** QAs can use the `Update with rebase` button from GitHub if:
+        - The PR has no `status-go` changes.
+        - The PR has no conflict with the develop branch.
+3. All possible conflicts have been resolved.
+4. Has the label: `request-manual-qa` or `skip-manual-qa`.
+5. PRs **MUST** identify what area is affected and should have a description.
 
+**NOTE:** Make sure that QAs are OK with that
+
+### Adding `skip-manual-qa`
+
+- Please ask another team member before adding the `skip-manual-qa` label (PR/Status community/DMs) so that there's a second opinion.
+- The PR MUST have a proper reasoning why manual QA is skipped.
+- The PR MUST include the steps of testing that has been done by the developer prior to moving it forward.
 **From the perspective of a developer it means that once work on PR is finished:**
 
 1. It should be rebased to the latest `develop`. If there are conflicts - they should be resolved if possible.
 2. If the PR was in the `Contributor` column - it should be moved to `Review` column.
 3. Wait for the review.
 4. Make sure that after review and before requesting manual QA your PR is rebased to current develop.
-5. Once the PR has been approved by reviewer(s) - label `request-manual-qa` should be applied to the PR
-6. Move PR to the E2E column when it is ready for testing (**mandatory for all PRs**). That will also trigger e2e tests run. QAs are monitoring PRs from E2E column and take it into test. 
+5. The PR **MUST** be moved to the E2E column when it is ready for testing (**mandatory for all PRs**).
+That will also trigger e2e tests run. QAs are monitoring PRs from E2E column and take it into test. 
+
+6. After that - PR will be taken into manual testing by the QA team.
+
+### E2E tests and analyzing the results
+
+This step cannot be skipped. So, at least one comment from the `status-im-auto` bot with results is a prerequisite for moving forward.
+Information on how to analyze tests can be found [here](https://github.com/status-im/status-mobile/blob/develop/doc/how-to-launch-e2e.md).
+Tests might be flaky, as they depend on infrastructure - SauceLabs and Waku.
+
+If there are `Failed tests` and you are not sure about the reason, you can always ping the mobile QAs for help (preferably in PRs by `@status-im/mobile-qa`).
 
-After that - PR will be taken into manual testing by the QA team.
+**For now, it is the only reliable source of regression in the app because, due to time constraints, it is not possible to test thoroughly each PR.
+Please, respect this rule.**
 
 ## Testing PR
 
 ### Manual testing
-- If you think PR needs and is ready for manual testing, please add the ```request-manual-qa``` label. 
+
+#### Prerequisites for manual testing
+1. PR has a description of the change and details the area of the application affected by the change.
+2. PR has automation results from the previous E2E test run step.
+3. PR includes a demo, test steps, and any possible regression(s)
+4. If appropriate, PR details what elements of a feature are out of scope.
+
+**If one of the prerequisites is missing, the PR will be moved to the contributor folder with a corresponding comment.**
+
+- If you think a PR needs and is ready for manual testing, please add the `request-manual-qa` label.
 - QA engineer picks up one of PRs with the ```request-manual-qa``` label, drags the item to the ```IN TESTING``` column and assigns it to themselves.
 - During testing, QA will add comments describing the issues found, and also review automation tests results.
 Usually found issues are numbered as "Issue 1, Issue 2", etc.
-When the first round of testing is completed and all issues for this stage are found, the QA can add the ```Tested - Issues``` label and drag the card to the ```CONTRIBUTOR``` column. These two actions are optional.
+When the first round of testing is completed and all issues for this stage are found, the QA can add the ```Tested - Issues``` label and drag the card to the ```CONTRIBUTOR``` column. These two actions are optional. 
+
+**IMPORTANT NOTE:** when the issues are fixed, developer **MUST** notify the QA that it is ready to be re-tested again by mention them in the PR.
+
 - When manual testing of the PR is fully completed and all the issues are fixed, the QA adds the ```Tested - OK``` label and drags the card to the ```Design review``` column (the cases when design review is necessary are described below).
 - If design review is not required, the QA drags the PR to the ```MERGE``` column. After that the developer merges PR into develop.
 - If design review has been done, the designer (```@Francesca-G```) drags the PR to the ```MERGE``` column.
@@ -68,7 +103,7 @@ There are three possible scenarios when the design review is completed:
 ---        
 
 #### Why my PR is in `Contributor` column?
-PR can be moved to this column by the ```status-github-bot``` or by QA engineer with label `Tested-issues`.
+PR can be moved to this column by the ```status-github-bot``` or by QA engineer with label `Tested-issues` or if one of the requirements for manual QA was not met.
 In the first case most often this happens due to conflicting files in PR.
 In the second case - after fixing of all found issues, the developer should ping the QA in the PR comments for retesting.
 
@@ -86,7 +121,6 @@ In the second case - after fixing of all found issues, the developer should ping
 6. In case of manual testing - the label ```Tested - OK``` from QA
 7. In case of design review - the approval from the designer
 
-
 You can merge your PR into develop - some useful clues you can find [here](https://notes.status.im/setup-e2e#3-Merging-PR) 
 
 HAPPY DEVELOPMENT! :tada: