Adding Conformance Guidelines document.

Author: brahmaroutu

contributors/devel/README.md

@@ -43,6 +43,9 @@ Guide](http://kubernetes.io/docs/admin/).
 * **Document Conventions** ([how-to-doc.md](how-to-doc.md))
   Document style advice for contributors.
 
+* **Conformance Tests Guidelines** ([conformance-guidelines.md](conformance-guidelines.md))
+  Documenting Conformance tests.
+
 * **Running a cluster locally** ([running-locally.md](running-locally.md)):
   A fast and lightweight local cluster deployment for development.
 

contributors/devel/conformance-guidelines.md

@@ -0,0 +1,75 @@
+# Conformance Test Guidelines
+
+The Kubernetes conformance test suite is a set of testcases, currently a
+subset of the integration/e2e tests, that the Architecture SIG has approved
+to define the core set of interoperable features that all Kubernetes
+deployments must support.
+
+For each Kubernetes release, a Conformance Document will be generated
+that lists all of the tests that comprise the conformance test suite, along
+with the formal specification of each test. This document will help people
+understand what features are being tested without having to look through
+the testcase's code directly.
+
+## Adding New Tests
+
+To include a testcase in the conformance test suite, the following
+steps must be taken:
+- the testcase must use the `framework.ConformanceIt()` function rather
+  than the `framework.It()` function
+- the testcase must include a comment immediately before the
+  `framework.ConformanceIt()` call that includes all of the required
+  metadata about the test (see the [Test Metadata](#test-metadata) section)
+
+The PR associated with this change will be automatically be blocked until the 
+Architecture SIG has reviewed and approved the change. The mechanism by which 
+the Architecture SIG is notified of the desired change is still TBD.
+
+## Test Metadata
+
+Each conformance test must include the following piece of metadata
+within its associated comment:
+
+- `Release`: indicates the Kubernetes release that the test was added to the
+  conformance test suite. If the test was modified in subsequent releases
+  then those releases should be included as well (comma separated)
+- `Testname`: a human readable short name of the test
+- `Description`: a detailed description of the test. This field must describe
+  the required behaviour of the Kubernetes components being tested using 
+  [RFC2119](https://tools.ietf.org/html/rfc2119) keywords. This field
+  is meant to be a "specification" of the tested Kubernetes features, as
+  such, it must be detailed enough so that readers can fully understand
+  the aspects of Kubernetes that are being tested without having to read
+  the test's code directly. Additionally, this test should provide a clear
+  distinction between the parts of the test that are there for the purpose
+  of validating Kubernetes rather than simply infrastructure logic that
+  is necessary to setup, or clean up, the test.
+
+## Sample Conformance Test
+
+The following snippet of code shows a sample conformance test's metadata:
+
+```
+/*
+  Release : v1.9
+  Testname: Kubelet: log output
+  Description: By default the stdout and stderr from the process being
+  executed in a pod MUST be sent to the pod's logs.
+*/
+framework.ConformanceIt("it should print the output to logs", func() {
+  ...
+})
+```
+
+The corresponding portion of the Conformance Document would then look
+like this:
+
+- - -
+## [Kubelet: log output](https://github.com/kubernetes/kubernetes/tree/release-1.9/test/e2e_node/kubelet_test.go#L47)
+
+Release : v1.9
+By default the stdout and stderr from the process being executed in a pod MUST be sent to the pod's logs.
+```
+- - -
+
+