Skip to content

Commit b41f965

Browse files
committed
added files to sig-security
Signed-off-by: Ayushman <[email protected]>
1 parent 16f9ec4 commit b41f965

28 files changed

+2003
-0
lines changed

docs/OWNERS

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
# See the OWNERS docs at https://go.k8s.io/owners
2+
3+
reviewers:
4+
- savitharaghunathan
5+
approvers:
6+
- savitharaghunathan
Binary file not shown.
Lines changed: 118 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,118 @@
1+
# Request for Proposal
2+
3+
## Kubernetes Third Party Security Audit
4+
5+
The Kubernetes Third-Party Audit Working Group (working group, henceforth) is soliciting proposals from select Information Security vendors for a comprehensive security audit of the Kubernetes Project.
6+
7+
### Eligible Vendors
8+
9+
Only the following vendors will be permitted to submit proposals:
10+
11+
- NCC Group
12+
- Trail of Bits
13+
- Cure53
14+
- Bishop Fox
15+
- Insomnia
16+
- Atredis Partners
17+
18+
If your proposal includes sub-contractors, please include relevant details from their firm such as CVs, past works, etc.
19+
20+
### RFP Process
21+
22+
This RFP will be open between 2018/10/29 and 2019/11/30.
23+
24+
The working group will answer questions for the first two weeks of this period.
25+
26+
Questions can be submitted [here](https://docs.google.com/forms/d/e/1FAIpQLSd5rXSDYQ0KMjzSEGxv0pkGxInkdW1NEQHvUJpxgX3y0o9IEw/viewform?usp=sf_link). All questions will be answered publicly in this document.
27+
28+
Proposals must include CVs, resumes, and/or example reports from staff that will be working on the project.
29+
30+
- 2018/10/29: RFP Open, Question period open
31+
- 2018/11/12: Question period closes
32+
- 2018/11/30: RFP Closes
33+
- 2018/12/11: The working group will announce vendor selection
34+
35+
## Audit Scope
36+
37+
The scope of the audit is the most recent release (1.12) of the core [Kubernetes project](https://github.com/kubernetes/kubernetes).
38+
39+
- Findings within the [bug bounty program](https://github.com/kubernetes/community/blob/master/contributors/guide/bug-bounty.md) scope are in scope.
40+
41+
We want the focus of the audit to be on bugs on Kubernetes. While Kubernetes relies upon a container runtimes such as Docker and CRI-O, we aren't looking for (for example) container escapes that rely upon bugs in the container runtime (unless, for example, the escape is made possible by a defect in the way that Kubernetes sets up the container).
42+
43+
### Focus Areas
44+
45+
The Kubernetes Third-Party Audit Working Group is specifically interested in the following areas. Proposals should indicate their level of expertise in these fields as it relates to Kubernetes.
46+
47+
- Networking
48+
- Cryptography
49+
- Authentication & Authorization (including Role Based Access Controls)
50+
- Secrets management
51+
- Multi-tenancy isolation: Specifically soft (non-hostile co-tenants)
52+
53+
### Out of Scope
54+
55+
Findings specifically excluded from the [bug bounty program](https://github.com/kubernetes/community/blob/master/contributors/guide/bug-bounty.md) scope are out of scope.
56+
57+
## Methodology
58+
59+
We are allowing 8 weeks for the audit, start date can be negioated after vendor selection. We recognize that November and December can be very high utilization periods for security vendors.
60+
61+
The audit should not be treated as a penetration test, or red team exercise. It should be comprehensive and not end with the first successful exploit or critical vulnerability.
62+
63+
The vendor should perform both source code analysis as well as live evaluation of Kubernetes.
64+
65+
The vendor should document the Kubernetes configuration and architecture that the audit was performed against for the creation of a "audited reference architecture" artifact. The working group must approve this configuration before the audit continues.
66+
67+
The working group will establish a 60 minute kick-off meeting to answer any initial questions and explain Kubernetes architecture.
68+
69+
The working group will be available weekly to meet with the selected vendor, will and provide subject matter experts for requested components.
70+
71+
The vendor must report urgent security issues immediately to both the working group and [email protected].
72+
73+
## Confidentiality and Embargo
74+
75+
All information gathered and artifacts created as a part of the audit must not be shared outside the vendor or the working group without the explicit consent of the working group.
76+
77+
## Artifacts
78+
79+
The audit should result in the following artifacts, which will be made public after any sensitive security issues are mitigated.
80+
81+
- Findings report, including an executive summary
82+
83+
- Audited reference architecture specification. Should take the form of a summary and associated configuration yaml files.
84+
85+
- Formal threat model
86+
87+
- Any proof of concept exploits that we can use to investigate and fix defects
88+
89+
- Retrospective white paper(s) on important security considerations in Kubernetes
90+
91+
*This artifact can be provided up to 3 weeks after deadline for the others.*
92+
93+
- E.g. [NCC Group: Understanding hardening linux containers](https://www.nccgroup.trust/globalassets/our-research/us/whitepapers/2016/april/ncc_group_understanding_hardening_linux_containers-1-1.pdf)
94+
- E.g. [NCC Group: Abusing Privileged and Unprivileged Linux
95+
Containers](https://www.nccgroup.trust/globalassets/our-research/us/whitepapers/2016/june/container_whitepaper.pdf)
96+
97+
## Q & A
98+
99+
| # | Question | Answer |
100+
|---|----------|--------|
101+
| 1 | The RFP says that any area included in the out of scope section of the k8s bug bounty programme is not in-scope of this review. There are some areas which are out of scope of the bug bounty which would appear to be relatively core to k8s, for example Kubernetes on Windows. Can we have 100% confirmation that these areas are out of scope? | Yes. If you encounter a vulnerability in Kubernetes' use of an out-of-scope element, like etcd or the container network interface (to Calico, Weave, Flannel, ...), that is in scope. If you encounter a direct vulnerability in a third-party component during the audit you should follow the embargo section of the RFP. |
102+
| 2 | On the subject of target Distribution and configuration option review:<br> The RFP mentions an "audited reference architecture".<br> - Is the expectation that this will be based on a specific k8s install mechanism (e.g. kubeadm)? <br> - On a related note is it expected that High Availability configurations (e.g. multiple control plane nodes) should be included.<br> - The assessment mentions Networking as a focus area. Should a specific set of network plugins (e.g. weave, calico, flannel) be considered as in-scope or are all areas outside of the core Kubernetes code for this out of scope.<br> - Where features of Kubernetes have been deprecated but not removed in 1.12, should they be considered in-scope or not? | 1. No, we are interested in the final topology -- the installation mechanism, as well as its default configuration, is tangental. The purpose is to contextualise the findings.<br>2. High-availability configurations should be included. For confinement of level of effort, vendor could create one single-master configuration and one high-availability configuration.<br>3. All plugins are out of scope per the bug bounty scope -- for clarification regarding the interface to plug-ins, please see the previous question.<br> 4. Deprecated features should be considered out of scope |
103+
| 3 | On the subject of dependencies:<br>- Will any of the project dependencies be in scope for the assessment? (e.g. https://github.com/kubernetes/kubernetes/blob/v1.14.3/Godeps/Godeps.json) | Project dependencies are in scope in the sense that they are **allowed** to be tested, but they should not be considered a **required** testing area. We would be interested in cases where Kubernetes is exploitable due to a vulnerability in a project depdendency. Vulnerabilities found in third-party dependencies should follow the embargo section of the RFP.|
104+
| 4 | Is the 8 weeks mentioned in the scope intended to be a limit on effort applied to the review, or just the timeframe for the review to occur in? | This is only a restriction on time frame, but is not intended to convey level of effort. |
105+
| 5| Will the report be released in its entirety after the issues have been remediated? | Yes. |
106+
| 6| What goals must be met to make this project a success? | We have several goals in mind:<br>1) Document a full and complete understanding of Kubernetes’ dataflow.<br>2) Achieve a reasonable understanding of potential vulnerability vectors for subsequent research.<br>3) Creation of artifacts that help third parties make a practical assessment of Kubernetes’ security position.<br>4) Eliminate design and architecture-level vulnerabilities.<br>5) Discover the most significant vulnerabilities, in both number and severity. |
107+
| 7 | Would you be open to two firms partnering on the proposal? | Yes, however both firms should collaborate on the proposal and individual contributors should all provide C.V.s or past works.|
108+
| 8| From a deliverables perspective, will the final report (aside from the whitepaper) be made public? | Yes. |
109+
| 9| The bug bounty document states the following is in scope, "Community maintained stable cloud platform plugins", however will the scope of the assessment include review of the cloud providers' k8s implementation? Reference of cloud providers: https://kubernetes.io/docs/concepts/cluster-administration/cloud-providers/ | Cloud provider-specific issues are excluded from the scope. |
110+
| 10| The bug bounty doc lists supply chain attacks as in scope and also says, "excluding social engineering attacks against maintainers". We can assume phishing these individuals is out of scope, but does the exclusion of social engineering against maintainers include all attacks involving individuals? For example, if we were to discover that one of these developers accidentally committed their SSH keys to a git repo unassociated with k8s and we could use these keys to gain access to the k8s project. Is that in scope? | Attacks against individual developers, such as the example provided, are out of scope for this engagement. |
111+
| 11| While suppression of logs is explicitly in scope, is log injection also in scope? | Log injection is in scope for the purposes of this audit.|
112+
| 12| Are all the various networking implementations in scope for the assessment? Ref: https://kubernetes.io/docs/concepts/cluster-administration/networking/#how-to-implement-the-kubernetes-networking-model | Please refer to question 1. |
113+
| 13| What does the working group refer to with formal threat model? Would STRIDE be a formal threat model in that sense?| A formal threat model should include a comprehensive dataflow diagram which shows data moving between different trust levels and assesses threats to that data using a system like STRIDE as the data moves between each process/component. Many good examples are present in Threat Modeling: Designing for Security by Adam Shostack. |
114+
| 14| Does Kubernetes uses any GoLang non-standard signing libraries? | An initial investigation has not uncovered any, however with a code base as large as Kubernetes, it is possible. |
115+
| 15| Does Kubernetes implement any cryptographic primitives on its own, i.e. primitives which are not part of the standard libraries? | An initial investigation has not uncovered any, however with a code base as large as Kubernetes, it is possible. |
116+
| 16| Presuming that live testing is part of the project, how does the working group see the "audited reference architecture" being defined? Is there a representative deployment, or a document describing a "default installation" that you foresee the engagement team using to inform the buildout of a test environment?| The purpose of the reference architecture is to define and document the configuration against which live testing was preformed. It should be generated collaboratively with the working group at the beginning of the project. We will want it to represent at least a common configuration, as in practice Kubernetes itself has no default configuration. It should take the form of a document detailing the set-up and configuration steps the vendor took to create their environment, ensuring an easily repeatable reference implementation. |
117+
| 17| The RFP describes ""networking and multi-tenancy isolation"" as one of the focus areas. <br/><br/>Can you describe for us what these terms mean to you? Can you also help us understand how you define a soft non-hostile co-tenant? Is a _hostile_ co-tenant also in scope?| By networking we mean vulnerabilities related to communication within and to/from the cluster: container to container, pod to pod, pod to service, and external to internal communications as described in [the networking documentation](https://kubernetes.io/docs/concepts/cluster-administration/networking/). <br/><br/>The concept of soft multi-tenancy is that you have a single cluster being shared by applications or groups within the same company or organization, with less intended restrictions of a hard multi-tenant platform like a PaaS that hosts multiple distinct and potentially hostile competing customers on a single cluster which requires stricter security assumptions. These definitions may vary by group and use case, but the idea is that you can have a cluster with multiple groups with their own namespaces, isolated by networking/storage/RBAC roles."|
118+
| 18| In the Artifacts section, you describe a Formal Threat Model as one of the outputs of the engagement. Can you expound on what this means to you? Are there any representative public examples you could point us to?| Please refer to question 13.|
Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,27 @@
1+
# Security Audit WG - RFP Decision Process
2+
3+
The Security Audit Working Group was tasked with leading the process of having a third party security audit conducted for the Kubernetes project. Our first steps were to select the vendors to be included in the Request for Proposal (RFP) process and create the RFP document setting out the goals of the audit. We were then responsible for evaluating the submitted proposals in order to select the vendor best suited to complete a security assessment against Kubernetes, a very complex and widely scoped project.
4+
5+
After publishing the initial RFP and distributing it to the eligible vendors, we had a period open for vendors to submit questions to better understand the project’s goals, which we made publicly available in the RFP document. While six (6) vendors were invited to participate, we ultimately received four (4) RFP responses, due to one vendor dropping out and two vendors partnering to submit a combined proposal.
6+
7+
The next stage of this project was more difficult: evaluating the responses and determining which vendor to use for the audit. With the list of eligible vendors already limited to a small set of very strong and well-known firms, it came as no surprise to us that they each had extremely compelling proposals that made choosing one over the other very difficult. The working group leads have years of experience on both sides of the table: writing proposals and conducting audits, as well as working with these vendors and their teams to assess companies we’ve worked at. To help us combine objective evaluations with our individual past experiences and knowledge of each of the vendors’ work and relevant experience (conference talks, white papers, published research and reports), we came up with a set of criteria that each of us used to rank the proposals on a scale of 1 to 5:
8+
9+
* Personnel fit and talent
10+
* Relevant understanding and experience (orchestration systems, containers, hardening, etc.)
11+
* The individual work products requested in the RFP:
12+
- Threat Model
13+
- Reference architecture
14+
- White paper
15+
- Assessment and report
16+
17+
While budget constraints became a part of the final selection, we wanted to leave cost out of the process as much as possible and focus on ensuring the community received the best possible audit. Based on this criteria, the scoring overall was extremely close, with the total scores all within a few points of each other.
18+
19+
| Vendor | Total Score |
20+
|--------|-------------|
21+
| Vendor A | 149 |
22+
| Vendor B | 149 |
23+
| Vendor C | 144 |
24+
| Vendor D | 135 |
25+
26+
After narrowing it down to our top two choices and some further discussions with those vendors, we decided to select the partnership of Atredis and Trail of Bits to complete this audit. We felt very strongly that the combination of these two firms, both composed of very senior and well known staff in the information security industry, would provide the best possible results. We look forward to working with them to kick off the actual audit process soon and for other Kubernetes contributors from the various SIGs to help partner with the working group on this assessment.
27+
Lines changed: 47 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,47 @@
1+
digraph K8S {
2+
subgraph cluster_apiserverinternal {
3+
node [style=filled];
4+
color=green;
5+
etcd[label="etcd"];
6+
label = "API Server Data Layer";
7+
}
8+
9+
subgraph cluster_apiserver {
10+
node [style=filled];
11+
color=blue;
12+
kubeapiserver[label="kube-apiserver"];
13+
kubeapiserver->etcd[label="HTTPS"]
14+
label = "API Server";
15+
}
16+
17+
subgraph cluster_mastercomponents {
18+
node [style=filled];
19+
label = "Master Control Plane Components";
20+
scheduler[label="Scheduler"];
21+
controllers[label="Controllers"]
22+
scheduler->kubeapiserver[label="Callback/HTTPS"];
23+
controllers->kubeapiserver[label="Callback/HTTPS"];
24+
color=black;
25+
}
26+
27+
subgraph cluster_worker {
28+
label="Worker"
29+
color="blue"
30+
kubelet->kubeapiserver[label="authenticated HTTPS"]
31+
kubeproxy[label="kube-proxy"]
32+
iptables->kubeproxy->iptables
33+
pods[label="pods with various containers"]
34+
pods->kubeproxy->pods
35+
}
36+
37+
subgraph cluster_internet {
38+
label="Internet"
39+
authuser[label="Authorized User via kubebctl"]
40+
generaluser[label="General User"]
41+
authuser->kubeapiserver[label="Authenticated HTTPS"]
42+
generaluser->pods[label="application-specific connection protocol"]
43+
}
44+
kubeapiserver->kubelet[label="HTTPS"]
45+
kubeapiserver->pods[label="HTTP",color=red]
46+
}
47+
Loading
Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
python3 tm.py --dfd > updated-dataflow.dot
2+
dot -Tpng < updated-dataflow.dot > updated-dataflow.png
3+
open updated-dataflow.png
Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
pytm==0.4

0 commit comments

Comments
 (0)