Skip to content

Commit fd12a28

Browse files
authored
Rethink should_panic and fail_uncoverable options as global conditions (#2967)
This PR is the next step to rework/introduce the `should_panic`/`fail_uncoverable` options as global conditions. Until now, we haven't had a concrete proposal to do so other than the implementation in #2532. This PR presents one for each option in their respective RFCs. I'd like to agree on this design before starting the code review for #2532. Related to #1905 #2272 #2299 #2516
1 parent 871c9e3 commit fd12a28

File tree

2 files changed

+27
-13
lines changed

2 files changed

+27
-13
lines changed

rfc/src/rfcs/0003-cover-statement.md

+11-2
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22
- **Feature Request Issue:** <https://github.com/model-checking/kani/issues/696>
33
- **RFC PR:** <https://github.com/model-checking/kani/pull/1906>
44
- **Status:** Unstable
5-
- **Version:** 0
5+
- **Version:** 1
66

77
-------------------
88

@@ -85,7 +85,16 @@ fn foo() {
8585
}
8686
```
8787

88-
We can consider adding an option that would cause verification to fail if a cover property was unsatisfiable or unreachable, e.g. `--fail-uncoverable`.
88+
The `--fail-uncoverable` option will allow users to fail the verification if a cover property is unsatisfiable or unreachable.
89+
This option will be integrated within the framework of [Global Conditions](https://model-checking.github.io/kani/rfc/rfcs/0007-global-conditions.html), which is used to define properties that depend on other properties.
90+
91+
Using the `--fail-uncoverable` option will enable the global condition with name `fail_uncoverable`.
92+
Following the format for global conditions, the outcome will be one of the following:
93+
1. `` - fail_uncoverable: FAILURE (expected all cover statements to be satisfied, but at least one was not)``
94+
2. `` - fail_uncoverable: SUCCESS (all cover statements were satisfied as expected)``
95+
96+
Note that the criteria to achieve a `SUCCESS` status depends on all properties of the `"cover"` class having a `SATISFIED` status.
97+
Otherwise, we return a `FAILURE` status.
8998

9099
### Inclusion in the Verification Summary
91100

rfc/src/rfcs/0005-should-panic-attr.md

+16-11
Original file line numberDiff line numberDiff line change
@@ -2,8 +2,10 @@
22
- **Feature Request Issue:** <https://github.com/model-checking/kani/issues/600>
33
- **RFC PR:** <https://github.com/model-checking/kani/pull/2272>
44
- **Status:** Unstable
5-
- **Version:** 0
6-
- **Proof-of-concept:** <https://github.com/model-checking/kani/pull/2315>
5+
- **Version:** 1
6+
- **Proof-of-concept:**
7+
* Version 0: <https://github.com/model-checking/kani/pull/2315>
8+
* Version 1: <https://github.com/model-checking/kani/pull/2532>
79

810
-------------------
911

@@ -95,20 +97,23 @@ Note that it's important that we provide the user with this feedback:
9597
2. **(Outcome)**: What's the actual result that Kani produced after the analysis?
9698
This will avoid a potential scenario where the user doesn't know for sure if the attribute has had an effect when verifying the harness.
9799

100+
Therefore, the representation must make clear both the expectation and the outcome.
98101
Below, we show how we'll represent this result.
99102

100-
#### Recommended Representation: Changes to overall result
101103

102-
The representation must make clear both the expectation and the outcome.
103-
Moreover, the overall result must change according to the verification results (i.e., the failures that were found).
104+
#### Recommended Representation: As a Global Condition
104105

105-
Using the `#[kani::should_panic]` attribute will return one of the following results:
106-
1. `VERIFICATION:- FAILED (encountered no panics, but at least one was expected)` if there were no failures.
107-
2. `VERIFICATION:- FAILED (encountered failures other than panics, which were unexpected)` if there were failures but not all them had `prop.property_class() == "assertion"`.
108-
3. `VERIFICATION:- SUCCESSFUL (encountered one or more panics as expected)` otherwise.
106+
The `#[kani::should_panic]` attribute essentially behaves as a property that depends on other properties.
107+
This makes it well-suited for integration within the framework of [Global Conditions](https://model-checking.github.io/kani/rfc/rfcs/0007-global-conditions.html).
109108

110-
Note that the criteria to achieve a `SUCCESSFUL` result depends on all failures having the property class `"assertion"`.
111-
If they don't, then the failed properties may contain UB, so we return a `FAILED` result instead.
109+
Using the `#[kani::should_panic]` attribute will enable the global condition with name `should_panic`.
110+
Following the format for global conditions, the outcome will be one of the following:
111+
1. `` - `should_panic`: FAILURE (encountered no panics, but at least one was expected)`` if there were no failures.
112+
2. `` - `should_panic`: FAILURE (encountered failures other than panics, which were unexpected)`` if there were failures but not all them had `prop.property_class() == "assertion"`.
113+
3. `` - `should_panic`: SUCCESS (encountered one or more panics as expected)`` otherwise.
114+
115+
Note that the criteria to achieve a `SUCCESS` status depends on all failed properties having the property class `"assertion"`.
116+
If they don't, then the failed properties may contain UB, so we return a `FAILURE` status instead.
112117

113118
### Multiple Harnesses
114119

0 commit comments

Comments
 (0)