Limit backports to 8.3+

[terrancedejesus]

Summary

As we move forward, we plan to limit automated backports to 8.3 within detection rules. We will continue to support older stacks, however as we add new features, we aim to prioritize forward progress within our schemas. This PR limits the automation to backport to 8.3, which is the first of future PRs that will be a sliding window as stack versions increase.

See the gdoc for more details.

[terrancedejesus]

Update - 01-25-2023

Known Issues:
✅ 1. After trimming lock versions file with trim-version-lock CLI command, we noticed that the previous and current min-stack entries in the version lock file were the same, causing version and SHA256 collisions.
✅ 2. After trimming the version lock file and running build-release --update-version-lock CLI command, we noticed that existing current versions were reduced (i.e. from 110 -> 10) but the buffers for previous locks still existing up to 109. This may be a result of the first issue listed.

After further review, we have concluded that the second issue is the result of the first issue. Since the updated version lock file from this PR was incorrect after trimming, thus when updating the version lock file via build-release --update-version-lock the issue was discovered.

Addressing the first issue, we changed the logic to outdated >= stack_version where all previous (forked) entries in the version lock file are compared with the trimmed stack version. This trim stacked version is the target version we want to use and simply remove anything lower from the previous (forked) entries. In this case, we used 8.3 and 8.4 for testing. With the updated logic we had the following outcome.

Rule 1 Scenario
8.4 - Trim Version
8.4 - Current Version
8.3 - Previous Version (8.3 >= 8.4 -> False (This forked entry will not be added back to the version lock file and is popped out))

Rule 2 Scenario
8.3 - Trim Version
8.4 - Current Version
8.3 - Previous Version (8.3 >= 8.3 -> True (This forked entry will be added back to the previous entries))
7.16 - Previous Version (7.16 >= 8.4 -> False (This forked entry will not be added back to the version lock file and is popped out))

In this case, the version.lock.json file was reviewed after trimming for both 8.3 and 8.4 to check that the results were desirable.

[Mikaayenson]

Related issues for further reason to drop support:

• [Bug] Resolve backport checks on 2470 by checking Version min_stack #2519
• [Bug] Check integrations cross major versions for older release support #2520

[Mikaayenson]

More related issues for further reason to drop support:

• [Bug] Patch fix for #2503 #2527
• [Bug] Patch fix for 2503 update addressing separate bugs #2528

[Mikaayenson]

Another related issue:

• [Test] Restrict host.os.type unit test to 8.3+ #2615

[terrancedejesus]

@approksiu as of the FF for 8.8 release, we will be releasing from 8.8, 8.7, 8.6 and 8.5. For any PRs to detection rules we currently manage backporting for 10 branches. What is our goal moving forward with this at the moment? I am happy to limit the backports for now to our target release.

[terrancedejesus]

Update 06/12/2023

We are moving forward with limiting these backports. At the moment, we will limit backports and trim the version lock file for 8.3+ only.

Once merged, we will ultimately detach our future dev from 8.2-. This means if we do any rule additions, rule validation or schema changes, we are not ensuring that these changes are valid with these stack versions, only 8.3+.

Our rules are still available in EPR for 8.2- as well as the repo tags in Detection rules. After merging this, we need to create a doc that helps users convert rules that they may want to use for out-of-date stacks.