Skip to content

Refactor scripts/create_repository.py, fix some issues #1639

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
22 commits
Select commit Hold shift + click to select a range
96f28f4
First create_repository.py refactoring
mbland Oct 29, 2024
4a1a8e8
Second create_repository.py refactoring
mbland Oct 29, 2024
91d549a
Third create_repository.py refactoring
mbland Oct 29, 2024
772344b
Small create_repository.py format updates, fixes
mbland Oct 30, 2024
5162cb2
Ensure Scala 2 libs get "_2" repo names in Scala 3
mbland Oct 30, 2024
c5244de
Bump Scalafmt to 3.8.3 in create_repository.py
mbland Oct 30, 2024
ad71483
Prevent create_repository.py from downgrading jars
mbland Oct 29, 2024
d0aef69
Ensure all artifact deps written to repo files
mbland Oct 30, 2024
0e06ea3
Update get_label, fix scala3 label bug
mbland Oct 31, 2024
d2f6d04
Replace `split_artifact_and_version`
mbland Oct 31, 2024
ab8bf1e
Check against current ResolvedArtifact instances
mbland Oct 31, 2024
05c55c4
Handle scalap, org.thesamet.scalapb special cases
mbland Oct 31, 2024
9cc23f8
Add --version, emit command, stderr only on error
mbland Oct 31, 2024
b71e9db
Raise and catch SubprocessError, return exit code
mbland Oct 31, 2024
8cbb842
Hoist output_dir from create_or_update_repo_file
mbland Oct 31, 2024
92ee9d5
Hoist OUTPUT_DIR as top level constant
mbland Oct 31, 2024
f625f23
Add --output_dir flag, add defaults to --help
mbland Oct 31, 2024
816cf5e
Create a new empty file if no previous file exists
mbland Oct 31, 2024
736d427
Remove redundant `file.exists()` check
mbland Nov 1, 2024
4c4afc4
Improve `create_repository.py --help` docstrings
mbland Nov 1, 2024
d22e72f
Refactor `get_label` in `create_repository.py`
mbland Nov 1, 2024
f05a0f2
Emit correct label for scala-collection-compat
mbland Nov 4, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 4 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -10,4 +10,7 @@ hash2
.vscode
unformatted-*.backup.scala
.scala-build
test/semanticdb/tempsrc
test/semanticdb/tempsrc

# From scripts/create_repository.py
repository-artifacts.json
115 changes: 80 additions & 35 deletions scripts/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,56 +6,101 @@
- [Debugging](#debugging)
- [Requirements](#requirements)

### About
The script allows to update a certain scala_x_x.bzl file and its content (artifact, sha, dependencies), by changing the value of `root_scala_version` variable.
It can be used to create non-existent file for chosen Scala version. <br>
It's using a [https://get-coursier.io/docs/](coursier) in order to **resolve** lists the transitive dependencies of dependencies and **fetch** the JARs of it.

### Usage
Usage from `/rules_scala/scripts`:
````
python3 create_repository.py
````

### Examples
Current value of `root_scala_versions`:
## About

The script allows to update a certain scala_x_x.bzl file and its content
(artifact, sha, dependencies), by changing the value of `root_scala_version`
variable.

It can be used to create non-existent file for chosen Scala version.

It's using a [https://get-coursier.io/docs/](coursier) in order to **resolve**
lists the transitive dependencies of dependencies and **fetch** the JARs of it.

## Usage

Usage from the rules_scala root directory:

```sh
./scripts/create_repository.py
```
root_scala_versions = ["2.11.12", "2.12.19", "2.13.14", "3.1.3", "3.2.2", "3.3.3", "3.4.3", "3.5.0"]

## Examples

Current value of `root_scala_versions`:

```py
root_scala_versions = [
"2.11.12",
"2.12.19",
"2.13.14",
"3.1.3",
"3.2.2",
"3.3.3",
"3.4.3",
"3.5.0",
]
```

To **update** content of `scala_3_4.bzl` file:
```
root_scala_versions = ["2.11.12", "2.12.19", "2.13.14", "3.1.3", "3.2.2", "3.3.3", "3.4.4", "3.5.0"]
^^^^^^^ <- updated version

```py
root_scala_versions = [
"2.11.12",
"2.12.19",
"2.13.14",
"3.1.3",
"3.2.2",
"3.3.3",
"3.4.4", # <- updated version
"3.5.0"
]
```

To **create** new `scala_3_6.bzl` file:
```
root_scala_versions = ["2.11.12", "2.12.19", "2.13.14", "3.1.3", "3.2.2", "3.3.3", "3.4.3", "3.5.0", "3.6.0"]
^^^^^^^ <- new version
```

### Debugging
Certain dependency version may not have a support for chosen Scala version e.g.
```py
root_scala_versions = [
"2.11.12",
"2.12.19",
"2.13.14",
"3.1.3",
"3.2.2",
"3.3.3",
"3.4.3",
"3.5.0",
"3.6.0", # <- new version
]
```

## Debugging

Certain dependency versions may not support a specific Scala versions, e.g.,

```py
kind_projector_version = "0.13.2" if scala_major < "2.13" else "0.13.3"
```

In order of that, there may be situations that script won't work. To debug that problem and adjust the values of hard-coded variables:
```
scala_test_major = "3" if scala_major >= "3.0" else scala_major
scala_fmt_major = "2.13" if scala_major >= "3.0" else scala_major
There may be situations in which the script won't work. To debug that problem
and adjust the values of hard-coded variables:

```py
scalatest_major = "3" if scala_major >= "3.0" else scala_major
scalafmt_major = "2.13" if scala_major >= "3.0" else scala_major
kind_projector_version = "0.13.2" if scala_major < "2.13" else "0.13.3"
f"org.scalameta:scalafmt-core_{scala_fmt_major}:{"2.7.5" if scala_major == "2.11" else scala_fmt_version}"
scalafmt_version = "2.7.5" if scala_major == "2.11" else SCALAFMT_VERSION
```
there is an option to print the output of these two subprocesses:

`output = subprocess.run(f'cs fetch {artifact}', capture_output=True, text=True, shell=True).stdout.splitlines()` <br>
there is an option to print the output of these two subprocesses:

```py
command = f'cs resolve {' '.join(root_artifacts)}'
output = subprocess.run(
command, capture_output=True, text=True, shell=True
).stdout.splitlines()
```
command = f'cs resolve {' '.join(root_artifacts)}'
output = subprocess.run(command, capture_output=True, text=True, shell=True).stdout.splitlines()
```

### Requirements
Installed [Coursier](https://get-coursier.io/) and [Python 3](https://www.python.org/downloads/)
## Requirements

Install [Coursier](https://get-coursier.io/) and
[Python 3](https://www.python.org/downloads/) before running the script.
Loading