-
Notifications
You must be signed in to change notification settings - Fork 109
Create new model for dependencies #447
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Comments
@JonoYang I remember you had committed a new model for the dependencies but we decided to postpone after the toolkit upgrade. |
Signed-off-by: Jono Yang <[email protected]>
@tdruez I started a new branch with the |
* Update scan_for_application_packages to save detected Package data to the CodebaseResource it is from, then iterate through the CodebaseResources with Package data and use the proper Package handler to process the Package data * Create DiscoveredDependency model * Add package_data JSON field to CodebaseResource Signed-off-by: Jono Yang <[email protected]>
* Increase field sizes in DiscoveredDependency Signed-off-by: Jono Yang <[email protected]>
Signed-off-by: Jono Yang <[email protected]>
Signed-off-by: Jono Yang <[email protected]>
Signed-off-by: Jono Yang <[email protected]>
Signed-off-by: Jono Yang <[email protected]>
Signed-off-by: Jono Yang <[email protected]>
* We never run into the situation where we are updating a DiscoveredDependency when scanning packages * Clean up migrations Signed-off-by: Jono Yang <[email protected]>
* We never run into the situation where we are updating a DiscoveredDependency when scanning packages * Clean up migrations Signed-off-by: Jono Yang <[email protected]>
* Update scancode-toolkit to 31.0.0rc3 #447 * Update expected test results Signed-off-by: Jono Yang <[email protected]>
* Update scancode-toolkit to 31.0.0rc3 #447 * Update expected test results * Update code formatting Signed-off-by: Jono Yang <[email protected]>
* Update scancode-toolkit to 31.0.0rc3 #447 * Update expected test results * Update code formatting Signed-off-by: Jono Yang <[email protected]>
* This is to reflect the changes made to the Package model in scancode-toolkit * Update test expectations Signed-off-by: Jono Yang <[email protected]>
* This is to reflect the changes made to the Package model in scancode-toolkit * Update test expectations Signed-off-by: Jono Yang <[email protected]>
* purl and dependency_uid are the only required fields on DiscoveredDependency * Update migrations * Check for created DiscoveredDependency in tests Signed-off-by: Jono Yang <[email protected]>
* Create DiscoveredDependencies in load_codebase pipeline * Update tests to check for DiscoveredDependencies * Update test expectations Signed-off-by: Jono Yang <[email protected]>
* Update test expectations Signed-off-by: Jono Yang <[email protected]>
* This will fail until scancode-toolkit 31.0.0rc4 is released * Update test expectations Signed-off-by: Jono Yang <[email protected]>
Signed-off-by: Jono Yang <[email protected]>
Signed-off-by: Jono Yang <[email protected]>
Signed-off-by: Jono Yang <[email protected]>
Signed-off-by: Jono Yang <[email protected]>
Signed-off-by: Jono Yang <[email protected]>
* Update DiscoveredDependency ordering * Update daglib test expectations Signed-off-by: Jono Yang <[email protected]>
* Update test expectations Signed-off-by: Jono Yang <[email protected]>
* Update test expectations Signed-off-by: Jono Yang <[email protected]>
Signed-off-by: Thomas Druez <[email protected]>
Signed-off-by: Thomas Druez <[email protected]>
Signed-off-by: Thomas Druez <[email protected]>
Signed-off-by: Thomas Druez <[email protected]>
Signed-off-by: Thomas Druez <[email protected]>
* Only show links in dependency for_package tab or dependency datafile_resource tab if there is a value Signed-off-by: Jono Yang <[email protected]>
* Use UpdateFromDataMixin in DiscoveredDependency * Create test for DiscoveredDependency.update_from_data() Signed-off-by: Jono Yang <[email protected]>
* Use UpdateFromDataMixin in DiscoveredDependency * Create test for DiscoveredDependency.update_from_data() Signed-off-by: Jono Yang <[email protected]>
Signed-off-by: Thomas Druez <[email protected]>
* Implement package assembly in scancode.io #447 Signed-off-by: Jono Yang <[email protected]> * Minor formatting changes for consistency #447 Signed-off-by: Thomas Druez <[email protected]> * Create DiscoveredPackages before other models #447 Signed-off-by: Jono Yang <[email protected]> * Revert "Create DiscoveredPackages before other models #447" This reverts commit c9b8bed. Sorting Packages, Dependencies, and Resources from DatafileHandler.assemble() will never work. The code needs to be changed in scancode-toolkit. Signed-off-by: Jono Yang <[email protected]> * Update migration #444 Signed-off-by: Jono Yang <[email protected]> * Return package_uids in for_packages #444 * This is so we are consistent with scancode-toolkit JSON output * Update expected test results Signed-off-by: Jono Yang <[email protected]> * Create directory Resources in docker pipeline #485 * Update test expectations Signed-off-by: Jono Yang <[email protected]> * Implement package assembly in scancode.io #447 Signed-off-by: Jono Yang <[email protected]> * Implement package assembly in scancode.io #447 Signed-off-by: Jono Yang <[email protected]> * Create DiscoveredDependency model #447 * Create new dependency list and detail views * Update assemble_packages() to create DiscoveredDependencies * Update test expectations Signed-off-by: Jono Yang <[email protected]> * Update fields on DiscoveredDependency #447 * Remove for_package_uid and replace with ForeignKey for_package * Remove datafile_path and replace with ForeignKey datafile_resource * Create properties for the two removed fields * Update dependency views to link to datafile_resource * Update expected test results Signed-off-by: Jono Yang <[email protected]> * Properly pluralize verbose name #447 Signed-off-by: Jono Yang <[email protected]> * Create new argument for create_from_data #447 * Add strip_datafile_path_root to DiscoveredDependency.create_from_data * This argument strips the root path segment from `datafile_path` before using the path to look up the corresponding CodebaseResource * This is used in the case where we are importing a scan from scancode-toolkit, where the root path segments are not stripped by default * Update expected test results Signed-off-by: Jono Yang <[email protected]> * Update prefetch_related #447 * Used cached_property for DiscoveredDependency properties Signed-off-by: Jono Yang <[email protected]> * Prefetch related models in output code #447 Signed-off-by: Jono Yang <[email protected]> * Import scancode.io 30.2.0 scans in load_codebase * Order DiscoveredDependencies by is_runtime, is_optional, is_resolved, and dependency_uid * Do not show dependency_uid value in DiscoveredDependency list view Signed-off-by: Jono Yang <[email protected]> * Revert changes for importing old scancode.io scans Signed-off-by: Jono Yang <[email protected]> * Regen migrations for DiscoveredDependency #447 Signed-off-by: Jono Yang <[email protected]> * Migrate DiscoveredPackage.dependencies #447 * Create migrations to generate new DiscoveredDependency objects from DiscoveredPackage.dependencies before removing the dependencies field Signed-off-by: Jono Yang <[email protected]> * Update test expectations #447 Signed-off-by: Jono Yang <[email protected]> * Remove accidentally committed files #447 Signed-off-by: Jono Yang <[email protected]> * Update migration logic #447 * Remove unnecessary else from DiscoveredDependency properties Signed-off-by: Jono Yang <[email protected]> * Add PackageURLMixin to DiscoveredDependency #447 Signed-off-by: Jono Yang <[email protected]> * Set DiscoveredDependencies purl fields #447 * Create migration that populates purl fields for existing DiscoveredDependencies Signed-off-by: Jono Yang <[email protected]> * Store purl values in purl fields #447 * Do not store dependency_uid in purl fields Signed-off-by: Jono Yang <[email protected]> * Remove purl field from DiscoveredDependency #447 * We are already storing this info in the purl fields * Create purl property on DiscoveredDependency for compatibility Signed-off-by: Jono Yang <[email protected]> * Update DependencyFilterSet #447 * Add search and purl fields Signed-off-by: Jono Yang <[email protected]> * Don't show DiscoveredDependencies purl fields #447 Signed-off-by: Jono Yang <[email protected]> * Update package detail view dependencies tab #447 Signed-off-by: Jono Yang <[email protected]> * Add package_type to dependency serializer #511 * Update test expectations Signed-off-by: Jono Yang <[email protected]> * Update expected test results Signed-off-by: Jono Yang <[email protected]> * Add dependency table column #447 Signed-off-by: Jono Yang <[email protected]> * Use tabset in dependency detail view #447 * Add package_type property to DiscoveredDependency Signed-off-by: Jono Yang <[email protected]> * Update dependency list view #447 * Use updated table header include * Update dependency presentation in package detail view * Show package uid on hover on for package tab Signed-off-by: Jono Yang <[email protected]> * Set DiscoveredDependency serializer fields #511 * Update DiscoveredDependency ordering Signed-off-by: Jono Yang <[email protected]> * Create donut chart for package type #447 Signed-off-by: Jono Yang <[email protected]> * Consolidate migrations #447 * Update DiscoveredDependency ordering * Update daglib test expectations Signed-off-by: Jono Yang <[email protected]> * Update dependency JSON ordering #447 * Update test expectations Signed-off-by: Jono Yang <[email protected]> * Set proper discovereddependencies related_name #447 Signed-off-by: Thomas Druez <[email protected]> * Fix template indentation #447 Signed-off-by: Thomas Druez <[email protected]> * Refactor update_from_data method into a UpdateFromDataMixin #447 Signed-off-by: Thomas Druez <[email protected]> * Fix the ProjectSerializer fields #447 Signed-off-by: Thomas Druez <[email protected]> * Fix test_scanpipe_api_project_detail unit test #447 Signed-off-by: Thomas Druez <[email protected]> * Add HTML title for list views #506 Signed-off-by: Thomas Druez <[email protected]> * Update dependency tabs #447 * Only show links in dependency for_package tab or dependency datafile_resource tab if there is a value Signed-off-by: Jono Yang <[email protected]> * Use UpdateFromDataMixin #447 * Use UpdateFromDataMixin in DiscoveredDependency * Create test for DiscoveredDependency.update_from_data() Signed-off-by: Jono Yang <[email protected]> * Fix formatting #447 Signed-off-by: Thomas Druez <[email protected]> Signed-off-by: Jono Yang <[email protected]> Signed-off-by: Thomas Druez <[email protected]> Co-authored-by: Thomas Druez <[email protected]>
PR merged. |
Actually getting an issue when applying the migrations on a production database.
|
Signed-off-by: Thomas Druez <[email protected]>
Signed-off-by: Thomas Druez <[email protected]>
Signed-off-by: Thomas Druez <[email protected]>
Signed-off-by: Thomas Druez <[email protected]>
Signed-off-by: Thomas Druez <[email protected]>
Signed-off-by: Thomas Druez <[email protected]>
Signed-off-by: Thomas Druez <[email protected]>
Migration issues fixed in #514 |
Signed-off-by: Thomas Druez <[email protected]>
scancode-toolkit has a new model called
DependentPackage
, which models the dependencies detected for a Package from a package manifest. TheDependentPackage
data shows up in the scancode JSON results in the top-levelDependencies
field. We should create a new model,DiscoveredDependency
(or some other name) to represent this in scancode.ioThe text was updated successfully, but these errors were encountered: