Skip to content

Collect NPM #101 #121

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 9 commits into from
Nov 1, 2019
Merged

Collect NPM #101 #121

merged 9 commits into from
Nov 1, 2019

Conversation

NavonilDas
Copy link
Contributor

Collecting vulnerabilities from npm registry.

Signed-off-by: Navonil Das <[email protected]>
Copy link
Member

@pombredanne pombredanne left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@NavonilDas thank you ++ for this. This is a very good start!

A few global comments before I can review further:

  1. can you ensure you rebase your branch on the latest develop branch?
  2. please use pep8 formatting
  3. use a docstring for function documentation rather than a # comment before the function
  4. most important of all, we need some unit tests for all this

import semantic_version
import re
import json
from urllib.request import urlopen
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you mind to sort imports?
Also can you add a standard license header as in the other files?

NPM_URL = 'https://registry.npmjs.org{}'
PAGE = '/-/npm/v1/security/advisories?page=1'

# Removes spaces and v Charecter in front of version
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you make this a docstring rather than a comment? Also plase make sure you use pep8-like formatting (e.g. 2 empty lines between top level members)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok, I will change the requested things.


# Removes spaces and v Charecter in front of version
def remove_spaces(x):
x = re.sub(r" +"," ",x) # Replace Multiple spaces to one
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please avoid end of line comments if possible. And use single ' quotes unless for docstrings.

That said, what is the purpose of this function? Are spaces an issue for the semantic version library?
And this is a rather complex function, so we absolutely need many tests for this

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This function removes multiple spaces and v character in front of the version (like v2.1.3). The semantic version module throws parsing exception due to these conditions.

yes, tests would be a good idea.

@haikoschol
Copy link
Collaborator

@NavonilDas Thanks for making those changes. However, your last commit is missing the "Signed-off-by" bit. Could you please add it? You can automate that for future commits with a git hook:

$ cat .git/hooks/prepare-commit-msg
#!/bin/sh

NAME=$(git config user.name)
EMAIL=$(git config user.email)

if [ -z "$NAME" ]; then
    echo "empty git config user.name"
    exit 1
fi

if [ -z "$EMAIL" ]; then
    echo "empty git config user.email"
    exit 1
fi

git interpret-trailers --if-exists doNothing --trailer \
    "Signed-off-by: $NAME <$EMAIL>" \
    --in-place "$1"

Your branch also still needs to be rebased on current develop. The conflicting code on both branches does the same thing, just with different syntax (string formatting with '{}'.format(...) vs f'{...}'). I prefer the f'{...}' syntax, but pick whichever one you want.

Let me know if you need any help. I can probably also do the rebase for you if you prefer that.

@NavonilDas
Copy link
Contributor Author

@haikoschol Thanks for the script.

After writing a few Tests I will rebase it to develop branch.

Signed-off-by: Navonil Das <[email protected]>
Signed-off-by: Navonil Das <[email protected]>
Signed-off-by: Navonil Das <[email protected]>
Signed-off-by: Navonil Das <[email protected]>
Signed-off-by: Navonil Das <[email protected]>
Signed-off-by: Navonil Das <[email protected]>
Copy link
Member

@pombredanne pombredanne left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for the update and the progress... See my comments inline.

README.md Outdated
@@ -68,6 +68,37 @@ DJANGO_DEV=1 python manage.py test vulnerabilities/tests
DJANGO_DEV=1 python manage.py import --all
```

If you want to run the import periodically, you can use a systemd timer:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not sure why this (which is from #124 ) ends up in your PR.... di you merge rather than rebase on develop?

@@ -127,7 +127,17 @@ def archlinux_dump(extract_data):
VulnerabilityReference.objects.create(
vulnerability=vulnerability,
reference_id=vulnerability_id,
<<<<<<< HEAD
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You have an unresolved merge conflict here... but why would your npm code be related to archlinux code?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is because we both made some trivial changes to the same lines of the archlinux code. That's what I was referring to at the end of my earlier comment (#121 (comment)):

The conflicting code on both branches does the same thing, just with different syntax (string formatting with '{}'.format(...) vs f'{...}'). I prefer the f'{...}' syntax, but pick whichever one you want.

Signed-off-by: Navonil Das <[email protected]>
@haikoschol
Copy link
Collaborator

It looks like all review comments have been addressed. There is only one commit failing the DCO check. Since most of the commits on this branch only contain styling or merge fixes, I'll squash them and merge the PR.

@haikoschol haikoschol merged commit 58e3ff3 into aboutcode-org:develop Nov 1, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants