Skip to content

Reduce use of blacklist/whitelist terminology #3961

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 8 commits into from
Mar 9, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 6 additions & 2 deletions ChangeLog
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,10 @@ Release date: TBA

* Introduce logic for checking deprecated attributes in DeprecationMixin.

* Reduce usage of blacklist/whitelist terminology. Notably, ``extension-pkg-allow-list`` is an
alternative to ``extension-pkg-whitelist`` and the message ``blacklisted-name`` is now emitted as
``disallowed-name``. The previous names are accepted to maintain backward compatibility.


What's New in Pylint 2.7.3?
===========================
Expand Down Expand Up @@ -5069,7 +5073,7 @@ Release date: 2003-11-20
__init__ files, and provides a new option ignore-interface-methods,
useful when you're using zope Interface implementation in your project

* base checker checks for black listed builtins call (controlled by the
* base checker checks for disallowed builtins call (controlled by the
bad-functions option) and for use of * and **

* format checker checks for use of <> and "l" as long int marker
Expand Down Expand Up @@ -5106,7 +5110,7 @@ Release date: 2003-11-20
* still fixes in format checker : don't check comment and docstring,
check first line after an indent

* black and white list now apply to all identifiers, not only
* allowed/prohibited names now apply to all identifiers, not only
variables, so changed the configuration option from
(good|bad)-variable-names to (good|bad)-names

Expand Down
8 changes: 4 additions & 4 deletions doc/technical_reference/c_extensions.rst
Original file line number Diff line number Diff line change
Expand Up @@ -9,11 +9,11 @@ Linting C extension modules is not supported out of the box, especially since
pylint has no way to get an AST object out of the extension module.

But **pylint** actually has a mechanism which you might use in case you
want to analyze C extensions. **pylint** has a flag, called **extension-pkg-whitelist**,
through which you can tell it to import that module and to build an AST from that
imported module::
want to analyze C extensions. **pylint** has a flag, called **extension-pkg-allow-list**
(formerly **extension-pkg-whitelist**), through which you can tell it to
import that module and to build an AST from that imported module::

$ pylint --extension-pkg-whitelist=your_c_extension
$ pylint --extension-pkg-allow-list=your_c_extension

Be aware though that using this flag means that extensions are loaded into the
active Python interpreter and may run arbitrary code, which you may not want. This
Expand Down
4 changes: 2 additions & 2 deletions doc/user_guide/run.rst
Original file line number Diff line number Diff line change
Expand Up @@ -118,8 +118,8 @@ configuration.

Other useful global options include:

--ignore=<file[,file...]> Add files or directories to the blacklist. They
should be base names, not paths.
--ignore=<file[,file...]> Files or directories to be skipped. They should be
base names, not paths.
--output-format=<format> Select output format (text, json, custom).
--msg-template=<template> Modify text output message template.
--list-msgs Generate pylint's messages.
Expand Down
2 changes: 1 addition & 1 deletion doc/whatsnew/1.7.rst
Original file line number Diff line number Diff line change
Expand Up @@ -631,7 +631,7 @@ Other Changes
which can redefine builtins, such as six.moves and future.builtins.

* A new option was added, ``ignore-patterns``, which is used for building a
blacklist of directories and files matching the regex patterns, similar to the
ignore list of directories and files matching the regex patterns, similar to the
``ignore`` option.


Expand Down
4 changes: 2 additions & 2 deletions doc/whatsnew/2.5.rst
Original file line number Diff line number Diff line change
Expand Up @@ -64,9 +64,9 @@ Other Changes

* Add new ``good-names-rgx`` and ``bad-names-rgx`` to enable permitting or disallowing of names via regular expressions

To enable better handling of whitelisting/blacklisting names, we added two new config options: good-names-rgxs: a comma-
To enable better handling of permitted/disallowed names, we added two new config options: good-names-rgxs: a comma-
separated list of regexes, that if a name matches will be exempt of naming-checking. bad-names-rgxs: a comma-
separated list of regexes, that if a name matches will be always marked as a blacklisted name.
separated list of regexes, that if a name matches will be always marked as a disallowed name.

* Mutable ``collections.*`` are now flagged as dangerous defaults.

Expand Down
4 changes: 4 additions & 0 deletions doc/whatsnew/2.8.rst
Original file line number Diff line number Diff line change
Expand Up @@ -16,3 +16,7 @@ New checkers

Other Changes
=============

* Reduce usage of blacklist/whitelist terminology. Notably, ``extension-pkg-allow-list`` is an
alternative to ``extension-pkg-whitelist`` and the message ``blacklisted-name`` is now emitted as
``disallowed-name``. The previous names are accepted to maintain backward compatibility.
13 changes: 9 additions & 4 deletions examples/pylintrc
Original file line number Diff line number Diff line change
Expand Up @@ -3,17 +3,22 @@
# A comma-separated list of package or module names from where C extensions may
# be loaded. Extensions are loading into the active Python interpreter and may
# run arbitrary code.
extension-pkg-allow-list=

# A comma-separated list of package or module names from where C extensions may
# be loaded. Extensions are loading into the active Python interpreter and may
# run arbitrary code. (This is an alternative name to extension-pkg-allow-list
# for backward compatibility.)
extension-pkg-whitelist=

# Specify a score threshold to be exceeded before program exits with error.
fail-under=10

# Add files or directories to the blacklist. They should be base names, not
# paths.
# Files or directories to be skipped. They should be base names, not paths.
ignore=CVS

# Add files or directories matching the regex patterns to the blacklist. The
# regex matches against base names, not paths.
# Files or directories matching the regex patterns are skipped. The regex
# matches against base names, not paths.
ignore-patterns=

# Python code to execute, usually for sys.path manipulation such as
Expand Down
8 changes: 5 additions & 3 deletions man/pylint.1
Original file line number Diff line number Diff line change
Expand Up @@ -47,9 +47,9 @@ In Python 3 porting mode, all checkers will be disabled and only messages emitte
.IP "--verbose, -v"
In verbose mode, extra non-checker-related info will be displayed.
.IP "--ignore=<file>[,<file>...]"
Add files or directories to the blacklist. They should be base names, not paths. [default: CVS]
Files or directories to be skipped. They should be base names, not paths. [default: CVS]
.IP "--ignore-patterns=<pattern>[,<pattern>...]"
Add files or directories matching the regex patterns to the blacklist. The regex matches against base names, not paths. [default: none]
Files or directories matching the regex patterns are skipped. The regex matches against base names, not paths. [default: none]
.IP "--persistent=<y_or_n>"
Pickle collected data for later comparisons. [default: yes]
.IP "--load-plugins=<modules>"
Expand All @@ -60,8 +60,10 @@ Specify a score threshold to be exceeded before program exits with error. [defau
Use multiple processes to speed up Pylint. Specifying 0 will auto-detect the number of processors available to use. [default: 1]
.IP "--limit-inference-results=<number-of-results>"
Control the amount of potential inferred values when inferring a single object. This can help the performance when dealing with large functions or complex, nested conditions. [default: 100]
.IP "--extension-pkg-whitelist=<pkg[,pkg]>"
.IP "--extension-pkg-allow-list=<pkg[,pkg]>"
A comma-separated list of package or module names from where C extensions may be loaded. Extensions are loading into the active Python interpreter and may run arbitrary code. [default: none]
.IP "--extension-pkg-whitelist=<pkg[,pkg]>"
A comma-separated list of package or module names from where C extensions may be loaded. Extensions are loading into the active Python interpreter and may run arbitrary code. (This is an alternative name to extension-pkg-allow-list for backward compatibility.) [default: none]
.IP "--suggestion-mode=<yn>"
When enabled, pylint would attempt to guess common misconfiguration and emit user-friendly hints instead of false-positive error messages. [default: yes]
.IP "--exit-zero"
Expand Down
2 changes: 1 addition & 1 deletion man/pyreverse.1
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ show help message and exit
.IP "-p<name>, --project=<name>"
set project name to <name> if not using \-c option. (default:'No Name')
.IP "--ignore=<file[,file...]>"
add files or directories to the blacklist. They should be base names, not
files or directories to be skipped. They should be base names, not
paths. [current: CVS]
.IP "-f<mode>, --filter-mode=<mode>"
filter attributes and functions according to <mode>. You can combine
Expand Down
37 changes: 21 additions & 16 deletions pylint/checkers/base.py
Original file line number Diff line number Diff line change
Expand Up @@ -1421,7 +1421,7 @@ def _check_misplaced_format_function(self, call_node):
"eval-used", "exec-used", "bad-reversed-sequence", "misplaced-format-function"
)
def visit_call(self, node):
"""visit a Call node -> check if this is not a blacklisted builtin
"""visit a Call node -> check if this is not a disallowed builtin
call and check for * or ** use
"""
self._check_misplaced_format_function(node)
Expand Down Expand Up @@ -1724,17 +1724,22 @@ def _create_naming_options():

class NameChecker(_BasicChecker):
msgs = {
"C0102": (
'Black listed name "%s"',
"blacklisted-name",
"Used when the name is listed in the black list (unauthorized names).",
),
"C0103": (
'%s name "%s" doesn\'t conform to %s',
"invalid-name",
"Used when the name doesn't conform to naming rules "
"associated to its type (constant, variable, class...).",
),
"C0104": (
'Disallowed name "%s"',
"disallowed-name",
"Used when the name matches bad-names or bad-names-rgxs- (unauthorized names).",
{
"old_names": [
("C0102", "blacklisted-name"),
]
},
),
"C0144": (
'%s name "%s" contains a non-ASCII unicode character',
"non-ascii-name",
Expand Down Expand Up @@ -1888,7 +1893,7 @@ def _create_naming_rules(self):

return regexps, hints

@utils.check_messages("blacklisted-name", "invalid-name", "non-ascii-name")
@utils.check_messages("disallowed-name", "invalid-name", "non-ascii-name")
def visit_module(self, node):
self._check_name("module", node.name.split(".")[-1], node)
self._bad_names = {}
Expand All @@ -1914,7 +1919,7 @@ def leave_module(self, node): # pylint: disable=unused-argument
self._raise_name_warning(*args)

@utils.check_messages(
"blacklisted-name", "invalid-name", "assign-to-new-keyword", "non-ascii-name"
"disallowed-name", "invalid-name", "assign-to-new-keyword", "non-ascii-name"
)
def visit_classdef(self, node):
self._check_assign_to_new_keyword_violation(node.name, node)
Expand All @@ -1924,7 +1929,7 @@ def visit_classdef(self, node):
self._check_name("attr", attr, anodes[0])

@utils.check_messages(
"blacklisted-name", "invalid-name", "assign-to-new-keyword", "non-ascii-name"
"disallowed-name", "invalid-name", "assign-to-new-keyword", "non-ascii-name"
)
def visit_functiondef(self, node):
# Do not emit any warnings if the method is just an implementation
Expand Down Expand Up @@ -1953,13 +1958,13 @@ def visit_functiondef(self, node):

visit_asyncfunctiondef = visit_functiondef

@utils.check_messages("blacklisted-name", "invalid-name", "non-ascii-name")
@utils.check_messages("disallowed-name", "invalid-name", "non-ascii-name")
def visit_global(self, node):
for name in node.names:
self._check_name("const", name, node)

@utils.check_messages(
"blacklisted-name", "invalid-name", "assign-to-new-keyword", "non-ascii-name"
"disallowed-name", "invalid-name", "assign-to-new-keyword", "non-ascii-name"
)
def visit_assignname(self, node):
"""check module level assigned names"""
Expand Down Expand Up @@ -2025,12 +2030,12 @@ def _raise_name_warning(
self.add_message(warning, node=node, args=args, confidence=confidence)
self.stats["badname_" + node_type] += 1

def _name_valid_due_to_whitelist(self, name: str) -> bool:
def _name_allowed_by_regex(self, name: str) -> bool:
return name in self.config.good_names or any(
pattern.match(name) for pattern in self._good_names_rgxs_compiled
)

def _name_invalid_due_to_blacklist(self, name: str) -> bool:
def _name_disallowed_by_regex(self, name: str) -> bool:
return name in self.config.bad_names or any(
pattern.match(name) for pattern in self._bad_names_rgxs_compiled
)
Expand All @@ -2056,11 +2061,11 @@ def _should_exempt_from_invalid_name(node):
clobbering, _ = utils.clobber_in_except(node)
if clobbering:
return
if self._name_valid_due_to_whitelist(name=name):
if self._name_allowed_by_regex(name=name):
return
if self._name_invalid_due_to_blacklist(name=name):
if self._name_disallowed_by_regex(name=name):
self.stats["badname_" + node_type] += 1
self.add_message("blacklisted-name", node=node, args=name)
self.add_message("disallowed-name", node=node, args=name)
return
regexp = self._name_regexps[node_type]
match = regexp.match(name)
Expand Down
2 changes: 1 addition & 1 deletion pylint/checkers/imports.py
Original file line number Diff line number Diff line change
Expand Up @@ -968,7 +968,7 @@ def _check_toplevel(self, node):
for name in node.names
]

# Get the full names of all the imports that are not whitelisted.
# Get the full names of all the imports that are only allowed at the module level
scoped_imports = [
name for name in module_names if name not in self._allow_any_import_level
]
Expand Down
4 changes: 2 additions & 2 deletions pylint/checkers/python3.py
Original file line number Diff line number Diff line change
Expand Up @@ -72,8 +72,8 @@ def _inferred_value_is_dict(value):
return isinstance(value, astroid.Instance) and "dict" in value.basenames


def _infer_if_relevant_attr(node, whitelist):
return node.expr.infer() if node.attrname in whitelist else []
def _infer_if_relevant_attr(node, relevant_attrs):
return node.expr.infer() if node.attrname in relevant_attrs else []


def _is_builtin(node):
Expand Down
2 changes: 1 addition & 1 deletion pylint/checkers/typecheck.py
Original file line number Diff line number Diff line change
Expand Up @@ -260,7 +260,7 @@ def _missing_member_hint(owner, attrname, distance_threshold, max_choices):
),
"I1101": (
"%s %r has no %r member%s, but source is unavailable. Consider "
"adding this module to extension-pkg-whitelist if you want "
"adding this module to extension-pkg-allow-list if you want "
"to perform analysis based on run-time introspection of living objects.",
"c-extension-no-member",
"Used when a variable is accessed for non-existent member of C "
Expand Down
4 changes: 2 additions & 2 deletions pylint/extensions/bad_builtin.py
Original file line number Diff line number Diff line change
Expand Up @@ -27,8 +27,8 @@ class BadBuiltinChecker(BaseChecker):
"W0141": (
"Used builtin function %s",
"bad-builtin",
"Used when a black listed builtin function is used (see the "
"bad-function option). Usual black listed functions are the ones "
"Used when a disallowed builtin function is used (see the "
"bad-function option). Usual disallowed functions are the ones "
"like map, or filter , where Python offers now some cleaner "
"alternative like list comprehension.",
)
Expand Down
29 changes: 24 additions & 5 deletions pylint/lint/pylinter.py
Original file line number Diff line number Diff line change
Expand Up @@ -164,7 +164,7 @@ def make_options():
"metavar": "<file>[,<file>...]",
"dest": "black_list",
"default": ("CVS",),
"help": "Add files or directories to the blacklist. "
"help": "Files or directories to be skipped. "
"They should be base names, not paths.",
},
),
Expand All @@ -175,8 +175,8 @@ def make_options():
"metavar": "<pattern>[,<pattern>...]",
"dest": "black_list_re",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't this be changed too ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Renaming the dest names (black_list, black_list_re, extension_pkg_whitelist etc.) could be breaking for plugins and other programmatic access since these are the member names in the config object. doc/how_tos/plugins.rst even sets black_list in the example.

It might be possible to clean this up in a way that's still backwards compatible -- I'll need to study optparse further since I'm not familiar with it. If optparse.Values doesn't provide a built-in way to alias the names, we could probably extend optparse.Values and add some @property annotations or __getattr__/__setattr__ to provide compatibility. Or as you've noted previously, maybe it's time for bigger rework of the config system. But because it won't make a difference to most users, this seems like something to discuss and address in a later issue/PR.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, lack of sleep, you're right, we should not touch it (not before 3.0 at least).

"default": (),
"help": "Add files or directories matching the regex patterns to the"
" blacklist. The regex matches against base names, not paths.",
"help": "Files or directories matching the regex patterns are"
" skipped. The regex matches against base names, not paths.",
},
),
(
Expand Down Expand Up @@ -367,7 +367,7 @@ def make_options():
},
),
(
"extension-pkg-whitelist",
"extension-pkg-allow-list",
{
"type": "csv",
"metavar": "<pkg[,pkg]>",
Expand All @@ -380,6 +380,21 @@ def make_options():
),
},
),
(
"extension-pkg-whitelist",
{
"type": "csv",
"metavar": "<pkg[,pkg]>",
"default": [],
"help": (
"A comma-separated list of package or module names"
" from where C extensions may be loaded. Extensions are"
" loading into the active Python interpreter and may run"
" arbitrary code. (This is an alternative name to"
" extension-pkg-allow-list for backward compatibility.)"
),
},
),
(
"suggestion-mode",
{
Expand Down Expand Up @@ -1106,7 +1121,11 @@ def open(self):
self.stats = {"by_module": {}, "by_msg": {}}
MANAGER.always_load_extensions = self.config.unsafe_load_any_extension
MANAGER.max_inferable_values = self.config.limit_inference_results
MANAGER.extension_package_whitelist.update(self.config.extension_pkg_whitelist)
MANAGER.extension_package_whitelist.update(self.config.extension_pkg_allow_list)
if self.config.extension_pkg_whitelist:
MANAGER.extension_package_whitelist.update(
self.config.extension_pkg_whitelist
)
for msg_cat in MSG_TYPES.values():
self.stats[msg_cat] = 0

Expand Down
6 changes: 3 additions & 3 deletions pylint/pyreverse/main.py
Original file line number Diff line number Diff line change
Expand Up @@ -141,9 +141,9 @@
{
"type": "csv",
"metavar": "<file[,file...]>",
"dest": "black_list",
"dest": "ignore_list",
"default": ("CVS",),
"help": "Add files or directories to the blacklist. They "
"help": "Files or directories to be skipped. They "
"should be base names, not paths.",
},
),
Expand Down Expand Up @@ -210,7 +210,7 @@ def run(self, args):
project = project_from_files(
args,
project_name=self.config.project,
black_list=self.config.black_list,
black_list=self.config.ignore_list,
)
linker = Linker(project, tag=True)
handler = DiadefsHandler(self.config)
Expand Down
4 changes: 2 additions & 2 deletions pylint/utils/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@
from pylint.utils.utils import (
HAS_ISORT_5,
IsortDriver,
_basename_in_blacklist_re,
_basename_in_ignore_list_re,
_check_csv,
_format_option_value,
_splitstrip,
Expand All @@ -69,7 +69,7 @@
"ASTWalker",
"HAS_ISORT_5",
"IsortDriver",
"_basename_in_blacklist_re",
"_basename_in_ignore_list_re",
"_check_csv",
"_format_option_value",
"_splitstrip",
Expand Down
Loading