diff -Nru pyflakes-2.1.1/debian/changelog pyflakes-2.2.0/debian/changelog --- pyflakes-2.1.1/debian/changelog 2020-02-19 08:56:58.000000000 -0800 +++ pyflakes-2.2.0/debian/changelog 2020-07-02 08:38:02.000000000 -0700 @@ -1,3 +1,25 @@ +pyflakes (2.2.0-1~20.04.2) focal; urgency=medium + + * Backport to focal. (LP: #1883175) + * Drop debhelper compat level to 12. + + -- Ted Kern Thu, 02 Jul 2020 08:38:02 -0700 + +pyflakes (2.2.0-1) unstable; urgency=medium + + * Team upload. + * New upstream release. + * Set Rules-Requires-Root: no. + * Bump debhelper compat level to 13. + + -- Ondřej Nový Fri, 12 Jun 2020 12:09:55 +0200 + +pyflakes (2.1.1-3) unstable; urgency=medium + + * Drop python2 support; Closes: #937439 + + -- Sandro Tosi Thu, 16 Apr 2020 18:39:15 -0400 + pyflakes (2.1.1-2) unstable; urgency=medium * Team upload. diff -Nru pyflakes-2.1.1/debian/control pyflakes-2.2.0/debian/control --- pyflakes-2.1.1/debian/control 2020-02-19 08:56:58.000000000 -0800 +++ pyflakes-2.2.0/debian/control 2020-07-02 08:38:02.000000000 -0700 @@ -6,9 +6,6 @@ Barry Warsaw Build-Depends: debhelper-compat (= 12), dh-python, - python-all, - python-nose, - python-setuptools, python3-all, python3-nose, python3-setuptools @@ -17,23 +14,7 @@ Testsuite: autopkgtest-pkg-python Vcs-Git: https://salsa.debian.org/python-team/applications/pyflakes.git Vcs-Browser: https://salsa.debian.org/python-team/applications/pyflakes - -Package: pyflakes -Architecture: all -Multi-Arch: foreign -Depends: python-pyflakes (=${binary:Version}), - ${misc:Depends}, - ${python:Depends}, -Recommends: pyflakes3 -Description: passive checker of Python 2 programs - Pyflakes is program to analyze Python programs and detect various - errors. It works by parsing the source file, not importing it, so it - is safe to use on modules with side effects. It's also much faster. - . - Unlike PyLint, Pyflakes checks only for logical errors in programs; - it does not perform any checks on style. - . - This is the Python 2 command line `pyflakes` program. +Rules-Requires-Root: no Package: pyflakes3 Architecture: all @@ -43,7 +24,6 @@ ${python3:Depends}, Breaks: pyflakes (<< 1.1.0-1) Replaces: pyflakes (<< 1.1.0-1) -Recommends: pyflakes Description: passive checker of Python 3 programs Pyflakes is program to analyze Python programs and detect various errors. It works by parsing the source file, not importing it, so it @@ -54,23 +34,6 @@ . This is the Python 3 command line `pyflakes3` program. -Package: python-pyflakes -Architecture: all -Replaces: pyflakes (<< 1.0.0-2~) -Breaks: pyflakes (<< 1.0.0-2~), - python-flake8 (<< 3.5.0), -Recommends: pyflakes -Depends: python-pkg-resources, ${misc:Depends}, ${python:Depends} -Description: passive checker of Python 2 programs - Python modules - Pyflakes is program to analyze Python programs and detect various - errors. It works by parsing the source file, not importing it, so it - is safe to use on modules with side effects. It's also much faster. - . - Unlike PyLint, Pyflakes checks only for logical errors in programs; - it does not perform any checks on style. - . - This is the Python 2 library version. - Package: python3-pyflakes Architecture: all Replaces: pyflakes (<< 1.0.0-2~) diff -Nru pyflakes-2.1.1/debian/pyflakes.1 pyflakes-2.2.0/debian/pyflakes.1 --- pyflakes-2.1.1/debian/pyflakes.1 2020-02-19 08:51:13.000000000 -0800 +++ pyflakes-2.2.0/debian/pyflakes.1 1969-12-31 16:00:00.000000000 -0800 @@ -1,33 +0,0 @@ -.TH "PYFLAKES" "1" "10/01/2007" "" "" -.\" disable hyphenation -.nh -.\" disable justification (adjust text to left margin only) -.ad l -.SH "NAME" -pyflakes - simple Python 2 source checker -.SH "SYNOPSIS" -.PP -\fBpyflakes\fR [\fIfile-or-directory\fR ...] -.SH "DESCRIPTION" -pyflakes is a simple program which checks Python source files for errors. -It is similar to PyChecker in scope, but differs in that it does not -execute the modules to check them. This is both safer and faster, although -it does not perform as many checks. Unlike PyLint, Pyflakes checks only -for logical errors in programs; it does not perform any checks on style. -.sp -All commandline arguments are checked, which have to be either regular files -or directories. If a directory is given, every \fB.py\fR file within -will be checked. -.sp -When no commandline arguments are given, data will be read from standard input. -.SH "OUTPUT" -Found warnings including unused imports, undefined variables -and unnecessary reimports, are printed on standard output. -Found errors including compile or encoding errors, are printed -on standard error. -.SH "EXIT STATUS" -The exit status is 0 when no warnings or errors are found. Otherwise the exit -status is 1. -.SH "AUTHOR" -This manual page was originally written by Bastian Kleineidam -for the Debian distribution of pyflakes (but can be used by others)\. diff -Nru pyflakes-2.1.1/debian/pyflakes.manpages pyflakes-2.2.0/debian/pyflakes.manpages --- pyflakes-2.1.1/debian/pyflakes.manpages 2020-02-19 08:51:13.000000000 -0800 +++ pyflakes-2.2.0/debian/pyflakes.manpages 1969-12-31 16:00:00.000000000 -0800 @@ -1 +0,0 @@ -debian/pyflakes.1 diff -Nru pyflakes-2.1.1/debian/rules pyflakes-2.2.0/debian/rules --- pyflakes-2.1.1/debian/rules 2020-02-19 08:51:13.000000000 -0800 +++ pyflakes-2.2.0/debian/rules 2020-07-02 08:38:02.000000000 -0700 @@ -12,16 +12,13 @@ %: - dh $@ --with python2,python3 --buildsystem=pybuild + dh $@ --with python3 --buildsystem=pybuild override_dh_installchangelogs: dh_installchangelogs NEWS.rst override_dh_auto_install: dh_auto_install - mkdir -p debian/pyflakes/usr/bin mkdir -p debian/pyflakes3/usr/bin - mv debian/python-pyflakes/usr/bin/pyflakes debian/pyflakes/usr/bin/pyflakes mv debian/python3-pyflakes/usr/bin/pyflakes debian/pyflakes3/usr/bin/pyflakes3 - sed -i '1s/3.*/2.7/' ./debian/pyflakes/usr/bin/pyflakes sed -i '1s/3.*/3/' ./debian/pyflakes3/usr/bin/pyflakes3 diff -Nru pyflakes-2.1.1/debian/tests/control pyflakes-2.2.0/debian/tests/control --- pyflakes-2.1.1/debian/tests/control 2020-02-19 08:51:13.000000000 -0800 +++ pyflakes-2.2.0/debian/tests/control 2020-07-02 08:38:02.000000000 -0700 @@ -1,8 +1,5 @@ -Test-Command: pyflakes --help -Depends: pyflakes - Test-Command: pyflakes3 --help Depends: pyflakes3 Tests: relative-import.sh -Depends: python-pyflakes, python3-pyflakes +Depends: python3-pyflakes diff -Nru pyflakes-2.1.1/debian/tests/relative-import.sh pyflakes-2.2.0/debian/tests/relative-import.sh --- pyflakes-2.1.1/debian/tests/relative-import.sh 2020-02-19 08:52:01.000000000 -0800 +++ pyflakes-2.2.0/debian/tests/relative-import.sh 2020-07-02 08:38:02.000000000 -0700 @@ -8,5 +8,4 @@ foo() EOF -python2 -m pyflakes relative.py python3 -m pyflakes relative.py diff -Nru pyflakes-2.1.1/NEWS.rst pyflakes-2.2.0/NEWS.rst --- pyflakes-2.1.1/NEWS.rst 2019-02-28 11:18:26.000000000 -0800 +++ pyflakes-2.2.0/NEWS.rst 2020-04-09 20:49:52.000000000 -0700 @@ -1,4 +1,32 @@ +2.2.0 (2020-04-08) + +- Include column information in error messages +- Fix ``@overload`` detection with other decorators and in non-global scopes +- Fix return-type annotation being a class member +- Fix assignment to ``_`` in doctests with existing ``_`` name +- Namespace attributes which are attached to ast nodes with ``_pyflakes_`` to + avoid conflicts with other libraries (notably bandit) +- Add check for f-strings without placeholders +- Add check for unused/extra/invalid ``'string literal'.format(...)`` +- Add check for unused/extra/invalid ``'string literal % ...`` +- Improve python shebang detection +- Allow type ignore to be followed by a code ``# type: ignore[attr-defined]`` +- Add support for assignment expressions (PEP 572) +- Support ``@overload`` detection from ``typing_extensions`` as well +- Fix ``@overload`` detection for async functions +- Allow ``continue`` inside ``finally`` in python 3.8+ +- Fix handling of annotations in positional-only arguments +- Make pyflakes more resistant to future syntax additions +- Fix false positives in partially quoted type annotations +- Warn about ``is`` comparison to tuples +- Fix ``Checker`` usage with async function subtrees +- Add check for ``if`` of non-empty tuple +- Switch from ``optparse`` to ``argparse`` +- Fix false positives in partially quoted type annotations in unusual contexts +- Be more cautious when identifying ``Literal`` type expressions + 2.1.1 (2019-02-28) + - Fix reported line number for type comment errors - Fix typing.overload check to only check imported names diff -Nru pyflakes-2.1.1/PKG-INFO pyflakes-2.2.0/PKG-INFO --- pyflakes-2.1.1/PKG-INFO 2019-02-28 11:25:22.000000000 -0800 +++ pyflakes-2.2.0/PKG-INFO 2020-04-09 20:51:46.000000000 -0700 @@ -1,6 +1,6 @@ Metadata-Version: 1.2 Name: pyflakes -Version: 2.1.1 +Version: 2.2.0 Summary: passive checker of Python programs Home-page: https://github.com/PyCQA/pyflakes Author: A lot of people @@ -94,7 +94,7 @@ Changelog --------- - Please see `NEWS.rst `_. + Please see `NEWS.rst `_. Platform: UNKNOWN Classifier: Development Status :: 6 - Mature diff -Nru pyflakes-2.1.1/pyflakes/api.py pyflakes-2.2.0/pyflakes/api.py --- pyflakes-2.1.1/pyflakes/api.py 2019-02-28 11:10:37.000000000 -0800 +++ pyflakes-2.2.0/pyflakes/api.py 2020-04-09 20:48:16.000000000 -0700 @@ -14,7 +14,7 @@ __all__ = ['check', 'checkPath', 'checkRecursive', 'iterSourceCode', 'main'] -PYTHON_SHEBANG_REGEX = re.compile(br'^#!.*\bpython[23w]?\b\s*$') +PYTHON_SHEBANG_REGEX = re.compile(br'^#!.*\bpython([23](\.\d+)?|w)?[dmu]?\s') def check(codeString, filename, reporter=None): @@ -118,8 +118,7 @@ except IOError: return False - first_line = text.splitlines()[0] - return PYTHON_SHEBANG_REGEX.match(first_line) + return PYTHON_SHEBANG_REGEX.match(text) def iterSourceCode(paths): @@ -194,14 +193,18 @@ def main(prog=None, args=None): """Entry point for the script "pyflakes".""" - import optparse + import argparse # Handle "Keyboard Interrupt" and "Broken pipe" gracefully _exitOnSignal('SIGINT', '... stopped') _exitOnSignal('SIGPIPE', 1) - parser = optparse.OptionParser(prog=prog, version=_get_version()) - (__, args) = parser.parse_args(args=args) + parser = argparse.ArgumentParser(prog=prog, + description='Check Python source files for errors') + parser.add_argument('-V', '--version', action='version', version=_get_version()) + parser.add_argument('path', nargs='*', + help='Path(s) of Python file(s) to check. STDIN if not given.') + args = parser.parse_args(args=args).path reporter = modReporter._makeDefaultReporter() if args: warnings = checkRecursive(args, reporter) diff -Nru pyflakes-2.1.1/pyflakes/checker.py pyflakes-2.2.0/pyflakes/checker.py --- pyflakes-2.1.1/pyflakes/checker.py 2019-02-28 11:10:37.000000000 -0800 +++ pyflakes-2.2.0/pyflakes/checker.py 2020-04-09 20:48:16.000000000 -0700 @@ -8,10 +8,12 @@ import ast import bisect import collections +import contextlib import doctest import functools import os import re +import string import sys import tokenize @@ -29,6 +31,8 @@ builtin_vars = dir(__import__('__builtin__' if PY2 else 'builtins')) +parse_format_string = string.Formatter().parse + if PY2: tokenize_tokenize = tokenize.generate_tokens else: @@ -69,18 +73,148 @@ if PY35_PLUS: FOR_TYPES = (ast.For, ast.AsyncFor) LOOP_TYPES = (ast.While, ast.For, ast.AsyncFor) + FUNCTION_TYPES = (ast.FunctionDef, ast.AsyncFunctionDef) else: FOR_TYPES = (ast.For,) LOOP_TYPES = (ast.While, ast.For) + FUNCTION_TYPES = (ast.FunctionDef,) + + +if PY38_PLUS: + def _is_singleton(node): # type: (ast.AST) -> bool + return ( + isinstance(node, ast.Constant) and + isinstance(node.value, (bool, type(Ellipsis), type(None))) + ) +elif not PY2: + def _is_singleton(node): # type: (ast.AST) -> bool + return isinstance(node, (ast.NameConstant, ast.Ellipsis)) +else: + def _is_singleton(node): # type: (ast.AST) -> bool + return ( + isinstance(node, ast.Name) and + node.id in {'True', 'False', 'Ellipsis', 'None'} + ) + + +def _is_tuple_constant(node): # type: (ast.AST) -> bool + return ( + isinstance(node, ast.Tuple) and + all(_is_constant(elt) for elt in node.elts) + ) + -# https://github.com/python/typed_ast/blob/55420396/ast27/Parser/tokenizer.c#L102-L104 +if PY38_PLUS: + def _is_constant(node): + return isinstance(node, ast.Constant) or _is_tuple_constant(node) +else: + _const_tps = (ast.Str, ast.Num) + if not PY2: + _const_tps += (ast.Bytes,) + + def _is_constant(node): + return ( + isinstance(node, _const_tps) or + _is_singleton(node) or + _is_tuple_constant(node) + ) + + +def _is_const_non_singleton(node): # type: (ast.AST) -> bool + return _is_constant(node) and not _is_singleton(node) + + +# https://github.com/python/typed_ast/blob/1.4.0/ast27/Parser/tokenizer.c#L102-L104 TYPE_COMMENT_RE = re.compile(r'^#\s*type:\s*') -# https://github.com/python/typed_ast/blob/55420396/ast27/Parser/tokenizer.c#L1400 -TYPE_IGNORE_RE = re.compile(TYPE_COMMENT_RE.pattern + r'ignore\s*(#|$)') -# https://github.com/python/typed_ast/blob/55420396/ast27/Grammar/Grammar#L147 +# https://github.com/python/typed_ast/blob/1.4.0/ast27/Parser/tokenizer.c#L1408-L1413 +ASCII_NON_ALNUM = ''.join([chr(i) for i in range(128) if not chr(i).isalnum()]) +TYPE_IGNORE_RE = re.compile( + TYPE_COMMENT_RE.pattern + r'ignore([{}]|$)'.format(ASCII_NON_ALNUM)) +# https://github.com/python/typed_ast/blob/1.4.0/ast27/Grammar/Grammar#L147 TYPE_FUNC_RE = re.compile(r'^(\(.*?\))\s*->\s*(.*)$') +MAPPING_KEY_RE = re.compile(r'\(([^()]*)\)') +CONVERSION_FLAG_RE = re.compile('[#0+ -]*') +WIDTH_RE = re.compile(r'(?:\*|\d*)') +PRECISION_RE = re.compile(r'(?:\.(?:\*|\d*))?') +LENGTH_RE = re.compile('[hlL]?') +# https://docs.python.org/3/library/stdtypes.html#old-string-formatting +VALID_CONVERSIONS = frozenset('diouxXeEfFgGcrsa%') + + +def _must_match(regex, string, pos): + # type: (Pattern[str], str, int) -> Match[str] + match = regex.match(string, pos) + assert match is not None + return match + + +def parse_percent_format(s): # type: (str) -> Tuple[PercentFormat, ...] + """Parses the string component of a `'...' % ...` format call + + Copied from https://github.com/asottile/pyupgrade at v1.20.1 + """ + + def _parse_inner(): + # type: () -> Generator[PercentFormat, None, None] + string_start = 0 + string_end = 0 + in_fmt = False + + i = 0 + while i < len(s): + if not in_fmt: + try: + i = s.index('%', i) + except ValueError: # no more % fields! + yield s[string_start:], None + return + else: + string_end = i + i += 1 + in_fmt = True + else: + key_match = MAPPING_KEY_RE.match(s, i) + if key_match: + key = key_match.group(1) # type: Optional[str] + i = key_match.end() + else: + key = None + + conversion_flag_match = _must_match(CONVERSION_FLAG_RE, s, i) + conversion_flag = conversion_flag_match.group() or None + i = conversion_flag_match.end() + + width_match = _must_match(WIDTH_RE, s, i) + width = width_match.group() or None + i = width_match.end() + + precision_match = _must_match(PRECISION_RE, s, i) + precision = precision_match.group() or None + i = precision_match.end() + + # length modifier is ignored + i = _must_match(LENGTH_RE, s, i).end() + + try: + conversion = s[i] + except IndexError: + raise ValueError('end-of-string while parsing format') + i += 1 + + fmt = (key, conversion_flag, width, precision, conversion) + yield s[string_start:string_end], fmt + + in_fmt = False + string_start = i + + if in_fmt: + raise ValueError('end-of-string while parsing format') + + return tuple(_parse_inner()) + + class _FieldsOrder(dict): """Fix order of AST node fields.""" @@ -530,29 +664,82 @@ return node.name -def is_typing_overload(value, scope): - def is_typing_overload_decorator(node): - return ( - ( - isinstance(node, ast.Name) and - node.id in scope and - isinstance(scope[node.id], ImportationFrom) and - scope[node.id].fullName == 'typing.overload' - ) or ( - isinstance(node, ast.Attribute) and - isinstance(node.value, ast.Name) and - node.value.id == 'typing' and - node.attr == 'overload' - ) +TYPING_MODULES = frozenset(('typing', 'typing_extensions')) + + +def _is_typing_helper(node, is_name_match_fn, scope_stack): + """ + Internal helper to determine whether or not something is a member of a + typing module. This is used as part of working out whether we are within a + type annotation context. + + Note: you probably don't want to use this function directly. Instead see the + utils below which wrap it (`_is_typing` and `_is_any_typing_member`). + """ + + def _bare_name_is_attr(name): + for scope in reversed(scope_stack): + if name in scope: + return ( + isinstance(scope[name], ImportationFrom) and + scope[name].module in TYPING_MODULES and + is_name_match_fn(scope[name].real_name) + ) + + return False + + return ( + ( + isinstance(node, ast.Name) and + _bare_name_is_attr(node.id) + ) or ( + isinstance(node, ast.Attribute) and + isinstance(node.value, ast.Name) and + node.value.id in TYPING_MODULES and + is_name_match_fn(node.attr) ) + ) + + +def _is_typing(node, typing_attr, scope_stack): + """ + Determine whether `node` represents the member of a typing module specified + by `typing_attr`. + + This is used as part of working out whether we are within a type annotation + context. + """ + return _is_typing_helper(node, lambda x: x == typing_attr, scope_stack) + + +def _is_any_typing_member(node, scope_stack): + """ + Determine whether `node` represents any member of a typing module. + + This is used as part of working out whether we are within a type annotation + context. + """ + return _is_typing_helper(node, lambda x: True, scope_stack) + +def is_typing_overload(value, scope_stack): return ( - isinstance(value.source, ast.FunctionDef) and - len(value.source.decorator_list) == 1 and - is_typing_overload_decorator(value.source.decorator_list[0]) + isinstance(value.source, FUNCTION_TYPES) and + any( + _is_typing(dec, 'overload', scope_stack) + for dec in value.source.decorator_list + ) ) +def in_annotation(func): + @functools.wraps(func) + def in_annotation_func(self, *args, **kwargs): + with self._enter_annotation(): + return func(self, *args, **kwargs) + return in_annotation_func + + def make_tokens(code): # PY3: tokenize.tokenize requires readline of bytes if not isinstance(code, bytes): @@ -634,11 +821,14 @@ ast.DictComp: GeneratorScope, } if PY35_PLUS: - _ast_node_scope[ast.AsyncFunctionDef] = FunctionScope, + _ast_node_scope[ast.AsyncFunctionDef] = FunctionScope nodeDepth = 0 offset = None traceTree = False + _in_annotation = False + _in_typing_literal = False + _in_deferred = False builtIns = set(builtin_vars).union(_MAGIC_GLOBALS) _customBuiltIns = os.environ.get('PYFLAKES_BUILTINS') @@ -670,6 +860,7 @@ for builtin in self.builtIns: self.addBinding(None, Builtin(builtin)) self.handleChildren(tree) + self._in_deferred = True self.runDeferred(self._deferredFunctions) # Set _deferredFunctions to None so that deferFunction will fail # noisily if called after we've run through the deferred functions. @@ -813,22 +1004,31 @@ def getParent(self, node): # Lookup the first parent which is not Tuple, List or Starred while True: - node = node.parent + node = node._pyflakes_parent if not hasattr(node, 'elts') and not hasattr(node, 'ctx'): return node def getCommonAncestor(self, lnode, rnode, stop): - if stop in (lnode, rnode) or not (hasattr(lnode, 'parent') and - hasattr(rnode, 'parent')): + if ( + stop in (lnode, rnode) or + not ( + hasattr(lnode, '_pyflakes_parent') and + hasattr(rnode, '_pyflakes_parent') + ) + ): return None if lnode is rnode: return lnode - if (lnode.depth > rnode.depth): - return self.getCommonAncestor(lnode.parent, rnode, stop) - if (lnode.depth < rnode.depth): - return self.getCommonAncestor(lnode, rnode.parent, stop) - return self.getCommonAncestor(lnode.parent, rnode.parent, stop) + if (lnode._pyflakes_depth > rnode._pyflakes_depth): + return self.getCommonAncestor(lnode._pyflakes_parent, rnode, stop) + if (lnode._pyflakes_depth < rnode._pyflakes_depth): + return self.getCommonAncestor(lnode, rnode._pyflakes_parent, stop) + return self.getCommonAncestor( + lnode._pyflakes_parent, + rnode._pyflakes_parent, + stop, + ) def descendantOf(self, node, ancestors, stop): for a in ancestors: @@ -866,7 +1066,7 @@ - `node` is the statement responsible for the change - `value` is the new value, a Binding instance """ - # assert value.source in (node, node.parent): + # assert value.source in (node, node._pyflakes_parent): for scope in self.scopeStack[::-1]: if value.name in scope: break @@ -888,7 +1088,7 @@ node, value.name, existing.source) elif not existing.used and value.redefines(existing): if value.name != '_' or isinstance(existing, Importation): - if not is_typing_overload(existing, self.scope): + if not is_typing_overload(existing, self.scopeStack): self.report(messages.RedefinedWhileUnused, node, value.name, existing.source) @@ -901,12 +1101,30 @@ self.scope[value.name] = value + def _unknown_handler(self, node): + # this environment variable configures whether to error on unknown + # ast types. + # + # this is silent by default but the error is enabled for the pyflakes + # testsuite. + # + # this allows new syntax to be added to python without *requiring* + # changes from the pyflakes side. but will still produce an error + # in the pyflakes testsuite (so more specific handling can be added if + # needed). + if os.environ.get('PYFLAKES_ERROR_UNKNOWN'): + raise NotImplementedError('Unexpected type: {}'.format(type(node))) + else: + self.handleChildren(node) + def getNodeHandler(self, node_class): try: return self._nodeHandlers[node_class] except KeyError: nodeType = getNodeType(node_class) - self._nodeHandlers[node_class] = handler = getattr(self, nodeType) + self._nodeHandlers[node_class] = handler = getattr( + self, nodeType, self._unknown_handler, + ) return handler def handleNodeLoad(self, node): @@ -1005,12 +1223,12 @@ parent_stmt = self.getParent(node) if isinstance(parent_stmt, (FOR_TYPES, ast.comprehension)) or ( - parent_stmt != node.parent and + parent_stmt != node._pyflakes_parent and not self.isLiteralTupleUnpacking(parent_stmt)): binding = Binding(name, node) elif name == '__all__' and isinstance(self.scope, ModuleScope): - binding = ExportBinding(name, node.parent, self.scope) - elif isinstance(getattr(node, 'ctx', None), ast.Param): + binding = ExportBinding(name, node._pyflakes_parent, self.scope) + elif PY2 and isinstance(getattr(node, 'ctx', None), ast.Param): binding = Argument(name, self.getScopeNode(node)) else: binding = Assignment(name, node) @@ -1022,11 +1240,11 @@ """ Return `True` if node is part of a conditional body. """ - current = getattr(node, 'parent', None) + current = getattr(node, '_pyflakes_parent', None) while current: if isinstance(current, (ast.If, ast.While, ast.IfExp)): return True - current = getattr(current, 'parent', None) + current = getattr(current, '_pyflakes_parent', None) return False name = getNodeName(node) @@ -1046,6 +1264,14 @@ except KeyError: self.report(messages.UndefinedName, node, name) + @contextlib.contextmanager + def _enter_annotation(self): + orig, self._in_annotation = self._in_annotation, True + try: + yield + finally: + self._in_annotation = orig + def _handle_type_comments(self, node): for (lineno, col_offset), comment in self._type_comments.get(node, ()): comment = comment.split(':', 1)[1].strip() @@ -1113,8 +1339,8 @@ self.isDocstring(node)): self.futuresAllowed = False self.nodeDepth += 1 - node.depth = self.nodeDepth - node.parent = parent + node._pyflakes_depth = self.nodeDepth + node._pyflakes_parent = parent try: handler = self.getNodeHandler(node.__class__) handler(node) @@ -1153,7 +1379,8 @@ self.scopeStack = [self.scopeStack[0]] node_offset = self.offset or (0, 0) self.pushScope(DoctestScope) - self.addBinding(None, Builtin('_')) + if '_' not in self.scopeStack[0]: + self.addBinding(None, Builtin('_')) for example in examples: try: tree = ast.parse(example.source, "") @@ -1172,6 +1399,7 @@ self.popScope() self.scopeStack = saved_stack + @in_annotation def handleStringAnnotation(self, s, node, ref_lineno, ref_col_offset, err): try: tree = ast.parse(s) @@ -1195,6 +1423,7 @@ self.handleNode(parsed_annotation, node) + @in_annotation def handleAnnotation(self, annotation, node): if isinstance(annotation, ast.Str): # Defer handling forward annotation. @@ -1207,7 +1436,8 @@ messages.ForwardAnnotationSyntaxError, )) elif self.annotationsFutureEnabled: - self.deferFunction(lambda: self.handleNode(annotation, node)) + fn = in_annotation(Checker.handleNode) + self.deferFunction(lambda: fn(self, annotation, node)) else: self.handleNode(annotation, node) @@ -1215,18 +1445,309 @@ pass # "stmt" type nodes - DELETE = PRINT = FOR = ASYNCFOR = WHILE = IF = WITH = WITHITEM = \ + DELETE = PRINT = FOR = ASYNCFOR = WHILE = WITH = WITHITEM = \ ASYNCWITH = ASYNCWITHITEM = TRYFINALLY = EXEC = \ EXPR = ASSIGN = handleChildren PASS = ignore # "expr" type nodes - BOOLOP = BINOP = UNARYOP = IFEXP = SET = \ - CALL = REPR = ATTRIBUTE = SUBSCRIPT = \ - STARRED = NAMECONSTANT = handleChildren + BOOLOP = UNARYOP = SET = \ + REPR = ATTRIBUTE = \ + STARRED = NAMECONSTANT = NAMEDEXPR = handleChildren + + def SUBSCRIPT(self, node): + if ( + ( + isinstance(node.value, ast.Name) and + node.value.id == 'Literal' + ) or ( + isinstance(node.value, ast.Attribute) and + node.value.attr == 'Literal' + ) + ): + orig, self._in_typing_literal = self._in_typing_literal, True + try: + self.handleChildren(node) + finally: + self._in_typing_literal = orig + else: + if _is_any_typing_member(node.value, self.scopeStack): + with self._enter_annotation(): + self.handleChildren(node) + else: + self.handleChildren(node) + + def _handle_string_dot_format(self, node): + try: + placeholders = tuple(parse_format_string(node.func.value.s)) + except ValueError as e: + self.report(messages.StringDotFormatInvalidFormat, node, e) + return + + class state: # py2-compatible `nonlocal` + auto = None + next_auto = 0 + + placeholder_positional = set() + placeholder_named = set() + + def _add_key(fmtkey): + """Returns True if there is an error which should early-exit""" + if fmtkey is None: # end of string or `{` / `}` escapes + return False + + # attributes / indices are allowed in `.format(...)` + fmtkey, _, _ = fmtkey.partition('.') + fmtkey, _, _ = fmtkey.partition('[') + + try: + fmtkey = int(fmtkey) + except ValueError: + pass + else: # fmtkey was an integer + if state.auto is True: + self.report(messages.StringDotFormatMixingAutomatic, node) + return True + else: + state.auto = False + + if fmtkey == '': + if state.auto is False: + self.report(messages.StringDotFormatMixingAutomatic, node) + return True + else: + state.auto = True + + fmtkey = state.next_auto + state.next_auto += 1 - NUM = STR = BYTES = ELLIPSIS = CONSTANT = ignore + if isinstance(fmtkey, int): + placeholder_positional.add(fmtkey) + else: + placeholder_named.add(fmtkey) + + return False + + for _, fmtkey, spec, _ in placeholders: + if _add_key(fmtkey): + return + + # spec can also contain format specifiers + if spec is not None: + try: + spec_placeholders = tuple(parse_format_string(spec)) + except ValueError as e: + self.report(messages.StringDotFormatInvalidFormat, node, e) + return + + for _, spec_fmtkey, spec_spec, _ in spec_placeholders: + # can't recurse again + if spec_spec is not None and '{' in spec_spec: + self.report( + messages.StringDotFormatInvalidFormat, + node, + 'Max string recursion exceeded', + ) + return + if _add_key(spec_fmtkey): + return + + # bail early if there is *args or **kwargs + if ( + # python 2.x *args / **kwargs + getattr(node, 'starargs', None) or + getattr(node, 'kwargs', None) or + # python 3.x *args + any( + isinstance(arg, getattr(ast, 'Starred', ())) + for arg in node.args + ) or + # python 3.x **kwargs + any(kwd.arg is None for kwd in node.keywords) + ): + return + + substitution_positional = set(range(len(node.args))) + substitution_named = {kwd.arg for kwd in node.keywords} + + extra_positional = substitution_positional - placeholder_positional + extra_named = substitution_named - placeholder_named + + missing_arguments = ( + (placeholder_positional | placeholder_named) - + (substitution_positional | substitution_named) + ) + + if extra_positional: + self.report( + messages.StringDotFormatExtraPositionalArguments, + node, + ', '.join(sorted(str(x) for x in extra_positional)), + ) + if extra_named: + self.report( + messages.StringDotFormatExtraNamedArguments, + node, + ', '.join(sorted(extra_named)), + ) + if missing_arguments: + self.report( + messages.StringDotFormatMissingArgument, + node, + ', '.join(sorted(str(x) for x in missing_arguments)), + ) + + def CALL(self, node): + if ( + isinstance(node.func, ast.Attribute) and + isinstance(node.func.value, ast.Str) and + node.func.attr == 'format' + ): + self._handle_string_dot_format(node) + + if ( + _is_typing(node.func, 'cast', self.scopeStack) and + len(node.args) >= 1 and + isinstance(node.args[0], ast.Str) + ): + with self._enter_annotation(): + self.handleNode(node.args[0], node) + + self.handleChildren(node) + + def _handle_percent_format(self, node): + try: + placeholders = parse_percent_format(node.left.s) + except ValueError: + self.report( + messages.PercentFormatInvalidFormat, + node, + 'incomplete format', + ) + return + + named = set() + positional_count = 0 + positional = None + for _, placeholder in placeholders: + if placeholder is None: + continue + name, _, width, precision, conversion = placeholder + + if conversion == '%': + continue + + if conversion not in VALID_CONVERSIONS: + self.report( + messages.PercentFormatUnsupportedFormatCharacter, + node, + conversion, + ) + + if positional is None and conversion: + positional = name is None + + for part in (width, precision): + if part is not None and '*' in part: + if not positional: + self.report( + messages.PercentFormatStarRequiresSequence, + node, + ) + else: + positional_count += 1 + + if positional and name is not None: + self.report( + messages.PercentFormatMixedPositionalAndNamed, + node, + ) + return + elif not positional and name is None: + self.report( + messages.PercentFormatMixedPositionalAndNamed, + node, + ) + return + + if positional: + positional_count += 1 + else: + named.add(name) + + if ( + isinstance(node.right, (ast.List, ast.Tuple)) and + # does not have any *splats (py35+ feature) + not any( + isinstance(elt, getattr(ast, 'Starred', ())) + for elt in node.right.elts + ) + ): + substitution_count = len(node.right.elts) + if positional and positional_count != substitution_count: + self.report( + messages.PercentFormatPositionalCountMismatch, + node, + positional_count, + substitution_count, + ) + elif not positional: + self.report(messages.PercentFormatExpectedMapping, node) + + if ( + isinstance(node.right, ast.Dict) and + all(isinstance(k, ast.Str) for k in node.right.keys) + ): + if positional and positional_count > 1: + self.report(messages.PercentFormatExpectedSequence, node) + return + + substitution_keys = {k.s for k in node.right.keys} + extra_keys = substitution_keys - named + missing_keys = named - substitution_keys + if not positional and extra_keys: + self.report( + messages.PercentFormatExtraNamedArguments, + node, + ', '.join(sorted(extra_keys)), + ) + if not positional and missing_keys: + self.report( + messages.PercentFormatMissingArgument, + node, + ', '.join(sorted(missing_keys)), + ) + + def BINOP(self, node): + if ( + isinstance(node.op, ast.Mod) and + isinstance(node.left, ast.Str) + ): + self._handle_percent_format(node) + self.handleChildren(node) + + def STR(self, node): + if self._in_annotation and not self._in_typing_literal: + fn = functools.partial( + self.handleStringAnnotation, + node.s, + node, + node.lineno, + node.col_offset, + messages.ForwardAnnotationSyntaxError, + ) + if self._in_deferred: + fn() + else: + self.deferFunction(fn) + + if PY38_PLUS: + def CONSTANT(self, node): + if isinstance(node.value, str): + return self.STR(node) + else: + NUM = BYTES = ELLIPSIS = CONSTANT = ignore # "slice" type nodes SLICE = EXTSLICE = INDEX = handleChildren @@ -1254,7 +1775,24 @@ self.report(messages.RaiseNotImplemented, node) # additional node types - COMPREHENSION = KEYWORD = FORMATTEDVALUE = JOINEDSTR = handleChildren + COMPREHENSION = KEYWORD = FORMATTEDVALUE = handleChildren + + _in_fstring = False + + def JOINEDSTR(self, node): + if ( + # the conversion / etc. flags are parsed as f-strings without + # placeholders + not self._in_fstring and + not any(isinstance(x, ast.FormattedValue) for x in node.values) + ): + self.report(messages.FStringMissingPlaceholders, node) + + self._in_fstring, orig = True, self._in_fstring + try: + self.handleChildren(node) + finally: + self._in_fstring = orig def DICT(self, node): # Complain if there are duplicate keys with different values @@ -1292,6 +1830,13 @@ ) self.handleChildren(node) + def IF(self, node): + if isinstance(node.test, ast.Tuple) and node.test.elts != []: + self.report(messages.IfTuple, node) + self.handleChildren(node) + + IFEXP = IF + def ASSERT(self, node): if isinstance(node.test, ast.Tuple) and node.test.elts != []: self.report(messages.AssertTuple, node) @@ -1343,13 +1888,15 @@ Handle occurrence of Name (which can be a load/store/delete access.) """ # Locate the name in locals / function / globals scopes. - if isinstance(node.ctx, (ast.Load, ast.AugLoad)): + if isinstance(node.ctx, ast.Load): self.handleNodeLoad(node) if (node.id == 'locals' and isinstance(self.scope, FunctionScope) and - isinstance(node.parent, ast.Call)): + isinstance(node._pyflakes_parent, ast.Call)): # we are doing locals() call in current scope self.scope.usesLocals = True - elif isinstance(node.ctx, (ast.Store, ast.AugStore, ast.Param)): + elif isinstance(node.ctx, ast.Store): + self.handleNodeStore(node) + elif PY2 and isinstance(node.ctx, ast.Param): self.handleNodeStore(node) elif isinstance(node.ctx, ast.Del): self.handleNodeDelete(node) @@ -1362,8 +1909,8 @@ # definition (not OK), for 'continue', a finally block (not OK), or # the top module scope (not OK) n = node - while hasattr(n, 'parent'): - n, n_child = n.parent, n + while hasattr(n, '_pyflakes_parent'): + n, n_child = n._pyflakes_parent, n if isinstance(n, LOOP_TYPES): # Doesn't apply unless it's in the loop itself if n_child not in n.orelse: @@ -1372,7 +1919,7 @@ break # Handle Try/TryFinally difference in Python < and >= 3.3 if hasattr(n, 'finalbody') and isinstance(node, ast.Continue): - if n_child in n.finalbody: + if n_child in n.finalbody and not PY38_PLUS: self.report(messages.ContinueInFinally, node) return if isinstance(node, ast.Continue): @@ -1433,6 +1980,10 @@ addArgs(node.args.args) defaults = node.args.defaults else: + if PY38_PLUS: + for arg in node.args.posonlyargs: + args.append(arg.arg) + annotations.append(arg.annotation) for arg in node.args.args + node.args.kwonlyargs: args.append(arg.arg) annotations.append(arg.annotation) @@ -1471,7 +2022,7 @@ self.pushScope() - self.handleChildren(node, omit='decorator_list') + self.handleChildren(node, omit=['decorator_list', 'returns']) def checkUnusedAssignments(): """ @@ -1684,14 +2235,14 @@ self.handleNode(node.value, node) def COMPARE(self, node): - literals = (ast.Str, ast.Num) - if not PY2: - literals += (ast.Bytes,) - left = node.left for op, right in zip(node.ops, node.comparators): - if (isinstance(op, (ast.Is, ast.IsNot)) and - (isinstance(left, literals) or isinstance(right, literals))): + if ( + isinstance(op, (ast.Is, ast.IsNot)) and ( + _is_const_non_singleton(left) or + _is_const_non_singleton(right) + ) + ): self.report(messages.IsLiteral, node) left = right diff -Nru pyflakes-2.1.1/pyflakes/__init__.py pyflakes-2.2.0/pyflakes/__init__.py --- pyflakes-2.1.1/pyflakes/__init__.py 2019-02-28 11:15:13.000000000 -0800 +++ pyflakes-2.2.0/pyflakes/__init__.py 2020-04-09 20:50:00.000000000 -0700 @@ -1 +1 @@ -__version__ = '2.1.1' +__version__ = '2.2.0' diff -Nru pyflakes-2.1.1/pyflakes/messages.py pyflakes-2.2.0/pyflakes/messages.py --- pyflakes-2.1.1/pyflakes/messages.py 2019-02-28 11:13:32.000000000 -0800 +++ pyflakes-2.2.0/pyflakes/messages.py 2020-04-09 20:48:16.000000000 -0700 @@ -13,8 +13,8 @@ self.col = getattr(loc, 'col_offset', 0) def __str__(self): - return '%s:%s: %s' % (self.filename, self.lineno, - self.message % self.message_args) + return '%s:%s:%s %s' % (self.filename, self.lineno, self.col+1, + self.message % self.message_args) class UnusedImport(Message): @@ -233,9 +233,16 @@ message = 'too many expressions in star-unpacking assignment' +class IfTuple(Message): + """ + Conditional test is a non-empty tuple literal, which are always True. + """ + message = '\'if tuple literal\' is always true, perhaps remove accidental comma?' + + class AssertTuple(Message): """ - Assertion test is a tuple, which are always True. + Assertion test is a non-empty tuple literal, which are always True. """ message = 'assertion is always true, perhaps remove parentheses?' @@ -265,4 +272,100 @@ class IsLiteral(Message): - message = 'use ==/!= to compare str, bytes, and int literals' + message = 'use ==/!= to compare constant literals (str, bytes, int, float, tuple)' + + +class FStringMissingPlaceholders(Message): + message = 'f-string is missing placeholders' + + +class StringDotFormatExtraPositionalArguments(Message): + message = "'...'.format(...) has unused arguments at position(s): %s" + + def __init__(self, filename, loc, extra_positions): + Message.__init__(self, filename, loc) + self.message_args = (extra_positions,) + + +class StringDotFormatExtraNamedArguments(Message): + message = "'...'.format(...) has unused named argument(s): %s" + + def __init__(self, filename, loc, extra_keywords): + Message.__init__(self, filename, loc) + self.message_args = (extra_keywords,) + + +class StringDotFormatMissingArgument(Message): + message = "'...'.format(...) is missing argument(s) for placeholder(s): %s" + + def __init__(self, filename, loc, missing_arguments): + Message.__init__(self, filename, loc) + self.message_args = (missing_arguments,) + + +class StringDotFormatMixingAutomatic(Message): + message = "'...'.format(...) mixes automatic and manual numbering" + + +class StringDotFormatInvalidFormat(Message): + message = "'...'.format(...) has invalid format string: %s" + + def __init__(self, filename, loc, error): + Message.__init__(self, filename, loc) + self.message_args = (error,) + + +class PercentFormatInvalidFormat(Message): + message = "'...' %% ... has invalid format string: %s" + + def __init__(self, filename, loc, error): + Message.__init__(self, filename, loc) + self.message_args = (error,) + + +class PercentFormatMixedPositionalAndNamed(Message): + message = "'...' %% ... has mixed positional and named placeholders" + + +class PercentFormatUnsupportedFormatCharacter(Message): + message = "'...' %% ... has unsupported format character %r" + + def __init__(self, filename, loc, c): + Message.__init__(self, filename, loc) + self.message_args = (c,) + + +class PercentFormatPositionalCountMismatch(Message): + message = "'...' %% ... has %d placeholder(s) but %d substitution(s)" + + def __init__(self, filename, loc, n_placeholders, n_substitutions): + Message.__init__(self, filename, loc) + self.message_args = (n_placeholders, n_substitutions) + + +class PercentFormatExtraNamedArguments(Message): + message = "'...' %% ... has unused named argument(s): %s" + + def __init__(self, filename, loc, extra_keywords): + Message.__init__(self, filename, loc) + self.message_args = (extra_keywords,) + + +class PercentFormatMissingArgument(Message): + message = "'...' %% ... is missing argument(s) for placeholder(s): %s" + + def __init__(self, filename, loc, missing_arguments): + Message.__init__(self, filename, loc) + self.message_args = (missing_arguments,) + + +class PercentFormatExpectedMapping(Message): + message = "'...' %% ... expected mapping but got sequence" + + +class PercentFormatExpectedSequence(Message): + message = "'...' %% ... expected sequence but got mapping" + + +class PercentFormatStarRequiresSequence(Message): + message = "'...' %% ... `*` specifier requires sequence" diff -Nru pyflakes-2.1.1/pyflakes/test/test_api.py pyflakes-2.2.0/pyflakes/test/test_api.py --- pyflakes-2.1.1/pyflakes/test/test_api.py 2019-02-28 11:10:37.000000000 -0800 +++ pyflakes-2.2.0/pyflakes/test/test_api.py 2020-04-09 20:48:16.000000000 -0700 @@ -9,7 +9,6 @@ import subprocess import tempfile -from pyflakes.checker import PY2 from pyflakes.messages import UnusedImport from pyflakes.reporter import Reporter from pyflakes.api import ( @@ -214,9 +213,35 @@ with open(pythonw, 'w') as fd: fd.write('#!/usr/bin/env pythonw\n') + python3args = os.path.join(self.tempdir, 'g') + with open(python3args, 'w') as fd: + fd.write('#!/usr/bin/python3 -u\n') + + python2u = os.path.join(self.tempdir, 'h') + with open(python2u, 'w') as fd: + fd.write('#!/usr/bin/python2u\n') + + python3d = os.path.join(self.tempdir, 'i') + with open(python3d, 'w') as fd: + fd.write('#!/usr/local/bin/python3d\n') + + python38m = os.path.join(self.tempdir, 'j') + with open(python38m, 'w') as fd: + fd.write('#! /usr/bin/env python3.8m\n') + + python27 = os.path.join(self.tempdir, 'k') + with open(python27, 'w') as fd: + fd.write('#!/usr/bin/python2.7 \n') + + # Should NOT be treated as Python source + notfirst = os.path.join(self.tempdir, 'l') + with open(notfirst, 'w') as fd: + fd.write('#!/bin/sh\n#!/usr/bin/python\n') + self.assertEqual( sorted(iterSourceCode([self.tempdir])), - sorted([python, python2, python3, pythonw])) + sorted([python, python2, python3, pythonw, python3args, python2u, + python3d, python38m, python27])) def test_multipleDirectories(self): """ @@ -423,7 +448,7 @@ with self.makeTempFile(source) as sourcePath: if PYPY: - message = 'EOF while scanning triple-quoted string literal' + message = 'end of file (EOF) while scanning triple-quoted string literal' else: message = 'invalid syntax' @@ -465,8 +490,8 @@ syntax error reflects the cause for the syntax error. """ with self.makeTempFile("if True:\n\tfoo =") as sourcePath: - column = 5 if PYPY else 7 - last_line = '\t ^' if PYPY else '\t ^' + column = 6 if PYPY else 7 + last_line = '\t ^' if PYPY else '\t ^' self.assertHasErrors( sourcePath, @@ -488,7 +513,12 @@ """ with self.makeTempFile(source) as sourcePath: if ERROR_HAS_LAST_LINE: - column = 9 if sys.version_info >= (3, 8) else 8 + if PYPY and sys.version_info >= (3,): + column = 7 + elif sys.version_info >= (3, 8): + column = 9 + else: + column = 8 last_line = ' ' * (column - 1) + '^\n' columnstr = '%d:' % column else: @@ -511,7 +541,12 @@ """ with self.makeTempFile(source) as sourcePath: if ERROR_HAS_LAST_LINE: - column = 14 if sys.version_info >= (3, 8) else 13 + if PYPY and sys.version_info >= (3,): + column = 12 + elif sys.version_info >= (3, 8): + column = 14 + else: + column = 13 last_line = ' ' * (column - 1) + '^\n' columnstr = '%d:' % column else: @@ -681,6 +716,12 @@ Tests of the pyflakes script that actually spawn the script. """ + # https://bitbucket.org/pypy/pypy/issues/3069/pypy36-on-windows-incorrect-line-separator + if PYPY and sys.version_info >= (3,) and WIN: + LINESEP = '\n' + else: + LINESEP = os.linesep + def setUp(self): self.tempdir = tempfile.mkdtemp() self.tempfilepath = os.path.join(self.tempdir, 'temp') @@ -721,9 +762,6 @@ if sys.version_info >= (3,): stdout = stdout.decode('utf-8') stderr = stderr.decode('utf-8') - # Workaround https://bitbucket.org/pypy/pypy/issues/2350 - if PYPY and PY2 and WIN: - stderr = stderr.replace('\r\r\n', '\r\n') return (stdout, stderr, rv) def test_goodFile(self): @@ -744,7 +782,7 @@ fd.write("import contraband\n".encode('ascii')) d = self.runPyflakes([self.tempfilepath]) expected = UnusedImport(self.tempfilepath, Node(1), 'contraband') - self.assertEqual(d, ("%s%s" % (expected, os.linesep), '', 1)) + self.assertEqual(d, ("%s%s" % (expected, self.LINESEP), '', 1)) def test_errors_io(self): """ @@ -754,7 +792,7 @@ """ d = self.runPyflakes([self.tempfilepath]) error_msg = '%s: No such file or directory%s' % (self.tempfilepath, - os.linesep) + self.LINESEP) self.assertEqual(d, ('', error_msg, 1)) def test_errors_syntax(self): @@ -766,8 +804,8 @@ with open(self.tempfilepath, 'wb') as fd: fd.write("import".encode('ascii')) d = self.runPyflakes([self.tempfilepath]) - error_msg = '{0}:1:{2}: invalid syntax{1}import{1} {3}^{1}'.format( - self.tempfilepath, os.linesep, 5 if PYPY else 7, '' if PYPY else ' ') + error_msg = '{0}:1:{2}: invalid syntax{1}import{1} {3}^{1}'.format( + self.tempfilepath, self.LINESEP, 6 if PYPY else 7, '' if PYPY else ' ') self.assertEqual(d, ('', error_msg, 1)) def test_readFromStdin(self): @@ -776,13 +814,14 @@ """ d = self.runPyflakes([], stdin='import contraband') expected = UnusedImport('', Node(1), 'contraband') - self.assertEqual(d, ("%s%s" % (expected, os.linesep), '', 1)) + self.assertEqual(d, ("%s%s" % (expected, self.LINESEP), '', 1)) class TestMain(IntegrationTests): """ Tests of the pyflakes main function. """ + LINESEP = os.linesep def runPyflakes(self, paths, stdin=None): try: diff -Nru pyflakes-2.1.1/pyflakes/test/test_checker.py pyflakes-2.2.0/pyflakes/test/test_checker.py --- pyflakes-2.1.1/pyflakes/test/test_checker.py 2019-02-28 11:10:37.000000000 -0800 +++ pyflakes-2.2.0/pyflakes/test/test_checker.py 2020-04-09 20:48:16.000000000 -0700 @@ -154,7 +154,7 @@ def test_type_comment_starts_with_word_ignore(self): ret = self._collect('x = 1 # type: ignore[T]') - self.assertSetEqual(ret, {(ast.Assign, ('# type: ignore[T]',))}) + self.assertSetEqual(ret, set()) def test_last_node_wins(self): """ diff -Nru pyflakes-2.1.1/pyflakes/test/test_code_segment.py pyflakes-2.2.0/pyflakes/test/test_code_segment.py --- pyflakes-2.1.1/pyflakes/test/test_code_segment.py 2019-02-28 11:10:37.000000000 -0800 +++ pyflakes-2.2.0/pyflakes/test/test_code_segment.py 2020-04-09 20:48:16.000000000 -0700 @@ -1,7 +1,9 @@ +from sys import version_info + from pyflakes import messages as m from pyflakes.checker import (FunctionScope, ClassScope, ModuleScope, Argument, FunctionDefinition, Assignment) -from pyflakes.test.harness import TestCase +from pyflakes.test.harness import TestCase, skipIf class TestCodeSegments(TestCase): @@ -124,3 +126,7 @@ self.assertIsInstance(function_scope_bar['g'], Argument) self.assertIsInstance(function_scope_bar['h'], Argument) self.assertIsInstance(function_scope_bar['i'], Argument) + + @skipIf(version_info < (3, 5), 'new in Python 3.5') + def test_scope_async_function(self): + self.flakes('async def foo(): pass', is_segment=True) diff -Nru pyflakes-2.1.1/pyflakes/test/test_doctests.py pyflakes-2.2.0/pyflakes/test/test_doctests.py --- pyflakes-2.1.1/pyflakes/test/test_doctests.py 2019-02-28 11:10:37.000000000 -0800 +++ pyflakes-2.2.0/pyflakes/test/test_doctests.py 2020-04-09 20:48:16.000000000 -0700 @@ -328,7 +328,9 @@ m.DoctestSyntaxError).messages exc = exceptions[0] self.assertEqual(exc.lineno, 4) - if sys.version_info >= (3, 8): + if PYPY: + self.assertEqual(exc.col, 27) + elif sys.version_info >= (3, 8): self.assertEqual(exc.col, 18) else: self.assertEqual(exc.col, 26) @@ -339,12 +341,14 @@ exc = exceptions[1] self.assertEqual(exc.lineno, 5) if PYPY: - self.assertEqual(exc.col, 13) + self.assertEqual(exc.col, 14) else: self.assertEqual(exc.col, 16) exc = exceptions[2] self.assertEqual(exc.lineno, 6) - if PYPY or sys.version_info >= (3, 8): + if PYPY: + self.assertEqual(exc.col, 14) + elif sys.version_info >= (3, 8): self.assertEqual(exc.col, 13) else: self.assertEqual(exc.col, 18) @@ -358,7 +362,9 @@ """ ''', m.DoctestSyntaxError).messages[0] self.assertEqual(exc.lineno, 5) - if PYPY or sys.version_info >= (3, 8): + if PYPY: + self.assertEqual(exc.col, 14) + elif sys.version_info >= (3, 8): self.assertEqual(exc.col, 13) else: self.assertEqual(exc.col, 16) @@ -377,7 +383,10 @@ m.DoctestSyntaxError, m.UndefinedName).messages self.assertEqual(exc1.lineno, 6) - self.assertEqual(exc1.col, 19) + if PYPY: + self.assertEqual(exc1.col, 20) + else: + self.assertEqual(exc1.col, 19) self.assertEqual(exc2.lineno, 7) self.assertEqual(exc2.col, 12) @@ -433,6 +442,16 @@ return 1 ''') + def test_globalUnderscoreInDoctest(self): + self.flakes(""" + from gettext import ugettext as _ + + def doctest_stuff(): + ''' + >>> pass + ''' + """, m.UnusedImport) + class TestOther(_DoctestMixin, TestOther): """Run TestOther with each test wrapped in a doctest.""" diff -Nru pyflakes-2.1.1/pyflakes/test/test_is_literal.py pyflakes-2.2.0/pyflakes/test/test_is_literal.py --- pyflakes-2.1.1/pyflakes/test/test_is_literal.py 2019-02-28 11:10:37.000000000 -0800 +++ pyflakes-2.2.0/pyflakes/test/test_is_literal.py 2020-04-09 20:48:16.000000000 -0700 @@ -198,3 +198,25 @@ if 4 < x is 'foo': pass """, IsLiteral) + + def test_is_tuple_constant(self): + self.flakes('''\ + x = 5 + if x is (): + pass + ''', IsLiteral) + + def test_is_tuple_constant_containing_constants(self): + self.flakes('''\ + x = 5 + if x is (1, '2', True, (1.5, ())): + pass + ''', IsLiteral) + + def test_is_tuple_containing_variables_ok(self): + # a bit nonsensical, but does not trigger a SyntaxWarning + self.flakes('''\ + x = 5 + if x is (x,): + pass + ''') diff -Nru pyflakes-2.1.1/pyflakes/test/test_other.py pyflakes-2.2.0/pyflakes/test/test_other.py --- pyflakes-2.1.1/pyflakes/test/test_other.py 2019-02-28 11:10:37.000000000 -0800 +++ pyflakes-2.2.0/pyflakes/test/test_other.py 2020-04-09 20:48:16.000000000 -0700 @@ -493,8 +493,10 @@ continue ''') + @skipIf(version_info > (3, 8), "Python <= 3.8 only") def test_continueInFinally(self): # 'continue' inside 'finally' is a special syntax error + # that is removed in 3.8 self.flakes(''' while True: try: @@ -1431,6 +1433,29 @@ self.flakes("a = foo if True else 'oink'", m.UndefinedName) self.flakes("a = 'moo' if True else bar", m.UndefinedName) + def test_if_tuple(self): + """ + Test C{if (foo,)} conditions. + """ + self.flakes("""if (): pass""") + self.flakes(""" + if ( + True + ): + pass + """) + self.flakes(""" + if ( + True, + ): + pass + """, m.IfTuple) + self.flakes(""" + x = 1 if ( + True, + ) else 2 + """, m.IfTuple) + def test_withStatementNoNames(self): """ No warnings are emitted for using inside or after a nameless C{with} @@ -1754,6 +1779,168 @@ print(f'\x7b4*baz\N{RIGHT CURLY BRACKET}') ''') + @skipIf(version_info < (3, 8), 'new in Python 3.8') + def test_assign_expr(self): + """Test PEP 572 assignment expressions are treated as usage / write.""" + self.flakes(''' + from foo import y + print(x := y) + print(x) + ''') + + +class TestStringFormatting(TestCase): + + @skipIf(version_info < (3, 6), 'new in Python 3.6') + def test_f_string_without_placeholders(self): + self.flakes("f'foo'", m.FStringMissingPlaceholders) + self.flakes(''' + f"""foo + bar + """ + ''', m.FStringMissingPlaceholders) + self.flakes(''' + print( + f'foo' + f'bar' + ) + ''', m.FStringMissingPlaceholders) + # this is an "escaped placeholder" but not a placeholder + self.flakes("f'{{}}'", m.FStringMissingPlaceholders) + # ok: f-string with placeholders + self.flakes(''' + x = 5 + print(f'{x}') + ''') + # ok: f-string with format specifiers + self.flakes(''' + x = 'a' * 90 + print(f'{x:.8}') + ''') + # ok: f-string with multiple format specifiers + self.flakes(''' + x = y = 5 + print(f'{x:>2} {y:>2}') + ''') + + def test_invalid_dot_format_calls(self): + self.flakes(''' + '{'.format(1) + ''', m.StringDotFormatInvalidFormat) + self.flakes(''' + '{} {1}'.format(1, 2) + ''', m.StringDotFormatMixingAutomatic) + self.flakes(''' + '{0} {}'.format(1, 2) + ''', m.StringDotFormatMixingAutomatic) + self.flakes(''' + '{}'.format(1, 2) + ''', m.StringDotFormatExtraPositionalArguments) + self.flakes(''' + '{}'.format(1, bar=2) + ''', m.StringDotFormatExtraNamedArguments) + self.flakes(''' + '{} {}'.format(1) + ''', m.StringDotFormatMissingArgument) + self.flakes(''' + '{2}'.format() + ''', m.StringDotFormatMissingArgument) + self.flakes(''' + '{bar}'.format() + ''', m.StringDotFormatMissingArgument) + # too much string recursion (placeholder-in-placeholder) + self.flakes(''' + '{:{:{}}}'.format(1, 2, 3) + ''', m.StringDotFormatInvalidFormat) + # ok: dotted / bracketed names need to handle the param differently + self.flakes("'{.__class__}'.format('')") + self.flakes("'{foo[bar]}'.format(foo={'bar': 'barv'})") + # ok: placeholder-placeholders + self.flakes(''' + print('{:{}} {}'.format(1, 15, 2)) + ''') + # ok: not a placeholder-placeholder + self.flakes(''' + print('{:2}'.format(1)) + ''') + # ok: not mixed automatic + self.flakes(''' + '{foo}-{}'.format(1, foo=2) + ''') + # ok: we can't determine statically the format args + self.flakes(''' + a = () + "{}".format(*a) + ''') + self.flakes(''' + k = {} + "{foo}".format(**k) + ''') + + def test_invalid_percent_format_calls(self): + self.flakes(''' + '%(foo)' % {'foo': 'bar'} + ''', m.PercentFormatInvalidFormat) + self.flakes(''' + '%s %(foo)s' % {'foo': 'bar'} + ''', m.PercentFormatMixedPositionalAndNamed) + self.flakes(''' + '%(foo)s %s' % {'foo': 'bar'} + ''', m.PercentFormatMixedPositionalAndNamed) + self.flakes(''' + '%j' % (1,) + ''', m.PercentFormatUnsupportedFormatCharacter) + self.flakes(''' + '%s %s' % (1,) + ''', m.PercentFormatPositionalCountMismatch) + self.flakes(''' + '%s %s' % (1, 2, 3) + ''', m.PercentFormatPositionalCountMismatch) + self.flakes(''' + '%(bar)s' % {} + ''', m.PercentFormatMissingArgument,) + self.flakes(''' + '%(bar)s' % {'bar': 1, 'baz': 2} + ''', m.PercentFormatExtraNamedArguments) + self.flakes(''' + '%(bar)s' % (1, 2, 3) + ''', m.PercentFormatExpectedMapping) + self.flakes(''' + '%s %s' % {'k': 'v'} + ''', m.PercentFormatExpectedSequence) + self.flakes(''' + '%(bar)*s' % {'bar': 'baz'} + ''', m.PercentFormatStarRequiresSequence) + # ok: single %s with mapping + self.flakes(''' + '%s' % {'foo': 'bar', 'baz': 'womp'} + ''') + # ok: does not cause a MemoryError (the strings aren't evaluated) + self.flakes(''' + "%1000000000000f" % 1 + ''') + # ok: %% should not count towards placeholder count + self.flakes(''' + '%% %s %% %s' % (1, 2) + ''') + # ok: * consumes one positional argument + self.flakes(''' + '%.*f' % (2, 1.1234) + '%*.*f' % (5, 2, 3.1234) + ''') + + @skipIf(version_info < (3, 5), 'new in Python 3.5') + def test_ok_percent_format_cannot_determine_element_count(self): + self.flakes(''' + a = [] + '%s %s' % [*a] + '%s %s' % (*a,) + ''') + self.flakes(''' + k = {} + '%(k)s' % {**k} + ''') + class TestAsyncStatements(TestCase): @@ -1841,6 +2028,7 @@ ''', m.BreakOutsideLoop) @skipIf(version_info < (3, 5), 'new in Python 3.5') + @skipIf(version_info > (3, 8), "Python <= 3.8 only") def test_continueInAsyncForFinally(self): self.flakes(''' async def read_data(db): diff -Nru pyflakes-2.1.1/pyflakes/test/test_type_annotations.py pyflakes-2.2.0/pyflakes/test/test_type_annotations.py --- pyflakes-2.1.1/pyflakes/test/test_type_annotations.py 2019-02-28 11:10:37.000000000 -0800 +++ pyflakes-2.2.0/pyflakes/test/test_type_annotations.py 2020-04-09 20:48:16.000000000 -0700 @@ -39,27 +39,101 @@ return s """) - def test_not_a_typing_overload(self): - """regression test for @typing.overload detection bug in 2.1.0""" + def test_typingExtensionsOverload(self): + """Allow intentional redefinitions via @typing_extensions.overload""" + self.flakes(""" + import typing_extensions + from typing_extensions import overload + + @overload + def f(s): # type: (None) -> None + pass + + @overload + def f(s): # type: (int) -> int + pass + + def f(s): + return s + + @typing_extensions.overload + def g(s): # type: (None) -> None + pass + + @typing_extensions.overload + def g(s): # type: (int) -> int + pass + + def g(s): + return s + """) + + @skipIf(version_info < (3, 5), 'new in Python 3.5') + def test_typingOverloadAsync(self): + """Allow intentional redefinitions via @typing.overload (async)""" + self.flakes(""" + from typing import overload + + @overload + async def f(s): # type: (None) -> None + pass + + @overload + async def f(s): # type: (int) -> int + pass + + async def f(s): + return s + """) + + def test_overload_with_multiple_decorators(self): + self.flakes(""" + from typing import overload + dec = lambda f: f + + @dec + @overload + def f(x): # type: (int) -> int + pass + + @dec + @overload + def f(x): # type: (str) -> str + pass + + @dec + def f(x): return x + """) + + def test_overload_in_class(self): self.flakes(""" - x = lambda f: f + from typing import overload + + class C: + @overload + def f(self, x): # type: (int) -> int + pass - @x - def t(): + @overload + def f(self, x): # type: (str) -> str pass - y = lambda f: f + def f(self, x): return x + """) + + def test_not_a_typing_overload(self): + """regression test for @typing.overload detection bug in 2.1.0""" + self.flakes(""" + def foo(x): + return x - @x - @y - def t(): + @foo + def bar(): pass - @x - @y - def t(): + def bar(): pass - """, m.RedefinedWhileUnused, m.RedefinedWhileUnused) + """, m.RedefinedWhileUnused) @skipIf(version_info < (3, 6), 'new in Python 3.6') def test_variable_annotations(self): @@ -270,7 +344,7 @@ return a + b ''') - def test_typeCommentsAdditionalComemnt(self): + def test_typeCommentsAdditionalComment(self): self.flakes(""" from mod import F @@ -315,3 +389,166 @@ x = 1 # type: F """) + + def test_typeIgnore(self): + self.flakes(""" + a = 0 # type: ignore + b = 0 # type: ignore[excuse] + c = 0 # type: ignore=excuse + d = 0 # type: ignore [excuse] + e = 0 # type: ignore whatever + """) + + def test_typeIgnoreBogus(self): + self.flakes(""" + x = 1 # type: ignored + """, m.UndefinedName) + + def test_typeIgnoreBogusUnicode(self): + error = (m.CommentAnnotationSyntaxError if version_info < (3,) + else m.UndefinedName) + self.flakes(""" + x = 2 # type: ignore\xc3 + """, error) + + @skipIf(version_info < (3,), 'new in Python 3') + def test_return_annotation_is_class_scope_variable(self): + self.flakes(""" + from typing import TypeVar + class Test: + Y = TypeVar('Y') + + def t(self, x: Y) -> Y: + return x + """) + + @skipIf(version_info < (3,), 'new in Python 3') + def test_return_annotation_is_function_body_variable(self): + self.flakes(""" + class Test: + def t(self) -> Y: + Y = 2 + return Y + """, m.UndefinedName) + + @skipIf(version_info < (3, 8), 'new in Python 3.8') + def test_positional_only_argument_annotations(self): + self.flakes(""" + from x import C + + def f(c: C, /): ... + """) + + @skipIf(version_info < (3,), 'new in Python 3') + def test_partially_quoted_type_annotation(self): + self.flakes(""" + from queue import Queue + from typing import Optional + + def f() -> Optional['Queue[str]']: + return None + """) + + def test_partially_quoted_type_assignment(self): + self.flakes(""" + from queue import Queue + from typing import Optional + + MaybeQueue = Optional['Queue[str]'] + """) + + def test_nested_partially_quoted_type_assignment(self): + self.flakes(""" + from queue import Queue + from typing import Callable + + Func = Callable[['Queue[str]'], None] + """) + + def test_quoted_type_cast(self): + self.flakes(""" + from typing import cast, Optional + + maybe_int = cast('Optional[int]', 42) + """) + + def test_type_cast_literal_str_to_str(self): + # Checks that our handling of quoted type annotations in the first + # argument to `cast` doesn't cause issues when (only) the _second_ + # argument is a literal str which looks a bit like a type annoation. + self.flakes(""" + from typing import cast + + a_string = cast(str, 'Optional[int]') + """) + + def test_quoted_type_cast_renamed_import(self): + self.flakes(""" + from typing import cast as tsac, Optional as Maybe + + maybe_int = tsac('Maybe[int]', 42) + """) + + @skipIf(version_info < (3,), 'new in Python 3') + def test_literal_type_typing(self): + self.flakes(""" + from typing import Literal + + def f(x: Literal['some string']) -> None: + return None + """) + + @skipIf(version_info < (3,), 'new in Python 3') + def test_literal_type_typing_extensions(self): + self.flakes(""" + from typing_extensions import Literal + + def f(x: Literal['some string']) -> None: + return None + """) + + @skipIf(version_info < (3,), 'new in Python 3') + def test_literal_type_some_other_module(self): + """err on the side of false-negatives for types named Literal""" + self.flakes(""" + from my_module import compat + from my_module.compat import Literal + + def f(x: compat.Literal['some string']) -> None: + return None + def g(x: Literal['some string']) -> None: + return None + """) + + @skipIf(version_info < (3,), 'new in Python 3') + def test_literal_union_type_typing(self): + self.flakes(""" + from typing import Literal + + def f(x: Literal['some string', 'foo bar']) -> None: + return None + """) + + @skipIf(version_info < (3,), 'new in Python 3') + def test_deferred_twice_annotation(self): + self.flakes(""" + from queue import Queue + from typing import Optional + + + def f() -> "Optional['Queue[str]']": + return None + """) + + @skipIf(version_info < (3, 7), 'new in Python 3.7') + def test_partial_string_annotations_with_future_annotations(self): + self.flakes(""" + from __future__ import annotations + + from queue import Queue + from typing import Optional + + + def f() -> Optional['Queue[str]']: + return None + """) diff -Nru pyflakes-2.1.1/pyflakes.egg-info/PKG-INFO pyflakes-2.2.0/pyflakes.egg-info/PKG-INFO --- pyflakes-2.1.1/pyflakes.egg-info/PKG-INFO 2019-02-28 11:25:22.000000000 -0800 +++ pyflakes-2.2.0/pyflakes.egg-info/PKG-INFO 2020-04-09 20:51:46.000000000 -0700 @@ -1,6 +1,6 @@ Metadata-Version: 1.2 Name: pyflakes -Version: 2.1.1 +Version: 2.2.0 Summary: passive checker of Python programs Home-page: https://github.com/PyCQA/pyflakes Author: A lot of people @@ -94,7 +94,7 @@ Changelog --------- - Please see `NEWS.rst `_. + Please see `NEWS.rst `_. Platform: UNKNOWN Classifier: Development Status :: 6 - Mature diff -Nru pyflakes-2.1.1/README.rst pyflakes-2.2.0/README.rst --- pyflakes-2.1.1/README.rst 2019-02-28 11:10:37.000000000 -0800 +++ pyflakes-2.2.0/README.rst 2020-04-09 20:48:16.000000000 -0700 @@ -86,4 +86,4 @@ Changelog --------- -Please see `NEWS.rst `_. +Please see `NEWS.rst `_.