What's New in Python 3.6.7 final? ================================= *Release date: 2018-10-20* There were no new changes in version 3.6.7. What's New in Python 3.6.7 release candidate 2? =============================================== *Release date: 2018-10-13* Core and Builtins ----------------- - bpo-34879: Fix a possible null pointer dereference in bytesobject.c. Patch by Zackery Spytz. - bpo-34320: Fix ``dict(od)`` didn't copy iteration order of OrderedDict. Library ------- - bpo-34769: Fix for async generators not finalizing when event loop is in debug mode and garbage collector runs in another thread. - bpo-34922: Fixed integer overflow in the :meth:`~hashlib.shake.digest()` and :meth:`~hashlib.shake.hexdigest()` methods for the SHAKE algorithm in the :mod:`hashlib` module. - bpo-34871: Fix inspect module polluted ``sys.modules`` when parsing ``__text_signature__`` of callable. - bpo-34872: Fix self-cancellation in C implementation of asyncio.Task - bpo-34819: Use a monotonic clock to compute timeouts in :meth:`Executor.map` and :func:`as_completed`, in order to prevent timeouts from deviating when the system clock is adjusted. - bpo-34521: Use :func:`socket.CMSG_SPACE` to calculate ancillary data size instead of :func:`socket.CMSG_LEN` in :func:`multiprocessing.reduction.recvfds` as :rfc:`3542` requires the use of the former for portable applications. - bpo-34282: Fix enum members getting shadowed by parent attributes. - bpo-34172: Fix a reference issue inside multiprocessing.Pool that caused the pool to remain alive if it was deleted without being closed or terminated explicitly. - bpo-33729: Fixed issues with arguments parsing in :mod:`hashlib`. Documentation ------------- - bpo-32174: chm document displays non-ASCII charaters properly on some MBCS Windows systems. Tests ----- - bpo-32962: Fixed test_gdb when Python is compiled with flags -mcet -fcf-protection -O0. macOS ----- - bpo-34370: Revert to using the released Tk 8.6.8 with macOS installers instead of the Tk 8.6.x development snapshot used with 3.7.1rc1 and 3.6.7rc1. The snapshot introduced at least one significant regression (bpo-34927). C API ----- - bpo-34910: Ensure that :c:func:`PyObject_Print` always returns ``-1`` on error. Patch by Zackery Spytz. What's New in Python 3.6.7 release candidate 1? =============================================== *Release date: 2018-09-26* Security -------- - bpo-17239: The xml.sax and xml.dom.minidom parsers no longer processes external entities by default. External DTD and ENTITY declarations no longer load files or create network connections. - bpo-34623: CVE-2018-14647: The C accelerated _elementtree module now initializes hash randomization salt from _Py_HashSecret instead of libexpat's default CSPRNG. - bpo-34405: Updated to OpenSSL 1.0.2p for Windows builds. - bpo-33871: Fixed sending the part of the file in :func:`os.sendfile` on macOS. Using the *trailers* argument could cause sending more bytes from the input file than was specified. - bpo-32533: Fixed thread-safety of error handling in _ssl. Core and Builtins ----------------- - bpo-34735: Fix a memory leak in Modules/timemodule.c. Patch by Zackery Spytz. - bpo-34588: Fix an off-by-one in the recursive call pruning feature of traceback formatting. - bpo-34527: On FreeBSD, Py_DecodeLocale() and Py_EncodeLocale() now also forces the ASCII encoding if the LC_CTYPE locale is "POSIX", not only if the LC_CTYPE locale is "C". - bpo-34400: Fix undefined behavior in parsetok.c. Patch by Zackery Spytz. - bpo-34377: Update valgrind suppression list to use ``_PyObject_Free``/``_PyObject_Realloc`` instead of ``PyObject_Free``/``PyObject_Realloc``. - bpo-24618: Fixed reading invalid memory when create the code object with too small varnames tuple or too large argument counts. - bpo-34068: In :meth:`io.IOBase.close`, ensure that the :attr:`~io.IOBase.closed` attribute is not set with a live exception. Patch by Zackery Spytz and Serhiy Storchaka. - bpo-34080: Fixed a memory leak in the compiler when it raised some uncommon errors during tokenizing. - bpo-34066: Disabled interruption by Ctrl-C between calling ``open()`` and entering a **with** block in ``with open()``. - bpo-33956: Update vendored Expat library copy to version 2.2.5. - bpo-24596: Decref the module object in :c:func:`PyRun_SimpleFileExFlags` before calling :c:func:`PyErr_Print()`. Patch by Zackery Spytz. - bpo-33451: Close directly executed pyc files before calling ``PyEval_EvalCode()``. - bpo-33312: Fixed clang ubsan (undefined behavior sanitizer) warnings in dictobject.c by adjusting how the internal struct _dictkeysobject shared keys structure is declared. - bpo-25750: Fix rare Python crash due to bad refcounting in ``type_getattro()`` if a descriptor deletes itself from the class. Patch by Jeroen Demeyer. - bpo-25862: Fix assertion failures in the ``tell()`` method of ``io.TextIOWrapper``. Patch by Zackery Spytz. Library ------- - bpo-34670: Add SSLContext.post_handshake_auth and SSLSocket.verify_client_post_handshake for TLS 1.3's post handshake authentication feature. - bpo-34652: Ensure :func:`os.lchmod` is never defined on Linux. - bpo-34625: Update vendorized expat library version to 2.2.6. - bpo-32270: The subprocess module no longer mistakenly closes redirected fds even when they were in pass_fds when outside of the default {0, 1, 2} set. - bpo-34610: Fixed iterator of :class:`multiprocessing.managers.DictProxy`. - bpo-34421: Fix distutils logging for non-ASCII strings. This caused installation issues on Windows. - bpo-34604: Fix possible mojibake in the error message of `pwd.getpwnam` and `grp.getgrnam`. Patch by William Grzybowski. - bpo-34530: ``distutils.spawn.find_executable()`` now falls back on :data:`os.defpath` if the ``PATH`` environment variable is not set. - bpo-34563: On Windows, fix multiprocessing.Connection for very large read: fix _winapi.PeekNamedPipe() and _winapi.ReadFile() for read larger than INT_MAX (usually 2^31-1). - bpo-13312: Avoids a possible integer underflow (undefined behavior) in the time module's year handling code when passed a very low negative year value. - bpo-34472: Improved compatibility for streamed files in :mod:`zipfile`. Previously an optional signature was not being written and certain ZIP applications were not supported. Patch by Silas Sewell. - bpo-6700: Fix inspect.getsourcelines for module level frames/tracebacks. Patch by Vladimir Matveev. - bpo-32947: Add OP_ENABLE_MIDDLEBOX_COMPAT and test workaround for TLSv1.3 for future compatibility with OpenSSL 1.1.1. - bpo-34341: Appending to the ZIP archive with the ZIP64 extension no longer grows the size of extra fields of existing entries. - bpo-18540: The :class:`imaplib.IMAP4` and :class:`imaplib.IMAP4_SSL` classes now resolve to the local host IP correctly when the default value of *host* parameter (``''``) is used. - bpo-34246: :meth:`smtplib.SMTP.send_message` no longer modifies the content of the *mail_options* argument. Patch by Pablo S. Blum de Aguiar. - bpo-31047: Fix ``ntpath.abspath`` for invalid paths on windows. Patch by Franz Woellert. - bpo-34263: asyncio's event loop will not pass timeouts longer than one day to epoll/select etc. - bpo-32215: Fix performance regression in :mod:`sqlite3` when a DML statement appeared in a different line than the rest of the SQL query. - bpo-19891: Ignore errors caused by missing / non-writable homedir while writing history during exit of an interactive session. Patch by Anthony Sottile. - bpo-940286: pydoc's ``Helper.showtopic()`` method now prints the cross references of a topic correctly. - bpo-34164: :func:`base64.b32decode` could raise UnboundLocalError or OverflowError for incorrect padding. Now it always raises :exc:`base64.Error` in these cases. - bpo-33967: functools.singledispatch now raises TypeError instead of IndexError when no positional arguments are passed. - bpo-34054: The multiprocessing module now uses the monotonic clock :func:`time.monotonic` instead of the system clock :func:`time.time` to implement timeout. - bpo-34010: Fixed a performance regression for reading streams with tarfile. The buffered read should use a list, instead of appending to a bytes object. - bpo-34019: webbrowser: Correct the arguments passed to Opera Browser when opening a new URL using the ``webbrowser`` module. Patch by Bumsik Kim. - bpo-33978: Closed existing logging handlers before reconfiguration via fileConfig and dictConfig. Patch by Karthikeyan Singaravelan. - bpo-14117: Make minor tweaks to turtledemo. The 'wikipedia' example is now 'rosette', decribing what it draws. The 'penrose' print output is reduced. The'1024' output of 'tree' is eliminated. - bpo-33974: Fixed passing lists and tuples of strings containing special characters ``"``, ``\``, ``{``, ``}`` and ``\n`` as options to :mod:`~tkinter.ttk` widgets. - bpo-27500: Fix getaddrinfo to resolve IPv6 addresses correctly. - bpo-24567: Improve random.choices() to handle subnormal input weights that could occasionally trigger an IndexError. - bpo-33871: Fixed integer overflow in :func:`os.readv` and :func:`os.writev` and in :func:`os.sendfile` with *headers* or *trailers* arguments (on BSD-based OSes and macOS). - bpo-33899: Tokenize module now implicitly emits a NEWLINE when provided with input that does not have a trailing new line. This behavior now matches what the C tokenizer does internally. Contributed by Ammar Askar. - bpo-33916: bz2 and lzma: When Decompressor.__init__() is called twice, free the old lock to not leak memory. - bpo-32568: Make select.epoll() and its documentation consistent regarding *sizehint* and *flags*. - bpo-33663: Convert content length to string before putting to header. - bpo-26544: Fixed implementation of :func:`platform.libc_ver`. It almost always returned version '2.9' for glibc. - bpo-27397: Make email module properly handle invalid-length base64 strings. - bpo-33476: Fix _header_value_parser.py when address group is missing final ';'. Contributed by Enrique Perez-Terron - bpo-33570: Change TLS 1.3 cipher suite settings for compatibility with OpenSSL 1.1.1-pre6 and newer. OpenSSL 1.1.1 will have TLS 1.3 cipers enabled by default. - bpo-33365: Print the header values besides the header keys instead just the header keys if *debuglevel* is set to >0 in :mod:`http.client`. Patch by Marco Strigl. - bpo-33336: ``imaplib`` now allows ``MOVE`` command in ``IMAP4.uid()`` (RFC 6851: IMAP MOVE Extension) and potentially as a name of supported method of ``IMAP4`` object. - bpo-32356: asyncio.transport.resume_reading() and pause_reading() are now idempotent. - bpo-31608: Raise a ``TypeError`` instead of crashing if a ``collections.deque`` subclass returns a non-deque from ``__new__``. Patch by Oren Milman. - bpo-29456: Fix bugs in hangul normalization: u1176, u11a7 and u11c3 Documentation ------------- - bpo-28617: Fixed info in the stdtypes docs concerning the types that support membership tests. - bpo-34065: Fix wrongly written basicConfig documentation markup syntax - bpo-33847: Add '@' operator entry to index. - bpo-25041: Document ``AF_PACKET`` in the :mod:`socket` module. Tests ----- - bpo-34587: test_socket: Remove RDSTest.testCongestion(). The test tries to fill the receiver's socket buffer and expects an error. But the RDS protocol doesn't require that. Moreover, the Linux implementation of RDS expects that the producer of the messages reduces its rate, it's not the role of the receiver to trigger an error. The test fails on Fedora 28 by design, so just remove it. - bpo-34661: Fix test_shutil if unzip doesn't support -t. - bpo-34200: Fixed non-deterministic flakiness of test_pkg by not using the scary test.support.module_cleanup() logic to save and restore sys.modules contents between test cases. - bpo-34594: Fix usage of hardcoded ``errno`` values in the tests. - bpo-34542: Use 3072 RSA keys and SHA-256 signature for test certs and keys. - bpo-34391: Fix ftplib test for TLS 1.3 by reading from data socket. - bpo-34399: Update all RSA keys and DH params to use at least 2048 bits. - bpo-33746: Fix test_unittest when run in verbose mode. - bpo-33901: Fix test_dbm_gnu on macOS with gdbm 1.15: add a larger value to make sure that the file size changes. - bpo-33873: Fix a bug in ``regrtest`` that caused an extra test to run if --huntrleaks/-R was used. Exit with error in case that invalid parameters are specified to --huntrleaks/-R (at least one warmup run and one repetition must be used). - bpo-32663: Making sure the `SMTPUTF8SimTests` class of tests gets run in test_smtplib.py. Build ----- - bpo-34710: Fixed SSL module build with OpenSSL & pedantic CFLAGS. - bpo-34582: Add JUnit XML output for regression tests and update Azure DevOps builds. - bpo-34121: Fix detection of C11 atomic support on clang. - bpo-30345: Add -g to LDFLAGS when compiling with LTO to get debug symbols. Windows ------- - bpo-34770: Fix a possible null pointer dereference in pyshellext.cpp. - bpo-34603: Fix returning structs from functions produced by MSVC - bpo-34225: Ensure INCLUDE and LIB directories do not end with a backslash. - bpo-34006: Revert line length limit for Windows help docs. The line-length limit is not needed because the pages appear in a separate app rather than on a browser tab. It can also interact badly with the DPI setting. - bpo-31546: Restore running PyOS_InputHook while waiting for user input at the prompt. The restores integration of interactive GUI windows (such as Matplotlib figures) with the prompt on Windows. - bpo-30237: Output error when ReadConsole is canceled by CancelSynchronousIo instead of crashing. - bpo-29097: Fix bug where :meth:`datetime.fromtimestamp` erronously throws an :exc:`OSError` on Windows for values between 0 and 86400. Patch by Ammar Askar. macOS ----- - bpo-34370: Have macOS 10.9+ installer builds for 3.7.1rc and 3.6.7rc use a development snapshot of Tk 8.6 (post-8.6.8) to mitigate certain scroller issues seen with IDLE and tkinter apps. - bpo-34405: Update to OpenSSL 1.0.2p for macOS installer builds. - bpo-31903: In :mod:`_scproxy`, drop the GIL when calling into ``SystemConfiguration`` to avoid deadlocks. IDLE ---- - bpo-34548: Use configured color theme for read-only text views. - bpo-1529353: Enable "squeezing" of long outputs in the shell, to avoid performance degradation and to clean up the history without losing it. Squeezed outputs may be copied, viewed in a separate window, and "unsqueezed". - bpo-34047: Fixed mousewheel scrolling direction on macOS. - bpo-34275: Make IDLE calltips always visible on Mac. Some MacOS-tk combinations need .update_idletasks(). Patch by Kevin Walzer. - bpo-34120: Fix unresponsiveness after closing certain windows and dialogs. - bpo-33975: Avoid small type when running htests. Since part of the purpose of human- viewed tests is to determine that widgets look right, it is important that they look the same for testing as when running IDLE. - bpo-33905: Add test for idlelib.stackview.StackBrowser. - bpo-33924: Change mainmenu.menudefs key 'windows' to 'window'. Every other menudef key is lowercase version of main menu entry. - bpo-33906: Rename idlelib.windows as window Match Window on the main menu and remove last plural module name. - bpo-33917: Fix and document idlelib/idle_test/template.py. The revised file compiles, runs, and tests OK. idle_test/README.txt explains how to use it to create new IDLE test files. - bpo-33904: IDLE: In rstrip, rename class RstripExtension as Rstrip - bpo-33907: For consistency and clarity, rename an IDLE module and classes. Module calltips and its class CallTips are now calltip and Calltip. In module calltip_w, class CallTip is now CalltipWindow. - bpo-33856: Add "help" in the welcome message of IDLE - bpo-33839: IDLE: refactor ToolTip and CallTip and add documentation and tests - bpo-33855: Minimally test all IDLE modules. Add missing files, import module, instantiate classes, and check coverage. Check existing files. Tools/Demos ----------- - bpo-32962: python-gdb now catchs ``UnicodeDecodeError`` exceptions when calling ``string()``. - bpo-32962: python-gdb now catchs ValueError on read_var(): when Python has no debug symbols for example. C API ----- - bpo-23927: Fixed :exc:`SystemError` in :c:func:`PyArg_ParseTupleAndKeywords` when the ``w*`` format unit is used for optional parameter.