NEP 23 — Backwards compatibility and deprecation policy — NumPy Enhancement Proposals

Ralf Gommers <ralf.gommers@gmail.com>
https://mail.python.org/pipermail/numpy-discussion/2021-January/081423.html
In this NEP we describe NumPy’s approach to backwards compatibility, its deprecation and removal policy, and the trade-offs and decision processes for individual cases where breaking backwards compatibility is considered.
NumPy has a very large user base. Those users rely on NumPy being stable and the code they write that uses NumPy functionality to keep working. NumPy is also actively maintained and improved – and sometimes improvements require, or are made easier, by breaking backwards compatibility. Finally, there are trade-offs in stability for existing users vs. avoiding errors or having a better user experience for new users. These competing needs often give rise to long debates and delay accepting or rejecting contributions. This NEP tries to address that by providing a policy as well as examples and rationales for when it is or isn’t a good idea to break backwards compatibility.
In addition, this NEP can serve as documentation for users about how the NumPy project treats backwards compatibility, and the speed at which they can expect changes to be made.
In scope for this NEP are:
Principles of NumPy’s approach to backwards compatibility.
How to deprecate functionality, and when to remove already deprecated functionality.
Decision making process for deprecations and removals.
How to ensure that users are well informed about any change.
Making concrete decisions about deprecations of particular functionality.
NumPy’s versioning scheme.
When considering proposed changes that are backwards incompatible, the main principles the NumPy developers use when making a decision are:
Changes need to benefit more than they harm users.
NumPy is widely used, so breaking changes should be assumed by default to be harmful.
Decisions should be based on how they affect users and downstream packages and should be based on usage data where possible. It does not matter whether this use contradicts the documentation or best practices.
The possibility of an incorrect result is worse than an error or even crash.
When assessing the costs of proposed changes, keep in mind that most users do not read the mailing list, do not notice deprecation warnings, and sometimes wait more than one or two years before upgrading from their old version. And that NumPy has millions of users, so “no one will do or use this” is likely incorrect.
Benefits of proposed changes can include improved functionality, usability and performance, as well as lower maintenance cost and improved future extensibility.
Fixes for clear bugs are exempt from this backwards compatibility policy. However, in case of serious impact on users even bug fixes may have to be delayed for one or more releases. For example, if a downstream library would no longer build or would give incorrect results.
Getting hard data on the impact of a deprecation of often difficult. Strategies that can be used to assess such impact include:
Use a code search engine ([1], [2]) or static ([3]) or dynamic ([4]) code analysis tools to determine where and how the functionality is used.
Test prominent downstream libraries against a development build of NumPy containing the proposed change to get real-world data on its impact.
Make a change on the main branch and revert it before release if it causes problems. We encourage other packages to test against NumPy’s main branch and if that’s too burdensome, then at least to test pre-releases. This often turns up issues quickly.
If the impact is unclear or significant, it is often good to consider alternatives to deprecations. For example, discouraging use in documentation only, or moving the documentation for the functionality to a less prominent place or even removing it completely. Commenting on open issues related to it that they are low-prio or labeling them as “wontfix” will also be a signal to users, and reduce the maintenance effort needing to be spent.
Deprecation warnings are necessary in all cases where functionality will eventually be removed. If there is no intent to remove functionality, then it should not be deprecated. A “please don’t use this for new code” in the documentation or other type of warning should be used instead, and the documentation can be organized such that the preferred alternative is more prominently shown.
shall include the version number of the release in which the functionality was deprecated.
shall include information on alternatives to the deprecated functionality, or a reason for the deprecation if no clear alternative is available. Note that release notes can include longer messages if needed.
shall use DeprecationWarning by default, and VisibleDeprecation for changes that need attention again after already having been deprecated or needing extra attention for some reason.
shall be listed in the release notes of the release where the deprecation is first present.
shall not be introduced in micro (bug fix) releases.
shall set a stacklevel, so the warning appears to come from the correct place.
shall be mentioned in the documentation for the functionality. A .. deprecated:: directive can be used for this.
Examples of good deprecation warnings (also note standard form of the comments above the warning, helps when grepping):
Removal of deprecated functionality:
shall be done after at least 2 releases assuming the current 6-monthly release cycle; if that changes, there shall be at least 1 year between deprecation and removal.
shall be listed in the release notes of the release where the removal happened.
can be done in any minor, but not bugfix, release.
For backwards incompatible changes that aren’t “deprecate and remove” but for which code will start behaving differently, a FutureWarning should be used. Release notes, mentioning version number and using stacklevel should be done in the same way as for deprecation warnings. A .. versionchanged:: directive shall be used in the docume