Skip to content

Let conftest.py skip its directory only with pytest.importorskip - #15143

Open
RonnyPfannschmidt wants to merge 5 commits into
mainfrom
claude/project-thread-zeq5mi
Open

RonnyPfannschmidt wants to merge 5 commits into
mainfrom
claude/project-thread-zeq5mi

Conversation

@RonnyPfannschmidt

@RonnyPfannschmidt RonnyPfannschmidt commented Oct 6, 2026 •

Copy link
Copy Markdown
Member

Requested by Ronny · project thread

🤖 Written by Claude Opus 5.5 via Claude Code for the pytest maintainers; I prompted it, it did the work, I read it.

Closes #15142.

Before: a conftest.py with tables = pytest.importorskip("tables") skipped its directory under pytest, but pytest tests/io/pytables crashed with a raw Skipped traceback. A plain pytest.skip() in a conftest silently skipped the whole package, and pytest.fail() crashed the same way.

After: pytest.importorskip() in a conftest skips its directory however pytest is invoked, including node ids below it. pytest.skip() in a conftest, with or without allow_module_level=True, still skips but warns (PytestRemovedIn10Warning) and becomes an error in pytest 10. pytest.fail()/pytest.xfail() are reported as a conftest import error.

How

  • New internal SkippedModule(Skipped) in _pytest.outcomes, raised by pytest.skip(allow_module_level=True), and its subclass SkippedImport, raised by pytest.importorskip() (including the minversion skip). Both set allow_module_level=True, so python.importtestmodule and anything reading the flag behave as before; their __qualname__ is Skipped, so reports are unchanged.
  • _importconftest: SkippedImport propagates; any other Skipped warns first, then propagates; any other OutcomeException becomes ConftestImportFailure like other exceptions (cause is typed Exception | OutcomeException). The skip is cached per conftest, so the conftest runs once even when it is both an initial conftest and reached by collection.
  • _set_initial_conftests leaves an anchor alone when a conftest on the way skips; the Dir collector re-raises the cached skip and reports it.
  • Session.collect: a skipped collector on the way to an argument counts as matching it, so pytest test_mod.py::test_a into a module-level skip reports the skip instead of ERROR: found no collectors (applies to test modules too).
  • --import-mode=importlib: _import_module_using_spec drops the module from sys.modules when executing it raises, like importlib does.

allow_module_level / SkippedModule assessment

Scan of 53k test files and conftests in 177 popular repos:

usage calls repos
importorskip at module top level 1297 76
skip(..., allow_module_level=True) at module top level 166 41
same, in a helper function called at import (e.g. numpy f2py) 6 5
same, in a package __init__.py 5 2
importorskip at conftest top level 10 6
skip(..., allow_module_level=True) in a conftest 0 0
explicit allow_module_level=False 0 0
reading .allow_module_level on an exception 0 0 (only pytest itself)
constructing Skipped(..., allow_module_level=...) directly 0 0

So the flag is only ever used as pytest.skip(..., allow_module_level=True), and nobody inspects it. That makes it a type in all but name; the subclasses let the conftest loader and importtestmodule ask isinstance() instead of reading an attribute. The allow_module_level parameter and attribute stay for compatibility.

Checklist

  • Tests: TestConftestImportOutcome (test_conftest.py), node-id and SkippedModule/SkippedImport tests (test_skipping.py), importlib sys.modules test (test_pathlib.py). The new tests fail on main.
  • Docs: how-to/skipping.rst "Skipping a whole directory", deprecations.rst entry.
  • Changelog: 15142.bugfix.rst, 15142.deprecation.rst.
  • AI agent credited in Co-authored-by trailer.

Full test suite passes locally (Python 3.13) apart from test_pdb_unittest_postmortem, a pexpect timing test that failed once and passed on 3 reruns; pre-commit run --all-files clean except zizmor, which needs GitHub API access not available here.

🤖 Generated with Claude Code

https://claude.ai/code/session_01GHmGPdXhVMcboqUeE6q62M

A conftest.py that raised an outcome exception while being imported got
different results depending on how pytest was invoked: a skip during
collection skipped the directory (even a plain pytest.skip()), while the
same conftest loaded as an initial conftest crashed with a raw Skipped
traceback, and pytest.fail()/xfail() escaped the conftest import error
handling.

- Add SkippedModule, an internal Skipped subclass raised by
  pytest.skip(allow_module_level=True) and pytest.importorskip().
- A conftest raising SkippedModule skips its directory, also for
  initial conftests; the skip is cached so the conftest runs once.
- A plain pytest.skip() in a conftest still skips but issues a
  PytestRemovedIn10Warning.
- pytest.fail()/xfail() in a conftest become a ConftestImportFailure.
- A node id below a skipped collector reports the skip instead of
  "found no collectors".
- --import-mode=importlib no longer leaves a module that raised while
  executing in sys.modules.

Closes #15142.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GHmGPdXhVMcboqUeE6q62M
@RonnyPfannschmidt RonnyPfannschmidt self-assigned this Oct 6, 2026
@psf-chronographer psf-chronographer Bot added the bot:chronographer:provided (automation) changelog entry is part of PR label Oct 6, 2026
Ronny Pfannschmidt and others added 2 commits October 6, 2026 07:44
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GHmGPdXhVMcboqUeE6q62M
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GHmGPdXhVMcboqUeE6q62M
@RonnyPfannschmidt
RonnyPfannschmidt marked this pull request as ready for review October 7, 2026 05:52
RonnyPfannschmidt and others added 2 commits October 7, 2026 01:52
Only pytest.importorskip() stays supported for skipping a directory from
its conftest.py: it now raises SkippedImport, a SkippedModule subclass,
and any other skip at conftest import warns with PytestRemovedIn10Warning.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GHmGPdXhVMcboqUeE6q62M
@RonnyPfannschmidt RonnyPfannschmidt changed the title Let conftest.py skip its directory only with a module-level skip Let conftest.py skip its directory only with pytest.importorskip Oct 7, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bot:chronographer:provided (automation) changelog entry is part of PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

conftest.py: support importorskip at import, make every other outcome exception a usage error

1 participant