Unable to Patch

Original link: https://www.kawabangga.com/posts/4706

Recently, I am trying to implement the feature of completely disabling color in pdir2 , making it the same as other cli: when stdout is TTY , the output with color is enabled by default, and when stdout is not TTY, no color is output, that is, there is no color-related escape . code . The pull request is here .

The implementation is relatively simple. In the case of non-TTY, a new Fake Color Render is created, which is directly output without any rendering.

But I encountered a problem when testing: the previous tests were written according to the default output color, and when pytest was running, there was obviously no TTY. My code changed this behavior, causing pytest to run the test, the color all disappeared.

I tried to use patch to solve this problem, set a global fixture, when pytest runs, sys.stdout.isatty() returns True, so that all previous test cases can still pass. Then I found a Very difficult problem to solve.

The place where sys.stdout.isatty() is used is in the global place of a module, and a global variable is set after the judgment, similar to the following:

 if sys.stdout.isatty():     use_color = True

And this module has been imported when pytest is collecting tests, so this use_color cached in the global of the module, even if I go to patch isatty() , it will not actually be called.

So I thought of deleting this cache. Doing this in Python3 is as simple as importlib.reload("pdir") .

However, there is another thorny problem: the pdir module, in fact, no longer exists after the import. In order to let users use: import pdir; pdir(foo) without from pdir import pdir; pdir(foo) , the author uses a trick: that is, in pdir.__init__ , directly put sys.module['pdir'] replaced: sys.modules[__name__] = PrettyDir , the advantage of this is that the import is no longer a module, but a class, which can be called directly. But the disadvantage is that we can no longer find the pdir module, and we cannot use importlib to reload.

In order to make it reload after the patch, I tried a lot of hacks, such as patching its global variables directly, but found that it would fail, because the patch could not find the target, and because the patch could not find the pdir module, it would prompt the class There is no attribute you want to patch; in addition, in the source code, sys.modules[__name__] = PrettyDir saves the original module to a new name before replacing it, but it doesn’t work. This import mechanism seems to have to make the module’s The name corresponds to the module.

After a good night’s sleep, I wondered why pytest runs the module’s init when it collects tests? Why not let me patch first, and then import it?

After reading the code carefully, I found that there are some test.py files, which import pdir at the beginning of the file. At this time, no matter how I patch, the init will be used when running the test later.

So to solve this problem, it is actually very simple:

1. The test file cannot import pdir , it must be imported in the test case, so that the module will not be initialized when collecting tests
2. Then I set a global fixture to patch isatty() . In this way, the logic of execution becomes: collect test (no pdir init) -> global fixture execution, isatty() patch is True -> execute test -> test internal import pdir -> pdir thinks stdout is a TTY

There is another disadvantage of this, that is, the global initialization is completed, and the logic that is not TTY can no longer be tested in the test.

Finally, I saw the deletion logic of sys.modules in the code. It seems that the author has also encountered similar requirements that global variables need to be reinitialized in the test. Put this code into the fixture and find that it works magically. The principle is very simple, that is, after I patch, I need to delete all pdir caches, so that when I import, I will re-init it again. It should be noted that you cannot just delete pdir , pdir.* need to be deleted.

 def remove_module_cache():     for module in (         'pdir',         'pdir.api',         'pdir.constants',         'pdir.format',         'pdir.configuration',         'pdir.color',         'pdir.utils',     ):         try:             del modules[module]         except KeyError:             pass  @pytest.fixture(scope="session") def tty():     with patch("sys.stdout.isatty") as faketty:         faketty.return_value = True         remove_module_cache()         yield

In this way, you only need to use the tty fixture when testing TTY. If you don’t use it, it will not be a TTY by default.

 def test_is_tty(tty):     import pdir     ...  def test_not_tty():     import pdir     ...

To solve this problem, there are some other possible ideas:

1. Make pytest restart a python interpreter for each test (or test file) and it’s completely clean. But I didn’t see that pytest has such a feature
2. Reduce the use of global variables, and judge whether it is the logic of TTY every time it is called. This is to modify the original logic for testing. I don’t like it very much.

The post Unable to Patch first appeared on Kawa Banga! .

This article is reprinted from: https://www.kawabangga.com/posts/4706
This site is for inclusion only, and the copyright belongs to the original author.