Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Run flake8 without creating physical files #1698

Closed
nbirillo opened this issue Sep 29, 2022 · 8 comments
Closed

Run flake8 without creating physical files #1698

nbirillo opened this issue Sep 29, 2022 · 8 comments

Comments

@nbirillo
Copy link

nbirillo commented Sep 29, 2022

Run flake8 without creating physical files

Hello! Can I use flake8 in a Python project via API? The documentation says: Flake8 3.0.0 presently does not have a public, stable Python API. I would like to run flake8 on a dataset to get code quality violations for each code snippet, but I don't want to create physical files with code since I have millions of code snippets. I currently found that flake8 uses FileChecker but didn't find other checkers, e.g. InMemoryChecker. Is it true that my use case is not supported?

And if it is true, do I understand correctly that if I implement my own сhecker, this will be enough to solve my problem?

The list of the plugins which I would like to use are:

flake8==3.9.0

# flake8 plugins
flake8-bugbear==21.4.3
flake8-builtins==1.5.3
flake8-comprehensions==3.4.0
flake8-eradicate==1.0.0
flake8-import-order==0.18.1
flake8-plugin-utils==1.3.1
flake8-polyfill==1.0.2
flake8-return==1.1.2
flake8-spellcheck==0.24.0
mccabe==0.6.1
pep8-naming==0.11.1
wps-light==0.15.2
flake8-broken-line==0.3.0
flake8-string-format==0.3.0
flake8-commas==2.0.0
cohesion==1.0.0
radon==4.5.0
@asottile
Copy link
Member

FileChecker isn't a supported api either -- the api lives in flake8.api.legacy though it's not great

your best bet is probably to call flake8 - with a subprocess and pass your file contents as stdin. or to use a NamedTemporaryFile and pass it that way

duplicate of #140

@nbirillo
Copy link
Author

@asottile How can I pass my file content as stdin?
I found the following function:

def is_using_stdin(paths: list[str]) -> bool:
    """Determine if we're going to read from stdin.

    :param paths:
        The paths that we're going to check.
    :returns:
        True if stdin (-) is in the path, otherwise False
    """
    return "-" in paths

That means that I need to pass just "-" as the path argument, isn't. But If I do it, I got the error: FileNotFoundError: [Errno 2] No such file or directory: 'stdin'

My Python code:

p = run(['flake8', '-'], stdout=PIPE, input=code, encoding='ascii')

@asottile
Copy link
Member

I can't see your screen. show your full output -- don't hide anything, don't edit it -- otherwise I can only guess at your problem

here's my framework for asking questions if you need some explicit pointers about how to ask a useful, helpful question

@nbirillo
Copy link
Author

I just asked how can I use flkae8 with stdin, because if you noted this way, you probably know how I can do it. It would be great if you share this knowledge

My code:

from subprocess import run, PIPE

code = u"""\
def is_using_stdin(paths: list[str]) -> bool:
    \"\"\"Determine if we're going to read from stdin.

    :param paths:
        The paths that we're going to check.
    :returns:
        True if stdin (-) is in the path, otherwise False
    \"\"\"
    return "-" in paths
"""

p = run(['flake8', '-'], stdout=PIPE, input=code, encoding='ascii')
print(p.returncode)
print(p.stdout)

My output:

Traceback (most recent call last):
  File "/Users/Anastasiia.Birillo/PycharmProjects/tmp/main/venv/bin/flake8", line 8, in <module>
    sys.exit(main())
  File "/Users/Anastasiia.Birillo/PycharmProjects/tmp/main/venv/lib/python3.8/site-packages/flake8/main/cli.py", line 22, in main
    app.run(argv)
  File "/Users/Anastasiia.Birillo/PycharmProjects/tmp/main/venv/lib/python3.8/site-packages/flake8/main/application.py", line 363, in run
    self._run(argv)
  File "/Users/Anastasiia.Birillo/PycharmProjects/tmp/main/venv/lib/python3.8/site-packages/flake8/main/application.py", line 351, in _run
    self.run_checks()
  File "/Users/Anastasiia.Birillo/PycharmProjects/tmp/main/venv/lib/python3.8/site-packages/flake8/main/application.py", line 264, in run_checks
    self.file_checker_manager.run()
  File "/Users/Anastasiia.Birillo/PycharmProjects/tmp/main/venv/lib/python3.8/site-packages/flake8/checker.py", line 323, in run
    self.run_serial()
  File "/Users/Anastasiia.Birillo/PycharmProjects/tmp/main/venv/lib/python3.8/site-packages/flake8/checker.py", line 307, in run_serial
    checker.run_checks()
  File "/Users/Anastasiia.Birillo/PycharmProjects/tmp/main/venv/lib/python3.8/site-packages/flake8/checker.py", line 589, in run_checks
    self.run_ast_checks()
  File "/Users/Anastasiia.Birillo/PycharmProjects/tmp/main/venv/lib/python3.8/site-packages/flake8/checker.py", line 494, in run_ast_checks
    for (line_number, offset, text, _) in runner:
  File "/Users/Anastasiia.Birillo/PycharmProjects/tmp/main/venv/lib/python3.8/site-packages/flake8_plugin_utils/plugin.py", line 75, in run
    self._load_file()
  File "/Users/Anastasiia.Birillo/PycharmProjects/tmp/main/venv/lib/python3.8/site-packages/flake8_plugin_utils/plugin.py", line 87, in _load_file
    with open(self._filename, 'rb') as f:
FileNotFoundError: [Errno 2] No such file or directory: 'stdin'
1

@asottile
Copy link
Member

right so whatever flake8_plugin_utils is does not read files correctly -- it should use the lines parameter to get the contents -- this isn't a flake8 issue and leading with that to begin with would have helped me identify that sooner rather than an additional round of back-and-forth

@nbirillo
Copy link
Author

Sorry for the stupid question, but what do you mean by using the lines parameter? How can I transfer it?
And sorry for wasting time on my problem

@asottile
Copy link
Member

looks like thy already fixed it -- afonasev/flake8-plugin-utils@1e07ff4

you can pip install 'flake8-plugin-utils>=1.3.2' probably to make that error go away

@nbirillo
Copy link
Author

It works, thank you a lot for the help!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants