Build CLI stuck / pip prompt not working with private pypi index #409

sbor23 · 2021-11-24T17:00:46Z

Hi

We're trying to get a build going with a simple pyproject.toml and setuptools als the build backend.
We have a pip.conf that specifies a private pypi index to be used for all installs.

Running python -m build results in the following output being stuck and not moving.

* Creating venv isolated environment...
* Installing packages in isolated environment... (setuptools>=45, wheels, setuptools_scm>=6.2)

Hitting enter twice reveals that the command was blocking the output of the pip install invocation in the isolated environment.
So pip is asking for user and password using the prompt, but the prompt is hidden.

Looking in indexes: [redacted]
User for [redacted]: ERROR: Exception:
Traceback (most recent call last):
  File "/tmp/build-env-ghmd794l/lib/python3.7/site-packages/pip/_internal/cli/base_command.py", line 188, in _main
    status = self.run(options, args)
  File "/tmp/build-env-ghmd794l/lib/python3.7/site-packages/pip/_internal/cli/req_command.py", line 185, in wrapper
    return func(self, options, args)
  File "/tmp/build-env-ghmd794l/lib/python3.7/site-packages/pip/_internal/commands/install.py", line 333, in run
    reqs, check_supported_wheels=not options.target_dir
  File "/tmp/build-env-ghmd794l/lib/python3.7/site-packages/pip/_internal/resolution/legacy/resolver.py", line 179, in resolve
    discovered_reqs.extend(self._resolve_one(requirement_set, req))
  File "/tmp/build-env-ghmd794l/lib/python3.7/site-packages/pip/_internal/resolution/legacy/resolver.py", line 362, in _resolve_one
    abstract_dist = self._get_abstract_dist_for(req_to_install)
  File "/tmp/build-env-ghmd794l/lib/python3.7/site-packages/pip/_internal/resolution/legacy/resolver.py", line 313, in _get_abstract_dist_for
    self._populate_link(req)
  File "/tmp/build-env-ghmd794l/lib/python3.7/site-packages/pip/_internal/resolution/legacy/resolver.py", line 279, in _populate_link
    req.link = self.finder.find_requirement(req, upgrade)
  File "/tmp/build-env-ghmd794l/lib/python3.7/site-packages/pip/_internal/index/package_finder.py", line 901, in find_requirement
    req.name, specifier=req.specifier, hashes=hashes,
  File "/tmp/build-env-ghmd794l/lib/python3.7/site-packages/pip/_internal/index/package_finder.py", line 883, in find_best_candidate
    candidates = self.find_all_candidates(project_name)
  File "/tmp/build-env-ghmd794l/lib/python3.7/site-packages/pip/_internal/index/package_finder.py", line 828, in find_all_candidates
    project_url, link_evaluator=link_evaluator,
  File "/tmp/build-env-ghmd794l/lib/python3.7/site-packages/pip/_internal/index/package_finder.py", line 792, in process_project_url
    html_page = self._link_collector.fetch_page(project_url)
  File "/tmp/build-env-ghmd794l/lib/python3.7/site-packages/pip/_internal/index/collector.py", line 612, in fetch_page
    return _get_html_page(location, session=self.session)
  File "/tmp/build-env-ghmd794l/lib/python3.7/site-packages/pip/_internal/index/collector.py", line 451, in _get_html_page
    resp = _get_html_response(url, session=session)
  File "/tmp/build-env-ghmd794l/lib/python3.7/site-packages/pip/_internal/index/collector.py", line 166, in _get_html_response
    "Cache-Control": "max-age=0",
  File "/tmp/build-env-ghmd794l/lib/python3.7/site-packages/pip/_vendor/requests/sessions.py", line 543, in get
    return self.request('GET', url, **kwargs)
  File "/tmp/build-env-ghmd794l/lib/python3.7/site-packages/pip/_internal/network/session.py", line 421, in request
    return super(PipSession, self).request(method, url, *args, **kwargs)
  File "/tmp/build-env-ghmd794l/lib/python3.7/site-packages/pip/_vendor/requests/sessions.py", line 530, in request
    resp = self.send(prep, **send_kwargs)
  File "/tmp/build-env-ghmd794l/lib/python3.7/site-packages/pip/_vendor/requests/sessions.py", line 650, in send
    r = dispatch_hook('response', hooks, r, **kwargs)
  File "/tmp/build-env-ghmd794l/lib/python3.7/site-packages/pip/_vendor/requests/hooks.py", line 31, in dispatch_hook
    _hook_data = hook(hook_data, **kwargs)
  File "/tmp/build-env-ghmd794l/lib/python3.7/site-packages/pip/_internal/network/auth.py", line 246, in handle_401
    username, password, save = self._prompt_for_password(parsed.netloc)
ValueError: not enough values to unpack (expected 3, got 2)
User for [redacted]: 
Traceback (most recent call last):
  File "/home/stefan/.pyenv/versions/3.7.12/lib/python3.7/site-packages/build/__main__.py", line 373, in main
    args.srcdir, outdir, distributions, config_settings, not args.no_isolation, args.skip_dependency_check
  File "/home/stefan/.pyenv/versions/3.7.12/lib/python3.7/site-packages/build/__main__.py", line 229, in build_package_via_sdist
    sdist = _build(isolation, builder, outdir, 'sdist', config_settings, skip_dependency_check)
  File "/home/stefan/.pyenv/versions/3.7.12/lib/python3.7/site-packages/build/__main__.py", line 140, in _build
    return _build_in_isolated_env(builder, outdir, distribution, config_settings)
  File "/home/stefan/.pyenv/versions/3.7.12/lib/python3.7/site-packages/build/__main__.py", line 108, in _build_in_isolated_env
    env.install(builder.build_system_requires)
  File "/home/stefan/.pyenv/versions/3.7.12/lib/python3.7/site-packages/build/env.py", line 211, in install
    _subprocess(cmd)
  File "/home/stefan/.pyenv/versions/3.7.12/lib/python3.7/site-packages/build/env.py", line 81, in _subprocess
    raise e
  File "/home/stefan/.pyenv/versions/3.7.12/lib/python3.7/site-packages/build/env.py", line 78, in _subprocess
    subprocess.check_output(cmd, stderr=subprocess.STDOUT)
  File "/home/stefan/.pyenv/versions/3.7.12/lib/python3.7/subprocess.py", line 411, in check_output
    **kwargs).stdout
  File "/home/stefan/.pyenv/versions/3.7.12/lib/python3.7/subprocess.py", line 512, in run
    output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['/tmp/build-env-ghmd794l/bin/python', '-Im', 'pip', 'install', '--use-pep517', '--no-warn-script-location', '-r', '/tmp/build-reqs-t6halb98.txt']' returned non-zero exit status 2.

ERROR Command '['/tmp/build-env-ghmd794l/bin/python', '-Im', 'pip', 'install', '--use-pep517', '--no-warn-script-location', '-r', '/tmp/build-reqs-t6halb98.txt']' returned non-zero exit status 2.

Which leads to two questions:

Would it be possible to display the prompt normally, so the user can manually enter the credentials to an actual promp? Because entering myusername<Enter>mypassword<Enter> seems to work just fine, but one has to figure that out first. Also the user is prompted multiple times, leading to multiple "hidden" prompts.

Is there a way to "inject" the pip call somehow to provide the credentials in a non-interactive way?
For example in the CI, we use HTTP basic auth in the form of pip install -i https://username:[email protected]/simple.
Another way could be to somehow pre-configure keyring to be installed and initialized.
Worst case we could probably use .netrc, but at least for local machines we would like to avoid this (see question 1 regarding multiple prompts.)

On a side note, I'm a bit puzzled that this didn't come up in #246 .

The text was updated successfully, but these errors were encountered:

FFY00 · 2021-11-24T18:12:36Z

Thanks for reporting, we should be passing --no-input to pip. How does passing --no-input to pip change the experience in your use case?

sbor23 · 2021-11-25T14:56:11Z

As expected, this leads the the same issue as if the remote was not known, so:

* Creating venv isolated environment...
* Installing packages in isolated environment... (setuptools>=45, setuptools_scm>=6.2, wheel)
Looking in indexes: [redacted]
ERROR: Could not find a version that satisfies the requirement wheel (from versions: none)
ERROR: No matching distribution found for wheel

Traceback (most recent call last):
  File "/home/stefan/projects/pyorgan/venv/lib/python3.10/site-packages/build/__main__.py", line 372, in main
    built = build_call(
  File "/home/stefan/projects/pyorgan/venv/lib/python3.10/site-packages/build/__main__.py", line 229, in build_package_via_sdist
    sdist = _build(isolation, builder, outdir, 'sdist', config_settings, skip_dependency_check)
  File "/home/stefan/projects/pyorgan/venv/lib/python3.10/site-packages/build/__main__.py", line 140, in _build
    return _build_in_isolated_env(builder, outdir, distribution, config_settings)
  File "/home/stefan/projects/pyorgan/venv/lib/python3.10/site-packages/build/__main__.py", line 108, in _build_in_isolated_env
    env.install(builder.build_system_requires)
  File "/home/stefan/projects/pyorgan/venv/lib/python3.10/site-packages/build/env.py", line 212, in install
    _subprocess(cmd)
  File "/home/stefan/projects/pyorgan/venv/lib/python3.10/site-packages/build/env.py", line 81, in _subprocess
    raise e
  File "/home/stefan/projects/pyorgan/venv/lib/python3.10/site-packages/build/env.py", line 78, in _subprocess
    subprocess.check_output(cmd, stderr=subprocess.STDOUT)
  File "/usr/lib64/python3.10/subprocess.py", line 420, in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
  File "/usr/lib64/python3.10/subprocess.py", line 524, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/tmp/build-env-jb75xsz0/bin/python', '-Im', 'pip', 'install', '--use-pep517', '--no-warn-script-location', '--no-input', '-r', '/tmp/build-reqs-ejsueipu.txt']' returned non-zero exit status 1.

ERROR Command '['/tmp/build-env-jb75xsz0/bin/python', '-Im', 'pip', 'install', '--use-pep517', '--no-warn-script-location', '--no-input', '-r', '/tmp/build-reqs-ejsueipu.txt']' returned non-zero exit status 1.

sbor23 · 2022-01-24T13:29:08Z

I guess this is related to: pypa/pip#10665

However, from my experience pip will use the keyring for credentials if available, while build doesn't.

stilley2 · 2022-06-10T23:03:54Z

For anybody looking for a workaround, set the environment variable PIP_NO_INPUT=1 before running build.

charlesdwright · 2022-06-16T22:03:35Z

Nope, that didn't work.

How on earth...

a019647 · 2022-09-01T18:35:06Z

I guess this is related to: pypa/pip#10665

And yet the stack trace and error message don't match at all.

krpatter-intc · 2023-02-09T18:17:46Z

if I did a pull request that added keyring as a dependency would that be acceptable to the devs?

henryiii · 2023-02-09T19:51:11Z

Not as a required dep (see #470 and similar discussions, we are keeping required deps to a minimum).

;

lazarillo mentioned this issue May 22, 2023

Build CLI stuck unless I hit <ENTER> twice #620

Open

ekalosak added a commit to ekalosak/moshi that referenced this issue Aug 9, 2023

fixed make build hanging using the workaround proposed in pypa/build#409

f6bae0f

;

sbor23 mentioned this issue Nov 17, 2023

Error creating pip environment when index requires a login pre-commit/pre-commit#1454

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Build CLI stuck / pip prompt not working with private pypi index #409

Build CLI stuck / pip prompt not working with private pypi index #409

sbor23 commented Nov 24, 2021

FFY00 commented Nov 24, 2021 •

edited

Loading

sbor23 commented Nov 25, 2021 •

edited

Loading

sbor23 commented Jan 24, 2022

stilley2 commented Jun 10, 2022

charlesdwright commented Jun 16, 2022

a019647 commented Sep 1, 2022

krpatter-intc commented Feb 9, 2023

henryiii commented Feb 9, 2023

Build CLI stuck / pip prompt not working with private pypi index #409

Build CLI stuck / pip prompt not working with private pypi index #409

Comments

sbor23 commented Nov 24, 2021

FFY00 commented Nov 24, 2021 • edited Loading

sbor23 commented Nov 25, 2021 • edited Loading

sbor23 commented Jan 24, 2022

stilley2 commented Jun 10, 2022

charlesdwright commented Jun 16, 2022

a019647 commented Sep 1, 2022

krpatter-intc commented Feb 9, 2023

henryiii commented Feb 9, 2023

FFY00 commented Nov 24, 2021 •

edited

Loading

sbor23 commented Nov 25, 2021 •

edited

Loading