WIP+RFC: How to treat quoting and escaping on the command line? #1109
Conversation
|
|
||
| @unittest.expectedFailure | ||
| def test_singlequoted_semicolons(self): | ||
| base = "python -c 'import alot; print(alot)'; echo 'done'" |
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
lucc
changed the title
|
Sorry to keep this hanging. @lucc I guess this is still relevant is it? I generally don't mind the changes and adding tests is certainly valuable. Could you rebase?
Well yes: it is used by the ui at the point where commandline input is interpreted. So this means in particular if you use |
Currently the command line can not contain any quoted semicolons. The python shlex module that is used for splitting the command string does two things with quotes: 1. it interprets them in the sense that everything between quotes is not split into tokens 2. it removes them from the generated tokens To prevent the second we do escape quotes before splitting the command string with shlex but that implicitly prevents the first point as well.
No caller did use these function arguments so they are not really needed.
lucc
force-pushed
the
fix/split_commandline
branch
from
d400ce1 to
e60fcaa
Compare
|
I rebased so we can better discuss this. There was some subtile issue with these changes but I have to think about this code thoroughly again to remember it. |
I was trying to execute
:call "import alot;ui.notify(alot.__version__)"and it did not work because the semicolon was used to split up the command line. This lead to the observation that our command line parsing code needs some hardening regarding quoting and escaping of special chars (actually only ";"). (Note that I have since solved my original problem, see newest entries in the wiki.)The tests I added show what I believe the command line parsing should do. Do you agree or differ? Please also post more
base = '...'; expected = ['...']examples that should also be put in the tests. I will gradually update the commits. Once we have finished the initial discussion what the parser should do I will turn this into a proper PR (you probably should wait until then when you want to do full reviews).The relevant part of the shlex docs is here. Some recent issues and PRs that also dealt with the command line parsing are #579, #985, #986 although none of them seems directly related.
The current solution for the first test is not complete. It doesn't work with single quotes and backslashes in quotes and nested quotes are not handled. In general this path (the current solution of doing some special
str.replacecalls before callingshlex.shlex) seems cumbersome in the long run and difficult to get right for all cases. I had an idea for another way of solving the issue in general: I could try to not setshlex.shlex.whitespaceandshlex.shlex.whitespace_splitto anything but the default. This will split at normal whitespace and we can then later on "split" the list of strings at entries that are just one semicolon. The lists for each individual command could then be joined with whitespace again. This would potentially change the whitespace so the question for @pazz is: Is this function used to process any strings where the amount of whitespace should not be changed (subject lines, email addresses, ...)?