* Python3: Update code to work with python3

[earboxer]

Close #18

Note that dependencies are now that fontforge needs to be installed with python3 bindings (--enable-python-scripting=3), and ruamel.yaml is needed as the yaml library.

[13rac1]

Awesome!

"ruamel.yaml is needed as the yaml library." Why?

[earboxer]

Hmm.. I'm not sure. I used it in another project, so I guess I thought it needed to be changed. I can try switching it back to pyyaml.

[earboxer]

Okay, it works fine with PyYaml. I've also added a requirements.txt to this project so you can just pip3 install -r requirements.txt to get the dependencies.

[zachdecook]

I pushed an update to make it compatible with either python 2 or 3 (whichever is in the path first as python). Presumably that would also be the one for which fontforge python bindings are set up for.
I haven't tested this yet.

[trinitronx]

I was trying to build eosrei/twemoji-color-font, and realized this now needs Python3 on Ubuntu 20.04. I ran 2to3 against the repo and it found some things to fix, but then it tried to escape all the \U and \u unicode sequences that were needed. After reverting that, I noticed this PR and realized I should just try merging it into my branch to build. It seemed to work, but it threw what looked to be some warnings. I'm not sure if they really matter:

cp assets/svg-bw/* build/svg-bw/
SCFBuild/bin/scfbuild -c scfbuild.yml -o build/TwitterColorEmoji-SVGinOT.ttf --font-version="13.0.0"
INFO:scfbuild.builder:Creating a new font
INFO:scfbuild.builder:Adding glyphs and ligatures
INFO:scfbuild.builder:Generating intermediate font file
Lookup subtable contains unused glyph four making the whole subtable invalid
Lookup subtable contains unused glyph numbersign making the whole subtable invalid
Lookup subtable contains unused glyph zero making the whole subtable invalid
Lookup subtable contains unused glyph three making the whole subtable invalid
Lookup subtable contains unused glyph one making the whole subtable invalid
Lookup subtable contains unused glyph two making the whole subtable invalid
Lookup subtable contains unused glyph uE0067 making the whole subtable invalid
Lookup subtable contains unused glyph nine making the whole subtable invalid
Lookup subtable contains unused glyph six making the whole subtable invalid
Lookup subtable contains unused glyph asterisk making the whole subtable invalid
Lookup subtable contains unused glyph uE0067 making the whole subtable invalid
Lookup subtable contains unused glyph five making the whole subtable invalid
Lookup subtable contains unused glyph seven making the whole subtable invalid
Lookup subtable contains unused glyph uE0067 making the whole subtable invalid
Lookup subtable contains unused glyph eight making the whole subtable invalid
Lookup subtable contains unused glyph four making the whole subtable invalid
Lookup subtable contains unused glyph numbersign making the whole subtable invalid
Lookup subtable contains unused glyph zero making the whole subtable invalid
Lookup subtable contains unused glyph three making the whole subtable invalid
Lookup subtable contains unused glyph one making the whole subtable invalid
Lookup subtable contains unused glyph two making the whole subtable invalid
Lookup subtable contains unused glyph uE0067 making the whole subtable invalid
Lookup subtable contains unused glyph nine making the whole subtable invalid
Lookup subtable contains unused glyph six making the whole subtable invalid
Lookup subtable contains unused glyph asterisk making the whole subtable invalid
Lookup subtable contains unused glyph uE0067 making the whole subtable invalid
Lookup subtable contains unused glyph five making the whole subtable invalid
Lookup subtable contains unused glyph seven making the whole subtable invalid
Lookup subtable contains unused glyph uE0067 making the whole subtable invalid
Lookup subtable contains unused glyph eight making the whole subtable invalid
INFO:scfbuild.builder:Reading intermediate font file
INFO:scfbuild.builder:Adding SVGinOT SVG files
INFO:scfbuild.builder:Saving output file: build/TwitterColorEmoji-SVGinOT.ttf
INFO:scfbuild.builder:Done!

Other than that... it seems like it works without the usual Python stacktraces when trying to run on Python3.

[trinitronx]

Considering Python 3 support, I should add one last set of details to consider:

• The shebang line: #!/usr/bin/env python did not work to use Python3 on Ubuntu 20.04 / Focal Fossa
• I had to change this locally to: #!/usr/bin/env python3
• The python-is-python3 package could remedy this situation, but it's not installed by default and could affect other things on the system.
  • For example: inkscape-0.92.5-1ubuntu1 still depends on python2
  • inkscape is required to build eosrei/twemoji-color-font
  • scfbuild is also required to build this font... but would now need python3
  • Therefore, we'd need to update inkscape to Python 3 also! (probably needs to happen anyway sometime soon)
• For dependencies:
  • Systems with Python 3 should either run: pip3 install -r scfbuild/requirements.txt
  • OR: A possible Debian package of this project would now need Build-Depends: on the python3-* package variants of these dependencies. (recommended)

Python 3 Migration

Arch Linux Details:

It appears that /usr/bin/python should work on this distro (2020-06-10 rolling release). /usr/bin/python links to python3 and is shipped with version: 3.8.3

Also according to the following info on ArchWiki:

To install the latest version of Python 3, install the python package.
To get the latest version of Python 2, install the python2 package.

Python 2 will happily run alongside Python 3. You need to specify python2 in order to run this version.

Any program requiring Python 2 needs to point to /usr/bin/python2, instead of /usr/bin/python, which points to Python 3. To do so, open the program or script in a text editor and change the first line. The line will show one of the following:

#!/usr/bin/env python

or

#!/usr/bin/python

In both cases, just change python to python2 and the program will then use Python 2 instead of Python 3.

References

• Python 3 on Arch Linux

Debian / Ubuntu Details:

Based on some quick research on the state of Python 3 in Debian & Ubuntu,
I found this:

Python 2 is officially EoL as of January 1st, 2020 no new bug reports,
fixes, or changes will be made to Python 2, and Python 2 is no longer supported.

Remaining packages in Ubuntu which require Python 2.7 have been updated to use
/usr/bin/python2 as their interpreter, and /usr/bin/python is not present by
default on any new installs.

python-is-python3 package, for a /usr/bin/python pointing to python3 instead.

What this does not mean:

• /usr/bin/python will point to Python 3. No, this is not going to happen (unless PEP 394 advocates otherwise, which is doubtful for the foreseeable future). /usr/bin/python and /usr/bin/python2 will point to Python 2.7 and /usr/bin/python3 will point to the latest supported Python 3 version.
• Python 2 will be removed from the archive. No, this is not going to happen. We expect Python 2.7 to remain supported and available in Ubuntu for quite a long time, given that PEP 373 promises upstream bug fix maintenance support until 2020. It would be nice if we could demote Python 2 to universe, but that's currently problematic for technical reasons relating to multi-Python version support in Debian/Ubuntu.

So, just using #!/usr/bin/env python will not work to set Python3 on a fresh Ubuntu 20.04 install "out of the box". In fact, because python2 is not installed by default anymore, this shebang line won't work at all without the python-is-python3 package, which is also not installed by default.

References:

• Sunsetting Python 2
• Ubuntu Focal Fossa (20.04) Release Notes: Python3 by default
• Python 3 on Ubuntu
• Python on Ubuntu

Gentoo Details:

The shebang line will work on this distro! The /usr/bin/python points to v3.6.5 on Gentoo Base v2.4.1 (VM built 2019-09-30). On a recent VM build of Gengoo Base v2.6, /usr/bin/python is v3.7.7 (VM built 2020-06-14). So everything should work well on this distro!

References:

• Python 3.7 to become the default target
• Gentoo Wiki: Python

Official Python PEP recommendations:

We're well into the End of Life of Python2, according to PEP 373:

The End Of Life date (EOL, sunset date) for Python 2.7 has been moved five years into the future, to 2020.

Based on the distro-specific inconsistencies still existing with /usr/bin/python, it's probable that package maintainers (e.g.: Debian / Ubuntu) would have to patch (e.g.: use quilt, gbp pq) or apply custom RegEx replacement (e.g. sed / perl) during post-install scripts to set the shebang line appropriately for the system.

When packaging third party Python scripts, distributors are encouraged to change less specific shebangs to more specific ones. This ensures software is used with the latest version of Python available, and it can remove a dependency on Python 2. The details on what specifics to set are left to the distributors; though. Example specifics could include:

• Changing python shebangs to python3 when Python 3.x is supported.
• Changing python shebangs to python2 when Python 3.x is not yet supported.
• Changing python3 shebangs to python3.8 if the software is built with Python 3.8.

At the very least, this upstream project should probably think about documenting the choice in shebang line so users, new developers and/or package maintainers don't get confused or bitten by the hidden complexities across different distros & platforms.

References:

• PEP 373
• PEP 394

[zachdecook]

scfbuild requires your fontforge installation to be built with the binding for the current python version. I know at least that Ubuntu 18.04 was shipping than as for python2, so this PR doesn't break that: it's assuming whatever is python can import fontforge.
However, if you do have fontforge bindings for python3, you can easily fix this problem by symlinking python3 as python earlier in your $PATH.
e.g.
sudo ln -s $(which python3) /usr/local/bin/python

[zachdecook]

(Or simply disregard the shebang like python3 bin/scfbuild)

[trinitronx]

However, if you do have fontforge bindings for python3, you can easily fix this problem by symlinking python3 as python earlier in your $PATH.
e.g.
sudo ln -s $(which python3) /usr/local/bin/python

Sorry, I didn’t mean to dump so much info in a confusing way as to say that there are not many valid solutions (which, including this one there are)

I think the main “executive summary” I was able to consolidate from all this was that it will probably work fine on most targeted distros that have moved to python3. For those like Debian (and also probably RHEL / CentOS), the shebang line becomes a packager’s responsibility to rewrite for that platform.

The Python PEP recommendations state it well:

When packaging third party Python scripts, distributors are encouraged to change less specific shebangs to more specific ones. This ensures software is used with the latest version of Python available, and it can remove a dependency on Python 2.

[13rac1]

I'm working on this now. I'm going to leave Python2 support in the code, but am changing the startup script to refer directly to Python3. Thanks for all your work on this!

#!/usr/bin/env python3

[13rac1]

Works well. I've decided to default to python3 in the shebang due to the Py2 EOL. Thank you @earboxer for figuring this out! :D