Skip to content

Commit

Permalink
Correct the declarations of I/O buffers as bytes-based
Browse files Browse the repository at this point in the history
* Correctly declare the parameter annotations as `BinaryIO`, because in fact it works only with byte-based buffers.

* `sys.stdin.read()` is always a string, so remove the Python2-specific check.
    * Correct the patch in the test which incorrectly sets `sys.stdin` to a bytes buffer which it never is.

* `sys.stdout.buffer` always exists, so remove the Python2-specific fallback.
    * Correct the patch in the test which incorrectly sets `sys.stdout` to a bytes buffer which it never is. And fix a matching mistake because this necessitated `stdout.read().decode()`.
  • Loading branch information
oprypin authored Nov 1, 2023
1 parent 2932f18 commit 8e517de
Show file tree
Hide file tree
Showing 3 changed files with 15 additions and 15 deletions.
7 changes: 7 additions & 0 deletions docs/changelog.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,13 @@ All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). See the [Contributing Guide](contributing.md) for details.

## [unreleased]

### Fixed

* Fix type annotations for `convertFile` - it accepts only bytes-based buffers.
Also remove legacy checks from Python 2 (#1400)

## [3.5.1] -- 2023-10-31

### Fixed
Expand Down
15 changes: 4 additions & 11 deletions markdown/core.py
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@
import sys
import logging
import importlib
from typing import TYPE_CHECKING, Any, Callable, ClassVar, Mapping, Sequence, TextIO
from typing import TYPE_CHECKING, Any, BinaryIO, Callable, ClassVar, Mapping, Sequence
from . import util
from .preprocessors import build_preprocessors
from .blockprocessors import build_block_parser
Expand Down Expand Up @@ -387,8 +387,8 @@ def convert(self, source: str) -> str:

def convertFile(
self,
input: str | TextIO | None = None,
output: str | TextIO | None = None,
input: str | BinaryIO | None = None,
output: str | BinaryIO | None = None,
encoding: str | None = None,
) -> Markdown:
"""
Expand Down Expand Up @@ -424,8 +424,6 @@ def convertFile(
input_file.close()
else:
text = sys.stdin.read()
if not isinstance(text, str): # pragma: no cover
text = text.decode(encoding)

text = text.lstrip('\ufeff') # remove the byte-order mark

Expand All @@ -448,12 +446,7 @@ def convertFile(
else:
# Encode manually and write bytes to stdout.
html = html.encode(encoding, "xmlcharrefreplace")
try:
# Write bytes directly to buffer (Python 3).
sys.stdout.buffer.write(html)
except AttributeError: # pragma: no cover
# Probably Python 2, which works with bytes by default.
sys.stdout.write(html)
sys.stdout.buffer.write(html)

return self

Expand Down
8 changes: 4 additions & 4 deletions tests/test_apis.py
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@
from logging import DEBUG, WARNING, CRITICAL
import yaml
import tempfile
from io import BytesIO
from io import BytesIO, StringIO, TextIOWrapper
import xml.etree.ElementTree as etree
from xml.etree.ElementTree import ProcessingInstruction

Expand Down Expand Up @@ -80,8 +80,8 @@ class TestConvertFile(unittest.TestCase):

def setUp(self):
self.saved = sys.stdin, sys.stdout
sys.stdin = BytesIO(bytes('foo', encoding='utf-8'))
sys.stdout = BytesIO()
sys.stdin = StringIO('foo')
sys.stdout = TextIOWrapper(BytesIO())

def tearDown(self):
sys.stdin, sys.stdout = self.saved
Expand Down Expand Up @@ -111,7 +111,7 @@ def testFileObjects(self):
def testStdinStdout(self):
markdown.markdownFromFile()
sys.stdout.seek(0)
self.assertEqual(sys.stdout.read().decode('utf-8'), '<p>foo</p>')
self.assertEqual(sys.stdout.read(), '<p>foo</p>')


class TestBlockParser(unittest.TestCase):
Expand Down

0 comments on commit 8e517de

Please sign in to comment.