initial year-end cleanup.

CHANGES

@@ -8,6 +8,12 @@ maintained as a mature library.  No new features are planned, but
 issues and pull requests for bugs are still welcome. Any changes to the 
 software will be noted here.
 
+01/01/20  Added an install.py script to make it easy to install PLY into
+          virtual environment if you just want to play with it.
+
+01/01/20  Some project reorganization. Moved the preprocessor and ctokens
+          file to the examples directory.  Bumped the version number.
+
 01/19/19  Some improvements to the preprocessor module contributed by
           Rob Reilink.  Issue #195 fixes the evaluation of expressions
           such as #if a != b.   Issue #196 fixes some some issues

CONTRIBUTING.md

@@ -8,8 +8,12 @@ of a bug report.
 Important note: The Github repo for PLY always contains the most
 up-to-date version of the software.  If you want to use the current
 version, you should COPY the contents of the `ply/` directory into
-your own project and use it.  PLY is not maintained in as a package
-installer.
+your own project and use it.  PLY is free software for you to use,
+but it is not maintained as a package that you install using pip
+or similar tools.
+
+
+
 
 
 

Makefile

@@ -1,17 +1,10 @@
 PYTHON ?= python
 
+install:
+	python install.py
+
 test:
 	cd test && $(PYTHON) testlex.py
 	cd test && $(PYTHON) testyacc.py
 
-wheel:
-	$(PYTHON) setup.py bdist_wheel
-
-sdist:
-	$(PYTHON) setup.py sdist
-
-upload: wheel sdist
-	$(PYTHON) setup.py bdist_wheel upload
-	$(PYTHON) setup.py sdist upload
-
-.PHONY: test wheel sdist upload
+.PHONY: install test

README.md

@@ -1,6 +1,6 @@
 # PLY (Python Lex-Yacc)
 
-Copyright (C) 2001-2019
+Copyright (C) 2001-2020
 David M. Beazley (Dabeaz LLC)
 All rights reserved.
 
@@ -88,6 +88,9 @@ import lex
 import yacc
 ```
 
+If you wish, you can use the install.py script to install PLY into
+virtual environment.
+
 PLY has no third-party dependencies. 
 
 The file doc/ply.html contains complete documentation on how to use
@@ -100,10 +103,9 @@ A simple example is found at the end of this document
 
 Requirements
 ============
-PLY requires the use of Python 2.6 or greater.  However, you should
+PLY requires the use of Python 3.6 or greater.  However, you should
 use the latest Python release if possible.  It should work on just
-about any platform.  PLY has been tested with both CPython and Jython.
-It also seems to work with IronPython.
+about any platform.  
 
 Resources
 =========
@@ -120,10 +122,6 @@ The GitHub page for PLY can be found at:
 
 * https://github.com/dabeaz/ply
 
-An old and inactive discussion group for PLY is found at:
-
-* http://groups.google.com/group/ply-hack
-
 Acknowledgments
 ===============
 A special thanks is in order for all of the students in CS326 who
@@ -135,14 +133,6 @@ Elias Ioup did the first implementation of LALR(1) parsing in PLY-1.x.
 Andrew Waters and Markus Schoepflin were instrumental in reporting bugs
 and testing a revised LALR(1) implementation for PLY-2.0.
 
-Special Note for PLY-3.0
-========================
-PLY-3.0 the first PLY release to support Python 3. However, backwards
-compatibility with Python 2.6 is still preserved. PLY provides dual
-Python 2/3 compatibility by restricting its implementation to a common
-subset of basic language features. You should not convert PLY using
-2to3--it is not necessary and may in fact break the implementation.
-
 Example
 =======
 

install.py

@@ -0,0 +1,28 @@
+import sys
+import shutil
+import ply, ply.lex, ply.yacc
+from pathlib import Path
+
+TARGET = "ply"
+
+def main(argv):
+    user = False
+    if len(argv) > 1:
+        if argv[1] != '--user':
+            raise SystemExit('usage: python install.py [--user]')
+        else:
+            user = True
+    site_packages = [p for p in sys.path if p.endswith('site-packages')]
+    path = Path(site_packages[0] if user else site_packages[-1]) / TARGET
+    if path.exists():
+        shutil.rmtree(path)
+    shutil.copytree(TARGET, path)
+    print(f'{TARGET} installed at {path}')
+
+if __name__ == '__main__':
+    main(sys.argv)
+
+
+
+
+

ply/__init__.py

@@ -1,5 +1,5 @@
 # PLY package
 # Author: David Beazley ([email protected])
 
-__version__ = '3.11'
+__version__ = '4.0'
 __all__ = ['lex','yacc']

ply/lex.py

@@ -1,7 +1,7 @@
 # -----------------------------------------------------------------------------
 # ply: lex.py
 #
-# Copyright (C) 2001-2019
+# Copyright (C) 2001-2020
 # David M. Beazley (Dabeaz LLC)
 # All rights reserved.
 #
@@ -33,7 +33,7 @@
 # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
 # -----------------------------------------------------------------------------
 
-__version__    = '3.11'
+__version__    = '4.0'
 __tabversion__ = '3.10'
 
 import re
@@ -43,13 +43,8 @@
 import os
 import inspect
 
-# This tuple contains known string types
-try:
-    # Python 2.6
-    StringTypes = (types.StringType, types.UnicodeType)
-except AttributeError:
-    # Python 3.0
-    StringTypes = (str, bytes)
+# This tuple contains acceptable string types
+StringTypes = (str, bytes)
 
 # This regular expression is used to match valid token names
 _is_identifier = re.compile(r'^[a-zA-Z0-9_]+$')

ply/yacc.py

@@ -1,7 +1,7 @@
 # -----------------------------------------------------------------------------
 # ply: yacc.py
 #
-# Copyright (C) 2001-2019
+# Copyright (C) 2001-2020
 # David M. Beazley (Dabeaz LLC)
 # All rights reserved.
 #
@@ -68,7 +68,7 @@
 import inspect
 import warnings
 
-__version__    = '3.11'
+__version__    = '4.0'
 __tabversion__ = '3.10'
 
 #-----------------------------------------------------------------------------
@@ -93,12 +93,6 @@
 
 pickle_protocol = 0            # Protocol to use when writing pickle files
 
-# String type-checking compatibility
-if sys.version_info[0] < 3:
-    string_types = basestring
-else:
-    string_types = str
-
 MAXINT = sys.maxsize
 
 # This object is a stand-in for a logging object created by the
@@ -3029,7 +3023,7 @@ def get_start(self):
     # Validate the start symbol
     def validate_start(self):
         if self.start is not None:
-            if not isinstance(self.start, string_types):
+            if not isinstance(self.start, str):
                 self.log.error("'start' must be a string")
 
     # Look for error handler
@@ -3115,12 +3109,12 @@ def validate_precedence(self):
                     self.error = True
                     return
                 assoc = p[0]
-                if not isinstance(assoc, string_types):
+                if not isinstance(assoc, str):
                     self.log.error('precedence associativity must be a string')
                     self.error = True
                     return
                 for term in p[1:]:
-                    if not isinstance(term, string_types):
+                    if not isinstance(term, str):
                         self.log.error('precedence items must be strings')
                         self.error = True
                         return

setup.md

@@ -2,19 +2,29 @@
 
 PLY is maintained software, but no longer produces package releases.
 There is no `setup.py` file.  It is not something that you install
-with `pip` or a similar tool. You must COPY the necessary code from
-PLY into your project and take ownership of it.
+with `pip` or a similar tool. PLY is free software which means that
+you are free to COPY the necessary code from PLY into your project and
+use it in any manner that you wish.
+
+If you'd simply like to play around with PLY in a virtual environment
+or install it into your normal Python distribution, use the included
+install.py script:
+
+    $ python install.py
 
 Why this policy? PLY is a highly specialized tool for expert-level
 programmers who are writing parsers and compilers.  If you are writing
 a compiler, there's a good chance that it's part of a substantially
-larger project.  Managing external dependencies (such as PLY) in such
-projects is an ongoing challenge.  However, the truth of the matter is
-that PLY just isn't that big.  All of the core functionality is
-contained in just two files.  PLY has no external dependencies of its
-own.  It changes very rarely.  Plus, there are various customizations
-that you might want to apply to how it works.  So, all things equal,
-it's probably better for you to copy it.  
+larger project.  Managing complexity and external dependencies (such
+as PLY) in such projects is an ongoing challenge.  However, the truth
+of the matter is that PLY just isn't that big.  All of the core
+functionality is contained in just two files.  PLY has no external
+dependencies of its own.  It changes very rarely.  Plus, there are
+various customizations that you might want to apply to how it works.
+So, all things equal, it's probably better for you to copy it.  This
+also protects you in the event that some other project decides to use
+PLY in a different way (or from a different version) than that used
+in your project.
 
 But what about getting all of the latest improvements and bug fixes?
 What improvements? PLY is implementing a 1970s-era parsing algorithm.