fix commitment issue with "union nonterminals"

[quasarbright]

example:

#lang racket

(require syntax-spec)

(syntax-spec
 (nonterminal a-expr
   (a))
 (nonterminal b-expr
   (b))
 (nonterminal a-or-b-expr
   a:a-expr
   b:b-expr)
 (host-interface/expression
   (dsl e:a-or-b-expr)
   #''good))

(dsl (b))

the parsing of a-or-b-expr seems to commit to the a-expr and not backtrack and try b-expr once a-expr fails. this naturally comes up when you want to do something like defn-or-expr in a block dsl. for the stlc dsl, I had to inline defns into defn-or-expr to get around this

[quasarbright]

from michael:

Yeah this is maybe more of a misfeature than a bug. It’s the result of straightforwardly compiling nonterminals to syntax-parse. Getting the behavior you want would require backtracking across nonterminals, which would bring up issues of backtracking through macro expansions when the nonterminals allow extension. And also possibly backtracking across binding, if an unbound reference could trigger backtracking. There’s probably a safe subset but we’d need to think hard and maybe implement a static check to recognize it. Would definitely be nice to do eventually, but for now understand matching a production as committed choice before examining subexpressions or checking references are well bound.