Add lex_commented and CommentedTokenStream to sway_parse (FuelL…

…abs#2123)

* Add `CommentedTokenStream` to `sway_parse`

This doesn't yet collect any comments, but adds the necessary structure
and attempts to preserve the original API and behaviour where possible.

Collecting of comments to be added in a follow-up commit.

* Collect multi-line comments in CommentedTokenStream

* Collect single-line comments in CommentedTokenStream

* Add token_trees and spanned impls for CommentedTokenStream

* Add Spanned impl for CommentedTokenTree. Add comment lexing test.

* Expose `lex_commented` function from root

* Add CommentedTree and CommentedGroup aliases

* Move CommentedTokenTree impl to better location

* Clean up by using CommentedTree type alias where applicable

Co-authored-by: Alex Hansen <[email protected]>
Co-authored-by: Chris O'Brien <[email protected]>

sway-parse/src/lib.rs

@@ -55,8 +55,8 @@ pub use crate::{
     path::{PathExpr, PathExprSegment, PathType, PathTypeSegment, QualifiedPathRoot},
     pattern::{Pattern, PatternStructField},
     statement::{Statement, StatementLet},
-    token::lex,
     token::LexError,
+    token::{lex, lex_commented},
     ty::Ty,
     where_clause::{WhereBound, WhereClause},
 };