tokenize method

  1. @override
List<CodeToken> tokenize(
  1. String code
)
override

Tokenizes code into a flat list of tokens covering the full input.

Implementation

@override
List<CodeToken> tokenize(String code) => _tokenize(
  code,
  keywords: _keywords,
  lineComment: '//',
  blockComment: const ('/*', '*/'),
);