Lexical Analyzer Architecture.
(or) How tokens are recognized


The main task of lexical analysis is to read input characters in the code and produce tokens. Lexical analyzer scans the entire source code of the program. It identifies each token one by one. Scanners are usually implemented to produce tokens only when requested by a parser. Here is how this works-
  1. "Get next token" is a command which is sent from the parser to the lexical analyzer.
  2. On receiving this command, the lexical analyzer scans the input until it finds the next token.
  3. It returns the token to Parser.

Lexical Analyzer skips whitespaces and comments while creating these tokens. If any error is present, then Lexical analyzer will correlate that error with the source file and line number.



Share to whatsapp

More Questions from System Software and Compiler Design Module 2

Lexical Analyzer Architecture.
(or) How tokens are recognized
View
Input Buffering in Compiler Design
View
Structure of a Compiler Design - Phases of Compiler
View
The linkage editor
View
With an algorithm, explain pass1 of a linking loader
View
Difference Between Linking Loader vs Linkage Editor
View
Write note on MASM assembler
View