Assignment 2: Lexical Analysis & State Diagrams
Lexical Analysis (Overview):
Lexical analysis is the process of converting a sequence of characters from source program into a sequence of tokens. A lexical analyzer is a pattern matcher for character strings:
A lexical analyzer is a “front-end” for the parser
A lexical analyzer identifies substrings of the source program that belong together.
A program which performs lexical analysis is termed as a lexical analyzer (lexer), tokenizer or scanner.
There are three approaches to building a lexical analyzer:
1.Write a formal description of the tokens and use a software tool that constructs a table-driven lexical analyzer from such a description
2.Design a state diagram that describes the tokens and write a program that implements the state diagram
3.Design a state diagram that describes the tokens and hand construct a table-driven implementation of the state diagram
Overview: For this assignment, you will design a state diagram that describes the tokens and write a program that implements the state diagram which recognizes a specific form of C-based comments: those that begin with /* and end with */
1.Design/present a state diagram which recognizes this form of C-based comments: those that begin with /* and end with */
2.In the programming language of your choice, write and test code which implements the state diagram of this problem (recognizes the C style comments described above)
3.Discussion report (written formally in full sentences):
a.Introduce the assignment/the problem to be solved
b.Briefly discuss the steps you took and decisions you made to start and solve this problem
c.Answer: did the state diagram help you design the code? Briefly explain your answer.
d.Concludes the report: wrap up the report (summary of assignment and what you did to meet the requirements
4.Deliverables: You will submit a single Word or PDF document which includes:
a.The written discussion elements found in #3 (in the above section)
b.Your state diagram (all components of the diagram must be clearly readable)
c.Video of your program .(the working URL to the video should be listed in the report document; at the end of the report):
ii. Showing the program run from start to finish with the output of the solution for three test cases
d.List (working URLs are sufficient) all resources used to complete this work (you are expected to explore at least two sources)
e.Text of your code copied at the end of the report
a.Compiling errors: Your solution must compile
b.Readability. Your code should meet basic readability principles:
i.Separate each component/part with white space.
ii.Align everything in a meaningful way.
c.Comments: you must include enough comments to ensure that the code is described in sufficient detail such that anyone else looking at the code can easily understand the design and the purpose of the code.
d.All code and work associated with this assignment is your own work/written by you.
Resources/reading on compilers, lexical analysis, etc.