This file defines the C++ lexical analysis tools. It defines the concept of a C++ 'token', also a template which parses tokens from character sources (anything with an iterator), and a few concrete instances of token streams. The token sources defined below use the lexer template, CPP_Token_Stream, to instantiate CPP_Token_Source's. That is, you can write your code using CPP_Tokens to analyze the parsed source. Instead of referring directly to one of the concrete lexers (CPP_Stream_Token_Source, CPP_File_Token_Source, or CPP_String_Token_Source) you can write your code using the abstract lexer base class, CPP_Token_Source. The virtual method call overhead is likely to be small compared to the looping constructs typically found inside the lexer (CPP_Token_Stream<>). More...
#include <fstream>
#include <string>
#include <portable_strstream.h>
Go to the source code of this file.
Classes | |
class | CPP_Buffer_Token_Source |
A source of tokens extracted from a character array. More... | |
class | CPP_File_Token_Source |
A source of CPP_Token's which comes from a named file. More... | |
class | CPP_Stream_Token_Source |
A source of CPP_Token's which comes from an istream. More... | |
class | CPP_String_Token_Source |
struct | CPP_Token |
An object that represents a C++ lexical token. CPP_Token objects are usually obtained from token streams. See CPP_Token_Stream's for an example use. More... | |
struct | CPP_Token_Source |
A generic representation of a stream of CPP_Token. More... | |
class | CPP_Token_Stream< Iterator > |
C++ token stream template (ie a source of CPP_Token objects). More... | |
class | CPP_Token_Stream_Prep |
A CPP_Token_Stream, does not include preprocessor directives. Doing so would greatly complicate any grammer. Instead, preprocessor directives are handled only if desired in an out of band style. To handle preprocessor directives, derive a class from this one (CPP_Token_Stream_Prep) and implement a virtual method which will be invoked by the stream as preprocessor directives are encountered. Note that the '#' and any spaces surrounding it will not be included in the text passed to the virtual method. More... | |
Namespaces | |
namespace | cxxtls |
The namespace that encapsulates most of the functionality in the CXX toolkit. | |
Functions | |
std::ostream & | operator<< (std::ostream &o, CPP_Token const &r) |
output a token to a stream |
This file defines the C++ lexical analysis tools. It defines the concept of a C++ 'token', also a template which parses tokens from character sources (anything with an iterator), and a few concrete instances of token streams. The token sources defined below use the lexer template, CPP_Token_Stream, to instantiate CPP_Token_Source's. That is, you can write your code using CPP_Tokens to analyze the parsed source. Instead of referring directly to one of the concrete lexers (CPP_Stream_Token_Source, CPP_File_Token_Source, or CPP_String_Token_Source) you can write your code using the abstract lexer base class, CPP_Token_Source. The virtual method call overhead is likely to be small compared to the looping constructs typically found inside the lexer (CPP_Token_Stream<>).
Definition in file cpp_token_stream.h.