TTokenList is a general tokenizer, very fast.
TLineToken breaks a PChar into Lines and filters all comments, very fast too.
Especially useful to get read access to lines of the IDE's EditStream.
For ClassBrowsers etc.

You can compile it with Delphi 2/3 and the C++ Builder.

Disclaimer:
   This is provided as is, expressly without a warranty of any kind.
   You use it at your own risc.

Have fun.

  Martin Waldenburg
  Landaeckerstrasse 27
  71642 Ludwigsburg
  Germany