antlr-3.0.1源码

Early Access ANTLR v3 ANTLR 3.0.1 August 13, 2007 Terence Parr, parrt at cs usfca edu ANTLR project lead and supreme dictator for life University of San Francisco INTRODUCTION [Java, C, Python, C# targets are available; others coming soon] Welcome to ANTLR v3! I've been working on this for nearly 4 years and it's finally ready! I have lots of features to add later, but this will be the first set. You should use v3 in conjunction with ANTLRWorks: http://www.antlr.org/works/index.html The book will also help you a great deal (printed May 15, 2007); you can also buy the PDF: http://www.pragmaticprogrammer.com/titles/tpantlr/index.html See the getting started document: http://www.antlr.org/wiki/display/ANTLR3/FAQ+-+Getting+Started You also have the examples plus the source to guide you. See the new wiki FAQ: http://www.antlr.org/wiki/display/ANTLR3/ANTLR+v3+FAQ and general doc root: http://www.antlr.org/wiki/display/ANTLR3/ANTLR+3+Wiki+Home Please help add/update FAQ entries. If all else fails, you can buy support or ask the antlr-interest list: http://www.antlr.org/support.html I have made very little effort at this point to deal well with erroneous input (e.g., bad syntax might make ANTLR crash). I will clean this up after I've rewritten v3 in v3. v3 is written in v2 at the moment. Per the license in LICENSE.txt, this software is not guaranteed to work and might even destroy all life on this planet: THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. ---------------------------------------------------------------------- EXAMPLES ANTLR v3 sample grammars (currently for C, C#, Java targets): http://www.antlr.org/download/examples-v3.tar.gz contains the following examples: LL-star, cminus, dynamic-scope, fuzzy, hoistedPredicates, island-grammar, java, python, scopes, simplecTreeParser, treeparser, tweak, xmlLexer. Also check out Mantra Programming Language for a prototype (work in progress) using v3: http://www.linguamantra.org/ ---------------------------------------------------------------------- What is ANTLR? ANTLR stands for (AN)other (T)ool for (L)anguage (R)ecognition and was originally known as PCCTS. ANTLR is a language tool that provides a framework for constructing recognizers, compilers, and translators from grammatical descriptions containing actions. Target language list: http://www.antlr.org/wiki/display/ANTLR3/Code+Generation+Targets ---------------------------------------------------------------------- How is ANTLR v3 different than ANTLR v2? See "What is the difference between ANTLR v2 and v3?" http://www.antlr.org/wiki/pages/viewpage.action?pageId=719 See migration guide: http://www.antlr.org/wiki/display/ANTLR3/Migrating+from+ANTLR+2+to+ANTLR+3 ---------------------------------------------------------------------- How do I install this damn thing? Just untar and you'll get: antlr-3.0.1/README.txt (this file) antlr-3.0.1/LICENSE.txt antlr-3.0.1/src/org/antlr/... antlr-3.0.1/lib/stringtemplate-3.0.jar (3.0.1 needs 3.0) antlr-3.0.1/lib/antlr-2.7.7.jar antlr-3.0.1/lib/antlr-3.0.1.jar Then you need to add all the jars in lib to your CLASSPATH. Please see the FAQ http://www.antlr.org/wiki/display/ANTLR3/ANTLR+v3+FAQ ---------------------------------------------------------------------- CHANGES INCOMPATIBILITY WARNING -- templates have changed; must regen output from grammars. Runtime libraries have also changed. Debug event listener interface has changed also. 3.0.1 - August 13, 2007 [See target pages on the wiki for more information on the non-Java targets] August 7, 2007 * added escaping of double quotes in DOTTreeGenerator July 22, 2007 * fixed dynamic scope implementation in lexers. They were not creating new scope entries on the stack. Unsupported feature! July 19, 2007 * implemented new unique ID; GC was causing non unique hash codes. Debugging tree grammars was messing up. July 17, 2007 * Added line/charposition to node socket events and event dump so we have more info during tree parsing. Only works if your tree adaptor returns a value Token object from getToken(treenode) with line/col set. Refactored consumeNode/LN to use deserializeNode(). * Fixed mismatched tree node exceptions; for imaginary nodes, it said "missing null". Now prints the token type we found. * Cleaned up exception stuff. MismatchedTreeNodeException was setting line/col, but only RecognitionException should do that. * If imaginary token gets a mismatch, there is no line info. Search backwards in stream if input node stream supports to find last node with good line/col info. E.g., ANTLRv3Tree.g: node from after line 156:72 mismatched tree node: EOA expecting <UP> which used to be: ANTLRv3Tree.g: node from line 0:0 mismatched tree node: null expecting <UP> * mismatched tree node exceptions were not sent to the debug event stream. Due to a type being slightly different on recoverFromMismatchedToken() in DebugTreeParser. Was calling BaseRecognizer version not subclass. Now we get: 9459: Recognition exception MismatchedTreeNodeException(0!=0) * List labels were not allowed as root nodes in tree rewrites like ^($listlabel ...). Had to add a template to AST.stg: /** Gen ^($label ...) where label+=... */ rewriteTokenListLabelRefRoot ::= rewriteTokenLabelRefRoot July 16, 2007 * fixed nextNode in RewriteRuleSubtreeStream was dup'ing too much, screwing up debug event stream. Also there was a bug in how the rewrite tree stream stuff decided to dup nodes. * fixed bug in LT for tree parsing; text was not transmitted properly; only single words worked. * made decision for rule put line/col on colon not first token of first alt. * remote ProxyToken now emits token index for easier debugging when looking at AW's event stream. For example, the @5 here is the token index: 31 Consume hidden [ /<64>,channel=99,30:7, @5] * same is true for consume nodes now: 25586 Consume node [')'/, <44>, 4712040,@1749] 25 When debugging tree parsers, it helps to track errors when you know what corresponding input symbol created this tree node. * Changed debug events associated with trees quite a bit. Passes nodes around now rather than text, type, unique IDs etc... Mostly affects internal stuff. Target developers will have some work in their runtime to do to match this change. :( BUT, there is only a slight tweak in the Dbg.stg and ASTDbg.stg templates. Interface just didn't make sense as is. If you turn on debugging, and want to track a node creation, you want the node pointer not its ID, text, etc... Added ProxyTree for passing across socket. Has line/charpos and tokenIndex July 15, 2007 * added null ptr protection in CommonTreeAdaptor. July 14, 2007 * null child in TreeAdaptor does nothing now. Changed interface and implementation. Changed DebugTreeAdaptor to not fire events on null add as well. July 12, 2007 * added get method for the line/col to DFA map in Grammar.java July 7, 2007 * fixed wrong order of test for exceptions in Lexer.getErrorMessage() June 28, 2007 * Added ability to set the port number in the constructor for the debug parser. June 5, 2007 * Changed (hidden) option -ver