Abstract Syntax Cheng-Chia Chen Concrete v s

Abstract Syntax Cheng-Chia Chen

Concrete v. s. Abstract Syntax • Parsing results can generally be represented in two forms: – concrete syntax tree (CST or Parse Tree) – abstract syntax tree (AST or syntax tree) • Concrete syntax trees (parse tree) is a record of the rules (and tokens) used to match some input text. • Abstract syntax tree records the structure of the input and is insensitive to the grammar that produced it. • AST is superior to CST since – CST reflects the structure of a sentence in a grammar • AST reflects the structure of a sentence in the language. • But a language can have a lot of grammars. – AST is generally more compact. – AST is usually more convenient for later analysis and processing. Cheng-Chia Chen 2

Example • Given the grammar : Expr + Term * Factor Number | (E) • and the input ‘ 2 + 5 ‘ : • We should have the following CST and AST: Cheng-Chia Chen 3

Expr Plus. Expr e 1 e 2 Num. Expr : 5 Expr Plus : + Term Bin. Expr op: plus e 1 e 2 Num. Expr : 2 Factor Number : 5 Num. Expr : 5 Number: 2 Possible ASTs for ‘ 2+5’ Cheng-Chia Chen The parse tree for ‘ 2+5’ 4

Abstract Syntax v. s. Concrete Syntax of Expressions Abstract Syntax • E E + E • E E – E • E E * E • E E / E • E id • E num Notes: 1. reflect the meaning of expressions; resulting AST more convenient for later processing 2. ambiguous; impractical to parsing Concrete Syntax (LALR(1)) • E E + F • E E – F • F F * T • F T / T • T id • T num • T (E) Notes: 1. quick parse possible 2. resulting CST is highly redundant Conclusion: generate AST (instead of CST) during concrete syntax parsing Cheng-Chia Chen 5

Building Syntax Tree for Expressions (using Java. CC) • First devise the Concrete Syntax (LL(K)) in EBNF Form. Start Exp Term ( “+” Term | “-” Tem)* Term Factor (“*” Factor | “/” Factor)* Factor <IDENTIFIER> | <INTEGER_LITERAL> | “(“ Exp “)” Cheng-Chia Chen 6

Design the AST classes : Class Hierarchy for expressions • • Exp Plus. Exp Minus. Exp Times. Exp Divide. Exp Identifier Interger. Literal Cheng-Chia Chen or simply a Binary. Exp 7

AST classes for expressions pubic abstract class Exp { pubic abstract int eval() } public class Plus. Exp extends Exp { private Exp e 1, e 2 ; public Plus. Exp(Exp a 1, Exp a 2) { e 1=a 1; e 2=a 2; } public int eval() { return e 1. eval() + e 2. eval(); } } …/* similar to Plus. Exp for class Minus. Exp, times. Exp and Divide. Exp. Omitted here */ public class Identifier extends Exp { pubic String f 0; public Identifier(String a 0) { f 0 = a 0 ; } public int eval() { return symbol. Table. lookup(f 0); }} Cheng-Chia Chen 8

AST classes public class Interger. Literal extends Exp { pubic String f 0; public Identifier(String a 0) { f 0 = a 0 ; } public int eval() { return Integer. parse. Int(f 0); } } Cheng-Chia Chen 9

Add actions to build syntax tree : Using the available AST classes Exp Start() : {Exp e; } { e = Exp() { return e; } } Exp() : {Exp e 1, e 2; } { e 1 = Term() ( “+” e 2 = Term() { e 1 = new Plus. Exp(e 1, e 2) ; } | “-” e 2 = Term() { e 1 = new Minus. Exp(e 1, e 2) ; } )* { return e 1; } } Cheng-Chia Chen 10

Building AST Exp Term() : {Exp e 1, e 2; } { e 1 = Factor() ( “*” e 2 = Factor() { e 1 = new Times. Exp(e 1, e 2) ; } | “/” e 2 = Factor() { e 1 = new Divide. Exp(e 1, e 2) ; } )* { return e 1; } } Exp Factor() : {Token t; Exp e; } { ( t = <IDENTIFIER> { e = new Identifier(t. image); } | t= <INTEGER_LITERAL> { e = new Integer. Literal(t. image); } | “(“ e = Exp() “)” ) { return e; } } Cheng-Chia Chen 11

$Exp. Parser. jj options { STATIC = false; … } PARSER_BEGIN(Exp. Parser) public class$

Exp. Parser. jj options { STATIC = false; … } PARSER_BEGIN(Exp. Parser) public class Exp. Parser { … public static void main( String[] args) { Exp. Parser p = new Exp. Parser(System. in); Exp ast = p. start(); System. out. prinln( “result = “ + ast. eval() ); } Token : { <IDENTIFIER: … > <INTEGER_LITERAL: …> } SKIP : { … } … Cheng-Chia Chen 12

• Tools for automated generation of AST classes definitions from abstract syntax definition – sable. CC version 3. X – JTB • Disadvantage – The syntax tree and relevant classes generated may be less abstract than one could desire. Cheng-Chia Chen 13

ASTs for Mini. Java • at package syntaxtree; Cheng-Chia Chen 14

Abstract Syntax for Mini. Java (I) Package syntaxtree; Program(Main. Class m, Class. Dec. List c 1) Main. Class(Identifier i 1, Identifier i 2, Statement s) --------------abstract class Class. Decl. Simple(Identifier i, Var. Decl. List vl, method. Decl. List m 1) Class. Decl. Extends(Identifier i, Identifier j, Var. Dec. List vl, Method. Decl. List ml) --------------Var. Decl(Type t, Identifier i) Method. Decl(Type t, Identifier I, Formal. List fl, Variable. Decl. List vl, Statement. List sl, Exp e) Formal(Type t, Identifier i) Cheng-Chia Chen 15

Abstract Syntax for Mini. Java (II) abstract class type Int. Array. Type() Boolean. Type() Integer. Type() Indentifier. Type(String s) -------------abstract class Statement Block(Statement. List sl) If(Exp e, Statement s 1, Statement s 2) While(Exp e, Statement s) Print(Exp e) Assign(Identifier i, Exp e) Array. Assign(Identifier i, Exp e 1, Exp e 2) ---------------------Cheng-Chia Chen 16

Abstract Syntax for Mini. Java (III) abstract class Exp And(Exp e 1, Exp e 2) Less. Than(Exp e 1, Exp e 2) Plus(Exp e 1, Exp e 2) Minus(Exp e 1, Exp e 2) Times(Exp e 1, Exp e 2) Not(Exp e) Array. Lookup(Exp e 1, Exp e 2) Array. Length(Exp e) Call(Exp e, Identifier i, Exp. List el) Integer. Literal(int i) True() False() Identifier. Exp(String s) This() New. Array(Exp e) New. Object(Identifier i) ------------------------Identifier(Sting s) --list classes------------Class. Dec. List() Exp. List() Formal. List() Method. Decl. List() Statement. LIst() Var. Decl. List() Cheng-Chia Chen 17

Syntax Tree Nodes - Details package syntaxtree; import visitor. Visitor; import visitor. Type. Visitor; public class Program { public Main. Class m; public Class. Decl. List cl; public Program(Main. Class am, Class. Decl. List acl) { m=am; cl=acl; } public void accept(Visitor v) { v. visit(this); } public Type accept(Type. Visitor v) { return v. visit(this); } } Cheng-Chia Chen 18

Class. Decl. java package syntaxtree; import visitor. Visitor; import visitor. Type. Visitor; public abstract class Class. Decl { public abstract void accept(Visitor v); public abstract Type accept(Type. Visitor v); } Cheng-Chia Chen 19

Class. Decl. Extends. java package syntaxtree; import visitor. Visitor; import visitor. Type. Visitor; public class Class. Decl. Extends extends Class. Decl { public Identifier i; public Identifier j; public Var. Decl. List vl; public Method. Decl. List ml; public Class. Decl. Extends(Identifier ai, Identifier aj, Var. Decl. List avl, Method. Decl. List aml) { i=ai; j=aj; vl=avl; ml=aml; } public void accept(Visitor v) { v. visit(this); } public Type accept(Type. Visitor v) { return v. visit(this); } } Cheng-Chia Chen 20

Statement. List. java package syntaxtree; import java. util. Vector; public class Statement. List { private Vector list; public Statement. List() { list = new Vector(); } public void add. Element(Statement n) { list. add. Element(n); } public Statement element. At(int i) { return (Statement)list. element. At(i); } public int size() { return list. size(); } } Cheng-Chia Chen 21

Package visitor/Visitor. java package visitor; import syntaxtree. *; public interface Visitor { public void visit(Program n); public void visit(Class. Decl. Simple n); public void visit(Var. Decl n); public void visit(Formal n); public void visit(Boolean. Type n); public void visit(Identifier. Type n); public void visit(If n); public void visit(Print n); public void visit(Array. Assign n); public void visit(Less. Than n); public void visit(Minus n); public void visit(Array. Lookup n); public void visit(Call n); public void visit(True n); public void visit(Identifier. Exp n); public void visit(New. Array n); public void visit(Not n); } Cheng-Chia Chen public public public public public void void void void void visit(Main. Class n); visit(Class. Decl. Extends n); visit(Method. Decl n); visit(Int. Array. Type n); visit(Integer. Type n); visit(Block n); visit(While n); visit(Assign n); visit(And n); visit(Plus n); visit(Times n); visit(Array. Length n); visit(Integer. Literal n); visit(False n); visit(This n); visit(New. Object n); visit(Identifier n); 22

AST for statement : X = y. m(1, 4+5) Statement s Identifier(“x”) Identifier. Exp(“y”) Assign (Identifier, Exp) Call(Exp, Identifier, Exp. List) Identifier(“m”) Exp. List: [ Integer. Literal(1) Integer. Literal(4) Cheng-Chia Chen , ] Plus(Exp, Exp) (Integer. Literal(5) 23

Mini. Java : Grammar(I) Program -> Main. Class. Decl * Program(Main. Class, Class. Decl. List) Program Goal() : { Main. Class m; Class. Decl. List cl = new Class. Decl. List(); Class. Decl c; } { m = Main. Class() ( c = Class. Decl() {cl. add. Element(c); })* <EOF> {return new Program(m, cl) } Cheng-Chia Chen 24

Mini. Java : Grammar(II) Main. Class -> class id { public static void main ( String [] id ) { Statement } } Main. Class( Identifier, Statement) Class. Decl -> class id { Var. Decl * Method. Decl * } -> class id extends id { Var. Decl* Method. Decl * } Class. Decl. Simple(…), Class. Dec. Extends(…) Var. Decl -> Type id ; Var. Decl(Type, Identifier) Method. Decl -> public Type id ( Formal. List ) { Var. Decl * Statement* return Exp ; } Method. Decl(Type, Identifier, Formal. List, Var. Decl. List Statement. List, Exp) Cheng-Chia Chen 25

Mini. Java : Grammar(III) Formal. List -> Formal ( “, ” Formal)* Formal -> Type id Formal(Type, Identifier ) Type -> int [] Int. Array. Type() -> boolean -> int -> id Identifier. Type() Cheng-Chia Chen 26

Mini. Java : Grammar(IV) Statement -> { Statement * } -> if ( Exp ) Statement else Statement -> while ( Exp ) Statement -> System. out. println ( Exp ) ; -> id = Exp ; -> id [ Exp ] = Exp ; Exp. List -> Exp (“, ” Exp ) * Cheng-Chia Chen 27

Mini. Java : Grammar(V) Exp -> Exp op Exp -> Exp [ Exp ] -> Exp. length -> Exp. Id ( Exp. List ) -> INTEGER_LITERAL -> true -> false -> id -> this -> new int [ Exp ] -> new id ( ) -> ! Exp -> ( Exp ) Cheng-Chia Chen 28