Metalinguistic

Parameter Passing
Metacircular

Macros

Outline
Data and Program Structure

Metalinguistic Abstraction
Metalinguistic Abstraction
(SICP 4.1 - 4.2)
Lecture IV

Ahmed Rezine
Linkopings Universitet

TDDA69, VT 2014

Metalinguistic

Metalinguistic

Metalinguistic Abstraction

Semantics of Programming Languages

Problem: No programming language is suitable to describe all kinds of complex

systems
Solution: Create a new language to help describe, reason and solve the kind of
problems at hand. This process is called metalinguistic abstraction
Examples:
Language of constraint networks with connectors and
constraints to capture equations and value propagation
Language of circuits with wires and function boxes to describe
events and signal propagation

Metacircular

Macros

Parameter Passing

There are three main approaches to give rigorous meanings to computer

programs :
Operational: describe how a valid program is interpreted as
sequences of computational steps.
Axiomatic: Predicate logic is used to describe what happens with the memory, the
program counter, etc. E.g. Hoare logic.
Denotational: constructs mathematical objects (e.g.. partial
functions) to describe the meanings of the expressions in the
language
See the Programming Theory course (TDDD47) for more on the Semantics of
programming languages.

Metalinguistic

Metacircular

Macros

Parameter Passing

Metalinguistic

Metacircular

Evaluators

Figure : The usual way of

programming

perform, when applied on an arbitrary expression of the

programming language, the required steps in order to evaluate the

Parameter Passing

Evaluators (cont.)

Interpreters or evaluators of a programming language are procedures that

Macros

expression
Method Assume computer D can evaluate expressions in language P1
Define a procedure in P1 that computes how one can evaluate
expressions in language P2
Computer D can now evaluate expressions in language P2
P1 is called the host language
P2 is called the embedded language
All languages can be regarded as evaluators of some particular
languages

Metalinguistic
Passing

Metacircular

Macros

Parameter

Universal Machines

The idea that an evaluator of a programming language can emulate the

evaluator of any other language has deep consequences

It suggests that what can be computed is independent of the

underlying language or machine and has to do with a notion of

Figure : Embedding a language

Metalinguistic

Metacircular

Macros

Parameter Passing

Evaluators (cont.)

computability

Alan Turing, in his 1936 paper, introduced a simple model, now called
Turing machines, and argued that any effective process can be
formulated as a program of such a machine (Church-Turing thesis).

We will go through several implementations of evaluators

We will discuss decisions and choices that are common and well

He also built a universal Turing Machine that evaluates programs of Turing

machines, and showed that there are problems that cannot be computed by
Turing machines (and hence by any effective process)

understood when it comes to the process of creating evaluators

Evaluators are not only used to implement new programming

languages. They can be regarded as a way to abstract several steps
as used in traditional programming.

Metacircular
A Metacircular Evaluator

Metalinguistic

Macros

Parameter Passing

Metalinguistic

Metacircular

Outline

Macros

Parameter Passing

Metacircular evaluators

An evaluator that is written in the same language that it evaluates is said to be metacircular evaluator.

We will build a metacircular evaluator for Scheme, where :

the "host language is Scheme, and

the "embedded language is %Scheme

We will build %Scheme with the environment model of evaluation

John McCarthy (the father of Lisp) already described in 1960 the eval-function of Lisp in Lisp

Metalinguistic

Metacircular

Implementing a Scheme evaluator

Macros

Parameter Passing

Metalinguistic

Metacircular

Macros

Parameter Passing

Implementing a Scheme evaluator (cont.)

Our metacircular evaluator is a formulation of the environment model of

Expression,

evaluation:

environment

In order to evaluate a compound expression other that a special form,

we evaluate the subexpressions and apply the value of the

A procedure to arguments

operator to the values of the operands

In order to apply a compound procedure to a set of arguments, we
eval

evaluate the body of the procedure in a new environment. This new

An expression in an environment

environment is obtained by extending the environment part of the

Procedure,

procedure object with a frame in which the formal parameters of the

arguments

procedure are bound to the arguments.

Observe how evaluation requires application that requires

Figure : The eval-apply cycle (SICP 4.1)

evaluation.
Metalinguistic

Metacircular

Macros

Parameter Passing

eval (cont.)

eval

Types of expressions and corresponding actions of eval:

Primitive expressions:

self-evaluating expressions: 3, #t, etc. The expression itself

variables are looked up in the environment for their values

The eval procedure:

Special forms:

takes as arguments an expression and an environment

is structured as a case analysis for the types of expressions the language can

Quoted expressions: return the quoted expressions

Assignment to (and definition of) a variable: evaluate the new value and
modify (create) the binding of the variable
If statement: evaluate the consequent if the predicate is true, otherwise
evaluate the alternative
Lambda expression: build a procedure object
Begin expression: evaluate expressions in their order
cond is transformed into a nest of if expressions and evaluated
Combinations:

evaluate

an abstract syntax is used to test the type of expressions and to select

its parts. This allows us to use the same evaluator with a different syntax
for the procedures

Metalinguistic

apply

Metacircular

Macros

Parameter Passing

Metalinguistic

evaluate operator and operands and pass resulting procedure and

arguments to apply.

Metacircular

Macros

Parameter Passing

Some issues to consider

The apply procedure :

A procedure and the list of arguments to which the procedure should

Maintain, keep track and maintain the environment

be applied
Depending on the type of the procedure, applies it on the arguments:
Primitive procedure: Apply the procedure directly (this is a frontier with
the host language).
Compound procedure: Sequentially evaluate the expressions in the
body of the procedure in an extended environment.

Evaluation order, special forms

Evaluation of the arguments, and its order

Add, change and look-up variables

Metacircular

Macros

Differences with the book

Syntax

Sicp defines eval and apply. In the labs, these are renamed to
eval-%scheme and apply-%scheme
Sicp defines an evaluator for Scheme in Scheme. It defines the eval and apply
procedures to unroll the evaluation-application cycle.
To clarify the difference between the host (Scheme) and the embedded
language (%Scheme), and to avoid name collisions, the labs prefix the
special forms of %Scheme with % and define eval-%scheme and apply%scheme to unroll the evaluation-application cycle.

Metalinguistic
Passing

Metacircular

Macros

Parameter

<numb>

<var>

(%quote <datum>)

(%set! <var> <expr>)

(%define <var> <expr>)

(%begin <expr>...<expr>

(%if <expr> <expr> <expr>) (<expr><expr> ... <expr>)

(%define (<fname> <var>...<var>) <expr>...<expr>)

eval-%scheme

(%lambda (<par>...<par>)
<expr>...<expr>)

(%cond
(<expr> <expr> ... <expr>)
(<expr> <expr> ... <expr>)
(... )
(%else <expr> ... <expr>))

Metalinguistic

Metacircular

Macros

Parameter Passing

(define (eval-%scheme exp env)

(cond ((self-evaluating? exp) exp)
((variable? exp) (lookup-variable-value exp env))
((quoted? exp) (text-of-quotation exp)) ((assignment? exp)
(eval-assignment exp env)) ((definition? exp) (evaldefinition exp env)) ((if? exp) (eval-if exp env))
((lambda? exp)
(make-procedure (lambdaparameters exp) (lambda-body
exp) env ) )
((begin? exp)
(eval-sequence (begin-actions exp) env))
((cond? exp) (eval-%scheme (cond->if exp) env))
((application? exp)
(apply-%scheme (eval-%scheme (operator exp) env) (listof-values (operands exp) env)))
(else
(error eval- % s cheme
"Unknown expression type: ~s" exp))))

Some aspects to remark in eval-%scheme

Altering variable values:
(define (eval-assignment exp env) (set-variable-value!
(assignment-variable exp) (eval-% scheme
(assignment-value exp) env) env )
ok )

Defining variables:

(define (eval-definition exp env)

(define(definition-variable exp)
variable!
(eval-%scheme
(definition-value exp) env) env )
ok )

Metacircular

Some aspects to remark in eval-%scheme

Macros

apply-%scheme

When to evaluate which expressions:

(define (eval-if exp env)
(if (true? (eval-%scheme (if-predicate exp) env)) (eval%scheme (if-consequent exp) env) (eval-%scheme (ifalternative exp) env)))

Packaging the environment in procedure objects

(define (make-procedure parameters body env)
(list procedure parameters body env))

Evaluating the arguments

(define (list-of-values exps env)
(if (no-operands? exps)
()
(cons (eval-%scheme (first-operand exps) env)
(list-of-values (rest-operands exps) env))))

Metalinguistic
Passing

Metacircular

Macros

Relying on Scheme in %Scheme

A table relates %Schemes primitive values to Scheme (SICP
p.382)

Parameter

(define (apply-%scheme procedure arguments)

(cond ((primitive-procedure? procedure)
(apply-primitive-procedure procedure argument s))
((c ompound-procedure ? procedure)
(eval-se qu ence
(procedure-body procedure)
(extend-environment
(procedure-parameters procedure)
arguments
(procedure-environment procedure))))
(else
(error apply-%scheme
"Unknown procedure type: ~s" procedure))))

Metalinguistic
Passing

Metacircular

Macros

Parameter

Metalinguistic

( de primitive-procedures
f i (lis
( lis
%car car)
Parameter Passing
Metacircular
t (lis
t
%cdr cdr)
t
(lis
%cons cons)
t
(lis
%null? null?)
t
(lis
%list list)
t
(lis
% eq? eq?)
t
(lis
%+ +)
t
(lis
%> >)
t
(lis
%newline newline)
t
(lis
%display display)))
t

Figure : Representing the environment

(define (apply-primitive-procedure proc
args) (apply (primitive-implementation
proc) args))

The table can be augmented ...

Macros

Parameter Passing

Metalinguistic

(define
(lookup-variable-value
env)environment
Finding
variable valuesvar
in the
(define (env-loop
env)
Metacircular
Macros
(define (scan vars vals)
(cond ((null? vars)
Defining
a variable in an environment
(env-loop (enclosing-environment env)))
((eq? var (mcar vars))
(mcar vals))
(else (scan (mcdr vars) (mcdr vals)))))
(if (eq? env the-empty-environment)
(error lookup-variable-value
"Unbound variable: ~s" var)
(define (define-variable!
var val env)
(let ((frame (first-frame
env)))
(let ((frame
env)))frame)
(scan(first-frame
(frame-variables
(define (scan(frame-values
vars vals) frame)))))
(cond env))
((null? vars)
(env-loop
(add-binding-to-frame! var val frame))
((eq? var (mcar vars))
(set-mcar! vals val))
(else (scan (mcdr vars) (mcdr vals)))))
(scan (frame-variables frame)
(frame-values frame))))

Metalinguistic

(define
(set-variable-value!
var in
val
Modifying
variable value
anenv)
environment
(define (env-loop env)
(define (scan vars vals)
(cond ((null? vars)
(env-loop (enclosing-environment env)))
((eq? var (mcar vars))
(set-mcar! vals val))
(else (scan (mcdr vars) (mcdr vals)))))
(if (eq? env the-empty-environment)
(error set-variable-value!
"Unbound variable: ~s" var)
(let ((frame (first-frame env)))
(scan (frame-variables frame)
(frame-values frame)))))
(env-loop env))

Parameter Passing

Metalinguistic
Passing

Metacircular

Macros

Parameter

Approaches to extend a language with new constructions

Add new primitive-procedures by extending the list of procedures that are mapped to procedures in the underlying Scheme, or written in %Scheme and evaluated with
eval-%scheme in Scheme.

For special forms, add a case in eval-%scheme with corresponding syntactic predicates and selection functions and evaluation procedures.

Extend the global environment with the new constructions in %Scheme

Allow the usage of macros in order to define new expressions using previously defined ones. These are then expended by the evaluator before they get also evaluated

Alternative parameter-passing models can allow certain constructions.

The first three approaches involve modifying the evaluator.

Metalinguistic

Examples of extending %Scheme in the labs (starting

from lab 3)
Parameter Passing

Metacircular

Metalinguistic

Metacircular

Macros

Metalinguistic

Parameter Passing

Macros

Parameter Passing

Macros

Parameter Passing

Outline

dolist A corresponding structure exists in Common Lisp.

(%dolist (var listexpr res) expl ... expm)

The semantics is as follows. The variable var is defined as
a local variable in a new environment. The expression
listexpr is evaluated and results in a list. For each value in
the list:

var is bound to the value

evaluate expressions exprl to exprm

Macros for Augumenting a Language

After that, the expression res is evaluated and its value is

returned as the value of the whole %dolist construction.
Finally the binding to the variable var is to be removed.

Metalinguistic

Metacircular

Macros

Parameter Passing

Macros for Augmenting a Language

Metalinguistic

Metacircular

Macros for Extending a Language (cont.)

Common Lisp allows macro procedures that give the possibility to write
procedures that transform a Lisp expression into another before it is evaluated.

A two steps process :

This is possible because programs are represented in data-structures that can

be manipulated (in fact as lists).
Macros expansion can take place:

During interpretation, the expansion is directly follwed of the evaluation of the

resulting expression

Can take the form of a pre-processing while reading a program, or before

compilation.

Scheme makes use of templates to transform the expressions.

These are used in the stream package of lab 2.

(
c
(
i

2
0

1.

Expand the expression to another new expression

2.

Evaluate the new expression

n
d
m
a(
e
v

(
l 2
l0
(
)
)
r
expansi
o l 2 onl
20
) 0 giv
)
luatio
n
es

(else 20))
gives

In lab 3,4

Macros for Extending a Language (cont.)

Metalinguistic

Metacircular

Macros

Parameter Passing

Metalinguistic

Metacircular

Different ways to
call

Outline

Macros

Parameter Passing

procedures

Issues and problems to think about:

Should the macro expansion take place directly after loading the
program as a pre-processing before evaluation or directly under
evaluation?
How will the expanded expression look like, how many times will
the expression be expanded? what about side effects?

Read thoroughly the code (sections 4.1.1-4.1.7). You will use the code in
almost all the remaining labs. Pay attention to how the environment is
managed.
load %Scheme, test and experiment
implement different mechanisms to augument the language

What assumptions can be made, what environment? is + the +

directly in the interpreter with new special forms

operation?
More to be looked at in the labs.

by adding macros and defininf new macros

Call-by-value: Arguments are evaluated before the procedure is applied to them

Metalinguistic Abstraction

-: Can result in un-needed, sometimes non-terminating, computations

+: easy to implement, efficient

Call-by-name: Arguments are sent, not-evaluated, to the procedure and only evaluated when needed

+: arguments are not evaluated if not needed

-: more complex to implement to take care of non-evaluated

Parameter Passing

Call-by-need: Similar to call-by-name, but each expression is evaluated at most once

+: more efficient than call-by-name

-: even more complex to implement than call-by-name

expressions

Metalinguistic

Issues to think about

Parameter Passing

Metacircular

Metalinguistic

Metacircular

Macros
Macros

Parameter Passing

Evaluation strategies and calling mechanisms

Parameter Passing

Metalinguistic

Metalinguistic

Metacircular

In lab 5

Macros

In lab 5 (cont.)

How to maintain the correct environment (thunks)? when to evaluate

Applicative order ^ Call-by-value

the arguments ? for example:

(f (%+ x y))
(%
define
(%
define
(%
define
(%
define

(
(
f
g
(
h
x

x
)
y
)
z
)
1
)

(g x) )
(h y))
(%+ z 3))
(%define y
2)

Normal order ^ Call-by-name

x in f bound to thunk:(%+ x y), GE y in

g bound to thunk: x, new env. z in h
bound to thunk: y, new env.

in order to evaluate (%+ z 3) y must be

Normal order + thunk optimization Call-by-need.

In functional programming:
Eager evaluation: evaluates as much as possible without
regards to whether the expression is needed or not
Lazy evaluation: only evaluate expressions if needed

evaluated to get z x must be evaluated to

get y (%+ x y) must be evaluated to get x

1.

Go through section 4.2 in Sicp.

The new model will allow us to define new functions that used to require

2.

You can download a version of %Scheme with possibilities for

special forms of macros: streams? dolist?

having call-by-name and call-by-need.

Extra assignment: In lecture 3 we used call-by-name to implement a sum

Study the code and follow the books description. It describes

procedure. Can it be realized in %Scheme? try, some additional changes

how un-evaluated arguments can be saved with their

are required, which? perform the changes in %Scheme.

environment in thunks, with memoization for call-by-need.

For next lecture: Check whether your favorite language allows for

You will then change %Scheme so that one can specify how

constructions that we can describe in this model. Code blocks in Ruby for

each parameter is passed.

example ?

3.

4.

(%define (f a (b %lazy) c (d %lazy-memo))

... )

Parameter Passing