xgo/README.md

The Tengo Language

Tengo is an embeddable script language for Go.

Tengo is fast as it's compiled to bytecode and executed on stack-based VM that's written in native Go.

>> Try Tengo Playground <<

Features

• Simple and intuitive syntax
• Dynamically typed with type coercions
• First-class functions and Closures
• Garbage collected (thanks to Go runtime)
• Easily extendible using customizable types
• Written in pure Go (no CGO, no external dependencies)
• Executable as a standalone language (without writing any Go code)

Benchmark

	fib(35)	fibt(35)	Type
Go	59ms	4ms	Go (native)
Tengo	4,809ms	5ms	VM on Go
Lua	1,752ms	3ms	Lua (native)
go-lua	5,236ms	5ms	Lua VM on Go
GopherLua	5,558ms	5ms	Lua VM on Go
Python	3,132ms	28ms	Python (native)
otto	85,765ms	22ms	JS Interpreter on Go
Anko	99,235ms	24ms	Interpreter on Go

fib(35) is a function to compute 35th Fibonacci number, and, fibt(35) is the tail-call version of the same function. You can see all the code used for this test in tengobench.

Tengo Syntax in 5 Minutes

Tengo supports line comments (//...) and block comments (/* ... */).

/* 
  multi-line block comments 
*/

a := 5 // line comments

Run in Playground

Tengo is a dynamically typed language, and, you can initialize the variables using := operator.

a := 1984 		// int
b := "aomame"		// string
c := -9.22		// float
d := true		// bool
e := '九'		// char
f := [1, false, "foo"]	// array
g := {			// map
    h: 439,
    i: 12.34,
    j: [0, 9, false]
}
k := func(l, m) {	// function
    return l + m
}

Run in Playground

After the variable is initialized, it can be re-assigned different value using = operator.

a := 1928		// int
a = "foo"		// string
f := func() {
    a := false		// 'a' is defined in the function scope
    a = [1, 2, 3]	// and thus does not affect 'a' in global scope.
}
a == "foo" 		// still "foo"

Run in Playground

Type is not directly specified, but, you can use type-coercion functions to convert between types.

s1 := string(1984)  // "1984"
i2 := int("-999")   // -999
f3 := float(-51)    // -51.0
b4 := bool(1)       // true
c5 := char("X")     // 'X'

Run in Playground

See Variable Types for more details on the variable types.

You can use the dot selector (.) and indexer ([]) operator to read or write elements of arrays, strings, or maps.

["one", "two", "three"][1]	// == "two"

m := {
    a: 1,
    b: [2, 3, 4],
    c: func() { return 10 }
}
m.a				// == 1
m["b"][1]			// == 3
m.c()				// == 10
m.x = 5				// add 'x' to map 'm'
//m.b[5] = 0			// but this is an error: index out of bounds

Run in Playground

For sequence types (string, bytes, array), you can use slice operator ([:]) too.

a := [1, 2, 3, 4, 5][1:3]	// == [2, 3]
b := [1, 2, 3, 4, 5][3:]	// == [4, 5]
c := [1, 2, 3, 4, 5][:3]	// == [1, 2, 3]
d := "hello world"[2:10]	// == "llo worl"

Run in Playground

In Tengo, functions are first-class citizen, and, it also supports closures, functions that captures variables in outer scopes. In the following example, the function returned from adder is capturing base variable.

adder := func(base) {
    return func(x) { return base + x }	// capturing 'base'
}
add5 := adder(5)
nine := add5(4)		// == 9

Run in Playground

For flow control, Tengo currently supports if-else, for, for-in statements.

// IF-ELSE
if a < 0 {
    // ...
} else if a == 0 {
    // ...
} else {
    // ...
}

// IF with init statement
if a := 0; a < 10 {
    // ...
} else {
    // ...
}

// FOR
for a:=0; a<10; a++ {
    // ...
}

// FOR condition-only (like WHILE in other languages)
for a < 10 {
    // ...
}

// FOR-IN
for x in [1, 2, 3] {		// array: element
    // ...
}
for i, x in [1, 2, 3] {		// array: index and element
    // ...
} 
for k, v in {k1: 1, k2: 2} {	// map: key and value
    // ...
}

An error object is created using error function-like keyword. An error can have any types of value and the underlying value of the error can be accessed using .value selector.

err1 := error("oops")   // error with string value
err2 := error(1+2+3)    // error with int value
if is_error(err1) {     // 'is_error' builtin function
    err_val := err1.value   // get underlying value 
}  

Run in Playground

You can load other scripts as modules using import expression.

Main script:

mod1 := import("./mod1") // assuming mod1.tengo file exists in the current directory 
                         // same as 'import("./mod1.tengo")' or 'import("mod1")'
mod1.func1(a)            // module function 
a += mod1.foo            // module variable
//mod1.foo = 5           // error: module variables are read-only

mod1.tengo file:

func1 := func(x) { print(x) }
foo := 2

Basically, import expression returns all the global variables defined in the module as a Map-like value. One can access the functions or variables defined in the module using . selector or ["key"] indexer, but, module variables are immutable.

Also, you can use import to load the standard libraries.

math := import("math")
a := math.abs(-19.84) // == 19.84

Embedding Tengo in Go

To execute Tengo code in your Go codebase, you should use Script. In the simple use cases, all you need is to do is to create a new Script instance and call its Script.Run() function.

import "github.com/d5/tengo/script"

var code = `
reduce := func(seq, fn) {
    s := 0
    for x in seq { fn(x, s) }
    return s
}

print(reduce([1, 2, 3], func(x, s) { s += x }))
`

func main() {
    s := script.New([]byte(code))
    if _, err := s.Run(); err != nil {
        panic(err)
    }
}

If you want to compile the source script once and execute it multiple times, you can use Script.Compile() function that returns Compiled instance.

import (
	"fmt"

	"github.com/d5/tengo/script"
)

func main() {
	s := script.New([]byte(`a := b + 20`))

	// define variable 'b'
	_ = s.Add("b", 10)

	// compile the source
	c, err := s.Compile()
	if err != nil {
		panic(err)
	}

	// run the compiled bytecode
	// a compiled bytecode 'c' can be executed multiple without re-compiling it
	if err := c.Run(); err != nil {
		panic(err)
	}

	// retrieve value of 'a'
	a := c.Get("a")
	fmt.Println(a.Int())
}

In the example above, a variable b is defined by the user before compilation using Script.Add() function. Then a compiled bytecode c is used to execute the bytecode and get the value of global variables. In this example, the value of global variable a is read using Compiled.Get() function.

If you need the custom data types (outside Tengo's primitive types), you can define your own struct that implements objects.Object interface (and optionally objects.Callable if you want to make function-like invokable objects).

import (
	"errors"
	"fmt"

	"github.com/d5/tengo/compiler/token"
	"github.com/d5/tengo/objects"
	"github.com/d5/tengo/script"
)

type Counter struct {
	value int64
}

func (o *Counter) TypeName() string {
	return "counter"
}

func (o *Counter) String() string {
	return fmt.Sprintf("Counter(%d)", o.value)
}

func (o *Counter) BinaryOp(op token.Token, rhs objects.Object) (objects.Object, error) {
	switch rhs := rhs.(type) {
	case *Counter:
		switch op {
		case token.Add:
			return &Counter{value: o.value + rhs.value}, nil
		case token.Sub:
			return &Counter{value: o.value - rhs.value}, nil
		}
	case *objects.Int:
		switch op {
		case token.Add:
			return &Counter{value: o.value + rhs.Value}, nil
		case token.Sub:
			return &Counter{value: o.value - rhs.Value}, nil
		}
	}

	return nil, errors.New("invalid operator")
}

func (o *Counter) IsFalsy() bool {
	return o.value == 0
}

func (o *Counter) Equals(t objects.Object) bool {
	if tc, ok := t.(*Counter); ok {
		return o.value == tc.value
	}

	return false
}

func (o *Counter) Copy() objects.Object {
	return &Counter{value: o.value}
}

func (o *Counter) Call(args ...objects.Object) (objects.Object, error) {
	return &objects.Int{Value: o.value}, nil
}

var code = []byte(`
arr := [1, 2, 3, 4]
for x in arr {
	c1 += x
}
out := c1()`)

func main() {
	s := script.New(code)

	// define variable 'c1'
	_ = s.Add("c1", &Counter{value: 5})

	// compile the source
	c, err := s.Run()
	if err != nil {
		panic(err)
	}

	// retrieve value of 'out'
	out := c.Get("out")
	fmt.Println(out.Int()) // prints "15" ( = 5 + (1 + 2 + 3 + 4) )
}

As an alternative to using Script, you can directly create and interact with the parser, compiler, and, VMs directly. There's no good documentation yet, but, check out Script code if you are interested.

Tengo CLI Tool

Although Tengo is designed as an embedded script language for Go, it can be compiled and executed as native binary using tengo tool.

Installing Tengo Tool

To install tengo tool, run:

go get github.com/d5/tengo/cmd/tengo

Compiling and Executing Tengo Code

You can directly execute the Tengo source code by running tengo tool with your Tengo source file (*.tengo).

tengo myapp.tengo

Or, you can compile the code into a binary file and execute it later.

tengo -c -o myapp myapp.tengo   # compile 'myapp.tengo' into binary file 'myapp'
tengo myapp                     # execute the compiled binary `myapp`	

Tengo REPL

You can run Tengo REPL if you run tengo with no arguments.

tengo

Roadmap

v0. (Current)

Things are experimental, and, the focus is on the core language features, stability, basic interoperability, and the performance optimization.

v1. Tengo as a Script Language

This will be the first versioned release, and, the main goal for v1 is to make Tengo as a fast embeddable script language for Go, which means Tengo will be comparable to other Go-based script languages such as Starlark, Lua VMs, and other interpreters.

• Interoperability with Go code
• More language features such as bound methods and switch-case statements

v2. Tengo as a Standalone Language

• Language-level concurrency support
• Tengo Standard Libraries
• Native executables compilation
• More language features