--- layout: default ---
This page is the authority on what Jsonnet programs should do. It defines Jsonnet lexing and syntax. It describes which programs should be rejected statically (i.e. before execution). Finally, it specifies the manner in which the program is executed, i.e. the JSON that is output, or the runtime error if there is one.
The specification is intended to be terse but precise. In particular, it illuminates all the subtleties and edge cases in order to allow fully-compatible language reimplementations and tools. The specification employs some standard theoretical computer science techniques, namely type systems and big step operational semantics. If that's not your cup of tea, then see the more discussive description of Jsonnet behavior in the tutorial.
A Jsonnet program is UTF-8 encoded text. The file is a sequence of tokens, separated by optional
whitespace and comments. Whitespace consists of space, tab, newline and carriage return. Tokens
are lexed greedily. Comments are either single line comments, beginning with a #
or a
//
, or block comments beginning with /*
and terminating at the first
*/
encountered within the comment.
Some identifiers are reserved as keywords, thus are not in the set id:
assert
else
error
false
for
function
if
import
importstr
in
local
null
tailstrict
then
self
super
true
"
and ending with the first subsequent non-quoted
"
'
and ending with the first subsequent non-quoted
'
@"
and ending with the first subsequent
non-quoted "
@'
and ending with the first subsequent
non-quoted '
|||
, followed by optional whitespace and a new-line.
The next line must be prefixed with some non-zero length whitespace W. The block ends at the
first subsequent line that does not begin with W, and it is an error if this line does not
contain some optional whitespace followed by |||
. The content of the string is the
concatenation of all the lines that began with W but with that prefix stripped. The line
ending style in the file is preserved in the string.Double- and single-quoted strings are allowed to span multiple lines, in which case whatever
dos/unix end-of-line character appears in the string. They both understand the following escape
characters: "'\bfnrt0
which have their standard meanings, as well as
\uXXXX
for hexadecimal unicode escapes.
Verbatim strings eschew all of the normal string escaping, including hexidecimal unicode escapes.
Every character in a verbatim string is processed literally, with the exception of doubled
end-quotes. Within a verbatim single-quoted string, ''
is processed as '
,
and a verbatim double-quoted string, ""
is processed as "
.
In the rest of this specification, the string is assumed to be canonicalized into a sequence of unicode codepoints with no record of the original quoting form as well and any escape characters removed.
{}[],.();
!$:~+-&|^=<>*/%
Subject to the following rules, which will cause the lexing to terminate with a shorter token:
//
is not allowed in an operator/*
is not allowed in an operator|||
is not allowed in an operator+-~!
The notation used here is as follows: { } denotes zero or more repetitions of a sequence of tokens,
and [ ] represents an optional sequence of tokens. This is not to be confused with { }
and [ ]
Which represent tokens in Jsonnet itself.
Note that although the lexer will generate tokens for a wide range of operators, only a finite set are currently parseable, the rest being reserved for possible future use.
expr ∈ Expr | ::= |
null |
true |
false |
self |
$ |
string |
number
|
| |
{
objinside
}
| |
| |
[
[ expr { , expr } [ , ]
]
| |
| |
[
expr
[ , ]
forspec compspec
]
| |
| |
expr
.
id
| |
| |
expr
[
expr
[ : [ expr
[ : [ expr ] ] ] ]
]
| |
| |
expr
[
: [ expr
[ : [ expr ] ] ] ]
]
| |
| |
super
.
id
| |
| |
super
[
expr
]
| |
| |
expr
(
[ args ]
)
| |
| | id | |
| |
local
bind
{
,
bind
}
;
expr
| |
| |
if
expr
then
expr
[ else
expr ]
| |
| | expr binaryop expr | |
| | unaryop expr | |
| |
expr
{
objinside
}
| |
| |
function
(
[ params ]
)
expr
| |
| |
assert
;
expr
| |
| |
import
string
| |
| |
importstr
string
| |
| |
error
expr
|
objinside | ::= |
member { , member } [ , ]
|
| |
{ objlocal , }
[
expr
]
:
expr
[ { , objlocal } [ , ] ]
forspec compspec
| |
member | ::= | objlocal | assert | field |
field ∈ Field | ::= |
fieldname
[ + ] h
e
|
| |
fieldname
(
[ params ]
)
[ + ] h
e
| |
h ∈ Hidden | ::= |
: | :: | :::
|
objlocal | ::= |
local bind
|
compspec ∈ CompSpec | ::= | { forspec | ifspec } |
forspec | ::= |
for
id
in
expr
|
ifspec | ::= |
if
expr
|
fieldname | ::= |
id |
string |
[
e
]
|
assert | ::= |
assert expr [ : expr ]
|
bind ∈ Bind | ::= |
id
=
e
|
| |
id
(
[ params ]
)
=
e
|
args | ::= |
e { , e } { , id = expr }
[ , ]
|
| |
id = expr { , id = expr }
[ , ]
|
params | ::= |
id { , id } { , id = expr }
[ , ]
|
| |
id = expr { , id = expr }
[ , ]
|
binaryop | ::= |
* |
/ |
% |
+ |
- |
<< |
>> |
< |
<= |
> |
>= |
== |
!= |
& |
^ |
| |
&& |
||
|
unaryop | ::= |
- |
+ |
! |
~
|
The abstract syntax by itself cannot unambiguously parse a sequence of tokens. Ambiguities are
resolved according to the following rules, which can also be overridden by adding parenthesis
symbols ()
.
Everything is left associative. In the case of assert
, error
,
function
, if
, import
, importstr
, and
local
, ambiguity is resolved by consuming as many tokens as possible on the right hand
side. For example the parentheses are redundant in local x = 1; (x + x)
. All
remaining ambiguities are resolved according to the following order of precedence:
e(...)
e[...]
e.f
(application and indexing)+
-
!
~
(the unary operators)*
/
%
(these, and the remainder below, are binary operators)+
-
<<
>>
<
>
<=
>=
==
!=
&
^
|
&&
||
To make the specification of Jsonnet as simple as possible, many of the language features are represented as syntax sugar. Below is defined the core syntax and the desugaring function from the abstract syntax to the core syntax. Both the static checking rules and the operational semantics are defined at the level of the core language, so it is possible to implement the desugaring in the parser.
The core language has the following simplifications:
$
, which is no-longer a special keyword.!=
, ==
, %
.[::]
are removed.+:
, +::
, and +:::
sugars are removed.e[e]
.super
can exist on its own.Commas are no-longer part of this abstract syntax but we may still write them in our notation to make the presentation more clear.
Also removed in the core language are import
and importstr
. The
semantics of these constructs is that they are replaced with either the contents of the file, or an
error construct if importing failed (e.g. due to I/O errors). In the first case, the file is
parsed, desugared, and subject to static checking before it can be substituted. In the latter case,
the file is substituted in the form of a string, so it merely needs to contain valid UTF-8.
A given Jsonnet file can be recursively imported via import
. Thus, the
implementation loads files lazily (i.e. during execution) as opposed to via static desugaring. The
imported Jsonnet file is parsed and statically checked in isolation. Therefore, the behavior of the
import is not affected by the environment into which it is imported. The files are cached by
filename, so that even if the file changes on disk during Jsonnet execution, referential
transparency is maintained.
e ∈ Core | ::= |
null |
true |
false |
self |
super |
string |
number
|
| |
{
{
assert e
}
{
[ e ] h e
}
}
| |
| |
{
[ e ] : e
for id in e
}
| |
| |
[
{ e }
]
| |
| |
e
[
e
]
| |
| |
e
(
{ e }
{ id = e }
)
| |
| | id | |
| |
local
id = e
{
id = e
}
;
e
| |
| |
if
e
then
e
else
e
| |
| | e binaryop e | |
| | unaryop e | |
| |
function
(
{ e }
{ id = e }
)
e
| |
| |
error
e
|
Desugaring removes constructs that are not in the core language by replacing them with constructs that are. It is defined via the following functions, which proceed by syntax-directed recursion. If a function is not defined on a construct then it simply recurses into the sub-expressions of that construct. Note that we import the standard library at the top of every file, and some of the desugarings call functions defined in the standard library. Their behavior is specified by implementation. However not all standard library functions are written in Jsonnet. The ones that are built into the interpreter (e.g. reflection) will be given special operational semantics rules with the rest of the core language constructs.
desugar: Expr → Core. This desugars a Jsonnet file. Let \(e_{std}\) be the parsed content of std.jsonnet.
desugarexpr: (Expr × Boolean) → Core: This desugars an expression. The second parameter of the function tracks whether we are within an object.
desugarassert: (Field × Boolean) → Field. This desugars object assertions.
desugarfield: (Field × Boolean) → Field. This desugars object fields.
Recall that h ranges over :
, ::
,
:::
.
desugarbind: (Field × Boolean) → Field. This desugars local bindings.
desugararrcomp: (Expr × CompSpec × Boolean) → Field. This desugars array comprehensions.
After the Jsonnet program is parsed and desugared, a syntax-directed algorithm is employed to
reject programs that contain certain classes of errors. This is presented like a static type
system, except that there are no static types. Programs are only rejected if they use undefined
variables, or if self
, super
or $
are used outside the bounds
of an object. In the core language, $
has been desugared to a variable, so its
checking is implicit in the checking of bound variables.
The static checking is described below as a judgement \(Γ ⊢ e\), where \(Γ\) is the set of
variables in scope of \(e\). The set \(Γ\) initially contains only std
, the implicit
standard library. In the case of imported files, each jsonnet file is checked independently of the
other files.
We present two sets of operational semantics rules. The first defines the judgement \(e ↓ v\) which represents the execution of Jsonnet expressions into Jsonnet values. The other defines the judgement \(v ⇓ j\) which represents manifestation, the process by which Jsonnet values are converted into JSON values.
We model both explicit runtime errors (raised by the error construct) and implicit runtime errors (e.g. array bounds errors) as stuck execution. Errors can occur both in the \(e ↓ v\) judgement and in the \(v ⇓ j\) judgement (because it is defined in terms of \(e ↓ v\)).
When executed, Jsonnet expressions yield Jsonnet values. These need to be manifested, an additional step, to get JSON values. The differences between Jsonnet values and JSON values are: 1) Jsonnet values contain functions (which are not representable in JSON). 2) Due to the lazy semantics, both object fields and array elements have yet to be executed to yield values. 3) Object assertions still need to be checked.
Execution of a statically-checked expression will never yield an object with duplicate field names. By abuse of notation, we consider two objects to be equivalent even if their fields and assertions are re-ordered. However this is not true of array elements or function parameters.
v | ∈ | Value | = | Primitive ∪ Object ∪ Function ∪ Array |
Primitive | ::= |
null | true | false | string | double | ||
o | ∈ | Object | ::= |
{ { assert e } { string h e } }
|
Function | ::= |
function ( { id } { id=e } ) e
| ||
a | ∈ | Array | ::= |
[ { e } ]
|
The hidden status of fields is preserved over inheritance if the right hand side uses the
:
form. This is codified with the following function:
The rules for capture-avoiding variable substitution [e/id] are an extension of those in the lambda calculus.
Let y ≠ x and z ≠ x.
self [e/x] = self
| |
super [e/x] = super
| |
x[e/x] = e' | |
y[e/x] = y | |
{
...
assert e'
...
[ e''] h e'''
...
} [e/x] =
{
...
assert e'[e/x]
...
[ e'[e/x]] h e''[e/x]
...
}
| |
{
[ e'] : e''
for x in e'''
} [e/x] =
{
[ e'] : e''
for x in e'''[e/x]
}
| |
{
[ e'] : e''
for y in e'''
} [e/x] =
{
[ e'[e/x]]: e''[e/x]
for y in e''' [e/x]
}
| |
(local ... y= e' ... ; e'')
[e/x] =
local ... y= e' ... ; e''
| (If any variable matches.) |
(local ... y= e' ... ; e'')
[e/x] =
local ... y= e'[e/x] ... ; e''[e/x]
| (If no variable matches.) |
(function (
... y ... z=e' ...
) e'')[e/x] =
function (
... y ... z=e' ...
) e''
| (If any param matches.) |
(function (
... y ... z=e' ...
) e'')[e/x] =
function (
... y ... z=e'[e/x] ...
) e''[e/x]
| (If no param matches.) |
Otherwise, e' [e/x] proceeds via syntax-directed recursion into subterms of e'. |
The rules for keyword substitution ⟦e/kw⟧ for kw ∈ {
self
, super
} avoid substituting keywords that are captured by nested
objects:
self
⟦e/self ⟧ = e
|
super
⟦e/super ⟧ = e
|
self
⟦e/super ⟧ = self
|
super
⟦e/self ⟧ = super
|
{
...
assert e'
...
[ e''] h e'''
...
}
⟦e/kw⟧ =
{
...
assert e'
...
[ e''⟦e/kw⟧] h e'''
...
}
|
{
[ e'] : e''
for x in e'''
} ⟦e/kw⟧ =
{
[ e'⟦e'/kw⟧] : e''
for x in e'''⟦e/kw⟧
}
|
Otherwise, e'⟦e'/kw⟧ proceeds via syntax-directed recursion into subterms of e'. |
The following big step operational semantics rules define the execution of Jsonnet programs, i.e. the reduction of a Jsonnet program e into its Jsonnet value v via the judgement \(e ↓ v\).
Let f range over strings, as used in object field names.
String concatenation will implicitly convert one of the values to a string if necessary. This is similar to Java. The referred function \(tostring\) returns its argument unchanged if it is a string. Otherwise it will manifest its argument as a JSON value \(j\) and unparse it as a single line of text. The referred function \(strlen\) returns the number of unicode characters in the string.
The numeric semantics are as follows:
*
, /
, +
, -
,
<
, ≤
, >
, ≥
, and unary +
and -
operate on numbers and have IEEE double precision floating point semantics,
except that the special states NaN, Infinity raise errors. Note that +
is also
overloaded on objects, arrays, and when either argument is a string. Also, <
,
≤
, >
, ≥
are overloaded on strings, and compare
lexically by unicode codepoint.
<<
,
>>
,
&
,
^
,
|
and
~
first convert their operands to signed 64 bit integers, then perform the operations
in a standard way, then convert back to IEEE double precision floating point.
std.pow(a, b)
,
std.floor(x)
,
std.ceil(x)
,
std.sqrt(x)
,
std.sin(x)
,
std.cos(x)
,
std.tan(x)
,
std.asin(x)
,
std.acos(x)
,
std.atan(x)
,
std.log(x)
,
std.exp(x)
,
std.mantissa(x)
,
std.exponent(x)
and
std.modulo(a, b)
. Also, std.codepoint(x)
take a single character string,
returning the unicode codepoint as a number, and std.char(x)
is its inverse.
The error
operator has no rule because we model errors (both from the language and
user-defined) as stuck execution. The semantics of error
are that its subterm is
evaluated to a Jsonnet value. If this is a string, then that is the error that is raised.
Otherwise, it is converted to a string using \(tostring\) like during string concatenation. The
specification does not specify how the error is presented to the user, and whether or not there is a
stack trace. Error messages are meant for human inspection, and there is therefore no need to
standardize them.
Finally, the function std.native(x)
takes a string and returns a function configured
by the user in a custom execution environment, thus its semantics cannot be formally described here.
The function std.extVar(x)
also takes a string and returns the value bound to that
external variable (always a string) at the time the Jsonnet environment was created.
After execution, the resulting Jsonnet value is manifested into a JSON value whose serialized form is the ultimate output. The Manifestation process removes hidden fields, checks assertions, and forces array elements and non-hidden object fields. Attempting to manifest a function raises an error since they do not exist in JSON. JSON values are formalized below.
By abuse of notation, we consider two objects to be equivalent even if their fields are re-ordered. However this is not true of array elements whose ordering is strict.
j | ∈ | JValue | = | Primitive ∪ JObject ∪ JArray |
Primitive | ::= |
null | true | false | string | double | ||
o | ∈ | JObject | ::= |
{ { string : j } }
|
a | ∈ | Array | ::= |
[ { j } ]
|
Note that JValue ⊂ Value.
Manifestation is the conversion of a Jsonnet value into a JSON value. It is represented with the judgement \(v⇓j\). The process requires executing arbitrary Jsonnet code fragments, so the two semantic judgements represented by \(↓\) and \(⇓\) are mutually recursive. Hidden fields are ignored during manifestation. Functions cannot be manifested, so an error is raised in that case (formalized as stuck execution).
Let D, E, F range over arbitrary expressions. Let ≡ mean contextual equivalence, i.e if D ≡ E then C[D] and C[E] will manifest to the same JSON, for any context C. Note that if D and E both yield errors, they are considered equivalent D ≡ E regardless of the text of the error messages.
Associativity | (D + E) + F ≡ D + (E + F) | |
Idempotence | D + D ≡ D | (if D does not contain super ) |
Commutativity | D + E ≡ E + D | (if D, E do not contain super and have no common fields) |
Identity | D + { } ≡ D | |
{ } + D ≡ D |