Table of Contents

Functions

Table of Contents

Functions take zero or more values as arguments and return a value. Argument values need to be of the type and range as documented for each function. Some functions are documented with a ..., in which case they take an arbitrary amount of arguments.

They can return scalar (string|int|real|bool), list (slist, ilist, rlist) and data values:

printf "one\ntwo\nthree\n" > /tmp/list.txt
printf "1\n2\n3\n"        >> /tmp/list.txt
printf "1.0\n2.0\n3.0"    >> /tmp/list.txt

bundle agent example_function_return_types
{

  classes:
      "this_file_exists" expression => fileexists( $(this.promise_filename) );

  vars:
      "my_string" string => concat( "Promises you cannot keep",
                                    " are no better than lies");

      "my_list_of_strings"
        slist => readstringlist( "/tmp/list.txt", # File to read
                                 "",               # Don't ignore any lines
                                 "\n",             # Split on newlines
                                 inf,              # Extract as many entries as possible
                                 inf);             # Read in as much data as possible

      "my_list_of_integers"
        ilist => readintlist( "/tmp/list.txt",     # File to read
                              "^(\D+)|(\d+[^\n]+)", # Ignore any lines that are not integers
                              "\n",                 # Split on newlines
                              inf,                  # Maximum number of entries
                              inf);                 # Maximum number of bytes to read

      "my_list_of_reals"
        rlist => readreallist( "/tmp/list.txt", # File to read
                              "^(\D+)",          # Ignore any lines that are not digits
                              "\n",              # Split on newlines
                              inf,               # Maximum number of entries
                              inf);              # Maximum number of bytes to read

      "my_integer" int => string_length( $(my_string) );

      "my_real" real => sum( my_list_of_integers );

      "my_data" data => mergedata( '{ "Hello": "world!" }' );

  reports:
      "my_string: '$(my_string)'";
      "my_list_of_strings includes '$(my_list_of_strings)'";
      "my_list_of_integers includes '$(my_list_of_integers)'";
      "my_list_of_reals includes '$(my_list_of_reals)'";
      "my_integer: '$(my_integer)'";
      "my_real: '$(my_real)'";
      "my_data: '$(with)'"
        with => string_mustache( "", my_data );

    this_file_exists::
      "This file exists.";

}
bundle agent __main__
{
  methods: "example_function_return_types";
}

R: my_string: 'Promises you cannot keep are no better than lies'
R: my_list_of_strings includes 'one'
R: my_list_of_strings includes 'two'
R: my_list_of_strings includes 'three'
R: my_list_of_strings includes '1'
R: my_list_of_strings includes '2'
R: my_list_of_strings includes '3'
R: my_list_of_strings includes '1.0'
R: my_list_of_strings includes '2.0'
R: my_list_of_strings includes '3.0'
R: my_list_of_integers includes '1'
R: my_list_of_integers includes '2'
R: my_list_of_integers includes '3'
R: my_list_of_reals includes '1'
R: my_list_of_reals includes '2'
R: my_list_of_reals includes '3'
R: my_list_of_reals includes '1.0'
R: my_list_of_reals includes '2.0'
R: my_list_of_reals includes '3.0'
R: my_integer: '48'
R: my_real: '6.000000'
R: my_data: '{
  "Hello": "world!"
}'
R: This file exists.

This policy can be found in /var/cfengine/share/doc/examples/function-return-types.cf and downloaded directly from github.

Boolean return type

Functions which return boolean, technically return a string any or !any. This is for compatibility with other functions and promise attributes which expect class expressions.

Normally, you don't need to worry about this, you use the function call to define classes, and use classes instead of boolean values:

bundle agent main
{
  vars:
      "five" int => "5";
  classes:
      "is_var" if => isvariable("five");
  reports:
    is_var::
      "Success!";
}

There is no boolean data type for vars promises. If you want to store or print the class expression, you can use concat():

bundle agent main
{
  vars:
      "five" int => "5";
      "expression" string => concat(isvariable("five"));
  classes:
      "is_var" if => "$(expression)"; # Will be expanded and evaluated
  reports:
    is_var::
      "Success: expression expanded to '$(expression)' and evaluated to true!";
}

Note: the truth of a class expression or the result of a function call may change during evaluation, but typically, a class, once defined, will stay defined.

Promise attributes and function calls

Promise attributes which use a class expression (string) as input, like if and unless, can take a function call which returns string or boolean as well.

A boolean function will be resolved to any, which is always true, or !any which is always false.
A string function will be resolved, and the returned string will be evaluated as a class expression.

bundle agent main
{
  vars:
      "five" int => "5";
      "is_var_class_expression" string => concat(isvariable("$(five)"));
  classes:
      "five_less_than_seven" expression => islessthan("$(five)", 7);
      "five_is_variable" if => "$(is_var_class_expression)";
  reports:
    any::
      "five: $(five)";
      "is_var_class_expression: $(is_var_class_expression)";
    five_less_than_seven::
      "$(five) is smaller than 7";
}

Function caching

During convergence, CFEngine's evaluation model will evaluate functions multiple times, which can be a performance concern.

Some system functions are particularly expensive:

execresult() and returnszero() for shell execution
regldap(), ldapvalue(), and ldaplist() for LDAP queries
findprocesses(), and processexists() for querying processes.
host2ip() and ip2host() for DNS queries
readtcp() for TCP interactions
hubknowledge(), and remotescalar() for hub queries

When enabled cached functions are not executed on every pass of convergence. Instead, the function will only be executed once during the agent evaluation step and its result will be cached until the end of that agent execution.

Note: Cached functions are executed multiple times during policy validation and pre-evaluation. Function caching is per-process, so results will not be cached between separate components e.g. cf-agent, cf-serverd and cf-promises. Additionally functions are cached by hashing the function arguments. If you have the exact same function call in two different promises (it does not matter if they are in the same bundle or not) only the first executed function will be cached. That cached result will be re-used for other identical function occurrences.

Function caching can be disabled by setting cache_system_functions in body common control to false.

Function Skipping

If a variable passed to a function is unable to be resolved the function will be skipped. The function will be evaluated during a later pass when all variables passed as arguments are able to be resolved. The function will never be evaluated if any argument contains a variable that never resolves.

Note: In some situations, a variable which never resolves cause errors, while in other cases it defaults to some behavior. For example:

bundle agent main
{
  classes:
      "a" unless => "$(no_such_var)";             # Won't error
      "b" unless => isvariable("$(no_such_var)"); # Will error
}

The behavior in these edge cases is likely to change, so don't rely on it.

Collecting Functions

Some function arguments are marked as collecting which means they can "collect" an argument from various sources. The data is normalized into the JSON format internally, so all of the following data types have consistent behavior.

If a key inside a data container is specified (mycontainer[key]), the value under that key is collected. The key can be a string for JSON objects or a number for JSON arrays.
If a single data container, CFEngine array, or slist is specified (mycontainer or myarray or myslist), the contents of it are collected.
If a single data container, CFEngine array, or slist is specified with @() around it (@(mycontainer) or @(myarray) or @(myslist)), the contents of it are collected.
If a function call that returns a data container or slist is specified, that function call is evaluated and the results are inserted, so you can say for instance sort(data_expand(...), "lex") to expand a data container then sort it.
If a list (slist, ilist, or rlist) is named, its entries are collected.
If any CFEngine "classic" array (array[key]) is named, it's first converted to a JSON key-value map, then collected.
If a literal JSON string like [ 1,2,3 ] or { "x": 500 } is provided, it will be parsed and used.
If any of the above-mentioned ways to reference variables are used inside a literal JSON string they will be expanded (or the function call will fail). This is similar to the behavior of Javascript, for instance. For example, mergedata('[ thing, { "mykey": otherthing[123] } ]') will wrap the thing in a JSON array; then the contents of otherthing[123] will be wrapped in a JSON map which will also go in the array.

Delayed Evaluation Functions

Since CFEngine 3.10, some functions are marked as delayed evaluation which means they can evaluate a function call across every element of a collection. This makes intuitive sense for the collection traversing functions maparray(), maplist(), and mapdata().

The practical use is for instance maplist(format("%03d", $(this)), mylist) which will evaluate that format() call once for every element of mylist.

Before 3.10, the same call would have resulted in running the format() function before the list is traversed, which is almost never what the user wants.

List of all functions

There are a large number of functions built into CFEngine. The following tables might make it easier for you to find the function you need.

Functions by Category

`communication`	`data`	`files`	`internal`	`io`	`system`	`utils`
`host2ip()`	`accumulated()`	`accessedbefore()`	`callstack_callers()`	`classfiltercsv()`	`data_sysctlvalues()`	`bundlestate()`
`hostrange()`	`ago()`	`changedbefore()`	`callstack_promisers()`	`countlinesmatching()`	`findprocesses()`	`classesmatching()`
`hostsseen()`	`and()`	`dirname()`		`data_readstringarray()`	`getenv()`	`classmatch()`
`hostswithclass()`	`basename()`	`diskfree()`		`data_readstringarrayidx()`	`getuid()`	`countclassesmatching()`
`hubknowledge()`	`bundlesmatching()`	`file_hash()`		`parseintarray()`	`getuserinfo()`	`datastate()`
`ip2host()`	`canonify()`	`fileexists()`		`parsejson()`	`getusers()`	`execresult()`
`iprange()`	`canonifyuniquely()`	`filesexist()`		`parserealarray()`	`groupexists()`	`getclassmetatags()`
`isipinsubnet()`	`classify()`	`filesize()`		`parsestringarray()`	`hostinnetgroup()`	`getvariablemetatags()`
`ldaparray()`	`concat()`	`filestat()`		`parsestringarrayidx()`	`now()`	`isvariable()`
`ldaplist()`	`data_expand()`	`findfiles()`		`parseyaml()`	`packagesmatching()`	`returnszero()`
`ldapvalue()`	`data_regextract()`	`isdir()`		`read_module_protocol()`	`packageupdatesmatching()`	`splayclass()`
`network_connections()`	`difference()`	`isexecutable()`		`readcsv()`	`processexists()`	`usemodule()`
`peerleader()`	`escape()`	`islink()`		`readdata()`	`registryvalue()`
`peerleaders()`	`eval()`	`isnewerthan()`		`readenvfile()`	`sysctlvalue()`
`peers()`	`every()`	`isplain()`		`readfile()`	`userexists()`
`readtcp()`	`expandrange()`	`laterthan()`		`readintarray()`
`regldap()`	`filter()`	`lsdir()`		`readintlist()`
`remoteclassesmatching()`	`format()`	`translatepath()`		`readjson()`
`remotescalar()`	`getfields()`			`readrealarray()`
`selectservers()`	`getgid()`			`readreallist()`
`url_get()`	`getindices()`			`readstringarray()`
	`getvalues()`			`readstringarrayidx()`
	`grep()`			`readstringlist()`
	`hash()`			`readyaml()`
	`hash_to_int()`			`regline()`
	`hashmatch()`
	`ifelse()`
	`intersection()`
	`irange()`
	`isgreaterthan()`
	`islessthan()`
	`join()`
	`lastnode()`
	`length()`
	`makerule()`
	`maparray()`
	`mapdata()`
	`maplist()`
	`max()`
	`mean()`
	`mergedata()`
	`min()`
	`none()`
	`not()`
	`nth()`
	`on()`
	`or()`
	`product()`
	`randomint()`
	`regarray()`
	`regcmp()`
	`regex_replace()`
	`regextract()`
	`reglist()`
	`reverse()`
	`rrange()`
	`shuffle()`
	`some()`
	`sort()`
	`splitstring()`
	`storejson()`
	`strcmp()`
	`strftime()`
	`string_downcase()`
	`string_head()`
	`string_length()`
	`string_mustache()`
	`string_replace()`
	`string_reverse()`
	`string_split()`
	`string_tail()`
	`string_upcase()`
	`sublist()`
	`sum()`
	`unique()`
	`variablesmatching()`
	`variablesmatching_as_data()`
	`variance()`

Functions by Return Type

`(i,r)range`	`(i,r,s)list`	`class`	`data`	`int`	`real`	`string`
`irange()`	`bundlesmatching()`	`accessedbefore()`	`bundlestate()`	`accumulated()`	`mean()`	`and()`
`rrange()`	`callstack_promisers()`	`changedbefore()`	`callstack_callers()`	`ago()`	`product()`	`basename()`
	`classesmatching()`	`classify()`	`classfiltercsv()`	`countclassesmatching()`	`sum()`	`canonify()`
	`difference()`	`classmatch()`	`data_expand()`	`countlinesmatching()`	`variance()`	`canonifyuniquely()`
	`expandrange()`	`every()`	`data_readstringarray()`	`diskfree()`		`concat()`
	`filter()`	`fileexists()`	`data_readstringarrayidx()`	`filesize()`		`dirname()`
	`findfiles()`	`filesexist()`	`data_regextract()`	`getfields()`		`escape()`
	`getclassmetatags()`	`groupexists()`	`data_sysctlvalues()`	`getgid()`		`eval()`
	`getindices()`	`hashmatch()`	`datastate()`	`getuid()`		`execresult()`
	`getusers()`	`hostinnetgroup()`	`findprocesses()`	`hash_to_int()`		`file_hash()`
	`getvalues()`	`hostrange()`	`getuserinfo()`	`length()`		`filestat()`
	`getvariablemetatags()`	`iprange()`	`mapdata()`	`now()`		`format()`
	`grep()`	`isdir()`	`mergedata()`	`on()`		`getenv()`
	`hostsseen()`	`isexecutable()`	`network_connections()`	`parseintarray()`		`hash()`
	`hostswithclass()`	`isgreaterthan()`	`packagesmatching()`	`parserealarray()`		`host2ip()`
	`intersection()`	`isipinsubnet()`	`packageupdatesmatching()`	`parsestringarray()`		`hubknowledge()`
	`ldaplist()`	`islessthan()`	`parsejson()`	`parsestringarrayidx()`		`ifelse()`
	`lsdir()`	`islink()`	`parseyaml()`	`randomint()`		`ip2host()`
	`maparray()`	`isnewerthan()`	`readcsv()`	`readintarray()`		`join()`
	`maplist()`	`isplain()`	`readdata()`	`readrealarray()`		`lastnode()`
	`peerleaders()`	`isvariable()`	`readenvfile()`	`readstringarray()`		`ldapvalue()`
	`peers()`	`laterthan()`	`readjson()`	`readstringarrayidx()`		`makerule()`
	`readintlist()`	`ldaparray()`	`readyaml()`	`selectservers()`		`max()`
	`readreallist()`	`none()`	`url_get()`	`string_length()`		`min()`
	`readstringlist()`	`processexists()`	`variablesmatching_as_data()`			`not()`
	`reverse()`	`read_module_protocol()`				`nth()`
	`shuffle()`	`regarray()`				`or()`
	`sort()`	`regcmp()`				`peerleader()`
	`splitstring()`	`regextract()`				`readfile()`
	`string_split()`	`regldap()`				`readtcp()`
	`sublist()`	`regline()`				`regex_replace()`
	`unique()`	`reglist()`				`registryvalue()`
	`variablesmatching()`	`remoteclassesmatching()`				`remotescalar()`
		`returnszero()`				`storejson()`
		`some()`				`strftime()`
		`splayclass()`				`string_downcase()`
		`strcmp()`				`string_head()`
		`usemodule()`				`string_mustache()`
		`userexists()`				`string_replace()`
						`string_reverse()`
						`string_tail()`
						`string_upcase()`
						`sysctlvalue()`
						`translatepath()`