Table of Contents

Augments

Table of Contents

Augments files can be used to define variables and classes for use by all CFEngine components before any parsing or evaluation happen. Augments are fundamentally JSON data files, so you should view and edit them with a JSON-aware editor if possible. This is a convenient way to override defaults defined in the Masterfiles Policy Framework without modifying the shipped policy itself.

There are two canonical augments files, $(sys.workdir)/data/host_specific.json (typically /var/cfengine/data/host_specific.json), which is read first and who's Variables take precedence, and $(sys.policy_entry_dirname)/def.json (typically /var/cfengine/inputs/def.json) which may load additional Augments as specified by the augments key.

The file def.json is found based on the location of the policy entry (the first policy file read by the agent):

with no arguments, it's in $(sys.inputdir)/def.json because $(sys.inputdir)/promises.cf is used
with -f /dirname/myfile.cf, it's in /dirname/def.json
with -f ./myfile.cf, it's in ./def.json

def_preferred.json will be used instead of def.json if it is present. This preferential loading can be disabled by providing the --ignore-preferred-augments option to the agent.

Values will be expanded, so you can use the variables from Special Variables.

An augments file can contain the following keys:

inputs

This key is supported in def.json, def_preferred.json, and augments loaded by the augments key.

Any filenames you put here will appear in the def.augments_inputs variable. The standard set of masterfiles refers to this variable and will autoload those files.

variables

This key is supported in both host_specific.json, def.json, def_preferred.json, and augments loaded by the augments key.

Variables defined here can target a namespace and or bundle scope explicitly. When defined from host_specific.json, variables default to the main bundle in the data __namespace_ ($(data:main.MyVariable)).

For example:

{
    "variables": {
        "VariableWithImplicitNamespaceAndBundle": {
            "value": "value"
        }
    }
}

Variables can target the implicit namespace while specifying the bundle.

For example:

{
    "variables": {
        "my_bundle.VariableWithImplicitNamespace": {
            "value": "value"
        }
    }
}

Variables can also target a namespace explicitly.

For example:

{
    "variables": {
        "MyNamespace:my_bundle.Variable": {
            "value": "value"
        }
    }
}

The comment key is optional, if supplied, the comment will be associated with the variable definition as if you had applied the comment attribute to a vars type promise.

For example, this JSON:

{
    "variables": {
        "MyNamespace:my_bundle.Variable": {
            "value": "value",
            "comment": "An optional note about why this variable is important"
        }
    }
}

Is equivalent to this policy:

body file control
{
      namespace => "MyNamespace";
}
bundle agent my_bundle
{
  vars:
      "Variable"
        string => "value",
        comment => "An optional note about why this variable is important";
}

The tags key is optional, if supplied, the tags will be associated with the variable definition as if you had applied the meta attribute to a vars type promise.

For example, this JSON:

{
    "variables": {
        "MyNamespace:my_bundle.Variable": {
            "value": "value",
            "tags": [ "inventory", "attribute_name=My Inventory" ]
        }
    }
}

Is equivalent to this policy:

body file control
{
      namespace => "MyNamespace";
}
bundle agent my_bundle
{
  vars:
      "Variable"
        string => "value",
        meta => { "inventory", "attribute_name=My Inventory" };
}

History:

Added in 3.18.0

vars

This key is supported in both host_sepecific.json, def.json, and def_preferred.json and augments loaded by the augments key.

Variables defined here can target a namespace and or bundle scope explicitly. When defined from def.json, variables default to the def bundle in the default __namespace_ ($(default:def.MyVariable)).

Thus:

{
    "vars": {
        "phone": "22-333-4444",
        "myplatform": "$(sys.os)",
        "MyBundle.MyVariable": "MyValue in MyBundle.MyVariable",
        "MyNamespace:MyBundle.MyVariable": "MyValue in MyNamespace:MyBundle.MyVariable"
    }
}

results in the variable default:def.phone with value 22-333-4444, default:def.myplatform with the value of your current OS, default:MyBundle.MyVariable with the value MyValue in MyBundle.MyVariable and MyNamespace:MyBundle.MyVariable with the value MyValue in MyNamespace:MyBundle.MyVariable.

Again, note that this happens before policy is parsed or evaluated.

You can see the list of variables thus defined in the output of cf-promises --show-vars (see Components and Common Control). They will be tagged with the tag source=augments_file. For instance, the above two variables (assuming you placed the data in $(sys.inputdir)/def.json) result in

cf-promises --show-vars=default:def
...
default:def.myplatform                   linux                                                        source=augments_file
default:def.phone                        22-333-4444                                                  source=augments_file

Variables of other types than string can be defined too, like in this example

"vars" : {
    "str1" : "string 1",
    "num1" : 5,
    "num2" : 3.5
    "slist1" : ["sliststr1", "sliststr2"]
    "array1" : {
        "idx1" : "val1",
        "idx2" : "val2"
    }
}

History:

3.18.0 gained ability to specify the namespace and bundle the variable should be defined in.

classes

This key is supported in both host_sepecific.json, def.json, def_preferred.json, and augments loaded by the augments key.

Any class defined via augments will be evaluated and installed as soft classes. This key supports both array and dict formats.

For an array each element of the array is tested against currently defined classes as an anchored regular expression unless the string ends with :: indicating it should be interpreted as a class expression.

For example:

{
    "classes": {
        "augments_class_from_regex_my_always": [ "any" ],
        "augments_class_from_regex_my_other_apache": [ "server[34]", "debian.*" ],
        "augments_class_from_regex_my_other_always": [ "augments_class_from_regex_my_always" ],
        "augments_class_from_regex_when_MISSING_not_defined": [ "^(?!MISSING).*" ],
        "augments_class_from_regex": [ "cfengine_\\d+" ],
        "augments_class_from_single_class_as_regex": [ "cfengine" ],
        "augments_class_from_single_class_as_expression": [ "cfengine::" ],
        "augments_class_from_classexpression_and": [ "cfengine.cfengine_3::" ],
        "augments_class_from_classexpression_not": [ "!MISSING::" ],
        "augments_class_from_classexpression_or": [ "cfengine|cfengine_3::" ],
        "augments_class_from_classexpression_complex": [ "(cfengine|cfengine_3).!MISSING::" ]
    }
}

The tags, comment, and the mutually exclusive class_expressions, and regular_expressions subkeys are supported when using the dict structure.

For example:

{
    "classes": {
        "myclass_defined_by_augments_in_def_json_3_18_0_v0": {
            "class_expressions": [ "linux.redhat::", "cfengine|linux::" ],
            "comment": "Optional description about why this class is important",
            "tags": [ "optional", "tags" ]
        },
        "myclass_defined_by_augments_in_def_json_3_18_0_v1": {
            "regular_expressions": [ "linux.*", "cfengine.*" ],
            "tags": [ "optional", "tags" ]
        }
    }
}

Note that augments is processed at the very beginning of agent evaluation. You can use any hard classes, persistent classes , or classes defined earlier in the augments list. Test carefully, custom soft classes may not be defined early enough for use. Thus:

{
    "classes": {
        "augments_class_from_regex_my_always": [ "any" ],
        "augments_class_from_regex_my_other_apache": [ "server[34]", "debian.*" ],
        "augments_class_from_regex_my_other_always": [ "augments_class_from_regex_my_always" ],
        "augments_class_from_regex_when_MISSING_not_defined": [ "^(?!MISSING).*" ],
        "augments_class_from_regex": [ "cfengine_\\d+" ],
        "augments_class_from_single_class_as_regex": [ "cfengine" ],
        "augments_class_from_single_class_as_expression": [ "cfengine::" ],
        "augments_class_from_classexpression_and": [ "cfengine.cfengine_3::" ],
        "augments_class_from_classexpression_not": [ "!MISSING::" ],
        "augments_class_from_classexpression_or": [ "cfengine|cfengine_3::" ],
        "augments_class_from_classexpression_complex": [ "(cfengine|cfengine_3).!MISSING::" ],
        "myclass_defined_by_augments_in_def_json_3_18_0_v0": {
            "class_expressions": [ "linux.redhat::", "cfengine|linux::" ],
            "comment": "Optional description about why this class is important",
            "tags": [ "optional", "tags" ]
        },
        "myclass_defined_by_augments_in_def_json_3_18_0_v1": {
            "regular_expressions": [ "linux.*", "cfengine.*" ],
            "tags": [ "optional", "tags" ]
        }
    }
}

results in * augments_class_from_rgex_my_always being always defined.

augments_class_from_regex_my_other_apache will be defined if the classes server3 or server4 are defined, or if any class starting with debian is defined.
augments_class_from_regex_my_other_always will be defined because augments_class_from_regex_my_always is listed first and always defined.
augments_class_from_regex_when_MISSING_not_defined will be defined if the class MISSING is not defined.
augments_class_from_single_class_as_regex will be defined because the class cfengine is always defined.
augments_class_from_single_class_as_expression will be defined because cfengine is defined when interpreted as a class expression.
augments_class_from_classexpression_and will be defined because the class cfengine and the class cfengine_3 are defined and the class expression cfengine.cfengine_3:: evaluates to true.
augments_class_from_classexpression_not will be defined because the class expression !MISSING:: evaluates to false since the class MISSING is not defined.
augments_class_from_classexpression_or will be defined because the class expression cfengine|cfengine_3:: evaluates to true since at least one of cfengine or cfengine_3 will always be defined by cfengine 3 agents.
augments_class_from_classexpression_complex will be defined because the class expression (cfengine|cfengine_3).!MISSING:: evaluates to true since at least one of cfengine or cfengine_3 will always be defined by cfengine 3 agents and MISSING is not defined.
myclass_defined_by_augments_in_def_json_3_18_0_v0 will be defined because the class expression cfengine|linux:: will always be true since there is always a cfengine class defined.
myclass_defined_by_augments_in_def_json_3_18_0_v1 will be defined because the expression cfengine.** will match at least one defined class, cfengine

You can see the list of classes thus defined through def.json in the output of cf-promises --show-classes (see Components and Common Control). They will be tagged with the tags source=augments_file. For instance:

% cf-promises --show-classes=my
Class name                                                   Meta tags                                Comment
augments_class_from_regex_my_always                          source=augments_file
augments_class_from_regex_my_other_always                    source=augments_file
augments_class_from_regex_my_other_apache                    source=augments_file
myclass_defined_by_augments_in_def_json_3_18_0_v0            optional,tags,source=augments_file       Optional description about why this class is important
myclass_defined_by_augments_in_def_json_3_18_0_v1            optional,tags,source=augments_file

See also:

Functions that use regular expressions with classes: classesmatching(), classmatch(), countclassesmatching()

History:

3.18.0
- Support for dict structure for classes and support for metadata (comment, tags) added.
- Classes are defined as soft classes instead of hard classes.

augments

This key is supported in def.json, def_preferred.json, and augments loaded by the augments key.

A list of file names that should be merged using mergedata() semantic

Example:

Here we merge a platform specific augments on to the def.json loaded next to the policy entry and see what the resulting variable values will be.

The def.json next to the policy entry:

{
  "vars":{
    "my_var": "defined in def.json",
    "my_other_var": "Defined ONLY in def.json"
  },
  "augments": [
    "/var/cfengine/augments/$(sys.flavor).json"
  ]
}

The platform specific augments on a CentOS 6 host:

/var/cfengine/augments/centos_6.json:

{
  "vars": {
    "my_var": "Overridden in centos_6.json",
    "centos_6_var": "Defined ONLY in centos_6.json"
  }
}

The expected values of the variables defined in the def bundle scope:

R: def.my_var == Overridden in centos_6.json
R: def.my_other_var == Defined ONLY in def.json
R: def.centos_6_var == Defined ONLY in centos_6.json

History

3.18.0
- Introduced variables key with support for metadata (comment, tags) and targeting the namespace and bundle.
- Introduced ability for vars to target namespace and bundle variables key with support for metadata (comment, tags).
- Introduced metadata (comment, tags) support for classes key.
- Introduced def_preferred.json and --ignore-preferred-augments to disable it.
- Classes defined from augments are now soft classes and not hard classes.
- Introduced parsing of $(sys.workdir)/data/host_specific.json
3.12.2, 3.14.0 introduced class expression interpretation (:: suffix) to classes key
3.12.0 introduced the augments key
3.7.3 back port def.json parsing in core agent and load def.json if present next to policy entry
3.8.2 removed core support for inputs key, load def.json if present next to policy entry
3.8.1 def.json parsing moved from policy to core agent for resolution of classes and variables to be able to affect control bodies
3.7.0 introduced augments concept into the Masterfiles Policy Framework