mergedata

Table of Contents

Prototype: mergedata(one, two, etc)

Return type: data

Description: Returns the merger of any named data containers or lists. Can also wrap and unwrap data containers.

The returned data container will have the keys from each of the named data containers, arrays, or lists.

If all the data containers are JSON arrays, they are merged into a single array, as you'd expect from merging two arrays.

If any of the data containers are JSON objects, all the containers are treated as JSON objects (for arrays, the key is the element's offset).

This function can accept many types of data parameters.

mergedata() is thus a convenient way, together with getindices() and getvalues(), to bridge the gap between data container and the traditional list and array data types in CFEngine.

Notes:

  • Bare values try to expand a named cfengine data container
  • It is only possible to wrap data containers in the current namespace.
  • true and false are reserved bare values
  • In the event of key collision the last key merged wins

Example:

body common control
{
      bundlesequence => { "test", "test2", "test3" };
}

bundle agent test
{
  vars:
      "d1" data => parsejson('{ "a": [1,2,3], "b": [] }');
      "d2" data => parsejson('{ "b": [4,5,6] }');
      "d3" data => parsejson('[4,5,6]');
      "list1" slist => { "element1", "element2" };
      "array1[mykey]" slist => { "array_element1", "array_element2" };
      "array2[otherkey]" string => "hello";

      "merged_d1_d2" data => mergedata("d1", "d2");
      "merged_d1_d3" data => mergedata("d1", "d3");
      "merged_d3_list1" data => mergedata("d3", "list1");

      "merged_d1_array1" data => mergedata("d1", "array1");
      "merged_d2_array2" data => mergedata("d2", "array2");

      "merged_d1_wrap_array_d2" data => mergedata("d1", "[ d2 ]");
      "merged_d1_wrap_map_d2" data => mergedata("d1", '{ "newkey": d2 }');

      "merged_d1_d2_str" string => format("merging %S with %S produced %S", d1, d2, merged_d1_d2);
      "merged_d1_wrap_array_d2_str" string => format("merging %S with wrapped [ %S ] produced %S", d1, d2, merged_d1_wrap_array_d2);
      "merged_d1_wrap_map_d2_str" string => format('merging %S with wrapped { "newkey": %S produced %S', d1, d2, merged_d1_wrap_map_d2);
      "merged_d1_d3_str" string => format("merging %S with %S produced %S", d1, d3, merged_d1_d3);
      "merged_d3_list1_str" string => format("merging %S with %S produced %S", d3, list1, merged_d3_list1);

      "merged_d1_array1_str" string => format("merging %S with %s produced %S", d1, array1, merged_d1_array1);
      "merged_d2_array2_str" string => format("merging %S with %s produced %S", d2, array2, merged_d2_array2);
  reports:
      "$(merged_d1_d2_str)";
      "$(merged_d1_wrap_array_d2_str)";
      "$(merged_d1_wrap_map_d2_str)";
      "$(merged_d1_d3_str)";
      "$(merged_d3_list1_str)";
      "$(merged_d1_array1_str)";
      "$(merged_d2_array2_str)";
}

bundle agent test2
{
  vars:
      "a"       data  => parsejson('{ "a": "1" }'), meta => { "mymerge" };
      "b"       data  => parsejson('{ "b": "2" }'), meta => { "mymerge" };
      "c"       data  => parsejson('{ "c": "3" }'), meta => { "mymerge" };
      "d"       data  => parsejson('{ "d": "4" }'), meta => { "mymerge" };
      "todo"    slist => variablesmatching(".*", "mymerge");

  methods:
      "go" usebundle => cmerge(@(todo)); # a, b, c, d

  reports:
      "$(this.bundle): merged containers with cmerge = $(cmerge.all_str)";
}

bundle agent cmerge(varlist)
{
  vars:
      "all"     data => parsejson('[]'),            policy => "free";
      "all"     data => mergedata(all, $(varlist)), policy => "free";
      "all_str" string => format("%S", all),        policy => "free";
}

bundle agent test3
{
    vars:
       "dest_files" slist => { "/tmp/default.json", "/tmp/epel.json" };
       "template_file" string => "repository.mustache";

       "process_templates" data => mergedata('{ "$(template_file)" : dest_files }');
       "process__templates_str" string => format("%S", "process_templates");

    reports:
        "$(this.bundle) $(process__templates_str)";
        "$(this.bundle) $(process_templates[$(template_file)])";
}

Output:

R: merging {"a":[1,2,3],"b":[]} with {"b":[4,5,6]} produced {"a":[1,2,3],"b":[4,5,6]}
R: merging {"a":[1,2,3],"b":[]} with wrapped [ {"b":[4,5,6]} ] produced {"0":{"b":[4,5,6]},"a":[1,2,3],"b":[]}
R: merging {"a":[1,2,3],"b":[]} with wrapped { "newkey": {"b":[4,5,6]} produced {"a":[1,2,3],"b":[],"newkey":{"b":[4,5,6]}}
R: merging {"a":[1,2,3],"b":[]} with [4,5,6] produced {"0":4,"1":5,"2":6,"a":[1,2,3],"b":[]}
R: merging [4,5,6] with { "element1", "element2" } produced [4,5,6,"element1","element2"]
R: merging {"a":[1,2,3],"b":[]} with array1 produced {"a":[1,2,3],"b":[],"mykey":["array_element1","array_element2"]}
R: merging {"b":[4,5,6]} with array2 produced {"b":[4,5,6],"otherkey":"hello"}
R: test2: merged containers with cmerge = {"a":"1","b":"2","c":"3","d":"4"}
R: test3 {"repository.mustache":["/tmp/default.json","/tmp/epel.json"]}
R: test3 /tmp/default.json
R: test3 /tmp/epel.json

Example:

bundle agent mergedata_last_key_wins
{
  vars:

      "one" data => '{ "color": "red",  "stuff": [ "one", "two" ], "thing": "one" }';
      "two" data => '{ "color": "blue", "stuff": [ "three" ] }';

  reports:
      "$(with)" with => storejson( mergedata( one, two ) );

}
bundle agent __main__
{
      methods: "mergedata_last_key_wins";
}

Output:

R: {
      "color": "blue",
      "stuff": [
                 "three"
                 ],
      "thing": "one"
}

History:

See also: getindices(), getvalues(), readjson(), parsejson(), readyaml(), parseyaml(), about collecting functions, and data documentation.