The Complete Examples and Tutorials
Table of Content
- Example Snippets
- General Examples
- Common Promise Patterns
- Change detection
- Copy single files
- Aborting execution
- Check filesystem space
- Distribute ssh keys
- Ensure a process is not running
- Updating from a central policy server
- Customize Message of the Day
- Restart a Process
- Set up sudo
- Set up name resolution with DNS
- Find the MAC address
- Install packages
- Ensure a service is enabled and running
- Create files and directories
- Set up time management through NTP
- Mount NFS filesystem
- CFEngine Administration Examples
- Measuring Examples
- Software Administration Examples
- Commands, Scripts, and Execution Examples
- File and Directory Examples
- Interacting with Directory Services
- File Template Examples
- Database Examples
- Network Examples
- System Security Examples
- System Information Examples
- System Administration Examples
- System File Examples
- Windows Registry Examples
- File Permissions
- User Management Examples
- Tutorials
- Enterprise API Examples
Links to Examples
- Example Snippets: This section is divided into topical areas and includes many examples of policy and promises. Each of the snippets can be easily copied or downloaded to a policy server and used as is.
Note: CFEngine also includes a small set of examples by default, which can be
found in /var/cfengine/share/doc/examples
.
See Also:
Tutorial for Running Examples
In this tutorial, you will perform the following:
- Create a simple "Hello World!" example policy file
- Make the example a standalone policy
- Make the example an executable script
- Add the example to the main policy file (
promises.cf
)
Note if your CFEngine administrator has enabled continuous deployment of the policy from a Version Control System, your changes may be overwritten!
"Hello World" Policy Example
Policies contain bundles, which are collections of promises. A promise is a declaration of intent. Bundles allow related promises to be grouped together, as illustrated in the steps that follow.
Following these steps, you will login to your policy server via the SSH protocol, use the vi command line editor to create a policy file named hello_world.cf, and create a bundle that calls a promise to display some text.
- Log into a running server machine using ssh (PuTTY may be used if using Windows).
- Type
sudo su
for super user (enter your password if prompted). - To get to the masterfiles directory, type
cd /var/cfengine/masterfiles
. - Create the file with the command:
vi hello_world.cf
In the vi editor, enter
i
for "Insert" and enter the following content (ie. copy and paste from a text editor):bundle agent hello_world { reports: any:: "Hello World!"; }
Exit the "Insert" mode by pressing the "esc" button. This will return to the command prompt.
Save the changes to the file by typing
:w
then "Enter".Exit vi by typing
:q
then "Enter".
In the policy file above, we have defined an agent bundle named hello_world
. Agent
bundles are only evaluated by cf-agent, the agent component of CFEngine.
This bundle promises to report on any class of hosts.
Activate a Bundle Manually
Activate the bundle manually by executing the following command at prompt:
/var/cfengine/bin/cf-agent --no-lock --file ./hello_world.cf --bundlesequence hello_world
This command instructs CFEngine to ignore locks, load
the hello_world.cf
policy, and activate the hello_world
bundle. See the output below:
# /var/cfengine/bin/cf-agent --no-lock --file ./hello_world.cf --bundlesequence hello_world
2013-08-20T14:03:43-0500 notice: R: Hello World!
As you get familiar with CFEngine, you'll probably start shortening this command to this equivalent:
/var/cfengine/bin/cf-agent -Kf ./hello_world.cf -b hello_world
Note the full path to the binary in the above command. CFEngine stores its binaries in /var/cfengine/bin on Linux and Unix systems. Your path might vary depending on your platform and the packages your are using. CFEngine uses /var because it is one of the Unix file systems that resides locally. Thus, CFEngine can function even if everything else fails (your other file systems, your network, and even system binaries) and possibly repair problems.
Make the Example Stand Alone
Instead of specifying the bundle sequence on the command line (as it was above), a body common
control section can be added to
the policy file. The body common control refers to those promises that are hard-coded into
all CFEngine components and therefore affect the behavior of all components. Note that only
one body common control
is allowed per agent activation.
Go back into vi by typing "vi" at the prompt. Then type i
to insert
body common control to hello_world.cf
. Place it above bundle agent hello_world, as
shown in the following example:
body common control
{
bundlesequence => { "hello_world" };
}
bundle agent hello_world
{
reports:
any::
"Hello World!";
}
Now press "esc" to exit the "Insert" mode, then type :w
to save the file changes and "Enter".
Exit vi by typing :q
then "Enter." This will return to the prompt.
Execute the following command:
console
/var/cfengine/bin/cf-agent --no-lock --file ./hello_world.cf
The output is shown below:
# /var/cfengine/bin/cf-agent --no-lock --file ./hello_world.cf
2013-08-20T14:25:36-0500 notice: R: Hello World!
Note: It may be necessary to add a reference to the standard library within the body common control section, and remove the bundlesequence line. Example:
body common control {
inputs => {
"libraries/cfengine_stdlib.cf",
};
}
Make the Example an Executable Script
Add the #!
marker ("shebang") to hello_world.cf
in order to invoke CFEngine policy as an executable script:
Again type "vi" then "Enter" then i
to insert the following:
#!/var/cfengine/bin/cf-agent --no-lock
Add it before body common control, as shown below:
#!/var/cfengine/bin/cf-agent --no-lock
body common control
{
bundlesequence => { "hello_world" };
}
bundle agent hello_world
{
reports:
any::
"Hello World!";
}
Now exit "Insert" mode by pressing "esc". Save file changes by typing :w
then "Enter"
then exit vi by typing :q
then "Enter". This will return to the prompt.
Make the policy file executable, and then run it, by typing the following two commands:
chmod +x ./hello_world.cf
Followed by:
./hello_world.cf
See the output below:
# chmod +x ./hello_world.cf
# ./hello_world.cf
2013-08-20T14:39:34-0500 notice: R: Hello World!
Integrating the Example into your Main Policy
Make the example policy part of your main policy by doing the following on your policy server:
Ensure the example is located in
/var/cfengine/masterfiles
.If the example contains a
body common control
section, delete it. That section will look something like this:body common control { bundlesequence => { "hello_world" }; }
You cannot have duplicate control bodies (i.e. two agent control bodies, one in the main file and one in the example) as CFEngine won't know which it should use and they may conflict.
To resolve this, copy the contents of the control body section from the
example into the identically named control body section in the main policy
file /var/cfengine/masterfiles/promises.cf
and then remove the control body
from the example.
Insert the example's bundle name in the
bundlesequence
section of the main policy file/var/cfengine/masterfiles/promises.cf
:bundlesequence => { ... "hello_world", ... };
Insert the policy file name in the
inputs
section of the main policy file/var/cfengine/masterfiles/promises.cf
:inputs => { ... "hello_world.cf", ... };
You must also remove any inputs section from the example that includes the external library:
inputs => { "libraries/cfengine_stdlib.cf" };
This is necessary, since
cfengine_stdlib.cf
is already included in the inputs section of the master policy.The example policy will now be executed every five minutes along with the rest of your main policy.
Notes: You may have to fill the example with data before it will work.
For example, the LDAP query in active_directory.cf
needs a domain name.
In the variable declaration, replace "cftesting" with your domain name:
vars:
# NOTE: Edit this to your domain, e.g. "corp"
"domain_name" string => "cftesting";
Example Snippets
- General Examples
- CFEngine Administration Examples
- Measuring Examples
- Software Administration Examples
- Commands, Scripts, and Execution Examples
- File and Directory Examples
- File Template Examples
- Database Examples
- Network Examples
- System Security Examples
- System Information Examples
- System Administration Examples
- System File Examples
- Windows Registry Examples
- User Management
General Examples
Basic Example
To get started with CFEngine, you can imagine the following template for entering examples. This part of the code is common to all the examples.
body common control
{
bundlesequence => { "main" };
inputs => { "$(sys.libdir)/stdlib.cf" };
}
bundle agent main
{
# example
}
The general pattern
The general pattern of the syntax is like this (colors in html version: red, CFEngine word; blue, user-defined word):
bundle component name(parameters)
{
what_type:
where_when::
# Traditional comment
"promiser" -> { "promisee1", "promisee2" },
comment => "The intention ...",
handle => "unique_id_label",
attribute_1 => body_or_value1,
attribute_2 => body_or_value2;
}
Hello world
body common control
{
bundlesequence => { "hello" };
}
bundle agent hello
{
reports:
linux::
"Hello world!";
}
Array example
body common control
{
bundlesequence => { "array" };
}
bundle common g
{
vars:
"array[1]" string => "one";
"array[2]" string => "two";
}
bundle agent array
{
vars:
"localarray[1]" string => "one";
"localarray[2]" string => "two";
reports:
linux::
"Global $(g.array[1]) and $(localarray[2])";
}
Common Promise Patterns
This section includes includes common promise patterns. Refer to them as you write policy for your system.
- Aborting execution
- Change detection
- Check filesystem space
- Copy single files
- Create files and directories
- Customize Message of the Day
- Distribute ssh keys
- Ensure a process is not running
- Ensure a service is enabled and running
- Find the MAC address
- Install packages
- Mount NFS filesystem
- Restart a Process
- Set up sudo
- Set up time management through NTP
- Set up name resolution with DNS
- Updating from a central policy server
Change detection
This policy will look for changes recursively in a directory.
body common control
{
bundlesequence => { "example" };
}
bundle agent example
{
files:
"/home/mark/tmp/web" # Directory to monitor for changes.
changes => detect_all_change,
depth_search => recurse("inf");
}
body changes detect_all_change
{
report_changes => "all";
update_hashes => "true";
}
body depth_search recurse(d)
{
depth => "$(d)";
}
This policy can be found in
/var/cfengine/share/doc/examples/change_detect.cf
and downloaded directly from
github.
Here is an example run.
First, let's create some files for CFEngine to monitor:
# mkdir /etc/example
# date > /etc/example/example.conf
CFEngine detects new files and adds them to the file integrity database:
# cf-agent -f unit_change_detect.cf
2013-06-06T20:53:26-0700 error: /example/files/'/etc/example':
File '/etc/example/example.conf' was not in 'md5' database - new file found
# cf-agent -f unit_change_detect.cf -K
If there are no changes, CFEngine runs silently:
# cf-agent -f unit_change_detect.cf
#
Now let's update the mtime, and then the mtime and content. CFEngine will notice the changes and record the new profile:
# touch /etc/example/example.conf # update mtime
# cf-agent -f unit_change_detect.cf -K
2013-06-06T20:53:52-0700 error: Last modified time for
'/etc/example/example.conf' changed 'Thu Jun 6 20:53:18 2013'
-> 'Thu Jun 6 20:53:49 2013'
# date >> /etc/example/example.conf # update mtime and content
# cf-agent -f unit_change_detect.cf -K
2013-06-06T20:54:01-0700 error: Hash 'md5' for '/etc/example/example.conf' changed!
2013-06-06T20:54:01-0700 error: /example/files/'/etc/example': Updating hash for
'/etc/example/example.conf' to 'MD5=8576cb25c9f78bc9ab6afd2c32203ca1'
2013-06-06T20:54:01-0700 error: Last modified time for '/etc/example/example.conf'
changed 'Thu Jun 6 20:53:49 2013' -> 'Thu Jun 6 20:53:59 2013'
#
Copy single files
This is a standalone policy example that will copy single files,
locally (local_cp
) and from a remote site (secure_cp
).
The CFEngine Standard Library (cfengine_stdlib.cf) should be
included in the /var/cfengine/inputs/libraries/
directory and
inputs list as below.
body common control
{
bundlesequence => { "mycopy" };
inputs => { "$(sys.libdir)/stdlib.cf" };
}
bundle agent mycopy
{
files:
"/tmp/test_plain"
Path and name of the file we wish to copy to
comment => "/tmp/test_plain promises to be an up-to-date copy of /bin/echo to demonstrate copying a local file",
copy_from => local_cp("$(sys.workdir)/bin/file");
Copy locally from path/filename
"/tmp/test_remote_plain"
comment => "/tmp/test_plain_remote promises to be a copy of cfengine://serverhost.example.org/repo/config-files/motd",
copy_from => secure_cp("/repo/config-files/motd", "serverhost.example.org");
}
Copy remotely from path/filename and specified host ```
This policy can be found in
/var/cfengine/share/doc/examples/copy_copbl.cf
and downloaded directly from
github.
Aborting execution
Sometimes it is useful to abort a bundle execution if certain conditions are not met,
for example when validating input to a bundle. The following policy uses a list of
regular expressions for classes, or class expressions that cf-agent
will watch out for.
If any of these classes becomes defined, it will cause the current bundle to be aborted.
body common control
{
bundlesequence => { "example" };
}
body agent control
{
abortbundleclasses => { "invalid" };
}
bundle agent example
{
vars:
#"userlist" slist => { "mark", "john" }; # contains all valid entries
"userlist" slist => { "mark", "john", "thomas" }; # contains one invalid entry
classes:
"invalid" not => regcmp("[a-z][a-z][a-z][a-z]","$(userlist)"); # The class 'invalid' is set if the user name does not
# contain exactly four un-capitalized letters (bundle
# execution will be aborted if set)
reports:
!invalid::
"User name $(userlist) is valid at 4 letters";
}
This policy can be found in
/var/cfengine/share/doc/examples/abort.cf
and downloaded directly from
github.
This is how the policy runs when the userlist is valid:
# cf-agent -f unit_abort.cf
R: User name mark is valid at 4 letters
R: User name john is valid at 4 letters
#
This is how the policy runs when the userlist contains an invalid entry:
# cf-agent -f unit_abort.cf
Bundle example aborted on defined class "invalid"
#
To run this example file as part of your main policy you need to make an additional change:
There cannot be two body agent control
in the main policy. Delete the
body agent control
section from /var/cfengine/masterfiles/unit_abort.cf
.
Copy and paste abortbundleclasses => { "invalid" };
into
/var/cfengine/masterfiles/controls/cf_agent.cf
. If you add it to
the end of the file it should look something like this:
...
# dryrun => "true";
abortbundleclasses => { "invalid" };
}
Check filesystem space
Check how much space (in KB) is available on a directory's current partition.
body common control
{
bundlesequence => { "example" };
}
bundle agent example
{
classes:
"has_space" expression => isgreaterthan($(free), 0);
vars:
"free" int => diskfree("/tmp");
reports:
has_space::
"The filesystem has free space";
!has_space::
"The filesystem has NO free space";
}
R: The filesystem has free space
This policy can be found in
/var/cfengine/share/doc/examples/diskfree.cf
and downloaded directly from
github.
Example output:
# cf-agent -f unit_diskfree.cf
R: Freedisk 48694692
# df -k /tmp
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/sda1 149911836 93602068 48694692 66% /
#
Distribute ssh keys
This example shows a simple ssh key distribution implementation.
The policy was designed to work with the services_autorun
feature in
the Masterfiles Policy Framework. The
services_autorun
feature can be enabled from the augments_file. If
you do not have a def.json
in the root of your masterfiles directory
simply create it with the following content.
{
"classes": {
"services_autorun": [ "any" ]
}
}
In the following example we will manage the authorized_keys
file for
bob
, frank
, and kelly
.
For each listed user the ssh_key_distribution
bundle is activated if
the user exists on the system. Once activated the
ssh_key_distribution
bundle ensures that proper permissions are set
on the users .ssh
directory (home is assumed to be in
/home/username
) and ensures that the users .ssh/authorized_keys
is
a copy of the users authorized_keys
file as found on the server as
defined in the ssh_key_info
bundle.
Let's assume we collected all users' public keys into a single directory on the server and that users exist on the clients (and have corresponding home directory).
Note: special variable $(sys.policy_hub)
contains the hostname of
the policy server.
To deploy this policy simply place it in the services/autorun
directory of your masterfiles.
body common control
{
bundlesequence => { "autorun_ssh_key_distribution" };
inputs => { "$(sys.libdir)/stdlib.cf" };
}
bundle common ssh_key_info
{
meta:
"description"
string => "This bundle defines common ssh key information, like which
directory and server keys should be sourced from.";
vars:
"key_server" string => "$(sys.policy_hub)";
# We set the path to the repo in a common bundle so that we can reference
# the same path when defining access rules and when copying files.
# This directory is expected to contain one file for each users authorized
# keys, named for the username. For example: /srv/ssh_authorized_keys/kelly
"repo_path" string => "/srv/ssh_authorized_keys";
}
bundle agent autorun_ssh_key_distribution
{
meta:
# Here we simply tag the bundle for use with the `services_autorun`
# feature.
"tags" slist => { "autorun" };
vars:
"users" slist => { "bob", "frank", "kelly" };
methods:
"Distribute SSH Keys"
usebundle => ssh_key_distribution( $(users) ),
if => userexists( $(users) ),
comment => "It's important that we make sure each of these users
ssh_authorized_keys file has the correct content and
permissions so that they can successfully log in, if
the user exists on the executing agents host.";
}
bundle agent ssh_key_distribution(users)
{
meta:
"description"
string => "Ensure that specified users are able to log in using their ssh
keys";
vars:
# We get the users UID so that we can set permission appropriately
"uid[$(users)]" int => getuid( $(users) );
files:
"/home/$(users)/.ssh/."
create => "true",
perms => mo( 700, "$(uid[$(users)])"),
comment => "It is important to set the proper restrictive permissions and
ownership so that the ssh authorized_keys feature works
correctly.";
"/home/$(users)/.ssh/authorized_keys"
perms => mo( 600, "$(uid[$(users)])" ),
copy_from => remote_dcp( "$(ssh_key_info.repo_path)/$(users)",
$(ssh_key_info.key_server) ),
comment => "We centrally manage and users authorized keys. We source each
users complete authorized_keys file from the central server.";
}
bundle server ssh_key_access_rules
{
meta:
"description"
string => "This bundle handles sharing the directory where ssh keys
are distributed from.";
access:
# Only hosts with class `policy_server` should share the path to ssh
# authorized_keys
policy_server::
"$(ssh_key_info.repo_path)"
admit => { @(def.acl) },
comment => "We share the ssh authorized keys with all authorized
hosts.";
}
This policy can be found in
/var/cfengine/share/doc/examples/simple_ssh_key_distribution.cf
and downloaded directly from
github.
Example Run:
First make sure the users exist on your system.
root@host001:~# useradd bob
root@host001:~# useradd frank
root@host001:~# useradd kelly
Then update the policy and run it:
root@host001:~# cf-agent -Kf update.cf; cf-agent -KI
info: Installing cfe_internal_non_existing_package...
info: Created directory '/home/bob/.ssh/.'
info: Owner of '/home/bob/.ssh' was 0, setting to 1002
info: Object '/home/bob/.ssh' had permission 0755, changed it to 0700
info: Copying from '192.168.33.2:/srv/ssh_authorized_keys/bob'
info: Owner of '/home/bob/.ssh/authorized_keys' was 0, setting to 1002
info: Created directory '/home/frank/.ssh/.'
info: Owner of '/home/frank/.ssh' was 0, setting to 1003
info: Object '/home/frank/.ssh' had permission 0755, changed it to 0700
info: Copying from '192.168.33.2:/srv/ssh_authorized_keys/frank'
info: Owner of '/home/frank/.ssh/authorized_keys' was 0, setting to 1003
info: Created directory '/home/kelly/.ssh/.'
info: Owner of '/home/kelly/.ssh' was 0, setting to 1004
info: Object '/home/kelly/.ssh' had permission 0755, changed it to 0700
info: Copying from '192.168.33.2:/srv/ssh_authorized_keys/kelly'
info: Owner of '/home/kelly/.ssh/authorized_keys' was 0, setting to 1004
Ensure a process is not running
This is a standalone policy that will kill the sleep
process. You can adapt
it to make sure that any undesired process is not running.
body common control
{
bundlesequence => { "process_kill" };
}
bundle agent process_kill
{
processes:
"sleep"
signals => { "term", "kill" }; #Signals are presented as an ordered list to the process.
#On Windows, only the kill signal is supported, which terminates the process.
}
This policy can be found in /var/cfengine/share/doc/examples/unit_process_kill.cf
.
Example run:
# /bin/sleep 1000 &
[1] 5370
# cf-agent -f unit_process_kill.cf
[1]+ Terminated /bin/sleep 1000
#
Now let's do it again with inform mode turned on, and CFEngine will show the process table entry that matched the pattern we specified ("sleep"):
# /bin/sleep 1000 &
[1] 5377
# cf-agent -f unit_process_kill.cf -IK
2013-06-08T16:30:06-0700 info: This agent is bootstrapped to '192.168.183.208'
2013-06-08T16:30:06-0700 info: Running full policy integrity checks
2013-06-08T16:30:06-0700 info: /process_kill/processes/'sleep': Signalled 'term' (15) to process 5377 (root 5377 3854 5377 0.0 0.0 11352 0 612 1 16:30 00:00:00 /bin/sleep 1000)
[1]+ Terminated /bin/sleep 1000
#
If we add the -v switch to turn on verbose mode, we see the /bin/ps command CFEngine used to dump the process table:
# cf-agent -f unit_process_kill.cf -Kv
...
2013-06-08T16:38:20-0700 verbose: Observe process table with /bin/ps -eo user,pid,ppid,pgid,pcpu,pmem,vsz,ni,rss,nlwp,stime,time,args
2013-06-08T16:38:20-0700 verbose: Matched 'root 5474 3854 5474 0.0 0.0 11352 0 612 1 16:38 00:00:00 /bin/sleep 1000'
...
Updating from a central policy server
This is a conceptual example without any test policy associated with it.
The default policy shipped with CFEngine contains a centralized updating of policy that covers more subtleties than this example, and handles fault tolerance. Here is the main idea behind it. For simplicity, we assume that all hosts are on network 10.20.30.* and that the central policy server is 10.20.30.123.
bundle agent update
{
vars:
"master_location" string => "/var/cfengine/masterfiles";
"policy_server" string => "10.20.30.123";
comment => "IP address to locate your policy host.";
files:
"$(sys.workdir)/inputs"
perms => system("600"),
copy_from => remote_cp("$(master_location)",$(policy_server)),
depth_search => recurse("inf"); # This ensures recursive copying of all subdirectories
"$(sys.workdir)/bin"
perms => system("700"),
copy_from => remote_cp("/usr/local/sbin","localhost"),
depth_search => recurse("inf"); # This ensures recursive copying of all subdirectories
}
In addition the server needs to grant access to the clients, this is done in the body server control
:
body server control
{
allowconnects => { "127.0.0.1" , "10.20.30.0/24" };
allowallconnects => { "127.0.0.1" , "10.20.30.0/24" };
trustkeysfrom => { "127.0.0.1" , "10.20.30.0/24" };
}
Since we assume that all hosts are on network 10.20.30.* they will be granted access. In the default policy this is set to $(sys.policy_hub)/16
, i.e. all hosts in the same class B network as the hub will gain access. You will need to modify the access control list in body server control
if you have clients outside of the policy server's class B network.
Granting access to files and folders needs to be done using access
type promises in a server
bundle, for example, bundle server my_access_rules()
:
bundle server my_access_rules()
{
access:
10_20_30_123::
"/var/cfengine/masterfiles"
admit => { "127.0.0.1", "10.20.30.0/24" };
}
Customize Message of the Day
The Message of the Day is displayed when you log in or connect to a server. It typically shows information about the operating system, license information, last login, etc.
It is often useful to customize the Message of the Day to inform your users
about some specifics of the system they are connecting to. In this example we
render a /etc/motd
using a mustache template and add useful information as:
- The role of the server ( staging / production )
- The hostname of the server
- The CFEngine version we are running on the host
- The CFEngine role of the server ( client / hub )
- The administrative contacts details conditionally to the environment
- The primary Ipv4 IP address
- The number of packages updates available for this host
The bundle is defined like this:
bundle agent env_classification
{
vars:
# We use presence of key files to know the hosts environment
"environment_semaphores" slist => { "/etc/prod", "/etc/staging" };
"environment"
string => ifelse( filesexist( @(environment_semaphores) ), "incoherent",
fileexists("/etc/prod"), "production",
fileexists("/etc/staging"), "staging",
"unknown" );
"env_issue_msg"
string => ifelse( strcmp( "incoherent", $(environment) ),
"WARNING: Environment incoherent (multiple environment semaphores)",
strcmp( "unknown", $(environment) ),
"WARNING: Environment unknown (missing environment semaphores)",
"Host environment classified as $(environment)");
# This is to extract the cfengine role ( hub or client )
"cf_role"
string => ifelse( "policy_server", "Policy Server", "Policy Client");
classes:
# We define a class for the selected environment. It's useful for making
# decisions in other policies
"env_$(environment)"
expression => "any",
scope => "namespace";
}
bundle agent env_info
{
vars:
## Based on the environment we define different contacts.
"admin_contact"
slist => { "admin@acme.com", "oncall@acme.com" },
if => strcmp( $(env_classification.environment), "production" );
"admin_contact"
slist => { "dev@acme.com" },
if => strcmp( $(env_classification.environment), "staging" );
"admin_contact"
slist => { "root@localhost" },
if => strcmp( $(env_classification.environment), "unknown" );
## This is to extract the available package updates status
"updates_available"
data => packageupdatesmatching(".*", ".*", ".*", ".*");
"count_updates" int => length("updates_available");
classes:
# We define a class indicating there are updates available, it might be
# useful for various different policies.
"have_updates_available"
expression => isgreaterthan( $(count_updates), 0),
scope => "namespace";
}
bundle agent motd {
vars:
"motd_path" string => "/etc/motd";
# It's considered best practice to prepare a data container holding the
# information you need to render the template instead of relying on
# current datastate()
# First we extract currently defined classes from datastate(), then we
# construct the template data.
"_state" data => datastate(),
if => not( isvariable ( $(this.promiser) ) ); # Limit recursive growth
# and multiple calls to
# datastate() over
# multiple passes.
"template_data"
data => mergedata('{ "fqhost": "$(sys.fqhost)",
"primary_ip": "$(sys.ipv4)",
"cf_version": "$(sys.cf_version)",
"issue_msg": "$(env_classification.env_issue_msg)",
"cf_role": "$(env_classification.cf_role)",
"count_updates": "$(env_info.count_updates)",
"contacts": env_info.admin_contact,
"classes": _state[classes]
}');
files:
"$(motd_path)"
create => "true",
template_method => "inline_mustache",
template_data => @(template_data),
edit_template_string => '# Managed by CFEngine
}
***
*** Welcome to }
***
* *** * CFEngine Role: }
* *** * CFEngine Version:}
* *** *
* * Host IP: }
*** } package updates available.No package updates available or status unknown.
* *
* *
* *
For support contact:
- }$(const.n)';
}
bundle agent __main__
{
methods:
"env_classification";
"env_info";
"motd";
}
This policy can be found in
/var/cfengine/share/doc/examples/mustache_template_motd.cf
and downloaded directly from
github.
Example run:
root@debian8:~/core/examples# cf-agent -KIf ./mustache_template_motd.cf; cat /etc/motd
info: Updated rendering of '/etc/motd' from mustache template 'inline'
info: files promise '/etc/motd' repaired
# Managed by CFEngine
WARNING: Environment unknown (missing environment semaphores)
***
*** Welcome to nickanderson-thinkpad-w550s
***
* *** * CFEngine Role: Policy Client
* *** * CFEngine Version:3.17.0
* *** *
* * Host IP: 192.168.42.189
*** No package updates available or status unknown.
* *
* *
* *
For support contact:
- root@localhost
Restart a Process
This is a standalone policy that will restart three CFEngine processes if they are not running.
body common control
{
bundlesequence => { "process_restart" };
}
bundle agent process_restart
{
vars:
"component" slist => { # List of processes to monitor
"cf-monitord",
"cf-serverd",
"cf-execd"
};
processes:
"$(component)"
restart_class => canonify("start_$(component)"); # Set the class "start_<component>" if it is not running
commands:
"/var/cfengine/bin/$(component)"
ifvarclass => canonify("start_$(component)"); # Evaluate the class "start_<component>", CFEngine will run
# the command if "start_<component> is set.
}
Notes: The canonify
function translates illegal characters to underscores, e.g. start_cf-monitord
becomes start_cf_monitord
. Only alphanumerics and underscores are allowed in CFEngine identifiers (names of variables, classes, bundles, etc.)
This policy can be found in /var/cfengine/share/doc/examples/unit_process_restart.cf
.
Example run:
# ps -ef |grep cf-
root 4305 1 0 15:14 ? 00:00:02 /var/cfengine/bin/cf-execd
root 4311 1 0 15:14 ? 00:00:05 /var/cfengine/bin/cf-serverd
root 4397 1 0 15:15 ? 00:00:06 /var/cfengine/bin/cf-monitord
# kill 4311
# ps -ef |grep cf-
root 4305 1 0 15:14 ? 00:00:02 /var/cfengine/bin/cf-execd
root 4397 1 0 15:15 ? 00:00:06 /var/cfengine/bin/cf-monitord
# cf-agent -f unit_process_restart.cf
# ps -ef |grep cf-
root 4305 1 0 15:14 ? 00:00:02 /var/cfengine/bin/cf-execd
root 4397 1 0 15:15 ? 00:00:06 /var/cfengine/bin/cf-monitord
root 8008 1 0 18:18 ? 00:00:00 /var/cfengine/bin/cf-serverd
#
And again, in Inform mode:
# kill 8008
# cf-agent -f unit_process_restart.cf -I
2013-06-08T18:19:51-0700 info: This agent is bootstrapped to '192.168.183.208'
2013-06-08T18:19:51-0700 info: Running full policy integrity checks
2013-06-08T18:19:51-0700 info: /process_restart/processes/'$(component)': Making a one-time restart promise for 'cf-serverd'
2013-06-08T18:19:51-0700 info: Executing 'no timeout' ... '/var/cfengine/bin/cf-serverd'
2013-06-08T18:19:52-0700 info: Completed execution of '/var/cfengine/bin/cf-serverd'
#
Set up sudo
Setting up sudo is straightforward, we recommend managing it by copying trusted files from a repository. The following bundle will copy a master sudoers file to /etc/sudoers
(/tmp/sudoers
in this example - change it to /etc/sudoers
to use in production).
body common control
{
bundlesequence => { "sudoers" };
inputs => { "libraries/cfengine_stdlib.cf" };
}
bundle agent sudoers
{
# Define the master location of the sudoers file
vars:
"master_location" string => "/var/cfengine/masterfiles";
# Copy the master sudoers file to /etc/sudoers
files:
"/tmp/sudoers" # change to /etc/sudoers to use in production
comment => "Make sure the sudo configuration is secure and up to date",
perms => mog("440","root","root"),
copy_from => secure_cp("$(master_location)/sudoers","$(sys.policy_hub)");
}
We recommend editing the master sudoers file using visudo
or a similar tool. It is possible to use CFEngine's file editing capabilities to edit sudoers directly, but this does not guarantee syntax correctness and you might end up locked out.
Example run:
# cf-agent -f temp.cf -KI
2013-06-08T19:13:21-0700 info: This agent is bootstrapped to '192.168.183.208'
2013-06-08T19:13:22-0700 info: Running full policy integrity checks
2013-06-08T19:13:23-0700 info: Copying from '192.168.183.208:/var/cfengine/masterfiles/sudoers'
2013-06-08T19:13:23-0700 info: /sudoers/files/'/tmp/sudoers': Object '/tmp/sudoers' had permission 0600, changed it to 0440
#
For reference we include an example of a simple sudoers file:
# /etc/sudoers
#
# This file MUST be edited with the 'visudo' command as root.
#
Defaults env_reset
# User privilege specification
root ALL=(ALL) ALL
# Allow members of group sudo to execute any command after they have
# provided their password
%sudo ALL=(ALL) ALL
# Members of the admin group may gain root privileges
%admin ALL=(ALL) ALL
john ALL=(ALL) ALL
Set up name resolution with DNS
There are many ways to configure name resolution. A simple and straightforward approach is to implement this as a simple editing promise for the /etc/resolv.conf
file.
body common control
{
bundlesequence => { "edit_name_resolution" };
}
bundle agent edit_name_resolution
{
files:
"/tmp/resolv.conf" # This is for testing, change to "$(sys.resolv)" to put in production
comment => "Add lines to the resolver configuration",
create => "true", # Make sure the file exists, create it if not
edit_line => resolver, # Call the resolver bundle defined below to do the editing
edit_defaults => empty; # Baseline memory model of file to empty before processing
# bundle edit_line resolver
}
bundle edit_line resolver
{
insert_lines:
any:: # Class/context where you use the below nameservers. Change to appropriate class
# for your system (if not any::, for example server_group::, ubuntu::, etc.)
# insert the search domain or name servers we want
"search mydomain.tld" location => start; # Replace mydomain.tld with your domain name
# The search line will always be at the start of the file
"nameserver 128.39.89.8";
"nameserver 128.39.74.66";
}
body edit_defaults empty
{
empty_file_before_editing => "true";
}
body location start
{
before_after => "before";
}
Example run:
# cf-agent -f unit_edit_name_resolution.cf # set up resolv.conf
# cat /tmp/resolv.conf # show resolv.conf
search mydomain.tld
nameserver 128.39.89.8
nameserver 128.39.74.66
# echo 'nameserver 0.0.0.0' >> /tmp/resolv.conf # mess up resolv.conf
# cf-agent -f ./unit_edit_name_resolution.cf -KI # heal resolv.conf
2013-06-08T18:38:12-0700 info: This agent is bootstrapped to '192.168.183.208'
2013-06-08T18:38:12-0700 info: Running full policy integrity checks
2013-06-08T18:38:12-0700 info: /edit_name_resolution/files/'/tmp/resolv.conf': Edit file '/tmp/resolv.conf'
# cat /tmp/resolv.conf # show healed resolv.conf
search mydomain.tld
nameserver 128.39.89.8
nameserver 128.39.74.66
#
Find the MAC address
Finding the ethernet address can vary between operating systems.
We will use CFEngine's built in function execresult
to execute commands
adapted for different operating systems, assign the output to variables,
and filter for the MAC address. We then report on the result.
body common control
{
bundlesequence => { "example" };
}
bundle agent example
{
vars:
linux::
"interface" string => execresult("/sbin/ifconfig eth0", "noshell");
solaris::
"interface" string => execresult("/usr/sbin/ifconfig bge0", "noshell");
freebsd::
"interface" string => execresult("/sbin/ifconfig le0", "noshell");
darwin::
"interface" string => execresult("/sbin/ifconfig en0", "noshell");
# Use the CFEngine function 'regextract' to match the MAC address,
# assign it to an array called mac and set a class to indicate positive match
classes:
linux::
"ok" expression => regextract(
".*HWaddr ([^\s]+).*(\n.*)*", # pattern to match
"$(interface)", # string to scan for pattern
"mac" # put the text that matches the pattern into this array
);
solaris|freebsd::
"ok" expression => regextract(
".*ether ([^\s]+).*(\n.*)*",
"$(interface)",
"mac"
);
darwin::
"ok" expression => regextract(
"(?s).*ether ([^\s]+).*(\n.*)*",
"$(interface)",
"mac"
);
# Report on the result
reports:
ok::
"MAC address is $(mac[1])"; # return first element in array "mac"
}
This policy can be found in /var/cfengine/masterfiles/example_find_mac_addr.cf
Example run:
# cf-agent -f example_find_mac_addr.cf
2013-06-08T16:59:19-0700 notice: R: MAC address is a4:ba:db:d7:59:32
#
While the above illustrates the flexiblity of CFEngine in running external commands and parsing their output, as of CFEngine 3.3.0, Nova 2.2.0 (2011), you can get the MAC address natively:
body common control
{
bundlesequence => { "example" };
}
bundle agent example
{
vars:
linux:: "interface" string => "eth0";
solaris:: "interface" string => "bge0";
freebsd:: "interface" string => "le0";
darwin:: "interface" string => "en0";
reports:
"MAC address of $(interface) is: $(sys.hardware_mac[$(interface)])";
}
Install packages
Install desired packages.
body common control
{
bundlesequence => { "install_packages" };
inputs => { "libraries/cfengine_stdlib.cf" };
}
bundle agent install_packages
{
vars:
"desired_packages"
slist => { # list of packages we want
"ntp",
"lynx"
};
packages:
"$(desired_packages)" # operate on listed packages
package_policy => "add", # What to do with packages: install them.
package_method => generic; # Infer package manager (e.g. apt, yum) from the OS.
}
Caution: package management is a dirty business. If things don't go smoothly
using the generic method, you may have to use a method specific to your package
manager and get to your elbows in the details. But try generic
first. You
may get lucky.
Mind package names can differ OS to OS. For example, Apache httpd is "httpd" on Red Hat, and "apache2" on Debian.
Version comparison can be tricky when involving multipart version identifiers with numbers and letters.
CFEngine downloads the necessary packages from the default repositories if they are not present on the local machine, then installs them if they are not already installed.
Example run:
# dpkg -r lynx ntp # remove packages so CFEngine has something to repair
(Reading database ... 234887 files and directories currently installed.)
Removing lynx ...
Removing ntp ...
* Stopping NTP server ntpd [ OK ]
Processing triggers for ureadahead ...
Processing triggers for man-db ...
# cf-agent -f install_packages.cf # install packages
# dpkg -l lynx ntp # show installed packages
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name Version Architecture Description
+++-===============================-====================-====================-====================================================================
ii lynx 2.8.8dev.12-2ubuntu0 all Text-mode WWW Browser (transitional package)
ii ntp 1:4.2.6.p3+dfsg-1ubu amd64 Network Time Protocol daemon and utility programs
#
There are examples in /var/cfengine/share/doc/examples/
of installing packages using specific package managers:
- Red Hat (unit_package_yum.cf)
- Debian (unit_package_apt.cf)
- MSI for Windows (unit_package_msi_file.cf)
- Solaris (unit_package_solaris.cf)
- SuSE Linux (unit_package_zypper.cf)
Ensure a service is enabled and running
This example shows how to ensure services are started or stopped appropriately.
body file control
{
inputs => { "$(sys.libdir)/services.cf", "$(sys.libdir)/commands.cf" };
}
bundle agent main
{
vars:
linux::
"enable[ssh]"
string => ifelse( "debian|ubuntu", "ssh", "sshd"),
comment => "The name of the ssh service varies on different platforms.
Here we set the name of the service based on existing
classes and defaulting to `sshd`";
"disable[apache]"
string => ifelse( "debian|ubuntu", "apache2", "httpd" ),
comment => "The name of the apache web service varies on different
platforms. Here we set the name of the service based on
existing classes and defaulting to `httpd`";
"enable[cron]"
string => ifelse( "debian|ubuntu", "cron", "crond" ),
comment => "The name of the cron service varies on different
platforms. Here we set the name of the service based on
existing classes and defaulting to `crond`";
"disable[cups]"
string => "cups",
comment => "Printing services are not needed on most hosts.";
"enabled" slist => getvalues( enable );
"disabled" slist => getvalues( disable );
services:
linux::
"$(enabled)" -> { "SysOps" }
service_policy => "start",
comment => "These services should be running because x, y or z.";
"$(disabled)" -> { "SysOps" }
service_policy => "stop",
comment => "These services should not be running because x, y or z.";
systemd::
"sysstat"
service_policy => "stop",
comment => "Standard service handling for sysstat only works with
systemd. Other inits need cron entries managed.";
}
This policy can be found in
/var/cfengine/share/doc/examples/services.cf
and downloaded directly from
github.
Note: Not all services behave in the standard way. Some services may require custom handling. For example it is not uncommon for some services to not provide correct return codes for status checks.
See also:
Example usage on systemd
We can see that before the policy run sysstat
is inactive, apache2
is
active, cups
is active, ssh
is active and cron
is inactive.
root@ubuntu:# systemctl is-active sysstat apache2 cups ssh cron
inactive
active
active
active
inactive
Now we run the policy to converge the system to the desired state.
root@ubuntu:# cf-agent --no-lock --inform --file ./services.cf
info: Executing 'no timeout' ... '/bin/systemctl --no-ask-password --global --system -q stop apache2'
info: Completed execution of '/bin/systemctl --no-ask-password --global --system -q stop apache2'
info: Executing 'no timeout' ... '/bin/systemctl --no-ask-password --global --system -q stop cups'
info: Completed execution of '/bin/systemctl --no-ask-password --global --system -q stop cups'
info: Executing 'no timeout' ... '/bin/systemctl --no-ask-password --global --system -q start cron'
info: Completed execution of '/bin/systemctl --no-ask-password --global --system -q start cron'
After the policy run we can see that systat
, apache2
, and cups
are
inactive. ssh
and cron
are active as specified in the policy.
root@ubuntu:/home/nickanderson/CFEngine/core/examples# systemctl is-active sysstat apache2 cups ssh cron
inactive
inactive
inactive
active
active
Example usage with System V
We can see that before the policy run sysstat
is not reporting status
correctly , httpd
is running, cups
is running, sshd
is running and
crond
is not running.
[root@localhost examples]# service sysstat status; echo $?
3
[root@localhost examples]# service httpd status; echo $?
httpd (pid 3740) is running...
0
[root@localhost examples]# service cups status; echo $?
cupsd (pid 3762) is running...
0
[root@localhost examples]# service sshd status; echo $?
openssh-daemon (pid 3794) is running...
0
[root@localhost examples]# service crond status; echo $?
crond is stopped
3
Now we run the policy to converge the system to the desired state.
[root@localhost examples]# cf-agent -KIf ./services.cf
info: Executing 'no timeout' ... '/etc/init.d/crond start'
info: Completed execution of '/etc/init.d/crond start'
info: Executing 'no timeout' ... '/etc/init.d/httpd stop'
info: Completed execution of '/etc/init.d/httpd stop'
info: Executing 'no timeout' ... '/etc/init.d/cups stop'
info: Completed execution of '/etc/init.d/cups stop'
After the policy run we can see that systat
is still not reporting status correctly (some services do not respond to standard checks), apache2
, and cups
are
inactive. ssh
and cron
are active as specified in the policy.
[root@localhost examples]# service sysstat status; echo $?
3
[root@localhost examples]# service httpd status; echo $?
httpd is stopped
3
[root@localhost examples]# service cups status; echo $?
cupsd is stopped
3
[root@localhost examples]# service sshd status; echo $?
openssh-daemon (pid 3794) is running...
0
[root@localhost examples]# service crond status; echo $?
crond (pid 3929) is running...
0
Create files and directories
The following is a standalone policy that will create the file
/home/mark/tmp/test_plain
and the directory /home/mark/tmp/test_dir/
and set permissions on both.
body common control
{
bundlesequence => { "example" };
}
bundle agent example
{
files:
"/home/mark/tmp/test_plain"
The promiser specifies the path and name of the file.
perms => system,
create => "true";
The perms
attribute sets the file permissions as defined in the system
body below. The create
attribute makes sure that the files exists. If it
doesn't, CFEngine will create it.
"/home/mark/tmp/test_dir/."
perms => system,
create => "true";
The trailing /.
in the filename tells CFEngine that the promiser is a
directory.
}
body perms system
{
mode => "0640";
}
This body sets permissions to "0640"
This policy can be found in
/var/cfengine/share/doc/examples/create_filedir.cf
and downloaded directly from
github.
Example output:
# cf-agent -f unit_create_filedir.cf -I
2013-06-08T14:56:26-0700 info: /example/files/'/home/mark/tmp/test_plain': Created file '/home/mark/tmp/test_plain', mode 0640
2013-06-08T14:56:26-0700 info: /example/files/'/home/mark/tmp/test_dir/.': Created directory '/home/mark/tmp/test_dir/.'
2013-06-08T14:56:26-0700 info: /example/files/'/home/mark/tmp/test_dir/.': Object '/home/mark/tmp/test_dir' had permission 0755, changed it to 0750
#
Set up time management through NTP
The following sets up a local NTP server that synchronizes with pool.ntp.org and clients that synchronize with your local NTP server. See bottom of this example if you don't want to build a server, but use a "brute force" method (repeated ntpdate syncs).
This example demonstrates you can have a lot of low-level detailed control if you want it.
bundle agent system_time_ntp
{
vars:
linux::
"cache_dir" string => "$(sys.workdir)/cache"; # Cache directory for NTP config files
"ntp_conf" string => "/etc/ntp.conf"; # Target file for NTP configuration
"ntp_server" string => "172.16.12.161"; #
"ntp_network" string => "172.16.12.0"; # IP address and netmask of your local NTP server
"ntp_mask" string => "255.255.255.0"; #
"ntp_pkgs" slist => { "ntp" }; # NTP packages to be installed to ensure service
# Define a class for the NTP server
classes:
any::
"ntp_hosts" or => { classmatch(canonify("ipv4_$(ntp_server)")) };
# Ensure that the NTP packages are installed
packages:
ubuntu::
"$(ntp_pkgs)"
comment => "setup NTP",
package_policy => "add",
package_method => generic;
# Ensure existence of file and directory for NTP drift learning statistics
files:
linux::
"/var/lib/ntp/ntp.drift"
comment => "Enable ntp service",
create => "true";
"/var/log/ntpstats/."
comment => "Create a statistic directory",
perms => mog("644","ntp","ntp"),
create => "true";
ntp_hosts::
# Build the cache configuration file for the NTP server
"/var/cfengine/cache/ntp.conf"
comment => "Build $(this.promiser) cache file for NTP server",
create => "true",
edit_defaults => empty,
edit_line => restore_ntp_master("$(ntp_network)","$(ntp_mask)");
centos.ntp_hosts::
# Copy the cached configuration file to its target destination
"$(ntp_conf)"
comment => "Ensure $(this.promiser) in a perfect condition",
copy_from => local_cp("$(cache_dir)/ntp.conf"),
classes => if_repaired("refresh_ntpd_centos");
ubuntu.ntp_hosts::
"$(ntp_conf)"
comment => "Ensure $(this.promiser) in a perfect condition",
copy_from => local_cp("$(cache_dir)/ntp.conf"),
classes => if_repaired("refresh_ntpd_ubuntu");
!ntp_hosts::
# Build the cache configuration file for the NTP client
"$(cache_dir)/ntp.conf"
comment => "Build $(this.promiser) cache file for NTP client",
create => "true",
edit_defaults => empty,
edit_line => restore_ntp_client("$(ntp_server)");
centos.!ntp_hosts::
# Copy the cached configuration file to its target destination
"$(ntp_conf)"
comment => "Ensure $(this.promiser) in a perfect condition",
copy_from => local_cp("$(cache_dir)/ntp.conf"),
classes => if_repaired("refresh_ntpd_centos");
ubuntu.!ntp_hosts::
"$(ntp_conf)"
comment => "Ensure $(this.promiser) in a perfect condition",
copy_from => local_cp("$(cache_dir)/ntp.conf"),
classes => if_repaired("refresh_ntpd_ubuntu");
# Set classes (conditions) for to restart the NTP daemon if there have been any changes to configuration
processes:
centos::
"ntpd.*"
restart_class => "refresh_ntpd_centos";
ubuntu::
"ntpd.*"
restart_class => "refresh_ntpd_ubuntu";
# Restart the NTP daemon if the configuration has changed
commands:
refresh_ntpd_centos::
"/etc/init.d/ntpd restart";
refresh_ntpd_ubuntu::
"/etc/init.d/ntp restart";
}
#######################################################
bundle edit_line restore_ntp_master(network,mask)
{
vars:
"list" string =>
"######################################
# ntp.conf-master
driftfile /var/lib/ntp/ntp.drift
statsdir /var/log/ntpstats/
statistics loopstats peerstats clockstats
filegen loopstats file loopstats type day enable
filegen peerstats file peerstats type day enable
filegen clockstats file clockstats type day enable
# Use public servers from the pool.ntp.org project.
# Please consider joining the pool (http://www.pool.ntp.org/join.html).
# Consider changing the below servers to a location near you for better time
# e.g. server 0.europe.pool.ntp.org, or server 0.no.pool.ntp.org etc.
server 0.centos.pool.ntp.org
server 1.centos.pool.ntp.org
server 2.centos.pool.ntp.org
# Permit time synchronization with our time source, but do not
# permit the source to query or modify the service on this system.
restrict -4 default kod nomodify notrap nopeer noquery
restrict -6 default kod nomodify notrap nopeer noquery
# Permit all access over the loopback interface. This could
# be tightened as well, but to do so would effect some of
# the administrative functions.
restrict 127.0.0.1
restrict ::1
# Hosts on local network are less restricted.
restrict $(network) mask $(mask) nomodify notrap";
insert_lines:
"$(list)";
}
#######################################################
bundle edit_line restore_ntp_client(serverip)
{
vars:
"list" string =>
"######################################
# This file is protected by cfengine #
######################################
# ntp.conf-client
driftfile /var/lib/ntp/ntp.drift
statsdir /var/log/ntpstats/
statistics loopstats peerstats clockstats
filegen loopstats file loopstats type day enable
filegen peerstats file peerstats type day enable
filegen clockstats file clockstats type day enable
# Permit time synchronization with our time source, but do not
# permit the source to query or modify the service on this system.
restrict -4 default kod nomodify notrap nopeer noquery
restrict -6 default kod nomodify notrap nopeer noquery
# Permit all access over the loopback interface. This could
# be tightened as well, but to do so would effect some of
# the administrative functions.
restrict 127.0.0.1
restrict ::1
server $(serverip)
restrict $(serverip) nomodify";
insert_lines:
"$(list)";
}
This policy can be found in /var/cfengine/share/doc/examples/example_ntp.cf
If you don't want to build a server, you might do like this:
bundle agent time_management
{
vars:
any::
"ntp_server" string => "no.pool.ntp.org";
commands:
any::
"/usr/sbin/ntpdate $(ntp_server)"
contain => silent;
}
This is a hard reset of the time, it corrects it immediately. This may cause problems if there are large deviations in time and you are using time sensitive software on your system. An NTP daemon setup as shown above, on the other hand, slowly adapts the time to avoid causing disruption. In addition, the NTP daemon can be configured to learn your system's time drift and automatically adjust for it without having to be in touch with the server at all times.
Mount NFS filesystem
Mounting an NFS filesystem is straightforward using CFEngine's storage promises. The following bundle specifies the name of a remote file system server, the path of the remote file system and the mount point directory on the local machine:
body common control
{
bundlesequence => { "mounts" };
}
bundle agent mounts
{
storage:
"/mnt" mount => nfs("fileserver","/home"); # "/mnt" is the local moint point
# "fileserver" is the remote fileserver
# "/home" is the path to the remote file system
}
body mount nfs(server,source)
{
mount_type => "nfs"; # Protocol type of remote file system
mount_source => "$(source)"; # Path of remote file system
mount_server => "$(server)"; # Name or IP of remote file system server
mount_options => { "rw" }; # List of option strings to add to the file system table ("fstab")
edit_fstab => "true"; # True/false add or remove entries to the file system table ("fstab")
}
This policy can be found in /var/cfengine/share/doc/examples/example_mount_nfs.cf
Here is an example run. At start, the filesystem is not in /etc/fstab and is not mounted:
# grep mnt /etc/fstab # filesystem is not in /etc/fstab
# df |grep mnt # filesystem is not mounted
Now we run CFEngine to mount the filesystem and add it to /etc/fstab:
# cf-agent -f example_mount_nfs.cf
2013-06-08T17:48:42-0700 error: Attempting abort because mount went into a retry loop.
# grep mnt /etc/fstab
fileserver:/home /mnt nfs rw
# df |grep mnt
fileserver:/home 149912064 94414848 47882240 67% /mnt
#
Note: CFEngine errors out after it mounts the filesystem and updates /etc/fstab. There is a ticket https://cfengine.com/dev/issues/2937 open on this issue.
CFEngine Administration Examples
Ordering promises
This counts to five by default. If we change ‘/bin/echo one’ to ‘/bin/echox one’, then the command will fail, causing us to skip five and go to six instead.
This shows how dependencies can be chained in spite of the order of promises in the bundle.
Normally the order of promises in a bundle is followed, within each promise type, and the types are ordered according to normal ordering.
body common control
{
bundlesequence => { "order" };
}
bundle agent order
{
vars:
"list" slist => { "three", "four" };
commands:
ok_later::
"/bin/echo five";
otherthing::
"/bin/echo six";
any::
"/bin/echo one" classes => d("ok_later","otherthing");
"/bin/echo two";
"/bin/echo $(list)";
preserved_class::
"/bin/echo seven";
}
body classes d(if,else)
{
promise_repaired => { "$(if)" };
repair_failed => { "$(else)" };
persist_time => "0";
}
Aborting execution
body common control
{
bundlesequence => { "testbundle" };
version => "1.2.3";
}
body agent control
{
abortbundleclasses => { "invalid.Hr16" };
}
bundle agent testbundle
{
vars:
"userlist" slist => { "xyz", "mark", "jeang", "jonhenrik", "thomas", "eben" };
methods:
"any" usebundle => subtest("$(userlist)");
}
bundle agent subtest(user)
{
classes:
"invalid" not => regcmp("[a-z][a-z][a-z][a-z]","$(user)");
reports:
!invalid::
"User name $(user) is valid at 4 letters";
invalid::
"User name $(user) is invalid";
}
Measuring Examples
Measurements
body common control
{
bundlesequence => { "report" };
}
body monitor control
{
forgetrate => "0.7";
histograms => "true";
}
bundle agent report
{
reports:
"
Free memory read at $(mon.av_free_memory_watch)
cf_monitord read $(mon.value_monitor_self_watch)
";
}
bundle monitor watch
{
measurements:
# Test 1 - extract string matching
"/home/mark/tmp/testmeasure"
handle => "blonk_watch",
stream_type => "file",
data_type => "string",
history_type => "weekly",
units => "blonks",
match_value => find_blonks,
action => sample_min("10");
# Test 2 - follow a special process over time
# using cfengine's process cache to avoid resampling
"/var/cfengine/state/cf_rootprocs"
handle => "monitor_self_watch",
stream_type => "file",
data_type => "int",
history_type => "static",
units => "kB",
match_value => proc_value(".*cf-monitord.*",
"root\s+[0-9.]+\s+[0-9.]+\s+[0-9.]+\s+[0-9.]+\s+([0-9]+).*");
# Test 3, discover disk device information
"/bin/df"
handle => "free_disk_watch",
stream_type => "pipe",
data_type => "slist",
history_type => "static",
units => "device",
match_value => file_system;
# Update this as often as possible
# Test 4
"/tmp/file"
handle => "line_counter",
stream_type => "file",
data_type => "counter",
match_value => scanlines("MYLINE.*"),
history_type => "log";
}
body match_value scanlines(x)
{
select_line_matching => "^$(x)$";
}
body action sample_min(x)
{
ifelapsed => "$(x)";
expireafter => "$(x)";
}
body match_value find_blonks
{
select_line_number => "2";
extraction_regex => "Blonk blonk ([blonk]+).*";
}
body match_value free_memory # not willy!
{
select_line_matching => "MemFree:.*";
extraction_regex => "MemFree:\s+([0-9]+).*";
}
body match_value proc_value(x,y)
{
select_line_matching => "$(x)";
extraction_regex => "$(y)";
}
body match_value file_system
{
select_line_matching => "/.*";
extraction_regex => "(.*)";
}
Software Administration Examples
- Software and patch installation
- Postfix mail configuration
- Set up a web server
- Add software packages to the system
- Application baseline
- Service management (windows)
- Software distribution
- Web server modules
- Ensure a service is enabled and running
- Managing Software
- Install packages
Software and patch installation
Example for Debian:
body common control
{
bundlesequence => { "packages" };
}
body agent control
{
environment => { "DEBIAN_FRONTEND=noninteractive" };
}
bundle agent packages
{
vars:
# Test the simplest case -- leave everything to the yum smart manager
"match_package" slist => {
"apache2"
# "apache2-mod_php5",
# "apache2-prefork",
# "php5"
};
packages:
"$(match_package)"
package_policy => "add",
package_method => apt;
}
body package_method apt
{
any::
# ii acpi 0.09-3ubuntu1
package_changes => "bulk";
package_list_command => "/usr/bin/dpkg -l";
package_list_name_regex => "ii\s+([^\s]+).*";
package_list_version_regex => "ii\s+[^\s]+\s+([^\s]+).*";
# package_list_arch_regex => "none";
package_installed_regex => ".*"; # all reported are installed
#package_name_convention => "$(name)_$(version)_$(arch)";
package_name_convention => "$(name)";
# Use these only if not using a separate version/arch string
# package_version_regex => "";
# package_name_regex => "";
# package_arch_regex => "";
package_add_command => "/usr/bin/apt-get --yes install";
package_delete_command => "/usr/bin/apt-get --yes remove";
package_update_command => "/usr/bin/apt-get --yes dist-upgrade";
#package_verify_command => "/bin/rpm -V";
}
Examples MSI for Windows, by name:
body common control
{
bundlesequence => { "packages" };
}
bundle agent packages
{
vars:
"match_package" slist => {
"7zip"
};
packages:
"$(match_package)"
package_policy => "update",
package_select => ">=",
package_architectures => { "x86_64" },
package_version => "3.00",
package_method => msi_vmatch;
}
body package_method msi_vmatch
{
package_changes => "individual";
package_file_repositories => { "$(sys.workdir)\software_updates\windows", "s:\su" };
package_installed_regex => ".*";
package_name_convention => "$(name)-$(version)-$(arch).msi";
package_add_command => "\"$(sys.winsysdir)\msiexec.exe\" /qn /i";
package_update_command => "\"$(sys.winsysdir)\msiexec.exe\" /qn /i";
package_delete_command => "\"$(sys.winsysdir)\msiexec.exe\" /qn /x";
}
Windows MSI by version:
body common control
{
bundlesequence => { "packages" };
}
bundle agent packages
{
vars:
"match_package" slist => {
"7zip"
};
packages:
"$(match_package)"
package_policy => "update",
package_select => ">=",
package_architectures => { "x86_64" },
package_version => "3.00",
package_method => msi_vmatch;
}
body package_method msi_vmatch
{
package_changes => "individual";
package_file_repositories => { "$(sys.workdir)\software_updates\windows", "s:\su" };
package_installed_regex => ".*";
package_name_convention => "$(name)-$(version)-$(arch).msi";
package_add_command => "\"$(sys.winsysdir)\msiexec.exe\" /qn /i";
package_update_command => "\"$(sys.winsysdir)\msiexec.exe\" /qn /i";
package_delete_command => "\"$(sys.winsysdir)\msiexec.exe\" /qn /x";
}
Examples for solaris:
bundle agent example_using_ips_package_method
{
packages:
solaris::
"shell/zsh"
package_policy => "add",
package_method => ips;
}
bundle agent example_using_solaris_package_method
{
files:
solaris::
"/tmp/$(admin_file)"
create => "true",
edit_defaults => empty_file, # defined in stdlib
edit_line => create_solaris_admin_file; # defined in stdlib
packages:
solaris::
"SMCzlib"
package_policy => "add",
package_method => solaris( "SMCzlib",
"zlib-1.2.3-sol10-sparc-local",
"$(admin_file)");
}
bundle agent example_using_solaris_install_package_method
{
packages:
solaris::
"SMCzlib"
package_method => solaris_install("/tmp/SMCzlib.adminfile")
}
bundle agent example_using_pkgsrc_module
{
packages:
solaris::
"vim"
policy => "present",
package_module => pkgsrc;
}
Examples for yum based systems:
body common control
{
bundlesequence => { "packages" };
inputs => { "cfengine_stdlib.cf" };
}
bundle agent packages
{
vars:
# Test the simplest case -- leave everything to the yum smart manager
"match_package" slist => {
"apache2",
"apache2-mod_php5",
"apache2-prefork",
"php5"
};
packages:
"$(match_package)"
package_policy => "add",
package_method => yum;
}
SuSE Linux's package manager zypper is the most powerful alternative:
body common control
{
bundlesequence => { "packages" };
inputs => { "cfengine_stdlib.cf" }
}
bundle agent packages
{
vars:
# Test the simplest case -- leave everything to the zypper smart manager
"match_package" slist => {
"apache2",
"apache2-mod_php5",
"apache2-prefork",
"php5"
};
packages:
"$(match_package)"
package_policy => "add",
package_method => zypper;
}
Postfix mail configuration
body common control
{
inputs => { "$(sys.libdir)/stdlib.cf" };
bundlesequence => { postfix };
}
bundle agent postfix
{
vars:
"prefix" string => "/etc";
"smtpserver" string => "localhost";
"mailrelay" string => "mailx.example.org";
files:
"$(prefix)/main.cf"
edit_line => prefix_postfix;
"$(prefix)/sasl-passwd"
create => "true",
perms => mo("0600","root"),
edit_line => append_if_no_line("$(smtpserver) _$(sys.fqhost):chmsxrcynz4etfrejizhs22");
}
bundle edit_line prefix_postfix
{
#
# Value have the form NAME = "quoted space separated list"
#
vars:
"ps[relayhost]" string => "[$(postfix.mailrelay)]:587";
"ps[mydomain]" string => "iu.hio.no";
"ps[smtp_sasl_auth_enable]" string => "yes";
"ps[smtp_sasl_password_maps]" string => "hash:/etc/postfix/sasl-passwd";
"ps[smtp_sasl_security_options]" string => "";
"ps[smtp_use_tls]" string => "yes";
"ps[default_privs]" string => "mailman";
"ps[inet_protocols]" string => "all";
"ps[inet_interfaces]" string => "127.0.0.1";
"parameter_name" slist => getindices("ps");
delete_lines:
"$(parameter_name).*";
insert_lines:
"$(parameter_name) = $(ps[$(parameter_name)])";
}
bundle edit_line AppendIfNSL(parameter)
{
insert_lines:
"$(parameter)"; # This is default
}
Set up a web server
Adapt this template to your operating system by adding multiple classes. Each web server runs something like the present module, which is entered into the bundlesequence like this:
bundle agent web_server(state)
{
vars:
"document_root" string => "/";
####################################################
# Site specific configuration - put it in this file
####################################################
"site_http_conf" string => "/home/mark/CFEngine-inputs/httpd.conf";
####################################################
# Software base
####################################################
"match_package" slist => {
"apache2",
"apache2-mod_php5",
"apache2-prefork",
"php5"
};
#########################################################
processes:
web_ok.on::
"apache2"
restart_class => "start_apache";
off::
"apache2"
process_stop => "/etc/init.d/apache2 stop";
#########################################################
commands:
start_apache::
"/etc/init.d/apache2 start"; # or startssl
#########################################################
packages:
"$(match_package)"
package_policy => "add",
package_method => zypper,
classes => if_ok("software_ok");
#########################################################
files:
software_ok::
"/etc/sysconfig/apache2"
edit_line => fixapache,
classes => if_ok("web_ok");
#########################################################
reports:
!software_ok.on::
"The web server software could not be installed";
#########################################################
classes:
"on" expression => strcmp("$(state)","on");
"off" expression => strcmp("$(state)","off");
}
bundle edit_line fixapache
{
vars:
"add_modules" slist => {
"ssl",
"php5"
};
"del_modules" slist => {
"php3",
"php4",
"jk"
};
insert_lines:
"APACHE_CONF_INCLUDE_FILES=\"$(web_server.site_http_conf)\"";
field_edits:
#####################################################################
# APACHE_MODULES="actions alias ssl php5 dav_svn authz_default jk" etc..
#####################################################################
"APACHE_MODULES=.*"
# Insert module "columns" between the quoted RHS
# using space separators
edit_field => quotedvar("$(add_modules)","append");
"APACHE_MODULES=.*"
# Delete module "columns" between the quoted RHS
# using space separators
edit_field => quotedvar("$(del_modules)","delete");
# if this line already exists, edit it
}
Add software packages to the system
body common control
{
inputs => { "$(sys.libdir)/packages.cf" }
bundlesequence => { "packages" };
}
bundle agent packages
{
vars:
"match_package" slist => {
"apache2",
"apache2-mod_php5",
"apache2-prefork",
"php5"
};
packages:
solaris::
"$(match_package)"
package_policy => "add",
package_method => solaris;
redhat|SuSE::
"$(match_package)"
package_policy => "add",
package_method => yum_rpm;
methods:
# equivalent in 3.6, no OS choices
"" usebundle => ensure_present($(match_package));
}
Note you can also arrange to hide all the differences between package managers on an OS basis, but since some OSs have multiple managers, this might not be 100 percent correct.
Application baseline
bundle agent app_baseline
{
methods:
windows::
"any" usebundle => detect_adobereader;
}
bundle agent detect_adobereader
{
vars:
windows::
"value1" string => registryvalue("HKEY_LOCAL_MACHINE\SOFTWARE\Adobe\Acrobat Reader\9.0\Installer", "ENU_GUID");
"value2" string => registryvalue("HKEY_LOCAL_MACHINE\SOFTWARE\Adobe\Acrobat Reader\9.0\Installer", "VersionMax");
"value3" string => registryvalue("HKEY_LOCAL_MACHINE\SOFTWARE\Adobe\Acrobat Reader\9.0\Installer", "VersionMin");
classes:
windows::
"is_correct" and => {
strcmp($(value1), "{AC76BA86-7AD7-1033-7B44-A93000000001}"),
strcmp($(value2), "90003"),
islessthan($(value3), "10001" )
};
reports:
windows.!is_correct::
'Adobe Reader is not correctly deployed - got "$(value1)", "$(value2)", "$(value3)"';
}
Service management (windows)
body common control
{
bundlesequence => { "winservice" };
}
bundle agent winservice
{
vars:
"bad_services" slist => { "Alerter", "ClipSrv" };
services:
windows::
"$(bad_services)"
service_policy => "disable",
comment => "Disable services that create security issues";
}
Software distribution
bundle agent check_software
{
vars:
# software to install if not installed
"include_software" slist => {
"7-zip-4.50-$(sys.arch).msi"
};
# this software gets updated if it is installed
"autoupdate_software" slist => {
"7-zip"
};
# software to uninstall if it is installed
"exclude_software" slist => {
"7-zip-4.65-$(sys.arch).msi"
};
methods:
# "any" usebundle => add_software( "@(check_software.include_software)", "$(sys.policy_hub)" );
# "any" usebundle => update_software( "@(check_software.autoupdate_software)", "$(sys.policy_hub)" );
# "any" usebundle => remove_software( "@(check_software.exclude_software)", "$(sys.policy_hub)" );
}
bundle agent add_software(pkg_name)
{
vars:
# dir to install from locally - can also check multiple directories
"local_software_dir" string => "C:\Program Files\Cfengine\software\add";
files:
"$(local_software_dir)"
copy_from => remote_cp("/var/cfengine/master_software_updates/$(sys.flavour)_$(sys.arch)/add", "$(srv)"),
depth_search => recurse("1"),
classes => if_repaired("got_newpkg"),
comment => "Copy software from remote repository";
packages:
# When to check if the package is installed ?
got_newpkg|any::
"$(pkg_name)"
package_policy => "add",
package_method => msi_implicit( "$(local_software_dir)" ),
classes => if_else("add_success", "add_fail" ),
comment => "Install new software, if not already present";
reports::
add_fail::
"Failed to install one or more packages";
}
#########################################################################
bundle agent update_software(sw_names)
{
vars:
# dir to install from locally - can also check multiple directories
"local_software_dir" string => "C:\Program Files\Cfengine\software\update";
files:
"$(local_software_dir)"
copy_from => remote_cp("/var/cfengine/master_software_updates/$(sys.flavour)_$(sys.arch)/update", "$(srv)"),
depth_search => recurse("1"),
classes => if_repaired("got_newpkg"),
comment => "Copy software updates from remote repository";
packages:
# When to check if the package is updated ?
got_newpkg|any::
"$(sw_names)"
package_policy => "update",
package_select => ">=", # picks the newest update available
package_architectures => { "$(sys.arch)" }, # install 32 or 64 bit package ?
package_version => "1.0", # at least version 1.0
package_method => msi_explicit( "$(local_software_dir)" ),
classes => if_else("update_success", "update_fail");
reports:
update_fail::
"Failed to update one or more packages";
}
#########################################################################
bundle agent remove_software(pkg_name)
{
vars:
# dir to install from locally - can also check multiple directories
"local_software_dir" string => "C:\Program Files\Cfengine\software\remove";
files:
"$(local_software_dir)"
copy_from => remote_cp("/var/cfengine/master_software_updates/$(sys.flavour)_$(sys.arch)/remove", "$(srv)"),
depth_search => recurse("1"),
classes => if_repaired("got_newpkg"),
comment => "Copy removable software from remote repository";
packages:
got_newpkg::
"$(pkg_name)"
package_policy => "delete",
package_method => msi_implicit( "$(local_software_dir)" ),
classes => if_else("remove_success", "remove_fail" ),
comment => "Remove software, if present";
reports::
remove_fail::
"Failed to remove one or more packages";
}
Web server modules
The problem of editing the correct modules into the list of standard modules for the Apache web server. This example is based on the standard configuration deployment of SuSE Linux. Simply provide the list of modules you want and another list that you don't want.
body common control
{
inputs => { "$(sys.libdir)/stdlib.cf" };
bundlesequence => {
apache
};
}
bundle agent apache
{
files:
SuSE::
"/etc/sysconfig/apache2"
edit_line => fixapache;
}
bundle edit_line fixapache
{
vars:
"add_modules" slist => {
"dav",
"dav_fs",
"ssl",
"php5",
"dav_svn",
"xyz",
"superduper"
};
"del_modules" slist => {
"php3",
"jk",
"userdir",
"imagemap",
"alias"
};
insert_lines:
"APACHE_CONF_INCLUDE_FILES=\"/site/masterfiles/local-http.conf\"";
field_edits:
#####################################################################
# APACHE_MODULES="authz_host actions alias ..."
#####################################################################
# Values have the form NAME = "quoted space separated list"
"APACHE_MODULES=.*"
# Insert module "columns" between the quoted RHS
# using space separators
edit_field => quoted_var($(add_modules), "append");
"APACHE_MODULES=.*"
# Delete module "columns" between the quoted RHS
# using space separators
edit_field => quoted_var($(del_modules), "delete");
# if this line already exists, edit it
}
Commands, Scripts, and Execution Examples
- Command or script execution
- Change directory for command
- Commands example
- Execresult example
- Methods
- Method validation
- Trigger classes
Command or script execution
Execute a command, for instance to start a MySQL service. Note that simple shell commands like rm or mkdir cannot be managed by CFEngine, so none of the protections that CFEngine offers can be applied to the process. Moreover, this starts a new process, adding to the burden on the system.
body common control
{
bundlesequence => { "my_commands" };
inputs => { "$(sys.libdir)/stdlib.cf" };
}
bundle agent my_commands
{
commands:
Sunday.Hr04.Min05_10.myhost::
"/usr/bin/update_db";
any::
"/etc/mysql/start"
contain => setuid("mysql");
}
Change directory for command
body common control
{
bundlesequence => { "example" };
}
body contain cd(dir)
{
chdir => "${dir}";
useshell => "true";
}
bundle agent example
{
commands:
"/bin/pwd"
contain => cd("/tmp");
}
Commands example
body common control
{
bundlesequence => { "my_commands" };
inputs => { "$(sys.libdir)/stdlib.cf" };
}
bundle agent my_commands
{
commands:
Sunday.Hr04.Min05_10.myhost::
"/usr/bin/update_db";
any::
"/etc/mysql/start"
contain => setuid("mysql");
}
Execresult example
body common control
{
bundlesequence => { "example" };
}
bundle agent example
{
vars:
"my_result" string => execresult("/bin/ls /tmp","noshell");
reports:
"Variable is $(my_result)";
}
Methods
body common control
{
bundlesequence => { "testbundle" };
version => "1.2.3";
}
bundle agent testbundle
{
vars:
"userlist" slist => { "mark", "jeang", "jonhenrik", "thomas", "eben" };
methods:
"any" usebundle => subtest("$(userlist)");
}
bundle agent subtest(user)
{
commands:
"/bin/echo Fix $(user)";
reports:
"Finished doing stuff for $(user)";
}
Method validation
body common control
{
bundlesequence => { "testbundle" };
version => "1.2.3";
}
body agent control
{
abortbundleclasses => { "invalid" };
}
bundle agent testbundle
{
vars:
"userlist" slist => { "xyz", "mark", "jeang", "jonhenrik", "thomas", "eben" };
methods:
"any" usebundle => subtest("$(userlist)");
}
bundle agent subtest(user)
{
classes:
"invalid" not => regcmp("[a-z][a-z][a-z][a-z]","$(user)");
reports:
!invalid::
"User name $(user) is valid at 4 letters";
invalid::
"User name $(user) is invalid";
}
Trigger classes
body common control
{
any::
bundlesequence => { "insert" };
}
bundle agent insert
{
vars:
"v" string => "
One potato
Two potato
Three potahto
Four
";
files:
"/tmp/test_insert"
edit_line => Insert("$(insert.v)"),
edit_defaults => empty,
classes => trigger("edited");
commands:
edited::
"/bin/echo make bananas";
reports:
edited::
"The potatoes are bananas";
}
bundle edit_line Insert(name)
{
insert_lines:
"Begin$(const.n) $(name)$(const.n)End";
}
body edit_defaults empty
{
empty_file_before_editing => "true";
}
body classes trigger(x)
{
promise_repaired => { $(x) };
}
File and Directory Examples
- Create files and directories
- Copy single files
- Copy directory trees
- Disabling and rotating files
- Add lines to a file
- Check file or directory permissions
- Commenting lines in a file
- Copy files
- Copy and flatten directory
- Copy then edit a file convergently
- Deleting lines from a file
- Deleting lines exception
- Delete files recursively
- Editing files
- Editing tabular files
- Inserting lines in a file
- Back references in filenames
- Add variable definitions to a file
- Linking files
- Listing files-pattern in a directory
- Locate and transform files
- BSD flags
- Search and replace text
- Selecting a region in a file
- Warn if matching line in file
Create files and directories
Create files and directories and set permissions.
body common control
{
bundlesequence => { "testbundle" };
}
bundle agent testbundle
{
files:
"/home/mark/tmp/test_plain"
perms => system,
create => "true";
"/home/mark/tmp/test_dir/."
perms => system,
create => "true";
}
body perms system
{
mode => "0640";
}
Copy single files
Copy single files, locally (local_cp) or from a remote site (secure_cp). The Community Open Promise-Body Library (COPBL; cfengine_stdlib.cf) should be included in the /var/cfengine/inputs/ directory and input as below.
body common control
{
bundlesequence => { "mycopy" };
inputs => { "$(sys.libdir)/stdlib.cf" };
}
bundle agent mycopy
{
files:
"/home/mark/tmp/test_plain"
copy_from => local_cp("$(sys.workdir)/bin/file");
"/home/mark/tmp/test_remote_plain"
copy_from => secure_cp("$(sys.workdir)/bin/file","serverhost");
}
Copy directory trees
Copy directory trees, locally (local_cp) or from a remote site (secure_cp). (depth_search => recurse("")) defines the number of sublevels to include, ("inf") gets entire tree.
body common control
{
bundlesequence => { "my_recursive_copy" };
inputs => { "$(sys.libdir)/stdlib.cf" };
}
bundle agent my_recursive_copy
{
files:
"/home/mark/tmp/test_dir"
copy_from => local_cp("$(sys.workdir)/bin/."),
depth_search => recurse("inf");
"/home/mark/tmp/test_dir"
copy_from => secure_cp("$(sys.workdir)/bin","serverhost"),
depth_search => recurse("inf");
}
Disabling and rotating files
Use the following simple steps to disable and rotate files. See the Community Open Promise-Body Library if you wish more details on what disable and rotate does.
body common control
{
bundlesequence => { "my_disable" };
inputs => { "$(sys.libdir)/stdlib.cf" };
}
bundle agent my_disable
{
files:
"/home/mark/tmp/test_create"
rename => disable;
"/home/mark/tmp/rotate_my_log"
rename => rotate("4");
}
Add lines to a file
There are numerous approaches to adding lines to a file. Often the order of a configuration file is unimportant, we just need to ensure settings within it. A simple way of adding lines is show below.
body common control
{
any::
bundlesequence => { "insert" };
}
bundle agent insert
{
vars:
"lines" string =>
"
One potato
Two potato
Three potatoe
Four
";
files:
"/tmp/test_insert"
create => "true",
edit_line => append_if_no_line("$(insert.lines)");
}
Also you could write this using a list variable:
body common control
{
any::
bundlesequence => { "insert" };
}
bundle agent insert
{
vars:
"lines" slist => { "One potato", "Two potato",
"Three potatoe", "Four" };
files:
"/tmp/test_insert"
create => "true",
edit_line => append_if_no_line("@(insert.lines)");
}
Check file or directory permissions
bundle agent check_perms
{
vars:
"ns_files" slist => {
"/local/iu/logs/admin",
"/local/iu/logs/security",
"/local/iu/logs/updates",
"/local/iu/logs/xfer"
};
files:
NameServers::
"/local/dns/pz"
perms => mo("644","dns"),
depth_search => recurse("1"),
file_select => exclude("secret_file");
"/local/iu/dns/pz/FixSerial"
perms => m("755"),
file_select => plain;
"$(ns_files)"
perms => mo("644","dns"),
file_select => plain;
"$(ftp)/pub"
perms => mog("644","root","other");
"$(ftp)/pub"
perms => m("644"),
depth_search => recurse("inf");
"$(ftp)/etc" perms => mog("111","root","other");
"$(ftp)/usr/bin/ls" perms => mog("111","root","other");
"$(ftp)/dev" perms => mog("555","root","other");
"$(ftp)/usr" perms => mog("555","root","other");
}
Commenting lines in a file
body common control
{
version => "1.2.3";
inputs => { "$(sys.libdir)/stdlib.cf" };
bundlesequence => { "testbundle" };
}
bundle agent testbundle
{
files:
"/home/mark/tmp/cf3_test"
create => "true",
edit_line => myedit("second");
}
bundle edit_line myedit(parameter)
{
vars:
"edit_variable" string => "private edit variable is $(parameter)";
replace_patterns:
# replace shell comments with C comments
"#(.*)"
replace_with => C_comment,
select_region => MySection("New section");
}
body replace_with C_comment
{
replace_value => "/* $(match.1) */"; # backreference 0
occurrences => "all"; # first, last all
}
body select_region MySection(x)
{
select_start => "\[$(x)\]";
select_end => "\[.*\]";
}
body common control
{
version => "1.2.3";
bundlesequence => { "testbundle" };
}
bundle agent testbundle
{
files:
"/home/mark/tmp/comment_test"
create => "true",
edit_line => comment_lines_matching;
}
bundle edit_line comment_lines_matching
{
vars:
"regexes" slist => { "one.*", "two.*", "four.*" };
replace_patterns:
"^($(regexes))$"
replace_with => comment("# ");
}
body replace_with comment(c)
{
replace_value => "$(c) $(match.1)";
occurrences => "all";
}
body common control
{
version => "1.2.3";
bundlesequence => { "testbundle" };
}
bundle agent testbundle
{
files:
"/home/mark/tmp/comment_test"
create => "true",
edit_line => uncomment_lines_matching("\s*mark.*","#");
}
bundle edit_line uncomment_lines_matching(regex,comment)
{
replace_patterns:
"#($(regex))$" replace_with => uncomment;
}
body replace_with uncomment
{
replace_value => "$(match.1)";
occurrences => "all";
}
Copy files
files:
"/var/cfengine/inputs"
handle => "update_policy",
perms => m("600"),
copy_from => u_scp("$(master_location)",@(policy_server)),
depth_search => recurse("inf"),
file_select => input_files,
action => immediate;
"/var/cfengine/bin"
perms => m("700"),
copy_from => u_scp("/usr/local/sbin","localhost"),
depth_search => recurse("inf"),
file_select => cf3_files,
action => immediate,
classes => on_change("reload");
Copy and flatten directory
body common control
{
bundlesequence => { "testbundle" };
version => "1.2.3";
}
bundle agent testbundle
{
files:
"/home/mark/tmp/testflatcopy"
comment => "test copy promise",
copy_from => mycopy("/home/mark/LapTop/words","127.0.0.1"),
perms => system,
depth_search => recurse("inf"),
classes => satisfied("copy_ok");
"/home/mark/tmp/testcopy/single_file"
comment => "test copy promise",
copy_from => mycopy("/home/mark/LapTop/Cfengine3/trunk/README","127.0.0.1"),
perms => system;
reports:
copy_ok::
"Files were copied..";
}
body perms system
{
mode => "0644";
}
body depth_search recurse(d)
{
depth => "$(d)";
}
body copy_from mycopy(from,server)
{
source => "$(from)";
servers => { "$(server)" };
compare => "digest";
verify => "true";
copy_backup => "true"; #/false/timestamp
purge => "false";
type_check => "true";
force_ipv4 => "true";
trustkey => "true";
collapse_destination_dir => "true";
}
body classes satisfied(x)
{
promise_repaired => { "$(x)" };
persist_time => "0";
}
body server control
{
allowconnects => { "127.0.0.1" , "::1" };
allowallconnects => { "127.0.0.1" , "::1" };
trustkeysfrom => { "127.0.0.1" , "::1" };
}
bundle server my_access_rules()
{
access:
"/home/mark/LapTop"
admit => { "127.0.0.1" };
}
Copy then edit a file convergently
To convergently chain a copy followed by edit, you need a staging file. First you copy to the staging file. Then you edit the final file and insert the staging file into it as part of the editing. This is convergent with respect to both stages of the process.
bundle agent master
{
files:
"$(final_destination)"
create => "true",
edit_line => fix_file("$(staging_file)"),
edit_defaults => empty,
perms => mo("644","root"),
action => ifelapsed("60");
}
bundle edit_line fix_file(f)
{
insert_lines:
"$(f)"
# insert this into an empty file to reconstruct
insert_type => "file";
replace_patterns:
"searchstring"
replace_with => With("replacestring");
}
Deleting lines from a file
body common control
{
bundlesequence => { "test" };
}
bundle agent test
{
files:
"/tmp/resolv.conf" # test on "/tmp/resolv.conf" #
create => "true",
edit_line => resolver,
edit_defaults => def;
}
bundle edit_line resolver
{
vars:
"search" slist => { "search iu.hio.no cfengine.com", "nameserver 128.39.89.10" };
delete_lines:
"search.*";
insert_lines:
"$(search)" location => end;
}
body edit_defaults def
{
empty_file_before_editing => "false";
edit_backup => "false";
max_file_size => "100000";
}
body location start
{
# If not line to match, applies to whole text body
before_after => "before";
}
body location end
{
# If not line to match, applies to whole text body
before_after => "after";
}
Deleting lines exception
body common control
{
bundlesequence => { "testbundle" };
}
bundle agent testbundle
{
files:
"/tmp/passwd_excerpt"
create => "true",
edit_line => MarkNRoot;
}
bundle edit_line MarkNRoot
{
delete_lines:
"mark.*|root.*" not_matching => "true";
}
Delete files recursively
The rm_rf and rm_rf_depth bundles in the standard library make it easy to prune directory trees.
Editing files
This is a huge topic. See also See Add lines to a file, See Editing tabular files, etc. Editing a file can be complex or simple, depending on needs.
Here is an example of how to comment out lines matching a number of patterns:
body common control
{
version => "1.2.3";
bundlesequence => { "testbundle" };
inputs => { "$(sys.libdir)/stdlib.cf" };
}
bundle agent testbundle
{
vars:
"patterns" slist => { "finger.*", "echo.*", "exec.*", "rstat.*",
"uucp.*", "talk.*" };
files:
"/etc/inetd.conf"
edit_line => comment_lines_matching("@(testbundle.patterns)","#");
}
Editing tabular files
body common control
{
version => "1.2.3";
bundlesequence => { "testbundle" };
}
bundle agent testbundle
{
vars:
"userset" slist => { "one-x", "two-x", "three-x" };
files:
# Make a copy of the password file
"/home/mark/tmp/passwd"
create => "true",
edit_line => SetUserParam("mark","6","/set/this/shell");
"/home/mark/tmp/group"
create => "true",
edit_line => AppendUserParam("root","4","@(userset)");
commands:
"/bin/echo" args => $(userset);
}
bundle edit_line SetUserParam(user,field,val)
{
field_edits:
"$(user):.*"
# Set field of the file to parameter
edit_field => col(":","$(field)","$(val)","set");
}
bundle edit_line AppendUserParam(user,field,allusers)
{
vars:
"val" slist => { @(allusers) };
field_edits:
"$(user):.*"
# Set field of the file to parameter
edit_field => col(":","$(field)","$(val)","alphanum");
}
body edit_field col(split,col,newval,method)
{
field_separator => $(split);
select_field => $(col);
value_separator => ",";
field_value => $(newval);
field_operation => $(method);
extend_fields => "true";
}
Inserting lines in a file
body common control
{
any::
bundlesequence => { "insert" };
}
bundle agent insert
{
vars:
"v" string => " One potato";
files:
"/tmp/test_insert"
create => "true",
edit_line => Insert("$(insert.v)");
}
bundle edit_line Insert(name)
{
insert_lines:
" $(name)"
whitespace_policy => { "ignore_leading", "ignore_embedded" };
}
body edit_defaults empty
{
empty_file_before_editing => "true";
}
body common control
{
any::
bundlesequence => { "insert" };
}
bundle agent insert
{
vars:
"v" string => "
One potato
Two potato
Three potatoe
Four
";
files:
"/tmp/test_insert"
create => "true",
edit_line => Insert("$(insert.v)"),
edit_defaults => empty;
}
bundle edit_line Insert(name)
{
insert_lines:
"Begin$(const.n)$(name)$(const.n)End";
}
body edit_defaults empty
{
empty_file_before_editing => "false";
}
body common control
{
any::
bundlesequence => { "insert" };
}
bundle agent insert
{
vars:
"v" slist => {
"One potato",
"Two potato",
"Three potatoe",
"Four"
};
files:
"/tmp/test_insert"
create => "true",
edit_line => Insert("@(insert.v)");
# edit_defaults => empty;
}
bundle edit_line Insert(name)
{
insert_lines:
"$(name)";
}
body edit_defaults empty
{
empty_file_before_editing => "true";
}
Back references in filenames
body common control
{
version => "1.2.3";
bundlesequence => { "testbundle" };
}
bundle agent testbundle
{
files:
# The back reference in a path only applies to the last link
# of the pathname, so the (tmp) gets ignored
"/tmp/(cf3)_(.*)"
edit_line => myedit("second $(match.2)");
# but ...
# "/tmp/cf3_test"
# create => "true",
# edit_line => myedit("second $(match.1)");
}
bundle edit_line myedit(parameter)
{
vars:
"edit_variable" string => "private edit variable is $(parameter)";
insert_lines:
"$(edit_variable)";
}
Add variable definitions to a file
body common control
{
bundlesequence => { "setvars" };
inputs => { "cf_std_library.cf" };
}
bundle agent setvars
{
vars:
# want to set these values by the names of their array keys
"rhs[lhs1]" string => " Mary had a little pig";
"rhs[lhs2]" string => "Whose Fleece was white as snow";
"rhs[lhs3]" string => "And everywhere that Mary went";
# oops, now change pig -> lamb
files:
"/tmp/system"
create => "true",
edit_line => set_variable_values("setvars.rhs");
}
Results in:
- lhs1= Mary had a little pig
- lhs2=Whose Fleece was white as snow
- lhs3=And everywhere that Mary went
An example of this would be to add variables to /etc/sysctl.conf on Linux:
body common control
{
bundlesequence => { "setvars" };
inputs => { "cf_std_library.cf" };
}
bundle agent setvars
{
vars:
# want to set these values by the names of their array keys
"rhs[net/ipv4/tcp_syncookies]" string => "1";
"rhs[net/ipv4/icmp_echo_ignore_broadcasts]" string => "1";
"rhs[net/ipv4/ip_forward]" string => "1";
# oops, now change pig -> lamb
files:
"/etc/sysctl"
create => "true",
edit_line => set_variable_values("setvars.rhs");
}
Linking files
body common control
{
version => "1.2.3";
bundlesequence => { "testbundle" };
}
bundle agent testbundle
{
files:
# Make a copy of the password file
"/home/mark/tmp/passwd"
link_from => linkdetails("/etc/passwd"),
move_obstructions => "true";
"/home/mark/tmp/linktest"
link_from => linkchildren("/usr/local/sbin");
#child links
}
body link_from linkdetails(tofile)
{
source => "$(tofile)";
link_type => "symlink";
when_no_source => "force"; # kill
}
body link_from linkchildren(tofile)
{
source => "$(tofile)";
link_type => "symlink";
when_no_source => "force"; # kill
link_children => "true";
when_linking_children => "if_no_such_file"; # "override_file";
}
body common control
{
any::
bundlesequence => {
"testbundle"
};
}
bundle agent testbundle
{
files:
"/home/mark/tmp/test_to" -> "someone"
depth_search => recurse("inf"),
perms => modestuff,
action => tell_me;
}
body depth_search recurse(d)
{
rmdeadlinks => "true";
depth => "$(d)";
}
body perms modestuff
{
mode => "o-w";
}
body action tell_me
{
report_level => "inform";
}
Listing files-pattern in a directory
body common control
{
bundlesequence => { "example" };
}
bundle agent example
{
vars:
"ls" slist => lsdir("/etc","p.*","true");
reports:
"ls: $(ls)";
}
Locate and transform files
body common control
{
any::
bundlesequence => {
"testbundle"
};
version => "1.2.3";
}
bundle agent testbundle
{
files:
"/home/mark/tmp/testcopy"
file_select => pdf_files,
transformer => "/usr/bin/gzip $(this.promiser)",
depth_search => recurse("inf");
}
body file_select pdf_files
{
leaf_name => { ".*.pdf" , ".*.fdf" };
file_result => "leaf_name";
}
body depth_search recurse(d)
{
depth => "$(d)";
}
BSD flags
body common control
{
bundlesequence => { "test" };
}
bundle agent test
{
files:
freebsd::
"/tmp/newfile"
create => "true",
perms => setbsd;
}
body perms setbsd
{
bsdflags => { "+uappnd","+uchg", "+uunlnk", "-nodump" };
}
Search and replace text
body common control
{
version => "1.2.3";
bundlesequence => { "testbundle" };
}
bundle agent testbundle
{
files:
"/tmp/replacestring"
create => "true",
edit_line => myedit("second");
}
bundle edit_line myedit(parameter)
{
vars:
"edit_variable" string => "private edit variable is $(parameter)";
replace_patterns:
# replace shell comments with C comments
"puppet"
replace_with => With("cfengine 3");
}
body replace_with With(x)
{
replace_value => $(x);
occurrences => "first";
}
body select_region MySection(x)
{
select_start => "\[$(x)\]";
select_end => "\[.*\]";
}
Selecting a region in a file
body common control
{
version => "1.2.3";
bundlesequence => { "testbundle" };
}
bundle agent testbundle
{
files:
"/tmp/testfile"
create => "true",
edit_line => myedit("second");
}
bundle edit_line myedit(parameter)
{
vars:
"edit_variable" string => "private edit variable is $(parameter)";
replace_patterns:
# comment out lines after start
"([^#].*)"
replace_with => comment,
select_region => ToEnd("Start.*");
}
body replace_with comment
{
replace_value => "# $(match.1)"; # backreference 0
occurrences => "all"; # first, last all
}
body select_region ToEnd(x)
{
select_start => $(x);
}
Warn if matching line in file
body common control
{
bundlesequence => { "testbundle" };
}
bundle agent testbundle
{
files:
"/var/cfengine/inputs/.*"
edit_line => DeleteLinesMatching(".*cfenvd.*"),
action => WarnOnly;
}
bundle edit_line DeleteLinesMatching(regex)
{
delete_lines:
"$(regex)" action => WarnOnly;
}
body action WarnOnly
{
action_policy => "warn";
}
Interacting with Directory Services
Active directory example
bundle agent active_directory
{
vars:
# NOTE: Edit this to your domain, e.g. "corp", may also need more DC's after it
"domain_name" string => "cftesting";
"user_name" string => "Guest";
# NOTE: We can also extract data from remote Domain Controllers
dummy.DomainController::
"domain_controller" string => "localhost";
"userlist" slist => ldaplist(
"ldap://$(domain_controller)",
"CN=Users,DC=$(domain_name),DC=com",
"(objectClass=user)",
"sAMAccountName",
"subtree",
"none");
classes:
dummy.DomainController::
"gotuser" expression => ldaparray(
"userinfo",
"ldap://$(domain_controller)",
"CN=$(user_name),CN=Users,DC=$(domain_name),DC=com",
"(name=*)",
"subtree",
"none");
reports:
dummy.DomainController::
'Username is "$(userlist)"';
dummy.gotuser::
"Got user data; $(userinfo[name]) has logged on $(userinfo[logonCount]) times";
}
Active list users directory example
bundle agent ldap
{
vars:
"userlist" slist => ldaplist(
"ldap://cf-win2003",
"CN=Users,DC=domain,DC=cf-win2003",
"(objectClass=user)",
"sAMAccountName",
"subtree",
"none");
reports:
'Username: "$(userlist)"';
}
Active directory show users example
bundle agent ldap
{
classes:
"gotdata" expression => ldaparray(
"myarray",
"ldap://cf-win2003",
"CN=Test Pilot,CN=Users,DC=domain,DC=cf-win2003",
"(name=*)",
"subtree",
"none");
reports:
gotdata::
"Got user data";
!gotdata::
"Did not get user data";
}
LDAP interactions
body common control
{
bundlesequence => { "ldap" , "followup"};
}
bundle agent ldap
{
vars:
# Get the first matching value for "uid"
"value" string => ldapvalue("ldap://eternity.iu.hio.no","dc=cfengine,dc=com","(sn=User)","uid","subtree","none");
# Get all matching values for "uid" - should be a single record match
"list" slist => ldaplist("ldap://eternity.iu.hio.no","dc=cfengine,dc=com","(sn=User)","uid","subtree","none");
classes:
"gotdata" expression => ldaparray("myarray","ldap://eternity.iu.hio.no","dc=cfengine,dc=com","(uid=mark)","subtree","none");
"found" expression => regldap("ldap://eternity.iu.hio.no","dc=cfengine,dc=com","(sn=User)","uid","subtree","jon.*","none");
reports:
linux::
"LDAP VALUE $(value) found";
"LDAP LIST VALUE $(list)";
gotdata::
"Found specific entry data ...$(ldap.myarray[uid]),$(ldap.myarray[gecos]), etc";
found::
"Matched regex";
}
bundle agent followup
{
reports:
linux::
"Different bundle ...$(ldap.myarray[uid]),$(ldap.myarray[gecos]),...";
}
File Template Examples
Templating
With CFEngine you have a choice between editing `deltas' into files or distributing more-or-less finished templates. Which method you should choose depends should be made by whatever is easiest.
If you are managing only part of the file, and something else (e.g. a package manager) is managing most of it, then it makes sense to use CFEngine file editing.
If you are managing everything in the file, then it makes sense to make the edits by hand and install them using CFEngine. You can use variables within source text files and let CFEngine expand them locally in situ, so that you can make generic templates that apply netwide.
Example template:
MYVARIABLE = something or other
HOSTNAME = $(sys.host) # CFEngine fills this in
To copy and expand this template, you can use a pattern like this:
bundle agent get_template(final_destination,mode)
{
vars:
# This needs to ne preconfigured to your site
"masterfiles" string => "/home/mark/tmp";
"this_template" string => lastnode("$(final_destination)","/");
files:
"$(final_destination).staging"
comment => "Get template and expand variables for this host",
perms => mo("400","root"),
copy_from => remote_cp("$(masterfiles)/templates/$(this_template)","$(policy_server)"),
action => if_elapsed("60");
"$(final_destination)"
comment => "Expand the template",
create => "true",
edit_line => expand_template("$(final_destination).staging"),
edit_defaults => empty,
perms => mo("$(mode)","root"),
action => if_elapsed("60");
}
The the following driving code (based on `copy then edit') can be placed in a library, after configuring to your environmental locations:
bundle agent get_template(final_destination,mode)
{
vars:
# This needs to ne preconfigured to your site
"masterfiles" string => "/home/mark/tmp";
"this_template" string => lastnode("$(final_destination)","/");
files:
"$(final_destination).staging"
comment => "Get template and expand variables for this host",
perms => mo("400","root"),
copy_from => remote_cp("$(masterfiles)/templates/$(this_template)","$(policy_server)"),
action => if_elapsed("60");
"$(final_destination)"
comment => "Expand the template",
create => "true",
edit_line => expand_template("$(final_destination).staging"),
edit_defaults => empty,
perms => mo("$(mode)","root"),
action => if_elapsed("60");
}
Database Examples
Database creation
body common control
{
bundlesequence => { "dummy" };
}
body knowledge control
{
#sql_database => "postgres";
sql_owner => "postgres";
sql_passwd => ""; # No passwd
sql_type => "postgres";
}
bundle knowledge dummy
{
topics:
}
body common control
{
bundlesequence => { "databases" };
}
bundle agent databases
{
#commands:
# "/usr/bin/createdb cf_topic_maps",
# contain => as_user("mysql");
databases:
"knowledge_bank/topics"
database_operation => "create",
database_type => "sql",
database_columns => {
"topic_name,varchar,256",
"topic_comment,varchar,1024",
"topic_id,varchar,256",
"topic_type,varchar,256",
"topic_extra,varchar,26"
},
database_server => myserver;
}
body database_server myserver
{
none::
db_server_owner => "postgres";
db_server_password => "";
db_server_host => "localhost";
db_server_type => "postgres";
db_server_connection_db => "postgres";
any::
db_server_owner => "root";
db_server_password => "";
db_server_host => "localhost";
db_server_type => "mysql";
db_server_connection_db => "mysql";
}
body contain as_user(x)
{
exec_owner => "$(x)";
}
Network Examples
- Find MAC address
- Client-server example
- Read from a TCP socket
- Set up a PXE boot server
- Resolver management
- Mount NFS filesystem
- Unmount NFS filesystem
- Find the MAC address
- Mount NFS filesystem
Find MAC address
Finding the ethernet address can be hard, but on Linux it is straightforward.
bundle agent test
{
vars:
linux::
"interface" string => execresult("/sbin/ifconfig eth0","noshell");
solaris::
"interface" string => execresult("/usr/sbin/ifconfig bge0","noshell");
freebsd::
"interface" string => execresult("/sbin/ifconfig le0","noshell");
darwin::
"interface" string => execresult("/sbin/ifconfig en0","noshell");
classes:
linux::
"ok" expression => regextract(
".*HWaddr ([^\s]+).*(\n.*)*",
"$(interface)",
"mac"
);
solaris::
"ok" expression => regextract(
".*ether ([^\s]+).*(\n.*)*",
"$(interface)",
"mac"
);
freebsd::
"ok" expression => regextract(
".*ether ([^\s]+).*(\n.*)*",
"$(interface)",
"mac"
);
darwin::
"ok" expression => regextract(
"(?s).*ether ([^\s]+).*(\n.*)*",
"$(interface)",
"mac"
);
reports:
ok::
"MAC address is $(mac[1])";
}
Client-server example
body common control
{
bundlesequence => { "testbundle" };
version => "1.2.3";
#fips_mode => "true";
}
bundle agent testbundle
{
files:
"/home/mark/tmp/testcopy"
comment => "test copy promise",
copy_from => mycopy("/home/mark/LapTop/words","127.0.0.1"),
perms => system,
depth_search => recurse("inf"),
classes => satisfied("copy_ok");
"/home/mark/tmp/testcopy/single_file"
comment => "test copy promise",
copy_from => mycopy("/home/mark/LapTop/Cfengine3/trunk/README","127.0.0.1"),
perms => system;
reports:
copy_ok::
"Files were copied..";
}
body perms system
{
mode => "0644";
}
body depth_search recurse(d)
{
depth => "$(d)";
}
body copy_from mycopy(from,server)
{
source => "$(from)";
servers => { "$(server)" };
compare => "digest";
encrypt => "true";
verify => "true";
copy_backup => "true"; #/false/timestamp
purge => "false";
type_check => "true";
force_ipv4 => "true";
trustkey => "true";
}
body classes satisfied(x)
{
promise_repaired => { "$(x)" };
persist_time => "0";
}
body server control
{
allowconnects => { "127.0.0.1" , "::1" };
allowallconnects => { "127.0.0.1" , "::1" };
trustkeysfrom => { "127.0.0.1" , "::1" };
# allowusers
}
bundle server my_access_rules()
{
access:
"/home/mark/LapTop"
admit => { "127.0.0.1" };
}
Read from a TCP socket
body common control
{
bundlesequence => { "example" };
}
bundle agent example
{
vars:
"my80" string => readtcp("research.iu.hio.no","80","GET /index.php HTTP/1.1$(const.r)$(const.n)Host: research.iu.hio.no$(const.r)$(const.n)$(const.r)$(const.n)",20);
classes:
"server_ok" expression => regcmp(".*200 OK.*\n.*","$(my80)");
reports:
server_ok::
"Server is alive";
!server_ok::
"Server is not responding - got $(my80)";
}
Set up a PXE boot server
Use CFEngine to set up a PXE boot server.
body common control
{
bundlesequence => { "pxe" };
inputs => { "$(sys.libdir)/stdlib.cf" };
}
bundle agent pxe
{
vars:
"software" slist => {
"atftp",
"dhcp-server",
"syslinux",
"apache2"
};
"dirs" slist => {
"/tftpboot",
"/tftpboot/CFEngine/rpm",
"/tftpboot/CFEngine/inputs",
"/tftpboot/pxelinux.cfg",
"/tftpboot/kickstart",
"/srv/www/repos"
};
"tmp_location" string => "/tftpboot/CFEngine/inputs";
# Distros that we can install
"rh_distros" slist => { "4.7", "5.2" };
"centos_distros" slist => { "5.2" };
# File contents of atftp configuration
"atftpd_conf" string =>
"
ATFTPD_OPTIONS=\"--daemon \"
ATFTPD_USE_INETD=\"no\"
ATFTPD_DIRECTORY=\"/tftpboot\"
ATFTPD_BIND_ADDRESSES=\"\"
";
# File contents of DHCP configuration
"dhcpd" string =>
"
DHCPD_INTERFACE=\"eth0\"
DHCPD_RUN_CHROOTED=\"yes\"
DHCPD_CONF_INCLUDE_FILES=\"\"
DHCPD_RUN_AS=\"dhcpd\"
DHCPD_OTHER_ARGS=\"\"
DHCPD_BINARY=\"\"
";
"dhcpd_conf" string =>
"
allow booting;
allow bootp;
ddns-update-style none; ddns-updates off;
subnet 192.168.0.0 netmask 255.255.255.0 {
range 192.168.0.20 192.168.0.254;
default-lease-time 3600;
max-lease-time 4800;
option routers 192.168.0.1;
option domain-name \"test.CFEngine.com\";
option domain-name-servers 192.168.0.1;
next-server 192.168.0.1;
filename \"pxelinux.0\";
}
group {
host node1 {
# Dummy machine
hardware ethernet 00:0F:1F:94:FE:07;
fixed-address 192.168.0.11;
option host-name \"node1\";
}
host node2 {
# Dell Inspiron 1150
hardware ethernet 00:0F:1F:0E:70:E7;
fixed-address 192.168.0.12;
option host-name \"node2\";
}
}
";
# File contains of Apache2 HTTP configuration
"httpd_conf" string =>
"
<Directory /srv/www/repos>
Options Indexes
AllowOverride None
</Directory>
Alias /repos /srv/www/repos
<Directory /tftpboot/distro/RHEL/5.2>
Options Indexes
AllowOverride None
</Directory>
Alias /distro/rhel/5.2 /tftpboot/distro/RHEL/5.2
<Directory /tftpboot/distro/RHEL/4.7>
Options Indexes
AllowOverride None
</Directory>
Alias /distro/rhel/4.7 /tftpboot/distro/RHEL/4.7
<Directory /tftpboot/distro/CentOS/5.2>
Options Indexes
AllowOverride None
</Directory>
Alias /distro/centos/5.2 /tftpboot/distro/CentOS/5.2
<Directory /tftpboot/kickstart>
Options Indexes
AllowOverride None
</Directory>
Alias /kickstart /tftpboot/kickstart
<Directory /tftpboot/CFEngine>
Options Indexes
AllowOverride None
</Directory>
Alias /CFEngine /tftpboot/CFEngine
";
# File contains of Kickstart for RHEL5 configuration
"kickstart_rhel5_conf" string =>
"
auth --useshadow --enablemd5
bootloader --location=mbr
clearpart --all --initlabel
graphical
firewall --disabled
firstboot --disable
key 77244a6377a8044a
keyboard no
lang en_US
logging --level=info
url --url=http://192.168.0.1/distro/rhel/5.2
network --bootproto=dhcp --device=eth0 --onboot=on
reboot
rootpw --iscrypted $1$eOnXdDPF$279sQ//zry6rnQktkATeM0
selinux --disabled
timezone --isUtc Europe/Oslo
install
part swap --bytes-per-inode=4096 --fstype=\"swap\" --recommended
part / --bytes-per-inode=4096 --fstype=\"ext3\" --grow --size=1
%packages
@core
@base
db4-devel
openssl-devel
gcc
flex
bison
libacl-devel
libselinux-devel
pcre-devel
device-mapper-multipath
-sysreport
%post
cd /root
rpm -i http://192.168.0.1/CFEngine/rpm/CFEngine-3.0.1b1-1.el5.i386.rpm
cd /etc/yum.repos.d
wget http://192.168.0.1/repos/RHEL5.Base.repo
rpm --import /etc/pki/rpm-gpg/*
yum clean all
yum update
mkdir -p /root/CFEngine_init
cd /root/CFEngine_init
wget -nd -r http://192.168.0.1/CFEngine/inputs/
/usr/local/sbin/cf-agent -B
/usr/local/sbin/cf-agent
";
# File contains of PXElinux boot menu
"pxelinux_boot_menu" string =>
"
boot options:
rhel5 - install 32 bit i386 RHEL 5.2 (MANUAL)
rhel5w - install 32 bit i386 RHEL 5.2 (AUTO)
rhel4 - install 32 bit i386 RHEL 4.7 AS (MANUAL)
centos5 - install 32 bit i386 CentOS 5.2 (Desktop) (MANUAL)
";
# File contains of PXElinux default configuration
"pxelinux_default" string =>
"
default rhel5
timeout 300
prompt 1
display pxelinux.cfg/boot.msg
F1 pxelinux.cfg/boot.msg
label rhel5
kernel vmlinuz-RHEL5U2
append initrd=initrd-RHEL5U2 load_ramdisk=1 ramdisk_size=16384 install=http://192.168.0.1/distro/rhel/5.2
label rhel5w
kernel vmlinuz-RHEL5U2
append initrd=initrd-RHEL5U2 load_ramdisk=1 ramdisk_size=16384 ks=http://192.168.0.1/kickstart/kickstart-RHEL5U2.cfg
label rhel4
kernel vmlinuz-RHEL4U7
append initrd=initrd-RHEL4U7 load_ramdisk=1 ramdisk_size=16384 install=http://192.168.0.1/distro/rhel/4.7
label centos5
kernel vmlinuz-CentOS5.2
append initrd=initrd-CentOS5.2 load_ramdisk=1 ramdisk_size=16384 install=http://192.168.0.1/distro/centos/5.2
";
# File contains of specified PXElinux default to be a RHEL5 webserver
"pxelinux_rhel5_webserver" string =>
"
default rhel5w
label rhel5w
kernel vmlinuz-RHEL5U2
append initrd=initrd-RHEL5U2 load_ramdisk=1 ramdisk_size=16384 ks=http://192.168.0.1/kickstart/kickstart-RHEL5U2.cfg
";
# File contains of a local repository for RHEL5
"rhel5_base_repo" string =>
"
[Server]
name=Server
baseurl=http://192.168.0.1/repos/rhel5/Server/
enable=1
[VT]
name=VT
baseurl=http://192.168.0.1/repos/rhel5/VT/
enable=1
[Cluster]
name=Cluster
baseurl=http://192.168.0.1/repos/rhel5/Cluster/
enable=1
[ClusterStorage]
name=Cluster Storage
baseurl=http://192.168.0.1/repos/rhel5/ClusterStorage/
enable=1
";
#####################################################
files:
packages_ok::
# Create files/dirs and edit the new files
"/tftpboot/distro/RHEL/$(rh_distros)/."
create => "true";
"/tftpboot/distro/CentOS/$(centos_distros)/."
create => "true";
"$(dirs)/."
create => "true";
"/tftpboot/pxelinux.cfg/boot.msg"
create => "true",
perms => mo("644","root"),
edit_line => append_if_no_line("$(pxelinux_boot_menu)"),
edit_defaults => empty;
"/tftpboot/pxelinux.cfg/default"
create => "true",
perms => mo("644","root"),
edit_line => append_if_no_line("$(pxelinux_default)"),
edit_defaults => empty;
"/tftpboot/pxelinux.cfg/default.RHEL5.webserver"
create => "true",
perms => mo("644","root"),
edit_line => append_if_no_line("$(pxelinux_rhel5_webserver)"),
edit_defaults => empty;
"/tftpboot/kickstart/kickstart-RHEL5U2.cfg"
create => "true",
perms => mo("644","root"),
edit_line => append_if_no_line("$(kickstart_rhel5_conf)"),
edit_defaults => empty;
"/srv/www/repos/RHEL5.Base.repo"
create => "true",
perms => mo("644","root"),
edit_line => append_if_no_line("$(rhel5_base_repo)"),
edit_defaults => empty;
# Copy files
"/tftpboot"
copy_from => local_cp("/usr/share/syslinux"),
depth_search => recurse("inf"),
file_select => pxelinux_files,
action => immediate;
"$(tmp_location)"
perms => m("644"),
copy_from => local_cp("/var/cfengine/inputs"),
depth_search => recurse("inf"),
file_select => input_files,
action => immediate;
# Edit atftp, dhcp and apache2 configurations
"/etc/sysconfig/atftpd"
edit_line => append_if_no_line("$(atftpd_conf)"),
edit_defaults => empty,
classes => satisfied("atftpd_ready");
"/etc/sysconfig/dhcpd"
edit_line => append_if_no_line("$(dhcpd)"),
edit_defaults => empty;
"/etc/dhcpd.conf"
edit_line => append_if_no_line("$(dhcpd_conf)"),
edit_defaults => empty,
classes => satisfied("dhcpd_ready");
"/etc/apache2/httpd.conf"
edit_line => append_if_no_line("$(httpd_conf)"),
edit_defaults => std_defs,
classes => satisfied("apache2_ok");
# Make a static link
"/tftpboot/pxelinux.cfg/C0A8000C"
link_from => mylink("/tftpboot/pxelinux.cfg/default.RHEL5.webserver");
# Hash comment some lines for apaches
apache2_ok::
"/etc/apache2/httpd.conf"
edit_line => comment_lines_matching_apache2("#"),
classes => satisfied("apache2_ready");
commands:
# Restart services
atftpd_ready::
"/etc/init.d/atftpd restart";
dhcpd_ready::
"/etc/init.d/dhcpd restart";
apache2_ready::
"/etc/init.d/apache2 restart";
#####################################################
packages:
ipv4_192_168_0_1::
# Only the PXE boot server
"$(software)"
package_policy => "add",
package_method => zypper,
classes => satisfied("packages_ok");
}
body file_select pxelinux_files
{
leaf_name => { "pxelinux.0" };
file_result => "leaf_name";
}
body copy_from mycopy_local(from,server)
{
source => "$(from)";
compare => "digest";
}
body link_from mylink(x)
{
source => "$(x)";
link_type => "symlink";
}
body classes satisfied(new_class)
{
promise_kept => { "$(new_class)"};
promise_repaired => { "$(new_class)"};
}
bundle edit_line comment_lines_matching_apache2(comment)
{
vars:
"regex" slist => { "\s.*Options\sNone", "\s.*AllowOverride\sNone", "\s.*Deny\sfrom\sall" };
replace_patterns:
"^($(regex))$"
replace_with => comment("$(comment)");
}
body file_select input_files
{
leaf_name => { ".*.cf",".*.dat",".*.txt" };
file_result => "leaf_name";
}
Resolver management
bundle common g # globals
{
vars:
"searchlist" slist => {
"search iu.hio.no",
"search cfengine.com"
};
"nameservers" slist => {
"128.39.89.10",
"128.39.74.16",
"192.168.1.103"
};
classes:
"am_name_server" expression => reglist("@(nameservers)","$(sys.ipv4[eth1])");
}
body common control
{
any::
bundlesequence => {
"g",
resolver(@(g.searchlist),@(g.nameservers))
};
domain => "iu.hio.no";
}
bundle agent resolver(s,n)
{
files:
# When passing parameters down, we have to refer to
# a source context
"$(sys.resolv)" # test on "/tmp/resolv.conf" #
create => "true",
edit_line => doresolv("@(this.s)","@(this.n)"),
edit_defaults => reconstruct;
# or edit_defaults => modify
}
bundle edit_line doresolv(s,n)
{
vars:
"line" slist => { @(s), @(n) };
insert_lines:
"$(line)";
}
body edit_defaults reconstruct
{
empty_file_before_editing => "true";
edit_backup => "false";
max_file_size => "100000";
}
body edit_defaults modify
{
empty_file_before_editing => "false";
edit_backup => "false";
max_file_size => "100000";
}
Mount NFS filesystem
body common control
{
bundlesequence => { "mounts" };
}
bundle agent mounts
{
storage:
"/mnt" mount => nfs("slogans.iu.hio.no","/home");
}
body mount nfs(server,source)
{
mount_type => "nfs";
mount_source => "$(source)";
mount_server => "$(server)";
#mount_options => { "rw" };
edit_fstab => "true";
unmount => "true";
}
Unmount NFS filesystem
body common control
{
bundlesequence => { "mounts" };
}
bundle agent mounts
{
storage:
# Assumes the filesystem has been exported
"/mnt" mount => nfs("server.example.org","/home");
}
body mount nfs(server,source)
{
mount_type => "nfs";
mount_source => "$(source)";
mount_server => "$(server)";
edit_fstab => "true";
unmount => "true";
}
System Security Examples
- Distribute root passwords
- Distribute ssh keys
- Distribute ssh keys
Distribute root passwords
body common control
{
version => "1.2.3";
inputs => { "$(sys.libdir)/stdlib.cf" };
bundlesequence => { "SetRootPassword" };
}
bundle common g
{
vars:
"secret_keys_dir" string => "/tmp";
}
bundle agent SetRootPassword
{
vars:
# Or get variables directly from server with Enterprise
"remote-passwd" string => remotescalar("rem_password","127.0.0.1","yes");
# Test this on a copy
files:
"/var/cfengine/ppkeys/rootpw.txt"
copy_from => secure_cp("$(sys.fqhost)-root.txt","master_host.example.org");
# or $(pw_class)-root.txt
"/tmp/shadow"
edit_line => SetRootPw;
}
bundle edit_line SetRootPw
{
vars:
# Assume this file contains a single string of the form root:passwdhash:
# with : delimiters to avoid end of line/file problems
"pw" int => readstringarray("rpw","$(sys.workdir)/ppkeys/rootpw.txt",
"#[^\n]*",":","1","200");
field_edits:
"root:.*"
# Set field of the file to parameter
edit_field => col(":","2","$(rpw[root][1])","set");
}
bundle server passwords
{
vars:
# Read a file of format
#
# classname: host1,host2,host4,IP-address,regex.*,etc
#
"pw_classes" int => readstringarray("acl","$(g.secret_keys_dir)/classes.txt",
"#[^\n]*",":","100","4000");
"each_pw_class" slist => getindices("acl");
access:
"/secret/keys/$(each_pw_class)-root.txt"
admit => splitstring("$(acl[$(each_pw_class)][1])" , ":" , "100"),
ifencrypted => "true";
}
Distribute ssh keys
bundle agent allow_ssh_rootlogin_from_authorized_keys(user,sourcehost)
{
vars:
"local_cache" string => "/var/cfengine/ssh_cache";
"authorized_source" string => "/master/CFEngine/ssh_keys";
files:
"$(local_cache)/$(user).pub"
comment => "Copy public keys from a an authorized cache into a cache on localhost",
perms => mo("600","root"),
copy_from => remote_cp("$(authorized_source)/$(user).pub","$(sourcehost)"),
action => if_elapsed("60");
"/root/.ssh/authorized_keys"
comment => "Edit the authorized keys into the user's personal keyring",
edit_line => insert_file_if_no_line_matching("$(user)","$(local_cache)/$(user).pub"),
action => if_elapsed("60");
}
bundle agent allow_ssh_login_from_authorized_keys(user,sourcehost)
{
vars:
"local_cache" string => "/var/cfengine/ssh_cache";
"authorized_source" string => "/master/CFEngine/ssh_keys";
files:
"$(local_cache)/$(user).pub"
comment => "Copy public keys from a an authorized cache into a cache on localhost",
perms => mo("600","root"),
copy_from => remote_cp("$(authorized_source)/$(user).pub","$(sourcehost)"),
action => if_elapsed("60");
"/home/$(user)/.ssh/authorized_keys"
comment => "Edit the authorized keys into the user's personal keyring",
edit_line => insert_file_if_no_line_matching("$(user)","$(local_cache)/$(user).pub"),
action => if_elapsed("60");
}
bundle edit_line insert_file_if_no_line_matching(user,file)
{
classes:
"have_user" expression => regline("$(user).*","$(this.promiser)");
insert_lines:
!have_user::
"$(file)"
insert_type => "file";
}
System Information Examples
- Change detection
- Hashing for change detection (tripwire)
- Check filesystem space
- Class match example
- Global classes
- Logging
- Check filesystem space
Change detection
body common control
{
bundlesequence => { "testbundle" };
inputs => { "cfengine_stdlib.cf" };
}
bundle agent testbundle
{
files:
"/usr"
changes => detect_all_change,
depth_search => recurse("inf"),
action => background;
}
Hashing for change detection (tripwire)
Change detection is a powerful and easy way to monitor your environment, increase awareness and harden your system against security breaches.
body common control
{
bundlesequence => { "testbundle" };
inputs => { "$(sys.libdir)/stdlib.cf" };
}
bundle agent testbundle
{
files:
"/home/mark/tmp/web" -> "me"
changes => detect_all_change,
depth_search => recurse("inf");
}
Check filesystem space
body common control
{
bundlesequence => { "example" };
}
bundle agent example
{
vars:
"free" int => diskfree("/tmp");
reports:
"Freedisk $(free)";
}
Class match example
body common control
{
bundlesequence => { "example" };
}
bundle agent example
{
classes:
"do_it" and => { classmatch(".*_3"), "linux" };
reports:
do_it::
"Host matches pattern";
}
Global classes
body common control
{
bundlesequence => { "g","tryclasses_1", "tryclasses_2" };
}
bundle common g
{
classes:
"one" expression => "any";
"client_network" expression => iprange("128.39.89.0/24");
}
bundle agent tryclasses_1
{
classes:
"two" expression => "any";
}
bundle agent tryclasses_2
{
classes:
"three" expression => "any";
reports:
one.three.!two::
"Success";
}
body common control
{
bundlesequence => { "g","tryclasses_1", "tryclasses_2" };
}
bundle common g
{
classes:
"one" expression => "any";
"client_network" expression => iprange("128.39.89.0/24");
}
bundle agent tryclasses_1
{
classes:
"two" expression => "any";
}
bundle agent tryclasses_2
{
classes:
"three" expression => "any";
reports:
one.three.!two::
"Success";
}
Logging
body common control
{
bundlesequence => { "test" };
}
bundle agent test
{
vars:
"software" slist => { "/root/xyz", "/tmp/xyz" };
files:
"$(software)"
create => "true",
action => logme("$(software)");
}
body action logme(x)
{
log_kept => "/tmp/private_keptlog.log";
log_failed => "/tmp/private_faillog.log";
log_repaired => "/tmp/private_replog.log";
log_string => "$(sys.date) $(x) promise status";
}
body common control
{
bundlesequence => { "one" };
}
bundle agent one
{
files:
"/tmp/xyz"
create => "true",
action => log;
}
body action log
{
log_level => "inform";
}
System Administration Examples
- Centralized Management
- Laptop support configuration
- Process management
- Kill process
- Restart process
- Mount a filesystem
- Manage a system process
- Set up HPC clusters
- Set up name resolution
- Set up sudo
- Environments (virtual)
- Environment variables
- Tidying garbage files
Centralized Management
These examples show a simple setup for starting with a central approach to management of servers. Centralization of management is a simple approach suitable for small environments with few requirements. It is useful for clusters where systems are all alike.
All hosts the same
Variation in hosts
Updating from a central hub
All hosts the same
This shows the simplest approach in which all hosts are the same. It is too simple for most environments, but it serves as a starting point. Compare it to the next section that includes variation.
body common control
{
bundlesequence => { "central" };
}
bundle agent central
{
vars:
"policy_server" string => "myhost.domain.tld";
"mypackages" slist => {
"nagios"
"gcc",
"apache2",
"php5"
};
files:
# Password management can be very simple if all hosts are identical
"/etc/passwd"
comment => "Distribute a password file",
perms => mog("644","root","root"),
copy_from => secure_cp("/home/mark/LapTop/words/RoadAhead","$(policy_server)");
packages:
"$(mypackages)"
package_policy => "add",
package_method => generic;
# Add more promises below ...
}
body server control
{
allowconnects => { "127.0.0.1" , "::1", "10.20.30.0/24" };
allowallconnects => { "127.0.0.1" , "::1", "10.20.30.0/24" };
trustkeysfrom => { "127.0.0.1" , "::1", "10.20.30.0/24" };
# allowusers
}
bundle server my_access_rules()
{
access:
# myhost.domain.tld makes this file available to 10.20.30*
myhost_domain_tld::
"/etc/passwd"
admit => { "127.0.0.1", "10.20.30.0/24" };
}
Variation in hosts
body common control
{
bundlesequence => { "central" };
}
bundle agent central
{
classes:
"mygroup_1" or => { "myhost", "host1", "host2", "host3" };
"mygroup_2" or => { "host4", "host5", "host6" };
vars:
"policy_server" string => "myhost.domain.tld";
mygroup_1::
"mypackages" slist => {
"nagios"
"gcc",
"apache2",
"php5"
};
mygroup_2::
"mypackages" slist => {
"apache"
"mysql",
"php5"
};
files:
# Password management can be very simple if all hosts are identical
"/etc/passwd"
comment => "Distribute a password file",
perms => mog("644","root","root"),
copy_from => secure_cp("/etc/passwd","$(policy_server)");
packages:
"$(mypackages)"
package_policy => "add",
package_method => generic;
# Add more promises below ...
}
body server control
{
allowconnects => { "127.0.0.1" , "::1", "10.20.30.0/24" };
allowallconnects => { "127.0.0.1" , "::1", "10.20.30.0/24" };
trustkeysfrom => { "127.0.0.1" , "::1", "10.20.30.0/24" };
# allowusers
}
bundle server my_access_rules()
{
access:
# myhost.domain.tld makes this file available to 10.20.30*
myhost_domain_tld::
"/etc/passwd"
admit => { "127.0.0.1", "10.20.30.0/24" };
}
Updating from a central hub
The configuration bundled with the CFEngine source code contains an example of centralized updating of policy that covers more subtleties than this example, and handles fault tolerance. Here is the main idea behind it. For simplicity, we assume that all hosts are on network 10.20.30.* and that the central policy server/hub is 10.20.30.123.
bundle agent update
{
vars:
"master_location" string => "/var/cfengine/masterfiles";
"policy_server" string => "10.20.30.123",
comment => "IP address to locate your policy host.";
files:
"$(sys.workdir)/inputs"
perms => system("600"),
copy_from => remote_cp("$(master_location)",$(policy_server)),
depth_search => recurse("inf");
"$(sys.workdir)/bin"
perms => system("700"),
copy_from => remote_cp("/usr/local/sbin","localhost"),
depth_search => recurse("inf");
}
body server control
{
allowconnects => { "127.0.0.1" , "10.20.30.0/24" };
allowallconnects => { "127.0.0.1" , "10.20.30.0/24" };
trustkeysfrom => { "127.0.0.1" , "10.20.30.0/24" };
}
bundle server my_access_rules()
{
access:
10_20_30_123::
"/var/cfengine/masterfiles"
admit => { "127.0.0.1", "10.20.30.0/24" };
}
Laptop support configuration
Laptops do not need a lot of confguration support. IP addresses are set by DHCP and conditions are changeable. But you want to set your DNS search domains to familiar settings in spite of local DHCP configuration, and another useful trick is to keep a regular backup of disk changes on the local disk. This won't help against disk destruction, but it is a huge advantage when your user accidentally deletes files while travelling or offline.
body common control
{
bundlesequence => {
"update",
"garbage_collection",
"main",
"backup",
};
inputs => {
"update.cf",
"site.cf",
"library.cf"
};
}
body agent control
{
# if default runtime is 5 mins we need this for long jobs
ifelapsed => "15";
}
body monitor control
{
forgetrate => "0.7";
}
body executor control
{
splaytime => "1";
mailto => "mark@iu.hio.no";
smtpserver => "localhost";
mailmaxlines => "30";
# Instead of a separate update script, now do this
exec_command => "$(sys.workdir)/bin/cf-agent -f failsafe.cf && $(sys.workdir)/bin/cf-agent";
}
bundle agent main
{
vars:
"component" slist => { "cf-monitord", "cf-serverd" };
# - - - - - - - - - - - - - - - - - - - - - - - -
files:
"$(sys.resolv)" # test on "/tmp/resolv.conf" #
create => "true",
edit_line => resolver,
edit_defaults => def;
processes:
"$(component)" restart_class => canonify("start_$(component)");
# - - - - - - - - - - - - - - - - - - - - - - - -
commands:
"$(sys.workdir)/bin/$(component)"
ifvarclass => canonify("start_$(component)");
}
bundle agent backup
{
files:
"/home/backup"
copy_from => cp("/home/mark"),
depth_search => recurse("inf"),
file_select => exclude_files,
action => longjob;
}
bundle agent garbage_collection
{
files:
"$(sys.workdir)/outputs"
delete => tidy,
file_select => days_old("3"),
depth_search => recurse("inf");
}
Process management
body common control
{
bundlesequence => { "test" };
}
bundle agent test
{
processes:
"sleep"
signals => { "term", "kill" };
}
body common control
{
bundlesequence => { "testbundle" };
}
bundle agent testbundle
{
processes:
"sleep"
process_count => up("sleep");
reports:
sleep_out_of_control::
"Out of control";
}
body process_count up(s)
{
match_range => "5,10"; # or irange("1","10");
out_of_range_define => { "$(s)_out_of_control" };
}
body common control
{
bundlesequence => { "testbundle" };
}
bundle agent testbundle
{
processes:
".*"
process_select => proc_finder("a.*"),
process_count => up("cfservd");
}
body process_count up(s)
{
match_range => "1,10"; # or irange("1","10");
out_of_range_define => { "$(s)_out_of_control" };
}
body process_select proc_finder(p)
{
stime_range => irange(ago("0","0","0","2","0","0"),now);
process_result => "stime";
}
body common control
{
bundlesequence => { "testbundle" };
}
bundle agent testbundle
{
processes:
".*"
process_select => proc_finder("a.*"),
process_count => up("cfservd");
}
body process_count up(s)
{
match_range => "1,10"; # or irange("1","10");
out_of_range_define => { "$(s)_out_of_control" };
}
body process_select proc_finder(p)
{
process_owner => { "avahi", "bin" };
command => "$(p)";
pid => "100,199";
vsize => "0,1000";
process_result => "command.(process_owner|vsize)";
}
body common control
{
bundlesequence => { "process_restart" };
}
bundle agent process_restart
{
processes:
"/usr/bin/daemon"
restart_class => "launch";
commands:
launch::
"/usr/bin/daemon";
}
body common control
{
bundlesequence => { "process_restart" };
}
bundle agent process_restart
{
vars:
"component" slist => {
"cf-monitord",
"cf-serverd",
"cf-execd"
};
processes:
"$(component)"
restart_class => canonify("start_$(component)");
commands:
"/var/cfengine/bin/$(component)"
ifvarclass => canonify("start_$(component)");
}
body common control
{
bundlesequence => { "testbundle" };
}
bundle agent testbundle
{
processes:
"cfservd"
process_count => up("cfservd");
cfservd_out_of_control::
"cfservd"
signals => { "stop" , "term" },
restart_class => "start_cfserv";
commands:
start_cfserv::
"/usr/local/sbin/cfservd";
}
body process_count up(s)
{
match_range => "1,10"; # or irange("1","10");
out_of_range_define => { "$(s)_out_of_control" };
}
Kill process
body common control
{
bundlesequence => { "test" };
}
bundle agent test
{
processes:
"sleep"
signals => { "term", "kill" };
}
Restart process
A basic pattern for restarting processes:
body common control
{
bundlesequence => { "process_restart" };
}
bundle agent process_restart
{
processes:
"/usr/bin/daemon"
restart_class => "launch";
commands:
launch::
"/usr/bin/daemon";
}
This can be made more sophisticated to handle generic lists:
body common control
{
bundlesequence => { "process_restart" };
}
bundle agent process_restart
{
vars:
"component" slist => {
"cf-monitord",
"cf-serverd",
"cf-execd"
};
processes:
"$(component)"
restart_class => canonify("start_$(component)");
commands:
"/var/cfengine/bin/$(component)"
ifvarclass => canonify("start_$(component)");
}
Why? Separating this into two parts gives a high level of control and conistency to CFEngine. There are many options for command execution, like the ability to run commands in a sandbox or as `setuid'. These should not be reproduced in processes.
Mount a filesystem
body common control
{
bundlesequence => { "mounts" };
}
bundle agent mounts
{
storage:
"/mnt" mount => nfs("slogans.iu.hio.no","/home");
}
body mount nfs(server,source)
{
mount_type => "nfs";
mount_source => "$(source)";
mount_server => "$(server)";
#mount_options => { "rw" };
edit_fstab => "true";
unmount => "true";
}
Manage a system process
Ensure running
Ensure not running
Prune processes
Ensure running
The simplest example might look like this:
bundle agent restart_process
{
processes:
"httpd"
comment => "Make sure apache web server is running",
restart_class => "restart_httpd";
commands:
restart_httpd::
"/etc/init.d/apache2 restart";
}
This example shows how the CFEngine components could be started using a pattern.
bundle agent CFEngine_processes
{
vars:
"components" slist => { "cf-execd", "cf-monitord", "cf-serverd", "cf-hub" };
processes:
"$(components)"
comment => "Make sure server parts of CFEngine are running",
restart_class => canonify("start_$(component)");
commands:
"$(sys.workdir)/bin/$(component)"
comment => "Make sure server parts of CFEngine are running",
ifvarclass => canonify("start_$(components)");
}
Ensure not running
bundle agent restart_process
{
vars:
"killprocs" slist => { "snmpd", "gameserverd", "irc", "crack" };
processes:
"$(killprocs)"
comment => "Ensure processes are not running",
signals => { "term", "kill" };
}
Prune processes
This example kills processes owned by a particular user that have exceeded 100000 bytes of resident memory.
body common control
{
bundlesequence => { "testbundle" };
}
bundle agent testbundle
{
processes:
".*"
process_select => big_processes("mark"),
signals => { "term" };
}
body process_select big_processes(o)
{
process_owner => { $(o) };
rsize => irange("100000","900000");
process_result => "rsize.process_owner";
}
Set up HPC clusters
HPC cluster machines are usually all identical, so the CFEngine configuration is very simple. HPC clients value CPU and memory resources, so we can shut down unnecessary services to save CPU. We can also change the scheduling rate of CFEngine to run less frequently, and save a little:
body executor control
{
splaytime => "1";
mailto => "cfengine@example.com";
smtpserver => "localhost";
mailmaxlines => "30";
# Once per hour, on the hour
schedule => { "Min00_05" };
}
bundle agent services_disable
{
vars:
# list all of xinetd services (case sensitive)
"xinetd_services" slist => {
"imap",
"imaps",
"ipop2",
"ipop3",
"krb5-telnet",
"klogin",
"kshell",
"ktalk",
"ntalk",
"pop3s",
};
methods:
# perform the actual disable all xinetd services according to the list above
"any" usebundle => disable_xinetd("$(xinetd_services)");
processes:
"$(xinetd_services)"
signals => { "kill" };
}
bundle agent disable_xinetd(name)
{
vars:
"status" string => execresult("/sbin/chkconfig --list $(name)", "useshell");
classes:
"on" expression => regcmp(".*on.*","$(status)");
commands:
on::
"/sbin/chkconfig $(name) off",
comment => "disable $(name) service";
reports:
on::
"disable $(name) service.";
}
Set up name resolution
There are many ways to do name resolution setup1 We write a reusable bundle using the editing features.
A simple and straightforward approach is to maintain a separate modular bundle for this task. This avoids too many levels of abstraction and keeps all the information in one place. We implement this as a simple editing promise for the /etc/resolv.conf file.
bundle agent system_files
{
files:
"$(sys.resolv)" # test on "/tmp/resolv.conf" #
comment => "Add lines to the resolver configuration",
create => "true",
edit_line => resolver,
edit_defaults => std_edits;
# ...other system files ...
}
bundle edit_line resolver
{
delete_lines:
# delete any old name servers or junk we no longer need
"search.*";
"nameserver 80.65.58.31";
"nameserver 80.65.58.32";
"nameserver 82.103.128.146";
"nameserver 78.24.145.4";
"nameserver 78.24.145.5";
"nameserver 128.39.89.10";
insert_lines:
"search mydomain.tld" location => start;
special_net::
"nameserver 128.39.89.8";
"nameserver 128.39.74.66";
!special_net::
"nameserver 128.38.34.12";
any::
"nameserver 212.112.166.18";
"nameserver 212.112.166.22";
}
A second approach is to try to conceal the operational details behind a veil of abstraction.
bundle agent system_files
{
vars:
"searchlist" string => "iu.hio.no CFEngine.com";
"nameservers" slist => {
"128.39.89.10",
"128.39.74.16",
"192.168.1.103"
};
files:
"$(sys.resolv)" # test on "/tmp/resolv.conf" #
create => "true",
edit_line => doresolv("$(s)","@(this.n)"),
edit_defaults => empty;
# ....
}
bundle edit_line doresolv(search,names)
{
insert_lines:
"search $(search)";
"nameserver $(names)";
}
bundle agent system_files { # ...
files: "/etc/hosts" comment => "Add hosts to the /etc/hosts file", edit_line => fix_etc_hosts; }
bundle edit_line fix_etc_hosts { vars: "names[127.0.0.1]" string => "localhost localhost.CFEngine.com"; "names[128.39.89.12]" string => "myhost myhost.CFEngine.com"; "names[128.39.89.13]" string => "otherhost otherhost.CFEngine.com"; # etc
"i" slist => getindices("names");
insert_lines: "$(i) $(names[$(i)])"; } ```
DNS is not the only name service, of course. Unix has its older /etc/hosts file which can also be managed using file editing. We simply append this to the system_files bundle.
bundle agent system_files
{
vars:
"searchlist" string => "iu.hio.no CFEngine.com";
"nameservers" slist => {
"128.39.89.10",
"128.39.74.16",
"192.168.1.103"
};
files:
"$(sys.resolv)" # test on "/tmp/resolv.conf" #
create => "true",
edit_line => doresolv("$(s)","@(this.n)"),
edit_defaults => empty;
# ....
}
bundle edit_line doresolv(search,names)
{
insert_lines:
"search $(search)";
"nameserver $(names)";
}
bundle agent system_files { # ...
files: "/etc/hosts" comment => "Add hosts to the /etc/hosts file", edit_line => fix_etc_hosts; }
bundle edit_line fix_etc_hosts { vars: "names[127.0.0.1]" string => "localhost localhost.CFEngine.com"; "names[128.39.89.12]" string => "myhost myhost.CFEngine.com"; "names[128.39.89.13]" string => "otherhost otherhost.CFEngine.com"; # etc
"i" slist => getindices("names");
insert_lines: "$(i) $(names[$(i)])"; } ```
Set up sudo
Setting up sudo is straightforward, and is best managed by copying trusted files from a repository.
bundle agent system_files
{
vars:
"masterfiles" string => "/subversion_projects/masterfiles";
# ...
files:
"/etc/sudoers"
comment => "Make sure the sudo configuration is secure and up to date",
perms => mog("440","root","root"),
copy_from => secure_cp("$(masterfiles)/sudoers","$(policy_server)");
}
Environments (virtual)
body common control
{
bundlesequence => { "my_vm_cloud" };
}
bundle agent my_vm_cloud
{
vars:
"vms[atlas]" slist => { "guest1", "guest2", "guest3" };
environments:
scope||any:: # These should probably be in class "any" to ensure uniqueness
"$(vms[$(sys.host)])"
environment_resources => virt_xml("$(xmlfile[$(this.promiser)])"),
environment_interface => vnet("eth0,192.168.1.100/24"),
environment_type => "test",
environment_host => "atlas";
# default environment_state => "create" on host, and "suspended elsewhere"
}
body environment_resources virt_xml(specfile)
{
env_spec_file => "$(specfile)";
}
body environment_interface vnet(primary)
{
env_name => "$(this.promiser)";
env_addresses => { "$(primary)" };
host1::
env_network => "default_vnet1";
host2::
env_network => "default_vnet2";
}
Environment variables
body common control
{
bundlesequence => { "my_vm_cloud" };
}
bundle agent my_vm_cloud
{
environments:
"centos5"
environment_resources => virt_xml,
environment_type => "xen",
environment_host => "ursa-minor";
# default environment_state => "create" on host, and "suspended elsewhere"
}
body environment_resources virt_xml
{
env_spec_file => "/srv/xen/centos5-libvirt-create.xml";
}
Tidying garbage files
Emulating the `tidy' feature of CFEngine 2.
body common control
{
any::
bundlesequence => { "testbundle" };
}
bundle agent testbundle
{
files:
"/tmp/test"
delete => tidy,
file_select => zero_age,
depth_search => recurse("inf");
}
body depth_search recurse(d)
{
#include_basedir => "true";
depth => "$(d)";
}
body delete tidy
{
dirlinks => "delete";
rmdirs => "false";
}
body file_select zero_age
{
mtime => irange(ago(1,0,0,0,0,0),now);
file_result => "mtime";
}
System File Examples
Editing password or group files
To change the password of a system, we need to edit a file. A file is a complex object – once open there is a new world of possible promises to make about its contents. CFEngine has bundles of promises that are specially for editing.
body common control
{
inputs => { "$(sys.libdir)/stdlib.cf" };
bundlesequence => { "edit_passwd" };
}
bundle agent edit_passwd
{
vars:
"userset" slist => { "user1", "user2", "user3" };
files:
"/etc/passwd"
edit_line => set_user_field("mark","7","/set/this/shell");
"/etc/group"
edit_line => append_user_field("root","4","@(main.userset)");
}
Editing password or group files custom
In this example the bundles from the Community Open Promise-Body Library are included directly in the policy instead of being input as a separate file.
body common control
{
bundlesequence => { "addpasswd" };
}
bundle agent addpasswd
{
vars:
# want to set these values by the names of their array keys
"pwd[mark]" string => "mark:x:1000:100:Mark Burgess:/home/mark:/bin/bash";
"pwd[fred]" string => "fred:x:1001:100:Right Said:/home/fred:/bin/bash";
"pwd[jane]" string => "jane:x:1002:100:Jane Doe:/home/jane:/bin/bash";
files:
"/tmp/passwd"
create => "true",
edit_line => append_users_starting("addpasswd.pwd");
}
bundle edit_line append_users_starting(v)
{
vars:
"index" slist => getindices("$(v)");
classes:
"add_$(index)" not => userexists("$(index)");
insert_lines:
"$($(v)[$(index)])",
ifvarclass => "add_$(index)";
}
bundle edit_line append_groups_starting(v)
{
vars:
"index" slist => getindices("$(v)");
classes:
"add_$(index)" not => groupexists("$(index)");
insert_lines:
"$($(v)[$(index)])",
ifvarclass => "add_$(index)";
}
Log rotation
body common control
{
bundlesequence => { "testbundle" };
}
bundle agent testbundle
{
files:
"/home/mark/tmp/rotateme"
rename => rotate("4");
}
body rename rotate(level)
{
rotate => "$(level)";
}
Garbage collection
body common control
{
bundlesequence => { "garbage_collection" };
inputs => { "cfengine_stdlib.cf" };
}
bundle agent garbage_collection
{
files:
Sunday::
"$(sys.workdir)/nova_repair.log"
comment => "Rotate the promises repaired logs each week",
rename => rotate("7"),
action => if_elapsed("10000");
"$(sys.workdir)/nova_notkept.log"
comment => "Rotate the promises not kept logs each week",
rename => rotate("7"),
action => if_elapsed("10000");
"$(sys.workdir)/promise.log"
comment => "Rotate the promises not kept logs each week",
rename => rotate("7"),
action => if_elapsed("10000");
any::
"$(sys.workdir)/outputs"
comment => "Garbage collection of any output files",
delete => tidy,
file_select => days_old("3"),
depth_search => recurse("inf");
"$(sys.workdir)/"
comment => "Garbage collection of any output files",
delete => tidy,
file_select => days_old("14"),
depth_search => recurse("inf");
# Other resources
"/tmp"
comment => "Garbage collection of any temporary files",
delete => tidy,
file_select => days_old("3"),
depth_search => recurse("inf");
"/var/log/apache2/.*bz"
comment => "Garbage collection of rotated log files",
delete => tidy,
file_select => days_old("30"),
depth_search => recurse("inf");
"/var/log/apache2/.*gz"
comment => "Garbage collection of rotated log files",
delete => tidy,
file_select => days_old("30"),
depth_search => recurse("inf");
"/var/log/zypper.log"
comment => "Prevent the zypper log from choking the disk",
rename => rotate("0"),
action => if_elapsed("10000");
}
Manage a system file
Simple template
Simple versioned template
Macro template
Custom editing
Simple template
bundle agent hand_edited_config_file
{
vars:
"file_template" string =>
"
127.0.0.1 localhost
::1 localhost ipv6-localhost ipv6-loopback
fe00::0 ipv6-localnet
ff00::0 ipv6-mcastprefix
ff02::1 ipv6-allnodes
ff02::2 ipv6-allrouters
ff02::3 ipv6-allhosts
10.0.0.100 host1.domain.tld host1
10.0.0.101 host2.domain.tld host2
10.0.0.20 host3.domain.tld host3
10.0.0.21 host4.domain.tld host4
";
##############################################################
files:
"/etc/hosts"
comment => "Define the content of all host files from this master source",
create => "true",
edit_line => append_if_no_lines("$(file_template)"),
edit_defaults => empty,
perms => mo("$(mode)","root"),
action => if_elapsed("60");
}
Simple versioned template
The simplest approach to managing a file is to maintain a master copy by hand, keeping it in a version controlled repository (e.g. svn), and installing this version on the end machine.
We'll assume that you have a version control repository that is located on some independent server, and has been checked out manually once (with authentication) in /mysite/masterfiles.
bundle agent hand_edited_config_file
{
vars:
"masterfiles" string => "/mysite/masterfiles";
"policy_server" string => "policy_host.domain.tld";
files:
"/etc/hosts"
comment => "Synchronize hosts with a hand-edited template in svn",
perms => m("644"),
copy_from => remote_cp("$(masterfiles)/trunk/hosts_master","$(policy_server)");
commands:
"/usr/bin/svn update"
comment => "Update the company document repository including manuals to a local copy",
contain => silent_in_dir("$(masterfiles)/trunk"),
ifvarclass => canonify("$(policy_server)");
}
Macro template
The next simplest approach to file management is to add variables to the template that will be expanded into local values at the end system, e.g. using variables like ‘$(sys.host)’ for the name of the host within the body of the versioned template.
bundle agent hand_edited_template
{
vars:
"masterfiles" string => "/mysite/masterfiles";
"policy_server" string => "policy_host.domain.tld";
files:
"/etc/hosts"
comment => "Synchronize hosts with a hand-edited template in svn",
perms => m("644"),
create => "true",
edit_line => expand_template("$(masterfiles)/trunk/hosts_master"),
edit_defaults => empty,
action => if_elapsed("60");
commands:
"/usr/bin/svn update"
comment => "Update the company document repository including manuals to a local copy",
contain => silent_in_dir("$(masterfiles)/trunk"),
ifvarclass => canonify("$(policy_server)");
}
The macro template file may contain variables, as below, that get expanded by CFEngine.
bundle agent hand_edited_template
{
vars:
"masterfiles" string => "/mysite/masterfiles";
"policy_server" string => "policy_host.domain.tld";
files:
"/etc/hosts"
comment => "Synchronize hosts with a hand-edited template in svn",
perms => m("644"),
create => "true",
edit_line => expand_template("$(masterfiles)/trunk/hosts_master"),
edit_defaults => empty,
action => if_elapsed("60");
commands:
"/usr/bin/svn update"
comment => "Update the company document repository including manuals to a local copy",
contain => silent_in_dir("$(masterfiles)/trunk"),
ifvarclass => canonify("$(policy_server)");
}
127.0.0.1 localhost $(sys.host) ::1 localhost ipv6-localhost ipv6-loopback fe00::0 ipv6-localnet ff00::0 ipv6-mcastprefix ff02::1 ipv6-allnodes ff02::2 ipv6-allrouters ff02::3 ipv6-allhosts 10.0.0.100 host1.domain.tld host1 10.0.0.101 host2.domain.tld host2 10.0.0.20 host3.domain.tld host3 10.0.0.21 host4.domain.tld host4
$(definitions.more_hosts) ```
Custom editing
If you do not control the starting state of the file, because it is distributed by an operating system vendor for instance, then editing the final state is the best approach. That way, you will get changes that are made by the vendor, and will ensure your own modifications are kept even when updates arrive.
bundle agent modifying_managed_file
{
vars:
"data" slist => { "10.1.2.3 sirius", "10.1.2.4 ursa-minor", "10.1.2.5 orion"};
files:
"/etc/hosts"
comment => "Append a list of lines to the end of a file if they don't exist",
perms => m("644"),
create => "true",
edit_line => append_if_no_lines("modifying_managed_file.data"),
action => if_elapsed("60");
}
Another example shows how to set the values of variables using a data-driven approach and methods from the standard library.
body common control
{
bundlesequence => { "testsetvar" };
}
bundle agent testsetvar
{
vars:
"v[variable_1]" string => "value_1";
"v[variable_2]" string => "value_2";
files:
"/tmp/test_setvar"
edit_line => set_variable_values("testsetvar.v");
}
Windows Registry Examples
Windows registry
body common control
{
bundlesequence => { "reg" };
}
bundle agent reg
{
vars:
"value" string => registryvalue("HKEY_LOCAL_MACHINE\SOFTWARE\Cfengine AS\Cfengine","value3");
reports:
windows::
"Value extracted: $(value)";
}
unit_registry_cache.cf
body common control
{
bundlesequence => {
# "registry_cache"
# "registry_restore"
};
}
bundle agent registry_cache
{
databases:
windows::
"HKEY_LOCAL_MACHINE\SOFTWARE\Adobe"
database_operation => "cache",
database_type => "ms_registry",
comment => "Save correct registry settings for Adobe products";
}
bundle agent registry_restore
{
databases:
windows::
"HKEY_LOCAL_MACHINE\SOFTWARE\Adobe"
database_operation => "restore",
database_type => "ms_registry",
comment => "Make sure Adobe products have correct registry settings";
}
unit_registry.cf
body common control
{
bundlesequence => { "databases" };
}
bundle agent databases
{
databases:
windows::
# Registry has (value,data) pairs in "keys" which are directories
# "HKEY_LOCAL_MACHINE\SOFTWARE\Cfengine AS"
# database_operation => "create",
# database_type => "ms_registry";
# "HKEY_LOCAL_MACHINE\SOFTWARE\Cfengine AS\Cfengine"
# database_operation => "create",
# database_rows => { "value1,REG_SZ,new value 1", "value2,REG_SZ,new val 2"} ,
# database_type => "ms_registry";
"HKEY_LOCAL_MACHINE\SOFTWARE\Cfengine AS\Cfengine"
database_operation => "delete",
database_columns => { "value1", "value2" } ,
database_type => "ms_registry";
# "HKEY_LOCAL_MACHINE\SOFTWARE\Cfengine AS\Cfengine"
# database_operation => "cache", # cache,restore
# registry_exclude => { ".*Windows.*CurrentVersion.*", ".*Touchpad.*", ".*Capabilities.FileAssociations.*", ".*Rfc1766.*" , ".*Synaptics.SynTP.*", ".*SupportedDevices.*8086", ".*Microsoft.*ErrorThresholds" },
# database_type => "ms_registry";
"HKEY_LOCAL_MACHINE\SOFTWARE\Cfengine AS"
database_operation => "restore",
database_type => "ms_registry";
}
File Permissions
ACL file example
body common control
{
bundlesequence => { "acls" };
}
bundle agent acls
{
files:
"/media/flash/acl/test_dir"
depth_search => include_base,
acl => template;
}
body acl template
{
acl_method => "overwrite";
acl_type => "posix";
acl_directory_inherit => "parent";
aces => { "user:*:r(wwx),-r:allow", "group:*:+rw:allow", "mask:x:allow", "all:r"};
}
body acl win
{
acl_method => "overwrite";
acl_type => "ntfs";
acl_directory_inherit => "nochange";
aces => { "user:Administrator:rw", "group:Bad:rwx(Dpo):deny" };
}
body depth_search include_base
{
include_basedir => "true";
}
ACL generic example
body common control
{
bundlesequence => { "acls" };
}
bundle agent acls
{
files:
"/media/flash/acl/test_dir"
depth_search => include_base,
acl => test;
}
body acl test
{
acl_type => "generic";
aces => {"user:bob:rwx", "group:staff:rx", "all:r"};
}
body depth_search include_base
{
include_basedir => "true";
}
ACL secret example
body common control
{
bundlesequence => { "acls" };
}
bundle agent acls
{
files:
windows::
"c:\Secret"
acl => win,
depth_search => include_base,
comment => "Secure the secret directory from unauthorized access";
}
body acl win
{
acl_method => "overwrite";
aces => { "user:Administrator:rwx" };
}
body depth_search include_base
{
include_basedir => "true";
}
User Management Examples
Local user management
There are many approaches to managing users. You can edit system files
like /etc/passwd
directly, you can use commands on some systems like
useradd
. However the easiest, and preferred way is to use
CFEngine's native users
type promise.
Ensuring a local user has a specific password
This example shows ensuring that the local users root
is managed if
there is a specific password hash defined.
body file control
{
# This policy uses parts of the standard library.
inputs => { "$(sys.libdir)/users.cf" };
}
bundle agent main
{
vars:
# This is the hashed password for 'vagrant'
debian_8::
"root_hash"
string => "$6$1nRTeNoE$DpBSe.eDsuZaME0EydXBEf.DAwuzpSoIJhkhiIAPgRqVKlmI55EONfvjZorkxNQvK2VFfMm9txx93r2bma/4h/";
users:
linux::
"root"
policy => "present",
password => hashed_password( $(root_hash) ),
if => isvariable("root_hash");
}
This policy can be found in
/var/cfengine/share/doc/examples/local_user_password.cf
and downloaded directly from
github.
root@debian-jessie:/core/examples# grep root /etc/shadow
root:!:16791:0:99999:7:::
root@debian-jessie:/core/examples# cf-agent -KIf ./local_user_password.cf
info: User promise repaired
root@debian-jessie:/core/examples# grep root /etc/shadow
root:$6$1nRTeNoE$DpBSe.eDsuZaME0EydXBEf.DAwuzpSoIJhkhiIAPgRqVKlmI55EONfvjZorkxNQvK2VFfMm9txx93r2bma/4h/:16791:0:99999:7:::
Ensuring local users are present
This example shows ensuring that the local users jack
and jill
are
present on all linux systems using the native users
type promise.
body file control
{
# This policy uses parts of the standard library.
inputs => { "$(sys.libdir)/files.cf" };
}
bundle agent main
{
vars:
"users" slist => { "jack", "jill" };
"skel" string => "/etc/skel";
users:
linux::
"$(users)"
home_dir => "/home/$(users)",
policy => "present",
home_bundle => home_skel( $(users), $(skel) );
}
bundle agent home_skel(user, skel)
{
files:
"/home/$(user)/."
create => "true",
copy_from => seed_cp( $(skel) ),
depth_search => recurse( "inf" );
}
This policy can be found in
/var/cfengine/share/doc/examples/local_users_present.cf
and downloaded directly from
github.
Lets check the environment to see that the users do not currently exist.
root@debian-jessie:/CFEngine/core/examples# egrep "jack|jill" /etc/passwd
root@debian-jessie:/core/examples# ls -al /home/{jack,jill}
ls: cannot access /home/jack: No such file or directory
ls: cannot access /home/jill: No such file or directory
Let's run the policy and inspect the state of the system afterwards.
root@debian-jessie:/core/examples# cf-agent -KIf ./users_present.cf
info: Created directory '/home/jack/.'
info: Copying from 'localhost:/etc/skel/.bashrc'
info: Copying from 'localhost:/etc/skel/.profile'
info: Copying from 'localhost:/etc/skel/.bash_logout'
info: User promise repaired
info: Created directory '/home/jill/.'
info: Copying from 'localhost:/etc/skel/.bashrc'
info: Copying from 'localhost:/etc/skel/.profile'
info: Copying from 'localhost:/etc/skel/.bash_logout'
info: User promise repaired
root@debian-jessie:/core/examples# egrep "jack|jill" /etc/passwd
jack:x:1001:1001::/home/jack:/bin/sh
jill:x:1002:1002::/home/jill:/bin/sh
root@debian-jessie:/core/examples# ls -al /home/{jack,jill}
/home/jack:
total 20
drwxr-xr-x 2 root root 4096 Dec 22 16:37 .
drwxr-xr-x 5 root root 4096 Dec 22 16:37 ..
-rw-r--r-- 1 root root 220 Dec 22 16:37 .bash_logout
-rw-r--r-- 1 root root 3515 Dec 22 16:37 .bashrc
-rw-r--r-- 1 root root 675 Dec 22 16:37 .profile
/home/jill:
total 20
drwxr-xr-x 2 root root 4096 Dec 22 16:37 .
drwxr-xr-x 5 root root 4096 Dec 22 16:37 ..
-rw-r--r-- 1 root root 220 Dec 22 16:37 .bash_logout
-rw-r--r-- 1 root root 3515 Dec 22 16:37 .bashrc
-rw-r--r-- 1 root root 675 Dec 22 16:37 .profile
Ensuring local users are locked
This example shows ensuring that the local users jack
and jill
are
locked if they are present on linux systems using the native users
type promise.
bundle agent main
{
vars:
"users" slist => { "jack", "jill" };
users:
linux::
"$(users)"
policy => "locked";
}
This policy can be found in
/var/cfengine/share/doc/examples/local_users_locked.cf
and downloaded directly from
github.
This output shows the state of the /etc/shadow
file before running
the example policy:
root@debian-jessie:/core/examples# egrep "jack|jill" /etc/shadow
jack:x:16791:0:99999:7:::
jill:x:16791:0:99999:7:::
root@debian-jessie:/core/examples# cf-agent -KIf ./local_users_locked.cf
info: User promise repaired
info: User promise repaired
root@debian-jessie:/core/examples# egrep "jack|jill" /etc/shadow
jack:!x:16791:0:99999:7::1:
jill:!x:16791:0:99999:7::1:
Ensuring local users are absent
This example shows ensuring that the local users jack
and jill
are
absent on linux systems using the native users
type promise.
bundle agent main
{
vars:
"users" slist => { "jack", "jill" };
users:
linux::
"$(users)"
policy => "absent";
}
This policy can be found in
/var/cfengine/share/doc/examples/local_users_absent.cf
and downloaded directly from
github.
Before activating the example policy, lets inspect the current state of the system.
root@debian-jessie:/core/examples# egrep "jack|jill" /etc/passwd
jack:x:1001:1001::/home/jack:/bin/sh
jill:x:1002:1002::/home/jill:/bin/sh
root@debian-jessie:/core/examples# ls -al /home/{jack,jill}
/home/jack:
total 20
drwxr-xr-x 2 root root 4096 Dec 22 16:37 .
drwxr-xr-x 5 root root 4096 Dec 22 16:37 ..
-rw-r--r-- 1 root root 220 Dec 22 16:37 .bash_logout
-rw-r--r-- 1 root root 3515 Dec 22 16:37 .bashrc
-rw-r--r-- 1 root root 675 Dec 22 16:37 .profile
/home/jill:
total 20
drwxr-xr-x 2 root root 4096 Dec 22 16:37 .
drwxr-xr-x 5 root root 4096 Dec 22 16:37 ..
-rw-r--r-- 1 root root 220 Dec 22 16:37 .bash_logout
-rw-r--r-- 1 root root 3515 Dec 22 16:37 .bashrc
-rw-r--r-- 1 root root 675 Dec 22 16:37 .profile
From the above output we can see that the local users jack
and
jill
are present, and that they both have home directories.
Now lets activate the example policy and insepect the result.
root@debian-jessie:/core/examples# cf-agent -KIf ./local_users_absent.cf
info: User promise repaired
info: User promise repaired
root@debian-jessie:/core/examples# egrep "jack|jill" /etc/passwd
root@debian-jessie:/core/examples# ls -al /home/{jack,jill}
/home/jack:
total 20
drwxr-xr-x 2 root root 4096 Dec 22 16:37 .
drwxr-xr-x 5 root root 4096 Dec 22 16:37 ..
-rw-r--r-- 1 root root 220 Dec 22 16:37 .bash_logout
-rw-r--r-- 1 root root 3515 Dec 22 16:37 .bashrc
-rw-r--r-- 1 root root 675 Dec 22 16:37 .profile
/home/jill:
total 20
drwxr-xr-x 2 root root 4096 Dec 22 16:37 .
drwxr-xr-x 5 root root 4096 Dec 22 16:37 ..
-rw-r--r-- 1 root root 220 Dec 22 16:37 .bash_logout
-rw-r--r-- 1 root root 3515 Dec 22 16:37 .bashrc
-rw-r--r-- 1 root root 675 Dec 22 16:37 .profile
From the above output we can see that the local users jack
and
jill
were removed from the system as desired. Note that their home
directories remain, and if we wanted them to be purged we would have
to have a separate promise to perform that cleanup.
Local group management
CFEngine does not currently have a native groups
type promsie so you
will need to either edit the necessary files using files
type
promises, or arrange for the proper commands to be run in order to
create or delete groups.
Ensure a local group is present
Add lines to the password file, and users to group if they are not already there.
This example uses the native operating system commands to show ensuring that a group is present.
body file control
{
# This policy uses parts of the standard library.
inputs => { "$(sys.libdir)/paths.cf" };
}
bundle agent main
{
classes:
"group_cfengineers_absent"
not => groupexists("cfengineers");
commands:
linux.group_cfengineers_absent::
"$(paths.groupadd)"
args => "cfengineers";
}
This policy can be found in
/var/cfengine/share/doc/examples/local_group_present.cf
and downloaded directly from
github.
First lets inspect the current state of the system.
root@debian-jessie:/core/examples# grep cfengineers /etc/group
Now lets activate the example policy and check the resulting state of the system.
root@debian-jessie:/core/examples# cf-agent -KIf ./local_group_present.cf
info: Executing 'no timeout' ... '/usr/sbin/groupadd cfengineers'
info: Completed execution of '/usr/sbin/groupadd cfengineers'
root@debian-jessie:/CFEngine/core2.git/examples# grep cfengineers /etc/group
cfengineers:x:1001:
Ensureing a user is a member of a secondary group
This example shows using the native users
type promise to ensure
that a user is a member of a particular group.
bundle agent main
{
users:
linux::
"jill"
policy => "present",
groups_secondary => { "cfengineers" };
}
This policy can be found in
/var/cfengine/share/doc/examples/local_user_secondary_group_member.cf
and downloaded directly from
github.
First lets inspect the current state of the system
root@debian-jessie:/core/examples# grep jill /etc/passwd
root@debian-jessie:/core/examples# grep jill /etc/group
Now lets actiavte the example policy and inspect the resulting state.
root@debian-jessie:/core/examples# cf-agent -KIf ./local_user_secondary_group_member.cf
info: User promise repaired
root@debian-jessie:/core/examples# grep jill /etc/passwd
jill:x:1001:1002::/home/jill:/bin/sh
root@debian-jessie:/core/examples# grep jill /etc/group
cfengineers:x:1001:jill
jill:x:1002:
It's important to remember we made no promise about the presence of
the cfengineers
group in the above example. We can see what would
happen when the cfengineers
group was not present.
root@debian-jessie:/core/examples# grep cfengineers /etc/group
root@debian-jessie:/core/examples# cf-agent -KIf ./local_user_secondary_group_member.cf
usermod: group 'cfengineers' does not exist
error: Command returned error while modifying user 'jill'. (Command line: '/usr/sbin/usermod -G "cfengineers" jill')
info: User promise not kept
Get a list of users
body common control
{
bundlesequence => { test };
}
bundle agent test
{
vars:
"allusers" slist => getusers("zenoss,mysql,at","12,0");
reports:
linux::
"Found user $(allusers)";
}
Tutorials
Familarize yourself with CFEngine by following these step by step tutorials.
Additional tutorials to help you get started including screen casts can be found in the CFEngine Learning Center.
JSON and YAML Support in CFEngine
Introduction
JSON is a well-known data language. It even has a specification (See http://json.org).
YAML is another well-known data language. It has a longer, much more complex specification (See http://yaml.org).
CFEngine has core support for JSON and YAML. Let's see what it can do.
Problem statement
We'd like to read, access, and merge JSON-sourced data structures: they should be weakly typed, arbitrarily nested, with consistent quoting and syntax.
We'd like to read, access, and merge YAML-sourced data structures just like JSON-sourced, to keep policy and internals simple.
In addition, we must not break backward compatibility with CFEngine
3.5 and older, so we'd like to use the standard CFEngine array a[b]
syntax.
Data containers
A new data type, the data container, was introduced in 3.6.
It's simply called data
. The documentation with some examples is at https://cfengine.com/docs/master/reference-promise-types-vars.html#data-container-variables
Reading JSON
There are many ways to read JSON data; here are a few:
readjson()
: read from a JSON file, e.g."mydata" data => readjson("/my/file", 100k);
parsejson()
: read from a JSON string, e.g."mydata" data => parsejson('{ "x": "y" }');
data_readstringarray()
anddata_readstringarrayidx()
: read text data from a file, split it on a delimiter, and make them into structured data.mergedata()
: merge data containers, slists, and classic CFEngine arrays, e.g."mydata" data => mergedata(container1, slist2, array3);
mergedata
in particular is very powerful. It can convert a slist or a classic CFEngine array to a data container easily: "mydata" data => mergedata(myslist);
Reading YAML
There are two ways to read YAML data:
readyaml()
: read from a YAML file, e.g."mydata" data => readyaml("/my/file.yaml", 100k);
parseyaml()
: read from a YAML string, e.g."mydata" data => parseyaml('- arrayentry1');
Since these functions return data containers, everything about JSON-sourced data structures applies to YAML-sourced data structures as well.
Accessing JSON
To access JSON data, you can use:
- the
nth()
function to access an array element, e.g."myx" string => nth(container1, 0);
- the
nth
function to access a map element, e.g."myx" string => nth(container1, "x");
- the
a[b]
notation, e.g."myx" string => "$(container1[x])";
. You can nest, e.g.a[b][c][0][d]
. This only works if the element is something that can be expanded in a string. So a number or a string work. A list of strings or numbers works. A key-value map underx
won't work. - the
getindices()
andgetvalues()
functions, just like classic CFEngine arrays
A full example
This example can be saved and run. It will load a key-value map where the keys are class names and the values are hostname regular expressions or class names.
- if your host name is
c
orb
or the classesc
orb
are defined, thedev
class will be defined - if your host name is
flea
or the classflea
is defined, theprod
class will be defined - if your host name is
a
or the classa
is defined, theqa
class will be defined - if your host name is
linux
or the classlinux
is defined, theprivate
class will be defined
Easy, right?
body common control
{
bundlesequence => { "run" };
}
bundle agent run
{
vars:
"bykey" data => parsejson('{ "dev": ["c", "b"], "prod": ["flea"], "qa": ["a"], "private": ["linux"] }');
"keys" slist => getindices("bykey");
classes:
# define the class from the key name if any of the items under the key match the host name
"$(keys)" expression => regcmp("$(bykey[$(keys)])", $(sys.host));
# define the class from the key name if any of the items under the key are a defined class
"$(keys)" expression => classmatch("$(bykey[$(keys)])");
reports:
"keys = $(keys)";
"I am in class $(keys)" ifvarclass => $(keys);
}
So, where's the magic? Well, if you're familiar with classic CFEngine
arrays, you will be happy to hear that the exact same syntax works
with them. In other words, data containers don't change how you use
CFEngine. You still use getindices
to get the keys, then iterate
through them and look up values.
Well, you can change
"bykey" data => parsejson('{ "dev": ["c", "b"], "prod": ["flea"], "qa": ["a"], "private": ["linux"] }');
with
"bykey" data => data_readstringarray(...);
and read the same container from a text file. The file should be formatted like this to produce the same data as above:
dev c b
prod flea
qa a
private linux
You can also use
"bykey" data => readjson(...);
and read the same container from a JSON file.
Summary
Using JSON and YAML from CFEngine is easy and does not change how you use CFEngine. Try it out and see for yourself!
Distribute files from a central location
CFEngine can manage many machines simply by distributing policies to all its hosts. This tutorial describes how to distribute files to hosts from a central policy server location. For this example, we will distribute software patches.
Files are centrally stored on the policy server (hub). In our example, they are stored in /storage/patches
.
These patch files must also exist on the agent host (client) in /storage/deploy/patches
. To do this,
perform the following instructions:
Check out masterfiles from your central repository
CFEngine stores the master copy of all policy in the /var/cfengine/masterfiles
directory.
Ensure that you are working with the latest version of your masterfiles
.
git clone url
or
git pull origin master
Make policy changes
Define locations
Before files can be copied we must know where files should be copied from and where files should be copied to. If these locations are used by multiple components, then defining them in a common bundle can reduce repetition. Variables and classes that are defined in common bundles are accessible by all CFEngine components. This is especially useful in the case of file copies because the same variable definition can be used both by the policy server when granting access and by the agent host when performing the copy.
The policy framework includes a common bundle called def
. In this example, we
will add two variables--dir_patch_store
and dir_patch_deploy
--to this existing bundle.
These variables provide path definitions for storing and deploying patches.
Add the following variable information to the masterfiles/def.cf
file:
"dir_patch_store"
string => "/storage/patches",
comment => "Define patch files source location",
handle => "common_def_vars_dir_patch_store";
"dir_patch_deploy"
string => "/storage/deploy/patches",
comment => "Define patch files deploy location",
handle => "common_def_vars_dir_patch_deploy";
}
These common variables can be referenced from the rest of the policy by using their fully
qualified names,
$(def.dir_patch_store)
and $(def.dir_patch_deploy)
Grant file access
Access must be granted before files can be copied. The right to access a file
is provided by cf-serverd
, the server component of CFEngine. Enter access information using the access
promise type in a server
bundle. The default access rules defined by the MPF (Masterfiles Policy Framework) can be found in
controls/cf_serverd.cf
.
There is no need to modify the vendored policy, instead define your own server bundle. For our example, add the following to services/main.cf
:
bundle server my_access_rules
{
access:
"$(def.dir_patch_store)"
handle => "server_access_grant_locations_files_patch_store_for_hosts",
admit => { ".*$(def.domain)", @(def.acl) },
comment => "Hosts need to download patch files from the central location";
}
Create a custom library for reusable synchronization policy
You might need to frequently synchronize or copy a directory structure from the policy server to an agent host. Thus, identifying reusable parts of policy and abstracting them for later use is a good idea. This information is stored in a custom library.
Create a custom library called lib/custom/files.cf
. Add the following content:
bundle agent sync_from_policyserver(source_path, dest_path)
# @brief Sync files from the policy server to the agent
#
# @param source_path Location on policy server to copy files from
# @param dest_path Location on agent host to copy files to
{
files:
"$(dest_path)/."
handle => "sync_from_policy_server_files_dest_path_copy_from_source_path_sys_policy_hub",
copy_from => sync_cp("$(source_path)", "$(sys.policy_hub)"),
depth_search => recurse("inf"),
comment => "Ensure files from $(sys.policy_hub):$(source_path) exist in $(dest_path)";
}
This reusable policy will be used to synchronize a directory on the policy server to a directory on the agent host.
Create a patch policy
Organize in a way that makes the most sense to you and your team. We recommend organizing policy by services.
Create services/patching.cf
with the following content:
# Patching Policy
bundle agent patching
# @brief Ensure various aspects of patching are handeled
# We can break down the various parts of patching into separate bundles. This
# allows us to become less overwhelmed by details if numerous specifics
# exist in one or more aspect for different host classifications.
{
methods:
"Patch Distribution"
handle => "patching_methods_patch_distribution",
usebundle => "patch_distribution",
comment => "Ensure patches are properly distributed";
}
bundle agent patch_distribution
# @brief Ensures that our patches are distributed to the proper locations
{
files:
"$(def.dir_patch_deploy)/."
handle => "patch_distribution_files_def_dir_patch_deploy_exists",
create => "true",
comment => "If the destination directory does not exist, we have no place
to which to copy the patches.";
methods:
"Patches"
handle => "patch_distribution_methods_patches_from_policyserver_def_dir_patch_store_to_def_dir_patch_deploy",
usebundle => sync_from_policyserver("$(def.dir_patch_store)", "$(def.dir_patch_deploy)"),
comment => "Patches need to be present on host systems so that we can use
them. By convention we use the policy server as the central
distribution point.";
}
The above policy contains two bundles. We have separated a top-level patching
bundle from a more specific patch_distribution
bundle. This is an
illustration of how to use bundles in order to abstract details. You
might, for example, have some hosts that you don’t want to fully
synchronize so you might use a different method or copy from a
different path. Creating numerous bundles allows you to move those details away from the top
level of what is involved in patching. If people are interested in what
is involved in patch distribution, they can view that bundle for specifics.
Integrate the policy
Now that all the pieces of the policy are in place, they must be integrated
into the policy so they can be activated. Add each policy file to the inputs
section which is found under body common control
. Once the policy file is included in
inputs, the bundle can be activated. Bundles can be activated by adding them to either the
bundlesequence
or they can be called as a methods
type promise.
Add the following entries to promises.cf
under body common control
-> inputs
:
"lib/custom/files.cf",
"services/patching.cf",
and the following to promises.cf
under body common control
-> bundlesequence
:
"patching",
Now that all of the policy has been edited and is in place, check for syntax errors by
running cf-promises -f ./promises.cf
. This promise is activated from the service_catalogue
bundle.
Commit Changes
Set up trackers in the Mission Portal (Enterprise Users Only)
Before committing the changes to your repository, log in to the Mission Portal and set up a Tracker so that you can see the policy as it goes out. To do this, perform the following:
Navigate to the Hosts section. Select All hosts. Select the Events tab, located in the right-hand panel. Click Add new tracker.
Name it Patch Failure. Set the Report Type to Promise not Kept. Under Watch, enter .patch. Set the Start Time to Now and then click Done to close the Start Time window. Click Start to save the new tracker. This tracker watches for any promise handle that includes the string patch where a promise is not kept.
Add another tracker called Patch Repaired. Set the Report Type to Promise Repaired. Enter the same values as above for Watch and Start Time. Click Start to save the new tracker. This tracker allows you to see how the policy reacts as it is activated on your infrastructure.
Deploy changes (Enterprise and Community Users)
Always inspect what you expect. git status
shows the status of your current branch.
git status
Inspect the changes contained in each file. Once satisfied, add them to Git's commit staging area.
git diff file
git add file
Iterate over using git diff, add, and status until all of the changes that you expected are listed as Changes to be committed. Check the status once more before you commit the changes.
git status
Commit the changes to your local repository.
git commit
Push the changes to the central repository so they can be pulled down to your policy server for distribution.
git push origin master
Reporting and Remediation of Security Vulnerabilities
Prerequisites
- CFEngine 3.6 Enterprise Hub
- At least one client vulnerable to CVE-2014-6271
Overview
Remediating security vulnerabilities is a common issue. Sometimes you want to know the extent to which your estate is affected by a threat. Identification of affected systems can help you prioritize and plan remediation efforts. In this tutorial you will learn how to inventory your estate and build alerts to find hosts that are affected by the #shellshock exploit. After identifying the affected hosts you will patch a subset of the hosts and then be able to see the impact on your estate. The same methodology can be applied to other issues.
Note: The included policy does not require CFEngine Enterprise. Only the reporting functionality (Mission Portal) requires the Enterprise version.
Inventory CVE-2013-6271
Writing inventory policy with CFEngine is just like any other CFEngine policy,
except for the addition of special meta
attributes used to augment the
inventory interface. First you must know how to collect the information you
want. In this case we know that a vulnerable system will have the word
vulnerable listed in the output of the command
env x='() { :;}; echo vulnerable' $(bash) -c 'echo testing CVE-2014-6271'
.
This bundle will check if the host is vulnerable to the CVE, define a class CVE_2014_6217 if it is vulnerable and augment Mission Portals Inventory interface in CFEngine Enterprise.
bundle agent inventory_CVE_2014_6271
{
meta:
"description" string => "Remote exploit vulnerability in bash http://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2014-6271";
"tags" slist => { "autorun" };
vars:
"env" string => "$(paths.env)";
"bash" string => "/bin/bash";
"echo" string => "$(paths.echo)";
"test_result" string => execresult("$(env) x='() { :;}; $(echo) vulnerable' $(bash) -c 'echo testing CVE-2014-6271'", "useshell");
CVE_2014_6271::
"vulnerable"
string => "CVE-2014-6271",
meta => { "inventory", "attribute_name=Vulnerable CVE(s)" };
classes:
"CVE_2014_6271"
expression => regcmp( "vulnerable.*", "$(test_result)" ),
scope => "namespace",
persistence => "10",
comment => "We persist the class for 2 agent runs so that bundles
activated before this bundle can use the class on the next
agent execution to coordinate things like package updates.";
reports:
DEBUG|DEBUG_cve_2014_6217::
"Test Result: $(test_result)";
CVE_2014_6271.(inform_mode|verbose_mode)::
"Tested Vulnerable for CVE-2014-6271: $($(this.bundle)_meta.description)";
}
What does this inventory policy do?
Meta type promises are used to attach additional information to bundles. We have set 'description' so that future readers of the policy will know what the policy is for and how to get more information on the vulnerability. For the sake of simplicity in this example set 'autorun' as a tag to the bundle. This makes the bundle available for automatic activation when using the autorun feature in the Masterfiles Policy Framework.
Next we set the paths to the binaries that we will use to exeucte our test command. As of this writing the paths for 'env' and 'echo' are both in the standard libraries paths bundle, but 'bash' is not. Note that you may need to adjust the path to bash for your platforms. Then we run our test command and place the command output into the 'test_result' variable. Since we have no CVE_2014_6271 class defined yet, the next promise to set the variable 'vulnerable' to 'CVE-2014-6271' will be skipped on the first pass. Then the classes type promise is evaluated and defines the class CVE_2014_6271 if the output matches the regular expression 'vulnerable.*'. Finally the reports are evaluated before starting the second pass. If the class 'DEBUG' or 'DEBUG_inventory_CVE_2014_6271' is set the test command output will be shown, and if the vulnerability is present agent is running in inform or verbose mode message indicating the host is vulnerable along with the description will be output.
On the second pass only that variable 'vulnerable' will be set with the value 'CVE-2014-6271' if the host is vulnerable. Note how this variable tagged with 'inventory' and 'attribute_name='. These are special meta tags that CFEngine Enterprise uses in order to display information.
Deploy the policy
As noted previously, in this example we will use autorun for simplicity. Please
ensure that the class "services_autorun" is defined. The easiest way to do this
is to change "services_autorun" expression => "!any";
to "services_autorun"
expression => "any";
in def.cf
.
Once you have autorun enabled you need only save the policy into
services/autorun/inventory_CVE_2014_6271.cf
.
Report on affected system inventory
Within 20 minutes of deploying the policy you should be able to see results in the Inventory Reporting interface.
A new Inventory attribute 'Vulnerable CVE(s)' is available.
Report showing CVEs that each host is vulnerable to.
Chart the Vulnerable CVE(s) and get a visual breakdown.
Build Dashboard Widget with Alerts
Let's add alerts for CVE(s) to the dashboard.
Give the dashboard widget a name.
Configure an general CVE alert for the dashboard.
Add an additional alert for this specific CVE.
See the dashboard alert in action.
Remediate Vulnerabilities
Now that we know the extent of exposure lets ensure bash gets updated on some
of the affected systems. Save the following policy into
services/autorun/remediate_CVE_2014_6271.cf
bundle agent remediate_CVE_2014_6271
{
meta:
"tags" slist => { "autorun" };
classes:
"allow_update" or => { "hub", "host001" };
methods:
allow_update.CVE_2014_6271::
"Upgrade_Bash"
usebundle => package_latest("bash");
}
What does this remediation policy do?
For simplicity of the example this policy defines the class allow_update on hub and host001, but you could use any class that makes sense to you. If the allow_update class is set, and the class CVE_2014_6271 is defined (indicating the host is vulnerable) then the policy ensures that bash is updated to the latest version available.
Report on affected systems inventory after remediation
Within 20 minutes or so of the policy being deployed you will be able to report on the state of remediation.
See the remediation efforts relfected in the dashboard.
Drill down into the dashboard and alert details.
Run an Inventory report to see hosts and their CVE status.
Chart the Vulnerable CVE(s) and get a visual breakdown.
Summary
In this tutorial you have learned how to use the reporting and inventory features of CFEngine Enterprise to discover and report on affected systems before and after remediation efforts.
Create, Modify, and Delete Files
Prerequisites
- Read the tutorial Tutorial for Running Examples
- Ensure you have read and understand the section on how to make an example stand alone
- Ensure you have read the note at the end of that section regarding modification of the body common control to the following:
body common control {
inputs => {
"libraries/cfengine_stdlib.cf",
};
}
Note: This change is not necessary for supporting each of the examples in this tutorial. It will be included only in those examples that require it.
List Files
Note: The following workflow assumes the directory /home/user already exists. If it does not either create the directory or adjust the example to a path of your choosing.
Create a file /var/cfengine/masterfiles/file_test.cf that includes the following text:
bundle agent list_file { vars: "ls" slist => lsdir("/home/user","test_plain.txt","true"); reports: "ls: $(ls)"; }
Run the following command to remove any existing test file at the location we wish to use for testing this example:
rm /home/user/test_plain.txt
Test to ensure there is no file /home/user/test_plain.txt, using the following command (the expected result is that there should be no file listed at the location /home/user/test_plain.txt):
ls /home/user/test_plain.txt
Run the following command to instruct CFEngine to see if the file exists (the expected result is that no report will be generated (because the file does not exist):
/var/cfengine/bin/cf-agent --no-lock --file /var/cfengine/masterfiles/file_test.cf --bundlesequence list_file
Create a file for testing the example, using the following command:
touch /home/user/test_plain.txt
Run the following command to instruct CFEngine to search for the file (the expected result is that a report will be generated, because the file exists):
/var/cfengine/bin/cf-agent --no-lock --file /var/cfengine/masterfiles/file_test.cf --bundlesequence list_file
Double check the file exists, using the following command (the expected result is that there will be a file listed at the location /home/user/test_plain.txt):
ls /home/user/test_plain.txt
Run the following command to remove the file:
rm /home/user/test_plain.txt
Create a File
bundle agent testbundle
{
files:
"/home/user/test_plain.txt"
perms => system,
create => "true";
}
bundle agent list_file
{
vars:
"ls" slist => lsdir("/home/user","test_plain.txt","true");
reports:
"ls: $(ls)";
}
bundle agent list_file_2
{
vars:
"ls" slist => lsdir("/home/user","test_plain.txt","true");
reports:
"ls: $(ls)";
}
body perms system
{
mode => "0640";
}
ls /home/user/test_plain.txt
/var/cfengine/bin/cf-agent --no-lock --file ./file_test.cf --bundlesequence list_file,testbundle,list_file_2
/var/cfengine/bin/cf-agent --no-lock --file ./file_test.cf --bundlesequence list_file,list_file_2
ls /home/user/test_plain.txt
rm /home/user/test_plain.txt
Delete a File
body common control {
inputs => {
"libraries/cfengine_stdlib.cf",
};
}
bundle agent testbundle
{
files:
"/home/user/test_plain.txt"
perms => system,
create => "true";
}
bundle agent test_delete
{
files:
"/home/user/test_plain.txt"
delete => tidy;
}
bundle agent list_file
{
vars:
"ls" slist => lsdir("/home/user","test_plain.txt","true");
reports:
"ls: $(ls)";
}
bundle agent list_file_2
{
vars:
"ls" slist => lsdir("/home/user","test_plain.txt","true");
reports:
"ls: $(ls)";
}
body perms system
{
mode => "0640";
}
rm /home/user/test_plain.txt
ls /home/user/test_plain.txt
/var/cfengine/bin/cf-agent --no-lock --file ./file_test.cf --bundlesequence list_file,testbundle,list_file_2
/var/cfengine/bin/cf-agent --no-lock --file ./file_test.cf --bundlesequence list_file,list_file_2
/var/cfengine/bin/cf-agent --no-lock --file ./file_test.cf --bundlesequence list_file,test_delete,list_file_2
ls /home/user/test_plain.txt
rm /home/user/test_plain.txt
(last command will throw an error because the file doesn't exist!)
Modify a File
rm /home/user/test_plain.txt
ls /home/user/test_plain.txt
/var/cfengine/bin/cf-agent --no-lock --file ./file_test.cf --bundlesequence list_file,testbundle,list_file_2
/var/cfengine/bin/cf-agent --no-lock --file ./file_test.cf --bundlesequence list_file,list_file_2
body common control {
inputs => {
"libraries/cfengine_stdlib.cf",
};
}
bundle agent testbundle
{
files:
"/home/user/test_plain.txt"
perms => system,
create => "true";
}
bundle agent test_delete
{
files:
"/home/user/test_plain.txt"
delete => tidy;
}
bundle agent list_file
{
vars:
"ls" slist => lsdir("/home/user","test_plain.txt","true");
reports:
"ls: $(ls)";
}
bundle agent list_file_2
{
vars:
"ls" slist => lsdir("/home/user","test_plain.txt","true");
reports:
"ls: $(ls)";
}
# Finds the file, if exists calls bundle to edit line
bundle agent outer_bundle_1
{
files:
"/home/user/test_plain.txt"
create => "false",
edit_line => inner_bundle_1;
}
# Finds the file, if exists calls bundle to edit line
bundle agent outer_bundle_2
{
files:
"/home/user/test_plain.txt"
create => "false",
edit_line => inner_bundle_2;
}
# Inserts lines
bundle edit_line inner_bundle_1
{
vars:
"msg" string => "Helloz to World!";
insert_lines:
"$(msg)";
}
# Replaces lines
bundle edit_line inner_bundle_2
{
replace_patterns:
"Helloz to World!"
replace_with => hello_world;
}
body replace_with hello_world
{
replace_value => "Hello World";
occurrences => "all";
}
body perms system
{
mode => "0640";
}
/var/cfengine/bin/cf-agent --no-lock --file ./file_test.cf --bundlesequence list_file,test_delete,list_file_2
ls /home/user/test_plain.txt
rm /home/user/test_plain.txt
Copy a File and Edit its Text
body common control {
inputs => {
"libraries/cfengine_stdlib.cf",
};
}
bundle agent testbundle
{
files:
"/home/ichien/test_plain.txt"
perms => system,
create => "true";
reports:
"test_plain.txt has been created";
}
bundle agent test_delete
{
files:
"/home/ichien/test_plain.txt"
delete => tidy;
}
bundle agent do_files_exist
{
vars:
"mylist" slist => { "/home/ichien/test_plain.txt", "/home/ichien/test_plain_2.txt" };
classes:
"exists" expression => filesexist("@(mylist)");
reports:
exists::
"test_plain.txt and test_plain_2.txt files exist";
!exists::
"test_plain.txt and test_plain_2.txt files do not exist";
}
bundle agent do_files_exist_2
{
vars:
"mylist" slist => { "/home/ichien/test_plain.txt", "/home/ichien/test_plain_2.txt" };
classes:
"exists" expression => filesexist("@(mylist)");
reports:
exists::
"test_plain.txt and test_plain_2.txt files both exist";
!exists::
"test_plain.txt and test_plain_2.txt files do not exist";
}
bundle agent list_file_1
{
vars:
"ls1" slist => lsdir("/home/ichien","test_plain.txt","true");
"ls2" slist => lsdir("/home/ichien","test_plain_2.txt","true");
"file_content_1" string => readfile( "/home/ichien/test_plain.txt" , "33" );
"file_content_2" string => readfile( "/home/ichien/test_plain_2.txt" , "33" );
reports:
#"ls1: $(ls1)";
#"ls2: $(ls2)";
"Contents of /home/ichien/test_plain.txt = $(file_content_1)";
"Contents of /home/ichien/test_plain_2.txt = $(file_content_2)";
}
bundle agent list_file_2
{
vars:
"ls1" slist => lsdir("/home/ichien","test_plain.txt","true");
"ls2" slist => lsdir("/home/ichien","test_plain_2.txt","true");
"file_content_1" string => readfile( "/home/ichien/test_plain.txt" , "33" );
"file_content_2" string => readfile( "/home/ichien/test_plain_2.txt" , "33" );
reports:
#"ls1: $(ls1)";
#"ls2: $(ls2)";
"Contents of /home/ichien/test_plain.txt = $(file_content_1)";
"Contents of /home/ichien/test_plain_2.txt = $(file_content_2)";
}
bundle agent outer_bundle_1
{
files:
"/home/ichien/test_plain.txt"
create => "false",
edit_line => inner_bundle_1;
}
# Copies file
bundle agent copy_a_file
{
files:
"/home/ichien/test_plain_2.txt"
copy_from => local_cp("/home/ichien/test_plain.txt");
reports:
"test_plain.txt has been copied to test_plain_2.txt";
}
bundle agent outer_bundle_2
{
files:
"/home/ichien/test_plain_2.txt"
create => "false",
edit_line => inner_bundle_2;
}
bundle edit_line inner_bundle_1
{
vars:
"msg" string => "Helloz to World!";
insert_lines:
"$(msg)";
reports:
"inserted $(msg) into test_plain.txt";
}
bundle edit_line inner_bundle_2
{
replace_patterns:
"Helloz to World!"
replace_with => hello_world;
reports:
"Text in test_plain_2.txt has been replaced";
}
body replace_with hello_world
{
replace_value => "Hello World";
occurrences => "all";
}
body perms system
{
mode => "0640";
}
/var/cfengine/bin/cf-agent --no-lock --file ./file_test.cf --bundlesequence test_delete,do_files_exist,testbundle,outer_bundle_1,copy_a_file,do_files_exist_2,list_file_1,outer_bundle_2,list_file_2
Tags for variables, classes, and bundles
Introduction
meta tags can be attached to any promise type using the meta
attribute.
These tags are useful for cross-referencing related promises. bundles
, vars
and classes
can be identified and leveraged in different ways within policy
using these tags.
Problem statement
We'd like to apply tags to variables and classes for many purposes, from stating their provenance (whence they came, why they exist, and how they can be used) to filtering them based on tags.
We'd also like to be able to include all the files in a directory and then run all the discovered bundles if they are tagged appropriately.
Syntax
Tagging variables and classes is easy with the meta
attribute. Here's an
example that sets the inventory
tag on a variable and names the attribute that
it represents. This one is actually built into the standard
MPF inventory policy,
so it's available out of the box in either Community or Enterprise.
bundle agent cfe_autorun_inventory_listening_ports
{
vars:
"ports" -> { "ENT-150" }
slist => sort( "mon.listening_ports", "int"),
meta => { "inventory", "attribute_name=Ports listening" },
if => some("[0-9]+", "mon.listening_ports"),
comment => "We only want to inventory the listening ports if we have
values that make sense.";
}
In the Enterprise Mission Portal, you can then make a report for "Ports listening" across all your machines. For more details, see Enterprise Reporting
Class tags work exactly the same way, you just apply them to a
classes
promise with the meta
attribute.
Tagging bundles is different because you have to use the meta
promise type (different from the meta
attribute).
An example is easiest:
bundle agent run_deprecated
{
meta:
"tags" slist => { "deprecated" };
}
This declares an agent bundle with a single tag.
Functions
Several new functions exist to give you access to variable and class tags, and to find classes and variables with tags.
classesmatching
: this used to be somewhat available with theallclasses.txt
file. You can now call a function to get all the defined classes, optionally filtering by name and tags. See classesmatchinggetvariablemetatags
: get the tags of a variable as an slist. See getvariablemetatagsvariablesmatching
: just likeclassesmatching
but for variables. See variablesmatchingvariablesmatching_as_data
: likevariablesmatching
but the matching variables and values are returned as a merged data container. See variablesmatching_as_datagetclassmetatags
: get the tags of a class as an slist. See getclassmetatagsbundlesmatching
: find the bundles matching some tags. See bundlesmatching (the example shows how you'd find adeprecated
bundle likerun_deprecated
earlier).
Module protocol
The module protocol has been extended to support tags. You set the tags on a line and they persist for every subsequent variable or class.
^meta=inventory
+x
=a=100
^meta=report,attribute_name=My vars
+y
=n=100
This will create class x
and variable a
with tag inventory
.
Then it will create class y
and variable b
with tags report
and
attribute_name=My vars
.
Enterprise Reporting with tags
In CFEngine Enterprise, you can build reports based on tagged variables and classes.
Please see Enterprise Reporting for a full tutorial, including troubleshooting possible errors. In short, this is an extremely easy way to categorize various data accessible to the agent.
Dynamic bundlesequence
Dynamic bundlesequences are extremely easy. First you find all the bundles whos name matches a regular expression and N tags.
vars:
"bundles" slist => bundlesmatching("regex", "tag1", "tag2", ...);
Then every bundle matching the regular expression regex
and all
the tags will be found and run.
methods:
"run $(bundles)" usebundle => $(bundles);
Note that the discovered bundle names will have the namespace prefix,
e.g. default:mybundle
. The regular expression has to match that. So
mybundle
as the regular expression would not work. See
bundlesmatching
for another detailed example.
In fact we found this so useful we implemented services autorun in the masterfiles policy framework.
There is only one thing to beware. All the bundles have to have the same number of arguments (0 in the case shown). Otherwise you will get a runtime error and CFEngine will abort. We recommend only using 0-argument bundles in a dynamic sequence to reduce this risk.
Summary
Tagging variables and classes and bundles in CFEngine is easy and allows more dynamic behavior than ever before. Try it out and see for yourself how it will change the way you use and think about system configuration policy and CFEngine.
Masterfiles Policy Framework Upgrade
Introduction
Upgrading the Masterfiles Policy Framework (MPF) is an optional but highly recommended first step when upgrading CFEngine.
Upgrading the MPF is not an exact process as the details highly depend on the
specifics of the changes made to the default policy. This tutorial leverages
git
and shows an example of upgrading a simple policy set based on 3.6.7 to
3.7.4 and can be used as a reference for upgrading your own policy sets.
Prepare a Git clone of your working masterfiles
If you are not using Git and instead editing directly in
$(sys.workdir/masterfiles)
you can simply copy your masterfiles into a new
directory and initalize a new Git repository.
If you're using Git already simply clone your repository and skip to the next step.
[root@hub MPF_upgrade]# rsync -a /var/cfengine/masterfiles/ MPF_upgrade/
Then initialize the new Git repository and add all the files to it.
[root@hub ~]# cd MPF_upgrade/
[root@hub MPF_upgrade]# git init
Initialized empty Git repository in /root/MPF_upgrade/.git/
[root@hub MPF_upgrade]# git add -A
[root@hub MPF_upgrade]# git commit -m "Before Upgrade"
[master (root-commit) 108c210] Before Upgrade
78 files changed, 19980 insertions(+)
create mode 100644 CUSTOM/policy1.cf
create mode 100644 cf_promises_release_id
create mode 100644 cf_promises_validated
create mode 100644 cfe_internal/CFE_cfengine.cf
create mode 100644 cfe_internal/CFE_hub_specific.cf
create mode 100644 cfe_internal/CFE_knowledge.cf
create mode 100644 cfe_internal/cfengine_processes.cf
create mode 100644 cfe_internal/ha/ha.cf
create mode 100644 cfe_internal/ha/ha_def.cf
create mode 100644 cfe_internal/host_info_report.cf
create mode 100644 controls/3.4/cf_serverd.cf
create mode 100644 controls/cf_agent.cf
create mode 100644 controls/cf_execd.cf
create mode 100644 controls/cf_hub.cf
create mode 100644 controls/cf_monitord.cf
create mode 100644 controls/cf_runagent.cf
create mode 100644 controls/cf_serverd.cf
create mode 100644 def.cf
create mode 100644 inventory/any.cf
create mode 100644 inventory/debian.cf
create mode 100644 inventory/generic.cf
create mode 100644 inventory/linux.cf
create mode 100644 inventory/lsb.cf
create mode 100644 inventory/macos.cf
create mode 100644 inventory/os.cf
create mode 100644 inventory/redhat.cf
create mode 100644 inventory/suse.cf
create mode 100644 inventory/windows.cf
create mode 100644 lib/3.5/bundles.cf
create mode 100644 lib/3.5/cfe_internal.cf
create mode 100644 lib/3.5/commands.cf
create mode 100644 lib/3.5/common.cf
create mode 100644 lib/3.5/databases.cf
create mode 100644 lib/3.5/feature.cf
create mode 100644 lib/3.5/files.cf
create mode 100644 lib/3.5/guest_environments.cf
create mode 100644 lib/3.5/monitor.cf
create mode 100644 lib/3.5/packages.cf
create mode 100644 lib/3.5/paths.cf
create mode 100644 lib/3.5/processes.cf
create mode 100644 lib/3.5/reports.cf
create mode 100644 lib/3.5/services.cf
create mode 100644 lib/3.5/storage.cf
create mode 100644 lib/3.6/bundles.cf
create mode 100644 lib/3.6/cfe_internal.cf
create mode 100644 lib/3.6/cfengine_enterprise_hub_ha.cf
create mode 100644 lib/3.6/commands.cf
create mode 100644 lib/3.6/common.cf
create mode 100644 lib/3.6/databases.cf
create mode 100644 lib/3.6/edit_xml.cf
create mode 100644 lib/3.6/examples.cf
create mode 100644 lib/3.6/feature.cf
create mode 100644 lib/3.6/files.cf
create mode 100644 lib/3.6/guest_environments.cf
create mode 100644 lib/3.6/monitor.cf
create mode 100644 lib/3.6/packages.cf
create mode 100644 lib/3.6/paths.cf
create mode 100644 lib/3.6/processes.cf
create mode 100644 lib/3.6/reports.cf
create mode 100644 lib/3.6/services.cf
create mode 100644 lib/3.6/stdlib.cf
create mode 100644 lib/3.6/storage.cf
create mode 100644 lib/3.6/users.cf
create mode 100644 lib/3.6/vcs.cf
create mode 100644 promises.cf
create mode 100644 services/autorun.cf
create mode 100644 services/autorun/custom_policy2.cf
create mode 100644 services/autorun/hello.cf
create mode 100644 services/file_change.cf
create mode 100644 sketches/meta/api-runfile.cf
create mode 100644 templates/host_info_report.mustache
create mode 100644 update.cf
create mode 100644 update/cfe_internal_dc_workflow.cf
create mode 100644 update/cfe_internal_local_git_remote.cf
create mode 100644 update/cfe_internal_update_from_repository.cf
create mode 100644 update/update_bins.cf
create mode 100644 update/update_policy.cf
create mode 100644 update/update_processes.cf
[root@hub MPF_upgrade]# git status
# On branch master
nothing to commit, working directory clean
Now we have a Git repository that we can start merging in the changes from upstream.
Merge the upstream changes to the MPF into your policy
Remove everything except the .git directory.
By first removing everything we will easily be able so see which files are new, changed, moved or removed upstream.
[root@hub MPF_upgrade]# rm -rf *
[root@hub MPF_upgrade]# git status
On branch master
Changes not staged for commit:
(use "git add/rm <file>..." to update what will be committed)
(use "git checkout -- <file>..." to discard changes in working directory)
deleted: CUSTOM/policy1.cf
deleted: cf_promises_release_id
deleted: cf_promises_validated
deleted: cfe_internal/CFE_cfengine.cf
deleted: cfe_internal/CFE_hub_specific.cf
deleted: cfe_internal/CFE_knowledge.cf
deleted: cfe_internal/cfengine_processes.cf
deleted: cfe_internal/ha/ha.cf
deleted: cfe_internal/ha/ha_def.cf
deleted: cfe_internal/host_info_report.cf
deleted: controls/3.4/cf_serverd.cf
deleted: controls/cf_agent.cf
deleted: controls/cf_execd.cf
deleted: controls/cf_hub.cf
deleted: controls/cf_monitord.cf
deleted: controls/cf_runagent.cf
deleted: controls/cf_serverd.cf
deleted: def.cf
deleted: inventory/any.cf
deleted: inventory/debian.cf
deleted: inventory/generic.cf
deleted: inventory/linux.cf
deleted: inventory/lsb.cf
deleted: inventory/macos.cf
deleted: inventory/os.cf
deleted: inventory/redhat.cf
deleted: inventory/suse.cf
deleted: inventory/windows.cf
deleted: lib/3.5/bundles.cf
deleted: lib/3.5/cfe_internal.cf
deleted: lib/3.5/commands.cf
deleted: lib/3.5/common.cf
deleted: lib/3.5/databases.cf
deleted: lib/3.5/feature.cf
deleted: lib/3.5/files.cf
deleted: lib/3.5/guest_environments.cf
deleted: lib/3.5/monitor.cf
deleted: lib/3.5/packages.cf
deleted: lib/3.5/paths.cf
deleted: lib/3.5/processes.cf
deleted: lib/3.5/reports.cf
deleted: lib/3.5/services.cf
deleted: lib/3.5/storage.cf
deleted: lib/3.6/bundles.cf
deleted: lib/3.6/cfe_internal.cf
deleted: lib/3.6/cfengine_enterprise_hub_ha.cf
deleted: lib/3.6/commands.cf
deleted: lib/3.6/common.cf
deleted: lib/3.6/databases.cf
deleted: lib/3.6/edit_xml.cf
deleted: lib/3.6/examples.cf
deleted: lib/3.6/feature.cf
deleted: lib/3.6/files.cf
deleted: lib/3.6/guest_environments.cf
deleted: lib/3.6/monitor.cf
deleted: lib/3.6/packages.cf
deleted: lib/3.6/paths.cf
deleted: lib/3.6/processes.cf
deleted: lib/3.6/reports.cf
deleted: lib/3.6/services.cf
deleted: lib/3.6/stdlib.cf
deleted: lib/3.6/storage.cf
deleted: lib/3.6/users.cf
deleted: lib/3.6/vcs.cf
deleted: promises.cf
deleted: services/autorun.cf
deleted: services/autorun/custom_policy2.cf
deleted: services/autorun/hello.cf
deleted: services/file_change.cf
deleted: sketches/meta/api-runfile.cf
deleted: templates/host_info_report.mustache
deleted: update.cf
deleted: update/cfe_internal_dc_workflow.cf
deleted: update/cfe_internal_local_git_remote.cf
deleted: update/cfe_internal_update_from_repository.cf
deleted: update/update_bins.cf
deleted: update/update_policy.cf
deleted: update/update_processes.cf
no changes added to commit (use "git add" and/or "git commit -a")
Install the new MPF
The MPF can be obtained from
any community package (in
$(sys.workdir)/share/CoreBase/
),
enterprise hub package (in
$(sys.workdir)/share/NovaBase/
),
masterfiles source tarball (requires
./configure
and make install
),
installed masterfiles tarball (ready
for extraction),
or
directly from github.
We will install the MPF from source obtained directly from github.
Note: You will need automake
to install from source.
First clone the masterfiles repository for the version you are installing. And verify you have the correct tag checked out.
Note: Directly checking out a tag as in the example below is only supported in Git versions 1.7.9.5 and newer.
[root@hub MPF_upgrade]# cd ..
[root@hub ~]# git clone -b 3.7.4 https://github.com/cfengine/masterfiles
[root@hub ~]# cd masterfiles
[root@hub ~]# git describe
3.7.4
Note: For systems without python 3 easily available (such as centos 6) you can use the following to get around problems in autogen with 3rdparty/core/determine-version.py, which requires python3.
export EXPLICIT_VERSION=$(git describe)
Now we will install the masterfiles from upstream into the directory where we are doing the integration.
First we build and install masterfiles to a temporary location.
./autogen.sh
[root@hub masterfiles]# ./autogen.sh
configure.ac:31: installing `./config.guess'
configure.ac:31: installing `./config.sub'
configure.ac:34: installing `./install-sh'
configure.ac:34: installing `./missing'
checking build system type... x86_64-unknown-linux-gnu
checking host system type... x86_64-unknown-linux-gnu
checking target system type... x86_64-unknown-linux-gnu
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking for a thread-safe mkdir -p... /bin/mkdir -p
checking for gawk... gawk
checking whether make sets $(MAKE)... yes
checking how to create a ustar tar archive... gnutar
checking whether to disable maintainer-specific portions of Makefiles... yes
checking for a thread-safe mkdir -p... /bin/mkdir -p
checking for a BSD-compatible install... /usr/bin/install -c
Summary of options:
Core directory -> not set - tests are disabled
Enterprise directory -> not set - some tests are disabled
Install prefix -> /var/cfengine
configure: generating makefile targets
configure: creating ./config.status
config.status: creating Makefile
config.status: creating controls/3.5/update_def.cf
config.status: creating controls/3.6/update_def.cf
config.status: creating controls/3.7/update_def.cf
config.status: creating modules/packages/Makefile
config.status: creating promises.cf
config.status: creating tests/acceptance/Makefile
config.status: creating tests/unit/Makefile
DONE: Configuration done. Run "make install" to install CFEngine Masterfiles.
[root@hub masterfiles]# ./configure --prefix /tmp/masterfiles-3.7.4
checking build system type... x86_64-unknown-linux-gnu
checking host system type... x86_64-unknown-linux-gnu
checking target system type... x86_64-unknown-linux-gnu
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking for a thread-safe mkdir -p... /bin/mkdir -p
checking for gawk... gawk
checking whether make sets $(MAKE)... yes
checking how to create a ustar tar archive... gnutar
checking whether to disable maintainer-specific portions of Makefiles... yes
checking for a thread-safe mkdir -p... /bin/mkdir -p
checking for a BSD-compatible install... /usr/bin/install -c
Summary of options:
Core directory -> not set - tests are disabled
Enterprise directory -> not set - some tests are disabled
Install prefix -> /tmp/masterfiles-3.7.4
configure: generating makefile targets
configure: creating ./config.status
config.status: creating Makefile
config.status: creating controls/3.5/update_def.cf
config.status: creating controls/3.6/update_def.cf
config.status: creating controls/3.7/update_def.cf
config.status: creating modules/packages/Makefile
config.status: creating promises.cf
config.status: creating tests/acceptance/Makefile
config.status: creating tests/unit/Makefile
DONE: Configuration done. Run "make install" to install CFEngine Masterfiles.
Then after running make install
we move the installed masterfiles into our
integration directory.
[root@hub masterfiles]# mv /tmp/masterfiles-3.7.4/masterfiles/* ../MPF_upgrade
[root@hub masterfiles]# cd ../MPF_upgrade/
Merge differences
Now we can use git status
to see an overview of the changes to the
repository between our starting point and the new MPF.
[root@hub MPF_upgrade]# git status
On branch master
Changes not staged for commit:
(use "git add/rm <file>..." to update what will be committed)
(use "git checkout -- <file>..." to discard changes in working directory)
deleted: CUSTOM/policy1.cf
deleted: cf_promises_release_id
deleted: cf_promises_validated
modified: cfe_internal/CFE_cfengine.cf
deleted: cfe_internal/CFE_hub_specific.cf
deleted: cfe_internal/CFE_knowledge.cf
deleted: cfe_internal/cfengine_processes.cf
deleted: cfe_internal/ha/ha.cf
deleted: cfe_internal/ha/ha_def.cf
deleted: cfe_internal/host_info_report.cf
deleted: controls/3.4/cf_serverd.cf
deleted: controls/cf_agent.cf
deleted: controls/cf_execd.cf
deleted: controls/cf_hub.cf
deleted: controls/cf_monitord.cf
deleted: controls/cf_runagent.cf
deleted: controls/cf_serverd.cf
deleted: def.cf
modified: inventory/any.cf
modified: inventory/linux.cf
modified: inventory/lsb.cf
modified: lib/3.5/cfe_internal.cf
modified: lib/3.5/common.cf
modified: lib/3.5/files.cf
modified: lib/3.5/packages.cf
deleted: lib/3.5/reports.cf
modified: lib/3.6/cfe_internal.cf
modified: lib/3.6/common.cf
modified: lib/3.6/files.cf
modified: lib/3.6/packages.cf
deleted: lib/3.6/reports.cf
modified: lib/3.6/services.cf
modified: lib/3.6/stdlib.cf
modified: promises.cf
deleted: services/autorun.cf
deleted: services/autorun/custom_policy2.cf
deleted: services/file_change.cf
modified: sketches/meta/api-runfile.cf
modified: update.cf
deleted: update/cfe_internal_dc_workflow.cf
deleted: update/cfe_internal_local_git_remote.cf
deleted: update/cfe_internal_update_from_repository.cf
deleted: update/update_bins.cf
deleted: update/update_policy.cf
deleted: update/update_processes.cf
Untracked files:
(use "git add <file>..." to include in what will be committed)
cfe_internal/core/
cfe_internal/enterprise/
cfe_internal/update/
controls/3.5/
controls/3.6/
controls/3.7/
inventory/freebsd.cf
lib/3.6/autorun.cf
lib/3.6/cfe_internal_hub.cf
lib/3.7/
services/main.cf
no changes added to commit (use "git add" and/or "git commit -a")
All of the Untracked files are new additions from upstream so they should be safe to take.
[root@hub MPF_upgrade]# git add cfe_internal/core/ \
cfe_internal/enterprise/ \
cfe_internal/update/ \
controls/3.5/ \
controls/3.6/ \
controls/3.7/ \
inventory/freebsd.cf \
lib/3.6/autorun.cf \
lib/3.6/cfe_internal_hub.cf \
lib/3.7/ \
services/main.cf
We can run git status again to see the current overview:
[root@hub MPF_upgrade]# git status
On branch master
Changes to be committed:
(use "git reset HEAD <file>..." to unstage)
new file: cfe_internal/core/deprecated/cfengine_processes.cf
new file: cfe_internal/core/host_info_report.cf
new file: cfe_internal/core/limit_robot_agents.cf
new file: cfe_internal/core/log_rotation.cf
new file: cfe_internal/core/main.cf
new file: cfe_internal/enterprise/CFE_hub_specific.cf
new file: cfe_internal/enterprise/CFE_knowledge.cf
new file: cfe_internal/enterprise/file_change.cf
new file: cfe_internal/enterprise/ha/ha.cf
new file: cfe_internal/enterprise/ha/ha_def.cf
new file: cfe_internal/enterprise/ha/ha_update.cf
new file: cfe_internal/enterprise/main.cf
new file: cfe_internal/update/cfe_internal_dc_workflow.cf
new file: cfe_internal/update/cfe_internal_local_git_remote.cf
new file: cfe_internal/update/cfe_internal_update_from_repository.cf
new file: cfe_internal/update/update_bins.cf
new file: cfe_internal/update/update_policy.cf
new file: cfe_internal/update/update_processes.cf
new file: controls/3.5/cf_agent.cf
new file: controls/3.5/cf_execd.cf
new file: controls/3.5/cf_hub.cf
new file: controls/3.5/cf_monitord.cf
new file: controls/3.5/cf_runagent.cf
new file: controls/3.5/cf_serverd.cf
new file: controls/3.5/def.cf
new file: controls/3.5/def_inputs.cf
new file: controls/3.5/reports.cf
new file: controls/3.5/update_def.cf
new file: controls/3.5/update_def_inputs.cf
new file: controls/3.6/cf_agent.cf
new file: controls/3.6/cf_execd.cf
new file: controls/3.6/cf_hub.cf
new file: controls/3.6/cf_monitord.cf
new file: controls/3.6/cf_runagent.cf
new file: controls/3.6/cf_serverd.cf
new file: controls/3.6/def.cf
new file: controls/3.6/def_inputs.cf
new file: controls/3.6/reports.cf
new file: controls/3.6/update_def.cf
new file: controls/3.6/update_def_inputs.cf
new file: controls/3.7/cf_agent.cf
new file: controls/3.7/cf_execd.cf
new file: controls/3.7/cf_hub.cf
new file: controls/3.7/cf_monitord.cf
new file: controls/3.7/cf_runagent.cf
new file: controls/3.7/cf_serverd.cf
new file: controls/3.7/def.cf
new file: controls/3.7/def_inputs.cf
new file: controls/3.7/reports.cf
new file: controls/3.7/update_def.cf
new file: controls/3.7/update_def_inputs.cf
new file: inventory/freebsd.cf
new file: lib/3.6/autorun.cf
new file: lib/3.6/cfe_internal_hub.cf
new file: lib/3.7/autorun.cf
new file: lib/3.7/bundles.cf
new file: lib/3.7/cfe_internal.cf
new file: lib/3.7/cfe_internal_hub.cf
new file: lib/3.7/cfengine_enterprise_hub_ha.cf
new file: lib/3.7/commands.cf
new file: lib/3.7/common.cf
new file: lib/3.7/databases.cf
new file: lib/3.7/edit_xml.cf
new file: lib/3.7/examples.cf
new file: lib/3.7/feature.cf
new file: lib/3.7/files.cf
new file: lib/3.7/guest_environments.cf
new file: lib/3.7/monitor.cf
new file: lib/3.7/packages.cf
new file: lib/3.7/paths.cf
new file: lib/3.7/processes.cf
new file: lib/3.7/services.cf
new file: lib/3.7/stdlib.cf
new file: lib/3.7/storage.cf
new file: lib/3.7/users.cf
new file: lib/3.7/vcs.cf
new file: services/main.cf
Changes not staged for commit:
(use "git add/rm <file>..." to update what will be committed)
(use "git checkout -- <file>..." to discard changes in working directory)
deleted: CUSTOM/policy1.cf
deleted: cf_promises_release_id
deleted: cf_promises_validated
modified: cfe_internal/CFE_cfengine.cf
deleted: cfe_internal/CFE_hub_specific.cf
deleted: cfe_internal/CFE_knowledge.cf
deleted: cfe_internal/cfengine_processes.cf
deleted: cfe_internal/ha/ha.cf
deleted: cfe_internal/ha/ha_def.cf
deleted: cfe_internal/host_info_report.cf
deleted: controls/3.4/cf_serverd.cf
deleted: controls/cf_agent.cf
deleted: controls/cf_execd.cf
deleted: controls/cf_hub.cf
deleted: controls/cf_monitord.cf
deleted: controls/cf_runagent.cf
deleted: controls/cf_serverd.cf
deleted: def.cf
modified: inventory/any.cf
modified: inventory/linux.cf
modified: inventory/lsb.cf
modified: lib/3.5/cfe_internal.cf
modified: lib/3.5/common.cf
modified: lib/3.5/files.cf
modified: lib/3.5/packages.cf
deleted: lib/3.5/reports.cf
modified: lib/3.6/cfe_internal.cf
modified: lib/3.6/common.cf
modified: lib/3.6/files.cf
modified: lib/3.6/packages.cf
deleted: lib/3.6/reports.cf
modified: lib/3.6/services.cf
modified: lib/3.6/stdlib.cf
modified: promises.cf
deleted: services/autorun.cf
deleted: services/autorun/custom_policy2.cf
deleted: services/file_change.cf
modified: sketches/meta/api-runfile.cf
modified: update.cf
deleted: update/cfe_internal_dc_workflow.cf
deleted: update/cfe_internal_local_git_remote.cf
deleted: update/cfe_internal_update_from_repository.cf
deleted: update/update_bins.cf
deleted: update/update_policy.cf
deleted: update/update_processes.cf
Next we want to bring back any of our custom policy files. Keeping your
polices organized together helps to make this process easy. The custom policy
files in the example policy set are CUSTOM/policy1.cf
and
services/autorun/custom_policy2.cf
. Restore them with git checkout
.
[root@hub MPF_upgrade] git checkout CUSTOM/policy1.cf services/autorun/custom_policy2.cf
The files marked as modified in the git status
output are files that have
changed upstream.
[root@hub MPF_upgrade]# git status | grep modified
modified: cfe_internal/CFE_cfengine.cf
modified: inventory/any.cf
modified: inventory/linux.cf
modified: inventory/lsb.cf
modified: lib/3.5/cfe_internal.cf
modified: lib/3.5/common.cf
modified: lib/3.5/files.cf
modified: lib/3.5/packages.cf
modified: lib/3.6/cfe_internal.cf
modified: lib/3.6/common.cf
modified: lib/3.6/files.cf
modified: lib/3.6/packages.cf
modified: lib/3.6/services.cf
modified: lib/3.6/stdlib.cf
modified: promises.cf
modified: sketches/meta/api-runfile.cf
modified: update.cf
For any files that you have not modified (like those in lib) simply add them
to gits staging area with git add
. Carefully review and merge or
re-integrate your custom changes on top of the upstream files.
The remaining files in git status
marked as deleted are files that have
been moved or removed from upstream.
NOTE: It is uncommon for any files to be moved or deleted between patch releases (e.g. 3.7.1 -> 3.7.2).
[root@hub MPF_upgrade]# git status | grep deleted
deleted: cf_promises_release_id
deleted: cf_promises_validated
deleted: cfe_internal/CFE_hub_specific.cf
deleted: cfe_internal/CFE_knowledge.cf
deleted: cfe_internal/cfengine_processes.cf
deleted: cfe_internal/ha/ha.cf
deleted: cfe_internal/ha/ha_def.cf
deleted: cfe_internal/host_info_report.cf
deleted: controls/3.4/cf_serverd.cf
deleted: controls/cf_agent.cf
deleted: controls/cf_execd.cf
deleted: controls/cf_hub.cf
deleted: controls/cf_monitord.cf
deleted: controls/cf_runagent.cf
deleted: controls/cf_serverd.cf
deleted: def.cf
deleted: lib/3.5/reports.cf
deleted: lib/3.6/reports.cf
deleted: services/autorun.cf
deleted: services/file_change.cf
deleted: update/cfe_internal_dc_workflow.cf
deleted: update/cfe_internal_local_git_remote.cf
deleted: update/cfe_internal_update_from_repository.cf
deleted: update/update_bins.cf
deleted: update/update_policy.cf
deleted: update/update_processes.cf
It's a good idea to review these files as some of them might have contained
modifications, especially def.cf
and any files under controls
. Always keep
track of the modifications you make to any of the files that ship with the
MPF. Make sure that any necessary customization's to the deleted files are
carried through to their new locations.
Once the files are no longer needed you can git rm
them.
[root@hub MPF_upgrade]# git rm def.cf cf_promises_release_id cf_promises_validated cfe_internal/CFE_hub_specific.cf cfe_internal/CFE_knowledge.cf cfe_internal/cfengine_processes.cf cfe_internal/ha/ha.cf cfe_internal/ha/ha_def.cf cfe_internal/host_info_report.cf controls/3.4/cf_serverd.cf controls/cf_agent.cf controls/cf_execd.cf controls/cf_hub.cf controls/cf_monitord.cf controls/cf_runagent.cf controls/cf_serverd.cf lib/3.5/reports.cf lib/3.6/reports.cf services/autorun.cf services/file_change.cf update/cfe_internal_dc_workflow.cf update/cfe_internal_local_git_remote.cf update/cfe_internal_update_from_repository.cf update/update_bins.cf update/update_policy.cf update/update_processes.cf
rm 'def.cf'
rm 'cf_promises_release_id'
rm 'cf_promises_validated'
rm 'cfe_internal/CFE_hub_specific.cf'
rm 'cfe_internal/CFE_knowledge.cf'
rm 'cfe_internal/cfengine_processes.cf'
rm 'cfe_internal/ha/ha.cf'
rm 'cfe_internal/ha/ha_def.cf'
rm 'cfe_internal/host_info_report.cf'
rm 'controls/3.4/cf_serverd.cf'
rm 'controls/cf_agent.cf'
rm 'controls/cf_execd.cf'
rm 'controls/cf_hub.cf'
rm 'controls/cf_monitord.cf'
rm 'controls/cf_runagent.cf'
rm 'controls/cf_serverd.cf'
rm 'lib/3.5/reports.cf'
rm 'lib/3.6/reports.cf'
rm 'services/autorun.cf'
rm 'services/file_change.cf'
rm 'update/cfe_internal_dc_workflow.cf'
rm 'update/cfe_internal_local_git_remote.cf'
rm 'update/cfe_internal_update_from_repository.cf'
rm 'update/update_bins.cf'
rm 'update/update_policy.cf'
rm 'update/update_processes.cf'
Review git status
and make sure that the policy validates then commit your
changes.
[root@hub MPF_upgrade]# git status
On branch master
Changes to be committed:
(use "git reset HEAD <file>..." to unstage)
deleted: cf_promises_release_id
deleted: cf_promises_validated
modified: cfe_internal/CFE_cfengine.cf
renamed: cfe_internal/cfengine_processes.cf -> cfe_internal/core/deprecated/cfengine_processes.cf
renamed: cfe_internal/host_info_report.cf -> cfe_internal/core/host_info_report.cf
new file: cfe_internal/core/limit_robot_agents.cf
new file: cfe_internal/core/log_rotation.cf
new file: cfe_internal/core/main.cf
renamed: cfe_internal/CFE_hub_specific.cf -> cfe_internal/enterprise/CFE_hub_specific.cf
renamed: cfe_internal/CFE_knowledge.cf -> cfe_internal/enterprise/CFE_knowledge.cf
renamed: services/file_change.cf -> cfe_internal/enterprise/file_change.cf
new file: cfe_internal/enterprise/ha/ha.cf
renamed: cfe_internal/ha/ha_def.cf -> cfe_internal/enterprise/ha/ha_def.cf
new file: cfe_internal/enterprise/ha/ha_update.cf
new file: cfe_internal/enterprise/main.cf
deleted: cfe_internal/ha/ha.cf
renamed: update/cfe_internal_dc_workflow.cf -> cfe_internal/update/cfe_internal_dc_workflow.cf
renamed: update/cfe_internal_local_git_remote.cf -> cfe_internal/update/cfe_internal_local_git_remote.cf
new file: cfe_internal/update/cfe_internal_update_from_repository.cf
renamed: update/update_bins.cf -> cfe_internal/update/update_bins.cf
renamed: update/update_policy.cf -> cfe_internal/update/update_policy.cf
renamed: update/update_processes.cf -> cfe_internal/update/update_processes.cf
deleted: controls/3.4/cf_serverd.cf
renamed: controls/cf_agent.cf -> controls/3.5/cf_agent.cf
new file: controls/3.5/cf_execd.cf
renamed: controls/cf_hub.cf -> controls/3.5/cf_hub.cf
renamed: controls/cf_monitord.cf -> controls/3.5/cf_monitord.cf
renamed: controls/cf_runagent.cf -> controls/3.5/cf_runagent.cf
renamed: controls/cf_serverd.cf -> controls/3.5/cf_serverd.cf
renamed: def.cf -> controls/3.5/def.cf
new file: controls/3.5/def_inputs.cf
renamed: lib/3.5/reports.cf -> controls/3.5/reports.cf
renamed: update.cf -> controls/3.5/update_def.cf
new file: controls/3.5/update_def_inputs.cf
new file: controls/3.6/cf_agent.cf
new file: controls/3.6/cf_execd.cf
new file: controls/3.6/cf_hub.cf
new file: controls/3.6/cf_monitord.cf
new file: controls/3.6/cf_runagent.cf
new file: controls/3.6/cf_serverd.cf
new file: controls/3.6/def.cf
new file: controls/3.6/def_inputs.cf
renamed: lib/3.6/reports.cf -> controls/3.6/reports.cf
new file: controls/3.6/update_def.cf
new file: controls/3.6/update_def_inputs.cf
new file: controls/3.7/cf_agent.cf
new file: controls/3.7/cf_execd.cf
new file: controls/3.7/cf_hub.cf
new file: controls/3.7/cf_monitord.cf
new file: controls/3.7/cf_runagent.cf
new file: controls/3.7/cf_serverd.cf
new file: controls/3.7/def.cf
new file: controls/3.7/def_inputs.cf
new file: controls/3.7/reports.cf
new file: controls/3.7/update_def.cf
new file: controls/3.7/update_def_inputs.cf
deleted: controls/cf_execd.cf
modified: inventory/any.cf
new file: inventory/freebsd.cf
modified: inventory/linux.cf
modified: inventory/lsb.cf
modified: lib/3.5/cfe_internal.cf
modified: lib/3.5/common.cf
modified: lib/3.5/files.cf
modified: lib/3.5/packages.cf
renamed: services/autorun.cf -> lib/3.6/autorun.cf
modified: lib/3.6/cfe_internal.cf
renamed: lib/3.6/cfe_internal.cf -> lib/3.6/cfe_internal_hub.cf
modified: lib/3.6/common.cf
modified: lib/3.6/files.cf
modified: lib/3.6/packages.cf
modified: lib/3.6/services.cf
modified: lib/3.6/stdlib.cf
new file: lib/3.7/autorun.cf
new file: lib/3.7/bundles.cf
new file: lib/3.7/cfe_internal.cf
new file: lib/3.7/cfe_internal_hub.cf
new file: lib/3.7/cfengine_enterprise_hub_ha.cf
new file: lib/3.7/commands.cf
new file: lib/3.7/common.cf
new file: lib/3.7/databases.cf
new file: lib/3.7/edit_xml.cf
new file: lib/3.7/examples.cf
new file: lib/3.7/feature.cf
new file: lib/3.7/files.cf
new file: lib/3.7/guest_environments.cf
new file: lib/3.7/monitor.cf
new file: lib/3.7/packages.cf
new file: lib/3.7/paths.cf
new file: lib/3.7/processes.cf
new file: lib/3.7/services.cf
new file: lib/3.7/stdlib.cf
new file: lib/3.7/storage.cf
new file: lib/3.7/users.cf
new file: lib/3.7/vcs.cf
modified: promises.cf
new file: services/main.cf
modified: sketches/meta/api-runfile.cf
modified: update.cf
deleted: update/cfe_internal_update_from_repository.cf
[root@hub MPF_upgrade]# cf-promises -cf ./promises.cf
[root@hub MPF_upgrade]# cf-promises -cf ./update.cf
[root@hub MPF_upgrade]# git commit -m "After Policy Upgrade"
100 files changed, 12521 insertions(+), 1493 deletions(-)
delete mode 100644 cf_promises_release_id
delete mode 100644 cf_promises_validated
rewrite cfe_internal/CFE_cfengine.cf (88%)
rename cfe_internal/{ => core/deprecated}/cfengine_processes.cf (95%)
rename cfe_internal/{ => core}/host_info_report.cf (98%)
create mode 100644 cfe_internal/core/limit_robot_agents.cf
create mode 100644 cfe_internal/core/log_rotation.cf
create mode 100644 cfe_internal/core/main.cf
rename cfe_internal/{ => enterprise}/CFE_hub_specific.cf (85%)
rename cfe_internal/{ => enterprise}/CFE_knowledge.cf (100%)
rename {services => cfe_internal/enterprise}/file_change.cf (58%)
create mode 100644 cfe_internal/enterprise/ha/ha.cf
rename cfe_internal/{ => enterprise}/ha/ha_def.cf (54%)
create mode 100644 cfe_internal/enterprise/ha/ha_update.cf
create mode 100644 cfe_internal/enterprise/main.cf
delete mode 100644 cfe_internal/ha/ha.cf
rename {update => cfe_internal/update}/cfe_internal_dc_workflow.cf (100%)
rename {update => cfe_internal/update}/cfe_internal_local_git_remote.cf (100%)
create mode 100644 cfe_internal/update/cfe_internal_update_from_repository.cf
rename {update => cfe_internal/update}/update_bins.cf (97%)
rename {update => cfe_internal/update}/update_policy.cf (92%)
rename {update => cfe_internal/update}/update_processes.cf (92%)
delete mode 100644 controls/3.4/cf_serverd.cf
rename controls/{ => 3.5}/cf_agent.cf (80%)
create mode 100644 controls/3.5/cf_execd.cf
rename controls/{ => 3.5}/cf_hub.cf (100%)
rename controls/{ => 3.5}/cf_monitord.cf (100%)
rename controls/{ => 3.5}/cf_runagent.cf (100%)
rename controls/{ => 3.5}/cf_serverd.cf (87%)
rename def.cf => controls/3.5/def.cf (74%)
create mode 100644 controls/3.5/def_inputs.cf
rename {lib => controls}/3.5/reports.cf (80%)
rename update.cf => controls/3.5/update_def.cf (59%)
create mode 100644 controls/3.5/update_def_inputs.cf
create mode 100644 controls/3.6/cf_agent.cf
create mode 100644 controls/3.6/cf_execd.cf
create mode 100644 controls/3.6/cf_hub.cf
create mode 100644 controls/3.6/cf_monitord.cf
create mode 100644 controls/3.6/cf_runagent.cf
create mode 100644 controls/3.6/cf_serverd.cf
create mode 100644 controls/3.6/def.cf
create mode 100644 controls/3.6/def_inputs.cf
rename {lib => controls}/3.6/reports.cf (78%)
create mode 100644 controls/3.6/update_def.cf
create mode 100644 controls/3.6/update_def_inputs.cf
create mode 100644 controls/3.7/cf_agent.cf
create mode 100644 controls/3.7/cf_execd.cf
create mode 100644 controls/3.7/cf_hub.cf
create mode 100644 controls/3.7/cf_monitord.cf
create mode 100644 controls/3.7/cf_runagent.cf
create mode 100644 controls/3.7/cf_serverd.cf
create mode 100644 controls/3.7/def.cf
create mode 100644 controls/3.7/def_inputs.cf
create mode 100644 controls/3.7/reports.cf
create mode 100644 controls/3.7/update_def.cf
create mode 100644 controls/3.7/update_def_inputs.cf
delete mode 100644 controls/cf_execd.cf
create mode 100644 inventory/freebsd.cf
rename {services => lib/3.6}/autorun.cf (50%)
rewrite lib/3.6/cfe_internal.cf (67%)
rename lib/3.6/{cfe_internal.cf => cfe_internal_hub.cf} (77%)
create mode 100644 lib/3.7/autorun.cf
create mode 100644 lib/3.7/bundles.cf
create mode 100644 lib/3.7/cfe_internal.cf
create mode 100644 lib/3.7/cfe_internal_hub.cf
create mode 100644 lib/3.7/cfengine_enterprise_hub_ha.cf
create mode 100644 lib/3.7/commands.cf
create mode 100644 lib/3.7/common.cf
create mode 100644 lib/3.7/databases.cf
create mode 100644 lib/3.7/edit_xml.cf
create mode 100644 lib/3.7/examples.cf
create mode 100644 lib/3.7/feature.cf
create mode 100644 lib/3.7/files.cf
create mode 100644 lib/3.7/guest_environments.cf
create mode 100644 lib/3.7/monitor.cf
create mode 100644 lib/3.7/packages.cf
create mode 100644 lib/3.7/paths.cf
create mode 100644 lib/3.7/processes.cf
create mode 100644 lib/3.7/services.cf
create mode 100644 lib/3.7/stdlib.cf
create mode 100644 lib/3.7/storage.cf
create mode 100644 lib/3.7/users.cf
create mode 100644 lib/3.7/vcs.cf
create mode 100644 services/main.cf
rewrite update.cf (78%)
delete mode 100644 update/cfe_internal_update_from_repository.cf
Now your Masterfiles Policy Framework is upgraded and ready to be tested.
Custom Inventory
This tutorial will show you how to add custom inventory attributes that can be leveraged in policy and reported on in the CFEngine Enterprise Mission Portal. For a more detailed overview on how the inventory system works please reference CFEngine 3 inventory modules.
Overview
This tutorial provides instructions for the following:
Choose an Attribute to Inventory
Writing inventory policy is incredibly easy. Simply add the inventory
and
attribute_name=
tags to any variable or namespace scoped
classes.
In this tutorial we will add Owner
information into the inventory. In this
example we will use a simple shared flat file data source
/vagrant/inventory_owner.csv
.
On your hosts create /vagrant/inventory_owner.csv
with the following contnet:
hub, Operations Team <ops@cfengine.com>
host001, Development <dev@cfengine.com>
Note: I am using the [CFEngine Enterprise Vagrant Environment][Installing Enterprise Vagrant Environemt] and files located in the vagrant project directory are automatically available to all hosts.
Create and Deploy Inventory Policy
Now that each of your hosts has access to a data source that provides the Owner information we will write an inventory policy to report that information.
Create /var/cfengine/masterfiles/services/tutorials/inventory/owner.cf
with the
following content:
bundle common tutorials_inventory_owner
# @brief Inventory Owner information
# @description Inventory owner information from `/vagrant/inventory_owner.csv`.
{
classes:
"have_owner"
expression => isvariable("my_owner");
vars:
"data_source" string => "/vagrant/inventory_owner.csv";
"owners" data => data_readstringarray( $(data_source), "", ", ", 100, 512 );
"my_owner"
string => "$(owners[$(sys.uqhost)][0])",
meta => { "inventory", "attribute_name=Owner" },
comment => "We need to tag the owner information so that it is correctly
reported via the UI.";
reports:
inform_mode.have_owner::
"$(this.bundle): Discovered Owner='$(my_owner)'";
}
Note: For the simplicity of this tutorial we assume that masterfiles is not configured for policy updates from a Git repository. If it is, please add the policy to your repository and ensure it gets to its final destination as needed.
This policy will not be activated until it has been included in inputs. For simplicity we will be adding it to common control, but note that files can include other files through the use of inputs in file control.
Add 'services/tutorials/inventory/owner.cf' to inputs and
'tutorials_inventory_owner' to the bundlesequence in
common control found in /var/cfengine/masterfiles/promises.cf
as shown below.
body common control
{
bundlesequence => {
"inventory_control",
@(inventory.bundles),
tutorials_inventory_owner,
def,
cfe_internal_hub_vars,
# snipped for brevity
service_catalogue,
};
inputs => {
# File definition for global variables and classes
"def.cf",
# Inventory policy
@(inventory.inputs),
"services/tutorials/inventory/owner.cf",
# snipped for brevity
# List of services here
"services/init_msg.cf",
"services/file_change.cf",
};
}
When you finish its a good idea to run cf-promises
(syntax analyzer) to
ensure the policy validates. You can also perform a manual policy run and check
that the correct owner is discovered.
Syntax Check:
[root@hub ~]# cf-promises -f /var/cfengine/masterfiles/promises.cf
[root@hub ~]#
No output and return code 0 indicate the policy was successfully validated.
Manual Policy Run:
[root@hub ~]# cf-agent -KIf /var/cfengine/masterfiles/promises.cf -b tutorials_inventory_owner
2014-06-16T19:24:58+0000 info: Using command line specified bundlesequence
R: tutorials_inventory_owner: Discovered Owner='Operations Team <ops@cfengine.com>'
Here I have run the policy without locks (-K) in inform mode (-I), using a specific policy entry (-f) and activating only a specific bundle (-b). The inform output helps me confirm that my hubs owner is discovered from our CSV properly.
Run Reports
Once you have integrated the policy into promises.cf it will be distributed and run by all agents. Once the hub has had a chance to collect reports the 'Owner' attribute will be available to select as a Table column for Inventory Reports. Custom attributes appear under the 'User defined' section.
Note: It may take up to 15 minutes for your custom inventory attributes to be collected and made available for reporting.
You will see the host owner as shown in the following report.
File Comparison
- Add the policy contents (also can be downloaded from file_compare_test.cf) to a new file, such as /var/cfengine/masterfiles/file_test.cf.
Run the following commands as root on the command line:
export AOUT_BIN="a.out" export GCC_BIN="/usr/bin/gcc" export RM_BIN="/bin/rm" export WORK_DIR=$HOME export CFE_FILE1="test_plain_1.txt" export CFE_FILE2="test_plain_2.txt" /var/cfengine/bin/cf-agent /var/cfengine/masterfiles/file_test.cf --bundlesequence robot,global_vars,packages,create_aout_source_file,create_aout,test_delete,do_files_exist_1,create_file_1,outer_bundle_1,copy_a_file,do_files_exist_2,list_file_1,stat,outer_bundle_2,list_file_2
Here is the order in which bundles are called in the command line above (some other support bundles are contained within file_test.cf but are not included here):
- robot - demonstrates use of
reports
. - global_vars - sets up some global variables for later use.
- packages - installs packages that will be used later on.
- create_aout_source_file - creates a source file.
- create_aout - compiles the source file.
- test_delete - deletes a file.
- do_files_exist_1 - checks the existence of files.
- create_file_1 - creates a file.
- outer_bundle_1 - adds text to a file.
- copy_a_file - copies the file.
- do_files_exist_2 - checks the existence of both files.
- list_file_1 - shows the contents of each file.
- stat - uses the stat command and the aout application to compare modified times of both files.
- outer_bundle_2 - modifies the contents of the second file.
- list_file_2 - shows the contents of both files and uses CFEngine functionality to compare the modified time for each file.
robot
Demonstrates use of reports
, using an ascii art representation of the CFEngine robot.
global_vars
Sets up some global variables that are used frequently by other bundles.
bundle common global_vars
{
vars:
"gccexec" string => getenv("GCC_BIN",255);
"rmexec" string => getenv("RM_BIN",255);
"aoutbin" string => getenv("AOUT_BIN",255);
"workdir" string => getenv("WORK_DIR",255);
"aoutexec" string => "$(workdir)/$(aoutbin)";
"file1name" string => getenv("CFE_FILE1",255);
"file2name" string => getenv("CFE_FILE2",255);
"file1" string => "$(workdir)/$(file1name)";
"file2" string => "$(workdir)/$(file2name)";
classes:
"gclass" expression => "any";
}
packages
Ensures that the gcc package is installed, for later use by the create_aout bundle.
bundle agent packages
{
vars:
"match_package" slist => {
"gcc"
};
packages:
"$(match_package)"
package_policy => "add",
package_method => yum;
reports:
gclass::
"Package gcc installed";
"*********************************";
}
create_aout_source_file
Creates the c source file that will generate a binary application in create_aout.
bundle agent create_aout_source_file
{
# This bundle creates the source file that will be compiled in bundle agent create_aout.
# See that bunlde's comments for more information.
vars:
# An slist is used here instead of a straight forward string because it doesn't seem possible to create
# line endings using \n when using a string to insert text into a file.
"c" slist => {"#include <stdlib.h>","#include <stdio.h>","#include <sys/stat.h>","#include <string.h>","void main()","{char file1[255];strcpy(file1,\"$(global_vars.file1)\");char file2[255];strcpy(file2,\"$(global_vars.file2)\");struct stat time1;int i = lstat(file1, &time1);struct stat time2;int j = lstat(file2, &time2);if (time1.st_mtime < time2.st_mtime){printf(\"Newer\");}else{if(time1.st_mtim.tv_nsec < time2.st_mtim.tv_nsec){printf(\"Newer\");}else{printf(\"Not newer\");}}}"};
files:
"$(global_vars.workdir)/a.c"
perms => system,
create => "true",
edit_line => Insert("@(c)");
reports:
"The source file $(global_vars.workdir)/a.c has been created. It will be used to compile the binary a.out, which will provide more accurate file stats to compare two files than the built in CFEngine functionality for comparing file stats, including modification time. This information will be used to determine of the second of the two files being compared is newer or not.";
"*********************************";
}
create_aout
This bundle creates a binary application from the source in create_aout_source_file that uses the stat library to compare two files, determine if the modified times are different, nd whether the second file is newer than the first.
The difference between this application and using CFEngine's built in support for getting file stats is that normally the accuracy is only to the second of the modified file time but in order to better compare two files requires parts of a second as well. The stat library provides the extra support for retrieving the additional information required.
bundle agent create_aout
{
classes:
"doesfileacexist" expression => fileexists("$(global_vars.workdir)/a.c");
"doesaoutexist" expression => fileexists("$(global_vars.aoutbin)");
vars:
# Removes any previous binary
"rmaout" string => execresult("$(global_vars.rmexec) $(global_vars.aoutexec)","noshell");
doesfileacexist::
"compilestr" string => "$(global_vars.gccexec) $(global_vars.workdir)/a.c -o $(global_vars.aoutexec)";
"gccaout" string => execresult("$(compilestr)","noshell");
reports:
doesfileacexist::
"gcc output: $(gccaout)";
"Creating aout using $(compilestr)";
!doesfileacexist::
"Cannot compile a.out, $(global_vars.workdir)/a.c does not exist.";
doesaoutexist::
"The binary application aout has been compiled from the source in the create_aout_source_file bundle. It uses the stat library to compare two files, determine if the modified times are different, and whether the second file is newer than the first. The difference between this application and using CFEngine's built in support for getting file stats (e.g. filestat, isnewerthan), which provides file modification time accurate to a second. However, in order to better compare two files might sometimes require parts of a second as well. The stat library provides the extra support for retrieving the additional information required to get better accuracy (down to parts of a second), and is utilized by the binary application a.out that is compiled within the create_aout bundle.";
"*********************************";
}
test_delete
Deletes any previous copy of the test files used in the example.
bundle agent test_delete
{
files:
"$(global_vars.file1)"
delete => tidy;
}
do_files_exist_1
Verifies whether the test files exist or not.
bundle agent do_files_exist_1
{
classes:
"doesfile1exist" expression => fileexists("$(global_vars.file1)");
"doesfile2exist" expression => fileexists("$(global_vars.file2)");
methods:
doesfile1exist::
"any" usebundle => delete_file("$(global_vars.file1)");
doesfile2exist::
"any" usebundle => delete_file("$(global_vars.file2)");
reports:
!doesfile1exist::
"$(global_vars.file1) does not exist.";
doesfile1exist::
"$(global_vars.file1) did exist. Call to delete it was made.";
!doesfile2exist::
"$(global_vars.file2) does not exist.";
doesfile2exist::
"$(global_vars.file2) did exist. Call to delete it was made.";
}
create_file_1
Creates the first test file, as an empty file.
bundle agent create_file_1
{
files:
"$(global_vars.file1)"
perms => system,
create => "true";
reports:
"$(global_vars.file1) has been created";
}
outer_bundle_1
Adds some text to the first test file.
bundle agent outer_bundle_1
{
files:
"$(global_vars.file1)"
create => "false",
edit_line => inner_bundle_1;
}
copy_a_file
Makes a copy of the test file.
bundle agent copy_a_file
{
files:
"$(global_vars.file2)"
copy_from => local_cp("$(global_vars.file1)");
reports:
"$(global_vars.file1) has been copied to $(global_vars.file2)";
}
do_files_exist_2
Verifies that both test files exist.
bundle agent do_files_exist_2
{
methods:
"any" usebundle => does_file_exist($(global_vars.file1));
"any" usebundle => does_file_exist($(global_vars.file2));
}
list_file_1
Reports the contents of each test file.
bundle agent list_file_1
{
methods:
"any" usebundle => file_content($(global_vars.file1));
"any" usebundle => file_content($(global_vars.file2));
reports:
"*********************************";
}
exec_aout
bundle agent exec_aout
{
classes:
"doesaoutexist" expression => fileexists("$(global_vars.aoutbin)");
vars:
doesaoutexist::
"aout" string => execresult("$(global_vars.aoutexec)","noshell");
reports:
doesaoutexist::
"*********************************";
"$(global_vars.aoutbin) determined that $(global_vars.file2) is $(aout) than $(global_vars.file1)";
"*********************************";
!doesaoutexist::
"Executable $(global_vars.aoutbin) does not exist.";
}
stat
Compares the modified time of each test file using the binary application compiled in create_aout to see if it is newer.
bundle agent stat
{
classes:
"doesfile1exist" expression => fileexists("$(global_vars.file1)");
"doesfile2exist" expression => fileexists("$(global_vars.file2)");
vars:
doesfile1exist::
"file1" string => "$(global_vars.file1)";
"file2" string => "$(global_vars.file2)";
"file1_stat" string => execresult("/usr/bin/stat -c \"%y\" $(file1)","noshell");
"file1_split1" slist => string_split($(file1_stat)," ",3);
"file1_split2" string => nth("file1_split1",1);
"file1_split3" slist => string_split($(file1_split2),"\.",3);
"file1_split4" string => nth("file1_split3",1);
"file2_stat" string => execresult("/usr/bin/stat -c \"%y\" $(file2)","noshell");
"file2_split1" slist => string_split($(file2_stat)," ",3);
"file2_split2" string => nth("file2_split1",1);
"file2_split3" slist => string_split($(file2_split2),"\.",3);
"file2_split4" string => nth("file2_split3",1);
methods:
"any" usebundle => exec_aout();
reports:
doesfile1exist::
"Parts of a second extracted extracted from stat for $(file1): $(file1_split4). Full stat output for $(file1): $(file1_stat)";
"Parts of a second extracted extracted from stat for $(file2): $(file2_split4). Full stat output for $(file2): $(file2_stat)";
"Using the binary Linux application stat to compare two files can help determine if the modified times between two files are different. The difference between the stat application using its additional flags and using CFEngine's built in support for getting and comparing file stats (e.g. filestat, isnewerthan) is that normally the accuracy is only to the second of the file's modified time. In order to better compare two files requires parts of a second as well, which the stat command can provide with some additional flags. Unfortunately the information must be extracted from the middle of a string, which is what the stat bundle accomplishes using the string_split and nth functions.";
"*********************************";
!doesfile1exist::
"stat: $(global_vars.file1) and probably $(global_vars.file2) do not exist.";
}
outer_bundle_2
Modifies the text in the second file.
bundle agent outer_bundle_2
{
files:
"$(global_vars.file2)"
create => "false",
edit_line => inner_bundle_2;
}
list_file_2
Uses filestat
and isnewerthan
to compare the two test files to see if the second one is newer. Sometimes the modifications already performed, such as copy and modifying text, happen too quickly and filestat and isnewerthan may both report that the second test file is not newer than the first, while the more accurate stat based checks in the stat bundle (see step 12) will recognize the difference.
bundle agent list_file_2
{
methods:
"any" usebundle => file_content($(global_vars.file1));
"any" usebundle => file_content($(global_vars.file2));
classes:
"ok" expression => isgreaterthan(filestat("$(global_vars.file2)","mtime"),filestat("$(global_vars.file1)","mtime"));
"newer" expression => isnewerthan("$(global_vars.file2)","$(global_vars.file1)");
reports:
"*********************************";
ok::
"Using isgreaterthan+filestat determined that $(global_vars.file2) was modified later than $(global_vars.file1).";
!ok::
"Using isgreaterthan+filestat determined that $(global_vars.file2) was not modified later than $(global_vars.file1).";
newer::
"Using isnewerthan determined that $(global_vars.file2) was modified later than $(global_vars.file1).";
!newer::
"Using isnewerthan determined that $(global_vars.file2) was not modified later than $(global_vars.file1).";
}
Full Policy
body common control {
inputs => {
"libraries/cfengine_stdlib.cf",
};
}
bundle agent robot
{
reports:
" 77777777777";
" 77777777777777";
" 777 7777 777";
" 7777777777777";
" 777777777777";
" 777 7777 77";
" ";
" ZZZZ ZZZ ZZZZ ZZZ ZZZZ";
" ZZZZZ ZZZZZZZZZZZZZZ ZZZZZ ";
" ZZZZZZZ ZZZZZZZZZZZZZ ZZZZZZZ";
" ZZZZ ------------- ZZZZZZ";
" ZZZZZ !CFENGINE! ZZZZZ";
" ZZZZ ------------- ZZZZZ";
" ZZZZZ ZZZZZZZZZZZZZZ ZZZZZ";
" ZZZ ZZZZZZZZZZZZZ ZZZ";
" ZZZZZ ZZZZZZZZZZZZZ ZZZZZ";
" ..?ZZZ+,,,,, ZZZZZZZZZZZZZZ ZZZZZ";
" ...ZZZZ~ ,:: ZZZZZZZZZZZZZ ZZZZ";
" ..,ZZZZZ,:::::: ZZZZZ";
" ZZZ ZZZ";
" ~ ===+";
" ZZZZZZZZZZZZZI??";
" ZZZZZZZZZZZZZ$???";
" 7Z$+ ZZ ZZZ???II";
" ZZZZZ+ ZZZZZIIII";
" ZZZZZ ZZZZZ III77";
" +++ +$ZZ??? ZZZ";
" +++??ZZZZZIIIIZZZZZ";
" ????ZZZZZIIIIZZZZZ";
" ??IIZZZZ 7777ZZZ";
" IIZZZZZ 77ZZZZZ";
" I$ZZZZ $ZZZZ";
}
bundle common global_vars
{
vars:
"gccexec" string => getenv("GCC_BIN",255);
"rmexec" string => getenv("RM_BIN",255);
"aoutbin" string => getenv("AOUT_BIN",255);
"workdir" string => getenv("WORK_DIR",255);
"aoutexec" string => "$(workdir)/$(aoutbin)";
"file1name" string => getenv("CFE_FILE1",255);
"file2name" string => getenv("CFE_FILE2",255);
"file1" string => "$(workdir)/$(file1name)";
"file2" string => "$(workdir)/$(file2name)";
classes:
"gclass" expression => "any";
}
bundle agent packages
{
vars:
"match_package" slist => {
"gcc"
};
packages:
"$(match_package)"
package_policy => "add",
package_method => yum;
reports:
gclass::
"Package gcc installed";
"*********************************";
}
bundle agent create_aout_source_file
{
# This bundle creates the source file that will be compiled in bundle agent create_aout.
# See that bunlde's comments for more information.
vars:
# An slist is used here instead of a straight forward string because it doesn't seem possible to create
# line endings using \n when using a string to insert text into a file.
"c" slist => {"#include <stdlib.h>","#include <stdio.h>","#include <sys/stat.h>","#include <string.h>","void main()","{char file1[255];strcpy(file1,\"$(global_vars.file1)\");char file2[255];strcpy(file2,\"$(global_vars.file2)\");struct stat time1;int i = lstat(file1, &time1);struct stat time2;int j = lstat(file2, &time2);if (time1.st_mtime < time2.st_mtime){printf(\"Newer\");}else{if(time1.st_mtim.tv_nsec < time2.st_mtim.tv_nsec){printf(\"Newer\");}else{printf(\"Not newer\");}}}"};
files:
"$(global_vars.workdir)/a.c"
perms => system,
create => "true",
edit_line => Insert("@(c)");
reports:
"The source file $(global_vars.workdir)/a.c has been created. It will be used to compile the binary a.out, which will provide more accurate file stats to compare two files than the built in CFEngine functionality for comparing file stats, including modification time. This information will be used to determine of the second of the two files being compared is newer or not.";
"*********************************";
}
bundle edit_line Insert(name)
{
insert_lines:
"$(name)";
}
bundle agent create_aout
{
classes:
"doesfileacexist" expression => fileexists("$(global_vars.workdir)/a.c");
"doesaoutexist" expression => fileexists("$(global_vars.aoutbin)");
vars:
# Removes any previous binary
"rmaout" string => execresult("$(global_vars.rmexec) $(global_vars.aoutexec)","noshell");
doesfileacexist::
"compilestr" string => "$(global_vars.gccexec) $(global_vars.workdir)/a.c -o $(global_vars.aoutexec)";
"gccaout" string => execresult("$(compilestr)","noshell");
reports:
doesfileacexist::
"gcc output: $(gccaout)";
"Creating aout using $(compilestr)";
!doesfileacexist::
"Cannot compile a.out, $(global_vars.workdir)/a.c does not exist.";
doesaoutexist::
"The binary application aout has been compiled from the source in the create_aout_source_file bundle. It uses the stat library to compare two files, determine if the modified times are different, and whether the second file is newer than the first. The difference between this application and using CFEngine's built in support for getting file stats (e.g. filestat, isnewerthan), which provides file modification time accurate to a second. However, in order to better compare two files might sometimes require parts of a second as well. The stat library provides the extra support for retrieving the additional information required to get better accuracy (down to parts of a second), and is utilized by the binary application a.out that is compiled within the create_aout bundle.";
"*********************************";
}
bundle agent test_delete
{
files:
"$(global_vars.file1)"
delete => tidy;
}
bundle agent delete_file(fname)
{
files:
"$(fname)"
delete => tidy;
reports:
"Deleted $(fname)";
}
body contain del_file
{
useshell => "useshell";
}
bundle agent do_files_exist_1
{
classes:
"doesfile1exist" expression => fileexists("$(global_vars.file1)");
"doesfile2exist" expression => fileexists("$(global_vars.file2)");
methods:
doesfile1exist::
"any" usebundle => delete_file("$(global_vars.file1)");
doesfile2exist::
"any" usebundle => delete_file("$(global_vars.file2)");
reports:
!doesfile1exist::
"$(global_vars.file1) does not exist.";
doesfile1exist::
"$(global_vars.file1) did exist. Call to delete it was made.";
!doesfile2exist::
"$(global_vars.file2) does not exist.";
doesfile2exist::
"$(global_vars.file2) did exist. Call to delete it was made.";
}
bundle agent create_file_1
{
files:
"$(global_vars.file1)"
perms => system,
create => "true";
reports:
"$(global_vars.file1) has been created";
}
bundle agent outer_bundle_1
{
files:
"$(global_vars.file1)"
create => "false",
edit_line => inner_bundle_1;
}
bundle agent copy_a_file
{
files:
"$(global_vars.file2)"
copy_from => local_cp("$(global_vars.file1)");
reports:
"$(global_vars.file1) has been copied to $(global_vars.file2)";
"*********************************";
}
bundle agent do_files_exist_2
{
methods:
"any" usebundle => does_file_exist($(global_vars.file1));
"any" usebundle => does_file_exist($(global_vars.file2));
}
bundle agent does_file_exist(filename)
{
vars:
"filestat" string => filestat("$(filename)","mtime");
classes:
"fileexists" expression => fileexists("$(filename)");
reports:
fileexists::
"$(filename) exists. Last Modified Time = $(filestat).";
!fileexists::
"$(filename) does not exist";
}
bundle agent list_file_1
{
methods:
"any" usebundle => file_content($(global_vars.file1));
"any" usebundle => file_content($(global_vars.file2));
reports:
"*********************************";
}
bundle agent exec_aout
{
classes:
"doesaoutexist" expression => fileexists("$(global_vars.aoutbin)");
vars:
doesaoutexist::
"aout" string => execresult("$(global_vars.aoutexec)","noshell");
reports:
doesaoutexist::
"*********************************";
"$(global_vars.aoutbin) determined that $(global_vars.file2) is $(aout) than $(global_vars.file1)";
"*********************************";
!doesaoutexist::
"Executable $(global_vars.aoutbin) does not exist.";
}
bundle agent stat
{
classes:
"doesfile1exist" expression => fileexists("$(global_vars.file1)");
"doesfile2exist" expression => fileexists("$(global_vars.file2)");
vars:
doesfile1exist::
"file1" string => "$(global_vars.file1)";
"file2" string => "$(global_vars.file2)";
"file1_stat" string => execresult("/usr/bin/stat -c \"%y\" $(file1)","noshell");
"file1_split1" slist => string_split($(file1_stat)," ",3);
"file1_split2" string => nth("file1_split1",1);
"file1_split3" slist => string_split($(file1_split2),"\.",3);
"file1_split4" string => nth("file1_split3",1);
"file2_stat" string => execresult("/usr/bin/stat -c \"%y\" $(file2)","noshell");
"file2_split1" slist => string_split($(file2_stat)," ",3);
"file2_split2" string => nth("file2_split1",1);
"file2_split3" slist => string_split($(file2_split2),"\.",3);
"file2_split4" string => nth("file2_split3",1);
methods:
"any" usebundle => exec_aout();
reports:
doesfile1exist::
"Parts of a second extracted extracted from stat for $(file1): $(file1_split4). Full stat output for $(file1): $(file1_stat)";
"Parts of a second extracted extracted from stat for $(file2): $(file2_split4). Full stat output for $(file2): $(file2_stat)";
"Using the binary Linux application stat to compare two files can help determine if the modified times between two files are different. The difference between the stat application using its additional flags and using CFEngine's built in support for getting and comparing file stats (e.g. filestat, isnewerthan) is that normally the accuracy is only to the second of the file's modified time. In order to better compare two files requires parts of a second as well, which the stat command can provide with some additional flags. Unfortunately the information must be extracted from the middle of a string, which is what the stat bundle accomplishes using the string_split and nth functions.";
"*********************************";
!doesfile1exist::
"stat: $(global_vars.file1) and probably $(global_vars.file2) do not exist.";
}
bundle agent outer_bundle_2
{
files:
"$(global_vars.file2)"
create => "false",
edit_line => inner_bundle_2;
}
bundle edit_line inner_bundle_1
{
vars:
"msg" string => "Helloz to World!";
insert_lines:
"$(msg)";
reports:
"inserted $(msg) into $(global_vars.file1)";
}
bundle edit_line inner_bundle_2
{
replace_patterns:
"Helloz to World!"
replace_with => hello_world;
reports:
"Text in $(global_vars.file2) has been replaced";
}
body replace_with hello_world
{
replace_value => "Hello World";
occurrences => "all";
}
bundle agent list_file_2
{
methods:
"any" usebundle => file_content($(global_vars.file1));
"any" usebundle => file_content($(global_vars.file2));
classes:
"ok" expression => isgreaterthan(filestat("$(global_vars.file2)","mtime"),filestat("$(global_vars.file1)","mtime"));
"newer" expression => isnewerthan("$(global_vars.file2)","$(global_vars.file1)");
reports:
"*********************************";
ok::
"Using isgreaterthan+filestat determined that $(global_vars.file2) was modified later than $(global_vars.file1).";
!ok::
"Using isgreaterthan+filestat determined that $(global_vars.file2) was not modified later than $(global_vars.file1).";
newer::
"Using isnewerthan determined that $(global_vars.file2) was modified later than $(global_vars.file1).";
!newer::
"Using isnewerthan determined that $(global_vars.file2) was not modified later than $(global_vars.file1).";
}
bundle agent file_content(filename)
{
vars:
"file_content" string => readfile( "$(filename)" , "0" );
"file_stat" string => filestat("$(filename)","mtime");
reports:
"Contents of $(filename) = $(file_content). Last Modified Time = $(file_stat).";
#"The report on contents will only show new content and modifications. Even if the method is called more than once, if the evaluation is exactly the same as the previous call then there will be no report (possibly because the bundle is not evaluated a second time?).";
}
body perms system
{
mode => "0640";
}
Enterprise API Examples
- Check installation status
- Manage users, roles
- Managing Settings
- Browse host information
- Issue flexible SQL queries against data collected from hosts by the CFEngine Server
- Schedule reports for email and later download
- Tracking changes performed by CFEngine
See also: Enterprise API Reference
SQL Query Examples
Synchronous Example: Listing Hostname and IP for Ubuntu Hosts
Request:
curl -k --user admin:admin https://test.cfengine.com/api/query -X POST -d '{ "query": "SELECT Hosts.HostName, Hosts.IPAddress FROM Hosts"}'
Response:
{
"data": [
{
"header": [
{
"columnName": "hostname",
"columnType": "STRING"
},
{
"columnName": "ipaddress",
"columnType": "STRING"
}
],
"query": "select hostname, ipaddress from hosts",
"queryTimeMs": 152,
"rowCount": 1001,
"rows": [
[
"ubuntu10-2.stage.cfengine.com",
"172.20.100.1"
],
[
"ubuntu10-3.stage.cfengine.com",
"172.20.100.2"
],
[
"ubuntu10-4.stage.cfengine.com",
"172.20.100.3"
]
],
}
],
"meta": {
"count": 1,
"page": 1,
"timestamp": 1437051092,
"total": 1
}
}
Subscribed Query Example: Creating A Subscribed Query
Here we create a new query to count file changes by name and have the result
sent to us by email. The schedule field is any CFEngine context expression.
The backend polls subscriptions in a loop and checks whether it's time to
generate a report and send it out. In the following example, user milton
creates a new subscription to a report which he names file-changes-report
,
which will be sent out every Monday night. His boss will get an email with a
link to a PDF version of the report.
Request:
curl -k --user admin:admin https://test.cfengine.com/api/user/milton/ subscription/query/file-changes-report -X PUT -d '{"to": "boss@megaco.com", "query": "SELECT FileName, Count(*) FROM FileChangesLog GROUP BY FileName", "schedule": "Monday.Hr23.Min59", "title": "A very important file changes report""description": "Text that will be included in email""outputTypes": [ "pdf" ] }'
Response:
204 No Content
Subscribed Query Example: Listing Report Subscriptions
Milton can list all his current subscriptions by issuing the following.
Request:
curl -k --user admin:admin https://test.cfengine.com/api/user/milton/subscription/query
Response:
{
"meta": {
"page": 1,
"count": 1,
"total": 1,
"timestamp": 1351003514
},
"data": [
{
"id": "file-changes-report",
"to": "boss@megaco.com",
"query": "SELECT FileName, Count(*) FROM FileChangesLog GROUP BY FileName",
"title": "A very important file changes report",
"description": "Text that will be included in email",
"schedule": "Monday.Hr23.Min59",
"outputTypes": [
"pdf"
]
}
]
}
Subscribed Query Example: Removing A Report Subscription
Request:
curl -k --user admin:admin https://test.cfengine.com/api/user/milton/subscription/query/file-changes-report -X DELETE
Response:
204 No Content
Checking Status
You can get basic info about the API by issuing /api. This status information may also be useful if you contact support, as it gives some basic diagnostics.
Request
curl -k --user admin:admin https://test.cfengine.com/api/
Response
{
"meta": {
"page": 1,
"count": 1,
"total": 1,
"timestamp": 1351154889
},
"data": [
{
"apiName": "CFEngine Enterprise API",
"apiVersion": "v1",
"enterpriseVersion": "3.0.0a1.81c0d4c",
"coreVersion": "3.5.0a1.f3649b2",
"databaseHostname": "127.0.0.1",
"databasePort": 27017,
"authenticated": "internal",
"license": {
"expires": 1391036400,
"installTime": 1329578143,
"owner": "Stage Environment",
"granted": 20,
"licenseUsage": {
"lastMeasured": 1351122120,
"samples": 1905,
"minObservedLevel": 7,
"maxObservedLevel": 30,
"meanUsage": 21.9689,
"meanCumulativeUtilization": 109.8446,
"usedToday": 7
}
}
}
]
}
Managing Settings
Settings support two operations, GET (view settings) and POST (update settings). When settings are updated, they are sanity checked individually and as a whole. All or no settings will be updated for a request.
Example: Viewing settings
Request
curl --user admin:admin http://test.cfengine.com/api/settings
Response
{
"meta": {
"page": 1,
"count": 1,
"total": 1,
"timestamp": 1350992335
},
"data": [
{
"hostIdentifier": "default.sys.fqhost",
"rbacEnabled": true,
"ldapEnabled": true,
"blueHostHorizon": 900
}
]
}
Example: Configuring LDAP
The setting ldapEnabled
turns external authentication on or off. LDAP settings
are managed by the LDAP API and not this Settings API.
Request
curl --user admin:admin http://test.cfengine.com/api/settings -X PATCH -d '{ "ldapEnabled": true }'
Response
204 No Content
Example: Changing The Log Level
The API uses standard Unix syslog to log a number of events. Additionally, log
events are sent to stderr
, which means they may also end up in your Apache
log. Log events are filtered based on the log level in settings. Suppose you
wanted to have greater visibility into the processing done at the back-end. The
standard log level is error
. Changing it to info
is done as follows.
NOTE: Change to API log level will only take effect after Apache has re-started.
Request
curl --user admin:admin http://test.cfengine.com/api/settings -X PATCH -d '{ "logLevel": "info" }'
Response
204 No Content
Managing Users and Roles
Users and Roles determine who has access to what data from the API. Roles are defined by regular expressions that determine which hosts the user can see, and what policy outcomes are restricted.
Example: Listing Users
Request
curl --user admin:admin http://test.cfengine.com/api/user
Response
{
"meta": {
"page": 1,
"count": 2,
"total": 2,
"timestamp": 1350994249
},
"data": [
{
"id": "calvin",
"external": true,
"roles": [
"Huguenots", "Marketing"
]
},
{
"id": "quinester",
"name": "Willard Van Orman Quine",
"email": "noreply@@aol.com",
"external": false,
"roles": [
"admin"
]
}
]
}
Example: Creating a New User
All users will be created for the internal user table. The API will never attempt to write to an external LDAP server.
Request
curl --user admin:admin http://test.cfengine.com/api/user/snookie -X PUT -d
{
"email": "snookie@mtv.com",
"roles": [
"HR"
]
}
Response
201 Created
}
Example: Updating an Existing User
Both internal and external users may be updated. When updating an external users, the API will essentially annotate metadata for the user, it will never write to LDAP. Consequently, passwords may only be updated for internal users. Users may only update their own records, as authenticated by their user credentials.
Request
curl --user admin:admin http://test.cfengine.com/api/user/calvin -X POST -d '{ "name": "Calvin" }'
Response
204 No Content
Example: Retrieving a User
It is possible to retrieve data on a single user instead of listing
everything. The following query is similar to issuing GET
/api/user?id=calvin
, with the exception that the previous query accepts
a regular expression for id
.
Request
curl --user admin:admin http://test.cfengine.com/api/user/calvin
Response
{
"meta": {
"page": 1,
"count": 1,
"total": 1,
"timestamp": 1350994249
},
"data": [
{
"id": "calvin",
"name": "Calvin",
"external": true,
"roles": [
"Huguenots", "Marketing"
]
},
]
}
Example: Adding a User to a Role
Adding a user to a role is just an update operation on the user. The full role-set is updated, so if you are only appending a role, you may want to fetch the user data first, append the role and then update. The same approach is used to remove a user from a role.
Request
curl --user admin:admin http://test.cfengine.com/api/user/snookie -X POST -d
{
"roles": [
"HR", "gcc-contrib"
]
}
Response
204 No Content
}
Example: Deleting a User
Users can only be deleted from the internal users table.
Request
curl --user admin:admin http://test.cfengine.com/api/user/snookie -X DELETE
Response
204 No Content
Tracking changes
Changes REST API allows to track the changes made by cf-agent in the infrastructure.
Example: Count changes
This examples shows how to count changes performed by cf-agent within last 24h hours.
Example is searching for changes that are performed by linux machines within generate_repairs bundle.
Request
curl --user admin:admin 'https://test.cfengine.com/api/v2/changes/policy/count?include[]=linux&bundlename=generate_repairs'
Response
{
"count": 381
}
Example: Show vacuum command executions
Show all vacuumdb executions within last 24 hours executed on policy server.
Example is searching for changes that are performed by policy_server machines that execute commands promise with command /var/cfengine/bin/vacuumdb% - there is '%' sign at the end which is a wildcard as vacuumdb is executed with different options across policy.
Request
curl --user admin:admin 'https://test.cfengine.com/api/v2/changes/policy?include[]=policy_server&promisetype=commands&promiser=/var/cfengine/bin/vacuumdb%'
Response
{
"data": [
{
"bundlename": "cfe_internal_postgresql_vacuum",
"changetime": 1437642099,
"hostkey": "SHA=6ddfd5eaa85ee681ec12ce833fd7206e4d21c76e496be5f8b403ad0ead60a6ce",
"hostname": "hub.provisioned.1436361289.cfengine.com.com",
"logmessages": [
"Executing 'no timeout' ... '/var/cfengine/bin/vacuumdb --analyze --quiet --dbname=cfdb'",
"Completed execution of '/var/cfengine/bin/vacuumdb --analyze --quiet --dbname=cfdb'"
],
"policyfile": "/var/cfengine/inputs/lib/cfe_internal_hub.cf",
"promisees": [],
"promisehandle": "cfe_internal_postgresql_maintenance_commands_run_vacuumdb",
"promiser": "/var/cfengine/bin/vacuumdb --analyze --quiet --dbname=cfdb",
"promisetype": "commands",
"stackpath": "/default/cfe_internal_management/methods/'CFEngine_Internals'/default/cfe_internal_enterprise_main/methods/'hub'/default/cfe_internal_postgresql_vacuum/commands/'/var/cfengine/bin/vacuumdb --analyze --quiet --dbname=cfdb'[0]"
}
],
"total": 1,
"next": null,
"previous": null
}
Browsing Host Information
A resource /api/host is added as an alternative interface for browsing host
information. For full flexibility we recommend using SQL
reports via /api/query for this. however, currently vital signs (data
gathered from cf-monitord
) is not part of the SQL reports data model.
Example: Listing Hosts With A Given Context
Request
curl --user admin:admin http://test.cfengine.com/api/host?context-include=windows.*
Response
{
"meta": {
"page": 1,
"count": 2,
"total": 2,
"timestamp": 1350997528
},
"data": [
{
"id": "1c8fafe478e05eec60fe08d2934415c81a51d2075aac27c9936e19012d625cb8",
"hostname": "windows2008-2.test.cfengine.com",
"ip": "172.20.100.43"
},
{
"id": "dddc95486d97e4308f164ddc1fdbbc133825f35254f9cfbd59393a671015ab99",
"hostname": "windows2003-2.test.cfengine.com",
"ip": "172.20.100.42"
}
]
}
Example: Looking Up Hosts By Hostname
Contexts, also known as classes, are powerful. You can use them to
categorize hosts according to a rich set of tags. For example, each
host is automatically tagged with a canonicalized version of its
hostname and IP-address. So we could lookup the host with hostname
windows2003-2.test.cfengine.com
as follows (lines split and indented
for presentability).
Request
curl --user admin:admin http://test.cfengine.com/api/host?context-include=
windows2003_2_stage_cfengine_com
Response
{
"meta": {
"page": 1,
"count": 1,
"total": 1,
"timestamp": 1350997528
},
"data": [
{
"id": "dddc95486d97e4308f164ddc1fdbbc133825f35254f9cfbd59393a671015ab99",
"hostname": "windows2003-2.test.cfengine.com",
"ip": "172.20.100.42"
}
]
}
Example: Looking Up Hosts By IP
Similarly we can lookup the host with hostname
windows2008-2.test.cfengine.com
by IP as follows (lines split and indented
for presentability).
Request
curl --user admin:admin http://test.cfengine.com/api/host?
context-include=172_20_100_43
Response
{
"meta": {
"page": 1,
"count": 1,
"total": 1,
"timestamp": 1350997528
},
"data": [
{
"id": "1c8fafe478e05eec60fe08d2934415c81a51d2075aac27c9936e19012d625cb8",
"hostname": "windows2008-2.stage.cfengine.com",
"ip": "172.20.100.43"
}
]
}
Example: Removing Host Data
If a host has been decommissioned from a Hub, we can explicitly remove data associated with the host from the Hub, by issuing a DELETE request (lines split and indented for presentability).
Request
curl --user admin:admin http://test.cfengine.com/api/host/\
SHA=1c8fafe478e05eec60fe08d2934415c81a51d2075aac27c9936e19012d625cb8 -X DELETE
Response
204 No Content
Example: Listing Available Vital Signs For A Host
Each host record on the Hub has a set of vital signs collected by cf-monitord
on the agent. We can view the list of vitals signs from as host as follows
(lines split and indented for presentability).
Request
curl --user admin:admin http://test.cfengine.com/api/host/\
SHA=4e913e2f5ccf0c572b9573a83c4a992798cee170f5ee3019d489a201bc98a1a/vital
Response
{
"meta": {
"page": 1,
"count": 4,
"total": 4,
"timestamp": 1351001799
},
"data": [
{
"id": "messages",
"description": "New log entries (messages)",
"units": "entries",
"timestamp": 1351001400
},
{
"id": "mem_swap",
"description": "Total swap size",
"units": "megabytes",
"timestamp": 1351001400
},
{
"id": "mem_freeswap",
"description": "Free swap size",
"units": "megabytes",
"timestamp": 1351001400
},
{
"id": "mem_free",
"description": "Free system memory",
"units": "megabytes",
"timestamp": 1351001400
},
}
Example: Retrieving Vital Sign Data
Each vital sign has a collected time series of values for up to one week. Here
we retrieve the time series for the mem_free
vital sign at host
4e913e2f5ccf0c572b9573a83c4a992798cee170f5ee3019d489a201bc98a1a
for October
23rd 2012 12:20pm to 12:45pm GMT (lines split and indented for
presentability).
Request
curl --user admin:admin http://test.cfengine.com/api/host/\
SHA=4e913e2f5ccf0c572b9573a83c4a992798cee170f5ee3019d489a201bc98a1a/vital/\
mem_free?from=1350994800&to=1350996300
Response
"meta": {
"page": 1,
"count": 1,
"total": 1,
"timestamp": 1351002265
},
"data": [
{
"id": "mem_free",
"description": "Free system memory",
"units": "megabytes",
"timestamp": 1351001700,
"values": [
[
1350994800,
36.2969
],
[
1350995100,
36.2969
],
[
1350995400,
36.2969
],
[
1350995700,
36.2969
],
[
1350996000,
36.1758
],
[
1350996300,
36.2969
]
]
}
]