The Complete Guide
Table of Content
- Introduction and System Overview
- Latest Release
- Installation and Configuration
- Writing and Serving Policy
- Design Center Overview
- Reporting
- FAQ
- Additional Resources
CFEngine is a configuration management system that provides a framework for automated management of IT infrastructure.
CFEngine is decentralized and highly scalable. It is powered by autonomous agents that can continuously monitor, self-repair, and update or restore an entire IT system, with negligible impact on system resources or performance.
See Also: Introduction and System Overview
CFEngine Features
- Defines the configuration of an entire IT system, including: Devices, Users, Applications, and Services.
- Helps maintain that system over time.
- Checks the system state at any given moment.
- Ensures compliance with a desired system state.
- Propagates real-time modifications or updates across the system.
Choose a CFEngine Version
CFEngine Enterprise is a licensed edition for enterprises that plan to use the tool in production environments. The Enterprise edition comes in several variants, including one that can be evaluated for free (up to 25 servers).
CFEngine Community, a free GPL v3 open source edition.
See also:
Install It
There are several steps to bring up a CFEngine installation within an organization:
- Prepare all appropriate machines for installation.
- Configure your network and security.
- Download the CFEngine software.
- Install CFEngine on the Policy Server(s).
- Bootstrap the Policy Server to itself.
- Initiate post-install configuration on the Policy Server.
- Install CFEngine on the Host machine(s).
- Bootstrap the Host(s) to a Policy Server.
See Installation and Configuration for a more detailed guide on how to get CFEngine up and running for various environments.
Try It
Walk through the examples, tutorials and how to guides to get a better feel for the power and value of CFEngine:
Learn More
Take a look at the CFEngine Guide to learn more about CFEngine's architecture and components, as well as how to write policy that can help manage IT systems.
Check out Additional Resources and the CFEngine Learning Center, for more guides, demos, and other resources from our CFEngine staff and our special CFEngine contributors.
Use our Help
Support and Community We provide a number of ways to connect you to CFEngine experts if you need more help. Contact us!
CFEngine Guide
- Introduction and System Overview
- Latest Release
- Installation and Configuration
- Pre-Installation Checklist
- General Installation
- [Upgrading to 3.9]
- Secure Bootstrap
- Writing and Serving Policy
- Reports
- Design Center Overview
- FAQ
- Additional Resources
Introduction and System Overview
CFEngine is a distributed system for managing and monitoring computers across an IT network. Machines on the network that have CFEngine installed, and have registered themselves with a policy server (see Installation and Configuration), will each be running a set of CFEngine component applications that manage and interpret textual files called policies. Policy files themselves contain sets of instructions to ensure machines on the network are in full compliance with a defined state. At the atomic level are sets, or bundles, of what are known in the CFEngine world as Promises. Promises are at the heart of Promise Theory, which is in turn what CFEngine is all about.
- Policy Language and Compliance
- CFEngine Policy Servers and Hosts
- CFEngine Component Applications and Daemons
Policy Language and Compliance
For many users, CFEngine is simply a configuration tool – i.e. software for deploying and patching systems according to a policy. Policy is described using promises. Every statement in CFEngine 3 is a promise to be kept at some time or location. More than this, however, CFEngine is not like other automation tools that "roll out" an image of some software once and hope for the best. Every promise that you make in CFEngine is continuously verified and maintained. It is not a one-off operation, but a self-repairing process should anything deviate from the policy.
CFEngine ensures that the actual state of a system is in compliance with the predefined model of desired state for the system. If it is not in compliance CFEngine will bring it into compliance. This is known as convergence.
That model is represented by one or more policies that have been written using the declarative CFEngine policy language. The policy language has been designed with a vocabulary that is intuitive, yet at the same time can still support the design of highly complex IT systems.
Those policies are distributed across all hosts within the system via download from the policy server. Every host will then interpret and execute each of the instructions it has been given in a predetermined order.
CFEngine continually monitors all of the hosts in real-time, and should the system’s current state begin to drift away from the intended state then CFEngine will automatically take corrective action to bring everything back into compliance.
See Also: Language Concepts, Writing and Serving Policy
CFEngine Policy Servers and Hosts
There are basically two categories of machines in a CFEngine environment: policy servers and their client hosts. Policy servers are responsible for making policy files available to each of the client hosts that have registered with it (a.k.a. bootstrapped), including itself. Hosts on the other hand are responsible for ensuring they continuously pull in the latest policies, or changes to policies, from the policy server. They are additionally responsible for ensuring they remain fully compliant with the instructions contained within the policy files, at all times.
The role of a particular machine where CFEngine is deployed determines which of the components will be installed and running at any given moment.
See Also: Writing and Serving Policy
CFEngine Component Applications and Daemons
There are a number of components in CFEngine, with each component performing a unique function: components responsible for implementing promises, components responsible for organizing large networks of agents, and other components responsible for providing the infrastructure of CFEngine.
These components form the basis of automation with CFEngine. They are independent software agents running on the various systems that make up your infrastructure. They communicate with one another as shown in the following figure, using a protocol that allows each host to distribute promises, act upon them, and report status to a central server.
All CFEngine software components exist in /var/cfengine/bin
.
Daemons
All machines, whether they are policy servers or hosts, will have these three important daemons running at all times:
cf-execd
cf-execd
is the scheduling daemon for cf-agent
, similar to cron. It executes and collects the output of cf-agent
and
e-mails any output to the configured e-mail address.
cf-execd
runs cf-agent
locally according to a schedule specified in policy code (executor control body). After a cf-agent
run is completed, cf-execd
gathers output from cf-agent
, and may be configured to email the output to a specified address. It may also be configured to splay
(randomize) the execution schedule to prevent synchronized cf-agent
runs across a network.
cf-execd
keeps the promises made in common bundles, and is affected by common and executor control bodies.
See also: cf-execd reference documentation.
cf-serverd
cf-serverd
is a socket listening daemon providing two services: it acts as a file server for remote file copying and it allows an authorized cf-runagent
to start a cf-agent
run. cf-agent
typically connects to a cf-serverd
instance to request updated policy code, but may also request additional files for download. cf-serverd
employs role based access control (defined in policy code) to authorize requests.
cf-serverd
keeps the promises made in common
and server
bundles, and is affected by common
and server
control bodies.
By starting this daemon you can set up a line of communication between hosts. The server is able to share files and receive requests to execute existing policy on an individual machine. It is not possible to send (push) new information to CFEngine from outside.
This daemon authenticates requests from the network and processes them according to rules specified in the server control body and server bundles containing access promises.
See also: cf-serverd reference documentation.
cf-monitord
cf-monitord
is the monitoring daemon for CFEngine. It samples probes defined in policy using measurements type promises and attempts to learn the normal system state based on current and past observations. Current estimates are made available as special variables (e.g. $(mon.av_cpu)) to cf-agent
, which may use them to inform policy decisions.
cf-monitord
keeps the promises made in common
and monitor
bundles, and is affected by common
and monitor
control bodies.
See also: cf-monitord reference documentation.
Other Component Applications
- /var/cfengine/bin/cf-agent
- /var/cfengine/bin/cf-key
- /var/cfengine/bin/cf-promises
- /var/cfengine/bin/cf-runagent
cf-agent
cf-agent
evaluates policy code and makes changes to the system. Policy bundles are evaluated in the order of the provided bundlesequence
(this is normally specified in the common control body and defaults to just the main
bundle if unspecified). For each bundle, cf-agent groups promise statements according to their type. Promise types are then evaluated in a preset order to ensure fast system convergence to policy.
cf-agent
keeps the promises made in common
and agent
bundles, and is affected by common
and agent
control bodies.
cf-agent
is the instigator of change. Everything that happens on a client machine
happens because of cf-agent
. The agent is the part of CFEngine that
manipulates system resources.
cf-agent
's only contact with the network is via remote copy requests. It
does not and cannot grant any access to a system from the network. It is only
able to request access to files from the server component.
See also: cf-agent
reference documentation.
cf-key
The CFEngine key generator makes key pairs for remote authentication.
See also: cf-key
reference documentation.
cf-promises
cf-promises
is CFEngine's promise verifier. It is used to run a "pre-check" of
policy code before cf-agent
attempts to execute.
cf-promises
operates by first parsing policy code checking for syntax errors. Second, it validates the integrity of policy consisting of multiple files. Third, it checks for semantic errors, e.g. specific attribute set rules. Finally, cf-promises
attempts to expose errors by partially evaluating the policy, resolving as many variable and classes promise statements as possible. At no point does cf-promises
make any changes to the system.
In 3.6.0 and later, cf-promises
will not evaluate function calls either. This may affect customers who use execresult for instance. Use the new --eval-functions yes command-line option (default is no) to retain the old behavior from 3.5.x and earlier.
See also: cf-promises
reference documentation.
cf-runagent
cf-runagent
is a helper program that can be used to run cf-agent
on a number of remote
hosts. It cannot be used to tell cf-agent
what to do, it can only ask
cf-serverd
on the remote host to run the cf-agent
with its existing
policy. It can thus be used to trigger an immediate deployment of new policy,
if their existing policy includes that they check for updates.
Privileges can be granted to users to provide a kind of Role Based Access Control (RBAC) to certain parts of the existing policy.
cf-runagent
connects to a list of running instances of cf-serverd
. It allows foregoing the usual cf-execd
schedule to activate cf-agent
. Additionally, a user may send classes to be defined on the remote host. Two kinds of classes may be sent: classes to decide on which hosts cf-agent
will be started, and classes that the user requests cf-agent
should define on execution. The latter type is regulated by cf-serverd
's role based access control.
See also: cf-runagent
reference documentation.
CFEngine Architecture and Design
CFEngine operates autonomously in a network, under your guidance. While CFEngine supports anything from 1 servers to 100,000+ servers, the essence of any CFEngine deployment is the same.
CFEngine supports networks of any size, from a handful of nodes to hundreds of thousands of computers. It is built to scale. If your site is very large (many thousands of servers) you should spend some time discussing your requirements with CFEngine experts. They will know how to tune promises and configurations to your environment as scale requires you to have more infrastructure, and a potentially more complicated configuration. No matter the scale, the essence of any CFEngine deployment is the same, but with great power comes great responsibility (a.k.a. don't break things before the weekend, on the weekend, or in fact on any other day).
CFEngine was designed to enable scalable configuration management in any kind of environment, with an emphasis on supporting large, Unix-like systems that are connected via TCP/IP.
CFEngine doesn't depend on or assume the presence of reliable infrastructure. It works opportunistically in any environment, using the fewest possible resources, and it has a limited set of software dependencies. It can run anywhere and this lean approach to CFEngine's architecture makes it possible to support both traditional server-based approaches to configuration as well as more novel platforms for configuration including embedded and mobile systems.
CFEngine's design allows you to create fault-tolerant, available systems which are independent of external requirements. CFEngine works in all the places you think it should, and all the new places you haven't even thought of yet.
Managing Expectations with Promises
CFEngine works on a simple notion of promises. A promise is the documentation of an intention to act or behave in some manner. When you make a promise, it is an effort to improve trust. Trust is an economic time-saver. If you can't trust you have to verify everything, and that is expensive.
Everything in CFEngine can be thought of as a promise to be kept by different
resources in the system. In a system that delivers a web site with Apache
httpd
, an important promise may be to make sure that the httpd
or apache
package is installed,
running, and accessible on port 80. In a system which needs to satisfy mid-day
traffic on a busy web site, a promise may be to ensure that there are 200
application servers running during normal business hours.
These promises are not top-down directives for a central authority to push through the system. A large organization can't run on top-down authority alone. A group of people can't be managed without empowering and trusting them to make independent decisions.
CFEngine is a system that emphasizes the promises a client makes to the overall CFEngine network. They are the rules which clients are responsible for implementing. We can create large systems of scale because we don't create a bulky centralized authority. There should be no single point-of-failure when managing machines and people.
Combining promises with patterns to describe where and when promises should apply is what CFEngine is all about.
Automation with CFEngine
Users are good at researching solutions and making design decisions, but awful at repeated execution. Machines are pitiful at making decisions, but very good at reliable implementation at very large scale. It makes sense to let each side do the job that they are good at. With CFEngine, users make decisions and write promises for machines to implement and satisfy.
A CFEngine user will declare a promise in CFEngine, and CFEngine will then translate this promise into a series of actions to implement. For the most part, CFEngine understands how to deliver on promises, and they don't need to be given explicit instructions for completing tasks. It is your job to make decisions about the systems you are managing and to describe those in suitable promises. It is CFEngine's job to automate and deliver a promise.
CFEngine is a distributed solution that is completely independent of host operating systems, network topology or system processes. You describe the ideal state of a given system by creating promises and the CFEngine agents ensures that the necessary steps are taken to achieve this state. Automation in CFEngine is executed through a series of components that run locally on hosts.
Phases of System Management
There are four commonly cited phases in managing systems with CFEngine: Build, Deploy, Manage, and Audit.
Build
A system is based on a number of decisions and resources that need to be `built' before they can be implemented. You don't need to decide every detail, just enough to build trust and predictability into your system. In CFEngine, what you build is a template of proposed promises for the machines being managed. If the machines in a system all make and keep these promises, the system will function seamlessly as planned.
Deploy
Deploying really means implementing the policy that was already decided. In transaction systems, one tries to push out changes one-by-one, hence `deploying' the decision. In CFEngine you simply publish your policy (in CFEngine parlance these are "promise proposals") and the machines see the new proposals and can adjust accordingly. Each machine runs an agent that is capable of implementing policies and maintaining them over time without further assistance.
Manage
Once a decision is made, unplanned events will occur. Such incidents traditionally set off alarms and humans rush to make new transactions to repair them. In CFEngine, the autonomous agent manages the system, and you only have to deal with rare events that cannot be dealt with automatically. This is the key difference of CFEngine, a focus on autonomy and creating agents that are smart enough to adapt to changing situations.
Audit
In traditional configuration systems, the outcome is far from clear after a one-shot transaction, so one audits the system to determine what actually happened. In CFEngine, changes are not just initiated once, but locally audited and maintained. Decision outcomes are assured by design in CFEngine and maintained automatically, so the main worry is managing conflicting. Users can sit back and examine regular reports of compliance generated by the agents, without having to arrange for new transactions to roll-out changes.
You should not think of CFEngine as a roll-out system, i.e. one that attempts to force out absolute changes and perhaps reverse them in case of error. Roll-out and roll-back are theoretically flawed concepts that only sometimes work in practice. With CFEngine, you publish a sequence of policy revisions, always moving forward (because like it or not, time only goes in one direction). All of the desired-state changes are managed locally by each individual host, and continuously repaired to ensure on-going compliance with policy.
See Also: Networking
CFEngine Directory Structure
The CFEngine application is fully contained within the /var/cfengine directory tree. Here is a quick breakdown of the directory structure and some of the files and functions associated with each subdirectory.
- /var/cfengine/bin
- Directories for Policy Files
- Output Directories
- Log Files in /var/cfengine
- Database Files in /var/cfengine
- Process (AKA PID) Files in /var/cfengine
- Sockets in /var/cfengine
- Datafiles in /var/cfengine
- Binary Files in /var/cfengine
- git in /var/cfengine/bin
- Misc. in /var/cfengine/bin
- Postgres in /var/cfengine/bin
- Redis in /var/cfengine/bin
/var/cfengine/bin
Agents
cf-agent
: Executes the promises.cf file; ensures that all promises are being keptcf-key
cf-promises
: Verifies CFEngine's configuration syntaxcf-runagent
: Contacts a remote system to run cf-agentcf-twin
Daemons
cf-execd
: Starts the cf-agent process at a specified time interval.cf-monitord
: Collects system statisticscf-serverd
: Provides network services; used to distribute policy and data filesrunalerts.sh
: Updates Mission Portal status and activates alert actions (Enterprise only)cf-consumer
: Responsible for moving collected reports from the redis queue into the postgres database. (CFEngine Enterprise only)cf-hub
: Responsible for collecting reports from remote agents. (CFEngine Enterprise only)
See Also: CFEngine Component Applications and Daemons
Directories for Policy Files
/var/cfengine/modules
Location of scripts used in commands
promises.
/var/cfengine/inputs
Cached policy repository on each CFEngine client. When cf-agent
is
invoked by cf-execd
, it reads only from this directory.
/var/cfengine/masterfiles
Policy repository which grants access to local or bootstrapped CFEngine
clients when they need to update their policies. Policies obtained from
/var/cfengine/masterfiles
are then cached in /var/cfengine/inputs
for
local policy execution. The cf-agent
executable does not execute policies
directly from this repository.
Output Directories
/var/cfengine/outputs
Directory where cf-agent
creates its output files. The outputs directory is
a record of spooled run-reports. These are often mailed to the administrator
by cf-execd
, or can be copied to another central location and viewed in an
alternative browser. However, not all hosts have an email capability or are
online, so the reports are kept here.
/var/cfengine/reports
Directory used to store reports. Reports are not tidied automatically, so you should delete these files after a time to avoid a build up.
/var/cfengine/state
State data such as current process identifiers of running processes, persistent classes and other cached data.
/var/cfengine/lastseen
Log data for incoming and outgoing connections.
/var/cfengine/cfapache
/var/cfengine/config
/var/cfengine/design-center
/var/cfengine/httpd
/var/cfengine/lib
Directory to store shared objects and dependencies that are in the bundled packages.
/var/cfengine/lib-twin
/var/cfengine/master_software_updates
/var/cfengine/plugins
/var/cfengine/ppkeys
Directory used to store encrypted public/private keys for CFEngine client/server network communications.
/var/cfengine/share
/var/cfengine/software_updates
/var/cfengine/ssl
Log Files in /var/cfengine
On hosts, CFEngine writes numerous logs and records to its private workspace.
CFEngine Enterprise provides solutions for centralization and network-wide reporting at an arbitrary scale.
cf3.[hostname].runlog
A time-stamped log of when each lock was released. This shows the last time each individual promise was verified.
cfagent.[hostname].log
Although ambiguously named (for historical reasons) this log contains the current list of setuid/setgid programs observed on the system. CFEngine warns about new additions to this list. This log has been deprecated.
cf_notkept.log
In CFEngine Enterprise, a list of promises, with handles and comments, that were not kept.
cf_repair.log
In CFEngine Enterprise, a list of promises, with handles and comments, that were repaired.
promise_summary.log
A time-stamped log of the percentage fraction of promises kept after each run.
Database Files in /var/cfengine
- bundles.lmdb
cf_classes.lmdb
A database of classes that have been defined on the current host, including their relative frequencies, scaled like a probability.
cf_lastseen.lmdb
A database of hosts that last contacted this host, or were contacted by this host, and includes the times at which they were last observed.
checksum_digests.lmdb
The database of hash values used in CFEngine's change management functions.
nova_agent_execution.lmdb
nova_track.lmdb
performance.lmdb
A database of last, average and deviation times of jobs recorded by
cf-agent
. Most promises take an immeasurably short time to check, but
longer tasks such as command execution and file copying are measured by
default. Other checks can be instrumented by setting a
measurement_class
in the action
body of a promise.
Process (AKA PID) Files in /var/cfengine
The CFEngine components keep their current process identifier number in `pid files' in the work directory.
cf-consumer.pid
cf-execd.pid
cf-hub.pid
cf-monitord.pid
cf-serverd.pid
Sockets in /var/cfengine
cf-hub-local
Datafiles in /var/cfengine
policy_server.dat
IP address of the policy server
Binary Files in /var/cfengine
randseed
git in /var/cfengine/bin
bin/git
bin/git-cvsserver
bin/gitk
bin/git-receive-pack
bin/git-shell
bin/git-upload-archive
bin/git-upload-pack
Misc. in /var/cfengine/bin
bin/curl
bin/lmdump
bin/openssl
bin/rpmvercmp
bin/rsync
bin/runalerts.sh
Postgres in /var/cfengine/bin
bin/clusterdb
bin/createdb
bin/createlang
bin/createuser
bin/dropdb
bin/droplang
bin/dropuser
bin/initdb
bin/pg_basebackup
bin/pg_config
bin/pg_controldata
bin/pg_ctl
bin/pg_dump
bin/pg_dumpall
bin/pg_isready
bin/pg_receivexlog
bin/pg_resetxlog
bin/pg_restore
bin/postgres
bin/postmaster
bin/psql
bin/reindexdb
bin/vacuumdb
Redis in /var/cfengine/bin
bin/redis-benchmark
bin/redis-check-aof
bin/redis-check-dump
bin/redis-cli
bin/redis-server
Not Verified
state/cf_lock.lmdb
A database of active and inactive locks and their expiry times. Deleting this database will reset all lock protections in CFEngine.
state/history.lmdb
CFEngine Enterprise maintains this long-term trend database.
state/cf_observations.lmdb
This database contains the current state of the observational history of
the host as recorded by cf-monitord
.
state/cf_state.lmdb
A database of persistent classes active on this current host.
state/nova_measures.lmdb
CFEngine Enterprise database of custom measurements.
state/nova_static.lmdb
CFEngine Enterprise database of static system discovery data.
state/cf_procs
A cache of the process table. This is useful formeasurement
promises about processes.state/cf_rootprocs
A cache of the process table of processes owned by the root user. This is useful formeasurement
promises about processes.state/cf_otherprocs
A cache of the process table for processes not owned by the root user. This is useful formeasurement
promises about processes.state/file_changes.log
A time-stamped log of which files have experienced content changes since the last observation, as determined by the hashing algorithms in CFEngine.
state/*_measure.log
CFEngine Enterprise maintains user-defined logs based on specifically promised observations of the system.
state/env_data
This file contains a list of currently discovered classes and variable values that characterize the anomaly alert environment. They are altered by the monitor daemon.
/var/logs/CFEngineHub-Install.log
This file contains logs related to the CFEngine package installation.
Networking
Starting cf-serverd
sets up a line of communication between
hosts. This daemon authenticates requests from the network and processes them
according to rules specified in the
server control
body and server bundles
containing access
promises.
The server can allow the network to access files or to execute CFEngine:
The only contact
cf-agent
makes to the server is via remote copy requests. It does not and cannot grant any access to a system from the network. It is only able to request access to files on the remote server.cf-runagent
can be used to runcf-agent
on a number of remote hosts.
Unlike other approaches to automation, CFEngine does not rely on SSH key authentication and configuring trust, the communication between hosts is very structured and also able to react to availability issues. This means that you don't have to arrange login credentials to get CFEngine to work. If large portions of your network stop working, then CFEngine on each individual host understands how to keep on running and delivering promises.
In particular, if the network is not working, CFEngine agents skip downloading new promises and continue with what they already have. CFEngine was specifically designed to be resilient against connectivity issues network failure may be in question. CFEngine is fault tolerant and opportunistic.
Connecting to server
In order to connect to the CFEngine server you need:
- A public-private key pair. It is automatically generated during package
installation or during bootstrap. To manually create a key pair,
run
cf-key
. - Network connectivity with an IPv4 or IPv6 address.
- Permission to connect to the server.
The
server control
body must grant access to your computer and public key by name or IP address, by listing it in the appropriate access lists (see below). - Mutual key trust.
Your public key must be trusted by the server, and you must trust the server's
public key. The first part is established by having the
trustkeysfrom
setting open on the server for the first connection of the agent. It should be closed later to avoid trusting new agents. The second part is established by bootstrapping the agent to the hub, or by executing acopy_from
files promise usingtrustkey=>"true"
. - Permission to access something.
Your host name or IP address must be mentioned in an
access
promise inside a server bundle, made by the file that you are trying to access.
If all of the above criteria are met, connection will be established and data will be transferred between client and server. The client can only send short requests, following the CFEngine protocol. The server can return data in a variety of forms, usually files, but sometimes console output.
Bootstrapping
Bootstrap is the manual first run of cf-agent that establishes
communication with the policy server.
Bootstrapping executes the failsafe.cf
policy that connects to
the server, establishes trust to the server's key, and that starts the
CFEngine daemon processes cf-execd
, cf-serverd
and cf-monitord
.
The host that other hosts are bootstrapped to
automatically assumes the role of policy server.
You should bootstrap the policy server first to itself:
$ /var/cfengine/bin/cf-agent --bootstrap [public IP of localhost]
Then execute the same step (using the exact same IP) on all hosts that should pull policy from that server. CFEngine will create keys if there are none present, and exchange those to establish trust.
CFEngine will output diagnostic information upon bootstrap. In case of error,
investigate the access
promises the server is making (run cf-serverd
in
verbose mode on the policy hub for more informative messages). Note that
by default, CFEngine's server daemon cf-serverd
trusts incoming connections
from hosts within the same /16
subnet.
After a host has been bootstrapped, the text file policy_server.dat
in
the CFEngine installation contains the IP address of the policy server.
Key exchange
The key exchange model used by CFEngine is based on that used by OpenSSH. It is a peer to peer exchange model, not a central certificate authority model. This means that there are no scalability bottlenecks (at least by design, though you might introduce your own if you go for an overly centralized architecture).
Key exchange is handled automatically by CFEngine and all you need to do is to
decide which keys to trust. The server (cf-serverd
) trusts new keys only
from addresses in trustkeysfrom
. Once a key has been
accepted you should close down trustkeysfrom
list. Then, even if a malicious peer
is spoofing an allowed IP address, its unknown key will be denied.
Once you have arranged for the right to connect to the server, you must decide
which hosts will have access to which files. This is done with access
promises.
bundle server access_rules()
{
access:
"/path/file"
admit => { "127.0.0.1", "127.0.0.2", "127.0.0.3" },
deny => { "192.168.0.0/8" };
}
On the client side, i.e. cf-runagent
and cf-agent
, there are three issues:
- Choosing which server to connect to.
- Trusting the key of any previously unknown servers
- Choosing whether data transfers should be encrypted (with
encrypt
) - not applicable if you are using newprotocol_version
.
There are two ways of managing trust of server keys by a client. One is an
automated option, setting the option trustkey
in a copy_from
files promise, e.g.
body copy_from example
{
# .. other settings ..
trustkey => "true";
}
Another way is to run cf-runagent
in interactive mode. When you run
cf-runagent
, unknown server keys are offered to you interactively (as with
ssh
) for you to accept or deny manually:
$ WARNING - You do not have a public key from host ubik.iu.hio.no = 128.39.74.25
$ Do you want to accept one on trust? (yes/no)
-->
Once public keys have been exchanged from client to server and from server to client, the issue of trust is solved according to public key authentication schemes. You only need to worry about trust when one side of a connection has never seen the other side before.
Time windows (races)
All security is based on a moment of trust that is granted by a user at some point in time – and is assumed thereafter (once given, hard to rescind). Cryptographic key methods only remove the need for a repeat of the trust decision. After the first exchange, trust is no longer needed, because the keys allow identity to be actually verified.
Even if you leave the trust options switched on, you are not blindly trusting
the hosts you know about. The only potential insecurity lies in any new keys
that you have not thought about. If you use wildcards or IP prefixes in the
trust rules, then other hosts might be able to spoof their way in on trust
because you have left open a hole for them to exploit. That is why it is
recommended to set the system to the state of zero trust
immediately after key transfer, by commenting or emptying out the trust options
(trustkeysfrom
on the server).
It is possible, though somewhat laborious, to transfer the keys out of band,
by copying /var/cfengine/ppkeys/localhost.pub
to
/var/cfengine/ppkeys/user-aaa.bbb.ccc.mmm
(assuming IPv4) on another host.
e.g.
localhost.pub -> root-128.39.74.71.pub
Other users than root
CFEngine normally runs as user "root" (except on Windows which does not normally have a root user), i.e. a privileged administrator. If other users are to be granted access to the system, they must also generate a key and go through the same process. In addition, the users must be added to the server configuration file.
Encryption
CFEngine has 2 communication protocols. classic
or 1
and 2
or latest
.
Each protocol provides different encryption options for keeping file contents
private during transfer.
However, the main role of encryption in configuration management is for authentication. Secrets should not be transferred through policy, encrypted or not. Policy files should be considered public, and any leakage should not reveal secret information.
Note: Connections from the cf-agent are cached as described in the
documentation for body copy_from
.
Protocol Classic
Encryption for Enterprise is symmetric AES 256 bit in CBC mode, using a session key exchanged during the RSA handshake.
In core/community as outgoing outlined in the
body copy_from encrypt
documentation the initial
connection is encrypted using the public/private keys for the client
and server hosts. After the initial connection is established
subsequent connections and data transfer is encrypted by a randomly
generated Blowfish key that is refreshed each session.
With the classic protocol cf-serverd has the ability to enforce that a
file transfer be encrypted by setting the
ifencrypted
access attribute. When ACLs that
require encryption have unencrypted access attempts cf-serverd logs an
error message indicating the file requires encryption. Access to files
that cf-serverd requires to be encrypted can be logged by setting the
body server control logencryptedtransfers
attribute.
Protocol 2
3.6 introduced a new protocol option for communication with cf-serverd. Protocol 2 is the default in 3.7+ and uses a TLS session for encryption.
Note: When protocol 2 is in use legacy encryption attributes are noop.
The following attributes are affected:
- encrypt` in copy from bodies
ifencrypted
in in access promiseslogencryptedtransfers
in body common control
The specific encryption algorithm used depends on the cipher
negotiated between the client and the server. You can control which
ciphers are allowed by cf-serverd for incoming connections by
setting the
body server control allowciphers
attribute. Controlling
which ciphers are allowed to be used in outgoing connections is
done by setting
body common control tls_ciphers
.
Additionally the minimum version of TLS required for incoming
connections can be set in
body server control allowtlsversion
and the minimum version of TLS required for outgoing connections
can be set in
body common control tls_min_version
.
There are debug and verbose level logs produced by cf-agent to indicate when TLS is in use.
The following was captured by running the agent update policy in debug mode.
/var/cfenigne/bin/cf-agent -Kdf update.cf
verbose: Connected to host 192.168.33.2 address 192.168.33.2 port 5308
debug: TLSVerifyCallback: no ssl->peer_cert
debug: TLSVerifyCallback: no conn_info->key
debug: This must be the initial TLS handshake, accepting
verbose: TLS version negotiated: TLSv1.2; Cipher: AES256-GCM-SHA384,TLSv1/SSLv3
verbose: TLS session established, checking trust...
verbose: Received public key compares equal to the one we have stored
verbose: Server is TRUSTED, received key 'SHA=5d20c01e4230aa53863eb36686eaa882094cdbddf53545616dfd588f00cc0659' MATCHES stored one.
debug: TLSRecvLines(): CFE_v2 cf-serverd 3.7.1.
debug: TLSRecvLines(): OK WELCOME USERNAME=root
cf-serverd emits verbose and debug log messages indicating when TLS is in use.
The following was captured by starting cf-serverd in the foreground with debug mode.
/var/cfenigne/bin/cf-serverd -Fd
verbose: New connection (from 192.168.33.3, sd 7), spawning new thread...
verbose: CollectCallHasPending: false
debug: Waiting at incoming select...
info: 192.168.33.3> Accepting connection
verbose: 192.168.33.3> Setting socket timeout to 600 seconds.
verbose: 192.168.33.3> Peeked nothing important in TCP stream, considering the protocol as TLS
debug: 192.168.33.3> Peeked data: ....2......ak.
debug: 192.168.33.3> TLSVerifyCallback: no ssl->peer_cert
debug: 192.168.33.3> TLSVerifyCallback: no conn_info->key
debug: 192.168.33.3> This must be the initial TLS handshake, accepting
verbose: 192.168.33.3> TLS version negotiated: TLSv1.2; Cipher: AES256-GCM-SHA384,TLSv1/SSLv3
verbose: 192.168.33.3> TLS session established, checking trust...
debug: 192.168.33.3> TLSRecvLines(): CFE_v2 cf-agent 3.7.1.
debug: 192.168.33.3> TLSRecvLines(): IDENTITY USERNAME=root.
verbose: 192.168.33.3> Setting IDENTITY: USERNAME=root
verbose: 192.168.33.3> Received public key compares equal to the one we have stored
verbose: 192.168.33.3> SHA=4f25279831eeaf579d2e3451345854a93fdefc856ad741bd59515b859fb84dea: Client is TRUSTED, public key MATCHES stored one.
Troubleshooting
When setting up cf-serverd
, you might see the error message
Unspecified server refusal
This means that cf-serverd
is unable or is unwilling to authenticate the
connection from your client machine. The message is deliberately non-specific
so that anyone attempting to attack or exploit the service will not be given
information which might be useful to them.
There is a simple checklist for curing this problem:
- Make sure that you have granted access to the client's address in the
server control
body. - Make sure the connecting client is granted access to the requested resources
(files usually) in the
access_rules
promise bundle. - See the verbose log of the server for the exact error message, since the
client always gets the "Unspecified server refusal" reply from the server.
To run the server in verbose, kill cf-serverd on the policy hub and run:
$ cf-serverd -v
and then manually run
cf-agent
on the client. - In the unlikely case that you still get no indication of the denial, try
increasing the agent run verbosity.
cf-agent -I
for info-level messages or evencf-agent -v
for verbose.
Latest Release
New in CFEngine Learn about the newest features in CFEngine 3.9
Supported Platforms and Versions These are the supported platforms for the current release.
Known Issues View any issues of which we are currently aware and investigating. View possible workarounds.
New in CFEngine
CFEngine 3.9 is the latest version of CFEngine. To see what's new in this
release, see the ChangeLog
and Enterprise ChangeLog
.
ChangeLog
For a history of changes in the CFEngine version you have installed, see the ChangeLog
and
ChangeLog.Enterprise
files in /var/cfengine/share/doc
. The running change log
for the core repository is also available
online.
3.9.2: - cf-serverd: Do not close connection when file does not exist. (CFE-2532) - Allow editing fields in lines longer than 4k (CFE-2438) - fix files promise not setting ACL properly on directories. (CFE-616) - Fix connection cache, reuse connections when possible. (CFE-2447) - When no file is provided when calling cf-promises with cf or json output, use promises.cf by default. This restores the previous behavior. (CFE-2375) - Fix apt_get package module incorrectly using interactive mode. - Allow specifying agent maxconnections via def.json (CFE-2461) - Fix crash on Solaris when ps ucb variant is not available. (CFE-2506) - Behaviour change: getvalues(inexistent_var) returns an empty list. Restores 3.7.x and earlier behaviour. (CFE-2479) - Add: Ensure appropriate permissions for SSL files (ENT-760) - ,Fix double logging of output_prefix, and log process name for cf-agent syslog messages. (CFE-2225) - Remove 2k limit on strings length when writing JSON policies (CFE-2383) - Bugfix: getvalues() now behaves correctly for old CFEngine arrays of depth 1. - Behaviour change: it always returns a list now. Even when v is a simple string (i.e. not an iterable) it will return an slist with one element: the value of the string variable. Known issues: getvalues() still misbehaves with double-indexed arrays (seeand. (CFE-2504, CFE-2536) - Fix ttime_range constraint to go higher than 2G as number of seconds. - fix cf-serverd crash when reporting corrupted data. (ENT-3023) - cf-serverd should reload def.json when reloading policy (CFE-2406) - Fixed parsing of host name/IP and port number in cf-runagent (CFE-546) - Fix cumulative() to accept up to 1000 years, like it's documented. - Enterprise: Fix Postgres database migration from 3.8- to 3.9+. (ENT-2760) - fix insert_lines related memory corruption (CFE-2520) - define kept outcome with action warn if edit_line is as expected (CFE-2424) 3.9.1: - Change: Log level for keeping verbatim JSON to DEBUG (CFE-2141) - Change (masterfiles): Definition of from_cfexecd for cf-execd initiated runs (CFE-2386) - Change: Switch processes restart_class logging to verbose - Change: Enable agent component management policy on systemd hosts (CFE-2429) - Change: Remove executable bit from systemd units (CFE-2436) - Change: Require network before cfengine services (CFE-2435) - Fix 'contain' attribute 'no_output' having no effect when the 'commands' promise is using 'module => "true"'. (CFE-2412) - Fix: Services starting or stopping unnecessarily (CFE-2421) - Fix occasional segfault when running getindices() on a variable that has indices of multiple depths (e.g. both "a[x]" and "a[x][y]"). (CFE-2397) - Fix bug in files promise when multiple owners are promised but first one doesn't exist, and improve logging . (CFE-2432) - fix: memory leaks 3.9.0: New features/additions: - Classes promise: allow classes without an expression to default to defined. - Add optional interface parameter to iprange() to match only one interface. - Allow '=' in symbolic modes (Redmine #7826) - Add: FreeBSD ports package module - New package module for FreeBSD pkg package manager. - Add support for adding/removing fifos in policy - Add Linux parsing of /proc/net/ data. - sys.inet - sys.inet6 - sys.interface_data - Data is returned as a data container. - See documentation for more details. (Jira CFE-1991) - sys.ip2iface: new reverse mapping variable from IP to interface name - Namespaced classes can now be specified on the command line. - namespaces can now be passed to cf-runagent -D and --remote-bundles (Redmine #7856) - Add 'cf-full' and 'json-full' to cf-promises '-p' option. They generate output based on the entire policy. The existing 'cf' already behaved this way, and it has now been changed to generate output only for a single file, which the existing 'json' option already does. - New language functions: processexists() and findprocesses() (Redmine #7633) - Implement new regex_replace() function. (Redmine #7346) - Add log rotation policy for state/classes.jsonl log. (Redmine #7951) - Added collect_vars utility bundle to stdlib - Intoduce report_class_log attribute to body agent common. (Redmine #7951) - Add standard_services service_method allowing for explicit usage - cf-promises --show-vars can now show JSON variables. - Add json_pipe mode to mapdata(), which allows piping a JSON container to an external program for manipulation and receiving JSON back. The `jq` tool is a good example where this mode can be useful. A corresponding `$(def.jq)` variable has also been added with a default path to this tool. See documentation for mapdata() for more information and examples. (Jira CFE-2071) - behaviour change: "true" is always defined and "false" is never defined in a context expression. - Add: nimclient package module for AIX This module provides basic functionality for using nimclient as a means to ensure packages are either present or absent. It does not support listing package updates available or provide any special caching. - Add callstack_callers() and callstack_promisers() functions. - Log variable definitions in debug output. (Redmine #7137) - Add: Memory information to host info report (Jira CFE-1177) - In Mustache templates, one can now use `` and `` tags to iterate over the top level element in a container. (Redmine #6545) - Add network_connections() function that parses /proc/net - Provide new -w argument to override the workdir for testing - New feature: Emails sent by cf-execd can be filtered to get rid of emails for unwanted log messages. The attributes mailfilter_include and mailfilter_exclude in body executor control control what to include. See documentation for cf-execd for more information. (Jira CFE-2283) - Add: file_make_mustache bundle to render mustache templates - Add '-n' flag to cf-key to avoid host name lookups. - cf-agent, cf-execd, cf-promises, cf-runagent and cf-serverd honor multiple -D, -N and -s arguments (Redmine #7191) - Add "canonify" mode to mapdata(). - Add: printfile bodies to stdlib - Add: New `results` classes body [] (Redmine #7418, #7481) - Implement cf-runagent --remote-bundles and cf-serverd "bundle" access promise. (Redmine #7581) - Add commands promise arglist attribute, augmenting args attribute. - It's now possible to reference variables in inline JSON, for example: `mergedata('[ thing, { "mykey": otherthing[123] } ]')`. `thing` and `otherthing[123]` will be resolved as variables, since they are unquoted. See the documentation for more details. (Redmine #7871) - Allow inline JSON to be used in the following function calls: - data_expand() - difference() - every() - filter() - format() - getindices() - getvalues() - grep() - intersection() - join() - length() - makerule() - mapdata() - maplist() - mean() - mergedata() - none() - nth() - parsejson() - product() - regarray() - reglist() - reverse() - shuffle() - some() - sort() - storejson() - string_mustache() - sublist() - sum() - unique() - url_get() - variance() For example: `mergedata('[ "thing", { "mykey": "myvalue" } ]')` See the documentation for more details. (Jira CFE-2253) - Add: edit_line contains_literal_string to stdlib - Add body agent control select_end_match_eof option. (Jira CFE-2390) Changes: - Change: classesmatching(): order of classes changed - Change: getindices(), getvalues(), variablesmatching(), maparray(): order of variables returned has changed - Change: set_quoted_values uses bundle scoped classes - Change: set_config_values uses bundle scoped classes - Change: set_variable_values uses bundle scoped classes - Change: set_config_values_matching uses bundle scoped classes - Change: manage_variable_values_ini uses bundle scoped classes - Change: set_line_based should use bundle scoped classes (Jira CFE-1959) - getvalues() will now return a list also for data containers, and will descend recursively into the containers. (Redmine #7116) - Change: Improve git drop user support - Use new package promise as default package promise implementation. (Jira CFE-2332) - Don't follow symbolic links when copying extended attributes. - When a bodydefault:_ body is defined, it will be used by all promises of type unless another body is explicitly used. - cf-serverd no longer appends "-I -Dcfruncommand" to cfruncommand, this has to be done manually in masterfiles body server control. (Redmine #7732) - eval() function arguments mode and options are now optional. - sort() function argument mode is now optional. - Change: returnszero() no longer outputs the output of a command. The output can be seen by enabling info mode (-I). - cfruncommand is not executed under shell. (Redmine #7409) - Remove: Apache CGI module - Change: Make maxbytes arg of readjson() and readyaml() optional - Classes matching agent control's abortclasses are now printed before exit, even if they are defined in common bundles. Previously the regex (in abortclasses) that matched the class was printed if the class was defined in a common bundle, but the class itself was printed if it was defined in an agent bundle. With this change, the defined class that caused the abort is always printed. - Remove: Support for email settings from augments_file (Redmine #7682) - Change: set_variable_values_ini uses bundle scoped classes - findfiles() now skips relative paths. (Redmine #7981) - Clients connections using non TLS protocol are rejected by default. . (Jira CFE-2339) - Change: Policy files specified in the "inputs" section of def.json will no longer be auto-loaded. One has to refer to the which are using the "inputs" field inside def.json. (Redmine #7961) - Change: Seperate binary details from policy update (Redmine #7662) - Add guard for binary upgrade during bootstrap (Redmine #7861) - Change: Modernize pkg module and package_method - Remove: Userdir apache module - filestat(path, "linktarget") now follows non-absolute links and returns full path of target. This introduces a change in behaviour. Here is an example: $ ls -l /bin/sh lrwxrwxrwx 1 root root 4 Jun 4 2015 /bin/sh -> dash Previously the filestat function would return "dash", and would also log an error that the file can not be read. Now it will return "/bin/dash" (or the final destination if it happens that /bin/dash is also a symlink). You can still get the previous behaviour by using filestat(path, "linktarget_shallow"). (Redmine #7404) - Define (bootstrap|failsafe)_mode during update.cf when triggerd from failsafe.cf (Redmine #7861) - Behavior change: The promise string of a processes promise now matches just the command line of each process instead of the line that is output by ps. This was done to reduce fragmentation between platforms, since ps is a very nonstandardized tool. (Jira CFE-2161) - Allowed namespace names made more strict, to disallow namespaces that cannot be reached. (Redmine #7903) - Behavior change: When using readintlist(), readreallist() or readstringlist(), parsing an empty file will no longer result in a failed function call, but instead an empty list. Failure to open the file will still result in a failed function call. - insert_lines is no longer implicitly matching EOF as end of the region if 'select_end' pattern is not matched . (Jira CFE-2263) - EOF is matched as an end of the region in edit_line promises only if 'select_end_match_eof' parameter is true. (Jira CFE-2263) Bug fixes: - Upgrade CFEngine dependencies to the following versions: - Curl 7.48.0 - libxml2 2.9.4 - LMDB 0.9.18 - OpenLDAP 2.4.44 - OpenSSL 1.0.2h - PCRE 8.38 (Jira ENT-2720) - Upgrade dependencies to latest minor versions. For Community / Enterprise: For Enterprise: - Fix bug which sometimes misparses user names in ps output. - Fix: Problem with git not dropping privileges soon enough - Allow def.json up to 5MB instead of 4K. - It is possible to edit the same value in multiple regions of one file. (Redmine #7460) - CFEngine on Windows no longer truncates log messages if the program in question is killed halfway through. - Fixed a bug which caused def.json not being able to define classes based on other hard classes. (Jira CFE-2333) - Change: Tighten Enterprise hub permissions (Jira ENT-2708) - Fix a regression which would sometimes cause "Permission denied" errors on files inside directories with very restricted permissions. (Redmine #7808) - Fix use-after-free in ArrayMap and HashMap (Redmine #7952) - Package repositories are no more hit every time package promise is evaluated on SUSE. - Fix a bug which sometimes caused package promises to be skipped with "XX Another cf-agent seems to have done this since I started" messages in the log, most notably in long running cf-agent runs (longer than one minute). (Redmine #7933) - TTY detection should be more reliable. (Redmine #7606) - cf-promises -p cf now produces valid cfengine code (Redmine #7956) - Fix ps options for FreeBSD to check processes only in current host and not in jails - cf-runagent now properly supports multiple -D or -s arguments (Redmine #7191) - Fix: Work around impaired class definition from augments (Jira CFE-2333) - Fix "No such file or directory" LMDB error on heavily loaded hosts. (Jira CFE-2300) - Check for empty server response in RemoteDirList after decryption (Redmine #7908) - Small performance optimization when cf-execd scans emails before sending. - Fix handling of closed connections during transactions (Redmine #7926) - The core ps parsing engine used for processes promises has been rewritten from scratch, and should be more robust than before. (Jira CFE-2161) - Fix the lexer which could not handle empty newline(s) before a ```@endif```. - groupexists() no longer fails to detect a group name starting with a digit. (Jira CFE-2351) - Fix HP-UX specific bug that caused a lot of log output to disappear. - Fix unresolved variable (Redmine #7931) - Change: Suppress standard services noise on SUSE (Redmine #6968) - Reduce verbosity of yum package module (Redmine #7485) - cf-runagent: Allow connections to localhost instead of failing silently. - Show errors regarding failure to copy extended attributes when doing a local file copy. Errors could happen when copying across two different mount points where the support for extended attributes is different between the mount points. - Fix classes being set because of hash collision in the implementation. (Redmine #7912) - fix build failure on FreeBSD 7.1 (Redmine #7415) - Improve logging when managing setuid/setgid - Reduce verbosity of apt_get package module (Redmine #7485) - packagesmatching() and packageupdatesmatching() should work when new package promise is used. (Jira CFE-2246) - Fix bug which could render host unable to recover from a syntax error, even if failsafe.cf was utilized. This could happen if the file containing the syntax error was specified in the def.json special file. (Redmine #7961) - Prevent crash in cf-execd email code when policy server is not set. - In case of networking error, assume checksum is wrong - Fix two cases where action_policy warn still produces errors (Redmine #7274) - Fix bad option nlwp to vzps on Proxmox / OpenVZ. (Redmine #6961) - @if minimum_version now correctly ignores lines starting with '@' (Redmine #7862) - No longer hang when changing permissions/ownership on fifos (Redmine #7030) - readfile() and read*list() should print an error if they fail to read file. (Redmine #7702) - The isvariable() function call now correctly accepts all array variables when specified inline. Previously it would not accept certain special characters, even though they could be specified indirectly by using a variable to hold it. (Redmine #7088) - Fix file descriptor leak when there are network errors. - Improve robustness of process table parsing on Solaris. (Jira CFE-2161) - Installing packages containing version numbers using yum now works correctly. (Redmine #7825) - Parse def.json vars, classes and inputs from the C code. This fixes a bug where certain entries in this file would be parsed too late to have any effect on the evaluation. (Redmine #7453, #7615) - Change package modules permissions on hub package so that hub can execute package promises. (Redmine #7602) - Fix: CFEngine choking on standard services (Jira CFE-2086) - Fix: cf-upgrade on SUSE - Fix: Stop cfengine choking on systemctl output (Jira CFE-2806) - storage: Properly initialize the list of current mounts (Jira CFE-1803) - Fix bug which caused empty emails to be sent from cf-execd if there was no previous output log and the new log was fully filtered by email filters. (Jira ENT-2739) - Don't send empty emails for logs where everything is filtered. (Jira ENT-2739) - Fix intermittent error message of type: "error: Process table lacks space for last columns: " (Jira CFE-2371) - Be less verbose if a network interface doesn't have a MAC address. (Jira CFE-1995) 3.8.2: Fixes: - Update library dependencies to latest version. Libraries upgraded: - curl 7.47.0 - LMDB 0.9.18 - MySQL 5.1.72 - OpenLDAP 2.4.44 - OpenSSL 1.0.2g - PostgreSQL 9.3.11 - Redis 3.0.7 - rsync 3.1.2 PHP was kept at 5.6.17 because of problems with the 5.6.19 version. - Reduce verbosity of apt_get package module (Redmine #7485) - Reduce verbosity of yum package module (Redmine #7485) - The isvariable() function call now correctly accepts all array variables when specified inline. Previously it would not accept certain special characters, even though they could be specified indirectly by using a variable to hold it. (Redmine #7088) - Don't follow symbolic links when copying extended attributes. - Fix a bug which sometimes caused package promises to be skipped with "XX Another cf-agent seems to have done this since I started" messages in the log, most notably in long running cf-agent runs (longer than one minute). (Redmine #7933) - Fix bug which could render host unable to recover from a syntax error, even if failsafe.cf was utilized. This could happen if the file containing the syntax error was specified in the def.json special file. (Redmine #7961) - Change: Policy files specified in the "inputs" section of def.json will no longer be auto-loaded. One has to refer to the $(def.augments_inputs) variable in the policy (the standard masterfiles policies include this by default). This only affects installations which are not based on the standard masterfiles, and which are using the "inputs" field inside def.json. (Redmine #7961) - Fix file descriptor leak when there are network errors. - Fix cf-serverd error messages with classic protocol clients (Redmine #7818) - Installing packages containing version numbers using yum now works correctly. (Redmine #7825) - Fix ps options for FreeBSD to check processes only in current host and not in jails - fix build failure on FreeBSD 7.1 (Redmine #7415) - Show errors regarding failure to copy extended attributes when doing a local file copy. Errors could happen when copying across two different mount points where the support for extended attributes is different between the mount points. - Fix classes being set because of hash collision in the implementation. (Redmine #7912) - Allow def.json up to 5MB instead of 4K. - Fix a regression which would sometimes cause "Permission denied" errors on files inside directories with very restricted permissions. (Redmine #7808) - Change: Suppress standard services noise on SUSE (Redmine #6968) Changes: - Change: classesmatching(): order of classes changed 3.8.1: Changes: - Upgrade CFEngine dependencies to the following versions: - OpenSSL 1.0.2e - PCRE 8.38 - libxml2 2.9.3 - OpenLDAP 2.4.43 - libcurl 7.46.0 - Upgrade LMDB to version 0.9.17. (Redmine #7879) Bug fixes: - @if minimum_version now correctly ignores lines starting with '@' (Redmine #7862) - Add guard for binary upgrade during bootstrap (Redmine #7861) - Namespaced classes can now be specified on the command line. - Fix bad option nlwp to vzps on Proxmox / OpenVZ. (Redmine #6961) - Fix two cases where action_policy warn still produces errors (Redmine #7274) - Parse def.json vars, classes and inputs from the C code. This fixes a bug where certain entries in this file would be parsed too late to have any effect on the evaluation. (Redmine #7453, #7615) - Fix HP-UX specific bug that caused a lot of log output to disappear. - Check for empty server response in RemoteDirList after decryption (Redmine #7908) - getvalues() will now return a list also for data containers, and will descend recursively into the containers. (Redmine #7116) - Define (bootstrap|failsafe)_mode during update.cf when triggerd from failsafe.cf (Redmine #7861) 3.8.0: New features/additions: - New feature: Bodies can now inherit attribute values from other bodies by specifying "inherit_from" with the name of the body to inherit from, plus any arguments it accepts. For example: body classes myclasses { inherit_from => classes_generic("myname"); } (Redmine #4309) - Add url_get() function. (Redmine #6480) - Add @if feature() syntax @if feature work like @if minimum_version but allows distinguishing between features choosen at compile time. - Extend module protocol to create persistent classes. To use it, have the module print a line with "^persistence= " before printing any class names. "persistence=0" goes back to non- persistent classes. (Redmine #7302) - Add: New `results` classes body (Redmine #7418) - Add: Debug reports in cfe_internal_cleanup_agent_reports - Add: Path to svcprop in stdlib - Add: masterfiles-stage script to contrib - Whitespace is now allowed in class expressions for readability, between class names and operators. (Redmine #7152) Changes: - Change: Clarify bootstrap/failsafe reports - Change: Improve in-line docs for internal log maintenance - Change: Improve efficiency and debug reports (Redmine #7527) - Remove: 3.5 support from masterfiles policy framework - Long promiser strings with multiple lines are now abbreviated in logs. (Redmine #3964) - Change: Reunify Version based policy split - Change: Seperate binary details from policy update (Redmine #7662) - Remove /var/cfengine/cf3. .runlog. (Redmine #6957) - Change: sys.libdir and sys.local_libdir to non version specific path - sys.libdir now resolves to $(sys.inputdir)/lib - sys.local_libdir now resolves to lib (Redmine #7559) - Moved the following files to /var/cfengine/log/: - /var/cfengine/promise_summary.log - /var/cfengine/cfagent. .log - Change: Seperate binary details from policy update (Redmine #7662) - Remove: Support for email settings from augments_file (Redmine #7682) Bug fixes: - It is possible to edit the same value in multiple regions of one file. (Redmine #7460) - Change package modules permissions on hub package so that hub can execute package promises. (Rednime #7602) (Redmine #7602) - Fix exporting CSV reports through HTTPS. (Redmine #7267) - cf-agent, cf-execd, cf-promises, cf-runagent and cf-serverd honor multiple -D, -N and -s arguments (Redmine #7191) - readfile() and read*list() should print an error if they fail to read file. (Redmine #7702) - No longer hang when changing permissions/ownership on fifos (Redmine #7030) - Fix broken HA policy for 3rd disaster-recovery node. - Fix: Policy errors for 3.5 and 3.6 - Mustache templates: Fix key when value is not a primitive. The old behavior, when iterating across a map or array of maps, was to abort if the key was requested with . The new behavior is to always replace with either the key name or the iteration position in the array. An error is printed if is used outside of a Mustache iteration section. - Fix build with musl libc. (Redmine #7455) - Fixed a bug which could cause daemons to not to be killed correctly when upgrading or manually running "service cfengine3 stop". (Redmine #7193) - Fix daemons not restarting correctly on upgrade on AIX. - Package promise: Fix inability to install certain packages with numbers. (Redmine #7421) - Redmine #6027 Directories should no more be changed randomly into files. (Redmine #6027) - Improve cf-serverd's lock contention because of getpwnam() call. (Redmine #7643) (Redmine #7643) - action_policy "warn" now correctly produces warnings instead of various other verbosity levels. (Redmine #7274) - If there is an error saving a mustache template file it is now logged with log-level error (was inform). - The JSON parser now supports unquoted strings as keys. - Reduce malloc() thread contention on heavily loaded cf-serverd, by not exiting early in the logging function, if no message is to be printed. (Redmine #7624) (Redmine #7624) - Fix a bug which caused daemons not to be restarted on upgrade. (Redmine #7528) - Include latest security updates for dependencies. - Fixed bug which would cause bff and depot packages not to run package scripts on removal. (Redmine #7193) - Fix upgrade causing error message under systemd because of open ports. - Fixed several bugs which prevented CFEngine from loading libraries from the correct location. This affected several platforms. (Redmine #6708) - Legacy package promise: Result classes are now defined if the package being promised is already up to date. (Redmine #7399) - failsafe.cf will be created when needed. (Redmine #7634) (Redmine #7634) - If file_select.file_types is set to symlink and there are regular files in the scanned directory, CFEngine no longer produces an unneccessary error message. (Redmine #6996) - Fix 'AIX_PREINSTALL_ALREADY_DONE.txt: cannot create' error message on AIX. - Fix package promise not removing dependant packages. (Redmine #7424) - Fix: Solaris packages no longer contain duplicate library files, but instead symlinks to them. (Redmine #7591) - Fix select_class not setting class when used in common bundle with slist. (Redmine #7482) - Fix "@endif" keyword sometimes being improperly processed by policy parser. (Redmine #7413) - Fix noise from internal policy to upgrade windows agents (Redmine #7456) - cfruncommand now works if it contains spaces, with the TLS protocol. (Redmine #7405) - Fix warning "Failed to parse csv file entry" with certain very long commands promises. (Redmine #7400) - CFEngine no longer erronously passes -M to useradd on HP-UX. (Redmine #6734) - cf-monitord no longer complains about missing thermal zone files. (Redmine #7238) - systemd is now detected correctly if it is a symlink (Redmine #7297) - TTY detection should be more reliable. (Redmine #7606) (Redmine #7606) 3.7.3 Fixes: - Reduce verbosity of yum package module (Redmine #7485) - Reduce verbosity of apt_get package module (Redmine #7485) - Upgrade dependencies to latest patch versions. Upgraded libraries: - curl 7.47.0 - libxml2 2.9.3 - LMDB 0.9.18 - MySQL 5.1.72 - OpenLDAP 2.4.44 - OpenSSL 1.0.2g - PCRE 8.38 - PostgreSQL 9.3.11 - Redis 2.8.24 - rsync 3.1.2 PHP was kept at 5.6.17 because of problems with the 5.6.19 version. - parse def.json vars, classes, and inputs in C (Redmine #7453) - Namespaced classes can now be specified on the command line. - getvalues() will now return a list also for data containers, and will descend recursively into the containers. (Redmine #7116) - @if minimum_version now correctly ignores lines starting with '@' (Redmine #7862) - Fix definition of classes from augments file - Don't follow symbolic links when copying extended attributes. - Fix ps options for FreeBSD to check processes only in current host and not in jails - Fix cf-serverd error messages with classic protocol clients (Redmine #7818) - Change: Suppress standard services noise on SUSE (Redmine #6968) - The isvariable() function call now correctly accepts all array variables when specified inline. Previously it would not accept certain special characters, even though they could be specified indirectly by using a variable to hold it. (Redmine #7088) - Show errors regarding failure to copy extended attributes when doing a local file copy. Errors could happen when copying across two different mount points where the support for extended attributes is different between the mount points. - Fix bad option nlwp to vzps on Proxmox / OpenVZ. (Redmine #6961) - Fix file descriptor leak when there are network errors. - Fix a regression which would sometimes cause "Permission denied" errors on files inside directories with very restricted permissions. (Redmine #7808) - Check for empty server response in RemoteDirList after decryption (Redmine #7908) - Allow def.json up to 5MB instead of 4K. - Add guard for binary upgrade during bootstrap (Redmine #7861) - Fix HP-UX specific bug that caused a lot of log output to disappear. - Fix a bug which sometimes caused package promises to be skipped with "XX Another cf-agent seems to have done this since I started" messages in the log, most notably in long running cf-agent runs (longer than one minute). (Redmine #7933) - Define (bootstrap|failsafe)_mode during update.cf when triggerd from failsafe.cf (Redmine #7861) - Fix two cases where action_policy warn still produces errors (Redmine #7274) - Fix classes being set because of hash collision in the implementation. (Redmine #7912) - fix build failure on FreeBSD 7.1 (Redmine #7415) - Installing packages containing version numbers using yum now works correctly. (Redmine #7825) Changes: - Change: classesmatching(): order of classes changed 3.7.3 Fixes: - Reduce verbosity of yum package module (Redmine #7485) - Reduce verbosity of apt_get package module (Redmine #7485) - Upgrade dependencies to latest patch versions. Upgraded libraries: - curl 7.47.0 - libxml2 2.9.3 - LMDB 0.9.18 - MySQL 5.1.72 - OpenLDAP 2.4.44 - OpenSSL 1.0.2g - PCRE 8.38 - PostgreSQL 9.3.11 - Redis 2.8.24 - rsync 3.1.2 PHP was kept at 5.6.17 because of problems with the 5.6.19 version. - parse def.json vars, classes, and inputs in C (Redmine #7453) - Namespaced classes can now be specified on the command line. - getvalues() will now return a list also for data containers, and will descend recursively into the containers. (Redmine #7116) - @if minimum_version now correctly ignores lines starting with '@' (Redmine #7862) - Fix definition of classes from augments file - Don't follow symbolic links when copying extended attributes. - Fix ps options for FreeBSD to check processes only in current host and not in jails - Fix cf-serverd error messages with classic protocol clients (Redmine #7818) - Change: Suppress standard services noise on SUSE (Redmine #6968) - The isvariable() function call now correctly accepts all array variables when specified inline. Previously it would not accept certain special characters, even though they could be specified indirectly by using a variable to hold it. (Redmine #7088) - Show errors regarding failure to copy extended attributes when doing a local file copy. Errors could happen when copying across two different mount points where the support for extended attributes is different between the mount points. - Fix bad option nlwp to vzps on Proxmox / OpenVZ. (Redmine #6961) - Fix file descriptor leak when there are network errors. - Fix a regression which would sometimes cause "Permission denied" errors on files inside directories with very restricted permissions. (Redmine #7808) - Check for empty server response in RemoteDirList after decryption (Redmine #7908) - Allow def.json up to 5MB instead of 4K. - Add guard for binary upgrade during bootstrap (Redmine #7861) - Fix HP-UX specific bug that caused a lot of log output to disappear. - Fix a bug which sometimes caused package promises to be skipped with "XX Another cf-agent seems to have done this since I started" messages in the log, most notably in long running cf-agent runs (longer than one minute). (Redmine #7933) - Define (bootstrap|failsafe)_mode during update.cf when triggerd from failsafe.cf (Redmine #7861) - Fix two cases where action_policy warn still produces errors (Redmine #7274) - Fix classes being set because of hash collision in the implementation. (Redmine #7912) - fix build failure on FreeBSD 7.1 (Redmine #7415) - Installing packages containing version numbers using yum now works correctly. (Redmine #7825) Changes: - Change: classesmatching(): order of classes changed 3.7.2: Bug fixes: - readfile() and read*list() should print an error if they fail to read file. (Redmine #7702) - Fix 'AIX_PREINSTALL_ALREADY_DONE.txt: cannot create' error message on AIX. - If there is an error saving a mustache template file it is now logged with log-level error (was inform). - Change: Clarify bootstrap/failsafe reports - Fixed several bugs which prevented CFEngine from loading libraries from the correct location. This affected several platforms. (Redmine #6708) - If file_select.file_types is set to symlink and there are regular files in the scanned directory, CFEngine no longer produces an unneccessary error message. (Redmine #6996) - Fix: Solaris packages no longer contain duplicate library files, but instead symlinks to them. (Redmine #7591) - cf-agent, cf-execd, cf-promises, cf-runagent and cf-serverd honor multiple -D, -N and -s arguments (Redmine #7191) - Fix "@endif" keyword sometimes being improperly processed by policy parser. (Redmine #7413) - It is possible to edit the same value in multiple regions of one file. (Redmine #7460) - Fix select_class not setting class when used in common bundle with slist. (Redmine #7482) - Fix broken HA policy for 3rd disaster-recovery node. - Directories should no more be changed randomly into files. (Redmine #6027) - Include latest security updates for 3.7. - Reduce malloc() thread contention on heavily loaded cf-serverd, by not exiting early in the logging function, if no message is to be printed. (Redmine #7624) - Improve cf-serverd's lock contention because of getpwnam() call. (Redmine #7643) - action_policy "warn" now correctly produces warnings instead of various other verbosity levels. (Redmine #7274) - Change: Improve efficiency and debug reports (Redmine #7527) - Change package modules permissions on hub package so that hub can execute package promises. (Redmine #7602) - No longer hang when changing permissions/ownership on fifos (Redmine #7030) - Fix exporting CSV reports through HTTPS. (Redmine #7267) - failsafe.cf will be created when needed. (Redmine #7634) - Mustache templates: Fix key when value is not a primitive. The old behavior, when iterating across a map or array of maps, was to abort if the key was requested with . The new behavior is to always replace with either the key name or the iteration position in the array. An error is printed if is used outside of a Mustache iteration section. - Legacy package promise: Result classes are now defined if the package being promised is already up to date. (Redmine #7399) - TTY detection should be more reliable. (Redmine #7606) Masterfiles: - Add: Path to svcprop in stdlib - Add: New `results` classes body [] (Redmine #7418, #7481) - Remove: Support for email settings from augments_file (Redmine #7682) 3.7.1: Bug fixes: - Fix daemons not restarting correctly on upgrade on AIX. (Redmine #7550) - Fix upgrade causing error message under systemd because of open ports. - Fix build with musl libc. (Redmine #7455) - Long promiser strings with multiple lines are now abbreviated in logs. (Redmine #3964) - Fixed a bug which could cause daemons to not to be killed correctly when upgrading or manually running "service cfengine3 stop". (Redmine #7193) - Package promise: Fix inability to install certain packages with numbers. - Fix package promise not removing dependent packages. (Redmine #7424) - Fix warning "Failed to parse csv file entry" with certain very long commands promises. (Redmine #7400) - Fix misaligned help output in cf-hub. (Redmine #7273) - Augmenting inputs from the augments_file (Redmine #7420) - Add support for failover to 3rd HA node located outside cluster. - Upgrade all dependencies for patch release. - Fix a bug which caused daemons not to be restarted on upgrade. (Redmine #7528) 3.7.0: New features: - New package promise implementation. The syntax is much simpler, to try it out, check out the syntax: packages: "mypackage" policy => "absent/present", # Optional, default taken from common control package_module => apt_get, # Optional, will only match exact version. May be # "latest". version => "32.0", # Optional. architecture => "x86_64"; - Full systemd support for all relevant platforms - New classes to determine whether certain features are enabled: * feature_yaml * feature_xml For the official CFEngine packages, these are always enabled, but packages from other sources may be built without the support. - New readdata() support for generic data input (CSV, YAML, JSON, or auto) - YAML support: new readyaml() function and in readdata() - CSV support: new readcsv() function and in readdata() - New string_mustache() function - New data_regextract() function - eval() can now be called with "class" as the "mode" argument, which will cause it to return true ("any") if the calculated result is non-zero, and false ("!any") if it is zero. - New list_ifelse() function - New mapjson() function as well as JSON support in maparray(). - filestat() function now supports "xattr" argument for extended attributes. - "ifvarclass" now has "if" as an alias, and "unless" as an inverse alias. - Ability to expand JSON variables directory in Mustache templates: Prefix the name with '%' for multiline expansion, '$' for compact expansion. - Ability to expand the iteration *key* in Mustache templates with @ - Canonical JSON output: JSON output has reliably sorted keys so the same data structure will produce the same JSON every time. - New "@if minimum_version(x.x)" syntax in order to hide future language improvements from versions that don't understand them. - compile time option (--with-statedir) to override the default state/ directory path. - Fix error messages/ handling in process signalling which no longer allowed any signals to fail silently - Also enable shortcut keyword for cf-serverd classic protocol, eg to simplify the bootstrap process for clients that have different sys.masterdir settings (Redmine #3697) - methods promises now accepts the bundle name in the promiser string, as long as it doesn't have any parameters. - In a services promise, if the service_method bundle is not specified, it defaults to the promiser string (canonified) with "service_" as a prefix. The bundle must be in the same namespace as the promise. - inline JSON in policy files: surrounding with parsejson() is now optional *when creating a new data container*. - New data_expand() function to interpolate variables in a data container. - Add configurable network bandwidth limit for all outgoing connections ("bwlimit" attribute in "body common control") . To enforce it in both directions, make sure the attribute is set on both sides of the connection. - Secure bootstrap has been facilitated by use of "cf-agent --boostrap HUB_ADDRESS --trust-server=no" - Implement new TLS-relevant options (Redmine #6883): - body common control: tls_min_version - body server control: allowtlsversion - body common control: tls_ciphers - body server control: allowciphers (preexisting) Changes: - Improved output format, less verbose, and messages are grouped. - cf-execd: agent_expireafter default was changed to 120 minutes (Redmine #7113) - All embedded databases are now rooted in the state/ directory. - TLS used as default for all outgoing connections. - process promise now reports kept status instead of repaired if a signal is not sent, even if the restart_class is set. The old behavior was to set the repaired status whenever the process was not running. (Redmine#7216). - Bootstrapping requires keys to be generated in advance using cf-key. - Disable class set on reverse lookup of interfaces IP addresses. (Redmine #3993, Redmine #6870) - Define a hard class with just the OS major version on FreeBSD. - Abort cf-agent if OpenSSL's random number generator can't be seeded securely. - Masterfiles source tarball now installs using the usual commands "./configure; make install". - Updated Emacs syntax highlighting template to support the latest syntax enhancements in 3.7. Deprecations: - Arbitrary arguments to cfruncommand (using "cf-runagent -o") are not acceptable any more. (Redmine #6978) - 3.4 is no longer supported in masterfiles. Bug fixes: - Fix server common bundles evaluation order (Redmine#7211). - Limit LMDB disk usage by preserving sparse areas in LMDB files (Redmine#7242). - Fixed LMDB corruption on HP-UX 11.23. (Redmine #6994) - Fixed insert_lines failing to converge if preserve_block was used. (Redmine #7094) - Fixed init script failing to stop/restart daemons on openvz/lxc hosts. (Redmine #3394) - rm_rf_depth now deletes base directory as advertised. (Redmine #7009) - Refactored cf-agent's connection cache to properly differentiate hosts using all needed attributes like host and port. (Redmine #4646) - Refactored lastseen database handling to avoid inconsistencies. (Redmine #6660) - cf-key --trust-key now supports new syntax to also update the lastseen database, so that clients using old protocol will trust the server correctly. - Fixed a bug which sometimes caused an agent or daemon to kill or stop itself. (Redmine #7075, #7244) - Fixed a bug which made it difficult to kill CFEngine daemons, particularly cf-execd. (Redmine #6659, #7193) - Fixed a bug causing systemd not to be detected correctly on Debian. (Redmine #7297) - "cf-promises -T" will now correctly report the checked out commit, even if you haven't checked out a Git branch. (Redmine #7332) - Reduce verbosity of harmless errors related to socket timeouts and missing thermal zone files. (Redmine #6486 and #7238) Masterfiles: Added: - Support for user specified overring of framework defaults without modifying policy supplied by the framework itself (see example_def.json) - Support for def.json class augmentation in update policy - Run vacuum operation on postgresql every night as a part of maintenance. - Add measure_promise_time action body to lib (3.5, 3.6, 3.7, 3.8) - New negative class guard `cfengine_internal_disable_agent_email` so that agent email can be easily disabled by augmenting def.json Changed: - Relocate def.cf to controls/VER/ - Relocate update_def to controls/VER - Relocate all controls to controls/VER - Only load cf_hub and reports.cf on CFEngine Enterprise installs - Relocate acls related to report collection from bundle server access_rules to controls/VER/reports.cf into bundle server report_access_rules - Re-organize cfe_internal splitting core from enterprise specific policies and loading the appropriate inputs only when necessary - Moved update directory into cfe_internal as it is not generally intended to be modified - services/autorun.cf moved to lib/VER/ as it is not generally intended to be modified - To improve predictibility autorun bundles are activated in lexicographical order - Relocate services/file_change.cf to cfe_internal/enterprise. This policy is most useful for a good OOTB experience with CFEngine Enterprise Mission Portal. - Relocate service_catalogue from promsies.cf to services/main.cf. It is intended to be a user entry. This name change correlates with the main bundle being activated by default if there is no bundlesequence specified. - Reduce benchmarks sample history to 1 day. - Update policy no longer generates a keypair if one is not found. (Redmine: #7167) - Relocate cfe_internal_postgresql_maintenance bundle to lib/VER/ - Set postgresql_monitoring_maintenance only for versions 3.6.0 and 3.6.1 - Move hub specific bundles from lib/VER/cfe_internal.cf into lib/VER/cfe_internal_hub.cf and load them only if policy_server policy if set. - Re-organize lib/VER/stdlib.cf from lists into classic array for use with getvalues Removed: - Diff reporting on /etc/shadow (Enterprise) - Update policy from promise.cf inputs. There is no reason to include the update policy into promsies.cf, update.cf is the entry for the update policy - _not_repaired outcome from classes_generic and scoped_classes generic (Redmine: # 7022) Fixes: - standard_services now restarts the service if it was not already running when using service_policy => restart with chkconfig (Redmine #7258) 3.6.5: Features: - Introduced "systemd" hard class. (Redmine #6995) - Added paths to dtrace, zfs and zpool on FreeBSD in masterfiles. Bug fixes: - Fixed build error on certain RHEL5 and SLES10 setups. (Redmine #6841) - Fixed a bug which caused dangling symlinks not to be removed. (Redmine #6582) - Fixed data_readstringarrayidx function not preserving the order of the array it's producing. (Redmine #6920) - Fixed a bug which sometimes caused CFEngine to kill the wrong daemon if both the host and a container inside the host were running CFEngine. (Redmine #6906) - Made sure the rm_rf_depth bundle also deletes the base directory. (Redmine #7009) - Fixed monitord reporting wrongly on open ports. (Redmine #6926) - Skip adding the class when its name is longer than 1024 characters. Fixed core dump when the name is too large. (Redmine #7013) - Fixed detection of stopped process on Solaris. (Redmine #6946) - Fixed infinite loop (Redmine #6992) plus a couple more minor bugs in edit_xml promises. 3.6.4: Features: - Introduced users promises support on HP-UX platform. - Introduced process promises support on HP-UX platform. Bug fixes: - Fixed bug on FreeBSD which sometimes led to the wrong process being killed (Redmine #2330) - Fixed package version comparison sometimes failing with rpm package manager (Redmine #6807) - Fixed a bug in users promises which would sometimes set the wrong password hash if the user would also be unlocked at the same time. - Fixed a bug on AIX which would occasionally kill the wrong process. - Improved error message for functions that require an absolute path. (Redmine #6877) - Fixed some spelling errors in examples. - Fixed error in out-of-tree builds when building cf-upgrade. - Fixed a bug which would make cf-agent exit with an error if it was built with a custom log directory, and that directory did not exist. - Fixed ordering of evaluating promises when depends_on is used. (Redmine #6484, Redmine #5462) - Skip non-empty directories silently when recursively deleting. (Redmine #6331) - Fix memory exhaustion with list larger than 4994 items. (Redmine # 6672) - Fix cf-execd segfault on IP address detection (Redmine #6905). - Fix hard class detection of RHEL6 ComputeNode (Redmine #3148). 3.6.3 New features: - support for HP-UX 11.23 and later - experimental support for Red Hat Enterprise Linux 7 Bug fixes: - fix getindices on multi-dimensional arrays (Redmine #6779) - fix mustache template method to run in dryrun mode (Redmine #6739) - set mailto and mailfrom settings for execd in def.cf (Redmine #6702) - fix conflation of multi-index entries in arrays (Redmine #6674) - fix promise locking when transferring using update.cf (Redmine #6623) - update JSON parser to return an error on truncation (Redmine #6608) - fix sys.hardware_addresses not expanded (Redmine #6603) - fix opening database txn /var/cfengine/cf_lastseen.lmdb: MDB_READERS_FULL when running cf-keys --show-hosts (Redmine #6602) - fix segfault (Null pointer dereference) when select_end in delete_lines never matches (Redmine #6589) - fix max_file_size => "0" not disabling or allowing any size (Redmine #6588) - fix ifvarclass, with iteration over list, failing when deleting files with time condition (Redmine #6577) - fix classes defined with "or" constraint are never set if any value doesn't evaluate to a scalar (Redmine #6569) - update "mailfrom" default in default policy (Redmine #6567) - fix logrotate ambiguity of filename (Redmine #6563) - fix parsing JSON files (Redmine #6549) - reduce write count activity to /var partition (Redmine #6523) - fix files delete attribute incorrectly triggering promise_kept (Redmine #6509) - update services bundle output related to chkconfig when run in inform mode. (Redmine #6492) - fix Solaris serverd tests (Redmine #6406) - fix broken bechaviour of merging arrays with readstringarray (Redmine #6369) - fix ifelapsed bug with bundle nesting (Redmine #6334) - fix handling cf_null in bundlesequence (Redmine #6119) - fix maparray reading whole input array when using subarray (Redmine #6033) - fix directories being randomly changed to files (Redmine #6027) - update defaults promise type to work with classes (Redmine #5748) - systemd integration in services promises (Redmine #5415) - fix touch attribute ignoring action = warn_only (Redmine #3172) - fix 4KB string limit in functions readfile, string_downcase, string_head, string_reverse, string_length, string_tail, string_upcase (Redmine #2912) 3.6.2 Bug fixes: - don't regenerate software_packages.csv every time (Redmine #6441) - improve verbose message for package_list_command - fix missing log output on AIX (Redmine #6434) - assorted fixes to dirname() esp on Windows (Redmine #4716) - fix package manager detection - fix build issues on FreeBSD - allow copying of dead symbolic links (Redmine #6175) - preserve order in readstringarrayidx (Redmine #6466) - fix passing of unexpanded variable references to arrays (Redmine #5893) - use entries for new {admin,deny}_{ips,hostnames} constraints in the relevant legacy lists (Redmine #6542) - cope with ps's numeric fields overflowing to the right - interpret failing function calls in ifvarclass as class not set (Redmine #6327) - remove unexpanded lists when extending lists (Redmine #6541) - infer start-time of a process from elapsed when needed (Redmine #4094) - fix input range definition for laterthan() function (Redmine #6530) - don't add trailing delimiter when join()'ing lists ending with a null-value (Redmine #6552) - 9999999999 (ten 9s) or higher has been historically used as an upper bound in CFEngine code and policy but because of overflow on 32-bit platforms it caused problems with big numbers. Fixed in two ways: first change all existing policy uses to 999999999 (nine 9s instead of eleven 9s), second fix the C code to not wrap-around in case of overflow, but use the LONG_MAX value (Redmine #6531). - cf-serverd and other daemons no longer reload their configuration every minute if CFEngine is built with an inputs directory outside of the work directory (not the default). (Redmine #6551) 3.6.1 New features: - Introduced Solaris and AIX support into the 3.6 series, with many associated build and bug fixes. Changes: - Short-circuit evaluation of classes promises if class is already set (Redmine #5241) - fix to assume all non-specified return codes are failed in commands promises (Redmine #5986) - cf-serverd logs reconfiguration message to NOTICE (was INFO) so that it's always logged in syslog Bug fixes: - File monitoring has been completely rewritten (changes attribute in files promise), which eliminates many bugs, particularly regarding files that are deleted. Upgrading will keep all monitoring data, but downgrading again will reinitialize the DB, so all files will be reported as if they were new. (Redmine #2917) - $(this.promiser) expands in files promises for 'transformer', 'edit_template', 'copy_from.source', 'file_select.exec_program', 'classes' and 'action' bodies (Redmine #1554, #1496, #3530, #1563) - 'body changes' notifies about disappeared files in file monitoring (Redmine #2917) - Fixed CFEngine template producing a zero sized file (Redmine #6088) - Add 0-9 A-Z _ to allowed context of module protocol (Redmine #6063) - Extend ps command column width and prepend zone name on Solaris - Fixed strftime() function on Solaris when called with certain specifiers. - Fixed users promise bug regarding password hashes in a NIS/NSS setup. - Fixed $(sys.uptime), $(sys.systime) and $(sys.sysday) in AIX. (Redmine #5148, #5206) - Fixed processes_select complaining about "Unacceptable model uncertainty examining processes" (Redmine #6337) - ps command for linux has been changed to cope with big rss values (Redmine #6337) - Address ps -axo shift on FreeBSD 10 and later (Redmine #5667) - methods and services promises respect action_policy => "warn" (Redmine #5924) - LMDB should no longer deadlock if an agent is killed on the hub while holding the DB lock. Note that the change only affects binary packages shipped by CFEngine, since the upstream LMDB project has not yet integrated the change. (Redmine #6013) 3.6.0 Changes: - Changes to logging output - add process name and pid in syslog message (GitHub #789) - cf-serverd logging levels are now more standardised: - INFO logs only failures - VERBOSE logs successful requests as well - DEBUG logs actual protocol traffic. - cf-serverd now logs the relevant client IP address on each message. - Logging contexts to local database (cf_classes.tcdb) has been deprecated. - 'usebundle' promisees are logged for all the bundle promises - output from 'reports' promises has nothing prefixed except 'R: ' - a log line with stack path is generated when the promise type evaluated changes - LMDB (symas.com/mdb) is the default database for local data storage : use version 0.9.9 or later cf-agent --self-diagnostics (-x) is only implemented for TCDB, not for LMDB - port argument in readtcp() and selectservers() may be a service name (e.g. "http", "pop3"). - Enable source file in agent copy_from promises to be a relative path. - file "changes" reporting now reports with log level "notice", instead of "error". - process_results default to AND'ing of set attributes if not specified (Redmine #3224) - interface is now canonified in sys.hardware_mac[interface] to align with sys.ipv4[interface] (Redmine #3418) - cf-promises no longer errors on missing bodies when run without --full-check (-c) - Linux flavor "SUSE" now correctly spelled with all uppercase in variables and class names (Redmine #3734). The "suse" lowercase version is also provided for convenience (Redmine #5417). - $(this.promise_filename) and $(..._dirname) variables are now absolute paths. (Redmine #3839) - including the same file multiple times in 'body control inputs' is not an error - portnumber in body copy_from now supports service names like "cfengine", "pop3" etc, check /etc/services for more. - The failsafe.cf policy, run on bootstrap and in some other unusual cases, has been extracted from C code into libpromises/failsafe.cf - masterfiles - cf_promises_validated is now in JSON format - timestamp key is timestamp (sec since unix epoch) of last time validated - the masterfiles now come from https://github.com/cfengine/masterfiles and are not in the core repository - cf-serverd calls cf-agent with -Dcfruncommand when executing cf-runagent requests - Mark as removed: promise_notkept_log_include, promise_notkept_log_exclude, promise_repaired_log_include, promise_repaired_log_exclude, classes_include, classes_exclude, variables_include, variables_exclude attributes from report_data_select body (syntax is valid but not functional). They have been replaced by the following attributes: promise_handle_include, promise_handle_exclude, metatags_include, metatags_exclude. New features: - New promise type "users" for managing local user accounts. - TLS authentication and fully encrypted network protocol. Additions specific to the new type of connections: - New attribute "allowlegacyconnects" in body server control, which enables serving policy via non-latest cfengine protocol, to the given list of hosts. If the option is absent, it defaults to allow all hosts. To refuse non-TLS connections, specify an empty list. - New attribute "protocol_version" in body copy_from, and body common control, which defines the preferred protocol for outgoing connections.. Allowed values at the moment: "0" or "undefined", "classic" or "1", "latest" or "2". By leaving the copy_from option as undefined the common control option is used, and if both are undefined then classic protocol is used by default. - The new networking protocol uses TLS for authentication, after which all dialog is encrypted within the established TLS session. cf-serverd is still able to speak the legacy protocol with old agents. - The 'skipverify' option in 'body server control' is deprecated and only left for compatibility; it does nothing - cf-serverd does not hang up the connection if some request fails, so that the client can add more requests. - For the connections using the new protocol, all of the paths in bundle server access_rules now differentiate between a directory and a file using the trailing slash. If the path exists then this is auto-detected and trailing slash appended automatically. You have to append a trailing slash manually to an inexistent or symbolic path (e.g. "/path/to/$(connection.ip)/") to force recursive access. - New in 'access' promises for 'bundle server access_rules' - Attributes "admit_ips", "admit_hostnames", "admit_keys", "deny_ips", "deny_hostnames", "deny_keys" - "admit_keys" and "deny_keys" add the new functionality of controlling access according to host identity, regardless of the connecting IP. - For these new attributes, regular expressions are not allowed, only CIDR notation for "admit/deny_ips", exact "SHA=..." strings for "admit/deny_keys", and exact hostnames (e.g. "cfengine.com") or subdomains (starting with dot, e.g. ".cfengine.com") for "admit/deny"_hostnames. Same rules apply to 'deny_*' attributes. - These new constraints and the paths in access_rules, can contain special variables "$(connection.ip)", "$(connection.hostname)", "$(connection.key)", which are expanded dynamically for every received connection. - For connections using the new protocol, "admit" and "deny" constraints in bundle server access_rules are being phased out, preferred attributes are now "admit_ips", "deny_ips", "admit_hostnames", "deny_hostnames", "admit_keys", "deny_keys". - New "shortcut" attribute in bundle server access_rules used to dynamically expand non-absolute request paths. - masterfiles - standard library split: lib/3.5 (compatibility) and lib/3.6 (mainline) - many standard library bundles and bodies, especially packages- and file-related, were revised and fixed - supports both Community and Enterprise - new 'inventory/' structure to provide OS, dmidecode, LSB, etc. system inventory (configured mainly in def.cf) - cf_promises_release_id contains the policy release ID which is the GIT HEAD SHA if available or hash of tree - a bunch'o'bundles to make starting with CFEngine easier: - file-related: file_mustache, file_mustache_jsonstring, file_tidy, dir_sync, file_copy, file_link, file_hardlink, file_empty, file_make - packages-related: package_absent, package_present, package_latest, package_specific_present, package_specific_absent, package_specific_latest, package_specific - XML-related: xml_insert_tree_nopath, xml_insert_tree, xml_set_value, xml_set_attribute - VCS-related: git_init, git_add, git_checkout, git_checkout_new_branch, git_clean, git_stash, git_stash_and_clean, git_commit, git - process-related: process_kill - other: cmerge, url_ping, logrotate, prunedir - New command line options for agent binaries - New options to cf-promises - '--show-classes' and '--show-vars' - '--eval-functions' controls whether cf-promises should evaluate functions - Colorized output for agent binaries with command line option '--color' (auto-enabled if you set CFENGINE_COLOR=1) - New language features - New variable type 'data' for handling of structured data (ie JSON), including supporting functions: - 'data_readstringarray' - read a delimited file into a data map - 'data_readstringarrayidx' - read a delimited file into a data array - 'datastate' - create a data variable with currently set classes and variables - 'datatype' - determine the type of the top element of a container - 'format' - %S can be used to serialize 'data' containers into a string - 'mergedata' - merge two data containers, slists/ilists/rlists, or "classic" arrays into a data container - 'parsejson' - create a data container from a JSON string - 'readjson' - create a data container from a file that contains JSON - 'storejson' - serialize a data container into a string - Most functions operating on lists can also operate on data containers - pass a data container to a bundle with the @(container) notation - the module protocol accepts JSON for data containers with the '%' sigil - Tagging of classes and variables allows annotating of language construct with meta data; supporting functionality: - The module protocol in 'commands' promises has been extended to allow setting of tags of created variables and classes, and the context of created variables - 'getclassmetatags' - returns list of meta tags for a class - 'getvariablemetatags' - returns list of meta tags for a variable - 'body file control' has an 'inputs' attribute to include library files and other dependencies - bundlesequences can be built with bundlesmatching() based on bundle name and tags - New attributes in existing promise types and bodies - New option 'preserve_all_lines' for insert_type in insert_lines promises - Caching of expensive system functions to avoid multiple executions of execresult() etc, can be controlled via cache_system_functions attribute in body common control - New option 'mailsubject' in body executor control allows defining the subject in emails sent by CFEngine - Support for Mustache templates in 'files' promises; use 'template_method' and 'template_data' attributes. Without 'template_data' specified, uses datastate(). - New and improved functions - 'bundlesmatching' - returns list of defined bundles matching a regex and tags - 'canonifyuniquely' - converts a string into a unique, legal class name - 'classesmatching' - returns list of set classes matching a regex and tags - 'eval' - evaluates mathematical expressions; knows SI k, m, g quantifiers, e.g. "100k" - 'findfiles' - list files matching a search pattern; use "**" for recursive searches - 'makerule' - evaluates whether a target file needs to be rebuilt from sources - 'max', 'min' - returns maximum and minimum of the numbers in a container or list (sorted by a 'sort' method) - 'mean' - returns the mean of the numbers in a container or list - 'nth' - learned to look up by key in a data container holding a map - 'packagesmatching' - returns a filtered list of installed packages. - 'readfile' - learned to read system files of unknown size like those in /proc - 'sort' - can sort lexicographically, numerically (int or real), by IP, or by MAC - 'string_downcase', 'string_upcase' - returns the lower-/upper-case version of a string - 'string_head', 'string_tail' - returns the beginning/end of a string - 'string_length' - returns the length of a string - 'string_reverse' - reverses a string - 'string_split' - improved implementation, deprecates 'splitstring' - 'variablesmatching' - returns a list of variables matching a regex and tags - 'variance' - returns the variance of numbers in a list or container - New hard classes - Introduced alias 'policy_server' for context 'am_policy_hub' (the latter will be deprecated) - all the time-based classes have GMT equivalents - New variables - 'sys.bindir' - the location of the CFEngine binaries - 'sys.failsafe_policy_path' - the location of the failsafe policy file - 'sys.inputdir' - the directory where CFEngine searches for policy files - 'sys.key_digest' - the digest of the host's cryptographic key - 'sys.libdir', 'sys.local_libdir' - the location of the CFEngine libraries - 'sys.logdir' - the directory where the CFEngine log files are saved - 'sys.masterdir' - the location of masterfiles on the policy server - 'sys.piddir' - the directory where the daemon pid files are saved - 'sys.sysday' - the number of days since the beginning of the UNIX epoch - 'sys.systime' - the number of seconds since the beginning of the UNIX epoch - 'sys.update_policy_path' - the name of the update policy file - 'sys.uptime' - the number of minutes the host has been online - 'this.promise_dirname' - the name of the file in which the current promise is defined - 'this.promiser_uid' - the ID of the user running cf-agent - 'this.promiser_gid' - the group ID of the user running cf-agent - 'this.promiser_ppid' - the ID of the parent process running cf-agent Deprecations: - 'splitstring' - deprecated by 'string_split' - 'track_value' - 'skipverify' Bug fixes: for a complete list of fixed bugs, see Redmine at https://cfengine.com/dev - various fixes in evaluation and variable resolution - Improve performance of list iteration (Redmine #1875) - Removed limitation of input length to internal buffer sizes - directories ending with "/" are not ignored - lsdir() always return a list now, never a scalar - 'abortclasses' fixed to work in common bundles and other cases - namespaced 'edit_line' bundles now work (Redmine#3781) - lists are interpolated in correct order (Redmine#3122) - cf-serverd reloads policies properly when they change - lots of leaks (memory and file descriptor) fixed 3.5.3 Changes: - Improved security checks of symlink ownership. A symlink created by a user pointing to resources owned by a different user will no longer be followed. - Changed the way package versions are compared in package promises. (Redmine #3314) In previous versions the comparison was inconsistent. This has been fixed, but may also lead to behavior changes in certain cases. In CFEngine 3.5.3, the comparison works as follows: For instance: apache-2.2.31 ">=" "2.2.0" will result in the package being installed. Bug fixes: - fix cf-monitord crash due to incorrect array initialization (Redmine #3180) - fix cf-serverd stat()'ing the file tree every second (Redmine #3479) - correctly populate sys.hardware_addresses variable (Redmine #2936) - add support for Debian's GNU/kfreebsd to build system (Redmine #3500) - fix possible stack corruption in guest_environments promises (Redmine #3552) - work-around hostname trunctation in HP-UX's uname (Redmine #3517) - fix body copy purging of empty directories (Redmine #3429) - make discovery and loading of avahi libraries more robust - compile and packaging fixes for HP-UX, AIX and Solaris - fix fatal error in lsdir() when directory doesn't exist (Redmine #3273) - fix epoch calculation for stime inrange calculation (Redmine #2921) 3.5.2 Bug fixes: - fix delayed abortclasses checking (Redmine #2316, #3114, #3003) - fix maplist arguments bug (Redmine #3256) - fix segfaults in cf-pomises (Redmine #3173, 3194) - fix build on Solaris 10/SmartOS (Redmine #3097) - sanitize characters from /etc/issue in sys.flavor for Debian (Redmine #2988) - Fix segfault when dealing with files or data > 4K (Redmine #2912, 2698) - Don't truncate keys to 126 characters in getindices (Redmine #2626) - files created via log_* actions now have mode 600 (Redmine #1578) - fix wrong log message when a promise is ignored due to 'ifvarclass' not matching - fix lifetime of persistent classes (Redmine #3259) - fix segfault when process_select body had no process_result attribute Default to AND'ed expression of all specified attributes (Redmine #3224) - include system message in output when acl promises fail - fix invocation of standard_services bundle and corresponding promise compliance (Redmine #2869) 3.5.1 Changes: - file changes are logged with log level Notice, not Error - the CFEngine Standard Library in masterfiles/libraries is now split into promise-type specific policy files, and lives in a version-specific directory. This should have no impact on current code, but allows more granular include of needed stdlib elements (Redmine #3044) Bug fixes: - fix recursive copying of files (Redmine #2965) - respect classes in templates (Redmine ##2928) - fix timestamps on Windows (Redmine #2933) - fix non-root cf-agent flooding syslog (Redmine #2980) - fix email flood from cf-execd due to timestamps in agent output (Redmine #3011) - Preserve security context when editing or copying local files (Redmine #2728) - fix path for sys.crontab on redhat systems (Redmine #2553) - prevent incorrect "insert_lines promise uses the same select_line_matching anchor" warning (Redmine #2778) - Fix regression of setting VIPADDRESS to 127.0.0.1 (Redmine #3010) - Fix "changes" promise not receiving status when file is missing (Redmine #2820) - Fix symlinks being destroyed when editing them (Redmine #2363) - Fix missing "promise kept" status for the last line in a file (Redmine #2943) 3.5.0 New features: - classes promises now take an optional scope constraint. - new built-in functions: every, none, some, nth, sublist, uniq, filter - every - none - some - nth - sublist - uniq - filter - classesmatching - strftime - filestat - ifelse - maparray - format - cf-promises flag --parse-tree is replaced by --policy-output-format=, requiring the user to specify the output format (none, cf, json) - cf-promises allows partial check of policy (without body common control) without integrity check; --full-check enforces integrity check - agent binaries support JSON input format (.json file as generated by cf-promises) - cf-key: new options --trust-key/-t and --print-digest/-p - Class "failsafe_fallback" is defined in failsafe.cf when main policy contains errors and failsafe is run because of this - add scope attribute for body classes (Redmine #2013) - Better diagnostics of parsing errors - Error messages from parser now show the context of error - new cf-agent option: --self-diagnostics - new output format, and --legacy-output - warnings for cf-promises. - Enable zeroconf-discovery of policy hubs for automatic bootstrapping if Avahi is present - Support for sys.cpus on more platforms than Linux & HPUX Changes: - parser no longer allows ',' after promiser or promisee. must be either ';' or lval - Make parser output in GCC compatible format the only supported format (remove --gcc-brief-format flag) - Silence license warnings in Enterprise Free25 installations - action_policy => "warn" causes not_kept classes to be set on promise needing repair. - command line option version (-V) now prints a shorter parsable version without graphic - implicit execution of server and common bundles taking arguments is skipped in cf-serverd. - WARNING: option --policy-server removed, require option to --bootstrap instead - process promises don't log if processes are out of range unless you run in verbose mode - reports promises are now allowed in any context (Redmine #2005) - cf-report has been removed - cf-execd: --once implies --no-fork - Version info removed from mail subject in the emails sent by cf-execd. The subject will only contain "[fqname/ipaddress]" instead of "communnity/nova [fqname/ipaddress]" Please change your email filters accordingly if necessary. - "outputs" promise type is retired. Their semantics was not clear, and the functionality is better suited for control body setting, not a promise. - Tokyo Cabinet databases are now automatically checked for correctness during opening. It should prevent a number of issues with corrupted TC databases causing binaries to hang. - Improved ACL handling on Windows, which led to some syntax changes. We now consistently use the term "default" to describe ACLs that can be inherited by child objects. These keywords have received new names: acl_directory_inherit -> acl_default specify_inherit_aces -> specify_default_aces The old keywords are deprecated, but still valid. In addition, a new keyword "acl_inherit" controls inheritance behavior on Windows. This feature does not exist on Unix platforms. (Redmine #1832) - Networking code is moved from libpromises to its own library, libcfnet. Work has begun on making the API more sane and thread-safe. Lots of legacy code was removed. - Add getaddrinfo() replacement in libcompat (borrowed from PostgreSQL). - Replace old deprecated and non thread-safe resolver calls with getaddrinfo() and getnameinfo(). - Hostname2IPString(), IPString2Hostname() are now thread-safe, and are returning error when resolution fails. - Running cf-execd --once now implies --no-fork, and also does not wait for splaytime to pass. - execresult(), returnszero() and commands promises no longer requires the first word word to be an absolute path when using the shell. (Part of Redmine #2143) - commands promises useshell attribute now accepts "noshell" and "useshell" values. Boolean values are accepted but deprecated. (Part of Redmine #2143) - returnszero() now correctly sets the class name in this scenario (Part of Redmine #2143): classes: "commandfailed" not => returnszero("/bin/nosuchcommand", "noshell"); Bugfixes: - bundles are allowed to be empty (Redmine #2411) - Fixed '.' and '-' not being accepted by a commands module. (Redmine #2384) - Correct parsing of list variables by a command module. (Redmine #2239) - Fixed issue with package management and warn. (Redmine #1831) - Fixed JSON crash. (Redmine #2151) - Improved error checking when using fgets(). (Redmine #2451) - Fixed error message when deleting nonexistent files. (Redmine #2448) - Honor warn-only when purging from local directory. (Redmine #2162) - Make sure "restart" and "reload" are recognized keywords in packages. (Redmine #2468) - Allocate memory dynamically to avoid out-of-buffer or out-of-hash situations - fix edit_xml update of existing attributes (Redmine #2034) - use failsafe policy from compile-time specified workdir (Redmine #1991) - ifvarclass checked from classes promises in common bundles - do not wait for splaytime when executing only once - disable xml editing functionality when libxml2 doesn't provide necessary APIs (Redmine #1937) - Out-of-tree builds should work again, fixed a bunch of related bugs. - Fixed race condition in file editing. (Redmine #2545) - Fixed memory leak in cf-serverd and others (Redmine #1758) 3.4.5 (Bugfix and Stability release) Bugfixes: - make qualified arrays expand correcty (Redmine #1998, Mantis #1128) - correct possible errors in tcdb files when opening - avoid possible db corruption when mixing read/write and cursor operations - Allow umask value of 002 (Redmine #2496) 3.4.4 (Bugfix and Stability release) Bugfixes: - prevent possible crash when archiving files (GitHub #316) - don't create symlinks to cf-know in update policy - don't enable xml support if libxml2 is too old (Redmine #1937) 3.4.3 (Bugfix and Stability release) Bugfixes: - Don't flood error messages when processes are out of defined range - prevent segmentation fault in cf-monitord -x (Redmine #2021) - when copying files, use same file mode as source file, rather than 0600 (Redmine #1804) - include xpath in messages generated by edit_xml operations (Redmine #2057) 3.4.2 (Bugfix and Stability release) Bugfixes: - Fixes to policies in masterfiles (see masterfiles/Changelog for details) - Fixes for OpenBSD (GitHub #278) - Do not canonify values specified in abortbundleclasses/abortclasses (Redmine #1786) - Fix build issues on NetBSD, SLES 12.2 - Improve error message when libxml2 support is not compiled (Redmine #1799) - fix potential segmentation fault when trimming network socket data (GitHub #233) - fix potential segmentation fault when address-lookups in lastseen db failed (GitHub #233) - execute background promise serially when max_children was reached, rather than skipping them (GitHub #233) - fix segmentation fault in cf-promises when invoked with --reports (Redmine #1931) - fix compilation with Sun Studio 12 (Redmine #1901) - silence type-pun warning when building on HP-UX (GitHub #287) 3.4.1 (Bugfix and Stability release) New feature/behavior: - cf-execd terminates agent processes that are not responsive for a configurable amount of time (see agent_expireafter in body executor control), defaulting to 1 week Bugfixes: - fix regression of classmatch() failing with hard classes (Redmine #1834) - create promise-defined and persistent classes in correct namespace (Redmine #1836) - several fixes to namespace support - fix several crash bugs caused by buffer overflow and race conditions in cf-serverd - regenerate time classes in cf-execd for each run (Redmine #1838) - edit_xml: fix select_xpath implementation and update documentation NOTE: code that uses select_xpath_region needs to be changed to select_xpath - edit_xml: make sure that text-modification functions don't overwrite child nodes - edit_xml: improve error logging 3.4.0 New features: - Added rpmvercmp utility to compare versions of RPM packages for accurate sorting of RPM packages for packages promises. - Implement network timeout on server side to avoid keeping stale connections for hours. - XML editing capabilities. See the documentation for edit_xml body. Note the new dependency: libxml2. - Implement inheritance of local classes by bundles called using "usebundle". By default classes are not inherited. See the examples/unit_inherit.cf for an example. - Moved from Nova/Enterprise: - POSIX ACL support, - "outputs" promise type, - remote syslog support. - packages_default_arch_command hook in packages promises, to specify default architecture of the packages on the system. - packages_version_less_command / packages_version_equal_command hooks in packages promises, to specify external command for native package manager versions comparison - agent_expireafter in body executor control allows you to set a timeout on all cf-agent runs, to enforce a threshold on the number of concurrent agents - Running in Solaris zone is now detected and classes "zone" and "zone_ " are created in this case. - VirtualBox support added to guest_environment promises. - guest_environment promises are supported under OS X. - The "depends_on" attribute is now active, for the partal ordering of promises. If a promise depends on another (referred by handle) it will only be considered if the depends_on list is either kept or repaired already. ** WARNING: When upgrading, make sure that any existing use of depends_on does not make some promises being unintentionally ignored. This can happen if you are currently referring to non-existent or never-run handles in depends_on attributes. - methods return values, initial implementation - New format for cf-key -s, includes timestamp of last connection - cf-promises --parse-tree option to parse policy file and dump it in JSON format - Namespaces support for bundles and bodies. See the examples/unit_namespace*.cf for the usage. - Default arguments for bundles. See the examples/unit_defaults.cf - Metadata promise type. See the examples/unit_meta.cf New semantics: - Methods promises now return the status of promises kept within them. If any promise was not kept, the method is not kept, else if any promise is repaired, the method was repaired else it was kept. - Remote variable access in namespaces by $(namespace:bundle.variable) Changed functionality: - cf-execd -F switch no longer implies 'run once'. New -O/--once option is added to achieve this behaviour. This makes cf-execd easier to run from systemd, launchd and other supervision systems. Misc: - Support for the following outdated platforms and corresponding classes has been removed. De facto those platforms were unsupported for a long time, as CFEngine codebase uses C99 language features unavailable on old platforms: - SunOS 3.x (sun3) - SunOS 4.x (sun4) - Ultrix (ultrix) - DEC OSF/1 AXP (osf) - Digital UNIX (digital) - Sony NEWS (newsos) - 4.3BSD (bsd4_3) - IRIX (irix, irix4, irix64) - IBM Academic Operating System (aos) - BSD/OS / BSDi / BSD/386 (bsdos) - NeXTSTEP (nextstep) - GNU Hurd (gnu) - NEC UX/4800 (ux4800) - (Old news) Since 3.3.0 the layout of CFEngine Community packages has changed slightly. cf-* binaries have been moved to /var/cfengine/bin, due to the following reasons: - cf-* binaries are linked to libraries installed to /var/cfengine/lib, so placing binaries in /usr/local/sbin does not increase reliability of the CFEngine, - keeping whole CFEngine under single prefix (/var/cfengine) makes packaging simpler, - it matches the layout of CFEngine Enterprise packages. Please adjust your policies (the recommended ways to deal with the move are either to adjust $PATH to include /var/cfengine or to create symlinks in /usr/local/sbin in case you are relying on binaries to be available in $PATH). - Workdir location is properly changed if --prefix or --enable-fhs options are supplied to configure (Mantis #1195). - Added check for broken libmysqlclient implementations (Mantis #1217). - Standard library is updated from COPBL repository. - cf-know is no longer built in Community releases. The only functionality useful in Community, namely the reference manual generation, is provided by new compile-time cf-gendoc tool. - Filename (for storing filechanges) changed from file_change.log -> file_changes.log (in /var/cfengine/state) New format for storing file changes introduced: [timestamp,filename, ,Message] N = New file found C = Content Changed S = Stats changed R = File removed - Acceptance test suite passes on Mac OS X. - Changed some port numbers to replace old services with imap(s) - archlinux hard class on Arch Linux. - Detect BSD Make and automatically switch to GNU Make during build. Bugfixes: - cfruncommand for cf-execd is an arbitrary shell command now (Mantis #1268). - Fixed broken "daily" splayclasses (Mantis #1307). - Allow filenames up to 4096 bytes in network transfers (Redmine #1199). - Fix stale state preserved during cf-serverd reload (Redmine #1487). - Free disk space calculation is fixed (Mantis #1120). - Numerous portability bugfixes (especially OpenBSD, Solaris, AIX-related). - Compatibility fixes for AIX, HP-UX, Solaris (Mantis #1185, Mantis #1177, Mantis #1109). - Fixed broken socklen_t configure check under OpenBSD (Mantis #1168). - Fixed hang in cf-promises under OpenBSD (Mantis #1113). - Fixed endless loop in evaluating "$()" construct (Mantis #1023). - Fixed check for old PCRE versions (Mantis #1262). - Fixed insertion of multi-line blocks at the start of file (Mantis #809). - Fixed numerous memory leaks. - Fixes for metadata that were not resolvable - Fixes for namespaces that would not support metadata and variable expansion - Point-to-point network interfaces are detected and reported by CFEngine (Mantis #1246) - Partial non-GNU userspace support in acceptance testsuite (Mantis #1255) Full list of issues fixed is available on https://cfengine.com/bugtracker/changelog_page.php (old bug tracker) and https://cfengine.com/dev/projects/core/versions/34 (new bug tracker) 3.3.9 (Bugfix and Stability release) Bugfixes: - Do not lose hard classes in cf-serverd during policy reload (Mantis #1218). - Implement receive network timeout in cf-serverd. Prevents overloading cf-serverd with stale connections. 3.3.8 (Bugfix and Stability release) Versions 3.3.6, 3.3.7 were internal and weren't released. Bugfixes: - Propery set sys.domain variable if hostname is fully-qualified. - Fixed several small memory leaks. - Make network timeout for network reads configurable. Previously it was hardcoded to be 30 seconds, which was not enough for cf-runagent invoking cf-agent on big policies (Mantis #1028). 3.3.5 (Bugfix and Stability release) Bugfixes: - Fixed cf-execd memory leak on hosts with cf-monitord running. - Robustify against wrongly-sized entires in embedded databases. Standard library: - Bugfixes from upstream COPBL repository. - standard_services bundle from upstream COPBL repository. 3.3.4 (Bugfix and Stability release) Evaluation of policies: - Fix wrong classes set after installation of several packages using packages promises (Mantis #829). - Fix segfault using edit_template on existing file (Mantis #1155). Misc: - Fix memory leak during re-read of network interfaces' information in cf-execd/cf-serverd. 3.3.3 (Bugfix and Stability release) Evaluation of policies: - Zero-length files are valid for readfile() and similar functions (Mantis #1136). - Unchoke agent in case it encounters symlinks in form ./foo (Similar to Mantis #1117). Misc: - Fix generation of reference manual on machines with umask more relaxed than 022. - Use statvfs(3) on OpenBSD to obtain filesystem information (Mantis #1135). 3.3.2 (Bugfix and Stability release) Evaluation of policies: - Do not segfault if file copy was interrupted due to network connectivity or server going away (Mantis #1089). - Do not segfault if log_failed attribute is present in body, but log_kept is not (Mantis #1107). - Do not mangle relative paths in symlinks during file copy Previously symlink a -> b was mangled to a -> ./b. (Mantis #1117) - Properly compare 1.0 and 1.0.1 in packages promises. Previously only versions with equal amount of "segments" were comparable (Mantis #890, #1066). Base policy: - Properly set permissions on files for /var/cfengine/lib on HP-UX (Mantis #1114). - Standard library (cfengine_stdlib.cf) is synced with COPBL repository. Misc: - Do not create huge file in case corrupted TokyoCabinet database is detected (Mantis #1106). - Fix file descriptor leak on error paths, may have caused crashes of cf-execd and cf-serverd (Issue #1096). - Fix intermittent segfault in cf-execd (Mantis #1116). - Impose an upper limit on amount of listening sockets reported by cf-monitord. Huge amounts of listening sockets caused cf-agent to segfault on next run (Mantis #1098). - Add missing function prototypes caused errors during compilation on HP-UX (Mantis #1109). - Fix compilation on Solaris 11 (Mantis #1091). 3.3.1 (Bugfix and Stability release) Evaluation of policies: - Do not cut off name of bundle in variables interpolation (Mantis #975). - Do not segfault in function evaluation guarded by ifvaclass clause (Mantis #1084, #864). - Do not segfault if "classes" promise does not declare any value to be evaluated (Mantis #1074). - Do not segfault in database promises if there is no database_operation provided (Mantis #1046). Built-in functions: - Fix countclassesmatching() function which was misbehaving trying to match classes starting with alphanumeric symbol (Mantis #1073). - Fix diskfree() to return kilobytes, as described in documentation (Mantis #980, #955). - Fix hostsseen() function to avoid treating all hosts as not being seen since 1970 (Mantis #886). - Do not output misleading error message if readtcp() is unable to connect (Mantis #1085). Command-line interface: - -d option previously reqired an argument, though help message disagreed (Mantis #1053). - Disable --parse-tree option, not ready for the release (Mantis #1063). - Acept -h as a --help option. - Ensure that cf-execd might be started right after being shut down. Misc: - Plug file descriptor leak after failed file copy (Mantis #990). - Fix unsafe admit rules in default promises.cf (Mantis #1040). - Fix splaytime to match documentation: it is specified in minutes, not seconds (Mantis #1099). Packaging: - Fix owner/group of initscript and profile.d snippet in RPM builds (Mantis #1061, #1058). - Fix location of libvirt socket CFEngine uses to connect to libvirtd (Mantis #1072). - Install CoreBase to /var/cfengine/masterfiles during installation (Mantis #1075). - Do not leave old cf-twin around after upgrade (Mantis #1068) - Do not leave rcS.d symlinks after purging .deb package (Mantis #1092). 3.3.0 New promise types: - Guest environments promises, which allow to manipulate virtual machines using libvirt. - Database promises, which allow to maintain schema of MySQL and PostgreSQL databases. Database promises are in "technical preview" status: this promise type is subject to change in future. - Services promises for Unix, allows abstraction of details on managing any service New built-in functions: - dirname() to complement lastnode() - lsdir() - maplist() to apply functions over lists New features: - Allow defining arrays from modules. - Allow both `process_stop' and `signals' constraints in `processes' promises at the same time. - cf-promises --gcc-brief-format option to output warnings and errors in gcc-compatible syntax which to ease use "go to next error" feature of text editors. - Iteration over lists is now allowed for qualified (non-local) lists. New built-in variables and classes (Linux): - Number of CPUs: $(sys.cpus), 1_cpu, 2_cpus etc New built-in variables and classes (Unices): - $(sys.last_policy_update) - timestamp when last policy change was seen by host - $(sys.hardware_addresses) - list of MAC adresses - $(sys.ip_addresses) - list of IP addresses - $(sys.interfaces) - list of network interfaces - $(sys.hardware_mac[$iface]) - MAC address for network interface - mac_ :: - discovered MAC addresses Changes: - Major cleanup of database handling code. Should radically decrease amount of database issues experienced under heavy load. *WARNING*: Berkeley DB and SQLite backends are *removed*, use Tokyo Cabinet or QDBM instead. Both Tokyo Cabinet and QDBM are faster than Berkeley DB in typical CFEngine workloads. Tokyo Cabinet requires C99 environment, so it should be available on every contemporary operating system. For the older systems QDBM, which relies only on C89, is a better replacement, and deemed to be as portable, as Berkeley DB. - Change of lastseen database schema. Should radically decrease I/O contention on lasteen database. - Automatic reload of policies by cf-execd. - Documentation is generated during build, PDF and HTML files are retired from repository. - Rarely used feature retired: peer connectivity intermittency calculation. - Memory and CPU usage improvements. - Testsuite now uses 'make check' convention and does not need root privileges anymore. - cf_promises_validated now filled with timestamp, allows digest-copy for policy instead of mtime copy which is safer when clocks are unsynchronised - The bundled failsafe.cf policy now has trustkey=false to avoid IP spoofing attacks in default policy - See the full list of bugfixes at https://cfengine.com/bugtracker/changelog_page.php 3.2.4 (Bugfix and Stability release) Fixed failure in network transfer in case of misbehaving peer A few tiny memory leaks on error paths fixed 3.2.3 (Bugfix and Stability release) A few tiny memory leaks fixed Improved performance of cf-serverd under heavy load with TokyoCabinet database Full list of issues fixed is available on https://cfengine.com/bugtracker/changelog_page.php 3.2.2 (Bugfix and Stability release) Enabled compilation in "large files" mode under AIX Alleviated problem with broken file transfers over unstable Internet links. Full list of issues fixed is available on https://cfengine.com/bugtracker/changelog_page.php 3.2.1 (Bugfix and Stability release) Fixed compilation under HP-UX and Solaris Enabled compilation using HP ANSI C compiler Full list of issues fixed is available on https://cfengine.com/bugtracker/changelog_page.php 3.2.0 New bootstrap method with single-command bootstrapping: - cf-agent --bootstrap --policy-server 123.456.789.123 - Associated policy template files are added, partially maintained by CFEngine Bug fixes for file-editing, package versioning, and embedded database corruption (We recommend using TokyoCabinet instead of BerkeleyDB if building from source). Improved upgrade path for Nova. Patches for improved run-agent concurrency Reorganization of documentation and community resources 100% on regression test suite on 3 operating systems (Ubuntu, Debian, SuSE on x86-64 hardware) Support for multiple release environments package_policy update and addupdate now check if user-supplied version is larger than currently installed - updates only if so Help text of cf-report -r corrected - a list of key hashes is required, not ip addresses. New Emacs mode for CFEngine policy files (thanks to Ted Zlatanov!) Warnings are on edit_line changes can now give greater degree of information without spamming promise logs Class expressions parser accepts '||' as an alias for '|' again. Invalidation of package list cache on installation/removal of packages. New option cf-key -r to remove host key by IP or hostname. Added detection of network interfaces which belong to BSD jails. Improve robustness of multi-threaded code, in particular fix problems with spurious access denials in server and losing of authentication rules after policy reload. cf-promises accepts option -b matching cf-agent, which causes it to do not complain about missing bundlesequence. New functions and(), not(), or() and concat() to ease use of ifvarclass() clause. Full list of issues fixed is available on https://cfengine.com/bugtracker/changelog_page.php 3.1.5 New class parser, '||' is no longer allowed in expressions (use '|'). Class setting in the promise types insert_lines, delete_lines, replace_patterns, field_edits, vars, classes is restored. suspiciousnames implemented. New function getvalues(). New functions parse{read,int,string}array to match read{read,int,string}array. Testsuite added to check for core functionality. Syslog prefix is fixed to say 'cf3' instead of 'community'. 3.1.4 (Bugfix and Stability release) Some urgent patches to 3.1.3. Class validation parse bug fixed. Global zone handling error for solaris fixed. Package architectures handled correctly (bug #456). Reading and writing of key name "root-.pub" eliminated (bug #442, #453). cf-serverd crash because of race condition on SERVER_KEYSEEN fixed. Lock purging to avoid remnant complexity explosion (bug #430). Some copyright notices added that got lost. 3.1.3 (Stability release) Major memory leaks in cf-monitord, cf-execd, cf-serverd fixed (bug #427). The daemons now show no growth even with very complex policies. cf-serverd crash due to race condition in DeleteScope() fixed (bug #406). Added 30 second timeout on recv() on Linux. package_noverify_returncode implemented (bug #256). A flexible mechanism for setting classes based on return codes of commands has been introduced. Allows for setting promise kept, repaired or failed based on any return codes. This is currently implemented for commands-promises, package-manager commands and transformer in files. In classes body, see attributes kept_returncodes, repaired_returncodes, failed_returncodes (bug #248, #329). New function ip2host - reverse DNS lookup (bug #146). 3.1.2 (Scalability/efficiency release) Big efficiency improvements by caching output from cf-promises. Can also be used for much more efficient policy deployment (only pull if changed). Caching state of ps command for greater efficiency. Reloaded for each bundle. Index class lookup improves efficiency of class evaluation for huge configurations. Fixed issue where certain promiser strings got corrupted. Minor memory access issues fixed. Iterator bug introduced in 3.1.0 fixed 3.1.1 (Bugfix release) Memory leaks in server tracked down and fixed. List expansion bug (one list items not executed) fixed. Security issue introduced by change of runcommand shell policy fixed. If users defined a runcommand for cf-runagent/cf-serverd communication, possible to execute commands. cf-key -s command for showing key hash/IP address identity pairs 3.1.0 Change in storage of public keys. Cfengine now hashes the public key and uses this as the keyname. Keys will be converted automatically. The old dynamic addresses lists are deprecated. Caching of dns and key information for greater server speed. Change in last-seen format reflects the public key usage. New package policy addupdate - installs package if not there and updates it otherwise. Support for package_changes => "bulk" in file repository as well. New special function readstringarrayidx, similar to readstringarray, but uses integer indices. Very useful if first row elements are not good identifiers (e.g. contains spaces, non-unique, etc.). Change two log formats to use time() instead of date() - filechanges - total compliance Change from using md5 to sha256 as default digest for commercial version, community retains md5 for compat. Commands not returning 0 in commands-promises are flagged as repair_failed. Adjustable timeout on connect(). Defaults to 10 seconds, adjustable with default_timeout in agent control. Redesign of the knowledge map infrastructure. Now possible to use variables to call methods, e.g methods: "name $(list)" usebundle => $(list)("abc"); See reference manual notes Changes to normal ordering to optimize execution. Increased stability by always initializing Attribute and Promise structures. When running cf-promises in dry-run mode (-n), the user does not need to put binaries in WORKDIR/bin. For example, non-privileged users can verify root policies. Source control revision added in version string if run in verbose mode (e.g. "cf-promises -vV"). This needs some refining, uses revision of a header now. New semantics in return values of list functions. Null values are now allowed and there is no iteration over empty lists. The value "cf_null" is reserved for use as a null iterator. 3.0.5p1 Showing paths allowed/denied access to when cf-serverd is run in verbose mode. Bug in server fixed for dynamic addresses. File handle closure bugfix - too many open databases. Seg fault in mount files fix. Twin used in cf-execd without checking. Check_root set wrong directory permissions at source not destination. Error message degraded in body definition. Undefined body not warned as error. Various build enahncements. Package_list_update called only once per manager, and fixed crash. Version number bug in packages. 3.0.5 Encryption problems fixed - client key buffer was uninitialized. Classes-promisers are now automatically canonified when class strings are defined, to simplifying the use of variables in classes. New scalars sys.cf_version and sys.nova_version that hold Cfengine version information. Attribute package_delete_convention added, to allow customizable package name in delete command during update. package_list_update_ifelapsed limit added. Private variable $(firstrepo) is available in package_name_convention and package_delete_convention in order to expand the full path to a package, which is required by some managers. Some of the threading code is rewritten and made more robust. This includes synchronizing access to the lastseen database from the server. Bad initialization of BSD flags fixed Multiple variable expansion issues in control fixed for server and agent Allow ignore_missing_bundles to affect methods: bundles too Run agent trust dialogue fixed Bug in CPU monitoring, increasing time scale caused linear decay of CPU measurement. Bug in Setuid log storage, fix. Hooks added for new Nova virtualization promises. Multithreading mutex failed to collide during cfservd leading to dropped authentication under heavy load. 3.0.4 Class cancellation in promises to create better class feedback, allows emulation of switch/case semantics etc Value of SA measurement promises Special function getenv() which returns the contents of an environment variable (on all platforms). New function translatepath for generic Windows New function escape() to escape literals as regular expressions (like SQL) New function host2ip for caching IP address lookup New function regextract for setting variables with backreferences New variables for the components $(sys.cf_agent), $(sys.cf_know) etc pointing to the binaries. More robust integrated database implementation; closing all handles when receiving signals, self-healing on corruption. Package installation on localhost without a manager like yum completed, multiple repositories searched, and universal methods. Numerous bugfixes 3.0.3 sha256 .. new hashes in openssl included in syntax tree. End of line autocropping in readfile (hopefully intelligent) hashmatch function incorrectly implemented - old debugging code left behind. Fix. sys.crontab variable Unknown user is now interpretated as "same user", so that we give cfengine a chance to fix Unregistered addresses no longer report "(Non registered IP)", but return as the address itself when doing reverse lookups. 3.0.2 IMPORTANT: Change in normal ordering of editing. replace comes after insert lines Much testing and minor bug fixing Memory leaks fixed Many hooks added for Nova enterprise extensions. promise_output reports now placed in WORKDIR/reports directory Initialization correction and self-correx in monitord Many new body constraints added. Code readied for enterprise version Nova. -b option can override the bundlesequence (must not contain parameters yet) collapse_destination_dir option added to copy so that files can be aggregated from subdirectories into a single destination. Preparation for release: unit_accessed_before.cf x unit_accumulated_time.cf x unit_acl.cf x unit_acl_generic.cf x unit_ago.cf x unit_arrays.cf x unit_backreferences_files.cf x unit_badpromise.cf x unit_badtype.cf x unit_bsdflags.cf x unit_cf2_integration.cf x unit_changedbefore.cf x unit_change_detect.cf x unit_chdir.cf x unit_classes_global.cf x unit_classmatch.cf x unit_classvar_convergence.cf x unit_compare.cf x unit_controlclasses.cf x unit_control_expand.cf x unit_copy.cf x unit_copy_edit.cf x unit_copylinks.cf x unit_createdb.cf x unit_create_filedir.cf x unit_definitions.cf x unit_deletelines.cf x unit_disable_and_rotate_files.cf x unit_dollar.cf x unit_edit_column_files.cf x unit_edit_comment_lines.cf x unit_edit_deletenotmatch.cf x unit_edit_insert_lines.cf x unit_edit_insert_lines_silly.cf x unit_edit_replace_string.cf x unit_edit_sectioned_file.cf x unit_edit_setvar.cf x unit_edit_triggerclass.cf x unit-env.cf x unit_epimenides.cf x unit_exec_args.cf x unit_execd.cf x unit_exec_in_sequence.cf x unit_execresult.cf x unit_expand.cf x unit_failsafe.cf x unit_file_change_detection.cf x unit_fileexists.cf x unit_file_owner_list_template.cf x unit_fileperms.cf x unit_filesexist2.cf x unit_filesexist.cf x unit_getgid.cf x unit_getindices.cf x unit_getregistry.cf x unit_getuid.cf x unit_global_list_expansion_2.cf x unit_global_list_expansion.cf x unit_groupexists.cf x unit_hash.cf x unit_hashcomment.cf x unit_hashmatch.cf x unit_helloworld.cf x unit_hostrange.cf x unit_intarray.cf x unit_iprange.cf x unit_irange.cf x unit_isdir.cf x unit_islink.cf x unit_isnewerthan.cf x unit_isplain.cf x unit_isvariable.cf x unit_iteration.cf x unit_knowledge_txt.cf x unit_lastnode.cf x unit_ldap.cf x unit_linking.cf x unit_literal_server.cf x unit_locate_files_and_compress.cf x unit_log_private.cf x unit_loops.cf x unit_measurements.cf x unit_method.cf x unit_method_validate.cf x unit_module_exec_2.cf unit_module_exec.cf unit_mount_fs.cf x unit_neighbourhood_watch.cf x unit_null_config.cf x unit_occurrences.cf x unit_ordering.cf x unit_package_apt.cf x unit_package_hash.cf x unit_package_rpm.cf x unit_package_yum.cf x unit_package_zypper.cf x unit_parallel_exec.cf x unit_pathtype.cf x unit_pattern_and_edit.cf x unit_peers.cf x unit_postfix.cf x unit_process_kill.cf x unit_process_matching2.cf x unit_process_matching.cf x unit_process_signalling.cf x unit_readlist.cf x unit_readtcp.cf x unit_regarray.cf x unit_registry.cf x unit_regline.cf x unit_reglist.cf x unit_remove_deadlinks.cf x unit_rename.cf x unit_report_state.cf x unit_reporttofile.cf x unit_returnszero.cf x unit_select_mode.cf x unit_select_region.cf x unit_selectservers.cf x unit_select_size.cf x unit_server_copy_localhost.cf x unit_server_copy_remote.cf x unit_server_copy_purge.cf x unit_splitstring.cf x unit_sql.cf x unit_storage.cf x unit_strcmp.cf x unit_stringarray.cf x unit_syslog.cf x unit_template.cf x unit_tidy_all_files.cf x unit_user_edit.cf x unit_user_edit_method.cf x unit_userexists.cf x unit_varclass.cf x unit_vars.cf x unit_warnifline.cf x unit_webserver.cf x 3.0.1 First standalone release, independent of cfengine 2 Purge old definitions and check consistency. NB: changed search_mode to be a list of matching values Reporting rationalized in cf-promises with -r only to avoid leaving output files everywhere. Hooks added for upcoming commercial additions to cfengine. Added classify() and hostinnetgroup() functions Added additional change management options for change detection Package management added - generic mechanisms. Limits on backgrounding added to avoid resource contention during cfengine runs. Image type added to cf-know. New classes for quartly shifts: Morning,Afternoon,Evening,Night Bug fixes in editfiles - line insertion for multiple line objects Change the name of the variables and context from the monitord for better separation of data, and shorter names. sys -> mon average -> av, stddev -> dev canonical name for windows changed from "nt" to "windows", also version names added "vista","xp" etc.. License notices updated for dual license editions. 3.0.0 First release of cfengine 3. Known omissions: - no support for ACLs - no support for packages - no support for interface configuration These will be added in the next release.
Enterprise ChangeLog
For the complete history of Enterprise-specific changes in the CFEngine version you have
installed, see the ChangeLog.Enterprise
file in /var/cfengine/share/doc
.
3.9.2: - Canonify class names stored in class history log. (ENT-2821) - Fix errors of type "No file object exsists in path" on Windows. 3.9.1: - Add: Inventory for system product name (model) (ENT-2780) - Add: Enterprise application log dir to rotation - Change: Disable autocomplete for login - Change: Enable strict transport security - Change: Reduce php info leak - Change: Rename duplicate bodies in ha_update.cf (ENT-2753) - Change: Disable RC4 Cipher for ssl in Mission Portal - Change: re-enable hub process maintainance for systemd hosts. - Change: Disable TCP for redis (ENT-2761) - Change: Reduce Enteprise webserver info - Change: Use more restrictive unix socket perms (ENT-2705) - Change: Disable http TRACE method - Fix: Hub package no longer depends on libltdl. (ENT-2714) - Fix cf-serverd being launched under wrong account on Windows. (ENT-2755) 3.9.0: - Change: Render Mission Portal httpd.conf with mustache (Jira ENT-2568) - Change: Switch to http redirect by default (Jira ENT-2071) - Fix exporting CSV reports through HTTPS. (Redmine #7267) - Add: Bundle to generate a self signed cert for Mission Portal - For call collect in Enterprise, default collect_window setting increased from 10 to 30 seconds for reliability reasons in large-scale environments. - Upgrade CFEngine dependencies to the following versions: - Apache 2.4.20 - Git 2.8.3 - PHP 5.6.22 - PostgreSQL 9.5.3 - Redis 3.0.7 - rsync 3.1.2 (Jira ENT-2720) - Fix scheduled report not beeing emailed when report type is set to only contain CSV file type. (Redmine #3780, #7619) - Introduce class json log. (Redmine #7951) - Introduce new promise log logging. (Redmine #7887) - Fix broken call collect. (Redmine #7701) - Fix error logging to MP via HTTPS. (Redmine #7687) 3.8.2: No Enterprise specific fixes for 3.8.2, see Community changelog. 3.8.1: Changes: - Upgrade CFEngine Enterprise dependencies to the following versions: - OpenSSL 1.0.2e - PCRE 8.38 - libxml2 2.9.3 - OpenLDAP 2.4.43 - Redis 3.0.6 - PHP 5.6.17 - libcurl 7.46.0 - Git 2.6.5 3.8.0: Changes: - Move hub_log from /var/cfengine to /var/cfengine/log. - Change in behaviour: when running "cf-hub -q -H" manual report collection, policy is parsed before collecting, so there must be valid policy in inputs directory. (Redmine #7542) - Introduce by field truncation for promise execution entries. (Redmine #7466) - Move promise_summary.log into /var/cfengine/log directory. Bug fixes: - CFEngine on Windows no longer truncates log messages if the program in question is killed halfway through. - For call collect in Enterprise, default collect_window setting increased from 10 to 30 seconds for reliability reasons in large-scale environments. - Fix package not installing on Windows 2008 32-bit. (Redmine #7478) - Fix not being able to delete log files while CFEngine is running on Windows. (Redmine #7149) - Fix: Typo in cf-hub error message - Fix resource restrictions of SQL API matching table names as substrings. (Redmine #7536) - Removed error message from cf-serverd when not finding software inventory. E.g. "Failed to access current state for report: 'software'". - Fix last agent run timestamp in Agents not reporting (health bar). (Redmine #7406) 3.7.2: Bug fixes: - For call collect in Enterprise, default collect_window setting increased from 10 to 30 seconds for reliability reasons in large-scale environments. - CFEngine on Windows no longer truncates log messages if the program in question is killed halfway through. - Fix: Typo in cf-hub error message - Removed error message from cf-serverd when not finding software inventory. E.g. "Failed to access current state for report: 'software'". 3.7.1: Behavior changes: - Change in behaviour: when running "cf-hub -q -H" manual report collection, policy is parsed before collecting, so there must be valid policy in inputs directory. (Redmine #7542) Bug fixes: - Fix resource restrictions of SQL API matching table names as substrings. (Redmine #7536) - Add truncation for promise attribute sizes to prevent from ignoring to long reports. (Redmine: #7466) - Fix last agent run timestamp in Agents not reporting (health bar). (Redmine #7406) - Fix noise from internal policy to upgrade windows agents (Redmine #7456) - Fix package not installing on Windows 2008 32-bit. (Redmine #7478) 3.7.0: Mission Portal: - Multiple dashboards - Dashboard sharing - 'Changes' report type added - 'Changes' widget introduced - Added more out-of-the-box inventory variables Bug fixes: - Fix for health status in header occasionally not loading - Fixed icons disappearing from host categorization dropdown after editing - Process matching on Windows has been rewritten, which should make process promises work more reliable there. (Redmine #6977) - Failure in output log cleanup on Windows has been fixed. (Redmine #7149) API: - Introduce Changes API - Remove PromiseExecutionsLog (replaced with Changes API) - Remove SoftwareUpdatesLog 3.6.6: Bug fixes: - Ignore empty log messages while logging promise executions in cf-agent evaluation. - Fix Postgres CPU usage spikes. - Fix upgrate for monitoring. - Fix duplicate key value violates unique constraint "status_pkey" error. - Reduce database size in high load hub by making vacuum strategy more aggressive. 3.6.5: Bug fixes: - Fix cleaning-up monitoring during upgrade. - Remove unused bundles.lmdb to reduce agent I/O usage. - Redesign classes and variables storage (for reporting) to reduce I/O usage. - Improve API performance for DELETE requests on /api/host/:id resources. Mission Portal: - Small CSS changes - Widgets & alerts view - UI changes - Updated links to support portal 3.6.4: Bug fixes: - Fix "cfe_autorun_inventory_dmidecode" error message on Windows if Powershell is not installed. - Fix bogus failed promise, "cfe_internal...", as a result of indexing packages for the inventory screen. (Redmine #6865) Mission Portal: - LDAP settings UI improvements - Unsaved SQL and Inventory Reports are preserved while refreshing/navigating in browser browser - Help text: Added instructions to turn on Monitoring data - Health bar dropdown labels and reports renamed Changes: - Introduce automatic rebase for the client if the client have not been successfully collected for defined period of time. Timeout is set by client_history_timeout attibute in hub body and if it is not set, it defaults to 6 hours. Note: During rebase all accumulated reports up till that event are ignored and not collected by the cf-hub. 3.6.3: Mission Portal: - Added license information to header - General UI cleanups and small bug fixes - Optimization of Software Updates alert - Inventory reports: - Made software filtering case insensitive - Updated help text - Performance improvements - Alerts: - Bug fix for duplicate alerts in overview - Bug fixes for deleting alerts & widgets - Settings: - LDAP search filter help text & validation 3.6.2: Changes: - Monitoring magnified and monitoring yearly database schema have been redesigned to reduce database disk space usage over time. - RBAC backend have been redesigned from dynamically generated tmp views to static global views that use session variables for passing context filters and host identifier. Filtering also switched from dynamically generated queries to Full Text Search. Mission Portal: - UI changes: redesigned alerts + conditions overview screen - Layout improvement of alert results view - Added navigation menu buttons to dashboard + alerts screens - High Availability status added to header bar - Custom notification script UI added to settings and alert editing - Added 'Low disk space' alert + 'System health' widget OOTB - Bug fixing/small UI improvements 3.6.1: Changes: - 'cf-key --install-license' installs hub-specific license key file "fqname-hostkey.dat" in $WORKDIR/licenses, where they can easily be managed centrally via a VCS - hub-specific license file is searched in $WORKDIR/licenses before license.dat is searched in $WORKDIR, $WORKDIR/inputs and $WORKDIR/masterfiles - Where appropriate, Enterprise API returns proper NULL json objects rather than literal "NULL" values Mission Portal: - streamlined UI for inventory reporting - fix username/role lookup failures if external authentication backend is case insensitive - reduce number of LDAP roundtrips - allow filtering of reports by category - allow reordering of widgets on dashboard - UI for bulk-deleting decommissioned hosts from "health" menu - various behind-the-scene fixes and improvements to speed up UI and reporting Platform support: - Introduced Windows support into the CFEngine 3.6 series. 3.6.0 Mission Portal: - UI and layout improvements and cleanups - Dashboard and alerts introduced - Inventory report type and view introduced - Report categories introduced - CFEngine health indicator added to UI with links to associated reports - Host number indicator added to UI - Inline help and help pop-ups added for new features - Welcome tour pop-up introduced - Host filter UI improvements - search host name, select/deselect all - About CFEngine page - license and version information has moved to a dedicated page in settings - Fixes for IE8 compatibility - Added option in UI to allow logging in to Mission Portal over https - Design Center sketch catalog redesign - sketches can now be filtered by category, tag, or search - UI to reset git settings in Design Center Hub: - Remove --cache / -a command line option from cf-hub binary - Remove --index / -i command line option from cf-hub binary - Remove --maintain / -m command line option from cf-hub binary (Maintenance process have been implemented in the policy) - Remove MongoDB Diagnostics - Promise repaired/notkept log have been removed from report collection. It have been replaced by promise executions report. - Total compliance report have been removed from report collection. - Setuid report have been removed. - Promise definitions report have been removed. - Promise and bundle compliance reports have been removed. Their functionality have been replaced with promise executions report. - Reporting database and report collection architecture have been redesigned to improve performance and scalability characteristics. - MongoDB reporting database have been replaced with PostgreSQL 9.3 - Context, Variable, Software Installed, Software Patches and Promise Execution reports support history over time. History length is controlled per report type and can be configured in cfe_internal_hub_maintain bundle. - Introduce new hub query type: 'rebase' ('full' query aliases 'rebase') for re-downloading full state of the client in current moment. Rebase query result overwrites all non-historical entries about the host in the database. - Include meta data contents to contexts and variables reports. Enterprise Rest API: - Rest interface for Design center - Additional information returned for host (lastreport and firstseen) - Rest API 2.2 (/rest API) have been removed. - Enterprise API performance have been improved. - SQL API table schema have been redesigned. - Remove API cache. - Pagination and sorting improvements. - Introduce 'hostIdentifier' setting to /api/settings. - Fixes in LDAP support. - Delete host API now additionally removes host from lastseen database as also removes host public key. Bug fixes: - Removed MongoDB - cf-serverd for Windows now binds to both IPv4 and IPv6 by default, not just IPv6. (Redmine #3980) - cf-agent now reports host packages installed and available by default. (Redmine #3257) - Fixed incorrect file diff generation when a line had moved within a file, and certain other corner cases. (Redmine #5015) - Windows fixes: - CFEngine now handles Windows newlines correctly within text files when editing or using the module protocol. Existing text files will keep their newline type (either LF or CRLF), whereas new files will get CRLF newlines. (Redmine #4733) - CFEngine will no longer display a blocking popup if it crashes. - CFEngine now reports uptime correctly on Windows. Enterprise extensions: - Remove promise_notkept_log_include, promise_notkept_log_exclude, promise_repaired_log_include, promise_repaired_log_exclude (syntax is valid but not functional) - Remove classes_include, classes_exclude, variables_include, variables_exclude (syntax is valid but not functional) - Introduce promise_handle_include, promise_handle_exclude attributes from report_data_select - Introduce metatags_include, metatags_exclude attributes from report_data_select - Deprecate export_zenoss attribute - Introduce promise_execution.log containing outcome and information about all executed promises. It can be found under cfengine/state/ data format is CSV. - Agent execution time have been included into benchmarks report. - After disabling report_data_select filtering rule, include last known value in next packaged report. 3.5.3 Bug fixes: - purge old data for promises with long promise handles (Redmine #3438) - fix constraint violation in PromiseDefinitions table which resulted in error everytime this table was loaded (Redmine #3370) - enable update of promise definitions database from policy - fix cfengine3 init.d script to correctly detect debian systems with yum installed (Redmine #3589) Mission Portal: - various layout and UI fixes - fix editing of event trackers - speed up listing of hosts for promises not kept - maintain host context (Redmine #3474) - ability to manually add context filter in the SQL app (Redmine #3466) - host identifier settings simplified (Redmine #3101) Packaging fixes: - Correct php.ini path in the packaged httpd (Redmine #3445) - Add missing mongodb tools in ubuntu/debian hub packages (Redmine #3444) - Fix manpath error for SLES (Redmine #3539) - Fix file permissions - some policy files had executable bit set (Redmine #3521) 3.5.2 Changes: - MongoDB has been upgraded to version 2.2.4 - monitoring data has moved into a separate database See db-move-monitoring-to-cfmonitor.js script to migrate data Bug Fixes: - Fix segfault of cf-serverd on HP-UX - Do not to start a mongodb repair unnecessarily - cf-hub -H now supports multiple hosts Mission Portal: - Reports can be published and shared between users - Various UI improvements - Optimizations in the report engine 3.5.1 Mission Portal: - SQL queries can be shared between users - Fix timing issues for downloading large SQL reports - Purge sketch data when no longer used by active sketches - Uninstall sketches that have no activation - Support for boolean, menu option and optional parameters in Design Center UI - UI fixes to user and role management pages - Delete navigation tree definitions of deleted uses - Fixes to password reset - General UI improvments - Fixes for IE8 compatibility Changes: - Perform a database repair from init script if unclean shutdown of mongod is detected - Redmine #3035 - Data collection and cf-hub - Improved database connection handling during report collection by cf-hub. - REST APIs support an optional disableCache flag; when set, the backend always hits the MongoDB - Redmine #2945 Bug Fixes: - don't generate ERR message during maintenance if environments couldn't be queried, changed to INFO message - Fix usemodule on Windows (Redmine #1884) - Fixed replica set detection (regression in 3.5) - Redmine #2806 - Set correct precision format when storing db diagnostics to avoid null-values - Fix possible division-by-zero bug in compliance meters (Redmine #2734) 3.5.0 New features: - Mission Portal - added Design Center UI to simplify sketch activation, including MP specific git settings to support version control of sketch configurations - re-focused apps support quick navigation - added persisting host and policy context between apps - extended the SQL builder interface with more tables - Added FirstReportTimeStamp into Hosts table in SQL REST API. This time value represent fist report time after bootstrap, already bootstrapped agents will set this with first report after update. - Added regular expression support to SQL queries - HostContext filter support in SQL REST API. - added global navigation trees which are only editable by admins, including the option to share trees with other users - REST API extensions - New optional parameters for REST API were added: hostContextInclude and hostContextExclude (array type) - PromiseContext filter support in SQL REST API. New optional parameter for REST API was added: promiseContext (input: all / user / system) - Data collection and cf-hub - Added set and clear triggers for persistently disabling CFEngine components. eg. to disable cf-monitord, run cf-agent with "-Dset_persistent_disable_cf_monitord" to re-enable use: "-Dclear_persistent_disable_cf_monitord" - Host side report content filter for class, variable, promise log and monitoring reports. Controlled by report_data_select body in access promise. - Diagnostics logging and SQL REST API for MongoDB, report collection and maintenance process on the enterprise hub. - Windows - Windows Powershell support. execresult(), returnszero() and commands promises now supports "powershell" as an option in addition to the "noshell" and "useshell" variants. "powershell" is also added as a hard class in order to test whether Powershell is available. Changes: - Mission Portal - new visual design - streamlined interactions for building new trees - trees are now loaded lazily - general clean-up to the tree controls - hosts in trees are no longer color coded - hosts are only classified as red, green or missing data - operating system tree is now loaded by default - SQL queries are now run by default after clicking their respective links, running a query is now primary action in the UI (#2393) - data and result sets can now by filtered based on navi-tree - logged-in user's name is visible again in the toolbar - removed beta apps - Windows - Improved ACL handling on Windows, which led to some syntax changes. We now consistently use the term "default" to describe ACLs that can be inherited by child objects. These keywords have received new names: acl_directory_inherit -> acl_default specify_inherit_aces -> specify_default_aces The old keywords are deprecated, but still valid. In addition, a new keyword "acl_inherit" controls inheritance behavior on Windows. This feature does not exist on Unix platforms. - Enterprise API - Export SQL results to sqlite3 database file - Data Collection and cf-hub - cf-hub has got an option -q to query reports from the running agent. This option used to reside in cf-runagent, and has been moved to cf-hub. - Full and delta reports send only mon and sys variables as also hosts excluding policy server are reporting only subset of monitoring data. This can be changed using access promise in default cf_serverd.cf policy. - Removed license checking on hosts. Bugfixes: - Mission Portal - promise finder now does string matching - assigning roles in user management now makes sense - report builder now has a "new query" button - blue hosts' lacking data history is indicated correctly now - CSS fixes - finders no longer load duplicate list items - Fix inconsistent behavior of black host status directly after install. - Enterprise API - Fix some REST queries not working on replica secondaries (eg. the /rest/host/:id) - Remove HTML output from Total Compliance report 3.0.x Removed unused options "[-t][-r][-u]" from cf-know 3.0.1 $(sys.licenses_installtime) variable removed from "Enterprise Free" 3.0.0 New Reporting Engine: A SQL interface to reports collected by hub. We allow all standardised SQL SELECT constructs to query the SQL reports database, with the following additions: - TIMESTAMP_UNIX() - seconds elapsed since 1970 - TIMESTAMP_UNIX_DAYS() - days elapsed since 1970 These are added to avoid use of non-portable SQL date/time functions. Enterprise API: Read + write REST interface for - report querying(utilizes underlying reporting engine) - user management - can be used with REST API v1 in parallel Configurable hostnames(host/system identifier) in reports - can take any of the sys variables(eg.$(sys.fqhost)) When decommissioning (deleting a host) from the Mission Portal/Enterprise API, the public keys of the clients are also removed Ability to delete multiple hosts from the mission portal Improvements on the hub maintenance process - less resource intensive and configurable - New option for cf-hub added (-m) for Enterprise database maintenance Fixes on database connections problems - If you were seeing "connection refused because too many open connections", in database log please consider upgrading Changed "nova>" to "enterprise>" in agent verbose output - Please update email filters Removed internal CFE promises from reporting New classes enterprise, enterprise_X, enterprise_X_Y, enterprise_X_Y_Z on CFEngine Enterprise, to reflect the version running. New variable sys.enterprise_version that holds the CFEngine Enterprise version. This complements the Nova classes and the sys.nova_version variable, which will eventually be deprecated. Fix file change report containing warning message as filename for new/deleted files File diff log (nova_diff.log) have been extended with promise handle name. Total compliance output in cf-agent verbose mode and promise_summary.log have been extended with user and cfengine internal compliance level. System variables are collected by hub in every delta query Fix software reports showing "(never)" in the "Last seen" column Fix "blue hosts" list being empty for clients that don't have class keys Sendmail is installed by default on the hub - required for emailing of reports 32-bit hub installations no longer supported Created a variable update_policy.mongodb_dir, for cases where MongoDB should not run out of /var/cfengine/state (could grow to tens of gigabytes). Removed commercial_customer class, as it was unused in internal policies. Please use enterprise_edition instead if you used this in your policies. New performance report events: DBPurgeHostsAll, DBMaintenance, DBMaintenanceTimestampsSingleHost, DBCacheCompliance, DBReportCollectAll. License verification is made more robust by not relying on the last-seen database anymore. This means you do not need to bootstrap a client to verify the license. See the cf-key --install-license option. 2.2.0 More diagnostics on report collection from cf-hub. Logging more useful information in cf-hub -l, measuring total collection time in benchmarks report, under id "ReportCollectAll". Fixed issue where client would show as green in the Mission Portal when no data was received, e.g. due to access or license error at client. Now correctly shows as blue in these cases. Greatly reduced amount of connections from cf-hub to localhost mongodb. Now there is one connection per cf-hub run (max 50), before it was three per client. Software and variable report now contains end-node discovery time. Software and patches available/status reports contain maximum 5-minute old data, improved from 6 hours in last release. This will only apply to clients that are upgraded to 2.2.0. Software report query from Mission Portal is much faster on larger data-sets due to removal of autocomplete feature. Upgraded mongod from version 1.8.2 to 2.0.4, which increases efficiency on concurrency and reduces memory usage. New function hostswithclass() that generates a list of hosts in a given class on the hub. The bundled failsafe.cf policy now has trustkey=false to avoid IP spoofing attacks in default policy 2.1.0 New interface with interactive graphs on Nova hub. User management on hub. Allowing TAB in file diff report. Better handling of large diffs and attempts to diff binaries. Support for adding notes to all hosts and reports. Support for policy staging environments. Nova reports can be exported to file and imported manually using cf-report -x and cf-report -i. Faster collection of monitoring (vitals) data due to new protocol and data structure. Long-term storage (one year) of diff and changes reports. The variables report got a last-seen column. Now variables are stored in the hub for a longer time like classes, and not overwritten on every update. The report of reports-promises are shown in the promise repaired log in the Mission Portal. New option cf-hub --cache, recreates the cache data needed by the web interface. Only showing the last seen host name and ip address when listing hosts and on the host page. Reliability improvements, especially when querying the variables report. 2.0.2 Much faster report querying. Allows to specify age interval when querying promise not kept and repaired. Promiser conflict identifcation. Built with Cfengine Community Edition 3.1.4. 2.0.1 Fixed promise query of not kept/promise repaired logs. 1.2.0 Windows Event Logs include output_prefix if set, and which component reported the event. The verbosity of event logs have been reduced by not including promise kept and repaired events, this can now be tuned with action.log_level. Take out network communications from total state calculation, as it gets counted twice. 1.1.2 Encryption problems fixed in Community Edition. Built with Cfengine Community Edition 3.0.4p3. 1.1.0 Regular expressions in file paths supported on Windows by using forward slash as path separator. CPU utilization report on Windows. Users logged in report on Windows. On the Windows cf-serverd, requests for /var/cfengine are translated to $(sys.workdir)\Cfengine, and path separators are automatically adjusted ("/" becomes "\"). This yields support for more platform-independent promises and allows for automatic copying of reports from Windows clients to the policy server. Scale on graphs in the Knowledge Map, and different background color gives more readability. Special functions added for accessing remote classes for distributed cooperation. 1.0.0 Reports added to cf-report for compliance, setuid, file_changes etc. Added csv format also Automating topic map integration of policy, with impact analysis using promisee and builds_on promises. Literal string lookup in server. Database SQL and registry functions added. Verification and sanity checking of SQL database table structure. Create and destroy databases convergently. Access control list support for Linux. Powerful and lightweight promises for Customizable monitoring and system discovery promises added to cf-monitord. Longterm memory for 3 year trend analysis.
Supported Platforms and Versions
CFEngine works on a wide range of platforms, and the CFEngine team strives to provide support for the platforms most frequently used by our users.
Enterprise Server
Platform | Versions | Architecture |
---|---|---|
CentOS | 5, 6, 7 | x86-64 |
Debian | 6, 7, 8 | x86-64 |
RHEL | 5, 6, 7 | x86-64 |
Ubuntu | 12.04 | x86-64 |
Any supported host can be a policy server in Community installations of CFEngine.
Hosts
Platform | Versions | Architectures |
---|---|---|
AIX | 5.3*, 6, 7 | PowerPC |
CentOS | 4, 5, 6, 7 | x86-64, x86 |
Debian | 6, 7, 8 | x86-64, x86 |
HP-UX | 11.23+ | Itanium |
RHEL | 4, 5, 6, 7 | x86-64, x86 |
SLES | 10, 11 | x86-64, x86 |
Solaris | 9 | SPARC |
Solaris | 10, 11 | UltraSparc |
Ubuntu | 10.04, 12.04 | x86-64, x86 |
Windows | 2008 | x86-64, x86 |
Windows | 2008, 2012 | x86-64 |
* AIX 5.3 is required to have "5300-05-CSP" or later
Known Issues also includes platform-specific notes.
CFEngine Enterprise has Virtual I/O Server (VIOS) Recognized status from IBM. This means that CFEngine Enterprise has been technically verified by IBM to be installed in and manage VIOS environments.
Hub/Host compatibility
An upgrade path from previous versions is available.
Some data will not be available from hosts on version 3.5.x or older, and the policy you serve needs to take into account hosts with different versions.
Future platform support
The CFEngine team will continue to support future releases of popular Host platforms, including RHEL, Debian, Ubuntu, as well as maintaining support for existing platforms important to users.
In general, CFEngine is known to run on a wide range of other platforms. As long as the platform is POSIX compliant and has a C compiler toolchain that fully implements the C99 standard, we are happy to work with you to make CFEngine available. Please contact our sales team for details.
Policy Framework Updates
CFEngine Policy Framework Updates for 3.9
If you follow the CFEngine masterfiles policy framework (the masterfiles you get out of the box) we encourage you to upgrade the policy framework each time you upgrade CFEngine. We recommend making as few changes as possible to the shipped masterfiles to make these upgrades as painless as possible. Generally the best way to accomplish that is to take your custom policy and integrate it on top of the new masterfiles.
3.9 introduces some minor re-orginization of policy, and some new features aimed at making policy framework upgrades easier.
Please consult The Policy Framework for a map to the policy framework.
What is new in the 3.9 masterfiles policy framework
CHANGELOG.md
Changelog
Notable changes to the framework should be documented here
[Unreleased][unreleased]
Added
- External watchdog policy to ensure that cf-execd is running so that policy will be
run on schedule.
- This policy configures /etc/cron.d/cfengine_watchdog if /etc/cron.d is present to check for cf-execd once a minute and launch it if it is not running.
- The policy can be enabled by defining the class cfe_internal_core_watchdog_enabled, or disabled by defining cfe_internal_core_watchdog_disabled. In the event both classes are defined at the same time enabled wins. ###### Fixed
- Augmenting inputs from the augments_file (Redmine #7420)
3.7.0
Added
- CHANGELOG.md
- Support for user specified overring of framework defaults without modifying policy supplied by the framework itself (see example_def.json)
- Support for def.json class augmentation in update policy
- Run vacuum operation on postgresql every night as a part of maintenance.
- Add measure_promise_time action body to lib (3.5, 3.6, 3.7, 3.8)
- New negative class guard
cfengine_internal_disable_agent_email
so that agent email can be easily disabled by augmenting def.json
Changed
- Relocate def.cf to controls/VER/
- Relocate update_def to controls/VER
- Relocate all controls to controls/VER
- Only load cf_hub and reports.cf on CFEngine Enterprise installs
- Relocate acls related to report collection from bundle server access_rules to controls/VER/reports.cf into bundle server report_access_rules
- Re-organize cfe_internal splitting core from enterprise specific policies and loading the appropriate inputs only when necessary
- Moved update directory into cfe_internal as it is not generally intended to be modified
- services/autorun.cf moved to lib/VER/ as it is not generally intended to be modified
- To improve predictibility autorun bundles are activated in lexicographical order
- Relocate services/file_change.cf to cfe_internal/enterprise. This policy is most useful for a good OOTB experience with CFEngine Enterprise Mission Portal.
- Relocate service_catalogue from promsies.cf to services/main.cf. It is intended to be a user entry. This name change correlates with the main bundle being activated by default if there is no bundlesequence specified.
- Reduce benchmarks sample history to 1 day.
- Update policy no longer generates a keypair if one is not found. (Redmine: #7167)
- Relocate cfe_internal_postgresql_maintenance bundle to lib/VER/
- Set postgresql_monitoring_maintenance only for versions 3.6.0 and 3.6.1
- Move hub specific bundles from lib/VER/cfe_internal.cf into lib/VER/cfe_internal_hub.cf and load them only if policy_server policy if set.
- Re-organize lib/VER/stdlib.cf from lists into classic array for use with getvalues
- inform_mode classes changed to DEBUG|DEBUG_$(this.bundle):: (Redmine: #7191)
- Enabled limit_robot_agents in order to work around multiple cf-execd processes after upgrade. (Redmine #7185)
Deprecated
Removed
- Diff reporting on /etc/shadow (Enterprise)
- Update policy from promise.cf inputs. There is no reason to include the update policy into promsies.cf, update.cf is the entry for the update policy
- not_repaired outcome from classes_generic and scopedclasses generic (Redmine: # 7022)
Fixed
- standard_services now restarts the service if it was not already running when using service_policy => restart with chkconfig (Redmine #7258)
- Fix process_result logic to match the purpose of body process_select days_older_than (Redmine #3009)
Security
Makefile
The masterfiles now installs in the traditional UNIX way
using autotools. For example to install it under /my/path/to/masterfiles
,
you should unpack it and do the following:
./configure --prefix=/my/path/to
make install
def.json
Many featues previously enabled in def.cf
can now be enabled via this
external data file. The benefit is fewer modifications to the policy framework
that need to be worked out during policy framework upgrades.
Known Issues
CFEngine defects are managed in our bug tracker. Please report bugs or unexpected behavior there, following the documented guideline for new bug reports.
- Core Issues affecting 3.9
The items below highlight issues that require additional awareness when starting with CFEngine or when upgrading from a previous version.
HP-UX specific
- Package promises do not have out-of-the-box support for the HP-UX specific package manager. The workaround is to call the package manager directly using commands promises.
- Some important system information is missing from the HP-UX inventory report, as well as from CFEngine hard classes and system variables. The workaround is to use system tools to obtain the required information and set classes based on this. * Disk free * Memory size * Several OS and architecture specific attributes * System version * System serial number * System manufacturer * CPU model * BIOS version * BIOS vendor
- Process promises depend on the
ps
native tool, which by default truncates lines at 128 columns on HP-UX. It is recommended to edit the file/etc/default/ps
and increase theDEFAULT_CMD_LINE_WIDTH
setting to 1024 to guarantee that process promises will work smoothly on the platform. - Upgrading CFEngine on HP-UX is not supported by the out-of-the-box policy. There is a support article with a workaround.
Enterprise emails sent for alert noticies come from 'admin@organization.com'.
There is currently no setting in Mission Portal to configure the sender email address. This issue is tracked in ENT-695 and will be addressed in a future release.
To change the setting you must edit the from email address in
/var/cfengine/share/GUI/application/config/appsettings.php
. Policy in the
Masterfiles Policy Framework will take care of updating the running config
during the next policy run.
// Default FROM email address
$config['appemail'] = 'admin@organisation.com';
Enterprise monitoring graphs
Monitoring graphs are disabled by default in CFEngine Enterprise 3.6 and later versions. To enable them, change monitoring_include in masterfiles/controls/VERSION/reports.cf to e.g. ".*". Note that this has a significant impact on the resource consumption of your hub.
Monitoring graphs are not supported on all platforms, currently AIX, HP-UX and Windows do not have this data.
Enterprise software inventory is not out-of-the-box
Software inventory is not out-of-the-box for reporting from the hub on other platforms than Debian, Ubuntu and Red Hat/CentOS.
In order to add software inventory for other platforms, please contact support for a custom policy.
Enterprise - Unable to edit directory based users with dots in username
Mission Portal does not allow users from a directory to be edited if they have dots in their username.
Enterprise Hub - PHP warnings after upgrading from 3.6.x
After upgrading from 3.6.x
PHP warns it is unable to
initialize the apc module.
notice: Q: "...hp/bin/php /var": PHP Warning: PHP Startup: apc: Unable to initialize module
Q: "...hp/bin/php /var": Module compiled with module API=20100525
Q: "...hp/bin/php /var": PHP compiled with module API=20131226
Q: "...hp/bin/php /var": These options need to match
Q: "...hp/bin/php /var": in Unknown on line 0
This warning can be resolved by removing
/var/cfengine/httpd/php/lib/apc.ini
and
/var/cfengine/httpd/php/lib/php/extensions/no-debug-non-zts-20131226/apc.so
Enterprise Hub - Mission Portal database not properly preserved
When upgrading from earlier versions, the Mission Portal database may not be preserved, resulting in Mission Portal to appear as a fresh installation.
Incompatible Apache config during upgrade
Mis-spellings in the 3.9.0
release of the MPF when run on a hub with 3.6.x
binaries results in Apache getting an incompatible configuration.
Fixed in 3.9.1
with this change.
Dynamic bundle actuation results in error about cf_null
Jira #CFE-2426
error: A method attempted to use a bundle 'cf_null' that was apparently not defined
This is a benign error. cf_null
is an internal implementation detail that is
used to handle empty lists.
Workarounds:
- Explicitly guard against iterating methods on an empty list.
This snippet shows one way to define a class if a list is not empty.
classes:
"have_some_zero_dynamic_role_bundles"
expression => some( ".*", "roles_dynamic.bundles" );
- Ignore missing bundles in body common control
body common control
{
#...
ignore_missing_bundles => "true";
#...
}
- Add an empty =cf_null= bundle
bundle common cf_null
{
reports:
!any::
"This works around an issue iterating over an list of bundles.";
}
Installation and Configuration
Installation
There are several steps to bring up a CFEngine installation within an organization:
- Prepare all appropriate machines for installation.
- Configure your network and security.
- Download the CFEngine software.
- Install CFEngine on the Policy Server(s).
- Bootstrap the Policy Server to itself.
- Initiate post-install configuration on the Policy Server.
- Install CFEngine on the Host machine(s).
- Bootstrap the Host(s) to a Policy Server.
See General Installation for a more detailed guide for how to install CFEngine, and links to installation guides for various versions of CFEngine and different configurations.
See Secure Bootstrap for a guide on bootstrapping CFEngine in untrusted networks.
See also: Pre-Installation Checklist, Supported Platforms and Versions
Setup & Configuration
Additional options for configuring CFEngine policy are as follows:
Controlling Frequency Learn how to control frequency settings for verifying CFEngine policy.
Version Control Learn how to put your CFEngine policies under version control.
The Policy Framework Learn what options are available out of the box in CFEngine to configure its masterfiles operation.
Pre-Installation Checklist
Download Packages
System requirements
Please see Installing Enterprise for Production for hardware and configuration requirements, and for Supported Platforms and Versions operating system support.
Required Knowledge
- Linux
- SSH
- bash
- command line text editing (e.g. vi/vim, Emacs)
See Also: Quick-Start Guide to Using vi, Quick-Start Guide to Using PuTTY
Quick-Start Guide to Using vi
Quick-Start Guide To Using vi
This guide is designed for the novice user of CFEngine tutorials—and will introduce the basic use of a powerful tool that is referenced in the CFEngine learning documentation: the vi visual editor.
What is a visual editor? It lets you see multiple lines of the document you are editing—rather than simply issuing commands in the shell prompt. This means you can insert a very large piece of text and navigate anywhere in that text, and make changes.
The vi editor was developed for unix—but can run from any shell prompt, like PuTTY on the Windows platform, and also the Mac. So whatever the user's platform may be, learning to use vi will be very useful in working through the CFEngine tutorials.
When working in the CFEngine tutorials, vi will be used to do things like open files, insert text, save files, and many other functions.
vi will also be used when the CFEngine user starts to actually use the CFEngine software—for things like writing and deploying promises, the core of the CFEngine technology.
Learning the basics of vi is quite simple. The best way is by walking through an example.
Step 1. Inside the shell prompt, simply type “vi”. This will allow the user to insert text and create a new file.
Step 2. type “i” then press the “Enter” key. This takes the user to the insert mode, and allow typing in text or copying and pasting.
Step 3. Type some text—for example, the obligatory “Hello World” (which will be the subject of a later tutorial). Now press "Enter" to go to the next line and type “My name is Gary, and it's nice to meet you.”
The output will look like this:
Hello World
My name is Gary, and it's nice to meet you
Step 4. Now exit the insert mode and go back to command mode by pressing the “esc” key.
Step 5. Save the file by typing “:w (filename)”
Step 6. exit vi by typing “:q”
You can also save and exit with one command, “:wq”
It is important to remember that there are two basic operation modes in vi: the command mode, with which the user opens, saves and exits from files, and the insert mode with which the user inserts text—either by typing it in, or by copying and pasting—and can then edit any part of the text in the file.
open file using vi filename
Quick-Start Guide to Using PuTTY
- Using PuTTY in Simple Steps
- Accessing AWS Virtual Machines via SSH on Windows Using PuTTY and PuTTYgen
Using PuTTY in Simple Steps
This guide is intended for Windows users who are not accustomed to using SSH, or need some additional support for understanding how to work with SSH from their machine (e.g. challenges with key pairs).
It describes how to start using the free, open-source program PuTTY, to securely connect a client computer to a remote Linux/Unix server.
Many of the tutorials to follow will refer to using PuTTY, which is a popular SSH client for Windows workstations.
The important thing about PuTTY is that it is a secure way to connect a client to a server, using the SSH network protocol. It has a powerful and easy-to-use graphical user interface (GUI) and is used to run a remote session over a network.
What is SSH? It is short-form for “Secure Shell,” which means it creates a secure channel over an insecure network—like the internet, for example.
How does SSH do this? By encrypting the communications between the client and the server, using public-key cryptography, which means that a key-pair is generated—one of them public, and the other private, or secret, known only to the user.
Since CFEngine is a client-server enterprise software system, it is essential to access the servers securely. This is true whether the CFEngine system is run on a cloud platform, like Amazon Web Services and many others—or on a private network.
That is where PuTTY comes into the picture, since it uses SSH protocol for connecting a client to a server.
The PuTTY software consists of two separate programs PuTTY and PuTTYgen: They can be downloaded at http://www.chiark.greenend.org.uk/~sgtatham/putty/download.html
PuTTYgen is used to generate the encryption key pair while PuTTY, a command-line interface, is used to securely access the CFEngine server, or hub, from a remote client machine, which is called a host in CFEngine terminology.
PuTTYgen is used only when setting up a new client machine on the CFEngine hub. The CFEngine hub will already have an encrypted key-pair that was created when setting up the hub. (See the tutorial, Installing CFEngine on RHEL Using AWS)
The following steps describe how to get the client machine, up and running using PuTTYgen and PuTTY. There are two distinct steps to this process:
Step 1. Use PuTTYgen to create an encrypted key-pair in the .ppk file format that PuTTY uses.
(It is important to note that the key-pair on the hub will probably be in a file format that is different from the PuTTYgen .ppk file format. For example on Amazon Web Services (AWS) and many other cloud computing services, the key-pair file format created when setting up the server (_hub_) will be in the .pem file format.)
Step 2. Configure the PuTTY application in order to securely access the CFEngine hub.
Step 1. consists of the following sequence: First, launch PuTTYgen by double-clicking on the puTTygen icon in the Windows programs menu tree; (It should be inside the PuTTY folder that was created when the PuTTY was downloaded and installed.)
Next, download the key-pair and save it on the local hard disk in the .ppk file format.
a. Click Load. The following Load private key window will pop up:
b. In the Load private key window select All Files (*.*) in the drop down menu next to the File name input box.
c. Navigate to the location on disk where the public-key file was downloaded in earlier steps, in this case a .pem file. Click Open. The following window will appear:
d. Enter a Passphrase and confirm the Passphrase. If no Passphrase is desired, leave those fields empty.
e. When the key has been loaded click the Save private key button.
f. If saving without a Passphrase a dialog box will pop up; click yes to save the key without a Passphrase
g. Now close PuTTYgen.
Accessing AWS Virtual Machines via SSH on Windows Using PuTTY and PuTTYgen
Get PuTTY and PuTTYgen
- To get PuTTY and PuTTYgen, first go to
http://www.chiark.greenend.org.uk/~sgtatham/putty/download.html and either:
- Download and install using the PuTTY binaries installer
- Or, download PuTTY and PuTTYgen individually
Prepare Private Key Using PuTTYgen
- After the binaries have been downloaded and/or installed either:
- Double click
puttygen.exe
from the download location, if downloaded directly. - Or, if the PuTTY installer was used above, one of either:
- Press the
Windows
key +R
key and then typeputtygen
in the field namedOpen
. Then press theEnter
key or clickOK
. - Alternatively, double click puttygen.exe under
C:\Program Files (x86)\PuTTY
(when using Windows 64 bit) orC:\Program Files\PuTTY
(when using Windows 32 bit).
- Double click
The Puttygen Interface. You will load the .pem file that you created in AWS.
- On the PuTTYgen interface click the load button.
- In the Load private key window select All Files (*.*) in the drop down menu next to the File name input box.
- Navigate to the location on disk where the .pem file was downloaded in earlier steps.
- When the key has been loaded click the Save private key button.
- When prompted with a warning about saving without a passphrase, click yes.
The Puttygen popup window. Click Yes
, to proceed without a passphrase. You can also protect your private key with a passphrase that you enter into Key Passprhase
and Confirm Key Passphrase
.
- Finally, navigate to a good location on disk to save the key file, enter a name for the private key, ensure PuTTY Private Key Files (*.ppk) type is selected, and then click the Save button.
- You can now close the Puttygen application. You will call up the .ppk file when you configure the virtual machines using PuTTY.
Configure PuTTY
- Before configuring PuTTY, go back to your AWS Console, then navigate to INSTANCES > Instances.
- Make a note of the 2 different Public DNS entries for the virtual machines that were setup earlier (e.g. ec2xxxxxxxxxxxx.uswest1.compute.amazonaws.com, where the x's represent numbers).
- Launch PuTTY by either:
- Double clicking
putty.exe
from the download location, if downloaded directly. - Or, if the PuTTY installer was used above, one of either:
- Press the
Windows
key +R
key and then typeputty
in the field namedOpen
. Then press theEnter
key or clickOK
. - Alternatively, double click
putty.exe
underC:\Program Files (x86)\PuTTY
(when using Windows 64 bit) orC:\Program Files\PuTTY
(when using Windows 32 bit). - On the PuTTY interface, select
Category > Session
on the left side navigation tree:
- Double clicking
The Putty interface, with Session
selected on the left-side navigation tree.
- Now, we will configure the Putty application, which we will use to set up the two AWS virtual machines.
- The first step is to create a Host Name for the first VM.
- The Host Name consists mainly of the public DNS entry that was created for one of the two virtual machines in AWS. But the DNS is preceded by a user name,
ec2-user
, followed by the@
symbol, which is then followed by the DNS entry.
Setting up the PuTTY configuration with the Host Name, and a Saved Sessions Name.
- Port should be set to
22
. - Connection type should be set to
SSH
. Saved Sessions
can be any label.
Once we have entered our Host Name and our Saved Sessions name, we take the following steps:
* Select Connection > SSH > Auth
on the left side navigation tree.
* Click the Browse
button to select the Private key for authentication
.
* In the Select private key file
window, navigate to the .ppk private key file created earlier, and double-click on it to enter it into PuTTY. Your PuTTY screen should look like this:
Note that Auth
has been selected on left-side tree, in order to bring up this screen.
- Now we go back and select
Category > Session
on the left side navigation tree and then press theSave
button. - Repeat the steps for the second virtual machine, starting from setting the Host Name through pressing the Save button (as described above). Your PuTTY screen should show the two saved virtual machines, which are here named
Examples 1 and 2.
- Note: It may be necessary to redo the steps from selecting
Connection > SSH > Auth
through selecting the .ppk private key file. In other words, when configuring the connection the private key file may not be persistently saved. - Wait a moment, and select
Yes
if prompted. - This prompt will generally only be necessary when trying to login for the very first time.
The PuTTY interface with the two virtual machines saved. We can now proceed to configure those virtual machines with CFEngine.
Login to Virtual Machines Using PuTTY
- If one of the two virtual machines is configured and its details loaded in the PuTTY interface, first select the machine, then click the Open button. This will close the above PuTTY interface and open a command-line window, from which we will setup CFEngine on each of the two machines. One machine will act as the Server and the other as the client, and they will each be set up with different software.
- Once the first virtual machine is logged into, right click the top of PuTTY's application window (e.g. the part of the window decoration displaying the virtual machine name).
- In the contextual menu that then shows click New Session.
- Select the second virtual machine entry in the Saved Sessions list.
- Click Load and then Open.
- Both virtual machines should now be accessed in two different PuTTY command-line windows. Below is an example of what the command-line window will look like.
The PuTTY command-line window, which we will use to configure the virtual machines with CFEngine.
General Installation
There are several steps to bring up a CFEngine installation within an organization:
- Prepare all appropriate machines for installation.
- Configure your network and security.
- Download the CFEngine software.
- Install CFEngine on the Policy Server(s).
- Bootstrap the Policy Server to itself.
- Initiate post-install configuration on the Policy Server.
- Install CFEngine on the Host machine(s).
- Bootstrap the Host(s) to a Policy Server.
Before Installation
Check the Pre-Installation Checklist and Supported Platforms and Versions for requirements and other information that is useful for the installation procedure.
Install Packages
CFEngine Enterprise is provided in two packages; one is for the Policy Server (hub) and the other is for each Host (client).
Note: See Installing Community for the community version of CFEngine)
Log in as root and then follow these steps to install CFEngine Enterprise:
On the designated Policy Server, install the
cfengine-nova-hub
package:[RedHat/CentOS/SUSE] $ rpm -i <server hub package>.rpm [Debian/Ubuntu] $ dpkg -i <server hub package>.deb
On each Host, install the
cfengine-nova
package:[RedHat/CentOS/SUSE] $ rpm -i <agent package>.rpm [Debian/Ubuntu] $ dpkg -i <agent package>.deb
Note: Install actions logged to /var/logs/cfengine-install.log
.
Bootstrap
Bootstrapping a client means to configure it initially. With CFEngine, the default bootstrap:
- records the server's address (accessible as
sys.policy_hub
) and public key, and gives the server the client's key to establish trust (see Bootstrapping) - copies all the contents of
/var/cfengine/masterfiles
on the policy server (AKAsys.masterdir
) to/var/cfengine/inputs
(AKAsys.inputdir
). Seeupdate.cf
for details.
Run the bootstrap command, first on the policy server:
Find the IP address of your Policy Server:
$ ifconfig
Run the bootstrap command:
$ sudo /var/cfengine/bin/cf-agent --bootstrap <IP address of policy server>
The bootstrap command must then be run on any client attaching itself to this server, using the ip address of the policy server (i.e. exactly the same as the command run on the policy server itself).
Post-Installation Configuration
CFEngine itself is configured through policy as well (see Components and Common Control and
The Policy Framework for details). The following basic changes to the default policy will configure
cf-serverd
and cf-execd
for your environment.
Configure agent email settings
By default an email a summary of any cf-agent
run initiated by cf-execd
. You
may want to adjust the mailto or mailfrom. If you have a centralized reporting
system like CFEngine Enterprise you may wish to disable agent emails all
together.
Configure mailto and mailfrom
The preferred way of setting def.mailfrom
is from the
augments file.
{
"vars": {
"mailfrom": "sender@your.domain.here",
"mailto": "recipient@your.domain.here"
}
}
Alternatively you can alter the setting in def.cf
.
Note: On some systems these modifications should hopefully work without needing to make any additional changes elsewhere. However, any emails sent from the system might also end up flagged as spam and sent directly to a user's junk mailbox.
Note: It's best practice to restart daemons after adjusting it's settings to ensure they have taken effect.
Disable agent emails
The preferred way to disable the agent from sending emails is to define
cfengine_internal_disable_agent_email
from the augments file.
{
"classes": {
"cfengine_internal_disable_agent_email": [ "any" ]
}
}
Alternatively you can define the class from def.cf
.
Note: It's best practice to restart daemons after adjusting it's settings to ensure they have taken effect.
Server IP Address and Hostname
Edit /etc/hosts
and add an entry for the IP address and hostname of the server.
CFEngine Enterprise Post-Installation Setup
See: What steps should I take after installing CFEngine Enterprise?
More Detailed Installation Guides
Although most install procedures follow the same general workflow, there are several ways of installing CFEngine depending on your environment and which version of CFEngine you are using.
- Installing Enterprise for Production
- Install and test the latest version using our native version, for free!
- Installing CFEngine on virtual machine instances using Amazon Web Services' (AWS) EC2 service
- This is especially useful for people running Windows on their workstation or laptop.
- Install and test the latest version using our pre-packaged Vagrant environment
- Installing CFEngine Community Edition
Next Steps
- Learn about Writing and Serving Policy
Using Amazon Web Services
This guide describes how to install CFEngine on two Red Hat® Enterprise Linux® (RHEL) virtual machines using Amazon Web Services™ (AWS) and SSH. At the time of writing, under certain conditions, setting up an AWS account and using micro-instances is free.
One of the two machines will be a policy server
, while the other will be a host
.
Although these instructions walk through the steps needed to install CFEngine Enterprise on two machines, up to 25 machines can be set up using the same procedure and scripts.
This tutorial will cover the following steps:
- Initial Configuration of the AWS Virtual Machines.
- Configuring the Security Group.
- Configuring SSH Access to the Virtual Machines Using PuTTY (for Windows machines).
- Configuring the Firewall on the Policy Server.
- Installing CFEngine on both the Policy Server and Host Virtual Machines.
Initial Configuration of the Virtual Machines in AWS
Configure 2 RHEL Virtual Machine Instances in AWS
- Login to AWS.
- Under
Create Instance
click onLaunch Instance
. - On the line
Red Hat Enterprise Linux 64 Bit Free tier eligible
press theSelect
button. - On the
Choose Instance Type
screen ensure theMicro Instances
tab on the left is selected.
Configure Instance Details
- Press
Next: Configure Instance Details
. - On the
Configure Instance Details
screen change the number of instances to 2. - Leave
Network
as the default. Subnet
can beNo preference
.- Ensure
Public IP
is checked. - Leave all else at their default values.
Review and Launch
- Click
Review and Launch
. - Make a note of
Security group
name on theReview Instance Launch
screen. - Click
Launch
. - Select
Create a new key
pair in the first drop down menu. - Enter anything as the
Key pair name
. - Click the
Download Key Pair
button and save the .pem file to your local computer. - After the .pem file is saved click the
Launch Instance
button. - On the
Launch Status
screen click theView Instances
button.
Configure the Security Group
- On the left hand side of the AWS console click
NETWORK & SECURITY > Security Groups
- Remembering the
Security group
name from earlier, click on the appropriate line item in the list. - Below the list of security group names will display details for the current security group.
- Click the
Inbound
tab. - Click "Edit" button. A popup window will appear with "SSH" rule already present.
- Click the
+Add Rule
button. SelectHTTP
from the drop-down list. Click "Add Rule" button again. - Select
Custom TCP rule
and enter5308
in thePort range
text entry. Select "Custom IP" from the drop-down menu in the "Source" column. - Copy the "Group ID" from the line containing your "Group Name" and copy the "Group ID" into the text entry in the last column. Click "Save."
- Click the "Edit" button again. On the "Custom TCP" Rule, select "Anywhere" from the "Source" drop-down list. Click "Save."
Accessing the Virtual Machines Using SSH
See: Quick-Start Guide to Using PuTTY
Install and Configure the Firewall
Install the Firewall
- Ensure you are logged into both virtual machines.
- In both enter
sudo yum install system-config-firewall
to install. - Hit 'y' if prompted.
Configure the Firewall on the Policy Server (AKA hub)
The following steps are only necessary for one of the two virtual machines, the one that is designated as the policy server; these steps can be omitted on the second (client machine). Note that CFEngine refers to a client machine by the name Host
:
- When system-config-firewall is installed, enter
sudo system-config-firewall
- In the
Firewall Configuration
screen use theTab
key to go to Customize. - Hit the
Enter
key. Below is theFirewall Configuration
window that comes up:
Open Port 80 (HTTPD)
- On the
Trusted Services
screen, scroll down toWWW (HTTP)
, AKA port 80. - Hit the
Space Bar
to toggle theWWW
entry (i.e. ensure it is on, showing an asterisk beside the name).
Open Port 5308 (CFEngine)
- Hit the
Tab
key again untilForward
is highlighted, then hitEnter
. - Hit the
Tab
key untilAdd
is highlighted, then hitEnter
. - Enter
5308
in thePort
section. - Hit the
Tab
key and entertcp
in theProtocol
section. - Hit the
Tab
key until OK is highlighted, and hitEnter
.
The Port and Protocol
are entered in the blue boxes, with entries of 5308
and tcp
respectively.
Then the Tab
key is used to highlight the OK
button, and the user presses Enter
.
Wrapping Up Firewall Configuration
- Hit the
Tab
key untilClose
is highlighted, and hitEnter
. - Hit the
Tab
key or arrow keys untilOK
is highlighted, and hitEnter
.
Disabling Firewall on a Host (Warning: Only Do This If Absolutely Necessary)
For the second virtual machine, which is the client machine (also called host
), you may need to do the following if you see an error when bootstrapping this virtual machine in later steps:
* In the Firewall Configuration
screen use the Tab
key to go to Firewall.
* Turn off the firewall by toggling the entry with the Space
bar.
Note: Turning off the firewall in a production environment is considered unsafe.
CFEngine Installation Overview
We ready now ready to install the CFEngine software on both the server and client virtual machines. These also referred to as the “hub” and “host” machines, respectively. During the course of the instructions outlined in this guide, you will perform the following tasks:
- Install CFEngine Enterprise onto a Policy Server and onto Hosts. A Policy Server (hub) is a CFEngine instance that contains promises (business policy) that get deployed to Hosts. Hosts are clients that retrieve and execute promises.
- Bootstrap the Policy Server to itself and then bootstrap each of the Hosts to the Policy Server. Bootstrapping establishes a trust relationship between the Policy Server and all Hosts. Thus, business policy that you create in the Policy Server can be deployed to Hosts throughout your company. Bootstrapping completes the installation process.
- Log in to the Mission Portal. The Mission Portal is a graphical user interface that allows you to verify the actual state of all your Hosts, thus ensuring that your promises are being executed. By using the Design Center inside the Mission Portal, you can also define new desired states (business policies) for your infrastructure.
- Try out the Tutorials. Links to three tutorials give you a head start on learning CFEngine.
Step 1. Download and install Enterprise on a Policy Server
Run the following script on your designated Policy Server (hub), the virtual machine with the configured firewall from earlier steps:
$ wget http://cfengine.package-repos.s3.amazonaws.com/quickinstall/quick-install-cfengine-enterprise.sh && sudo bash ./quick-install-cfengine-enterprise.sh hub
This script installs the latest CFEngine Enterprise Policy Server on your server machine.
Step 2. Bootstrap the Policy Server
- The Policy Server must be bootstrapped to itself. Find the IP address of your Policy Server:
$ ifconfig
. Run the bootstrap command:
sudo /var/cfengine/bin/cf-agent --bootstrap <IP address of policy server>
Example:
$ sudo /var/cfengine/bin/cf-agent --bootstrap 172.31.3.25
Upon successful completion, a confirmation message appears: "Bootstrap to '172.31.3.25' completed successfully!"
Type the following to check which version of CFEngine your are running:
/var/cfengine/bin/cf-promises --version
The Policy Server is now installed.
Step 3. Install Enterprise on Host (Client)
- Ensure you are logged into the host machine setup earlier.
- Install CFEngine client version using the following:
$ wget http://cfengine.package-repos.s3.amazonaws.com/quickinstall/quick-install-cfengine-enterprise.sh && sudo bash ./quick-install-cfengine-enterprise.sh agent
Note: The installation will work on 64-bit and 32-bit client machines (the host requires a 64-bit machine).
The client software (host), has been installed on the second virtual machine.
Note: You can install CFEngine Enterprise on up to 25 hosts using the script above.
Step 4. Bootstrap the Host to the Policy Server
- All hosts must be bootstrapped to the Policy Server in order to establish a connection between the
Host
and thePolicy Server
. Run the same commands that you ran in Step 2,
$ sudo /var/cfengine/bin/cfagent bootstrap <IP address of policy server>
.Example:
$ sudo /var/cfengine/bin/cfagent bootstrap 172.31.3.25
The installation process is complete and CFEngine Enterprise is up and running on your system.
Step 5. Log in to the Mission Portal
- The Mission Portal is immediately accessible. Connect to the Policy Server through your web browser at: http://
(Note: The External IP address is available in the AWS console). - The default username for the Mission Portal is
admin
, and the password is alsoadmin
. - The Mission Portal runs TCP port 80 by default. Configure mission portal to use HTTPS instead of HTTP.
- During the initial setup, the Host(s) might take a few minutes to show up in the Mission Portal. Refresh the web page and login again if necessary.
What Next?
Tutorials
Configure and deploy a policy using sketches in the Design Center.
This tutorial teaches you how to configure and deploy business policy by using the Design Center application in the Mission Portal. Next, it shows you how to verify that your business policy is being activated by viewing the Reports in the Mission Portal.
Distribute files from a central location.
Whereas the first tutorial in this list teaches you how to deploy business policy through the Mission Portal, this advanced, command-line tutorial shows you how to distribute policy files from the Policy Server to all pertinent Hosts.
Recommended Reading
Installing Enterprise 25 Free
These instructions describe how to install the latest version of CFEngine Enterprise 25 Free. This is the full version of CFEngine Enterprise, but the number of Hosts (clients) is limited to 25.
Note the following requirements:
- To install this version of CFEngine Enterprise, your machine must be running a recent version of Linux. This installation script has been tested on RHEL 5 and 6, SLES 11, CentOS 5 and 6, and Debian 6 and 7.
- You need a minimum of 2 GB of available memory and a modern 64 bit processor.
- Plan for approximately 100MB of disk space per host. You should provide an extra 2G to 4G of disk space if you plan to bootstrap more hosts later.
- You need a least two VMs/servers, one for the Policy Server and one for a Host (client). They must be on the same network.
- The Policy Server needs to run on a dedicated OS with a vanilla installation (i.e. it only has repositories and packages officially supported by the OS vendor)
Installation Overview
During the course of the instructions outlined in this guide, you will perform the following tasks:
- Install CFEngine Enterprise onto a Policy Server and onto Hosts. A Policy Server (hub) is a CFEngine instance that contains promises (business policy) that get deployed to Hosts. Hosts are clients that retrieve and execute promises.
- Bootstrap the Policy Server to itself and then bootstrap each of the Hosts to the Policy Server. Bootstrapping establishes a trust relationship between the Policy Server and all Hosts. Thus, business policy that you create in the Policy Server can be deployed to Hosts throughout your company. Bootstrapping completes the installation process.
- Log in to the Mission Portal. The Mission Portal is a graphical user interface that allows you to verify the the actual state of all your Hosts, thus ensuring that your promises are being executed. By using the Design Center inside the Mission Portal, you can also define new desired states (business policies) for your infrastructure.
- Try out the Tutorials. Links to three tutorials give you a head start on learning CFEngine.
1. Download and install Enterprise on a Policy Server
Please Note: Internet access is required from the host if you wish to use the quick install script.
Run the following script on your designated Policy Server (hub) 64-bit machine (32-bit is not supported on the Policy Server):
$ wget http://cfengine.package-repos.s3.amazonaws.com/quickinstall/quick-install-cfengine-enterprise.sh && sudo bash ./quick-install-cfengine-enterprise.sh hub
This script installs the latest CFEngine Enterprise Policy Server on your machine.
2. Bootstrap the Policy Server
The Policy Server must be bootstrapped to itself. Find the IP address of your Policy Server (type $ ifconfig).
Run the bootstrap command:
$ sudo /var/cfengine/bin/cf-agent --bootstrap <IP address of policy server>
Example: $ sudo /var/cfengine/bin/cf-agent --bootstrap 192.168.1.12
Upon successful completion, a confirmation message appears: "Bootstrap to '192.168.1.12' completed successfully!"
Type the following to check which version of CFEngine your are running:
$ /var/cfengine/bin/cf-promises --version
The Policy Server is installed.
3. Install Enterprise on Hosts
Install Enterprise on your designated Host(s) by running the script below. Per the Free 25 agreement, you can install Enterprise on 25 Hosts. Note that the Hosts must be on the same network as the Policy Server that you just installed in Step 2.
$ wget http://cfengine.package-repos.s3.amazonaws.com/quickinstall/quick-install-cfengine-enterprise.sh && sudo bash ./quick-install-cfengine-enterprise.sh agent
Note that this installation works on 64- and 32-bit machines.
4. Bootstrap the Host to the Policy Server
All Hosts must be bootstrapped to the Policy Server in order to establish a connection between the Host and the Policy Server. Run the same commands that you ran in Step 3.
$ sudo /var/cfengine/bin/cf-agent --bootstrap <IP address of policy server>
Example: $ sudo /var/cfengine/bin/cf-agent --bootstrap 192.168.1.12
The installation process is complete and CFEngine Enterprise is up and running on your system.
5. Log in to the Mission Portal
The Mission Portal is immediately accessible. Connect to the Policy Server through your web browser at:
http://<IP address of your Policy Server>
username: admin password: admin
The Mission Portal runs TCP port 80 by default. (Click here to configure the Mission Portal to use HTTPS instead of HTTP.) During the initial setup, the Host(s) might take a few minutes to show up in the Mission Portal. Simply refresh the web page and login again if necessary.
Note: If you are running Enterprise with Vagrant, you must add the
correct port: http://localhost:
Tutorials
Configure and deploy a policy using sketches in the Design Center.
This tutorial teaches you how to configure and deploy business policy by using the Design Center application in the Mission Portal. Next, it shows you how to verify that your business policy is being activated by viewing the Reports in the Mission Portal.
Distribute files from a central location.
Whereas the first tutorial in this list teaches you how to deploy business policy through the Mission Portal, this advanced, command-line tutorial shows you how to distribute policy files from the Policy Server to all pertinent Hosts.
Recommended Reading
Rate your experience
Everyone is a first-time user a some point. We want to make the CFEngine Enterprise installation process easy for all of our new users. Before you forget your first-time experience, we would love for you to let us know how we can improve on this process.
Using Vagrant
The CFEngine Enterprise Vagrant Environment provides an easy way to test and explore CFEngine Enterprise. This guide describes how to set up a client-server model with CFEngine and, through policy, manage both machines. Vagrant will create one VirtualBox VM to be the Policy Server (server), and another machine that will be the Host Agent (client), or host that can be managed by CFEngine. Both will will run CentOS 6.5 64-bit and communicate on a host-only network. Apart from a one-time download of Vagrant and VirtualBox, this setup requires just one command and takes between 5 and 15 minutes to complete (determined by your Internet connection and disk speed). Upon completion, you are ready to start working with CFEngine.
Requirements
- 2G disk space
- 1G memory
- CPU with VT extensions capable of running 64bit guests
Note: VirtualBox requires that your computer support hardware virtualization in order to make use of the CentOS 64-bit virtual machines mentioned above. This is sometimes turned on or off in BIOS settings, but not all processors and motherboards necessarily support hardware virtualization.
If your system lacks this support you will need to choose another computer to take advantage of the 64-bit virtual machines or install CFEngine using a different approach.
Overview
- Install Vagrant
- Install Virtualbox
- Start the CFEngine Enterprise Vagrant Environment
- Log in to the Mission Portal
- Stop CFEngine Enterprise
- Uninstall
Install Vagrant
This tutorial uses Vagrant to configure your VMs. It is available for Linux, Windows and MacOS and can be downloaded from vagrantup.com (this guide has been tested with version 1.7.4). After downloading Vagrant, install it on your computer. You may want to reference the Windows Mac or Linux vagrant install guides.
Install Virtualbox
This tutorial uses VirtualBox to create virtual machines on your computer, to which Vagrant deploys CFEngine. VirtualBox can be downloaded from virtualbox.org (this guide has been tested with version 5.0.2). After downloading VirtualBox, install it on your computer.
Note: To avoid problems, disable other virtualization environments you are running.
Start the CFEngine Enterprise 3.9 Vagrant Environment
Step 1. Download our ready-made Vagrant project tar-file.
Step 2. Save and unpack the file anywhere on your drive; this creates a Vagrant Project directory.
Step 3. Open a terminal and navigate to the Vagrant Project directory (e.g.
/home/user/CFEngine_Enterprise_vagrant_quickstart-3.9.1-1
, or C:\CFEngine_Enterprise_vagrant_quickstart-3.9.1-1
) and enter the following command:
$ vagrant up
Vagrant performs the following processes:
- Downloads the CentOS basebox used for both the hub and the client (if it has not already been cached by vagrant.
- Provisions, installs and bootstraps the hub
- Provisions, installs and bootstraps clients
The basebox is ~500MB.
Note: If you want to use more hosts in this environment, you can edit the Vagrantfile text file in the directory that you have just created. Change the line that says "hosts = 1" to the number of hosts that you want in the setup. The maximum supported in this evaluation version of CFEngine is 25.
Log in to the Mission Portal
At the end of the setup process, you can use your browser to log in to the Mission Portal:
http://localhost:9002
username: admin
password: admin
Note: It may take up to 15 minutes before the hosts register in Mission Portal.
That's all there is to it, the install is complete! Move on and explore the environment.
Exploring the Environment
Accessing VMs
Accessing via SSH
The standard vagrant ssh key is configured. To ssh to a host run vagrant ssh
myhost
where myhost
is the name of a running vm as seen in the vagrant
status
output. Both the 'root' and 'vagrant' users passwords are set to
'vagrant'.
Example:
$ vagrant ssh hub
Last login: Fri Jun 13 18:58:10 2014 from 10.0.2.2
Accessing via GUI
If you launch the virtualbox GUI you should find the vagrant vms named
CFEngine Enterprise 3.9.1-1 hub
, and CFEngine Enterprise 3.9.1-1 agent host001
. Additionally, you can uncomment the v.gui=true
option in the Vagrantfile
to have the console gui start with the vms.
Note: There are two v.gui
settings to uncomment; one for the hub, and one
for the clients.
Check the status of the vms
Running vagrant status
from the vagrant project directroy will produce
output like this.
$ vagrant status
Current machine states:
hub not created (virtualbox)
host001 not created (virtualbox)
This environment represents multiple VMs. The VMs are all listed
above with their current state. For more information about a specific
VM, run `vagrant status NAME`.
Start or resume the environment
To start or resume a halted environment simply run vagrant up
from within the
vagrant project directory.
$ vagrant up
Stop the environment (Halt/Suspend/Destroy)
To shut down the vms run vagrant halt
. This will preserve the vms and any
changes made inside.
$ vagrant suspend
==> hub: Saving VM state and suspending execution...
==> host001: Saving VM state and suspending execution...
To suspend the vms run vagrant suspend
. This will freeze the state of each vm
and allows for latter resuming of the environment.
$ vagrant halt
==> host001: Attempting graceful shutdown of VM...
==> hub: Attempting graceful shutdown of VM...
At any time you can run vagrant destroy
to remove the provisioned vms. This will
delete the vms and any modifications made to the environment will be lost.
$ vagrant destroy
host001: Are you sure you want to destroy the 'host001' VM? [y/N] y
==> host001: Forcing shutdown of VM...
==> host001: Destroying VM and associated drives...
==> host001: Running cleanup tasks for 'shell' provisioner...
==> host001: Running cleanup tasks for 'shell' provisioner...
==> host001: Running cleanup tasks for 'shell' provisioner...
hub: Are you sure you want to destroy the 'hub' VM? [y/N] y
==> hub: Forcing shutdown of VM...
==> hub: Destroying VM and associated drives...
==> hub: Running cleanup tasks for 'shell' provisioner...
==> hub: Running cleanup tasks for 'shell' provisioner...
==> hub: Running cleanup tasks for 'shell' provisioner...
Uninstall Vagrant Environment
When you have completed your evaluation are ready to use CFEngine on production servers, remove the VMs that you created above by following these simple instructions:
To remove the VMs entirely, type: vagrant destroy
If you are completely done and do not anticipate using them anymore, you can
also remove the base box centos-6.5-x86_64-cfengine_enterprise-vagrant-201501201245
that was
downloaded. You can see it by typing vagrant box list
. To delete the basebox
run vagrant box remove centos-6.5-x86_64-cfengine_enterprise-vagrant-201501201245 virtualbox
.
Note: Running vagrant up
from the vagrant project directory again will
re-download this basebox.
Vagrant and VirtualBox are useful general purpose programs, so you might want to keep them around. If not, follow the standard procedures for your OS to remove these applications.
Next Steps
See also
Installing Enterprise for Production
These instructions describe how to install the latest version of CFEngine Enterprise in a production environment using pre-compiled rpm and deb packages for Ubuntu, Debian, Redhat, CentOS, and SUSE.
General Requirements
CFEngine recommends the following:
Host Memory
During normal operation the CFEngine processes consume about 30 MB of resident memory (RSS) on hosts with the agent only (not acting as Policy Server).
However there might be spikes due to e.g. commands executed from the CFEngine policy so it is generally recommended to have at least 256 MB available memory in order to run the CFEngine agent software.
Host disk
So that the agent is not affected by full disks it is recommended that
/var/cfengine
be on it's own partition.
On Unix-like systems, under normal operation CFEngine can consume
100 MB of the partition mounted where CFEngine is installed (usually
/var/cfengine
). On Windows systems, CFEngine can consume up to 1GB
(usually C:\Program Files\Cfengine
). The higher disk usage on
Windows is due to lack of support for sparse files on this platform,
which is utilized by a dependency of CFEngine (lmdb) when available.
The agent builds local differential reports for promise outcomes. The
longer the period between collections from the enterprise hub the more
resources are required to calculate these differentials. You can
control the maximum disk space used by diff reports (contexts,
variables, software installed, software patches, lastseen hosts and
promise executions) by adjusting def.max_client_history_size
.
Network
Verify that the machine’s network connection is working and that port 5308 (used by CFEngine) is open for both incoming and outgoing connections.
If a firewall is active on your operating system, adapt it to it to allow for communication on port 5308 or disable it.
CFEngine bundles all critical dependencies into the package; therefore, additional software is not required.
Requirements for VIOS
CFEngine Enterprise has Virtual I/O Server (VIOS) Recognized status from IBM. This means that CFEngine Enterprise has been technically verified by IBM to be installed in and manage VIOS environments.
During testing, CFEngine Enterprise was seen to use up to 2% of the
VIOS CPU during cf-agent
runs with the default CFEngine policy. The
resource utilization may vary depending on the policy CFEngine is
running. The VIOS should be configured with Shared Processors in
Uncapped mode.
Policy Server Requirements
Please note that the resource requirements below are meant as minimum guidelines and have been obtained with syntethic testing, and it is always better to leave some headroom if intermittent bottlenecks should occur. The key drivers for the vertical scalability of the Policy Servers are 1) the number of agents bootstrapped and 2) the size and complexity of the CFEngine policy.
cfapache and cfpostgres users
The CFEngine Server requires two users: cfapache and cfpostgres. If these users do not exist during installation of the server package, they will be created, so if there are constraints on user creation, please ensure that these users exists prior to installation.
These users are not required nor created by the agent package.
Dedicated OS
The CFEngine Server is only supported when installed on a dedicated, vanilla OS (i.e. it only has repositories and packages officially supported by the OS vendor). This is because the CFEngine Server uses services, e.g. apache, that are configured for CFEngine and may conflict with other custom application configurations.
One option, especially for smaller installations, is to run the CFEngine Server in a VM. But please consider the performance requirements when doing this.
CPU
A modern 64-bit processor with 12 or more cores for handling up to 5000 bootstrapped agents. This number is also linear with respect to the number of bootstrapped agents (so 6 cores would suffice for 2500 agents).
Memory
Minimum 2GB memory, but not lower than 8MB per bootstrapped agent. This means that, for a server with 5000 hosts, you should have at least 40GB of memory.
Disk sizing and partitioning
So that the agent is not affected by full disks it is recommended that
/var/cfengine
be on it's own partition.
It is recommended that $(sys.workdir)/state/pg
is mounted on a
separate disk. This will give PostgreSQL, which can be very disk I/O
intensive, dedicated resources.
Plan for approximately 100MB of disk space per bootstrapped agent. This means that, for a server with 5000 hosts, you should have at least 500 GB available on the database partition.
xfs is strongly recommended as the file system type for the file
system mounted on $(sys.workdir)/state/pg
. ext4 can be used as an
alternative, but ext3 should be avoided.
Disk speed
For 5000 bootstrapped agents, the disk that serves PostgreSQL
($(sys.workdir)/state/pg
) should be able to perform at least 1000
IOPS (in 16KiB block size) and 10 MB/s. The disk mounted on
$(sys.workdir)
should be able to perform at least 500 IOPS and 0.5
MB/s. SSD is recommended for the disk that serves PostgreSQL
($(sys.workdir)/state/pg
).
If you do not have separate partitions for $(sys.workdir)
and
$(sys.workdir)/state/pg
, the speed required by the disk serving
$(sys.workdir)
adds up (for 5000 bootstrapped agents it would be
1500 IOPS and 10.5 MB/s).
Note Your storage IOPS specification may be given in 4KiB block size, in which case you would need to divide it by 4 to get the corresponding 16KiB theoretical maximum.
Network
For serving policy and collecting reports for up to 5000 bootstrapped agents, plan for at least 30 MB/s (240 MBit) speed on the interface that connects the Policy Server with the agents.
Open file descriptors
The policy server should ideally be able to accept connections from
all clients; i.e. to allow at least as many incoming connections as
there are clients. The system limit for this is controlled by ulimit
-n
; so the parent process from which you bootstrap should, for a
server with 5000 hosts, run ulimit -n 5000
first. You should also
add such a ulimit -n
command to the script that implements service
cfengine3 start
(and restart
) and to any policy that starts
cf-serverd
or cf-hub
. For very large numbers of clients, it may
be advantageous to build a custom kernel to allow setting ulimit -n
high enough. You should also amend the value of maxconnections
set
in cf_serverd.cf
under $(sys.workdir)/masterfiles/controls/
to the
number of clients, likewise.
Download Packages
Install Packages
CFEngine Enterprise is provided in two packages; one is for the Policy Server (hub) and the other is for each Host (client).
Log in as root and then follow these steps to install CFEngine Enterprise:
On the designated Policy Server, install the
cfengine-nova-hub
package:[RedHat/CentOS/SUSE] # rpm -i <hub package>.rpm [Debian/Ubuntu] # dpkg -i <hub package>.deb
On each Host, install the
cfengine-nova
package:[RedHat/CentOS/SUSE] # rpm -i <agent package>.rpm [Debian/Ubuntu] # dpkg -i <agent package>.deb [Solaris] # pkgadd -d <agent package>.pkg all [AIX] # installp -a -d <agent package>.bff cfengine.cfengine-nova [HP-UX] # swinstall -s <full path to agent package>.depot cfengine-nova
Note: Install actions logged to /var/logs/cfengine-install.log
.
Bootstrap
Run the bootstrap command, first on the policy server and then on each host:
# /var/cfengine/bin/cf-agent --bootstrap <IP address of the Policy Server>
After bootstrapping the hub run the policy to complete the hub configuration.
# /var/cfengine/bin/cf-agent -Kf update.cf; /var/cfengine/bin/cf-agent -K
Licensed installations
If you are evaluating CFEngine Enterprise or otherwise using it in an environment with less than 25 agents connecting to a Policy Server, you do not need a license and there is no expiry.
If you are a customer, please send the Policy Server's public key to CFEngine support to obtain a license.
It's best to pack the public key into an archive so that it does not get corrupt in transit.
# tar --create --gzip --directory /var/cfengine --file $(hostname)-ppkeys.tar.gz ppkeys/localhost.pub
CFEngine will send you a license.dat
file. Install the obtained
license with cf-key
.
# cf-key --install-license ./license.dat
Next Steps
When bootstrapping is complete, CFEngine is up and running on your system.
The Mission Portal is immediately accessible. Connect to the Policy Server
through your web browser at http://<IP address of your Policy Server>
.
To be able to use the Mission Portal's Design Center front-end, continue with integrating Mission Portal with git.
Learn more about CFEngine by using the following resources:
Tutorial: Tutorial for Running Examples
Tutorial: Configure and deploy a policy using sketches in the Design Center.
Installing Community
These instructions describe how to download and install the latest version of CFEngine Community using pre-compiled rpm and deb packages for Ubuntu, Debian, Redhat, CentOS, and SUSE.
It also provides instructions for the following:
- Install CFEngine on a Policy Server (hub) and on a Host (client). A Policy Server (hub) is a CFEngine instance that contains promises (business policy) that get deployed to Hosts. Hosts are clients that retrieve and execute promises.
- Bootstrap the Policy Server to itself and then bootstrap the Host(s) to the Policy Server. Bootstrapping establishes a trust relationship between the Policy Server and all Hosts. Thus, business policy that you create in the Policy Server can be deployed to Hosts throughout your company. Bootstrapping completes the installation process.
Tutorials, recommended reading. and production environment recommendations appear at the end of this page.
Quick Setup Installation Script
Please Note: Internet access is required from the host if you wish to use the quick install script.
Use the following script to install CFEngine on your 32- or 64-bit machine.
$ wget -O- http://cfengine.package-repos.s3.amazonaws.com/quickinstall/quick-install-cfengine-community.sh | sudo bash
- Run this script on your designated Policy Server machine and on your designated Host machine(s).
- Bootstrap the Policy Server to itself and then bootstrap your Host(s) to the Policy Server by running the following command:
$ sudo /var/cfengine/bin/cf-agent --bootstrap <IP address of policy server>
If you require more details on bootstrapping, review Step 3 below. Bootstrapping completes the installation. - Go to the Tutorials section to learn how to use CFEngine.
1. Download Packages
Select the package to download that matches your operating system. This stores the cfengine-community_3.6.1-1_* file onto your machine.
Redhat/CentOS/SUSE 64-bit:
$ wget http://cfengine.package-repos.s3.amazonaws.com/community_binaries/cfengine-community-3.6.1-1.x86_64.rpm
Redhat/CentOS/SUSE 32-bit:
$ wget http://cfengine.package-repos.s3.amazonaws.com/community_binaries/cfengine-community-3.6.1-1.i386.rpm
Ubuntu/Debian 64-bit:
$ wget http://cfengine.package-repos.s3.amazonaws.com/community_binaries/cfengine-community_3.6.1-1_amd64.deb
Ubuntu/Debian 32-bit:
$ wget http://cfengine.package-repos.s3.amazonaws.com/community_binaries/cfengine-community_3.6.1-1_i386.deb
2. Install CFEngine on a Policy Server
Install the package on a machine designated as a Policy Server. A Policy Server is a CFEngine instance that contains promises (business policy) that get deployed to Hosts. Hosts are instances (clients) that retrieve and execute promises.
Choose the right command for your operating system:
Redhat/CentOS/SUSE 64-bit:
$ sudo rpm -i cfengine-community-3.6.1-1.x86_64.rpm
Redhat/CentOS/SUSE 32-bit:
$ sudo rpm -i cfengine-community_3.6.1-1.i386.rpm
Ubuntu/Debian 64-bit:
$ sudo dpkg -i cfengine-community_3.6.1-1_amd64.deb
Ubuntu/Debian 32-bit:
$ sudo dpkg -i cfengine-community_3.6.1-1_i386.deb
Note: You might get a message like this: "Policy is not found in /var/cfengine/inputs, not starting CFEngine." Do not worry; this is taken care of during the bootstrapping process.
3. Bootstrap the Policy Server
The Policy Server must be bootstrapped to itself. Find the IP address of your Policy Server (type $ ifconfig).
Run the bootstrap command:
$ sudo /var/cfengine/bin/cf-agent --bootstrap <IP address of policy server>
Example: $ sudo /var/cfengine/bin/cf-agent --bootstrap 192.168.1.12
Upon successful completion, a confirmation message appears: "Bootstrap to '192.168.1.12' completed successfully!"
Type the following to check which version of CFEngine your are running:
$ /var/cfengine/bin/cf-promises --version
The Policy Server is installed.
4. Install CFEngine on a Host
As stated earlier, Hosts are instances that retrieve and execute promises from the Policy Server. Install a package on your Host. Use the same package you installed on the Policy Server in Step 2. Note that you must have access to at least one more VM or server and it must be on the same network as the Policy Server that you just installed.
5. Bootstrap the Host to the Policy Server
The Host(s) must be bootstrapped to the Policy Server in order to establish a connection between the Host and the Policy Server. Run the same commands that you ran in Step 3.
$ sudo /var/cfengine/bin/cf-agent --bootstrap <IP address of policy server>
Example: $ sudo /var/cfengine/bin/cf-agent --bootstrap 192.168.1.12
The CFEngine installation process is complete.
Tutorials
Create a policy ("hello world")
Step 1. Create a file called hello_world.cf and add the following content:
body common control
{
bundlesequence => { "test" };
}
bundle agent test
{
reports: # This is a promise type.
cfengine_3:: # This means the promise will only
# be kept on a CFEngine_3 system.
"Hello World"; # This is a simple promise; it generates a report
# that says "Hello world".
}
Step 2. Run the policy:
$ sudo /var/cfengine/bin/cf-agent hello_world.cf
The policy displays the following output:
R: Hello world
Find more policy examples in the Examples section.
Create a distributed policy
Create a policy that ensures (promises) that a file called example.txt will always exist on the Host.
Step 1. On the Policy Server, create a file called mypolicy.cf and add it to the /var/cfengine/masterfiles directory:
$ sudo <editor> /var/cfengine/masterfiles/mypolicy.cf
Step 2. Add the following lines to the file:
bundle agent example
{
files:
cfengine_3:: # This is a class context (the promise will only
# be kept on a CFEngine_3 system)
"/home/vagrant/example.txt" # Path and name of the file we wish to ensure exists
create => "true"; # Make sure the file exists; create if it does not
}
Step 3. Update /var/cfengine/masterfiles/promises.cf to include this new policy. To do so, modify the promises.cf file to ensure that (1 the mypolicy.cf file is being included in the next policy distribution and that (2 example is in the bundlesequence.
$ sudo <editor> /var/cfengine/masterfiles/promises.cf
...
bundlesequence => {
# Common bundles first for best practice
"def",
"example",
...
inputs => {
# Global common bundles
"def.cf",
"mypolicy.cf",
...
The process is complete. The next time CFEngine runs on the Host (which by default is every 5 minutes), it will pull down the latest policy update and ensure that the example.txt file exists (this is the desired state). In fact, any Host that has installed CFEngine will contain the example.txt file (because we defined the cfengine_3:: class above).
Try these advanced tutorials:
- Tutorial for Running Examples
- Distribute files from a central location. This advanced, command-line tutorial shows you how to distribute policy files from the Policy Server to all pertinent Hosts.
Recommended Reading
- CFEngine language concepts
Production Environment
If you plan to use Community in a production environment, complete the following general requirements:
Host(s) Memory
256 MB available memory in order to run the CFEngine agent software (cf-agent).
Disk Storage
A full installation of CFEngine requires 25 MB. Additional disk usage depends on your specific policies, especially those that concern reporting.
Network
Verify that the machine’s network connection is working and that port 5308 (used by CFEngine) is open for both incoming and outgoing connections.
If iptables are active on your operating system, stop this service or adapt it to allow for communication on the above ports. If applicable, type the following two commands: /
etc/init.d/iptables stop
andchkconfig iptables off
Rate your experience
Everyone is a first-time user a some point. We want to make the CFEngine Enterprise installation process easy for all of our new users. Before you forget your first-time experience, we would love for you to let us know how we can improve on this process.
Secure Bootstrap
This guide presumes that you already have CFEngine properly installed and running on the policy hub, the machine that distributes the policy to all the clients. It also presumes that CFEngine is installed, but not yet configured, on a number of clients.
We present a step-by-step procedure to securely bootstrapping a number of servers (referred to as clients) to the policy hub, over a possibly unsafe network.
Introduction
CFEngine's trust model is based on the secure exchange of keys. This exchange of keys between client and hub, can either happen manually or automatically. Usually this step is automated as a dead-simple "bootstrap" procedure:
cf-agent --bootstrap $HUB_IP
It is presumed that during this first key exchange, the network is trusted, and no attacker will hijack the connection. After "bootstrapping" is complete, the node can be deployed in the open internet, and all connections are considered secure.
However there are cases where initial CFEngine deployment is happening over an insecure network, for example the Internet. In such cases we already have a secure channel to the clients, usually ssh, and we use this channel to manually establish trust from the hub to the clients and vice-versa.
Manual Trust Establishment
This procedure concerns CFEngine version 3.6 or earlier. While this fully manual procedure should always work, from version 3.7 onwards there is a simpler semi-automatic procedure for establishing trust.
On the policy hub
We must change the policy we're distributing to fully locked-down
settings. So after we have set-up our hub (using the standard procedure
of cf-agent --bootstrap $HUB_IP
) we take care of the following:
Cf-serverd must never accept a connection from a client presenting an untrusted key. So in
body server control
we must empty thetrustkeysfrom
attribute:trustkeysfrom => {};
Since we will be manually bootstrapping the clients, we need to distribute a proper
failsafe.cf
policy. (**NOTE:**failsafe.cf
is a file auto-generated in theinputs
directory when we runcf-agent --bootstrap
).In order to do that, we copy hub's
failsafe.cf
tomasterfiles
directory:cp /var/cfengine/inputs/failsafe.cf /var/cfengine/masterfiles/
We'll edit that copy in
masterfiles
in the next step.All
copy_from
files promises must never connect to an untrusted server, which means that the following line should not be found anywhere:trustkey => "true"
All occurrences of
trustkey
inmasterfiles
directory should be changed to "false", or be removed (since it defaults to false anyway). It is certain thatfailsafe.cf
that we copied in the previous step will contain such occurrences that should be changed. (Those occurrences are the reason that automatic bootstrapping requires a trusted network).The previous changes in
masterfiles
need to be properly propagated to theinputs
directory. The automated way to do that is to run the update.cf policy:cf-agent -f update.cf
Get the hub's key fingerprint, we'll need it later:
HUB_KEY=`cf-key -p /var/cfengine/ppkeys/localhost.pub`
On each client we deploy
We should not follow the automatic method, i.e. the
cf-agent --bootstrap
command. We will perform a
manual bootstrap.
Generate a private/public key pair by running
cf-key
.Get the client's key fingerprint, we'll need it later:
CLIENT_KEY=`cf-key -p /var/cfengine/ppkeys/localhost.pub`
Write the policy hub's IP address to
policy_server.dat
:echo $HUB_IP > /var/cfengine/policy_server.dat
Manually copy the modified
failsafe.cf
from hub'smasterfiles
directory into client'sinputs
directory. You should do it in a secure manner, for example usingscp
with properly trusted fingerprint of the remote host.NOTE: At this step, you can try running the failsafe policy. Because trust between the two hosts has not been established, you will get a failure, which is totally expected and means everything is correct and secure:
$# cf-agent -f failsafe.cf error: TRUST FAILED, server presented untrusted key: MD5=cc27570b8b831192d9f20b54d07dd80b error: No suitable server responded to hail error: TRUST FAILED, server presented untrusted key: MD5=cc27570b8b831192d9f20b54d07dd80b error: No suitable server responded to hail [ ... ]
Put the hub's key into the client's trusted keys:
scp $HUB_IP:/var/cfengine/ppkeys/localhost.pub /var/cfengine/ppkeys/root-${HUB_KEY}.pub
Final steps
Put the client's key into the hub's trusted keys. So on the hub, run:
scp $CLIENT_IP:/var/cfengine/ppkeys/localhost.pub /var/cfengine/ppkeys/root-${CLIENT_KEY}.pub
If you now run the failsafe policy on each and every client, it should succeed:
cf-agent -f failsafe.cf
Congratulations, you have performed a fully manual bootstrap procedure for your clients!
Upgrading
This guide documents our recommendation on how to upgrade an existing installation of CFEngine Community 3.7 and 3.8 and CFEngine Enterprise 3.7 and 3.8 to CFEngine 3.9.
We recommend upgrading the Masterfiles Policy Framework first so that you can verify that the new policy will work with your old clients across the infrastructure. Once the latest policy has been deployed successfully we recommend upgrading the Policy Server and finally the remote agents.
Upgrading to 3.9 from versions older than 3.7 is more complicated as some functionality introduced in 3.9 is not compatible with versions 3.6 and earlier. For more information about upgrading from 3.6 see Upgrade from 3.6.
Upgrade masterfiles and Policy Server (3.9.X to 3.9.X+1)
If you are doing a minor-minor 3.9 upgrade (e.g. from 3.9.0 to 3.9.1), the upgrade is easier.
We would however still recommend to perform a masterfiles upgrade (ideally in a test environment first) to get all the enhancements and fixes.
The Masterfiles Policy Framework is available in the hub package and separately on the download page
Normally most files can be replaced with new ones, the only ones that are likely changed by you are def.cf and promises.cf. For these two files, we would need to do a diff between your version and the new version and integrate the diff instead of replacing the whole file.
For more detailed information on how to upgrade masterfiles please see Prepare masterfiles for upgrade section below.
When the new masterfiles have been created and cf-promises promises.cf and cf-promises update.cf succeeds, you are ready to upgrade the Policy Server. That entails to
- replace /var/cfengine/masterfiles with your new integrated masterfiles
- stop the CFEngine services
- upgrade the hub package
- replace (or merge with your changes) /var/cfengine/state/pg/data/postgresql.conf with /var/cfengine/share/postgresql/postgresql.conf.cfengine to update your database configuration.
- restart the CFEngine services
Check the version with /var/cfengine/bin/cf-promises -V, and if you are running Enterprise, the Mission Portal About page.
If your clients get promise failures (not kept) similar to "Can't stat file '/var/cfengine/master_software_updates/cf-upgrade/linux.x86_64/cf-upgrade' on '
If everything looks good, you are ready to upgrade the clients, please skip to Prepare Client upgrade (all versions) followed by Complete Client upgrade (all versions) below.
Upgrade Policy Server (3.7 to 3.9.X)
Make a backup of the Policy Server, a full backup of
/var/cfengine
(or yourWORKDIR
equivalent) is recommended.cp -r /var/cfengine/ppkeys/ /root/3.7/ppkeys
tar cvzf /root/3.7/cfengine.tar.gz /var/cfengine
Save the list of hosts currently connecting to the Policy Server.
cf-key -s > /root/3.7/hosts
Prepare masterfiles following instructions in the Prepare masterfiles for upgrade section below.
Copy the merged masterfiles from the preparation you did above.
rm -rf /var/cfengine/masterfiles/*
cp /root/3.9/masterfiles/* /var/cfengine/masterfiles/
On your existing Policy Server, stop the CFEngine services.
service cfengine3 stop
- Verify that the output of
ps -e | grep cf
is empty.
Note: Clients will continue to execute the policy that they have.
Install the new CFEngine Policy Server package (you may need to adjust the package name based on CFEngine edition, version and distribution).
rpm -U cfengine-nova-hub-3.9.1-1.x86_64.rpm
# Red Hat based distributiondpkg --install cfengine-nova-hub_3.9.1-1_amd64.deb
# Debian based distribution
Bootstrap the Policy Server to itself (this step might not be needed if Policy Server is reporting correctly).
/var/cfengine/bin/cf-agent -B <POLICY-SERVER-IP>
Any error messages regarding processes can be corrected by running
cf-agent -f update.cf -IK
Take the Policy Server online.
- Verify with
cf-key -s
that connections from all clients have been established within 5-10 minutes. - Select some clients to confirm that they have received the new policy and are running it without error.
- Verify with
Prepare masterfiles for upgrade
- Merge your masterfiles with the CFEngine 3.9 policy framework on an infrastructure separate from your existing CFEngine installation.
- Identify existing modifications to the masterfiles directory. If patches from version control are unavailable or require verification, a copy of /var/cfengine/masterfiles from a clean installation of your previous version can help identify changes which will need to be applied to a new 3.9 install.
- The 3.9 masterfiles can be found in a clean installation of CFEngine (hub package on Enterprise), under /var/cfengine/masterfiles. Apply any customizations against a copy of the 3.9 masterfiles in a well-known location, e.g.
/root/3.9/masterfiles
. - Use
cf-promises
to verify that the policy runs with 3.9, by runningcf-promises /root/3.9/masterfiles/promises.cf
andcf-promises /root/3.9/masterfiles/update.cf
. - Use
cf-promises
to verify that the policy runs with you previous version of CFEngine (e.g. 3.7), by running the same commands as above on a node with that CFEngine version. - The merged masterfiles should now be based on the 3.9 framework, include your policies and work on both the version you are upgrading from and with 3.9.
Upgrade from 3.6
As 3.6 policy is not compatible with 3.9 some additional steps must be performed to fulfill the upgrade procedure.
- Beginning with version 3.9 the Masterfiles Policy Framework defaults to the new packages promise implementation for inventory of packages installed and packages updates. See package_inventory in body common control for details on modifying the default sources for package inventory.
There are
body common control
package_inventory
andpackage_module
attributes which are not recognized by versions 3.6 and earlier. While upgrading from 3.6.x make sure that both are commented, so that existing 3.6.x hosts can communicate with the 3.9 hub and can validate policy. After migrating all the clients to the newest CFEngine version, make sure that both previously commented parameters are uncommented so that the new package promise can be used as the default one.
Prepare Client upgrade (all versions)
- Make client packages available on the Policy Server in
/var/cfengine/master_software_updates
, under the appropriate directories for the OS distributions you use. Turn on the auto-upgrade policy by setting the
trigger_upgrade
class. Setmasterfiles/controls/VER/update_def.cf
or theaugments_file
also known asdef.json
for a small set of clients. For example in the appropriateupdate_def.cf
file(s) change!any
to an appropriate class like an IP networkipv4_10_10_1|ipv4_10_10_2
or indef.json
Verify that the selected hosts are upgrading successfully.
As an Enterprise user, confirm that the hosts start appearing in Mission Portal after 5-10 minutes. Otherwise, log manually into a set of hosts to confirm the successful upgrade.
Complete Client upgrade (all versions)
- Widen the group of hosts on which the
trigger_upgrade
class is set. - Continue to verify from
cf-key -s
or in the Enterprise Mission Portal that hosts are upgraded correctly and start reporting in. - Verify that the list of hosts you captured before the upgrade, e.g. in
/root/3.7/hosts
correspond to what you see is now reporting in.
Optional steps
The steps listed here are not necessary unless you have special needs.
Migrating Mission Portal database
This step is not needed unless you are upgrading from CFEngine 3.8 or lower, to CFEngine 3.9 or higher, and you are unable to use the automatic migration.
Normally the package upgrade will do the migration for you, but if you have a
very big database, or for other reasons don't have enough space to hold database
backup files in the /var/cfengine/state/pg
directory, you may perform these
steps manually.
- Before installing the new version of CFEngine, dump the current content of
the database to a file using
pg_dump
. You need to do this for each of the three databases, like this:
su cfpostgres -c "/var/cfengine/bin/pg_dump cfdb > cfdb-backup.sql"
su cfpostgres -c "/var/cfengine/bin/pg_dump cfsettings > cfsettings-backup.sql"
su cfpostgres -c "/var/cfengine/bin/pg_dump cfmp > cfmp-backup.sql"
Shut down CFEngine and then delete or move the
/var/cfengine/state/pg/data
directory in order to prevent the automatic migration by the package scripts.Install the new CFEngine package.
Restore the database dump into the new PostgreSQL database by running:
su cfpostgres -c "/var/cfengine/bin/psql cfdb < cfdb-backup.sql"
su cfpostgres -c "/var/cfengine/bin/psql cfsettings < cfsettings-backup.sql"
su cfpostgres -c "/var/cfengine/bin/psql cfmp < cfmp-backup.sql"
Version Control
CFEngine is policy is stored in /var/cfengine/masterfiles
on the policy
server. It is common that this directory is backed by a version control system
(VCS), such as git or subversion. In this document we will focus on git, but
CFEngine is VCS agnostic.
Please note that the following applies to CFEngine Community or Enterprise, but in Enterprise there are built-in facilities that can make the following unnecessary. Please see Version Control and Configuration Policy for details.
Repository synchronization
When /var/cfengine/masterfiles
is backed by VCS, it may be useful to have an
agent policy that periodically checks the VCS server for the latest version
fetches any updates. Again, note that CFEngine Enterprise has this built-in.
After installing CFEngine on the policy server and before bootstrapping the agent to itself, we want to create a git repository out of our masterfiles. Assuming that we have a functioning git installation, and a previously configured github account:
$ cd /var/cfengine/masterfiles
$ git init
$ git remote add origin git@github.com:Username/cf-engine-repo.git
$ git add *
$ git commit -m "Initial post installation commit of masterfiles directory"
Now we are good to go and push the content of our /masterfiles directory. We push it with the -ff option to overwrite any possible content of our (supposedly empty) github repository
$ git push -ff origin master
The next step is to create a policy that will periodically update the content of our /masterfiles directory on the hub, pulling the changes we did on the git repo.
The following policy uses git pull
with the --ff-only flag to avoid
potentially bad merges. This assumes that no development takes place in
/var/cfengine/masterfiles
itself.
Note that we specify policy_server:: here, as we want only the hub to pull code from github, while nodes will be updated through CFEngine.
bundle agent vcs_update
{
commands:
policy_server::
"/usr/bin/git"
args => "pull --ff-only origin master",
contain => masterfiles_contain;
}
body contain masterfiles_contain
{
chdir => "/var/cfengine/masterfiles";
}
This policy will then regularly and periodically pulling the masterfiles (and therefore your new policies) from your git repo to the hub, so that your nodes will get configured accordingly. But, there is a catch: two files in the /masterfiles directory should be excluded. Those are the cf_promises_release_id, and the timestamp of the release (cf_promises_validated). So we add them to a .gitignore file and push it to the github repo.
$vim .gitignore
cf_promises_release_id
cf_promises_validated
And then we push it upstream. We then add the vcs_update policy to promises.cf and we are done.
Commit hooks
Commit hooks are scripts that are run when a repository is updated. We can use
a hook to notify a policy developer if an update causes a syntax error. While
the agent on the policy server should not copy from
/var/cfengine/masterfiles
to /var/cfengine/inputs
if the new policy does
not pass validation, it can nevertheless be helpful to employ VCS commit
hooks. A hook needs to be installed on the VCS server. Git and subversion
store their hooks on the server, under directories .git/hooks
and hooks
,
respectively.
Example git update hook
We can use a git update hook to prevent a change from being made unless it
passes syntax checking. The idea is to check out the revision in a temporary
directory and run cf-promises
on it. Here is an example hook.
#!/bin/sh
# --- Command line
REF_NAME="$1"
OLD_REV="$2"
NEW_REV="$3"
GIT=/usr/bin/git
TAR=/bin/tar
CF_PROMISES=/home/a10021/Source/core/cf-promises/cf-promises
TMP_CHECKOUT_DIR=/tmp/cfengine-post-commit-syntax-check/
MAIN_POLICY_FILE=promises.cf
echo "Creating temporary checkout directory at ${TMP_CHECKOUT_DIR}"
mkdir -p ${TMP_CHECKOUT_DIR}
echo "Clearing potential data in temporary checkout directory"
rm -rf ${TMP_CHECKOUT_DIR}/*
rm -rf ${TMP_CHECKOUT_DIR}/.svn
echo "Checking out revision ${REV} from ${REPOS} to file://${TMP_CHECKOUT_DIR}"
${GIT} archive ${NEW_REV} | tar -x -C ${TMP_CHECKOUT_DIR}
if [ $? -ne 0 ]; then
echo "Error checking out repository to temporary folder during post-commit syntax checking!" >&2
return 1
fi
echo "Running cf-promises -cf on ${TMP_CHECKOUT_DIR}/${MAIN_POLICY_FILE}"
${CF_PROMISES} -cf ${TMP_CHECKOUT_DIR}/${MAIN_POLICY_FILE}
if [ $? -ne 0 ]; then
echo "There were policy errors in pushed revision ${REV}" >&2
return 1
else
echo "Policy check completed successfully!"
return 0
fi
Example subversion post-commit hook
For subversion, the principle is essentially the same. Note that for a post-commit hook the check is run after update, so the repository may be left with a syntax error, but the committer is notified.
#!/bin/sh
REPOS="$1"
REV="$2"
SVN=/usr/bin/svn
CF_PROMISES=/home/a10021/Source/core/cf-promises/cf-promises
TMP_CHECKOUT_DIR=/tmp/cfengine-post-commit-syntax-check/
MAIN_POLICY_FILE=trunk/promises.cf
echo "Creating temporary checkout directory at ${TMP_CHECKOUT_DIR}"
mkdir -p ${TMP_CHECKOUT_DIR}
echo "Clearing potential data in temporary checkout directory"
rm -rf ${TMP_CHECKOUT_DIR}/*
rm -rf ${TMP_CHECKOUT_DIR}/.svn
echo "Checking out revision ${REV} from ${REPOS} to file://${TMP_CHECKOUT_DIR}"
${SVN} co -r ${REV} file://${REPOS} ${TMP_CHECKOUT_DIR}
if [ $? -ne 0 ]; then
echo "Error checking out repository to temporary folder during post-commit syntax checking!" >&2
return 1
fi
echo "Running cf-promises -cf on ${TMP_CHECKOUT_DIR}/${MAIN_POLICY_FILE}"
${CF_PROMISES} -cf ${TMP_CHECKOUT_DIR}/${MAIN_POLICY_FILE}
if [ $? -ne 0 ]; then
echo "There were policy errors in committed revision ${REV}" >&2
return 1
else
echo "Policy check completed successfully!"
return 0
fi
Writing and Serving Policy
- About Policies and Promises
- Policy Workflow
- How Promises Work
- Best Practices
- Layers of Abstraction in Policy
- Promises Available in CFEngine
- Authoring Policy Tools & Workflow
About Policies and Promises
Central to CFEngine's effectiveness in system administration is the concept of a "promise," which defines the intent and expectation of how some part of an overall system should behave.
CFEngine emphasizes the promises a client makes to the overall CFEngine network. Combining promises with patterns to describe where and when promises should apply is what CFEngine is all about.
This document describes in brief what a promise is and what a promise does. There are other resources for finding out additional details about "promises" in the See Also section at the end of this document.
What Are Promises
A promise is the documentation or definition of an intention to act or behave in some manner. They are the rules which CFEngine clients are responsible for implementing.
The Value of a Promise
When you make a promise it is an effort to improve trust, which is an economic time-saver. If you have trust then there is less need to verify, which in turn saves time and money.
When individual components are empowered with clear guidance, independent decision making power, and the trust that they will fulfil their duties, then systems that are complex and scalable, yet still manageable, become possible.
Anatomy of a Promise
bundle agent hello_world
{
reports:
any::
"Hello World!"
comment => "This is a simple promise saying hello to the world.";
}
How Promises Work
Everything in CFEngine can be thought of as a promise to be kept by different resources in the system. In a system that delivers a web site using Apache, an important promise may be to make sure that the httpd
or apache
package is installed, running, and accessible on port 80.
Summary for Writing, Deploying and Using Promises
Writing, deploying, and using CFEngine promises
will generally follow these simple steps:
- Using a text editor, create a new file (e.g.
hello_world.cf
). - Create a bundle and promise in the file (see "Hello World" Policy Example).
- Save the file on the policy server somewhere under
/var/cfengine/masterfiles
(can be under a sub-directory). - Let CFEngine know about the
promise
on thepolicy server
, generally in the file/var/cfengine/masterfiles/promises.cf
, or a file elsewhere but referred to inpromises.cf
. * Optional: it is also possible to call a bundle manually, usingcf-agent
. - Verify the
policy file
was deployed and successfully run.
See Tutorial for Running Examples for a more detailed step by step tutorial.
Policy Workflow
CFEngine does not make absolute choices for you, like other tools. Almost everything about its behavior is a matter of policy and can be changed.
In order to keep operations as simple as possible, CFEngine maintains a
private working directory on each machine, referred to in documentation as WORKDIR
and in policy by
the variable $(sys.workdir) By default, this is located at
/var/cfengine
or C:\var\CFEngine
. It contains everything CFEngine needs to
run.
The figure below shows how decisions flow through the parts of a system.
It makes sense to have a single point of coordination. Decisions are therefore usually made in a single location (the Policy Definition Point). The history of decisions and changes can be tracked by a version control system of your choice (e.g. git, Subversion, CVS etc.).
Decisions are made by editing CFEngine's policy file
promises.cf
(or one of its included sub-files). This process is carried out off-line.Once decisions have been formalized and coded, this new policy is copied to a decision distribution point, $(sys.masterdir) which defaults to
/var/cfengine/masterfiles
on all policy distribution servers.Every client machine contacts the policy server and downloads these updates. The policy server can be replicated if the number of clients is very large, but we shall assume here that there is only one policy server.
Once a client machine has a copy of the policy, it extracts only those promise proposals that are relevant to it, and implements any changes without human assistance. This is how CFEngine manages change.
CFEngine tries to minimize dependencies by decoupling processes. By following this pull-based architecture, CFEngine will tolerate network outages and will recover from deployment errors easily. By placing the burden of responsibility for decision at the top, and for implementation at the bottom, we avoid needless fragility and keep two independent quality assurance processes apart.
Best Practices
Policy Style Guide This covers punctuation, whitespace, and other styles to remember when writing policy.
Bundles Best Practices Refer to this page as you decide when to make a bundle and when to use classes and/or variables in them.
Testing Policies This page describes how to locally test CFEngine and play with configuration files.
See Also
Layers of Abstraction in Policy
CFEngine offers a number of layers of abstraction. The most fundamental atom in CFEngine is the promise. Promises can be made about many system issues, and you described in what context promises are to be kept.
CFEngine is designed to handle high level simplicity (without sacrificing low level capability) by working with configuration patterns. After all, configuration is all about promising consistent patterns in the resources of the system. Lists, for instance, are a particularly common kind of pattern: for each of the following... make a similar promise. There are several ways to organize patterns, using containers, lists and associative arrays.
Menu level
At this high level, a user selects
from a set of pre-defined services
(or
bundles in CFEngine parlance). The selection is not made by every host, rather
one places hosts into roles that will keep certain promises.
bundle agent service_catalogue # menu
{
methods:
any:: # selected by everyone
"everyone" usebundle => time_management,
comment => "Ensure clocks are synchronized";
"everyone" usebundle => garbage_collection,
comment => "Clear junk and rotate logs";
mailservers:: # selected by hosts in class
"mail server" -> { "goal_3", "goal_1", "goal_2" }
usebundle => app_mail_postfix,
comment => "The mail delivery agent";
"mail server" -> goal_3,
usebundle => app_mail_imap,
comment => "The mail reading service";
"mail server" -> goal_3,
usebundle => app_mail_mailman,
comment => "The mailing list handler";
}
Bundle level
At this level, users can switch on and off predefined features, or re-use standard methods, e.g. for editing files:
body common control
{
bundlesequence => {
webserver("on"),
dns("on"),
security_set("on"),
ftp("off")
};
}
The set of bundles that can be selected from is extensible by the user.
Promise level
This is the most detailed level of configuration, and gives full convergent promise behavior to the user. At this promise level, you can specify every detail of promise-keeping behavior, and combine promises together, reusing bundles and methods from standard libraries, or creating your own.
bundle agent addpasswd
{
vars:
# want to set these values by the names of their array keys
"pwd[mark]" string => "mark:x:1000:100:Mark B:/home/mark:/bin/bash";
"pwd[fred]" string => "fred:x:1001:100:Right Said:/home/fred:/bin/bash";
"pwd[jane]" string => "jane:x:1002:100:Jane Doe:/home/jane:/bin/bash";
files:
"/etc/passwd" # Use standard library functions
create => "true",
comment => "Ensure listed users are present",
perms => mog("644","root","root"),
edit_line => append_users_starting("addpasswd.pwd");
}
Promises Available in CFEngine
meta - information about promise bundles
Meta-data promises have no internal function. They are intended to be used to represent arbitrary information about promise bundles. Formally, meta promises are implemented as variables, and the values map to a variable context called bundlename_meta. The values can be used as variables and will appear in CFEngine Enterprise variable reports.
See meta
.
vars - a variable, representing a value
Variables in CFEngine are defined as promises that an identifier of a certain type represents a particular value. Variables can be scalars or lists of types string, int, real or data.
The allowed characters in variable names are alphanumeric (both upper and lower case) and undercore. Associative arrays using the string type and square brackets [] to enclose an arbitrary key are being deprecated in favor of the data variable type.
See vars
.
defaults - a default value for bundle parameters
Defaults promises are related to variables. If a variable or parameter in a promise bundle is undefined, or its value is defined to be invalid, a default value can be promised instead.
CFEngine does not use Perl semantics: i.e. undefined variables do not map to the empty string, they remain as variables for possible future expansion. Some variables might be defined but still contain unresolved variables. To handle this you will need to match the $(abc)
form of the variables.
See defaults
.
classes - a class, representing a state of the system
Classes promises may be made in any bundle. Classes that are set in common bundles are global in scope, while classes in all other bundles are local.
Note: The term class and context are sometimes used interchangeably.
See classes
.
users - add or remove users
User promises are promises made about local users on a host. They express which users should be present on a system, and which attributes and group memberships the users should have.
Every user promise has at least one attribute, policy, which describes whether or not the user should be present on the system. Other attributes are optional; they allow you to specify UID, home directory, login shell, group membership, description, and password.
A bundle can be associated with a user promise, such as when a user is created in order to do housekeeping tasks in his/her home directory, like putting default configuration files in place, installing encryption keys, and storing a login picture.
History: Introduced in CFEngine 3.6.0
See users
.
files - configure a file
Files promises are an umbrella for attributes of files. Operations fall basically into three categories: create, delete and edit.
See files
.
packages - install a package
CFEngine supports a generic approach to integration with native operating support for packaging. Package promises allow CFEngine to make promises regarding the state of software packages conditionally, given the assumption that a native package manager will perform the actual manipulations. Since no agent can make unconditional promises about another, this is the best that can be achieved.
See packages
.
guest_environments
Guest environment promises describe enclosed computing environments that can host physical and virtual machines, Solaris zones, grids, clouds or other enclosures, including embedded systems. CFEngine will support the convergent maintenance of such inner environments in a fixed location, with interfaces to an external environment.
CFEngine currently seeks to add convergence properties to existing interfaces for automatic self-healing of guest environments. The current implementation integrates with libvirt, supporting host virtualization for Xen, KVM, VMWare, etc. Thus CFEngine, running on a virtual host, can maintain the state and deployment of virtual guest machines defined within the libvirt framework. Guest environment promises are not meant to manage what goes on within the virtual guests. For that purpose you should run CFEngine directly on the virtual machine, as if it were any other machine.
See guest_environments
.
methods - take on a whole bundle of other promises
Methods are compound promises that refer to whole bundles of promises. Methods may be parameterized. Methods promises are written in a form that is ready for future development. The promiser object is an abstract identifier that refers to a collection (or pattern) of lower level objects that are affected by the promise-bundle. Since the use of these identifiers is for the future, you can simply use any string here for the time being.
See methods
.
processes - start or terminate processes
Process promises refer to items in the system process table, i.e., a command in some state of execution (with a Process Control Block). Promiser objects are patterns that are unanchored, meaning that they match line fragments in the system process table.
See processes
.
services - start or stop services
A service is a set of zero or more processes. It can be zero if the service is not currently running. Services run in the background, and do not require user intervention.
Service promises may be viewed as an abstraction of process and commands promises. An important distinguisher is however that a single service may consist of multiple processes. Additionally, services are registered in the operating system in some way, and get a unique name. Unlike processes and commands promises, this makes it possible to use the same name both when it is running and not.
Some operating systems are bundled with a lot of unused services that are running as default. At the same time, faulty or inherently insecure services are often the cause of security issues. With CFEngine, one can create promises stating the services that should be stopped and disabled.
The operating system may start a service at boot time, or it can be started by CFEngine. Either way, CFEngine will ensure that the service maintains the correct state (started, stopped, or disabled). On some operating systems, CFEngine also allows services to be started on demand, when they are needed. This is implemented though the inetd or xinetd daemon on Unix. Windows does not support this.
CFEngine also allows for the concept of dependencies between services, and can automatically start or stop these, if desired. Parameters can be passed to services that are started by CFEngine.
See services
.
commands - execute a command
Commands and processes are separated cleanly. Restarting of processes must be coded as a separate command. This stricter type separation allows for more careful conflict analysis to be carried out.
See commands
.
storage - verify attached storage
Storage promises refer to disks and filesystem properties.
See storage
.
databases - configure a database
CFEngine can interact with commonly used database servers to keep promises about the structure and content of data within them.
There are two main cases of database management to address: small embedded databases and large centralized databases.
Databases are often centralized entities that have a single point of management. While large monolithic database can be more easily managed with other tools, CFEngine can still monitor changes and discrepancies. In addition, CFEngine can also manage smaller embedded databases that are distributed in nature, whether they are SQL, registry or future types.
For example, creating 100 new databases for test purposes is a task for CFEngine; but adding a new item to an important production database is not a recommended task for CFEngine.
See databases
.
access - grant or deny access to file objects
Access promises are conditional promises made by resources living on the server.
The promiser is the name of the resource affected and is interpreted to be a path, unless a different resource_type is specified. Access is then granted to hosts listed in admit_ips, admit_keys and admit_hostnames, or denied using the counterparts deny_ips, deny_keys and deny_hostnames.
You layer the access policy by denying all access and then allowing it only to selected clients, then denying to an even more restricted set.
See access
.
roles - allow certain users to activate certain classes
Roles promises are server-side decisions about which users are allowed to define soft-classes on the server's system during remote invocation of cf-agent. This implements a form of Role Based Access Control (RBAC) for pre-assigned class-promise bindings. The user names cited must be attached to trusted public keys in order to be accepted. The regular expression is anchored, meaning it must match the entire name.
See roles
.
measurements - measure or sample data from the system
This is an Enterprise-only feature.
By default,CFEngine's monitoring component cf-monitord records performance data about the system. These include process counts, service traffic, load average and CPU utilization and temperature when available.
CFEngine Enterprise extends this in two ways. First it adds a three year trend summary based any 'shift'-averages. Second, it adds customizable measurements promises to monitor or log very specific user data through a generic interface. The end-result is to either generate a periodic time series, like the above mentioned values, or to log the results to custom-defined reports.
Promises of type measurement are written just like all other promises within a bundle destined for the agent concerned, in this case monitor. However, it is not necessary to add them to the bundlesequence, because cf-monitord executes all bundles of type monitor.
See measurements
.
reports - report a message
Reports promises simply print messages. Outputting a message without qualification can be a dangerous operation. In a large installation it could unleash an avalanche of messaging.
See reports
.
Authoring Policy Tools & Workflow
There are several ways to approach authoring promises and ensuring they are copied into and then deployed properly from the masterfiles
directory:
- Create or modify files directly in the
masterfiles
directory. - Copy new or modified files into the
masterfiles
directory (e.g. local file copy usingcp
,scp
overssh
). - Utilize a version control system (e.g. Git) to push/pull changes or add new files to the
masterfiles
directory. - Utilize CFEngine Enterprise's integrated Git respository. The CFEngine Enterprise Guide contains more information.
Authoring on a Workstation and Pushing to the Hub Using Git + GitHub
General Summary
- The "masterfiles" directory contains the promises and other related files (this is true in all situations).
- Replace the out of the box setup with an initialized
git
repository and remote to a clone hosted on GitHub. - Add a promise to
masterfiles
that tells CFEngine to check thatgit
repository for changes, and if there are any to merge them intomasterfiles
. - When an author wants to create a new promise, or modify an existing one, they clone the same repository on GitHub so that they have a local copy on their own computer.
- The author will make their edits or additions in their local version of the
masterfiles
repository. - After the author is done making their changes commit them using
git commit
. - After the changes are committed they are then pushed back to the remote repository on GitHub.
- As described in step3, CFEngine will pull any new changes that were pushed to GitHub (sometime within a five minute time interval).
- Those changes will first exist in
masterfiles
, and then afterwards will be deployed to CFEngine hosts that are bootstrapped to the hub.
Create a Repository on GitHub for Masterfiles
There are two methods possible with GitHub: one is to use the web interface at GitHub.com; the second is to use the GitHub application.
Method One: Create Masterfiles Repository Using GitHub Web Interface
1a. In the GitHub web interface, click on the New repository
button.
1b. Or from the +
drop down menu on the top right hand side of the screen select New repository
.
2. Fill in a value in the Repository name
text entry (e.g. cfengine-masterfiles).
3. Select private
for the type of privacy desired (public
is also possible, but is not recommended in most situations).
4. Optionally, check the Initialize this repository with a README
box. (not required):""
Method Two: Create Masterfiles Repository Using the GitHub Application
- Open the GitHub app and click on the "+ Create" sign to create a new repository.
- Fill in a value in the
Repository name
text entry (e.g. cfengine-masterfiles). - Select
private
for the type of privacy desired (public
is also possible, but is not recommended in most situations). - Select one of your "Accounts" where you want the new repository to be created.
- Click on the "Create" button at the bottom of the screen. A new repository will be created in your local GitHub folder.
Initialize Git Repository in Masterfiles on the Hub
> cd /var/cfengine/masterfiles
> git init
> git commit -m "First commit"
> git remote add origin https://github.com/GitUserName/cfengine-masterfiles.git
> git push -u origin master
Using the above steps on a private repository will fail with a 403 error. There are different approaches to deal with this:
A) Generate a key pair and add it to GitHub
- As root, type
ssh-keygen -t rsa
. - Hit enter when prompted to
Enter file in which to save the key (/root/.ssh/id_rsa):
. - Hit enter again when prompted to
Enter passphrase (empty for no passphrase):
. - Type
ssh-agent bash
and then the enter key. - Type
ssh-add /root/.ssh/id_rsa
. - Type
exit
to leavessh-agent bash
. - To test, type
ssh -T git@github.com
. - Open the generated key file (e.g.
vi /root/.ssh/id_rsa.pub
). - Copy the contents of the file to the clipboard (e.g. Ctrl+Shift+C).
- In the GitHub web interface, click the user account settings button (the icon with the two tools in the top right hand corner).
- On the next screen, on the left hand side, click
SSH keys
. - Click
Add SSH key
on the next screen. - Provide a
Title
for the label (e.g. CFEngine). - Paste the key contents from the clipboard into the
Key
textarea. - Click
Add key
. - If prompted to do so, provide your GitHub password, and then click the
Confirm
button.
B) Or, change the remote url to https://GitUserName@password:github.com/GitUserName/cfengine-masterfiles.git
. This is not safe in a production environment and should only be used for basic testing purposes (if at all).
Create a Remote in Masterfiles on the Hub to Masterfiles on GitHub
- Change back to the
masterfiles
directory, if not already there:> cd /var/cfengine/masterfiles
- Create the remote using the following pattern:
> git remote add upstream ssh://git@github.com/GitUserName/cfengine-masterfiles.git
.
- Verify the remote was registered properly by typing
git remote -v
and pressing enter.- You will see the remote definition in a list alongside any other previously defined remote enteries.
Add a Promise that Pulls Changes to Masterfiles on the Hub from Masterfiles on GitHub
- Create a new file in
/var/cfengine/masterfiles with a unique filename
(e.g.vcs_update.cf
) - Add the following text to the
vcs_update.cf
file:
bundle agent vcs_update
{
commands:
"/usr/bin/git"
args => "pull --ff-only upstream master",
contain => masterfiles_contain;
}
body contain masterfiles_contain
{
chdir => "/var/cfengine/masterfiles";
}
- Save the file.
- Add bundle and file information to
/var/cfengine/masterfiles/promises.cf
. Example (where...
represents existing text in the file, omitted for clarity:
body common control
{
bundlesequence => {
...
vcs_update,
};
inputs => {
...
"vcs_update.cf",
};
- Save the file.
<!--- End include:
/home/jenkins/workspace/build-documentation-3.9/label/DOCUMENTATION_x86_64_linux_debian_6/documentation/guide/writing-and-serving-policy/authoring-policy-tools-and-workflow.markdown
-->
Policy Style Guide
Style is a very personal choice and the contents of this guide should only be considered suggestions. We invite you to contribute to the growth of this guide.
Style Summary
- one indent = 2 spaces
- avoid letting line length surpass 80 characters.
- vertically align opening and closing curly braces unless on same line
- promise type = 1 indent
- context class expression = 2 indents
- promiser = 3 indents
- promise attributes = (we suggest 3 or 4 indents)
Promise Ordering
There are two common styles that are used when writing policy. The Normal Order style dictates that promises should be written in in the Normal Order that the agent evaluates promises in. The other is reader optimized where promises are written in the order they make sense to the reader. Both styles have their merits, but there seems to be a trend toward the reader optimized style.
1) Normal Order
Here is an example of a policy written in the Normal Order. Note how
packages
are listed after files
. This could confuse a novice who
thinks that it is necessary for the files promise to only be attempted
after the package promsie is kept. However this style can be useful to
a policy expert who is familiar with Normal Ordering.
bundle agent main
{
vars:
"sshd_config"
string => "/etc/ssh/sshd_config";
files:
"$(sshd_config)"
edit_line => insert_lines("PermitRootLogin no"),
classes => results("bundle", "sshd_config");
packages:
"ssh"
policy => "present";
package_module => apt_get;
services:
sshd_config_repaired::
"ssh"
service_policy => "restart",
comment => "After the sshd config file has been repaired, the
service must be reloaded in order for the new
settings to take effect.";
}
2) Reader Optimized
Here is an example of a policy written to be optimized for the reader. Note how packages are listed before files in the order which users think about taking imperitive action. This style can make it significantly easier for a novice to understand the desired state, but it is important to remember that Normal Ordering still applies and that the promises will not be actuated in the order they are written.
bundle agent main
{
vars:
"sshd_config"
string => "/etc/ssh/sshd_config";
packages:
"ssh"
policy => "present";
package_module => apt_get;
files:
"$(sshd_config)"
edit_line => insert_lines("PermitRootLogin no"),
classes => results("bundle", "sshd_config");
services:
sshd_config_repaired::
"ssh"
service_policy => "restart",
comment => "After the sshd config file has been repaired, the
service must be reloaded in order for the new
settings to take effect.";
}
Whitespace and Line Length
Spaces are preferred to tab characters. Lines should not have trailing whitespace. Generally line length should not surpass 80 characters.
Curly brace alignment
Generally if opening and closing braces are not on a single line they should be aligned vertically.
Example:
bundle agent example
{
vars:
"people" slist => {
"Obi-Wan Kenobi",
"Luke Skywalker",
"Chewbacca",
"Yoda",
"Darth Vader",
};
"cuddly" slist => { "Chewbacca", "Yoda" };
}
Promise types
Promise types should have 1 indent and each promise type after the first listed should have a blank line before the next promise type.
This example illustrates the blank line before the "classes" type.
bundle agent example
{
vars:
"policyhost" string => "MyPolicyServerHostname";
classes:
"EL5" or => { "centos_5", "redhat_5" };
"EL6" or => { "centos_6", "redhat_6" };
}
Context class expressions
Context class expressions should have 2 indents and each context class expression after the first listed within a given promise type should have a blank line preceding it.
This example illustrates the blank line before the second context class expression (solaris) in the files type promise section:
bundle agent example
{
files:
any::
"/var/cfengine/inputs/"
copy_from => update_policy( "/var/cfengine/masterfiles","$(policyhost)" ),
classes => policy_updated( "policy_updated" ),
depth_search => recurse("inf");
solaris::
"/var/cfengine/inputs"
copy_from => update_policy( "/var/cfengine/masterfiles", "$(policyhost" ),
classes => policy_updated( "policy_updated" );
}
Policy Comments
In-line policy comments are useful for debugging and explaining why something is done a specific way. We encourage you to document your policy thoroughly.
Comments about general body and bundle behavior and parameters should be placed after the body or bundle definition, before the opening curly brace and should not be indented. Comments about specific promise behavior should be placed before the promise at the same indention level as the promiser or on the same line after the attribute.
bundle agent example(param1)
# This is an example bundle to illustrate comments
# param1 - string -
{
vars:
"copy_of_param1" string => "$(param1)";
"jedi" slist => {
"Obi-Wan Kenobi",
"Luke Skywalker",
"Yoda",
"Darth Vader", # He used to be a Jedi, and since he
# tossed the emperor into the Death
# Star's reactor shaft we are including
# him.
};
classes:
# Most of the time we don't need differentiation of redhat and centos
"EL5" or => { "centos_5", "redhat_5" };
"EL6" or => { "centos_6", "redhat_6" };
}
Policy Reports
It is common and useful to include reports in policy to get detailed information about what is going on. During a normal agent run the goal is to have 0 output so reports should always be guarded with a class. Carefully consider when your policy should generate report output. For policy degbugging type information (value of variables, classes that were set or not) the following style is recommended:
bundle agent example
{
reports:
DEBUG|DEBUG_example::
"DEBUG $(this.bundle): Desired Report Output";
}
As of version 3.7 variables can be used in double colon class expressions. If your policy will only be parsed by 3.7 or newer agents the following style is recommended:
bundle agent example
{
reports:
"DEBUG|DEBUG_$(this.bundle)"::
"DEBUG $(this.bundle): Desired Report Output";
}
Following this style keeps policy debug reports from spamming logs. It avoids
polluting the inform_mode
and verbose_mode
output, and it allows you to get
debug output for ALL policy or just a select bundle which is incredibly useful
when debugging a large policy set.
Promise Handles
Promise handles uniquely identify a promise within a policy. We suggest a simple naming
scheme of bundle_name_promise_type_class_restriction_promiser
to keep handles unique and
easily identifiable. Often it may be easier to omit the handle.
bundle agent example
{
commands:
dev::
"/usr/bin/git"
args => "pull",
contain => in_dir("/var/srv/myrepo"),
ifvarclass => "redhat",
handle => "example_commands_dev_redhat_git_pull";
}
Hashrockets (=>)
You may align hash rockets within a promise body scope and for grouped single line promises.
Example:
bundle agent example
{
files:
any::
"/var/cfengine/inputs/"
copy_from => update_policy( "/var/cfengine/masterfiles","$(policyhost)" ),
classes => policy_updated( "policy_updated" ),
depth_search => recurse("inf");
"/var/cfengine/modules"
copy_from => update_policy( "/var/cfengine/modules", "$(policyhost" ),
classes => policy_updated( "modules_updated" );
classes:
"EL5" or => { "centos_5", "redhat_5" };
"EL6" or => { "centos_6", "redhat_6" };
}
You may also simply leave them as they are:
bundle agent example
{
files:
any::
"/var/cfengine/inputs/"
copy_from => update_policy( "/var/cfengine/masterfiles","$(policyhost)" ),
classes => policy_updated( "policy_updated" ),
depth_search => recurse("inf");
"/var/cfengine/modules"
copy_from => update_policy( "/var/cfengine/modules", "$(policyhost" ),
classes => policy_updated( "modules_updated" );
classes:
"EL5" or => { "centos_5", "redhat_5" };
"EL6" or => { "centos_6", "redhat_6" };
}
Which one do you prefer?
Naming Conventions
Classes
Classes are intended to describe an aspect of the system, and they are combined in expressions to restrict when and where a promise should be actuated. To make this desired state easier to read classes should be named to describe the current state, not an action that should take place.
For example, here is a policy that uses a class that indicates an action that should be taken after having repaired the sshd config.
bundle agent main
{
vars:
"sshd_config" string => "/etc/ssh/sshd_config";
files:
"$(sshd_config)"
edit_line => insert_lines("PermitRootLogin no"),
classes => if_repaired("restart_sshd");
services:
!windows::
"ssh"
service_policy => "start",
comment => "We always want ssh to be running so that we have
administrative access";
restart_sshd::
"ssh"
service_policy => "restart",
comment => "Here it's kind of hard to tell *why* we are
restarting sshd";
}
Here is a slightly improved version that shows using classes to
describe the current state, or what happened as the result of the
promise. Note how it's easier to determine why the ssh service
should be restarted. Using the
results
,
scoped_classes_generic
,
or classes_generic
classes bodies can help improve class name consistency and are highly
reccomended.
bundle agent main
{
vars:
"sshd_config" string => "/etc/ssh/sshd_config";
files:
"$(sshd_config)"
edit_line => insert_lines("PermitRootLogin no"),
classes => results("bundle", "sshd_config");
services:
!windows::
"ssh"
service_policy => "start",
comment => "We always want ssh to be running so that we have
administrative access";
sshd_config_repaired::
"ssh"
service_policy => "restart",
comment => "After the sshd config file has been repaired, the
service must be reloaded in order for the new
settings to take effect.";
}
Internal variables & classes
Variables and classes that have no centralized reporting value are considered "internal". By convention internal variables and classes should be prefixed with an underscore "_".
Deprecating Bundles
As your policy library changes over time you may want to deprecate various bundles in favor of newer implimentations. To indicate that a bundle is deprecated we recommend the following style.
bundle agent old
{
meta:
"tags" slist => {
"deprecated=3.6.0",
"deprecation-reason=More feature rich implimentation",
"replaced-by=newbundle",
};
}
Automatic reindentation
reindent.pl
is available from the core repository. You can run reindent.pl
FILE1.cf FILE2.c FILE3.h
to reindent files, if you don't want to set
up Emacs. It will rewrite them with the new indentation, using Emacs
in batch mode.
Some editors also have support for automatic re-indentation.
Bundles Best Practices
The following contains practices to remember when creating bundles as you write policy.
How to choose and name bundles
Use the name of a bundle to represent a meaningful aspect of system administration, We recommend using a two- or three-part name that explains the context, general subject heading, and special instance. Names should be service-oriented in order to guide non-experts to understand what they are about.
For example:
- app_mail_postfix
- app_mail_mailman
- app_web_apache
- app_web_squid
- app_web_php
- app_db_mysql
- garbage_collection
- security_check_files
- security_check_processes
- system_name_resolution
- system_xinetd
- system_root_password
- system_processes
- system_files
- win_active_directory
- win_registry
- win_services
When to make a bundle
Put items into a single bundle if:
- They belong to the same conceptual aspect of system administration.
- They do not need to be switched on or off independently.
Put items into different bundles if:
- All of the promises in one bundle need to the checked before all of the promises in another bundle.
- You need to re-use the promises with different parameters.
In general, keep the number of bundles to a minimum. This is a knowledge-management issue. Clarity comes from differentiation, but only if the number of items is small.
When to use a paramaterized bundle or method
If you need to arrange for a managed convergent collection or sequence of promises that will occur for a list of (multiple) names or promisers, then use a bundle to simplify the code.
Write the promises (which may or may not be ordered) using a parameter for the different names, and then call the method passing the list of names as a parameter to reduce the amount of code.
bundle agent testbundle
{
vars:
"userlist" slist => { "mark", "jeang", "jonhenrik", "thomas", "eben" };
methods:
"any" usebundle => subtest("$(userlist)");
}
###########################################
bundle agent subtest(user)
{
commands:
"/bin/echo Fix $(user)";
files:
"/home/$(user)/."
create => "true";
reports:
linux::
"Finished doing stuff for $(user)";
}
When to use classes in common bundles
- When you need to use them in multiple bundles (because classes defined in common bundles have global scope).
When to use variables in common bundles
- For rationality, if the variable does not belong to any particular bundle, because it is
used elsewhere. (Qualified variable names such as
$(mybundle.myname)
are always globally accessible, so this is a cosmetic issue.)
When to use variables in local bundles
- If they are not needed outside the bundles.
- If they are used for iteration (without qualified scope).
- If they are tied to a specific aspect of system maintenance represented by the bundle, so
that accessing
$(bundle.var)
adds clarity.
External Data
It is common to integrate CFEngine with external data sources. External data sources could be hand edited data files, the cached result of an API call or generated by other tooling. This is especially useful for integrating CFEngine with other infrastructure components like a CMDB.
CFEngine can load structured data defined in JSON, YAML, CSV using the
readjson()
, readyaml()
, readcsv()
and readdata()
functions or by custom
parsing with data_resdstringarray()
and data_readstringarrayidx()
.
Additionally CFEngine provides the augments file as a way to define variables and classes that are available from the beginning of policy evaluation.
The augments file can be distributed globally as part of your policy by creating
def.json
in the root of your masterfiles, or it could be generated by the
agent itself for use in subsequent runs.
If the augments file is generated on the agent itself we recommend doing so from
a separate policy like update.cf
.
The Policy Framework
The CFEngine policy framework is called the Masterfiles Policy Framework,
MPF, or simply masterfiles because the files live in
/var/cfengine/masterfiles
on the policy server (on the clients, and
note the policy server is typically also a client, they are cached in
/var/cfengine/inputs
).
The following configuration files are part of the default CFEngine
installation in /var/cfengine/masterfiles
, and have special roles.
The Masterfiles Policy Framework is continually updated. You can track its development on github. Notable changes to the framework are documented in the changelog.
Setting up
First, review the update_def
and def
bundles found in
controls/update_def.cf
and controls/def.cf
respectively. Many settings you
need to change will live here.
It is recommended to use an augments file to specify things
traditionally set in update_def.cf
and def.cf
.
We will cover the policy in the order it is activated, starting with
update.cf
and its bundlesequence followed by promises.cf
and its
bundlesequence.
Special Classes
persistent_disable_*DAEMON*
Description: Disable a CFEngine Enterprise daemon component persistently.
DAEMON
can be one of cf_execd
, cf_monitord
or cf_serverd
.
This will stop the AGENT from starting automatically.
clear_persistent_disable_*DAEMON*
Description: Re-enable a previously disabled CFEngine Enterprise daemon component.
DAEMON
can be one of cf_execd
, cf_monitord
or cf_serverd
.
update.cf
Synchronizing clients with the policy server happens here, in
update.cf
. Its main job is to copy all the files on the policy
server (usually the hub) under $(sys.masterdir)
(usually
/var/cfengine/masterfiles
) to the local host into $(sys.inputdir)
(usually /var/cfengine/inputs
).
This file should rarely if ever change. Should you ever change it (or
when you upgrade CFEngine), take special care to ensure the old and
the new CFEngine can parse and execute this file successfully. If not,
you risk losing control of your system (that is, if CFEngine cannot
successfully execute update.cf
, it has no mechanism for distributing
new policy files).
By default, the policy defined in update.cf is executed at the
beginning of a cf-execd
scheduled agent run (see schedule
and
exec_command
as defined in body executor control
in
controls/cf_execd.cf
). When the update policy completes
(regardless of success or failure) the policy defined in promises.cf
is activated.
This is a standalone policy file. You can actually run it with
cf-agent -KI -f ./update.cf
but if you don't understand what that
command does, please hold off until you've gone through the CFEngine
documentation. The contents of update.cf
duplicate other things
under lib
sometimes, in order to be completely standalone.
To repeat, when update.cf
is broken, things go bonkers. CFEngine
will try to run a backup failsafe.cf
you can find in the C core
under libpromises/failsafe.cf
(that .cf
file is written into the C
code and can't be modified). If things get to that point, you probably
have to look at why corrupted policies made it into production.
As is typical for CFEngine, the policy and the configuration are mixed. In
controls/update_def.cf
you'll find some very useful settings. Keep referring
to controls/update_def.cf
as you read this. We are skipping the nonessential
ones.
How it works
There are multiple stages in update.cf
. This document covers each
bundle in the order defined by the bundlesequence.
update_def (bundle)
This bundle is defined in
controls/update_def.cf
.
bundle common update_def
defines settings and variables that are
used throughout the update policy.
As of CFEngine version 3.7 it is recommended that these setting
changes are specified in def.json
to ease policy framework updates.
input_name_patterns (variable)
A list of regular expressions defining which files should be considered for copying during update.
masterfiles_perms_mode (variable)
Usually you want to leave this at 0600
meaning the inputs will be
readable only by their owner.
trigger_upgrade (class)
Off by default
When this class is set, the internal CFEngine upgrade mechanism is enabled. Currently this upgrade policy is specific to CFEngine Enterprise.
cfengine_internal_masterfiles_update (class)
Off by default.
This class enables masterfiles automatic update from a version control repository. Currently this policy relies on tooling available in CFEngine Enterprise.
Turn this on (set to any
) to auto-deploy policies on the policy
server, it has no effect on clients. See
Version Control and Configuration Policy
for details on how to use it.
This may result in DATA LOSS.
cfengine_internal_encrypt_transfers (class)
Off by default.
This class enables encryption during policy updates. If you are
running CFEngine versions 3.6 with protocol => "2"
or protocol =>
"latest"
this settings is unnecessary as all traffic will be
encapsulated inside of TLS. CFEngine Version 3.7 uses protocol =>
"2"
by default.
Turn this on (set to any
) to encrypt your policy transfers.
Note it has a duplicate in def.cf
, see below. If they are not
synchronized, you may get unexpected behavior.
cfengine_internal_purge_policies (class)
Off by default.
This class causes the update behavior to change from only copying changed files down to performing a synchronization by purging files on the client that do not exist on the server.
Turn this on (set to any
) to delete any files in your
$(sys.inputdir)
that are not in the policy server's masterfiles.
This may result in DATA LOSS.
Note it has a duplicate in def.cf
, see below. If they are not
synchronized, you may get unexpected behavior.
cfengine_internal_preserve_permissions (class)
Off by default.
Turn this on (set to any
) to preserve the permissions of the policy
server's masterfiles when they are copied.
This may result in FUNCTIONALITY LOSS if your scripts lose their exec bits unexpectedly
Note it has a duplicate in def.cf
, see below. If they are not
synchronized, you may get unexpected behavior.
cfengine_internal_disable_cf_promises_validated (class)
Off by default.
Turn this on (set to any
) to have remote agents always scan all of
masterfiles for changes and update accordingly.
This is not recommended as it both removes a safety mechanism that checks for policy to be valid before allowing clients to download updates, and the increased load on the hub will affect scalability.
Consider using time_based, select_class
or dist
based
classes instead of any to retain some of the benefits.
enable_cfengine_enterprise_hub_ha (class)
Off by default.
This class enables the HA policy for CFEngine Enterprise hubs. This class is not set by default.
cfe_internal_dc_workflow (bundle)
This bundle implements the auto-deployment of policies. See
Version Control and Configuration Policy
and cfengine_internal_masterfiles_update
below for details. This
policy is currently specific to CFEngine Enterprise.
cfe_internal_update_policy (bundle)
This bundle is defined in cfe_internal/update/update_policy.cf
.
It updates the policy files themselves. Basically it's a
check step that looks at $(sys.inputdir)/cf_promises_validated
and
compares it with the policy server's
$(sys.masterdir)/cf_promises_validated
. Then there's the actual
copy, which happens only if the cf_promises_validated
file was
updated in the check step. You can bypass this check and perform a
full scan by running cf-agent -KIf update.cf -D validated_updates_ready
.
Implementation (warning: advanced usage):
bundle agent cfe_internal_update_policy
{
vars:
"inputs_dir" string => translatepath("$(sys.inputdir)"),
comment => "Directory containing CFEngine policies",
handle => "cfe_internal_update_policy_vars_inputs_dir";
windows::
"master_location" string => "/var/cfengine/masterfiles", # NB! NOT $(sys.workdir) on Windows !
comment => "The master CFEngine policy directory on the policy host",
handle => "cfe_internal_update_policy_vars_master_location_windows";
"modules_dir" string => "/var/cfengine/modules", # NB! NOT $(sys.workdir) on Windows !
comment => "Directory containing CFEngine modules",
handle => "cfe_internal_update_policy_vars_modules_dir_windows";
"plugins_dir" string => "/var/cfengine/plugins", # NB! NOT $(sys.workdir) on Windows !
comment => "Directory containing CFEngine plugins",
handle => "cfe_internal_update_policy_vars_plugins_dir_windows";
!windows::
"master_location" string => "$(sys.masterdir)",
comment => "The master CFEngine policy directory on the policy host",
handle => "cfe_internal_update_policy_vars_master_location";
"modules_dir" string => translatepath("$(sys.workdir)/modules"),
comment => "Directory containing CFEngine modules",
handle => "cfe_internal_update_policy_vars_modules_dir";
"plugins_dir" string => translatepath("$(sys.workdir)/plugins"),
comment => "Directory containing CFEngine plugins",
handle => "cfe_internal_update_policy_vars_plugins_dir";
any::
"file_check" string => translatepath("$(inputs_dir)/promises.cf"),
comment => "Path to a policy file",
handle => "cfe_internal_update_vars_file_check";
"ppkeys_file" string => translatepath("$(sys.workdir)/ppkeys/localhost.pub"),
comment => "Path to public key file",
handle => "cfe_internal_update_policy_vars_ppkeys_file";
"postgresdb_dir" string => "$(sys.workdir)/state/pg/data",
comment => "Directory where Postgres database files will be stored on hub -",
handle => "cfe_internal_update_policy_postgresdb_dir";
"postgresdb_log" string => "/var/log/postgresql.log",
comment => "File where Postgres database files will be logging -",
handle => "cfe_internal_update_policy_postgresdb_log_file";
"redis_conf_file" string => translatepath("$(sys.workdir)/config/redis.conf"),
comment => "Path to Redis configuration file",
handle => "cfe_internal_update_policy_redis_conf_file";
classes:
"validated_updates_ready"
expression => "cfengine_internal_disable_cf_promises_validated",
comment => "If cf_promises_validated is disabled, then updates are
always considered validated.";
any::
"local_files_ok" expression => fileexists("$(file_check)"),
comment => "Check for $(sys.masterdir)/promises.cf",
handle => "cfe_internal_update_classes_files_ok";
# create a global files_ok class
"cfe_internal_trigger" expression => "local_files_ok",
classes => u_if_else("files_ok", "files_ok");
files:
!am_policy_hub:: # policy hub should not alter inputs/ uneccessary
"$(inputs_dir)/cf_promises_validated"
comment => "Check whether a validation stamp is available for a new policy update to reduce the distributed load",
handle => "cfe_internal_update_policy_check_valid_update",
copy_from => u_rcp("$(master_location)/cf_promises_validated", @(update_def.policy_servers)),
action => u_immediate,
classes => u_if_repaired("validated_updates_ready");
!am_policy_hub.!windows::
"$(modules_dir)"
comment => "Always update modules files on client side",
handle => "cfe_internal_update_policy_files_update_modules",
copy_from => u_rcp("$(modules_dir)", @(update_def.policy_servers)),
depth_search => u_recurse("inf"),
perms => u_m("755"),
action => u_immediate;
"$(plugins_dir)"
comment => "Always update plugins files on client side",
handle => "cfe_internal_update_policy_files_update_plugins",
copy_from => u_rcp("$(plugins_dir)", @(update_def.policy_servers)),
depth_search => u_recurse("inf"),
perms => u_m("755"),
action => u_immediate;
!am_policy_hub.windows::
"$(sys.workdir)\modules"
comment => "Always update modules files on client side (Windows)",
handle => "cfe_internal_update_policy_files_update_modules_windows",
copy_from => u_rcp("$(modules_dir)", @(update_def.policy_servers)),
depth_search => u_recurse("inf"),
perms => u_m("755"),
action => u_immediate;
"$(sys.workdir)\plugins"
comment => "Always update plugins files on client side (Windows)",
handle => "cfe_internal_update_policy_files_update_plugins_windows",
copy_from => u_rcp("$(plugins_dir)", @(update_def.policy_servers)),
depth_search => u_recurse("inf"),
perms => u_m("755"),
action => u_immediate;
am_policy_hub|validated_updates_ready:: # policy hub should always put masterfiles in inputs in order to check new policy
"$(inputs_dir)"
comment => "Copy policy updates from master source on policy server if a new validation was acquired",
handle => "cfe_internal_update_policy_files_inputs_dir",
copy_from => u_rcp("$(master_location)", @(update_def.policy_servers)),
depth_search => u_recurse("inf"),
file_select => u_input_files,
action => u_immediate;
!policy_server.enable_cfengine_enterprise_hub_ha::
"$(sys.workdir)/policy_server.dat"
comment => "Copy policy_server.dat file from server",
handle => "cfe_internal_update_ha_policy_server",
copy_from => u_rcp("$(sys.workdir)/state/master_hub.dat", @(update_def.policy_servers)),
action => u_immediate,
classes => u_if_repaired("replica_failover"); # not needed ?
am_policy_hub::
"$(master_location)/."
comment => "Make sure masterfiles folder has right file permissions",
handle => "cfe_internal_update_policy_files_sys_workdir_masterfiles",
perms => u_m($(update_def.masterfiles_perms_mode)),
depth_search => u_recurse_basedir("inf"),
action => u_immediate;
}
body perms u_m(p)
{
mode => "$(p)";
}
body perms u_mo(p,o)
{
mode => "$(p)";
owners => {"$(o)"};
}
body perms u_shared_lib_perms
{
!hpux::
mode => "0644";
hpux::
mode => "0755"; # Mantis 1114, Redmine 1179
}
body file_select u_cf3_files
{
leaf_name => { "cf-.*" };
file_result => "leaf_name";
}
body file_select u_input_files
{
leaf_name => { @(update_def.input_name_patterns) };
file_result => "leaf_name";
}
body copy_from u_rcp(from,server)
{
source => "$(from)";
compare => "digest";
trustkey => "false";
!am_policy_hub::
servers => { "$(server)" };
cfengine_internal_encrypt_transfers::
encrypt => "true";
cfengine_internal_purge_policies::
purge => "true";
cfengine_internal_preserve_permissions::
preserve => "true";
}
body copy_from u_cp(from)
{
source => "$(from)";
compare => "digest";
}
body copy_from u_cp_nobck(from)
{
source => "$(from)";
compare => "digest";
copy_backup => "false";
}
body action u_immediate
{
ifelapsed => "0";
}
body depth_search u_recurse(d)
{
depth => "$(d)";
exclude_dirs => { "\.svn", "\.git", "git-core" };
}
body depth_search u_recurse_basedir(d)
{
include_basedir => "true";
depth => "$(d)";
exclude_dirs => { "\.svn", "\.git", "git-core" };
}
body classes u_if_repaired(x)
{
promise_repaired => { "$(x)" };
}
body classes u_if_repaired_then_cancel(y)
{
cancel_repaired => { "$(y)" };
}
body classes u_if_else(yes,no)
{
promise_repaired => { "$(yes)" };
repair_failed => { "$(no)" };
repair_denied => { "$(no)" };
repair_timeout => { "$(no)" };
}
body contain u_in_shell
{
useshell => "true";
}
body contain u_in_shell_and_silent
{
useshell => "true";
no_output => "true";
}
body contain u_postgres
{
useshell => "useshell";
exec_owner => "cfpostgres";
chdir => "/tmp";
no_output => "true";
}
body action u_ifwin_bg
{
windows::
background => "true";
}
body service_method u_bootstart
{
service_autostart_policy => "boot_time";
}
body contain u_silent_in_dir(s)
{
chdir => "$(s)";
no_output => "true";
}
body link_from u_ln_s(x)
{
link_type => "symlink";
source => "$(x)";
when_no_source => "force";
}
body delete u_tidy
{
dirlinks => "delete";
rmdirs => "true";
}
cfe_internal_update_bins (bundle)
This step does a self-update of CFEngine. See the Enterprise documentation for details; this functionality is unsupported in CFEngine Community.
cfe_internal_update_processes (bundle)
This step manages the running processes, ensuring cf-execd
and
cf-serverd
and cf-monitord
are running and doing some other tasks.
promises.cf
How it works
promises.cf
is your main run file. Keep referring to your
installation's promises.cf
as you read this.
promises.cf
is the first file that cf-agent
with no arguments will
try to look for. So whenever you see cf-agent
with no flile
parameter, read it as "run my promises.cf
".
It should contain all of the basic configuration
settings, including a list of other files to include. In normal
operation, it must also have a bundlesequence
.
bundlesequence
The bundlesequence
acts like the 'genetic makeup' of the configuration. Edit
the bundlesequence
to add any bundles you have defined, or are pre-defined.
Consider using the services_autorun
facility, an augments file or
integrating your bundles as methods
type promises so you don't have to edit
this setting at all.
BEWARE THAT ONLY VALID (KNOWN) BUNDLES CAN BE ADDED.
By default, the inventory modules, then internal hub modules, then
Design Center sketches, then the autorun services, and finally
internal management bundles are in the bundlesequence
.
In a large configuration, you might want to have a different
bundlesequence
for different classes of host, so that you can build
a complete system like a check-list from different combinations of
building blocks. You can construct different lists by composing them
from other lists, or you can use methods promises as an alternative
for composing bundles for different classes. This is an advanced topic
and a risky area (if you get it wrong, your policies will not
validate) so make sure you test your changes carefully!
inventory_control (bundle)
The inventory policy was added in CFEngine version 3.6. Inventory modules are desinged to define classes and variables based on the inspected state of the system. These classes and variales can be levereged when writing policy, and in CFEngine Enterprise they can be reported on from Mission Portal. You can disable pieces of it (inventory modules) or the whole thing if you wish.
disable_inventory (class)
This class is off by default (meaning the inventory is on by default). Here's the master switch to disable all inventory modules.
disable_inventory_lsb (class)
LSB is the Linux Standard Base, see https://wiki.linuxfoundation.org/en/LSB
By default, this class is turned off (and the module is on) if the LSB
executable /usr/bin/lsb_release
can be found. This inventory module
will populate inventory reports and variables for you with LSB
details. For details, see LSB
disable_inventory_dmidecode (class)
By default, this class is turned off (and the module is on) if the
executable /usr/sbin/dmidecode
can be found. This inventory module
will populate inventory reports and variables for you. For details,
see DMI decoding
disable_inventory_LLDP (class)
LLDP is a protocol for Link Layer Discovery. See http://en.wikipedia.org/wiki/Link_Layer_Discovery_Protocol
By default, this class is turned off (and the module is on) if the
executable /usr/bin/lldpctl
can be found. This inventory module will
populate variables for you. For details, see LLDP
disable_inventory_package_refresh (class)
By default, this class is turned off (and the module is on). This inventory module will populate the installed packages for you. On CFEngine Enterprise, the available packages will also be populated.
disable_inventory_mtab (class)
By default, this class is turned off (and the module is on) if
/etc/mtab
exists. This inventory module will populate variables for
you based on the mounted filesystems. For details, see mtab
disable_inventory_fstab (class)
By default, this class is turned off (and the module is on) if
$(sys.fstab)
(usually /etc/fstab
or /etc/vfstab
) exists. This
inventory module will populate variables for you based on the defined
filesystems. For details, see fstab
disable_inventory_proc (class)
By default, this class is turned off (and the module is on) if /proc
is a directory. This inventory module will populate variables for you
from some of the contents of /proc
. For details, see procfs
disable_inventory_cmdb (class)
By default, this class is turned on (and the module is off).
Turn this on (set to any
) to allow each client to load a me.json
file from the server and load its contents. For details, see CMDB
@(inventory.bundles) (bundle)
This bundle is defined as bundle common inventory
in promises.cf
.
Inventory bundles may vary between platform and other classes.
def (bundle)
This bundle is defined as bundle common def
in controls/def.cf
def
has some crucial settings used by the rest of CFEngine. It's
expected that users may edit it, but won't normally change the rest of
masterfiles except in services
or if they know it's
necessary. This bundle should be configured in conjunction with
update_def
as there are some settings that should be kept in sync
between the two policies.
As of CFEngine version 3.7 it is recommended that these setting
changes are specified in def.json
to ease policy framework updates.
Keep referring to def.cf
as you read this.
Implementation (warning: advanced usage):
bundle common def
{
classes:
"_workaround_CFE_2333" -> { "https://tracker.mender.io/browse/CFE-2333" }
or => { "cfengine_3_7_3", "cfengine_3_8_1", "cfengine_3_8_2" };
# If the augments_file is parsed from C then we do not need ot do this work
# from policy
!(feature_def_json_preparse)|(_workaround_CFE_2333)::
"have_augments_file" expression => fileexists($(augments_file)), scope => "bundle";
"have_augments_classes" expression => isvariable("augments[classes]"), scope => "bundle";
"have_augments_inputs" expression => isvariable("augments[inputs]"), scope => "bundle";
have_augments_classes.!(feature_def_json_preparse)|(_workaround_CFE_2333)::
"$(augments_classes_data_keys)"
expression => classmatch("$(augments[classes][$(augments_classes_data_keys)])"),
meta => { "augments_class", "derived_from=$(augments_file)" };
vars:
!(feature_def_json_preparse)|(_workaround_CFE_2333)::
"augments_file" string => "$(this.promise_dirname)/../../def.json";
"defvars" slist => variablesmatching("default:def\..*", "defvar");
have_augments_file.!feature_def_json_preparse|(_workaround_CFE_2333)::
"augments" data => readjson($(augments_file), 100k), ifvarclass => "have_augments_file";
"augments_inputs" slist => getvalues("augments[inputs]");
"override_vars" slist => getindices("augments[vars]");
"override_data_$(override_vars)" data => mergedata("augments[vars][$(override_vars)]");
"override_data_s_$(override_vars)" string => format("%S", "override_data_$(override_vars)");
any::
"augments_inputs"
slist => {},
ifvarclass => not( isvariable( "augments_inputs" ) ),
comment => "It's important that we define this list, even if it's empty
or we get errors about the list being unresolved.";
have_augments_classes.!(feature_def_json_preparse)|(_workaround_CFE_2333)::
"augments_classes_data" data => mergedata("augments[classes]");
"augments_classes_data_keys" slist => getindices("augments_classes_data");
any::
# Begin change
# Your domain name, for use in access control
# Note: this default may be inaccurate!
"domain"
string => "$(sys.domain)",
comment => "Define a global domain for all hosts",
handle => "common_def_vars_domain",
ifvarclass => not(isvariable("domain"));
# Mail settings used by body executor control found in controls/cf_execd.cf
"mailto"
string => "root@$(def.domain)",
ifvarclass => not(isvariable("mailto"));
"mailfrom"
string => "root@$(sys.uqhost).$(def.domain)",
ifvarclass => not(isvariable("mailfrom"));
"smtpserver"
string => "localhost",
ifvarclass => not(isvariable("smtpserver"));
# List here the IP masks that we grant access to on the server
# Only define here if we are not capable of parsing augments from C
"acl"
slist => getvalues("override_data_acl"),
comment => "JSON-sourced: Define an acl for the machines to be granted accesses",
handle => "common_def_json_vars_acl",
ifvarclass => and(isvariable("override_data_acl"), "!feature_def_json_preparse"),
meta => { "defvar" };
"acl"
slist => {
# Allow everything in my own domain.
# Note that this:
# 1. requires def.domain to be correctly set
# 2. will cause a DNS lookup for every access
# ".*$(def.domain)",
# Assume /16 LAN clients to start with
"$(sys.policy_hub)/16",
# Uncomment below if HA is used
#"@(def.policy_servers)"
# "2001:700:700:3.*",
# "217.77.34.18",
# "217.77.34.19",
},
comment => "Define an acl for the machines to be granted accesses",
handle => "common_def_vars_acl",
ifvarclass => and(not(isvariable("override_data_acl")), not(isvariable("acl"))),
meta => { "defvar" };
# Out of the hosts in allowconnects, trust new keys only from the
# following ones. This is open by default for bootstrapping.
# Only define here if we are not capable of parsing augments from C
"trustkeysfrom"
slist => getvalues("override_data_trustkeysfrom"),
comment => "JSON-sourced: define from which machines keys can be trusted",
ifvarclass => and(isvariable("override_data_trustkeysfrom"), "!feature_def_json_preparse"),
meta => { "defvar" };
"trustkeysfrom"
slist => {
# COMMENT THE NEXT LINE OUT AFTER ALL MACHINES HAVE BEEN BOOTSTRAPPED.
"0.0.0.0/0", # allow any IP
},
comment => "Define from which machines keys can be trusted",
ifvarclass => and(not(isvariable("override_data_trustkeysfrom")),
not(isvariable("trustkeysfrom"))),
meta => { "defvar" };
# Agent Controls
"control_agent_maxconnections"
int => "30",
ifvarclass => not( isvariable( "control_agent_maxconnections" ) );
debian::
"environment_vars"
handle => "common_def_vars_environment_vars",
comment => "Environment variables of the agent process. The values
of environment variables are inherited by child commands.",
slist => {
"DEBIAN_FRONTEND=noninteractive",
# "APT_LISTBUGS_FRONTEND=none",
# "APT_LISTCHANGES_FRONTEND=none",
};
# End change #
any::
"dir_masterfiles" string => translatepath("$(sys.masterdir)"),
comment => "Define masterfiles path",
handle => "common_def_vars_dir_masterfiles";
"dir_reports" string => translatepath("$(sys.workdir)/reports"),
comment => "Define reports path",
handle => "common_def_vars_dir_reports";
"dir_software" string => translatepath("$(sys.workdir)/master_software_updates"),
comment => "Define software path",
handle => "common_def_vars_dir_software";
"dir_bin" string => translatepath("$(sys.bindir)"),
comment => "Define binary path",
handle => "common_def_vars_dir_bin";
"dir_modules" string => translatepath("$(sys.workdir)/modules"),
comment => "Define modules path",
handle => "common_def_vars_dir_modules";
"dir_plugins" string => translatepath("$(sys.workdir)/plugins"),
comment => "Define plugins path",
handle => "common_def_vars_dir_plugins";
"cf_apache_user" string => "cfapache",
comment => "User that CFEngine Enterprise webserver runs as",
handle => "common_def_vars_cf_cfapache_user";
"cf_apache_group" string => "cfapache",
comment => "Group that CFEngine Enterprise webserver runs as",
handle => "common_def_vars_cf_cfapache_group";
solaris::
"cf_runagent_shell"
string => "/usr/bin/sh",
comment => "Define path to shell used by cf-runagent",
handle => "common_def_vars_solaris_cf_runagent_shell";
!(windows|solaris)::
"cf_runagent_shell"
string => "/bin/sh",
comment => "Define path to shell used by cf-runagent",
handle => "common_def_vars_cf_runagent_shell";
any::
"base_log_files" slist =>
{
"$(sys.workdir)/cf3.$(sys.uqhost).runlog",
"$(sys.workdir)/promise_summary.log",
};
"enterprise_log_files" slist =>
{
"$(sys.workdir)/cf_notkept.log",
"$(sys.workdir)/cf_repair.log",
"$(sys.workdir)/state/cf_value.log",
"$(sys.workdir)/outputs/dc-scripts.log",
"$(sys.workdir)/state/promise_log.jsonl",
"$(sys.workdir)/state/classes.jsonl",
};
"hub_log_files" slist =>
{
"$(sys.workdir)/httpd/logs/access_log", # Mission Portal
"$(sys.workdir)/httpd/logs/error_log", # Mission Portal
};
"max_client_history_size" -> { "cf-hub", "CFEngine Enterprise" }
int => "50M",
comment => "The threshold of report diffs which will trigger purging of
diff files.";
enterprise.!am_policy_hub::
# CFEngine's own log files
"cfe_log_files" slist => { @(base_log_files), @(enterprise_log_files) };
enterprise.am_policy_hub::
# CFEngine's own log files
"cfe_log_files" slist => { @(base_log_files), @(enterprise_log_files), @(hub_log_files) };
!enterprise::
# CFEngine's own log files
"cfe_log_files" slist => { @(base_log_files) };
# Directories where logs are rotated and old files need to be purged.
any::
"log_dir[outputs]" string => "$(sys.workdir)/outputs";
"log_dir[reports]" string => "$(sys.workdir)/reports";
enterprise.am_policy_hub::
"log_dir[application]" string => "$(sys.workdir)/httpd/htdocs/application/logs";
any::
"cfe_log_dirs" slist => getvalues( log_dir );
# Enterprise HA Related configuration
# enable_cfengine_enterprise_hub_ha is defined below
# Disabled by default
enable_cfengine_enterprise_hub_ha::
"standby_servers" slist => filter("$(sys.policy_hub)", "ha_def.ips", false, true, 10);
"policy_servers" slist => { "$(sys.policy_hub)", "@(standby_servers)" };
!enable_cfengine_enterprise_hub_ha::
"policy_servers" slist => {"$(sys.policy_hub)"};
classes:
### Enable special features policies. Set to "any" to enable.
# Auto-load files in "services/autorun" and run bundles tagged "autorun".
# Disabled by default!
"services_autorun" -> { "jira:CFE-2135" }
comment => "This class enables the automatic parsing running of bundles
tagged with 'autorun'. Evaluation limitations require that
this class is set at the beginning of the agent run, so it
must be defined in the augments file (def.json), or as an
option to the agent with --define or -D. Changing the
expression here will *NOT* work correctly. Setting the class
here will result in an error due to the autorun bundle not
being found.",
expression => "!any";
# Internal CFEngine log files rotation
"cfengine_internal_rotate_logs" expression => "any";
# Enable or disable agent email output (also see mailto, mailfrom and
# smtpserver)
"cfengine_internal_agent_email" expression => "any";
"cfengine_internal_disable_agent_email" expression => "!any";
# Enable or disable external watchdog to ensure cf-execd is running
"cfe_internal_core_watchdog_enabled" expression => "!any";
"cfe_internal_core_watchdog_disabled" expression => "!any";
# Transfer policies and binaries with encryption
# you can also request it from the command line with
# -Dcfengine_internal_encrypt_transfers
# NOTE THAT THIS CLASS ALSO NEEDS TO BE SET IN update.cf
"cfengine_internal_encrypt_transfers" expression => "!any";
# Purge policies that don't exist on the server side.
# you can also request it from the command line with
# -Dcfengine_internal_purge_policies
# NOTE THAT THIS CLASS ALSO NEEDS TO BE SET IN update.cf
"cfengine_internal_purge_policies" expression => "!any";
# Preserve permissions of the policy server's masterfiles.
# you can also request it from the command line with
# -Dcfengine_internal_preserve_permissions
# NOTE THAT THIS CLASS ALSO NEEDS TO BE SET IN update.cf
"cfengine_internal_preserve_permissions" expression => "!any";
# Allow the hub to edit sudoers in order for the Apache user to
# run passwordless sudo cf-runagent. Enable this if you want the
# Mission Portal to be able to troubleshoot failed Design Center
# sketch activations on a host.
"cfengine_internal_sudoers_editing_enable" expression => "!any";
# Class defining which versions of cfengine are (not) supported
# by this policy version.
# Also note that this policy will only be run on enterprise policy_server
"postgresql_maintenance_supported"
expression => "(policy_server.enterprise.!enable_cfengine_enterprise_hub_ha)|(policy_server.enterprise.enable_cfengine_enterprise_hub_ha.hub_active)";
# This class is for PosgreSQL maintenance
# pre-defined to every Sunday at 2 a.m.
# This can be changed later on.
"postgresql_full_maintenance" expression => "postgresql_maintenance_supported.Sunday.Hr02.Min00_05";
# Run vacuum job on database
# pre-defined to every night except Sunday when full cleanup is executed.
"postgresql_vacuum" expression => "postgresql_maintenance_supported.!Sunday.Hr02.Min00_05";
# Enable CFEngine Enterprise HA Policy
"enable_cfengine_enterprise_hub_ha" expression => "!any";
#"enable_cfengine_enterprise_hub_ha" expression => "enterprise_edition";
# Enable failover to node which is outside cluster
#"failover_to_replication_node_enabled" expression => "enterprise_edition";
# Enable cleanup of agent report diffs when they exceed
# `def.max_client_history_size`
"enable_cfe_internal_cleanup_agent_reports" -> { "cf-hub", "CFEngine Enterprise" }
expression => "enterprise_edition",
comment => "If reports are not collected for an extended period of time
the disk may fill up or cause additional collection
issues.";
reports:
DEBUG|DEBUG_def::
"DEBUG: $(this.bundle)";
"$(const.t) def.json was found at $(augments_file)"
ifvarclass => fileexists( $(augments_file) );
"$(const.t) override request $(override_vars) to '$(override_data_s_$(override_vars))'; new value '$($(override_vars))'"
ifvarclass => isvariable("override_data_$(override_vars)");
"$(const.t) defined class '$(augments_classes_data_keys)' because of classmatch('$(augments[classes][$(augments_classes_data_keys)])')"
ifvarclass => "$(augments_classes_data_keys)";
"$(const.t) $(defvars) = $($(defvars))";
"DEBUG $(this.bundle): Agent parsed augments_file"
ifvarclass => "have_augments_file.feature_def_json_preparse";
"DEBUG $(this.bundle): Policy parsed augments_file"
ifvarclass => "have_augments_file.!feature_def_json_preparse";
}
bundle common inventory_control
{
vars:
"lldpctl_exec" string => ifelse(fileexists("/usr/sbin/lldpctl"), "/usr/sbin/lldpctl",
fileexists("/usr/local/bin/lldpctl"), "/usr/local/bin/lldpctl",
"/usr/sbin/lldpctl");
"lldpctl_json" string => "$(lldpctl_exec) -f json",
unless => isvariable("def.lldpctl_json");
"lldpctl_json" string => "$(def.lldpctl_json)",
if => isvariable("def.lldpctl_json");
"lsb_exec" string => "/usr/bin/lsb_release";
"mtab" string => "/etc/mtab";
"proc" string => "/proc";
vars:
freebsd::
"dmidecoder" string => "/usr/local/sbin/dmidecode";
!freebsd::
"dmidecoder" string => "/usr/sbin/dmidecode";
classes:
# setting this disables all the inventory modules except package_refresh
"disable_inventory" expression => "!any";
# disable specific inventory modules below
# by default disable the LSB inventory if the general inventory
# is disabled or the binary is missing. Note that the LSB
# binary is typically not very fast.
"disable_inventory_lsb" expression => "disable_inventory";
"disable_inventory_lsb" not => fileexists($(lsb_exec));
# by default disable the dmidecode inventory if the general
# inventory is disabled or the binary does not exist. Note that
# typically this is a very fast binary.
"disable_inventory_dmidecode" expression => "disable_inventory";
"disable_inventory_dmidecode" not => fileexists($(dmidecoder));
# by default disable the LLDP inventory if the general inventory
# is disabled or the binary does not exist. Note that typically
# this is a reasonably fast binary but still may require network
# I/O.
"disable_inventory_LLDP" expression => "disable_inventory";
"disable_inventory_LLDP" not => fileexists($(lldpctl_exec));
# by default run the package inventory refresh every time, even
# if disable_inventory is set
"disable_inventory_package_refresh" expression => "!any";
# by default disable the mtab inventory if the general inventory
# is disabled or $(mtab) is missing. Note that this is very
# fast.
"disable_inventory_mtab" expression => "disable_inventory";
"disable_inventory_mtab" not => fileexists($(mtab));
# by default disable the fstab inventory if the general
# inventory is disabled or $(sys.fstab) is missing. Note that
# this is very fast.
"disable_inventory_fstab" expression => "disable_inventory";
"disable_inventory_fstab" not => fileexists($(sys.fstab));
# by default disable the proc inventory if the general
# inventory is disabled or /proc is missing. Note that
# this is typically fast.
"disable_inventory_proc" expression => "disable_inventory|freebsd";
"disable_inventory_proc" not => isdir($(proc));
# by default don't run the CMDB integration every time, even if
# disable_inventory is not set
"disable_inventory_cmdb" expression => "any";
reports:
verbose_mode.disable_inventory::
"$(this.bundle): All inventory modules disabled";
verbose_mode.!disable_inventory_lsb::
"$(this.bundle): LSB module enabled";
verbose_mode.!disable_inventory_dmidecode::
"$(this.bundle): dmidecode module enabled";
verbose_mode.!disable_inventory_LLDP::
"$(this.bundle): LLDP module enabled";
verbose_mode.!disable_inventory_mtab::
"$(this.bundle): mtab module enabled";
verbose_mode.!disable_inventory_fstab::
"$(this.bundle): fstab module enabled";
verbose_mode.!disable_inventory_proc::
"$(this.bundle): proc module enabled";
verbose_mode.!disable_inventory_package_refresh::
"$(this.bundle): package_refresh module enabled";
verbose_mode.!disable_inventory_cmdb::
"$(this.bundle): CMDB module enabled";
}
augments_file (variable)
This variable defines the path to a JSON file used to "augment" the current definitions.
The augments_file
is intended to ease policy framework upgrades by
providing a standard location for site specific settings to be
defined. Currently the augments file supports defining additional
inputs and classes as well as overriding variables matching defvars
.
By default the augments_file
is expected to exist in the root of
your policy. Should you want to deliver different augments files to
different clients, you may consider pointing this to a file
outside of the masterfiles tree that is downloaded by the
individual clients.
defvars (variable)
This variable defines the list of variables that are allowed to be
overridden by the augments_file
.
augments (variable)
This variable contains the augments data as loaded from the
augments_file
. By default this is limited to 100k. If your
augments_file
is larger than 100k you will want to adjust this
limit.
Here is an example augments_file
:
{ "classes": { "my_apache": [ "server1", "server2", "redhat.*" ], "my_other_apache": [ "server[34]", "debian.*" ], "my_filehost": [ "server3" ], "my_gateway": [ "server3" ], "my_yum_role": [ "redhat.*" ], "my_redhat_role": [ "redhat.*" ], "my_apt_role": [ "debian.*" ], "my_debian_role": [ "debian.*" ], "services_autorun": [ "any" ], "cfengine_internal_disable_agent_email": [ "enterprise_edition" ], "cfe_internal_core_watchdog_enabled": [ "any" ] }, "inputs": [ "$(sys.libdir)/bundles.cf" ], "vars": { "acl": [ ".*$(def.domain)", "$(sys.policy_hub)/16" ], "control_agent_maxconnections": 100, "input_name_patterns": [ ".*\\.cf",".*\\.dat",".*\\.txt", ".*\\.conf", ".*\\.mustache", ".*\\.sh", ".*\\.pl", ".*\\.py", ".*\\.rb", "cf_promises_release_id", ".*\\.json", ".*\\.yaml", ".*\\.js" ] } }
domain (variable)
Set your domain
to the right value. By default it's used for mail
and to deduce your file access ACLs.
mailto (variable)
This variable defines the email address that agent run output is sent to.
mailfrom (variable)
This variale defines the email address that emails containing agent run output come from.
smtpserver (variable)
This variale defines the smtp server to use when sending agent emails.
acl (variable)
The acl
is crucial. This is used by every host, not just the
policy server. Make sure you only allow hosts you want to allow.
trustkeysfrom (variable)
trustkeysfrom
tells the policy server from which IPs it should accept
connections even if the host's key is unknown, trusting it at connect
time. This is only useful to be open during for bootstrapping these
hosts. As the comments say, empty it after your hosts have been
bootstrapped to avoid unpleasant surprises.
services_autorun (class)
Off by default.
When enabled files in services/autorun
suffixed with .cf
are automatically
added to inputs and bundles that are tagged with autorun
are automatically
actuated (as a policy level feature).
This class needs to be defined from the beginning of policy execution and should be defined using an augments file. Here is an example:
{
"classes":{
"services_autorun":[
"any"
]
}
}
Note: It is not sufficient to define this class within the policy itself in order for files to be discovered and added to inputs appropriately.
To tag a bundle with autorun simply define tags
as a meta
slist with a value
that inclused autorun
. Here's a simple example of such a bundle in
services/autorun/hello.cf
:
bundle agent hello_world_autorun
{
meta:
"tags" slist => { "autorun" };
reports:
verbose_mode::
"$(this.bundle): Hello, this is an automatically loaded bundle";
}
cfengine_internal_rotate_logs (class)
On by default.
Rotates CFEngine's own logs. Here is the cfe_internal_log_rotation
bundle implementation:
bundle agent cfe_internal_log_rotation
{
methods:
cfengine_internal_rotate_logs::
# CFEngine generates internal log files that need to be rotated.
# Have a look at def.cf to enable rotation of these files
"Rotate CFEngine log files"
handle => "cfe_internal_log_rotation_rotate_log_files",
usebundle => logrotate( @(def.cfe_log_files), 1M, 10 ),
comment => "To keep the disk from getting to full we want to rotate
log files when they reach 1MB in size. So that we have
some history , we keep 10 versions.";
"Prune old log files"
handle => "cfe_internal_log_rotation_prune_log_dirs",
usebundle => prunedir( @(def.cfe_log_dirs), 30 ),
comment => "Scheduled activities like agent runs and reports can
create log files or reports that stack up over time. So
that we dont fill the disk, but have some historical
information available locally we purge log files older
than 30 days.";
reports:
DEBUG|DEBUG_cfe_internal_log_rotation::
"DEBUG $(this.bundle): Check CFEngine log file for rotation '$(def.cfe_log_files)'";
"DEBUG $(this.bundle): Check CFEngine log directory for old logs '$(def.cfe_log_dirs)'";
}
cfengine_internal_agent_email (class)
On by default.
This class enables agent email output from cf-execd
.
cfengine_internal_encrypt_transfers (class)
Duplicate of the one in update.cf
. They should be set in unison or
you may get unexpected behavior.
cfengine_internal_purge_policies (class)
Duplicate of the one in update.cf
. They should be set in unison or
you may get unexpected behavior.
cfengine_internal_preserve_permissions (class)
Duplicate of the one in update.cf
. They should be set in unison or
you may get unexpected behavior.
cfengine_internal_sudoers_editing_enable (class)
Off by default. Only used on the CFEngine Enterprise hub.
Turn this on (set to any
) to allow the hub to edit sudoers in order
for the Apache user to run passwordless sudo cf-runagent (part of
Mission Portal troubleshooting).
postgresql_maintenance_supported (class)
On by default only for CFEngine Enterprise Hubs.
This class enables maintaince routines for the database used in CFEngine Enterprise.
postgresql_full_maintenance (class)
On by default only on Sundays at 2am when postgresql_maintenance_supported is defined.
Set this class accordingly if you want to schedule database maintenance operations at a different time.
postgresql_vacuum (class)
On by default at 2am when postgresql_maintenance_supported is defined except for Sundays.
Set this class accordingly if you want to schedule database maintenance operations at a different time.
enable_cfengine_enterprise_hub_ha (class)
Off by default.
Set this class when you want to enable the CFEngine Enterprise HA policies.
enable_cfe_internal_cleanup_agent_reports (class)
Off by default for core. On by default for CFEngine Enteprise clients.
This class enables policy that cleans up report diffs when they exceed
def.maxclient_history_size
.
@(cfengine_enterprise_hub_ha.classification_bundles) (bundle)
Bundles related to classification for CFEngine Enterprise HA.
cfsketch_run (bundle)
This bundle activates sketches deployed by the Design Center tooling.
services_autorun (bundle)
This bundle loads policies found in services/autorun
that are
tagged for autorun.
See services_autorun
@(services_autorun.bundles) (bundle)
This activates bundles found by services_autorun.
cfe_internal_management (bundle)
This bundle activates policy related to CFEngine itself. For example rotation of logs generated by the agent.
main (bundle)
This bundle is defined as bundle agent main
in services/main.cf
, it is the main entry into custom policy.
@(cfengine_enterprise_hub_ha.management_bundles) (bundle)
These bundles activate policy that manages HA in CFEngine Enterprise.
inputs
In order to find bundles, CFEngine needs to know where to look. This
list defines what files are needed. Note there are several dynamic
entries here, coming from other bundles. CFEngine will keep evaluating
the inputs
and bundlesequence
until all the bundles are found and
resolved.
Make sure to add any of your own services
files here if you don't
use the services_autorun
facility, to ensure the bundles in them are
found.
failsafe.cf
The failsafe.cf
file ensures that your system can survive errors and
can upgrade gracefully to new versions even when mistakes are made.
It's literally a failsafe if promises.cf
and update.cf
should
fail.
This file is generated during the bootstrapping process, and should
normally never be changed. The only job of failsafe.cf
is to execute
the update bundle in a “standalone” context should there be a syntax
error somewhere in the main set of promises. In this way, if a client
machine's policies are ever corrupted after downloading erroneous
policy from a server, that client will have a failsafe method for
downloading a corrected policy once it becomes available on the
server. Note that by “corrupted” and “erroneous” we typically mean
“broken via administrator error” - mistakes happen, and the
failsafe.cf
file is CFEngine's way of being prepared for that
eventuality.
If you ever change failsafe.cf
(or when you upgrade CFEngine), make
sure the old and the new CFEngine can successfully parse and execute
this file. If not, you risk losing control of your system (that is, if
CFEngine cannot successfully execute this policy file, it has no
failsafe/fallback mechanism for distributing new policy files).
Some general rules (but again, note you can completely break your
CFEngine installation by editing failsafe.cf
):
- Upgrade the software first, then add new features to the configuration.
- Never use advanced features in the failsafe or update file.
- Avoid using library code (including any bodies from
stdlib.cf
or the files it includes). Copy/paste any bodies you need using a unique name that does not collide with a name in library (we recommend simply adding the prefix “u_”). This may mean that you create duplicate functionality, but that is okay in this case to ensure a 100% functioning standalone update process). The promises which manage the update process should not have any dependencies on any other files.
CFEngine will fail-over to the failsafe.cf
configuration if it is
unable to read or parse the contents successfully. That means that any
syntax errors you introduce (or any new features you utilize in a
configuration) will cause a fail-over, because the parser will not be
able to interpret the policy. If the failover is due to the use of new
features, they will not parse until the software itself has been
updated (so we recommend that you always update CFEngine before
updating policy to use new features). If you accidentally cause a bad
(i.e., unparseable) policy to be distributed to client machines, the
failsafe.cf
policy on those machines will run (and will eventually
download a working policy, once you fix it on the policy host).
Further structure
cfe_internal
: internal CFEngine policies you shouldn't modify or you will get unexpected behaviorcontrols
: configuration of components, e.g. thecf-agent
orcf-serverd
, beyond whatdef.cf
can offer. You'll see3.6
which is for backwards compatibility and used only by3.6
agents.3.7
+ agents will use the library files directly inside of this directory.def.cf
: defaults you can and should configure, see aboveinventory
: inventory modules (loaded before anything else to discover facts about the system) live here; see abovelib
: main library directory. You'll see3.5
and3.6
and3.7
under it. These are the supported versions for masterfiles backwards compatibility.promises.cf
: main policy, you will need to configure this, see aboveservices
: your site's policies go hereservices_autorun
: see abovesketches
: Design Center installations use this; do not touch or you will get unexpected behaviorupdate
andupdate.cf
: functionality for updating inputs and CFEngine itself, see above. You only modify files underupdate
if you know the impact of what you are doing or you may get unexpected behavior.
cf_promises_validated
Several CFEngine components that read policy (e.g. cf-agent
,
cf-execd
, cf-serverd
) run cf-promises
to validate the syntax of
their input files before actually running the policy. To illustrate
this, if cf-promises
runs every 5 minutes then there will be 12
checks occurring every hour, 24 hours a day, 7 days a week -- a total
of 2016 possible validation checks. Each of those individual
validation sessions can take some number of seconds to perform
depending on the system, scale, circumstances and configuration.
Starting with CFEngine 3.1.2, the outcome of every run of
cf-promises
was cached, which lets agents skip the validation of
input files that have not changed since the previous run.
Starting with CFEngine 3.6, outcome on both hosts and hubs is stored
in the file $(sys.workdir)/masterfiles/cf_promises_validated
(usually sys.workdir
is /var/cfengine
). The file can be created by
cf-agent
after it has successfully verified the policy with
cf-promises
. The file can also be created by a user with
cf-promises -T DIRECTORY
which is useful for validating an entire
directory.
When the hash content of any file under WORKDIR/inputs
changes, and
validates to be syntactically correct, then a timestamp in
cf_promises_validated
is updated. If not, the run of cf-promises
is skipped and, at the same time, the cf-execd, cf-serverd and
cf-monitord daemons will not reload the policy unless
cf_promises_validated
has an updated timestamp, which cf-agent
will normally take care of.
In the default installation, the masterfiles are populated automatically on the policy server and you can even auto-deploy them from a version control system.
You should configure the masterfiles as described above. Leaving them at their default settings may expose your masterfiles or worse, especially the cf-serverd ACL settings. If you are not sure of the terms used below or what it all means, come back to this page after you've learned about writing policy and the CFEngine syntax.
CFEngine 3 Inventory Modules
The CFEngine 3 inventory modules are pieces of CFEngine policy that
are loaded and used by the promises.cf
mechanism in order to
inventory the system.
CFEngine Enterprise has specific functionality to show and use inventory data, but users of the Community Version can use them as well locally on each host.
How It Works
The inventory modules are called in promises.cf
:
body common control
{
bundlesequence => {
# Common bundle first (Best Practice)
inventory_control,
@(inventory.bundles),
...
As you see, this calls the inventory_control
bundle, and then each
bundle in the list inventory.bundles
. That list is built in the
top-level common inventory
bundle, which will load the right things
for some common cases. The any.cf
inventory module is always loaded;
the rest are loaded if they are appropriate for the platform. For
instance, Debian systems will load debian.cf
and linux.cf
and
lsb.cf
but may load others as needed.
The effect for users is that the right inventory modules will be loaded and evaluated.
The inventory_control
bundle lives in def.cf
and defines what
inventory modules should be disabled. You can simply set
disable_inventory
to avoid the whole system, or you can look for the
disable_inventory_xyz
class to disable module xyz
.
Any inventory module works the same way, by doing some discovery work
and then tagging its classes and variables with the report
or
inventory
tags. For example:
vars:
"ports" slist => { @(mon.listening_ports) },
meta => { "inventory", "attribute_name=Ports listening" };
This defines a reported attribute "Ports listening" which contains a list of strings representing the listening ports. More on this in a second.
Your Very Own Inventory Module
The good news is, writing an inventory module is incredibly easy.
They are just CFEngine bundles. You can see a simple one that collects
the listening ports in any.cf
:
bundle agent cfe_autorun_inventory_listening_ports
# @brief Inventory the listening ports
#
# This bundle uses `mon.listening_ports` and is always enabled by
# default, as it runs instantly and has no side effects.
{
vars:
"ports" slist => { @(mon.listening_ports) },
meta => { "inventory", "attribute_name=Ports listening" };
}
Well, the slist copy is a CFEngine detail (we get the listening ports
from the monitoring daemon), so just assume that the data is correct.
What's important is the second line that starts with
meta
. That
defines metadata for the promise that CFEngine will use to determine
that this data is indeed inventory data and should be reported to the
CFEngine Enterprise Hub.
That's it. Really. The comments are optional but nice to have. You
don't have to put your new bundle in a file under the inventory
directory, either. The variables and classes can be declared anywhere
as long as they have the right tags. So you can use the services
directory or whatever else makes sense to you.
CFEngine Enterprise vs. Community
In CFEngine Enterprise, the reported data is aggregated in the hub and reported across the whole host population.
In CFEngine Community, users can use the classesmatching()
and
variablesmatching()
functions to collect all the inventory variables
and classes and report them in other ways.
Implementation Best Practice for CFEngine Enterprise
It is important that inventory variables and classes are continually defined. Only inventory variables and classes defined during the last reported run are available for use by the inventory reporting interface.
Inventory items that change frequently can create a burden on the Enterprise reporting infrastructure. Generally, inventory attributes should change infrequently.
If you wish to inventory attributes that frequently change or are expensive to discover consider implementing a sample interval and caching mechanism.
What Modules Are Available?
As soon as you use the promises.cf
provided in the parent directory,
quite a few inventory modules will be enabled (if appropriate for your
system). Here's the list of modules and what they provide. Note they
are all enabled by code in def.cf
as explained above.
LSB
- lives in:
lsb.cf
- applies to: LSB systems (most Linux distributions, basically)
- runs:
lsb_release -a
- sample data:
Distributor ID: Ubuntu
Description: Ubuntu 14.04 LTS
Release: 14.04
Codename: trusty
- provides:
- classes
lsb_$(os)
,lsb_$(os)_$(release)
,lsb_$(os)_$(codename)
- variables:
inventory_lsb.os
(Distributor ID),inventory_lsb.codename
,inventory_lsb.release
,inventory_lsb.flavor
,inventory_lsb.description
- classes
- implementation:
bundle agent inventory_lsb
{
classes:
"have_lsb" expression => fileexists($(lsb_exec));
"_inventory_lsb_found" expression => regcmp("^[1-9][0-9]*$", $(dim)),
scope => "namespace";
_inventory_lsb_found::
"lsb_$(os)" expression => "any",
comment => "LSB Distributor ID",
depends_on => { "inventory_lsb_os" },
scope => "namespace",
meta => { "inventory", "attribute_name=none" };
"lsb_$(os)_$(release)" expression => "any",
comment => "LSB Distributor ID and Release",
depends_on => { "inventory_lsb_os", "inventory_lsb_release" },
scope => "namespace",
meta => { "inventory", "attribute_name=none" };
"lsb_$(os)_$(codename)" expression => "any",
comment => "LSB Distributor ID and Codename",
depends_on => { "inventory_lsb_os", "inventory_lsb_codename" },
scope => "namespace",
meta => { "inventory", "attribute_name=none" };
vars:
"lsb_exec" string => "$(inventory_control.lsb_exec)";
have_lsb::
"data" string => execresult("$(lsb_exec) -a", "noshell");
"dim" int => parsestringarray(
"lsb",
$(data),
"\s*#[^\n]*",
"\s*:\s+",
"15",
"4095"
);
_inventory_lsb_found::
"lsb_keys" slist => getindices("lsb");
"os" string => canonify("$(lsb[Distributor ID][1])"),
handle => "inventory_lsb_os",
comment => "LSB-provided OS name",
meta => { "inventory", "attribute_name=none" };
"codename" string => canonify("$(lsb[Codename][1])"),
handle => "inventory_lsb_codename",
comment => "LSB-provided OS code name",
meta => { "inventory", "attribute_name=none" };
"release" string => "$(lsb[Release][1])",
handle => "inventory_lsb_release",
comment => "LSB-provided OS release",
meta => { "inventory", "attribute_name=none" };
"flavor" string => canonify("$(lsb[Distributor ID][1])_$(lsb[Release][1])"),
handle => "inventory_lsb_flavor",
comment => "LSB-provided OS flavor",
meta => { "inventory", "attribute_name=none" };
"description" string => "$(lsb[Description][1])",
handle => "inventory_lsb_description",
comment => "LSB-provided OS description",
meta => { "inventory", "attribute_name=none" };
reports:
(DEBUG|DEBUG_inventory_lsb)._inventory_lsb_found::
"DEBUG $(this.bundle): OS = $(os), codename = $(codename), release = $(release), flavor = $(flavor), description = $(description)";
"DEBUG $(this.bundle): got $(dim) LSB keys";
"DEBUG $(this.bundle): prepared LSB key $(lsb_keys) = '$(lsb[$(lsb_keys)][1])'";
(DEBUG|DEBUG_inventory_lsb).!_inventory_lsb_found::
"DEBUG $(this.bundle): LSB inventory not found";
}
- sample output:
% cf-agent -KI -binventory_control,inventory_lsb
R: inventory_lsb: OS = Ubuntu, codename = trusty, release = 14.04, flavor = Ubuntu_14_04, description = Ubuntu 14.04 LTS
SUSE
- lives in:
suse.cf
- applies to: SUSE Linux
- provides classes:
suse_pure
andsuse_derived
- implementation:
bundle common inventory_suse
{
classes:
"suse_pure" expression => "(sles|sled).!opensuse",
comment => "pure SUSE",
meta => { "inventory", "attribute_name=none" };
"suse_derived" expression => "opensuse.!suse_pure",
comment => "derived from SUSE",
meta => { "inventory", "attribute_name=none" };
}
Debian
- lives in:
debian.cf
- applies to: Debian and its derivatives
- provides:
- variables:
inventory_debian.mint_release
andinventory_debian.mint_codename
- classes:
debian_pure
,debian_derived
,linuxmint
,lmde
,linuxmint_$(mint_release)
,linuxmint_$(mint_codename)
,$(mint_codename)
- variables:
- implementation:
bundle common inventory_debian
{
vars:
has_lsb_release::
"lsb_release_info" string => readfile("/etc/lsb-release","256"),
comment => "Read more OS info" ;
has_etc_linuxmint_info::
"linuxmint_info" string => readfile("/etc/linuxmint/info","1024"),
comment => "Read Linux Mint specific info" ;
"lm_info_count"
int => parsestringarray("mint_info", # array to populate
"$(linuxmint_info)", # data to parse
"\s*#[^\n]*", # comments
"=", # split
100, # maxentries
2048) ; # maxbytes
"mint_release" string => "$(mint_info[RELEASE][1])" ;
"mint_codename" string => "$(mint_info[CODENAME][1])" ;
classes:
any::
"debian_derived_evaluated"
scope => "bundle",
or => { "has_os_release", "has_lsb_release", "has_etc_linuxmint_info" } ;
"linuxmint"
expression => "has_etc_linuxmint_info",
comment => "this is a Linux Mint system, of some sort",
meta => { "inventory", "attribute_name=none" } ;
has_lsb_release::
"linuxmint"
expression => regcmp("(?ms).*^DISTRIB_ID=LinuxMint$.*", "$(lsb_release_info)"),
comment => "this is a Linux Mint system, of some sort",
meta => { "inventory", "attribute_name=none" } ;
linuxmint.has_os_release::
"lmde"
expression => regcmp('(?ms).*^NAME="Linux Mint LMDE"$.*', "$(inventory_linux.os_release_info)"),
comment => "this is a Linux Mint Debian Edition",
meta => { "inventory", "attribute_name=none", "derived-from=inventory_linux.os_release_info" } ;
linuxmint.has_lsb_release::
"lmde"
expression => regcmp('(?ms).*^DISTRIB_DESCRIPTION="LMDE.*', "$(lsb_release_info)"),
comment => "this is a Linux Mint Debian Edition",
meta => { "inventory", "attribute_name=none", "derived-from=inventory_debian.lsb_release_info" } ;
has_etc_linuxmint_info::
"lmde"
expression => regcmp('(?ms).*^DESCRIPTION="LMDE.*',"$(linuxmint_info)"),
comment => "this is a Linux Mint Debian Edition",
meta => { "inventory", "attribute_name=none", "derived-from=inventory_debian.linuxmint_info" } ;
debian_derived_evaluated.has_etc_linuxmint_info.!lmde::
# These need to be evaluated only after debian_derived_evaluated is defined
# to ensure that the mint_info array has been evaluated as well.
# Failing to do that will create meaningless classes
# On non-LMDE Mint systems, this will create classes like, e.g.:
# linuxmint_14, nadia, linuxmint_nadia
"linuxmint_$(mint_release)" expression => "any",
meta => { "inventory", "attribute_name=none" } ;
"$(mint_codename)" expression => "any",
meta => { "inventory", "attribute_name=none" } ;
"linuxmint_$(mint_codename)" expression => "any",
meta => { "inventory", "attribute_name=none" } ;
debian_derived_evaluated::
"debian_pure" expression => "debian.!(ubuntu|linuxmint)",
comment => "pure Debian",
meta => { "inventory", "attribute_name=none" };
"debian_derived" expression => "debian.!debian_pure",
comment => "derived from Debian",
meta => { "inventory", "attribute_name=none" };
any::
"has_lsb_release" expression => fileexists("/etc/lsb-release"),
comment => "Check if we can get more info from /etc/lsb-release";
"has_etc_linuxmint_info" expression => fileexists("/etc/linuxmint/info"),
comment => "If this is a Linux Mint system, this *could* be available";
}
Red Hat
- lives in:
redhat.cf
- applies to: Red Hat and its derivatives
- provides classes:
redhat_pure
,redhat_derived
- implementation:
bundle common inventory_redhat
{
classes:
"redhat_pure" expression => "redhat.!centos.!oracle",
comment => "pure Red Hat",
meta => { "inventory", "attribute_name=none" };
"redhat_derived" expression => "redhat.!redhat_pure",
comment => "derived from Red Hat",
meta => { "inventory", "attribute_name=none" };
}
Windows
- lives in:
windows.cf
Mac OS X
- lives in:
macos.cf
Generic (unknown OS)
- lives in:
generic.cf
(seeany.cf
for generally applicable inventory modules)
LLDP
- lives in:
any.cf
- runs
inventory_control.lldpctl_exec
through a Perl filter - provides variables:
cfe_autorun_inventory_LLDP.K
for eachK
returned by the LLDB executable - implementation:
bundle agent cfe_autorun_inventory_LLDP
{
classes:
"lldpctl_exec_exists" expression => fileexists($(inventory_control.lldpctl_exec));
vars:
!disable_inventory_LLDP.lldpctl_exec_exists::
"info" data => parsejson(execresult($(inventory_control.lldpctl_json), "noshell"));
}
mtab
- lives in:
any.cf
- parses:
/etc/mtab
- provides classes:
have_mount_FSTYPE
andhave_mount_FSTYPE_MOUNTPOINT
- implementation:
bundle agent cfe_autorun_inventory_mtab
{
vars:
have_mtab::
"mount_count" int => readstringarrayidx("mounts",
$(inventory_control.mtab),
"\s*#[^\n]*",
"\s+",
500,
50000);
"idx" slist => getindices("mounts");
classes:
"have_mtab" expression => fileexists($(inventory_control.mtab));
# define classes like have_mount_ext4__var for a ext4 /var mount
"have_mount_$(mounts[$(idx)][2])_$(mounts[$(idx)][1])"
expression => "any",
scope => "namespace";
# define classes like have_mount_ext4 if there is a ext4 mount
"have_mount_$(mounts[$(idx)][2])"
expression => "any",
scope => "namespace";
reports:
verbose_mode::
"$(this.bundle): we have a $(mounts[$(idx)][2]) mount under $(mounts[$(idx)][1])";
}
- sample output (note this is verbose mode with
-v
because there's a lot of output):
% cf-agent -Kv -binventory_control,cfe_autorun_inventory_mtab|grep 'cfe_autorun_inventory_mtab: we have'
R: cfe_autorun_inventory_mtab: we have a ext4 mount under /
...
R: cfe_autorun_inventory_mtab: we have a cgroup mount under /sys/fs/cgroup/systemd
R: cfe_autorun_inventory_mtab: we have a tmpfs mount under /run/shm
fstab
- lives in:
any.cf
- parses:
sys.fstab
- provides classes:
have_fs_FSTYPE
have_fs_MOUNTPOINT
andhave_fs_FSTYPE_MOUNTPOINT
- implementation:
bundle agent cfe_autorun_inventory_fstab
{
vars:
have_fstab::
"mount_count" int => readstringarrayidx("mounts",
$(sys.fstab),
"\s*#[^\n]*",
"\s+",
500,
50000);
"idx" slist => getindices("mounts");
classes:
"have_fstab" expression => fileexists($(sys.fstab));
# define classes like have_fs_ext4__var for a ext4 /var entry
"have_fs_$(mounts[$(idx)][2])_$(mounts[$(idx)][1])"
expression => "any",
scope => "namespace";
# define classes like have__var for a /var entry
"have_fs_$(mounts[$(idx)][1])"
expression => "any",
scope => "namespace";
# define classes like have_fs_ext4 if there is a ext4 entry
"have_fs_$(mounts[$(idx)][2])"
expression => "any",
scope => "namespace";
reports:
verbose_mode::
"$(this.bundle): we have a $(mounts[$(idx)][2]) fstab entry under $(mounts[$(idx)][1])";
}
- sample output (note this is verbose mode with
-v
because there's a LOT of output):
% cf-agent -Kv -binventory_control,cfe_autorun_inventory_fstab|grep 'cfe_autorun_inventory_fstab: we have'
R: cfe_autorun_inventory_fstab: we have a ext4 fstab entry under /
R: cfe_autorun_inventory_fstab: we have a cifs fstab entry under /backups/load
R: cfe_autorun_inventory_fstab: we have a auto fstab entry under /mnt/cdrom
CMDB
- lives in:
any.cf
- parses:
me.json
(which is copied from the policy server; see implementation) - provides classes:
CLASS
for each CLASS found under theclasses
key in the JSON data - provides variables:
inventory_cmdb_load.VARNAME
for each VARNAME found under thevars
key in the JSON data - implementation:
bundle agent inventory_cmdb_load(file)
{
classes:
"have_cmdb_data" expression => isvariable("cmdb");
"$(ckeys)" expression => "any", scope => "namespace";
vars:
"cmdb" data => readjson($(file), "999999");
"cmdb_string" string => format("%S", cmdb);
"bkeys" slist => getindices("cmdb[vars]");
"vkeys_$(bkeys)" slist => getindices("cmdb[vars][$(bkeys)]");
"$(vkeys_$(bkeys))" string => nth("cmdb[vars][$(bkeys)]", $(vkeys));
"ckeys" slist => getindices("cmdb[classes]");
reports:
DEBUG|DEBUG_inventory_cmdb_load::
"DEBUG $(this.bundle): Got CMDB data from $(file): $(cmdb_string)"
ifvarclass => "have_cmdb_data";
"DEBUG $(this.bundle): Got CMDB key = $(vkeys_$(bkeys)), CMDB value = $((vkeys_$(bkeys)))"
ifvarclass => "have_cmdb_data";
"DEBUG $(this.bundle): Got CMDB class = $(ckeys)"
ifvarclass => "have_cmdb_data";
"DEBUG $(this.bundle): Could not read the CMDB data from $(file)"
ifvarclass => "!have_cmdb_data";
}
DMI decoding
- lives in:
any.cf
- runs:
dmidecode
- provides variables:
cfe_autorun_inventory_dmidecode.dmi[K]
for each key K in thedmidecode
output - implementation:
bundle agent cfe_autorun_inventory_dmidecode
{
vars:
"dmidefs" data => parsejson('
{
"bios-vendor": "BIOS vendor",
"bios-version": "BIOS version",
"system-serial-number": "System serial number",
"system-manufacturer": "System manufacturer",
"system-version": "System version",
"system-product-name": "System product name",
}');
- sample output (sudo is needed to access the DMI):
% sudo /var/cfengine/bin/cf-agent -KI -binventory_control,cfe_autorun_inventory_dmidecode
R: cfe_autorun_inventory_dmidecode: Obtained BIOS vendor = 'Intel Corp.'
R: cfe_autorun_inventory_dmidecode: Obtained BIOS version = 'BLH6710H.86A.0146.2013.1555.1888'
R: cfe_autorun_inventory_dmidecode: Obtained System serial number = ''
R: cfe_autorun_inventory_dmidecode: Obtained System manufacturer = ''
R: cfe_autorun_inventory_dmidecode: Obtained System version = ''
R: cfe_autorun_inventory_dmidecode: Obtained CPU model = 'Intel(R) Core(TM) i7-2600 CPU @ 3.40GHz'
Listening ports
- lives in:
any.cf
- provides variables:
cfe_autorun_inventory_listening_ports.ports
as a copy ofmon.listening_ports
- implementation:
bundle agent cfe_autorun_inventory_listening_ports
{
vars:
"ports" -> { "ENT-150" }
slist => sort( "mon.listening_ports", "int"),
meta => { "inventory", "attribute_name=Ports listening" },
ifvarclass => some("[0-9]+", "mon.listening_ports"),
comment => "We only want to inventory the listening ports if we have
values that make sense.";
}
Disk space
- lives in:
any.cf
- provides variables:
cfe_autorun_inventory_disk.free
as a copy ofmon.value_diskfree
- implementation:
bundle agent cfe_autorun_inventory_disk
{
vars:
enterprise::
"free" string => "$(mon.value_diskfree)",
meta => { "inventory", "attribute_name=Disk free (%)" };
}
Available memory
- lives in:
any.cf
- provides variables:
cfe_autorun_inventory_memory.free
as a copy ofmon.value_mem_free
andcfe_autorun_inventory_memory.total
as a copy ofmon.value_mem_total
- implementation:
bundle agent cfe_autorun_inventory_memory
{
vars:
enterprise::
# due to a Windows issue this is set to 0 there for now
"total" string => ifelse("windows", "0",
$(mon.value_mem_total)),
meta => { "inventory", "attribute_name=Memory size (MB)" };
"free" string => ifelse("windows", "0",
$(mon.value_mem_free)),
meta => { "report" };
}
Load average
- lives in:
any.cf
- provides variables:
cfe_autorun_inventory_loadaverage.value
as a copy ofmon.value_loadavg
- implementation:
bundle agent cfe_autorun_inventory_loadaverage
{
vars:
enterprise::
"value" string => "$(mon.value_loadavg)",
meta => { "report" };
}
procfs
- lives in:
any.cf
- parses:
consoles
,cpuinfo
,modules
,partitions
,version
- provides variables:
cfe_autorun_inventory_proc.console_count
,cfe_autorun_inventory_proc.cpuinfo[K]
for each CPU info key,cfe_autorun_inventory_proc.paritions[K]
for each partition key - provides classes:
_have_console_CONSOLENAME
,have_module_MODULENAME
- implementation:
bundle agent cfe_autorun_inventory_proc
{
vars:
"basefiles" slist => { "consoles", "cpuinfo", "modules", "partitions", "version" };
"files[$(basefiles)]" string => "$(inventory_control.proc)/$(basefiles)";
_have_proc_consoles::
"console_count" int => readstringarrayidx("consoles",
"$(files[consoles])",
"\s*#[^\n]*",
"\s+",
500,
50000);
"console_idx" slist => getindices("consoles");
_have_proc_modules::
"module_count" int => readstringarrayidx("modules",
"$(files[modules])",
"\s*#[^\n]*",
"\s+",
2500,
250000);
"module_idx" slist => getindices("modules");
_have_proc_cpuinfo::
# this will extract all the keys in one bunch, so you won't get
# detailed info for processor 0 for example
"cpuinfo_count" int => readstringarrayidx("cpuinfo_array",
"$(files[cpuinfo])",
"\s*#[^\n]*",
"\s*:\s*",
500,
50000);
"cpuinfo_idx" slist => getindices("cpuinfo_array");
"cpuinfo[$(cpuinfo_array[$(cpuinfo_idx)][0])]" string => "$(cpuinfo_array[$(cpuinfo_idx)][1])";
"cpuinfo_keys" slist => getindices("cpuinfo");
_have_proc_partitions::
"partitions_count" int => readstringarrayidx("partitions_array",
"$(files[partitions])",
"major[^\n]*",
"\s+",
500,
50000);
"partitions_idx" slist => getindices("partitions_array");
"partitions[$(partitions_array[$(partitions_idx)][4])]" string => "$(partitions_array[$(partitions_idx)][3])";
"partitions_keys" slist => getindices("partitions");
_have_proc_version::
"version" string => readfile("$(files[version])", 2048);
classes:
"have_proc" expression => isdir($(inventory_control.proc));
have_proc::
"_have_proc_$(basefiles)"
expression => fileexists("$(files[$(basefiles)])");
_have_proc_consoles::
"have_console_$(consoles[$(console_idx)][0])"
expression => "any",
scope => "namespace";
_have_proc_modules::
"have_module_$(modules[$(module_idx)][0])"
expression => "any",
scope => "namespace";
reports:
_have_proc_consoles.verbose_mode::
"$(this.bundle): we have console $(consoles[$(console_idx)][0])";
_have_proc_modules.verbose_mode::
"$(this.bundle): we have module $(modules[$(module_idx)][0])";
_have_proc_cpuinfo.verbose_mode::
"$(this.bundle): we have cpuinfo $(cpuinfo_keys) = $(cpuinfo[$(cpuinfo_keys)])";
_have_proc_partitions.verbose_mode::
"$(this.bundle): we have partitions $(partitions_keys) with $(partitions[$(partitions_keys)]) blocks";
_have_proc_version.verbose_mode::
"$(this.bundle): we have kernel version '$(version)'";
}
- sample output (note this is verbose mode with
-v
because there's a LOT of output):
% cf-agent -Kv -binventory_control,cfe_autorun_inventory_proc|grep 'cfe_autorun_inventory_proc: we have'
R: cfe_autorun_inventory_proc: we have console tty0
R: cfe_autorun_inventory_proc: we have module snd_seq_midi
...
R: cfe_autorun_inventory_proc: we have module ghash_clmulni_intel
R: cfe_autorun_inventory_proc: we have cpuinfo flags = fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx lahf_lm ida arat epb xsaveopt pln pts dtherm tpr_shadow vnmi flexpriority ept vpid
...
R: cfe_autorun_inventory_proc: we have cpuinfo model name = Intel(R) Core(TM) i7-2600 CPU @ 3.40GHz
R: cfe_autorun_inventory_proc: we have partitions sr0 with 1048575 blocks
...
R: cfe_autorun_inventory_proc: we have partitions sda with 468851544 blocks
R: cfe_autorun_inventory_proc: we have kernel version 'Linux version 3.11.0-15-generic (buildd@roseapple) (gcc version 4.8.1 (Ubuntu/Linaro 4.8.1-10ubuntu8) ) #25-Ubuntu SMP Thu Jan 30 17:22:01 UTC 2014'
Testing Policies
One of the practical advantages of CFEngine is that you can test it without the need for root or administrator privileges. This is useful if you are concerned about manipulating important system files, but naturally limits the possibilities for what CFEngine is able to do.
CFEngine operates with the notion of a work-directory. The default work
directory for the root
user is /var/cfengine
. For any other user, the work
directory lies in the user's home directory, named ~/.cfagent
.
CFEngine prefers you to keep certain files here. You should not resist this too strongly or you will make unnecessary trouble for yourself. The decision to have this 'known directory' was made to simplify a lot of configuration.
To test CFEngine as an ordinary user, do the following:
Copy the binaries into the work directory:
host$ mkdir -p ~/.cfagent/inputs
host$ mkdir -p ~/.cfagent/bin
host$ cp /var/cfengine/bin/cf-* ~/.cfagent/bin
host$ cp /var/cfengine/inputs/*.cf ~/.cfagent/inputs
You can test the software and play with configuration files by editing the
basic directly in the ~/.cfagent/inputs
directory. For example, try the
following:
host$ ~/.cfagent/bin/cf-promises
host$ ~/.cfagent/bin/cf-promises --verbose
This is always the way to start checking a configuration in CFEngine 3. If a
configuration does not pass this check/test, you will not be allowed to use
it, and cf-agent
will look for the file failsafe.cf
.
Controlling Frequency
When checking a series of expensive functions and verifying complex promises, you may want to make sure that CFEngine is not checking too frequently. One way of doing this is classes and class expression, another is using locks.
CFEngine incorporates a series of locks which prevent it from checking promises too often, and which prevent it from spending too long trying to check promises it has recently verified. This locking mechanism works in such a way that you can start several CFEngine components simultaneously without them interfering with each other. You can control two things about each kind of action in CFEngine:
ifelapsed
The minimum time (in minutes) which should have passed since the last time that promise was verified. It will not be executed again until this amount of time has elapsed. Default time is 1 minute.
expireafter
The maximum amount (in minutes) of time cf-agent
should wait for an old
instantiation to finish before killing it and starting again. You can think
about expireafter
as a timeout to use when a promise verification may
involve an operation that could wait indefinitely. Default time is 120
minutes.
You can set these values either globally (for all actions) or for each action
separately. If you set global and local values, the local values override the
global ones. All times are written in units of minutes. The following global
setting is defined in body agent control
.
body agent control
{
ifelapsed => "60"; # one hour
}
This setting tells CFEngine not to verify promises until 60 minutes have
elapsed, ie ensures that the global frequency for all promise verification is
one hour. This global setting of one hour could be changed for a specific
promise body by setting ifelapsed
in the promise body.
body action example
{
ifelapsed => "90"; # 1.5 hours
}
This promise which overrides the global 60 minute time period and defines a frequency of 90 minutes.
These locks do not prevent the whole of cf-agent
from running, only
atomic promise checks on the same objects (packages, users, files,
etc.). Several different cf-agent
instances can run concurrently.
The locks ensure that promises will not be verified by two cf-agents
at the same time or too soon after a verification.
Design Center Overview
The Design Center is a public repository for data-driven policy templates called sketches that is deeply integrated with CFEngine Enterprise.
Sketches are ready-to-use components (e.g. collections of bundles, support files, etc.) that can be directly imported and used in CFEngine policies. Most sketches are specialized for achieving specific tasks, or for maintaining a specific piece of software, but their scope and capabilities can range beyond. They are organized in categories and tagged according to their functionality.
Sketches are managed (installed, configured, enabled and uninstalled)
through a specialized tool called cf-sketch
which in turn talks to a
specialized Design Center API. That API is available for third-party
integrations.
In CFEngine Enterprise, the Design Center interactions are driven by the Mission Portal Design Center App for great ease of use.
This is a guide to the Design Center functionality. For the reference pages for its API and code structure, start with Design Center.
- Use Design Center Sketches to Deploy Policy
- The Design Center Sketch Workflow
- Value of Sketches to Users
- Design Center Terminology
- See Also
Use Design Center Sketches to Deploy Policy
The CFEngine Design Center is a collection of data-driven policy templates called sketches. Sketches can be configured and deployed ("activated") without prior knowledge of the CFEngine language.
The Design Center also contains tools that help you to manipulate and manage sketches:
Enterprise users manage and deploy sketches in the Mission Portal Design Center App. Deploy your first policy in the Design Center.
Community users manage and deploy sketches on the command line by using the
cf-sketch
tool.third-part integrators can use the Design Center API from the command line or through the CFEngine Enterprise API. For more information, see The Design Center API. <!-- (??? LINK TO Enterprise DC API depends on https://dev.cfengine.com/issues/6011 ???) -->
The Design Center Sketch Workflow
A sketch is installed, configured, and deployed (the whole process is called an "activation"), as shown in the diagram below:
Install a sketch from a master repository. At this point, the sketch is merely a template that cannot do anything because it doesn’t contain parameters. With CFEngine Enterprise, installations are handled invisibly.
Configure the sketch by providing parameters. Now, you can create sketch configurations. One sketch can have multiple configurations with different parameter sets that will get applied under different conditions. With CFEngine Enterprise, you do this from a GUI screen.
Deploy the runfile. This makes the sketch visible to CFEngine, so it can begin to execute. Typically the intermediate stage here is to generate a runfile (the policy glue that enables all the Design Center magic) and check it into your version control repository. With CFEngine Enterprise, this step is handled invisibly, you just enter a commit message for the change you've made.
Value of Sketches to Users
Sketches provide value to CFEngine users, especially Enterprise customers:
Instead of writing a single policy for each desired state, you can use data-driven policies (sketches) to implement, activate, or enforce policy. Users can configure the sketch for the appropriate environment and then deploy it to specific hosts or groups of hosts without knowing the intricacies of CFEngine policy language.
You can have infinitely flexible configurations of sketches across your infrastructure based on static (such as CPU or OS) or dynamic (such as time of day or number of users) conditions (classes). Every time CFEngine runs, it chooses which sketch(es) to execute with which parameter sets based on those conditions. As conditions change, the execution of sketches can change, just as the behavior of regular CFEngine policy can change.
There's a large list of sketches you can use already and the list is growing. You can use them as they are, or learn from their techniques and write your own.
Design Center Terminology
Design Center
Refers to the collection of sketches and the tools that allow you to manipulate and manage them.Design Center app (UI)
Refers to the Design Center user interface app that is located on the Mission Portal console for CFEngine Enterprise users.Design Center in GitHub
Refers to the CFEngine github repository of sketches, tools, and policy examples.Design Center API
Refers to the API which performs all operations related to sketches, parameter sets, environments, validations, and deployment.
See Also
- Deploy your first Policy (Enterprise) This Enterprise-specific tutorial illustrates how to use a sketch to configure and deploy a simple policy by using the Design Center App on the Mission Portal console.
- Configure Sketches on the Command Line (Community) This section is for Community users who plan to install, configure, and deploy sketches.
- Write a new Sketch (Enterprise and Community) This is an advanced section for both Enterprise and Community users who plan to write new sketches.
Command Line Sketches
The CFEngine Design Center is a repository of pre-made components
called sketches that allow you to use the full power of CFEngine
without having to learn the CFEngine policy language. Although
sketches are themselves written in the CFEngine policy language, you
can make use of them by simply installing them, configuring them using
the appropriate parameters, and deploying them on your
infrastructure. For users of the Community edition of CFEngine, this can be easily done
using both a command-line interface called cf-sketch
.
Please note that CFEngine Enterprise comes with the Design Center App to make the work below effortless.
Overview
This page provides instructions on how a Design Center sketch can be found, installed, configured, and executed as policy, using command-line tools with CFEngine community. The overview is as follows:
Before you Begin
Instructions
Step 1. Check out the Design Center repository
Step 5. Activate a sketch Define the parameter set and environment, and run the activate command.
Step 6. Deploy the sketch Generate and execute the runfile.
Additional resources are included at the end of this page.
Before you Begin
Complete the software requirements and review the basic concepts of sketches before you begin the instructions:
Requirements
To follow these instructions, you will need the following:
- A Unix-like system
- The following programs installed (all of these are either included or easily available in the package repositories for most operating system):
- CFEngine Community 3.5.0 or newer, or CFEngine Enterprise 3.5.0 or newer. Many of the components will work with CFEngine Community 3.4.0 or later (Enterprise 3.0.0 or later), but you need 3.5.0 to have access to the full range of features and sketches.
- Optional but recommended: the
Term::ReadLine::Gnu
Perl module. On many systems it is available in the standard package repositories (for example, on Ubuntu Linux you can install it usingapt-get install libterm-readline-gnu-perl
). You can also install it using thecpan
utility included with Perl.
Basic Concepts
The following concepts provide a better understanding of how the Design Center works. The Design Center framework contains the following:
- Sketches contain the code that is executed by CFEngine to
perform some task. Sketches are contributed by the CFEngine
community and hosted in the Design Center repositories. Most
sketches take a few parameters to configure their precise
behavior. For example, the
System::tzconfig
sketch contains the code to update the appropriate files in the system to set the correct timezone. - Parameter sets contain the parameters that tell a CFEngine sketch
the details of what to do. For example, the
System::tzconfig
sketch takes a parameter that tells it which timezone should be set in the system. The Design Center framework can store several parameter sets (each one identified by a name) for the same sketch, and from which you can choose to apply according to arbitrary circumstances. - Environments contain conditions, expressed as CFEngine Class Expressions, that indicate when and where a particular sketch will be executed with a particular set of parameters. Environments also contain expressions that determine when and where test and verbose modes will be enabled for a sketch. The Design Center can also store multiple named environment definitions, which you can combine in arbitrary ways to fine tune the execution of each sketch on each machine.
Sketches, Parameters and Environments, by themselves, do nothing. They have to be combined into Activations:
An activation is a combination of a sketch, a parameter set, and an
environment (all three specified by name) that defines which sketches
must be executed under which conditions, and with which parameters. It is
only through activations that you put sketches to work and make the
decisions as to how different parts of your infrastructure will be
configured. For example, you can have two different parameter sets
for the System::tzconfig
sketch, one for Linux and another one for
Solaris machines. You can then, on the central CFEngine Policy
Server, create two activations:
- Activation #1: Sketch
System::tzconfig
, parameter settzconfig-linux
, Environmentlinux
- Activation #2: Sketch
System::tzconfig
, parameter settzconfig-solaris
, Environmentsolaris
When these activations are distributed by the policy server to all the
clients, only Linux and Solaris machines will execute the
System::tzconfig
sketch, and each one of them will apply the
appropriate parameters.
Other concepts exist that can make your use of the Design Center even more powerful, but these are enough to get you started and to be able to follow these instructions.
Instructions
Step 1. Check out the Design Center repository
The Design Center is an open source project and is hosted on GitHub. Access the repository at https://github.com/cfengine/design-center.
The best way to get the Design Center at the moment is to check out
its git repository. For these instructions, we check it out
under the $HOME/source/
directory (you can use any location you
want, just replace it throughout the following instructions):
mkdir $HOME/source
cd $HOME/source
git clone https://github.com/cfengine/design-center.git
A directory called design-center
is created. We
call this the CHECKOUT directory and henceforth refer to it as
$CHECKOUT
. In the examples that follow, CHECKOUT is
$HOME/source/design-center
. You can save some typing by typing
export CHECKOUT=$HOME/source/design-center
at the prompt. From that point on, all command-line interaction can
use $CHECKOUT
and it will expand to the installation directory.
Step 2. Run cf-sketch
You can run cf-sketch in interactive mode directly from its directory
under $CHECKOUT
:
cd $CHECKOUT/tools/cf-sketch
./cf-sketch.pl
Welcome to cf-sketch version 3.5.0b1.
CFEngine AS, 2013.
Enter any command to cf-sketch, use 'help' for help, or 'quit' or '^D' to quit.
cf-sketch> _
By default, cf-sketch
provides an interactive prompt where you can
type the commands you want to execute. Type help
at the prompt
to see the descriptions of all the available commands. You can run
commands non-interactively by passing them as arguments to the
cf-sketch.pl
script from the command line.
Step 3. Search for sketches
The first step is to find some sketches to install on your system. The search command provides, without any parameters, a list of all available sketches:
cf-sketch> search
The following sketches are available:
Applications::Memcached Sketch for installing, configuring, and starting memcached.
...
Yale::stdlib Yale standard library
This is useful for exploration, but might be too much information. You can also provide a regular expression to search for a particular set of sketches:
cf-sketch> search system
The following sketches match your query:
System::Logrotate Sets defaults and user permissions in the sudoers file
System::Routes Sets defaults and user permissions in the sudoers file
System::Sudoers Sets defaults and user permissions in the sudoers file
System::Syslog Configures syslog
System::access Manage access.conf values
System::config_resolver Configure DNS resolver
System::cron Manage crontab and /etc/cron.d contents
System::etc_hosts Manage /etc/hosts
System::motd Configure the Message of the Day
System::set_hostname Set system hostname. Domain name is also set on Mac, Red Hat and and Gentoo derived distributions (but not Debian).
System::sysctl Manage sysctl values
System::tzconfig Manage system timezone configuration
Use the info
command to get additional information for a sketch, including details
about its parameters:
cf-sketch> info -v System::motd
The following sketches match your query:
Sketch System::motd
Description: Configure the Message of the Day
Authors: Ben Heilman <bheilman@enova.com>
Version: 1.00
License: MIT
Tags: cfdc
Installed: No
Parameters:
For bundle entry
motd: string
motd_path: string
prepend_command: string
dynamic_path: string
symlink_path: string
Step 4. Install a sketch
The first step in using a sketch is to install it. As an example, install
the System::motd
sketch:
cf-sketch> install System::motd
Sketch System::motd installed under /home/vagrant/.cfagent/inputs/sketches.
Sketch README.md installed under System::motd.
Sketch main.cf installed under System::motd.
Sketch params/debian_squeeze.json installed under System::motd.
Sketch params/debian_wheezy.json installed under System::motd.
Sketch params/example.json installed under System::motd.
Sketch params/simple.json installed under System::motd.
Sketch test.cf installed under System::motd.
Verify that the sketch has been installed using the list
command:
cf-sketch> list
The following sketches are installed:
CFEngine::dclib Design Center standard library
CFEngine::stdlib The portions of the CFEngine standard library (also known as COPBL) that are compatible with 3.4.0 releases
System::motd Configure the Message of the Day
Note that the CFEngine::dclib
and CFEngine::stdlib
are
automatically installed as dependencies of the System::motd
sketch.
Step 5. Activate a sketch
Activating sketches includes adding two elements: a parameter set and an environment.
Define the parameter set
Define the parameter set that contains the values that will
be used by the System::motd
sketch:
cf-sketch> define params System::motd
Please enter a name for the new parameter set (default: System::motd-entry-000): motd_params
Querying configuration for parameter set 'motd_params' for bundle 'entry'.
Please enter parameter motd (Message of the Day (aka motd)).
(enter STOP to cancel)
motd : Hello there!
Please enter parameter motd_path (Location of the primary, often only, MotD file).
(enter STOP to cancel)
motd_path [/etc/motd]: /etc/motd
Please enter parameter prepend_command (Command output to prepend to MotD).
(enter STOP to cancel)
prepend_command [/bin/uname -snrvm]: /bin/uname -snrvm
Please enter parameter dynamic_path (Location of the dynamic part of the MotD file).
(enter STOP to cancel)
dynamic_path :
Please enter parameter symlink_path (Location of the symlink to the motd file).
(enter STOP to cancel)
symlink_path :
Defining parameter set 'motd_params' with the entered data.
Parameter set motd_params successfully defined.
Confirm that the parameter set has been properly defined using
the list
command:
cf-sketch> list -v params
The following parameter sets are defined:
motd_params: Sketch System::motd
[System::motd][dynamic_path]:
[System::motd][motd]: Hello there!
[System::motd][motd_path]: /etc/motd
[System::motd][prepend_command]: /bin/uname -snrvm
[System::motd][symlink_path]:
Define an environment
For this example, define an environment that is always active:
cf-sketch> define environment -n walkthrough any
Environment 'walkthrough' successfully defined.
Use the list
command to verify that the environment was
defined:
cf-sketch> list -v env walk
The following environments match your query:
walkthrough
[activated]: any
[test]: !any
[verbose]: !any
Note that the test
and verbose
fields are optional and default to
!any
, which is equivalent to "never" in CFEngine terms.
Run the activate command
Activate the sketch by tying together the sketch name, parameter set, and environment:
cf-sketch> activate System::motd motd_params walkthrough
Using generated activation ID 'System::motd-1'.
Using existing parameter definition 'motd_params'.
Using existing environment 'walkthrough'.
Activating sketch System::motd with parameters motd_params.
Verify that the activation has been created:
cf-sketch> list activations
The following activations are defined:
Activation ID System::motd-1
Sketch: System::motd
Parameter sets: [ motd_params ]
Environment: 'walkthrough'
This means that when the sketches are deployed, the System::motd
sketch will be executed with the values defined in the
motd_params
parameter set, and on the hosts that satisfy the
conditions defined in the walkthrough
environment (which includes
all machines for now).
Step 6. Deploy the sketch: Generate and execute the runfile
So far all the definitions of parameters, environments, and activations
are known only to the cf-sketch
tool. You must deploy these
changes, which creates the appropriate CFEngine policy files for
executing the activated sketches. We have two commands for this:
The run
command allows you to quickly test the execution of the
sketches on the local host. It generates a standalone runfile that
encodes all the necessary information for the activated sketches, and
then executes it using cf-agent
:
cf-sketch> run
Runfile /var/cfengine/inputs/api-runfile-standalone.cf successfully generated.
Now executing the runfile with: /var/cfengine/bin/cf-agent -f /var/cfengine/inputs/api-runfile-standalone.cf
The deploy
command generates a non-standalone runfile that is meant
to be loaded and executed from your main promises.cf
file:
cf-sketch> deploy
Runfile /var/cfengine/inputs/api-runfile.cf successfully generated.
More information
The Design Center framework provides an API that takes care of managing all the backend framework, and cf-sketch offers an "expert" mode in addition to the interactive mode described in these instructions.
Once you are familiar with the basic concepts and want to learn more about how things work internally, visit the advanced discussion on configuring sketches.
Visit the Design Center API for reference.
Once you are ready to start writing Design Center sketches, refer to the sketch structure documentation.
Advanced Walkthrough
This walkthrough illustrates how a Design Center sketch can be found, installed, configured, and executed as policy. Many items are already discussed at Command Line Sketches. This Walkthrough provides a more advanced look at sketches in that it describes internal and backend processes.
Please note the following instructions will work with CFEngine Enterprise as well, but the Mission Portal Design Center App is a better user interface for most of the topics covered here.
Before you Begin
Make certain you have installed all necessary software and have checked out the Design Center repository:
Complete the software requirements
Review the basics concepts of sketches
Check out the Design Center repository
Overview
The following topics are discussed in this Walkthrough:
- Search for sketches with the Design Center API
- Search for sketches with cf-sketch in expert mode
- Search for sketches with cf-sketch in interactive mode
- Install a sketch with the Design Center API
- Install a sketch with cf-sketch in expert mode
- Install a sketch with cf-sketch in interactive mode
- Activate a sketch with the Design Center API
- Activate a sketch with cf-sketch in expert mode
- Activate a sketch with cf-sketch in interactive mode
Generate and execute the runfile
- Generate and execute the runfile with the Design Center API
- Generate and execute the runfile with cf-sketch in expert mode
- Generate and execute the runfile with cf-sketch in interactive mode
Prepare the config.json file
To interact with the Design Center, you must tell its API where things
are installed. Copy the config.json
file to your CFEngine personal
directory:
mkdir ~/.cfagent
cp $CHECKOUT/tools/cf-sketch/config.json ~/.cfagent/dc-api-config.json
Note: A config-root.json
file exists that is intended for privileged
(root) usage. It sets things up under /var/cfengine
but still
expects the checkout under ~/source/design-center
.
Open the config.json file. It contains the following JSON data:
{
log: "STDERR",
log_level: 4,
repolist: [ "~/.cfagent/inputs/sketches" ],
recognized_sources: [ "~/source/design-center/sketches" ],
runfile: { location: "~/.cfagent/inputs/api-runfile.cf", standalone: true, relocate_path: "sketches", filter_inputs: [] },
vardata: "~/.cfagent/vardata.conf",
}
You can change any setting you want but keeping the default values is recommended.
The paths you see are all relative to your home directory. The log
can be set to a file. The log_level
can be lowered to 1 if you want
less noise (the level is set to 4 in this example).
vardata
: is where the Design Center API stores all configurations.
runfile
: describes where the Design Center API will save an
executable policy with all the sketches you have installed and
activated.
repolist
: is where sketches will be installed.
recognized_sources
: is where sketches will be found. It can be
a URL such as
https://github.com/cfengine/design-center/blob/master/sketches/cfsketches.json
.
You can save some typing by entering
export DCJ=~/.cfagent/dc-api-config.json
at the prompt. From that point on, all command-line interaction can
use $DCJ
and it will expand to the filename above.
From this point on, refer to this command
$CHECKOUT/tools/cf-sketch/cf-dc-api.pl
as $CFAPI
for brevity.
You can save some typing by entering
export CFAPI=$CHECKOUT/tools/cf-sketch/cf-dc-api.pl
at the prompt. From that point on, all command-line interaction can
use $CFAPI
and it will expand to the command above.
Search for sketches
This section explains many details that the later sections in the Walkthrough will skip for brevity.
Search for sketches with the Design Center API
This is what all the other Design Center tools use. You do not need to know this protocol or know that it is used.
echo '{ dc_api_version: "3.6.0", request: {search: true } }' | $CFAPI $DCJ
If you get errors here, you might be missing Perl modules or the CFEngine agent. Look at the Design Center Wiki for possible solutions to your problem.
Output:
DCAPI::log3(DCAPI.pm:173): Successfully loaded vardata file /home/tzz/.cfagent/vardata.conf
DCAPI::log(DCAPI.pm:376): Searching location ~/source/design-center/sketches for terms true
DCAPI::Sketch::matches(Repo.pm:118): sketch Applications::Memcached matched terms true
...
DCAPI::Sketch::matches(Repo.pm:118): sketch Webserver::Install matched terms true
All of the above is debugging output. With log
set to a file name,
the output will go to that file. With log_level
set to 1, only the
essential errors will be shown. The actual API response is:
{
"api_ok":
{
"warnings":[],"success":true,"errors":[],"error_tags":{},
"data":
{
"search":
{
"/home/tzz/source/design-center/sketches":
{
"System::config_resolver":"System::config_resolver", ... ,"System::set_hostname":"System::set_hostname"
}
}
},
"log":[],"tags":{}
}
}
The response says (with many sketch names omitted for brevity): Here, these are all my sketches.
Note that this is not what you would use daily.
Search for sketches with cf-sketch
in expert mode
$CHECKOUT/tools/cf-sketch/cf-sketch.pl --expert --cfpath=/var/cfengine/bin --apiconfig $DCJ --search | sort
Output:
Applications::Memcached Sketch for installing, configuring, and starting memcached.
...
You piped the command above through the standard sort
command to
sort the results. You can omit the sort
.
Search for sketches with cf-sketch
in interactive mode
Run:
$CHECKOUT/tools/cf-sketch/cf-sketch.pl --apiconfig $DCJ
You'll see this prompt, or something like it:
Welcome to cf-sketch version 3.6.0.
CFEngine AS, 2013.
Enter any command to cf-sketch, use 'help' for help, or 'quit' or '^D' to quit.
cf-sketch>
Now enter the search
command:
cf-sketch> search
The following sketches are available:
Applications::Memcached Sketch for installing, configuring, and starting memcached.
...
Yale::stdlib Yale standard library
cf-sketch>
This is the easiest method but you might want the expert or direct API interaction for specific purposes. All three are shown throughout this Walkthrough.
Install a sketch
Install a sketch with the Design Center API
echo '{ dc_api_version: "3.6.0", request: {install: {sketch:"System::motd", force:true} } }' | $CFAPI $DCJ
The force
parameter tells the Design Center API to overwrite the
sketch even if it is installed already.
Output:
DCAPI::log3(DCAPI.pm:173): Successfully loaded vardata file /home/tzz/.cfagent/vardata.conf
...
DCAPI::log(DCAPI.pm:576): Installing sketch: {"source":["~/source/design-center/sketches"],"target":"~/.cfagent/inputs/sketches","sketch":"System::motd","force":true}
DCAPI::log4(Repo.pm:179): Installing sketch System::motd: copying /home/tzz/source/design-center/sketches/system/motd/README.md to /home/tzz/.cfagent/inputs/sketches/system/motd/README.md
DCAPI::log4(Repo.pm:179): Installing sketch System::motd: copying /home/tzz/source/design-center/sketches/system/motd/main.cf to /home/tzz/.cfagent/inputs/sketches/system/motd/main.cf
DCAPI::log4(Repo.pm:179): Installing sketch System::motd: copying /home/tzz/source/design-center/sketches/system/motd/params/debian_squeeze.json to /home/tzz/.cfagent/inputs/sketches/system/motd/params/debian_squeeze.json
DCAPI::log4(Repo.pm:179): Installing sketch System::motd: copying /home/tzz/source/design-center/sketches/system/motd/params/debian_wheezy.json to /home/tzz/.cfagent/inputs/sketches/system/motd/params/debian_wheezy.json
DCAPI::log4(Repo.pm:179): Installing sketch System::motd: copying /home/tzz/source/design-center/sketches/system/motd/params/example.json to /home/tzz/.cfagent/inputs/sketches/system/motd/params/example.json
DCAPI::log4(Repo.pm:179): Installing sketch System::motd: copying /home/tzz/source/design-center/sketches/system/motd/params/simple.json to /home/tzz/.cfagent/inputs/sketches/system/motd/params/simple.json
DCAPI::log4(Repo.pm:179): Installing sketch System::motd: copying /home/tzz/source/design-center/sketches/system/motd/test.cf to /home/tzz/.cfagent/inputs/sketches/system/motd/test.cf
After lots of activity (again, remember to drop down to log_level
1 or 0 if you want to skip all these messages) the sketch is installed.
Finally the API returns:
{
"api_ok":
{
"warnings":[],"success":true,"errors":[],"error_tags":{},
"data":
{
"install":
{
"System::motd":
{
"test.cf":"/home/tzz/.cfagent/inputs/sketches/system/motd/test.cf",
"params/debian_squeeze.json":"/home/tzz/.cfagent/inputs/sketches/system/motd/params/debian_squeeze.json",
"README.md":"/home/tzz/.cfagent/inputs/sketches/system/motd/README.md",
"params/example.json":"/home/tzz/.cfagent/inputs/sketches/system/motd/params/example.json",
"params/debian_wheezy.json":"/home/tzz/.cfagent/inputs/sketches/system/motd/params/debian_wheezy.json",
"main.cf":"/home/tzz/.cfagent/inputs/sketches/system/motd/main.cf",
"params/simple.json":"/home/tzz/.cfagent/inputs/sketches/system/motd/params/simple.json"
},
"~/.cfagent/inputs/sketches":{"System::motd":1}
},
"inventory_save":1
},
"log":[],"tags":{"System::motd":1,"installation":8}
}
}
The above output says that System::motd
was installed in
/home/tzz/.cfagent/inputs/sketches/system/motd/
(because my
config.json
says so), and that the sketch inventory was saved
afterwards.
Install a sketch with cf-sketch
in expert mode
$CHECKOUT/tools/cf-sketch/cf-sketch.pl --expert --cfpath=/var/cfengine/bin --install System::motd --apiconfig $DCJ
Output:
Sketch System::motd is already in target repo; you must uninstall it first
So, we need to force it...
$CHECKOUT/tools/cf-sketch/cf-sketch.pl --expert --cfpath=/var/cfengine/bin --install System::motd --apiconfig $DCJ --force
Output: nothing! In expert mode, when everything is OK, nothing is printed. Only the command return code will tell you if everything went well.
So, we need to make it verbose...
$CHECKOUT/tools/cf-sketch/cf-sketch.pl --expert --cfpath=/var/cfengine/bin --install System::motd --apiconfig $DCJ --force --verbose
Output:
... lots of verbose output, including the API interaction, omitted ...
OK: Got successful result: ... the API result is here, omitted for brevity
The cf-sketch
expert mode is a thin layer over the API for testing
and unattended work, so the above verbose output is not really meant
for everyday use.
Install a sketch with cf-sketch
in interactive mode
Run:
$CHECKOUT/tools/cf-sketch/cf-sketch.pl --apiconfig $DCJ
Now enter the uninstall System::motd
and install System::motd
commands, because just installing an already-installed sketch will not do
anything interesting:
cf-sketch> uninstall System::motd
Deactivated System::motd.
Sketch 'System::motd' was uninstalled.
cf-sketch> install System::motd
Sketch System::motd installed under /home/tzz/.cfagent/inputs/sketches.
cf-sketch>
To view verbose output, use
--verbose
with the interactive cf-sketch
call and view the output.
Activate a sketch
Activate a sketch with the Design Center API
We are going to define a run environment, which will tell the Design Center API that we want an activated sketch, not in test mode, and with verbose output:
echo '{ dc_api_version: "3.6.0", request: {define_environment: { walkthrough: { activated: true, test: false, verbose: true } } } }' | $CFAPI $DCJ
Then we will define the parameters for the System::motd
sketch:
echo '{ dc_api_version: "3.6.0", request: {define: { "motd_params": { "System::motd": { "motd": "\\n ! System is under the control of CFEngine, local changes may by overwritten.\\n", "prepend_command": null } } } } }' | $CFAPI $DCJ
and finally, use the run environment and the parameters to activate the sketch:
echo '{ dc_api_version: "3.6.0", request: {activate: { "System::motd": { environment: "walkthrough", params: [ "motd_params" ] } } } }' | $CFAPI $DCJ
Output (omitting log lines and reformatted):
{"api_ok":{"warnings":[],"success":true,"errors":[],"error_tags":{},
"data":{"define_environment":{"walkthrough":1}},
"log":[],"tags":{"walkthrough":1}}}
{"api_ok":{"warnings":[],"success":true,"errors":[],"error_tags":{},
"data":{"define":{"motd_params":1}},
"log":[],"tags":{"motd_params":1}}}
{"api_ok":{"warnings":[],"success":true,"errors":[],"error_tags":{},
"data":{"activate":{"System::motd":{"params":["motd_params"],"environment":"walkthrough"}}},
"log":[],"tags":{"System::motd":1}}}
This tells us that the API has recorded that we want the sketch
System::motd
to run with the run environment walkthrough
and the
parameters motd_params
.
Activate a sketch with cf-sketch
in expert mode
We'll try the simple.json
parameters that come with System::motd
.
You can look at that file; it is the same as the motd_params
above.
$CHECKOUT/tools/cf-sketch/cf-sketch.pl --expert --apiconfig $DCJ --activate System::motd=$CHECKOUT/sketches/system/motd/params/simple.json --verbose --activated
Output:
...
DCAPI::log(DCAPI.pm:1061): Activations for sketch System::motd are now [{"params":["motd_params"],"environment":"walkthrough"},{"params":["parameter definition from /home/tzz/source/design-center/sketches/system/motd/params/simple.json"],"environment":"cf_sketch_testing","target":"/home/tzz/.cfagent/inputs/sketches"}]
This says we now have two activations. One from the API call above and one we just created. Let's undo the activations:
$CHECKOUT/tools/cf-sketch/cf-sketch.pl --expert --apiconfig $DCJ --deactivate-all
$CHECKOUT/tools/cf-sketch/cf-sketch.pl --expert --apiconfig $DCJ --activate System::motd=$CHECKOUT/sketches/system/motd/params/simple.json --verbose --activated
Output:
...
DCAPI::log(DCAPI.pm:1403): Deactivating all activations: {"System::motd":[{"params":["motd_params"],"environment":"walkthrough"},{"params":["parameter definition from /home/tzz/source/design-center/sketches/system/motd/params/simple.json"],"environment":"cf_sketch_testing","target":"/home/tzz/.cfagent/inputs/sketches"}]}
...
OK: Got successful result: {"success":true,"warnings":[],"errors":[],"error_tags":{},"log":[],"data":{"activate":{"System::motd":{"environment":"cf_sketch_testing","params":["parameter definition from /home/tzz/source/design-center/sketches/system/motd/params/simple.json"],"target":"/home/tzz/.cfagent/inputs/sketches"}}},"tags":{"System::motd":1}}
Looks like it worked! The cf_sketch_testing
environment is created
by cf-sketch
on the fly and will include the same things as the
walkthrough
run environment. The --activated
and --verbose
flags turn on the environment activated
and verbose
flags.
There's also a --test
flag, but we will not use it here.
Activate a sketch with cf-sketch
in interactive mode
Run:
$CHECKOUT/tools/cf-sketch/cf-sketch.pl
Then:
cf-sketch> define params System::motd
Please enter a name for the new parameter set (default: System::motd-entry-000): motd_params
Querying configuration for parameter set 'motd_params' for bundle 'entry'.
Please enter parameter motd (Message of the Day (aka motd), ).
motd : Hello there!
Please enter parameter motd_path (Location of the primary, often only, MotD file, ).
motd_path [/etc/motd]: /etc/motd
Please enter parameter prepend_command (Command output to prepend to MotD, ).
prepend_command [/bin/uname -snrvm]: /bin/uname -snrvm
Please enter parameter dynamic_path (Location of the dynamic part of the MotD file, ).
dynamic_path : null
Please enter parameter symlink_path (Location of the symlink to the motd file, ).
symlink_path : null
Defining parameter set 'motd_params' with the entered data.
Parameter set motd_params successfully defined.
cf-sketch> activate System::motd motd_params walkthrough
Using existing parameter definition 'motd_params'.
Using existing environment 'walkthrough'.
Activating sketch System::motd with parameters motd_params.
...
DCAPI::log(DCAPI.pm:1061): Activations for sketch System::motd are now [{"params":["parameter definition from /home/tzz/source/design-center/sketches/system/motd/params/simple.json"],"environment":"cf_sketch_testing","target":"/home/tzz/.cfagent/inputs/sketches"},{"params":["motd_params"],"environment":"walkthrough","target":"/home/tzz/.cfagent/inputs/sketches","identifier":"System::motd-1"}]
cf-sketch>
As you can see, the expert and interactive modes have completely different usage patterns.
Generate and execute the runfile
Generate and execute the runfile with the Design Center API
Generating the runfile is easy:
echo '{ dc_api_version: "3.6.0", request: {regenerate: { } } }' | $CFAPI $DCJ
Output:
DCAPI::log(DCAPI.pm:249): Saving runfile /home/tzz/.cfagent/inputs/api-runfile.cf
{"api_ok":{"warnings":[],"success":true,"errors":[],"error_tags":{},"data":{},"log":[],"tags":{}}}
Note there is no user control over the location of the runfile, it's
entirely defined in the API's config.json
file. This is by design.
Time to run the policy!!! We know the name of the runfile, so we can run it.
cf-agent -KI -f ~/.cfagent/inputs/api-runfile.cf
We are not in test mode, so we will get errors if we run as a non-privileged user.
2013-06-04T20:12:04-0400 info: This agent is not bootstrapped
2013-06-04T20:12:04-0400 info: Running full policy integrity checks
2013-06-04T20:12:04-0400 error: Unable to open destination file '/etc/motd.cf-after-edit' for writing. (fopen: Permission denied)
2013-06-04T20:12:04-0400 error: /cfsketch_run/methods/'___001_System_motd_entry'/cfdc_motd:entry/files/'$(main_path)': Unable to save file '/etc/motd' after editing
2013-06-04T20:12:04-0400 error: chmod failed on '/etc/motd'. (chmod: Operation not permitted)
2013-06-04T20:12:04-0400 notice: R: cfdc_motd:entry: System::motd license = MIT
2013-06-04T20:12:04-0400 notice: R: cfdc_motd:entry: System::motd dependencies = CFEngine::dclib, CFEngine::stdlib
2013-06-04T20:12:04-0400 notice: R: cfdc_motd:entry: System::motd version 1.00 by Ben Heilman <bheilman@enova.com> starting up...
2013-06-04T20:12:04-0400 notice: R: cfdc_motd:entry: imported environment 'cf_sketch_testing' var 'activated' with value '1'
2013-06-04T20:12:04-0400 notice: R: cfdc_motd:entry: imported environment 'cf_sketch_testing' var 'test' with value ''
2013-06-04T20:12:04-0400 notice: R: cfdc_motd:entry: imported environment 'cf_sketch_testing' var 'verbose' with value '1'
2013-06-04T20:12:04-0400 notice: R: cfdc_motd:entry: imported environment 'cf_sketch_testing' class 'activated' because 'default:runenv_cf_sketch_testing_activated' was defined
2013-06-04T20:12:04-0400 notice: R: cfdc_motd:entry: imported environment 'cf_sketch_testing' class 'verbose' because 'default:runenv_cf_sketch_testing_verbose' was defined
2013-06-04T20:12:04-0400 notice: R: cfdc_motd:entry: running in verbose mode
Complete.
Generate and execute the runfile with cf-sketch
in expert mode
$CHECKOUT/tools/cf-sketch/cf-sketch.pl --expert --apiconfig $DCJ --generate
Output:
...
DCAPI::log(DCAPI.pm:249): Saving runfile /home/tzz/.cfagent/inputs/api-runfile.cf
Run the runfile:
cf-agent -KI -f ~/.cfagent/inputs/api-runfile.cf
The output will be the same as in the previous section.
Generate and execute the runfile with cf-sketch
in interactive mode
Run:
$CHECKOUT/tools/cf-sketch/cf-sketch.pl
You'll see:
cf-sketch> generate
...
DCAPI::log(DCAPI.pm:249): Saving runfile /home/tzz/.cfagent/inputs/api-runfile.cf
Runfile /home/tzz/.cfagent/inputs/api-runfile.cf successfully generated.
Run it:
cf-agent -KI -f ~/.cfagent/inputs/api-runfile.cf
The output will be the same as in the previous section.
Write a new Sketch
Enterprise and Community Users can Write Sketches
Overview
This page describes how to create a Design Center sketch from a basic CFEngine agent bundle. In effect the sketch is a wrapper, taking parameters from the Design Center API or the CFEngine Enterprise Mission Portal, and passing them down to the agent bundle.
The Bundle
The bundle we will wrap with a sketch is a simple use of the users
promise type:
bundle agent ensure_users(users, group, homedir, shell)
{
users:
"$(users)" -> {"PCI-DSS-2", "Baseline_developers_2_3"}
policy => "present",
home_dir => "$(homedir)/$(users)",
group_primary => $(group),
shell => $(shell),
handle => "ensure_user_setup",
home_bundle => setup_home_dir($(users), $(homedir));
}
bundle agent setup_home_dir(user, group, homedir)
{
files:
"$(homedir)/$(user)/." create => "true";
}
If you don't understand what it does, please look at Tutorials and the CFEngine Guide to learn more about CFEngine's policy language, syntax, and operation. We will not worry about the internals except to apply namespaces and testing guards.
The sketch.json Metadata
The first step is to define the sketch metadata. This is the annoying
administrivia that makes a package system such as Design Center
useful. Simply take an existing sketch.json
file such as the one below and modify it:
{
manifest:
{
"main.cf": { description: "main file" },
"README.md": { documentation: true },
},
metadata:
{
name: "System::Users",
description: "Configure users with parameters",
version: "1.00",
license: "MIT",
tags: [ "cfdc", "users", "enterprise_compatible", "enterprise_3_6" ],
authors: [ "Ted Zlatanov <tzz@lifelogs.com>" ],
depends: { "CFEngine::sketch_template": {}, cfengine: { version: "3.6.0" }, os: [{ "linux": "Linux", "solaris": "Solaris", "aix": "AIX", "windows": "Windows" }] }
},
api:
{
// this is the name of the bundle!
ensure_users:
[
{ type: "bundle_options", name: "Ensure the users exist as specified" },
{ type: "environment", name: "runenv", },
{ type: "metadata", name: "mymetadata", },
{ type: "list", name: "users", validation: "LIST_OF_STRING_NONEMPTY", description: "User names to add (separate by commas)" },
{ type: "string", name: "group", validation: "STRING_NONEMPTY", description: "Primary user group" },
{ type: "string", name: "homedir", default: "/home", validation: "PATH_ABSOLUTE_UNIX_OR_WINDOWS", description: "Location of the user's home directory" },
{ type: "string", name: "shell", default: "/bin/bash", choice: [ "/bin/sh", "/bin/bash", "/bin/csh", "/bin/tcsh", "/bin/zsh" ], description: "User shell" },
],
},
namespace: "cfdc_users",
interface: [ "main.cf" ],
}
- add any files you distribute with the sketch to the
manifest
- set the
authors
,tags
,name
,description
, etc. metadata - define an API, which mirrors the bundle we have. Here we do a few things extra:
- add
bundle_options
with the name of the bundle we want to show - add
environment
andmetadata
parameters, which carry the "glue" between Design Center and CFEngine - bring in the
users
,group
,homedir
, andshell
parameters with atype
, adefault
if needed, avalidation
or achoice
as needed, and adescription
- add
- set the
namespace
as shown to ensure this bundle won't conflict with others - set the
interface
to the list of files that have to be included by CFEngine for the sketch to work, normally justmain.cf
main.cf: The Converted Bundle
Now main.cf
will start with the original bundle, but we'll modify it.
body file control
{
namespace => "cfdc_users";
}
bundle agent ensure_users(runenv, metadata, users, group, homedir, shell)
{
#@include "REPO/sketch_template/standard.inc"
users:
!dc_test::
"$(users)" -> {"PCI-DSS-2", "Baseline_developers_2_3"}
policy => "present",
home_dir => "$(homedir)/$(users)",
group_primary => $(group),
shell => $(shell),
handle => "ensure_user_setup",
home_bundle => setup_home_dir($(users), $(homedir));
reports:
dc_verbose.dc_test::
"$(dcbundle): simulating user = $(users) with group $(group), home dir $(homedir) and shell $(shell)";
dc_verbose.!dc_test::
"$(dcbundle): ensuring user = $(users) with group $(group), home dir $(homedir) and shell $(shell)";
}
bundle agent setup_home_dir(user, group, homedir)
{
files:
"$(homedir)/$(user)/." create => "true";
}
- add a
namespace
matchingsketch.json
- add the
#@include
statement which sets up the Design Center machinery - add the
runenv
andmetadata
parameters (note you don't use them!) - report on what you'll do, using the classes
dc_verbose
anddc_test
which mean "you're in verbose mode" and "you're in test mode" respectively - only make the users if not in
dc_test
mode
Package The Sketch
This part is really too easy. We should make it harder so you have something to complain about!
There are two steps:
- put your files in a directory (say
/my/repo/sketches/xyz
if your sketches will live under/my/repo
and the one you made is calledxyz
). This is just the files from the manifest:README.md
, which you could cut from the manifest or auto-generate, is just a README filemain.cf
which you just saw above- plus
sketch.json
as shown above
- regenerate the sketch index for
/my/repo
and install your sketch into/var/cfengine/design-center/sketches
(the "live" repository of sketches). Run the following commands:
cp -rp /var/cfengine/share/*Base/sketches/sketch_template /my/repo/sketches/
/var/cfengine/design-center/bin/cf-sketch --make_cfsketches --inputs /my/repo --is=/my/repo/sketches/cfsketches.json
/var/cfengine/design-center/bin/cf-sketch --make_readme --is=/my/repo/sketches/cfsketches.json
/var/cfengine/design-center/bin/cf-sketch --install-all --is=/my/repo/sketches/cfsketches.json --inputs=/var/cfengine/design-center
See Maintaining your own sketch repository for these exact commands to run, with a longer explanation for each one.
You're Done!
That's all there is to writing a sketch. You should now look at
sketchify - Write A New Sketch From An Existing Bundle
for a guide to using the sketchify
tool. See
Maintaining your own sketch repository
to find out how to create your own sketch repository and install sketches from it.
Your users can then use the sketch you've written as described in Deploy your first Policy.
sketchify - Write A New Sketch From An Existing Bundle
Enterprise and Community Users can Write Sketches from the Command Line
Overview
This page describes how to create a Design Center sketch by converting an existing policy
into a sketch and by using the sketchify
tool in cf-sketch
to complete the process.
These steps are followed:
Step 1. Select a policy to convert into a sketch
Step 3. Define the sketch interface
Step 4. Revise the policy file as necessary
Step 5. Use the sketchify command to wrap the policy file into a sketch structure
Step 6. Verify that the new sketch is ready for installation and use
Refer to the password_expiration()
bundle example as you
follow the steps outlined on this page.
This information is from Diego Zamboni's book, Learning CFEngine 3. Used with permission. It has been updated, reformatted and edited for website consistency.
Before you Begin
This is an advanced topic; we assume that you know how to write CFEngine policy, and that you are familiar with the Design Center command line tools and, if you are an Enterprise user, with the Design Center UI.
Instructions
Step 1. Select a policy to convert into a sketch
The foundation of any Design Center sketch should be a working piece of CFEngine
policy in the form of a bundle of type agent that performs the appropriate functionality.
This bundle can call other bundles or bodies as appropriate, but it should be callable
as a single point of entry. Until you become more familiar with how sketches
are structured, write your bundles first as regular CFEngine
policy, and then convert them to sketches. The instructions on this page create a sketch from the
password_expiration()
bundle.
Step 2. Define a sketch name
Arbitrary names are acceptable, but
the Design Center by convention encourages us to use names of the form Category::Sketch
,
or even Category::Subcategory::Sketch
. For our password-expiration configuration
sketch, use Security::password_expiration
.
Step 3. Define the sketch interface
Identify and name configurable parameters
In our original example, all the
parameters are specified as variables inside the password_expiration()
bundle. For
a sketch, however, we want those values as parameters to be specified by the user when they
configure the sketch. The next step, then, is to look through the original code, make a
list of what those configurable parameters should be, and decide on their names:
pass_max_days
The maximum password age in days.
pass_min_days
The minimum password age, also in days.
pass_warn_age
The warning period before a password expires, in days.
min_uid
The minimum UID for setting password-expiration parameters. Users with UID below this threshold will not be modified.
skipped_users
A comma-separated list of usernames to skip when setting password-expiration parameters.
skipped_uids
A comma-separated list of UIDs to skip when setting password-expiration parameters.
All of these parameters can be specified as strings, just as they are in the original policy code.
Create a namespace
You must also decide on a namespace in which to place the sketch. Namespaces are top-level naming divisions that help avoid conflicts in bundle, body, or class names. Use a namespace that contains the following:
- a reference to the origin of the sketch. For example, all CFEngine-produced sketches have namespaces that start with cfdc_ for CFEngine Design Center
- the name of the sketch, or a shortened, representative version of it.
For this example, use cflearn_password_expiration.
Step 4. Revise the policy file as necessary
Once the sketch interface is defined, rewrite the policy file as necessary to make it ready to use as a sketch. Below is the updated code, with some comments about the changes made As you go through these, compare them to the original code in the password_expiration() bundle:
bundle agent password_expiration(pass_max_days, pass_min_days, pass_warn_age,
min_uid, skipped_users, skipped_uids) # <1>
{
vars:
# We store the individual parameters in an array,
# for easier reference and file editing
"logindefs[PASS_MAX_DAYS]" string => "$(pass_max_days)"; # <2>
"logindefs[PASS_MIN_DAYS]" string => "$(pass_min_days)";
"logindefs[PASS_WARN_AGE]" string => "$(pass_warn_age)";
# Position of each parameter in /etc/shadow
"fieldnum[PASS_MIN_DAYS]" string => "4";
"fieldnum[PASS_MAX_DAYS]" string => "5";
"fieldnum[PASS_WARN_AGE]" string => "6";
# List of parameters to modify
"params" slist => getindices("logindefs");
# Get list of users, and also generate them in canonified form
# This list already excludes users specified by UID or name.
"users" slist => getusers("$(skipped_users)", "$(skipped_uids)");
"cusers[$(users)]" string => canonify("$(users)");
classes:
# Define classes for users that must not be modified by UID threshold
"skip_$(cusers[$(users)])" expression => islessthan(getuid("$(users)"),
"$(min_uid)");
files:
linux:: # <3>
"/etc/login.defs"
handle => "edit_logindefs",
comment => "Set desired login.defs parameters",
edit_line =>
default:set_config_values(
"cflearn_password_expiration:password_expiration.logindefs"); # <4>
"/etc/shadow"
handle => "edit_shadow_$(params)",
comment => "Modify $(params) for individual users.",
edit_defaults => default:backup_timestamp, # <5>
edit_line => default:set_user_field("$(users)",
"$(fieldnum[$(params)])",
"$(logindefs[$(params)])"),
ifvarclass => "!skip_$(cusers[$(users)])";
reports:
!linux:: # <6>
"Warning: Security::password_expiration only works on Linux for now.";
}
Point by Point:
The logic of the code has not changed, but a few things have been updated or rearranged:
<1> We have added all the configurable parameters we determined earlier as arguments
to our password_expiration()
bundle. All of these values are now accepted as arguments
instead of being hardcoded into the policy. This is the entry point for our sketch.
<2> We use the new parameters throughout the code, instead of the hard-coded values we had before.
<3> We have added a class expression to limit the execution of the sketch to systems
that support its behavior. This is necessary because a sketch might be activated on
many different systems, and it needs to do the right thing regardless of
where it is running. In this case, we have limited it to Linux systems, in which we
know the password-expiration
parameters are configured using the /etc/login.defs
file.
<4> Here is the first use of namespaces, in two places: We have added the
default:
namespace specification to the standard library bundle
set_config_values()
, and we have specified our sketch namespace in the fully-qualified
name of the logindefs
array that we pass to set_config_values()
. The
fully-qualified name of the array (cflearn_password_expiration:password_expiration.logindefs
)
contains the namespace, the bundle name, and the array name.
<5> We must add the default:
namespace to all the standard library components we
use and, in this case, also to the backup_timestamp
body and the set_user_field()
bundle.
<6> Finally, and to complement the limitation of functionality of the sketch to Linux
systems, we have added a reports:
promise that prints a warning on non-Linux systems to let
users know that the sketch is non-functional on them.
We now have the policy file in a shape that is well suited for conversion into a sketch.
Step 5. Use the sketchify command to wrap the policy file into a sketch structure
The next step is to actually wrap the revised policy file into the appropriate structure required
by a sketch, which includes putting the file into its own directory. Add to that directory
a README
file and a file named sketch.json
that contains all the metadata about the
sketch, as well as all the information needed to configure and invoke it. You can find
the full specification in the
Writing a Design Center Sketch
guide, but you can also
use the sketchify
command in cf-sketch
to do it automatically. The sketchify
command reads the
policy file, asks you for the appropriate information, and produces a ready-to-use sketch
in your local checkout of the Design Center repository.
The sketchify
command takes as its only argument the file that contains our policy file,
which it reads and analyzes for bundles of type agent
. Our policy file contains only one
bundle, so it is used automatically as the entry point for the sketch (if more than one
agent bundle is found, you will be asked which one you want to use as the sketch entry
point):
# /var/cfengine/design-center/bin/cf-sketch sketchify /vagrant/password_expiration.cf
Reading file '/vagrant/password_expiration.cf'.
Automatically choosing the only agent bundle in /vagrant/password_expiration.cf:
'password_expiration'
I will now prompt you for the data needed to generate the sketch.
Please enter STOP at any prompt to interrupt the process.
Note: The Design Center framework supports sketches with more than one entry point, but sketchify as of this writing lets you choose only one of them.
Next, sketchify asks for general information about the sketch, including its name, description, version number, license (most sketches in the Design Center use the MIT license), tags, and author information. You can also enter the names of other CFEngine policy files that should be included in this sketch. Most sketches are contained in a single .cf file, but if you have a very complex sketch, the ability to package multiple .cf files within the same sketch could be useful:
Sketch name: Security::password_expiration
One-line description for the new sketch: Manage password expiration and warning periods
Sketch version number: 1.0
Sketch license: MIT
Sketch tags (comma-separated list): security,cflearn,passwords
Authors (comma-separated list, preferably of the form 'Name <email>'): Diego Zamboni <diego.zamboni@cfengine.com>
Please enter any other files that need to be included with this sketch (press Enter to stop):
Note: This has nothing to do with sketch dependencies—any file(s) you specify
here will be included within the sketch you are creating. As of this writing,
sketchify does not handle sketch dependencies. You must include
them by hand in the generated sketch.json
file.
Next, sketchify
queries us for the information needed for defining
the sketch API. For each parameter of the entry bundle, sketchify
prompts for its type, a description, and a optional default and
example values (the example value is shown in the Design Center app in
CFEngine Enterprise). In our example, we give default values for all
the parameters except skipped_users
and skipped_uids
:
Thank you. I will now prompt you for the information regarding the parameters
of the entry point for the sketch.
For each parameter, you need to provide a type, a description, and optional default and example values.
(enter STOP at any prompt to abort)
For parameter 'pass_max_days':
Type [(1) string, (2) boolean, (3) list, (4) array]: : 1
Short description: : Maximum password age in days
Default value (empty for no default): : 180
Example value (empty for no example): : 180
For parameter 'pass_min_days':
Type [(1) string, (2) boolean, (3) list, (4) array]: : 1
Short description: : Minimum password age in days
Default value (empty for no default): : 5
Example value (empty for no example): : 5
For parameter 'pass_warn_age':
Type [(1) string, (2) boolean, (3) list, (4) array]: : 1
Short description: : Warning period before password expires, in days
Default value (empty for no default): : 2
Example value (empty for no example): : 2
For parameter 'min_uid':
Type [(1) string, (2) boolean, (3) list, (4) array]: : 1
Short description: : Minimum UID to consider when updating existing accounts
Default value (empty for no default): : 500
Example value (empty for no example): : 500
For parameter 'skipped_users':
Type [(1) string, (2) boolean, (3) list, (4) array]: : 1
Short description: : Comma-separated list of usernames to skip when updating existing accounts
Default value (empty for no default): :
Example value (empty for no example): : diego,joe
For parameter 'skipped_uids':
Type [(1) string, (2) boolean, (3) list, (4) array]: : 1
Short description: : Comma-separated list of UIDs to skip when updating existing accounts
Default value (empty for no default): :
Example value (empty for no example): : 550,1027
We are done with the API!
Having defined the sketch API, sketchify now queries you for information about the namespace to use for this sketch. We decided before which namespace to use, but the namespace declaration does not yet appear in the policy file we are using, so sketchify offers to insert it automatically:
Now checking the namespace declaration.
The file '/vagrant/password_expiration.cf' does not have a namespace declaration.
It is recommended that every sketch has its own namespace to avoid potential naming conflicts with other sketches or policies.
I can insert the appropriate namespace declaration, and have generated a suggested namespace for you: cfdc_security_password_expiration
Please enter the namespace to use for this sketch: : cfdc_security_password_expiration
Note: If you insert the namespace declaration in the policy file by hand, before running it through sketchify, the command will automatically detect and use the declaration.
In addition to the parameters defined in the API, a sketch entry bundle can receive two
special parameters of type environment
and metadata
. If used, these parameters will be
automatically generated and passed by the Design Center framework when executing
the sketch.
The
environment
parameter contains the name of the environment with which the sketch has been activated. This allows the sketch to access the characteristics of the environment, including the verbose and testing fields (interpreted as classes) so that the sketch can easily use them as conditions to alter its behavior.The
metadata
parameter contains the name of an array in which the Design Center framework automatically stores all the sketch metadata, including its name and description, authors, etc.
If these parameters are not already passed to the entry bundle in the input file, sketchify will ask you if you want to add them:
The entry point 'password_expiration' doesn't seem to receive
parameters of type 'environment' or 'metadata'.
These arguments are useful for the sketch to respond to different run
environment parameters (i.e. test or verbose mode) or to have access
to its own metadata. I can automatically add these parameters to the
bundle, together with some boilerplate code to put their information
in classes and variables.
Would you like me to add environment/metadata parameters and code to
the sketch? (Y/n) : y
In addition to adding the parameters to the bundle, sketchify also adds some boilerplate code to do the following:
Extract the values of all fields defined in the active environment (at least
activated
,verbose
, andtesting
, and possibly others if defined) into both classes and variables. For example, it will create a string variable namedverbose
that contains the class expression stored in that field, and also a class namedverbose
that will be set to the result of evaluating that class expression. You can then use that class within your sketch to easily enable additional reports when verbose mode has been activated in the current environment.Add some other information for better integration of the sketch into the CFEngine Enterprise Design Center app.
As of this writing, the following code is automatically inserted by sketchify at the top
of the bundle. This line is automatically expanded into the contents
of the template file which can be found at
/var/cfengine/design-center/sketches/sketch_template/standard.inc
.
#@include "REPO/sketch_template/standard.inc"
sketchify
now asks you for the location under the currently-used
sketch repository where the new sketch should be stored:
Thank you! We are almost done.
Please enter the directory where the new sketch will be stored.
If you enter a relative path, it will be used within the currently configure sketch repository (/var/cfengine/design-center/sketches). If you enter an absolute path, it will be used as-is. The directory will be created if needed.
I have generated a suggestion based on your sketch name: security/password_expiration
Directory: security/password_expiration
Before writing the sketch, sketchify
shows you a menu with all the
parameters you entered, and gives you a chance to modify them. If you
made any mistakes or want to change anything, enter the number of the
corresponding parameter and sketchify
will prompt you for the values
again.
You now have a chance to modify any of the information you entered.
These are the current sketch parameters:
1. Sketch name: Security::password_expiration
2. One-line description for the new sketch: Manage password expiration and warning periods
3. Sketch version number: 1.0
4. Sketch license: MIT
5. Sketch tags: cflearn, enterprise_compatible, passwords, security, sixified, sketchify_generated
6. Authors: Diego Zamboni <diego.zamboni@cfengine.com>
7. Extra manifest files:
8. Sketch API:
For bundle password_expiration
pass_max_days: string (Maximum password age in days) [default value: '180']
pass_min_days: string (Minimum password age in days) [default value: '5']
pass_warn_age: string (Warning period before password expires, in days) [default value: '2']
min_uid: string (Minimum UID to consider when updating existing accounts) [default value: '500']
skipped_users: string (Comma-separated list of users to skip when updating existing accounts)
skipped_uids: string (Comma-separated list of UIDs to skip when updating existing accounts)
9. Namespace: cfdc_security_password_expiration
10. Runenv and metadata parameters: Environment and metadata parameters and boilerplate code WILL be added
11. Output directory: /var/cfengine/design-center/sketches/
Please enter the number of the part you want to modify (1-11, Enter to
continue)
Finally, when you press "Enter" in the prompt above, sketchify
writes all the files for the sketch in the appropriate directory:
Your new sketch will be stored under /var/cfengine/design-center/sketches/security/password_expiration
Writing /var/cfengine/design-center/sketches/security/password_expiration/sketch.json
Transferring /vagrant/password_expiration.cf to /var/cfengine/design-center/sketches/security/password_expiration/password_expiration.cf
Regenerating sketch index in /var/cfengine/design-center/sketches
Generating a README file for the new sketch.
We are done! Please check your new sketch under
/var/cfengine/design-center/sketches/security/password_expiration.
The sketch is created; the process is complete.
Step 6. Verify that the new sketch is ready for installation and use
Verify this using cf-sketch
. Search for the password sketch:
cf-sketch> search password
The following sketches match your query:
Security::password_expiration Manage password expiration and warning periods
cf-sketch> install Security::password_expiration
Sketch Security::password_expiration installed under
/var/cfengine/masterfiles/sketches.
cf-sketch> info -v Security::password_expiration
The following sketches match your query:
Sketch Security::password_expiration
Description: Manage password expiration and warning periods
Authors: Diego Zamboni <diego.zamboni@cfengine.com>
Version: 1.0
License: MIT
Tags: passwords, security, sketchify_generated, cflearn
Installed: Yes, under /var/cfengine/masterfiles/sketches
Activated: No
Parameters:
For bundle password_expiration
pass_max_days: string (Maximum password age in days) [default value: '180']
pass_min_days: string (Minimum password age in days) [default value: '5']
pass_warn_age: string (Warning period before password expires, in days)
[default value: '2']
min_uid: string (Minimum UID to consider when updating existing accounts)
[default value: '500']
skipped_users: string (Comma-separated list of usernames to skip when updating existing accounts)
skipped_uids: string (Comma-separated list of UIDs to skip when updating existing accounts)
While sketchify automates most of the process of creating a sketch from an existing bundle, it cannot handle a few items. Thus, look at the files it generates for sanity checking. Here are some of the things you might want or need to fix by hand:
Dependencies: If your sketch depends on other sketches, you must add them by hand to the
depends
metadata element in the generated sketch.json file. At the moment, sketchify automatically inserts a dependency on CFEngine 3.5.0, which is the minimum recommended version of using Design Center sketches.Multiple entry points: The Design Center framework supports multiple entry points per sketch (to different bundles). This is not supported at the moment by sketchify, so you must add any additional entry points by hand.
Calls to standard library bundles and bodies need to be prefixed with
default:
so that they are correctly found when called from the sketch namespace.
Example
password_expiration() bundle
Below is the existing policy example that is used to turn into a sketch. Refer to it as you follow the steps for creating a sketch.
bundle agent password_expiration
{
vars:
# Maximum password age
"logindefs[PASS_MAX_DAYS]" string => "180";
# Minimum password age (minimum days between changes)
"logindefs[PASS_MIN_DAYS]" string =>"10";
# Warning period (in days) before password expires
"logindefs[PASS_WARN_AGE]" string => "5";
# Position of each parameter in /etc/shadow
"fieldnum[PASS_MIN_DAYS]" string => "4";
"fieldnum[PASS_MAX_DAYS]" string => "5";
"fieldnum[PASS_WARN_AGE]" string => "6";
# List of parameters to modify
"params" slist => getindices("logindefs");
# UIDs below this threshold will not be touched
"uidthreshold" int => "500";
# Additionally, these users and UIDs will not be touched.
# These are comma-separated lists.
"skipped_users" string => "vboxadd,nobody";
"skipped_uids" string => "1000,1005";
# Get list of users, and also generate them in canonified form
"users" slist => getusers("$(skipped_users)", "$(skipped_uids)");
"cusers[$(users)]" string => canonify("$(users)");
classes:
# Define classes for users that must not be modified,
# either by UID threshold or by username
"skip_$(cusers[$(users)])" expression => islessthan(getuid("$(users)"),
"$(uidthreshold)");
files:
"/etc/login.defs"
handle => "edit_logindefs",
comment => "Set desired login.defs parameters",
edit_line => set_config_values("password_expiration.logindefs");
"/etc/shadow"
handle => "edit_shadow_$(params)",
comment => "Modify $(params) for individual users.",
edit_defaults => backup_timestamp,
edit_line => set_user_field("$(users)",
"$(fieldnum[$(params)])",
"$(logindefs[$(params)])"),
ifvarclass => "!skip_$(cusers[$(users)])";
}
Reporting
No promises made in CFEngine imply automatic aggregation of data to a central location. In CFEngine Enterprise (our commercial version), an optimized aggregation of standardized reports is provided, but the ultimate decision to aggregate must be yours.
Monitoring and reporting capabilities in CFEngine depend on your installation:
Enterprise Edition Reporting
The CFEngine Enterprise edition offers a framework for configuration management that goes beyond building and deploying systems. Features include compliance management, reporting and business integration, and tools for handling the necessary complexity.
In a CFEngine Enterprise installation, the CFEngine Server aggregates information about the environment in a centralized database. By default data is collected every 5 minutes from all bootstrapped hosts and includes information about:
- logs about promises kept, not kept and repaired
- current host contexts and classifications
- variables
- software information
- file changes
This data can be mined using SQL queries and then used for inventory management, compliance reporting, system diagnostics, and capacity planning.
Access to the data is provided through:
Command-Line Reporting
Community Edition
Basic output to file or logs can be customized on a per-promise basis. Users can design their own log and report formats, but data processing and extraction from CFEngine's embedded databases must be scripted by the user.
Note:
If you have regular reporting needs, we recommend using our commercially-supported version of CFEngine, Enterprise. It will save considerable time and resources in programming, and you will have access to the latest developments through the software subscription.
Monitoring and Reporting
What are Monitoring and Reporting?
Monitoring is the sampling of system variables at regular intervals in order to present an overview of actual changes taking place over time. Monitoring data are often presented as extensive views of moving-line time series. Monitoring has the ability to detect anomalous behavior by comparing past and present.
The term reporting is usually taken to mean the creation of short summaries of specific system properties suitable for management. System reports describe both promises about the system, such as compliance, discovered changes and faults.
The challenge of both these activities is to compare intended or promised, behavior with the actual observed behavior of the system.
Should Monitoring and Configuration be Separate?
The traditional view of IT operations is that configuration, monitoring, and reporting are three different things that should not be joined. Traditionally, all three have been independent centralized processes. This view has emerged historically, but it has a major problem: Humans are needed to glue these parts back together. Monitoring as an independent activity is inherently non-scalable. When numbers of hosts grow beyond a few thousands, centralized monitoring schemes fail to manage the information. Tying configuration (and therefore repair) to monitoring at the host level is essential for the effective management of large and distributed data facilities. CFEngine foresaw this need in 1998, with its Computer Immunology initiative, and continues to develop this strategy.
CFEngine's approach is to focus on scalability. The commercial editions of
CFEngine provide what meaningful information they can in a manner that can
be scaled to tens of thousands of machines.
<!--- End include: /home/jenkins/workspace/build-documentation-3.9/label/DOCUMENTATION_x86_64_linux_debian_6/documentation/guide/reporting/monitoring-reporting.markdown
-->
Command-Line Reports
Command-line reporting is available to Enterprise and Community users.
Overview
The following report topics are included:
CFEngine output levels
CFEngine's default behavior is to report to the console (known as standard output). It's default behavior is to report nothing except errors that are judged to be of a critical nature.
By using CFEngine with the inform flag, you can alter the default to report on action items (actual changes) and warnings:
# cf-agent -I
# cf-agent --inform
By using CFEngine with the verbose flag, you can alter the default to report all of its thought-processes. You should not interpret a message that only appears in CFEngine's verbose mode as an actual error, only as information that might be relevant to decisions being made by the agent:
# cf-agent -v
# cf-agent --verbose
Creating custom reports
CFEngine allows you to use reports
promises to make reports of your own. A simple
example of this is shown below.
body common control
{
bundlesequence => { "test" };
}
#
bundle agent test
{
reports:
cfengine_3::
"$(sys.date),This is a report"
report_to_file => "/tmp/test_log";
}
We can apply this idea to make more useful custom reports. In this example, the agent tests for certain software package and creates a simple HTML file of existing software:
body common control
{
bundlesequence => { "test" };
}
#
bundle agent test
{
vars:
"software" slist => { "gpg", "zip", "rsync" };
classes:
"no_report" expression => fileexists("/tmp/report.html");
"have_$(software)" expression => fileexists("/usr/bin/$(software)");
reports:
no_report::
"
<html>
Name of this host is: $(sys.host)<br>
Type of this host is: $(sys.os)<br>
"
report_to_file => "/tmp/report.html";
#
"
Host has software $(software)<br>
"
ifvarclass => "have_$(software)",
report_to_file => "/tmp/report.html";
#
"
</html>
"
report_to_file => "/tmp/report.html";
}
The outcome of this promise is a file called /tmp/report.html which contains the following output:
<html>
Name of this host is: atlas<br>
Type of this host is: linux<br>
Host has software gpg<br>
Host has software zip<br>
Host has software rsync<br>
</html>
The mechanism shown above can clearly be used to create a wide variety of report formats, but it requires a lot of coding and maintenance by the user.
Including data in reports
CFEngine generates information internally that you might want to use in reports.
For example, the agent cf-agent
interfaces with the local light-weight monitoring agent
cf-monitord
so that system state can be reported simply:
body common control
{
bundlesequence => { "report" };
}
###########################################################
bundle agent report
{
reports:
linux::
"/etc/passwd except $(const.n)"
showstate => { "otherprocs", "rootprocs" };
}
A bonus to this is that you can get CFEngine to report system anomalies:
reports:
rootprocs_high_dev2::
"RootProc anomaly high 2 dev on $(mon.host) at approx $(mon.env_time)
measured value $(mon.value_rootprocs)
average $(mon.average_rootprocs) pm $(mon.stddev_rootprocs)"
showstate => { "rootprocs" };
entropy_www_in_high&anomaly_hosts.www_in_high_anomaly::
"High entropy incoming www anomaly on $(mon.host) at $(mon.env_time)
measured value $(mon.value_www_in)
average $(mon.average_www_in) pm $(mon.stddev_www_in)"
showstate => { "incoming.www" };
This produces the following standard output:
R: State of otherprocs peaked at Tue Dec 1 12:12:21 2014
R: The peak measured state was q = 98:
R: Frequency: [kjournald] |** (2/98)
R: Frequency: [pdflush] |** (2/98)
R: Frequency: /var/cfengine/bin/cf-execd|** (2/98)
R: Frequency: COMMAND |* (1/98)
R: Frequency: init [5] |* (1/98)
R: Frequency: [kthreadd] |* (1/98)
R: Frequency: [migration/0] |* (1/98)
R: Frequency: [ksoftirqd/0] |* (1/98)
R: Frequency: [events/0] |* (1/98)
R: Frequency: [khelper] |* (1/98)
R: Frequency: [kintegrityd/0] |* (1/98)
Finally, you can quote lines from files in your data for convenience:
body common control
{
bundlesequence => { "report" };
}
###########################################################
bundle agent report
{
reports:
linux::
"/etc/passwd except $(const.n)"
printfile => pr("/etc/passwd","5");
}
######################################################################
body printfile pr(file,lines)
{
file_to_print => "$(file)";
number_of_lines => "$(lines)";
}
This produces the following output:
R: /etc/passwd except
R: at:x:25:25:Batch jobs daemon:/var/spool/atjobs:/bin/bash
R: avahi:x:103:105:User for Avahi:/var/run/avahi-daemon:/bin/false
R: beagleindex:x:104:106:User for Beagle indexing:/var/cache/beagle:/bin/bash
R: bin:x:1:1:bin:/bin:/bin/bash
R: daemon:x:2:2:Daemon:/sbin:/bin/bash
Creating custom logs
Logs can be attached to any promise. In this example, an executed shell command logs a message to the standard output. CFEngine recognizes thestdoutfilename for Standard Output, in the Unix/C standard manner:
bundle agent test
{
commands:
"/tmp/myjob",
action => logme("executor");
}
############################################
body action logme(x)
{
log_repaired => "stdout";
log_string => " -> Started the $(x) (success)";
}
In the following example, a file creation promise logs different outcomes (success or failure) to different log files:
body common control
{
bundlesequence => { "test" };
}
bundle agent test
{
vars:
"software" slist => { "/root/xyz", "/tmp/xyz" };
files:
"$(software)"
create => "true",
action => logme("$(software)");
}
#
body action logme(x)
{
log_kept => "/tmp/private_keptlog.log";
log_failed => "/tmp/private_faillog.log";
log_repaired => "/tmp/private_replog.log";
log_string => "$(sys.date) $(x) promise status";
}
This generates three different logs with the following output:
atlas$ more /tmp/private_keptlog.log
Sun Dec 6 11:58:16 2009 /tmp/xyz promise status
Sun Dec 6 11:58:43 2009 /tmp/xyz promise status
Redirecting output to logs
CFEngine interfaces with the system logging tools in different ways. Syslog
is the
default log for Unix-like systems, while the event logger
is the default on Windows.
You may choose to copy a fixed level of CFEngine's standard screen messaging to the
system logger on a per-promise basis:
body common control
{
bundlesequence => { "one" };
}
bundle agent one
{
files:
"/tmp/xyz"
create => "true",
action => log;
}
body action log
{
log_level => "inform";
}
Change detection: tripwires
Doing a change detection scan is a convergent process, but it can still detect changes and present the data in a compressed format that is often more convenient than a full-scale audit. The result is less precise, but there is a trade-off between precision and cost.
To make a change tripwire, use a files promise, as shown below:
body common control
{
bundlesequence => { "testbundle" };
}
#
bundle agent testbundle
{
files:
"/home/mark/tmp" -> "me"
changes => scan_files,
depth_search => recurse("inf");
}
# library code ...
body changes scan_files
{
report_changes => "all";
update_hashes => "true";
}
body depth_search recurse(d)
{
depth => "$(d)";
}
In CFEngine Enterprise, reports of the following form are generated when these promises are kept by the agent:
Change detected File change
Sat Dec 5 18:27:44 2013 group for /tmp/testfile changed 100 -> 0
Sat Dec 5 18:27:44 2013 /tmp/testfile
Sat Dec 5 18:20:45 2013 /tmp/testfile
These reports are generated automatically in Enterprise, and are integrated into the
web-browsable knowledge map. Community edition users must extract the data and create
these themselves.
<!--- End include: /home/jenkins/workspace/build-documentation-3.9/label/DOCUMENTATION_x86_64_linux_debian_6/documentation/guide/reporting/command-line-reports.markdown
-->
FAQ
- Enterprise Installation
- Enterprise Scalability
- Policy Distribution
- Manual Execution
- Agent Email Reports
- Policy Writing
Enterprise Installation and Configuration
What steps should I take after installing CFEngine Enterprise
There are general steps to be taken outlined in Post-Installation Configuration.
In addition to this, Enterprise 3.6 uses the local mail relay, and it is assumed that the server where CFEngine Enterprise is installed on has proper mail setup.
The default FROM email for all emails sent from the Mission Portal is currently admin@organization.com. This can be changed on the CFEngine Hub in /var/cfengine/httpd/htdocs/application/config/appsettings.php:$config['appemail']
.
Can I use an existing PostgreSQL installation
No. Although CFEngine keeps its assumptions about Postgres to a bare minimum, CFEngine should use a dedicated PostgreSQL database instance to ensure there is no conflict with an existing installation.
Do I need experience with PostgreSQL
PostgreSQL is highly configurable and you should have some in-house expertise to properly configure your database installation. The defaults are well tuned for common cases but you may find optimizations depending on your hardware and OS.
What is the system user for the CFEngine dedicated PostgreSQL database and Apache server
Starting with CFEngine 3.6 there will be a system user called cfpostgres
for running the dedicated CFEngine PostgreSQL database
installation.
Similarly there will be a cfapache
system user for the Apache web server.
What are the requirements for installing CFEngine Enterprise
General Information
Users and Permissions
- CFEngine Enterprise makes an attempt to create the local users
cfapache
andcfpostgres
, as well as groupcfapache
when installing 3.6. The server must allow creation of these users and groups.
Enterprise Scalability
Is it normal to have many cf-hub processes running
- Yes, it is expected to have ~ 50 cf-hub processes running on your hub
Policy Distribution
I have added new files in masterfiles but my remote clients are not getting updates
Check that the files you expect to be distributed have matching leaf_name
pattern. If newly bootstrapped clients get those files but existing clients don't, this is certainly the problem, because bootstrapping and failsafe operation ignore leaf_name
and copy everything.
In CFEngine 3.6 masterfiles policy framework this is configurable with
input_name_patterns
in the update_def
bundle in def.cf
. See The Policy Framework for more information.
I have updated some non policy files and changes are not distributed to clients
cf_promises_validated
gates client updates. This file is only updated on the
policy server when new policy is validated. Edits to non policy files do not
trigger an update of cf_promises_validated
. You can use a separate promise to
ensure those files are continually distributed, instead of only on policy
updates.
For details see cf_promises_validated and cfe_internal_update_policy
My policy server has changed its IP address and new bootstraps don't work!
(thanks to Dan Langille in https://groups.google.com/forum/#!topic/help-cfengine/jcdIh12_lNI)
Symptom:
After the policy server was restarted with the new IP address, clients would not connect:
error: Not authorized to trust public key of server '192.168.14.113' (trustkey = false)
error: Authentication dialogue with '192.168.14.113' failed
Bootstrapping the clients also fails:
[root@dev /var/cfengine] /var/cfengine/bin/cf-agent --bootstrap 192.168.14.113
2014-06-23T13:57:07-0400 notice: R: This autonomous node assumes the role of voluntary client
2014-06-23T13:57:07-0400 notice: R: Failed to copy policy from policy server at 192.168.14.113:/var/cfengine/masterfiles
Please check
* cf-serverd is running on 192.168.14.113
* network connectivity to 192.168.14.113 on port 5308
* masterfiles 'body server control' - in particular allowconnects, trustkeysfrom and skipverify
* masterfiles 'bundle server' -> access: -> masterfiles -> admit/deny
It is often useful to restart cf-serverd in verbose mode (cf-serverd -v) on 192.168.14.113 to diagnose connection issues.
When updating masterfiles, wait (usually 5 minutes) for files to propagate to inputs on 192.168.14.113 before retrying.
2014-06-23T13:57:07-0400 notice: R: Did not start the scheduler
2014-06-23T13:57:07-0400 error: Bootstrapping failed, no input file at '/var/cfengine/inputs/promises.cf' after bootstrap
Solution:
Assuming that 661df12c960af9afdde093e0cb339b4d
is the MD5 hostkey and 192.168.14.113
is the new IP address:
cd /var/cfengine/ppkeys && mv -i root-MD5=661df12c960af9afdde093e0cb339b4d.pub root-192.168.14.113.pub
Manual Execution
How do I run a standalone policy file
The --file
or -f
option to cf-agent
specifys the policy file. The -K
or --no-lock
flag and the -I
or --inform
options are commonly used in combination with the -f
option to ensure that
all promises are skipped because of locking and for the agent to produce
informational output like successful repairs.
cf-agent -KIf ./my_standalone_policy.cf
In 3.6.1 and later, a standalone policy file may choose not to
specify a bundlesequence
. In that case, the bundlesequence
defaults to main
so you'll need a bundle called main
.
You can avoid that requirement by using the -b BUNDLENAME
flag which
specifies an explicit bundlesequence
, see below.
Why do I get Undefined body when I try to run my policy
cf-promises -f ./large-files.cf
:
./large-files.cf:14:0: error: Undefined body tidy with type delete
./large-files.cf:16:0: error: Undefined body recurse with type depth_search
The above errors indicate that the tidy
and recurse
bodies are not able
to be found by CFEngine. This is because they are not found in the file or in
one of the files it includes. Either define the body within the same policy
file or include the file that defines the body using inputs in either body
common control or body file
control.
Example: Add stdlib via body common control
body common control
{
bundlesequence => { "file_remover" };
inputs => { "$(sys.libdir)/stdlib.cf" };
}
Example: Add stdlib via body file control Body file control allows you to build modular policy. Body file control inputs are typically relative to the policy file itself.
bundle file_remover_control
{
vars:
"inputs" slist => { "$(this.promise_dir)/$(sys.local_libdir)/stdlib.cf" };
}
body file control
{
inputs => { @(file_remover_control.inputs) };
}
This policy will work correctly whether it's included by another
policy file or not. Note the body file
control
option is new since
CFEngine 3.6, so you should not use if your policy could be seen by
3.5 or earlier CFEngine clients.
How do I run a specific bundle
A specific bundle can be activated by passing the -b
or --bundlesequence
options to cf-agent
. This may be used to activate a specific bundle within a
large policy set or to run a standalone policy that does not include a body
common control
.
cf-agent -b my_bundle
If you want to activate multiple bundles in a sequence simply separate them with commas (no spaces between).
cf-agent --bundlesequence bundle1,bundle2
How do I define a class for a single run
You can use the --define
or -D
options of cf-agent
.
cf-agent -D my_class
And if you want to define multiple, simply separate them with commas (no spaces between).
cf-agent --define my_class,my_other_class
Multiple -D
flags are not supported, you have to put all the classes in one comma-separated list.
Showing Classes and variables with cf-promsies
cf-promises --show-classes
and cf-promises --show-vars
will only show
classes and variables found on a first pass through the policy, since
cf-promises
does not evaluate agent promises.
Agent Email Reports
How do I set the email where agent reports are sent
The agent report email functionality is configured in body executor control
https://github.com/cfengine/masterfiles/blob/master/controls/cf_execd.cf. It
defaults to root@$(def.domain)
which is configured in bundle common def
https://github.com/cfengine/masterfiles/blob/master/def.cf.
For details see domain.
How do I disable agent email output
You can simply remove or comment out the settings.
In 3.6.x there is a convenience class cfengine_internal_agent_email
avaiable
in bundle common def
to switch on/off agent email.
For details see cfengine_internal_agent_email.
Mustache Templating
How can I pass a data variable to template_data?
Just use template_data => @(mycontainer)
.
If you need to extract a portion of the container or merge it with another, use
template_data => mergedata("mycontainer[piece]", "othercontainer")
.
Can I render a Mustache template into a string?
Yes, see string_mustache()
.
How do I render a section only if a given class is defined?
In this Mustache example the word 'Enterprise' will only be rendered if the class 'enterprise' is defined.
This template should not be passed a data container; it uses the datastate()
of the CFEngine system. That's where classes.enterprise
and vars.sys.cf_version
came from.
Version: CFEngine Enterprise
How do I iterate over a list?
This template should not be passed a data container; it uses the datastate()
of the CFEngine system. That's where vars.mon.listening_tcp4_ports
came from.
{{#vars.mon.listening_tcp4_ports}}
* {{.}}
{{/vars.mon.listening_tcp4_ports}}
How do I ensure that a local user is locked?
To ensure that a local user exists but is locked (for example a service
account) simply specify policy => "locked"
.
bundle agent service_accounts
{
vars:
"users" slist => { "apache", "libuuid" };
users:
!windows::
"$(users)"
policy => "locked";
}
Policy Writing
Common questions asked about policy writing.
How do I pass a data type variable
Data type variables also known as "data containers" are passed using the same syntax as passing a list.
bundle agent example
{
vars:
# First you must have a data type variable, define it inline or read from a
# file using `readjson()`.
"data" data => parsejson('[ { "x": 1 }, { "y": 2 } ]');
methods:
"use data"
usebundle => use_data(@(data));
}
bundle agent use_data(dc)
{
vars:
# Use the data
# Get its keys, or its index
"dc_index" slist => getindices(dc);
classes:
"have_x" expression => isvariable("dc[$(dc_index)][x]");
"have_z" expression => isvariable("dc[$(dc_index)][z]");
reports:
"CFEngine version '$(sys.cf_version)'";
have_x::
"Index '$(dc_index)' has key for x";
have_z::
"Index '$(dc_index)' has key for z";
}
$ cf-agent -Kf ./example.cf -b example
R: CFEngine version '3.6.4'
R: Index '0' has key for x
R: Index '1' has key for x
Enterprise report collection
How does CFEngine Enterprise collect reports?
cf-hub
makes connections from the hub to remote agents currently registered in
the lastseen database (viewable with cf-key -s
)
on [body hub control port
]body hub control port. The hub
tries to collect from up to the licensed number of hosts for each collection
round as identified by hub_schedule
as defined
in body hub control
.
Note: No ordering is specified, so if the number of entries in the lastseen database is greater than the number of licensed hosts it is not possible to determine which hosts will be collected from and which hosts will be skipped.
See Also:
hostsseen()
,hostswithclass()
How are agents not running determined?
Hosts who's last agent execution status is "FAIL" will show up under "Agents not running". A hosts last agent execution status is set to "FAIL" when the hub notices that there are no promise results within 3x of the expected agent run interval. The agents average run interval is computed by a geometric average based on the 4 most recent agent executions.
You can inspect hosts last execution time, execution status (from the hubs perspective), and average run interval using the following SQL.
SELECT Hosts.HostName AS "Host name",
AgentStatus.LastAgentLocalExecutionTimeStamp AS "Last agent local execution
time", cast(AgentStatus.AgentExecutionInterval AS integer) AS "Agent execution
interval", AgentStatus.LastAgentExecutionStatus AS "Last agent execution status"
FROM AgentStatus INNER JOIN Hosts ON Hosts.HostKey = AgentStatus.HostKey
This can be queried over the API most easily by placing the query into a json
file. And then using the query
API.
agent_execution_time_interval_status.query.json
:
{
"query": "SELECT Hosts.HostName, AgentStatus.LastAgentLocalExecutionTimeStamp, cast(AgentStatus.AgentExecutionInterval AS integer), AgentStatus.LastAgentExecutionStatus FROM AgentStatus INNER JOIN Hosts ON Hosts.HostKey = AgentStatus.HostKey"
}
$ curl -s -u admin:admin http://hub/api/query -X POST -d @agent_execution_time_interval_status.query.json | jq ".data[0].rows"
[
[
"hub",
"2016-07-25 16:53:23+00",
"296",
"OK"
],
[
"host001",
"2016-07-25 16:06:50+00",
"305",
"FAIL"
]
]
See Also: Enterprise API Reference
, Enterprise API Examples
How are hosts not reporting determined?
Hosts that have not been collected from within blueHostHorizon
seconds will
show up under "Hosts not reporting".
blueHostHorizon
defaults to 900 seconds (15 minutes). You can inspect the
current value of blueHostHorizon
from Mission Portal or via the API:
$ curl -s -u admin:admin http://hub/api/settings/ | jq ".data[0].blueHostHorizon"
900
See Also: Enterprise API Reference
, Enterprise API Examples
, Enterprise Settings
Troubleshooting report collection
The following steps can be used to help diagnose and potentially restore reporting for hosts experiencing issues.
Perform manual delta collection for a single host
Performing back to back delta collections and comparing the data received can help to expose so called patching issues. If the same amount of data is collected twice a rebase may resolve it.
[root@hub ~]# cf-hub -q delta -H 192.168.33.2 -v
verbose: ----------------------------------------------------------------
verbose: Initialization preamble
verbose: ----------------------------------------------------------------
# <snipped for brevity>
verbose: Connecting to host 192.168.33.2, port 5308 as address 192.168.33.2
verbose: Waiting to connect...
verbose: Setting socket timeout to 10 seconds.
verbose: Connected to host 192.168.33.2 address 192.168.33.2 port 5308 (socket descriptor 4)
verbose: TLS version negotiated: TLSv1.2; Cipher: AES256-GCM-SHA384,TLSv1/SSLv3
verbose: TLS session established, checking trust...
verbose: Received public key compares equal to the one we have stored
verbose: Server is TRUSTED, received key 'SHA=e77d408e9802e2c549417d5e3379c43050d2ad5928a198855dbb7e9c8af9a6f1' MATCHES stored one.
verbose: Key digest for address '192.168.33.2' is SHA=e77d408e9802e2c549417d5e3379c43050d2ad5928a198855dbb7e9c8af9a6f1
verbose: Will request from host 192.168.33.2 (digest = SHA=e77d408e9802e2c549417d5e3379c43050d2ad5928a198855dbb7e9c8af9a6f1) data later than timestamp 1481901790
verbose: Successfully opened extension plugin 'cfengine-report-collect.so' from '/var/cfengine/lib/cfengine-report-collect.so'
verbose: Successfully loaded extension plugin 'cfengine-report-collect.so'
verbose: Sending query at Fri Dec 16 15:24:23 2016
verbose: h>s QUERY delta 1481901790 1481901863
verbose: Sending query at Fri Dec 16 15:24:23 2016
verbose: Received reply of 5050 bytes at Fri Dec 16 15:24:23 2016 -> Xfer time 0 seconds (processing time 0 seconds)
verbose: Processing report: MOM (items: 44)
verbose: Processing report: MOY (items: 48)
verbose: Processing report: MOH (items: 22)
verbose: Processing report: EXS (items: 1)
verbose: Received 5 kb of report data with 115 individual items
verbose: Connection to 192.168.33.2 is closed
Perform manual rebase collection for a single host
A rebase
causes the hub to throw away all reports since the last collection
and collect only the output from the most recent run.
[root@hub ~]# cf-hub -q rebase -H 192.168.33.2 -v
verbose: ----------------------------------------------------------------
verbose: Initialization preamble
verbose: ----------------------------------------------------------------
# <snipped for brevity>
verbose: Connecting to host 192.168.33.2, port 5308 as address 192.168.33.2
verbose: Waiting to connect...
verbose: Setting socket timeout to 10 seconds.
verbose: Connected to host 192.168.33.2 address 192.168.33.2 port 5308 (socket descriptor 4)
verbose: TLS version negotiated: TLSv1.2; Cipher: AES256-GCM-SHA384,TLSv1/SSLv3
verbose: TLS session established, checking trust...
verbose: Received public key compares equal to the one we have stored
verbose: Server is TRUSTED, received key 'SHA=e77d408e9802e2c549417d5e3379c43050d2ad5928a198855dbb7e9c8af9a6f1' MATCHES stored one.
verbose: Key digest for address '192.168.33.2' is SHA=e77d408e9802e2c549417d5e3379c43050d2ad5928a198855dbb7e9c8af9a6f1
verbose: Successfully opened extension plugin 'cfengine-report-collect.so' from '/var/cfengine/lib/cfengine-report-collect.so'
verbose: Successfully loaded extension plugin 'cfengine-report-collect.so'
verbose: Sending query at Fri Dec 16 15:35:10 2016
verbose: h>s QUERY rebase 0 1481902510
verbose: Sending query at Fri Dec 16 15:35:10 2016
verbose: Received reply of 128157 bytes at Fri Dec 16 15:35:10 2016 -> Xfer time 0 seconds (processing time 0 seconds)
verbose: Processing report: CLD (items: 46)
verbose: Processing report: VAD (items: 52)
verbose: Processing report: LSD (items: 13)
verbose: Processing report: SDI (items: 327)
verbose: Processing report: SPD (items: 143)
verbose: Processing report: ELD (items: 205)
verbose: ts #0 > 1481902510
verbose: Received 125 kb of report data with 786 individual items
verbose: Connection to 192.168.33.2 is closed
Note: The Enterprise hub automatically schedules rebase queries if it has
been unable to collect from a given candidate for client_history_timeout
hours.
If a manual rebase collection does not restore reporting functionality for a host continue on to restarting the report collection components.
Restart report collection components
Sometimes it is necessary to restart the report collection subsystem in order to
re-synchronize the caching layer with the database. To restart the report
collection subsystem simply kill cf-hub
, cf-consumer
, redis-server
, and
run the update policy.
For systemd hosts this can be accomplished by simply restarting the cf-hub
service. The related component restarts are automatically handled via the unit
dependencies:
[root@hub ~]# systemctl restart cf-hub
For non-systemd hosts:
[root@hub ~]# pkill cf-consumer
[root@hub ~]# pkill cf-hub
[root@hub ~]# pkill redis-server
[root@hub ~]# cf-agent -KIf update.cf
info: Executing 'no timeout' ... '/var/cfengine/bin/redis-server /var/cfengine/config/redis.conf'
info: Command related to promiser '/var/cfengine/bin/redis-server /var/cfengine/config/redis.conf' returned code defined as promise kept 0
info: Completed execution of '/var/cfengine/bin/redis-server /var/cfengine/config/redis.conf'
info: Executing 'no timeout' ... '/var/cfengine/bin/cf-consumer'
info: Command related to promiser '/var/cfengine/bin/cf-consumer' returned code defined as promise kept 0
info: Completed execution of '/var/cfengine/bin/cf-consumer'
info: Executing 'no timeout' ... '"/var/cfengine/bin/cf-hub"'
info: Command related to promiser '"/var/cfengine/bin/cf-hub"' returned code defined as promise kept 0
info: Completed execution of '"/var/cfengine/bin/cf-hub"'
Additional Resources
Use the following links to learn more about CFEngine:
Reading
Learn by reading information brought to you by CFEngine experts:
Learning CFEngine 3 by Diego Zamboni
CFEngine 3 Tutorial and Cookbook by Neil Watson, a Senior UNIX/Linux system admin and a CFEngine Champion.
CFEngine Resources by Vertical Sysadmin, Inc, a sysadmin training company and an authorized CFEngine training partner.
CFEngine Development blog Posts on configuration management best practices from the CFEngine team.
Training
Online Training An introduction to CFEngine by our founder, Mark Burgess. These video recordings explain the basic principles and syntax of the CFEngine language and suggests some examples to try out.
Demos Videos, Webinars, and Keynotes which demonstrate the key capabilities of CFE Enterprise and Community editions.
CFEngine as a powerful security tool: Three presentations by Diego Zamboni
Tools
Download CFEngine code editors.
Sign Up
On-Site Training Sign up for professional training courses that provide a better understanding of CFEngine and how it can help improve configuration management in your organization.
Contact us to get more info on training courses.
Support and Community
Support Desk
- CFEngine Enterprise Support Desk Enterprise users have access to our support desk.
Forums
Help from our CFEngine community is available to all users on our Google Groups forums:
Support for CFEngine Enterprise users Help for users who have downloaded the free version of CFEngine 3 Enterprise.
help-cfengine General help for all your CFEngine questions.
Learning Resources
Sometimes the best help is already written.
- Visit our learning resources for guides, demos, training videos, and tools.
Social Media
Stay in touch. Follow us:
The #cfengine IRC channel on the irc.freenode.org network.
If you want to learn more about how CFEngine can help you and your organization, contact us.
Contribute to CFEngine
CFEngine Github
Public Bug Tracker
- Bugs and improvement suggestions can be registered with our development team in our public bug tracker. Read the bug tracker information before you submit a bug.