Table of Contents
ChangeLog
Table of Contents
See Also: Enterprise Changelog, Masterfiles Changelog
3.14.0:
- A bootstrap_id.dat file is now generated on every bootstrap
(CFE-2977)
- Added options to cf-net to set minimum TLS version and ciphers
(ENT-4617)
- Added --no-truncate option to cf-key
This option, when used with --show-hosts changes the formatting
of the output. Instead of padding and truncating each of the
fields, they are printed, in full, with no padding, and separated
by a single tab character. The output is not as pretty, but should
be more useful for parsing by other scripts / tooling. (CFE-3036)
- Added a new option --skip-db-check to agent and execd
This option allows you to enable/disable database (LMDB) consistency
checks. Initially it is disabled by default, but this will likely
change. (CFE-2893)
- Added a new utility to contrib: cf-remote
cf-remote is a python + fabric tool to log in to remote hosts
you have ssh access to. It can be used to download, transfer,
and install cfengine packages as well as bootstrapping etc.
At this point, cf-remote is not packaged with CFEngine, but can be
installed separately from:
https://github.com/cfengine/core/tree/master/contrib/cf-remote
(CFE-2889)
- Added derived-from-file tags to hard classes based on /etc/debian_version and /etc/issue
- Added a function to filter CSV-files by classes (CFE-2768)
- Forward slash is now an allowed character in module protocol commands
(CFE-2478)
- Augments files can now handle class expressions by appending '::'
A condition in an augments file is treated as a class expression
if it ends in `::`. Otherwise it is treated as a regular
expression. (CFE-2954)
- Internal ps command can now handle longer usernames (CFE-2951)
- Made copylink_pattern honor '/../' in copy source (CFE-2960)
- CSV parser now supports CRLF inside double quotes (ENT-4504)
- Added an error when a function defining a variables still fails at
pass 3 (CFE-2983)
- Documented cf-execd and cf-serverd response to SIGHUP in manpage
(CFE-2853)
- Stopped trimming leading zeroes in ubuntu minor version class
The old version detection logic (using /etc/debian_version) was
converting the minor version part to an integer, defining
ubuntu_18_4 instead of ubuntu_18_04. The new platform detection
(based on /etc/os-release) defines ubuntu_18_04. Since both old
and new methods are running to maximize compatibility, both
ubuntu_18_04 and ubuntu_18_4 were defined.
This commit ensures that the old detection logic treats the
minor version (the 04 part) as a string, not an integer. The
change is specific to Ubuntu, and should affect Ubuntu 18.04,
16.04, 14.04, etc. (CFE-2882)
- SUID log permissions are now properly enforced (CFE-2919)
- Agent log file names are now always lowercase
- Extended module with file protocol for data (CFE-3050)
- Fixed a segfault in 'cf-promises -p json-full' (CFE-3019)
- Added cf-key help output to indicate ability to delete by key digest
(CFE-2997)
- Fixed disabling TLS 1.0 (CFE-3068)
- Fixed growing memory footprint of daemons (CFE-3032)
- Fixed the log message about setting collect_window (ENT-4238)
- Fixed the log message when parsing TIME in 'ps' output fails
- Fixed parsing of YAML values starting with numbers (CFE-2033)
- Fixed sys.flavor on AIX (ENT-3970)
- Fixed 6 cases where promises could get the wrong outcome
All cases were related to error handling and detected using
static code analysis (LGTM). They were limited to cf-monitord
and cf-agent (guest_environments and files promise types). Due
to a programming mistake, promise results would sometimes be
overwritten with 'skipped' outcome. Keeping the previous
value or making the promises 'not kept' is expected behavior.
Added a query to our CI (LGTM) to make sure we catch this error
on new contributions.
- Fixed an issue while parsing ps output on AIX (ENT-4295)
- Fixed a memory leak in filesexist function (ENT-4313)
- Fixed a memory leak in mustache rendering (ENT-4313)
- Fixed a memory leak in: differences(), intersection(), unique()
(ENT-4586)
- Fixed a segfault in policy parser (ENT-4022)
- Connection cache is now global (CFE-2678)
- Increased verbosity of AcquireLock permission error (ENT-4395)
- Message about invalid class characters from module protocol moved to VERBOSE
(CFE-2887, CFE-3008)
- Prevented buffer overflows in cf-monitord data parsing
- Private keys generated by cf-key are no longer encrypted
Private key files encrypted with a broken cipher and default
hard coded passphrase provide no real security, and is only an
inconvenience. Maybe it was intended to add a password prompt
later, but it's been 10 years now, and the cipher and passphrase
remain untouched. The function which reads keys still supports
both encrypted and unencrypted keys, it will decrypt if necessary.
- Reduce SSL/TLS shutdowns on bad networks (CFE-3023)
- Removed programming error in handling of process_count body
Previously, having a failing function call inside in_range_define
or out_of_range_define would cause a programming error when
trying to define that as a class. Fixed it by detecting the
case, printing a normal error, and skipping defining the class. (CFE-2067)
- Set policy->release_id to "failsafe"/"bootstrap" when running failsafe.cf
(CFE-3031)
- Switched permissions of various temporary files in state to 0600
These files were created with 0644 permissions, and then
repaired in policy. However, since they are deleted / recreated
periodically, it causes INFO noise. Safer and better user
experience to create them with restricted permissions to
begin with.
Affected files:
* `$(sys.statedir)/cf_procs`
* `$(sys.statedir)/cf_rootprocs`
* `$(sys.statedir)/cf_otherprocs`
(ENT-4601)
- string_split segments are now truncated to 1024 bytes instead of
crashing (CFE-3047)
- Unresolved function calls in process_select body are now skipped
Function calls which always fail, like getuid("nosuchuser"), are
never resolved. Previously this would cause a programming error,
since the body is expected to have a list of strings, not
unresolved function calls.
The function calls are silently skipped (with a verbose message)
as this matches the behavior of calling the functions in a vars
promise, and using that as a body parameter.
(CFE-1968)
- cf-check directories can now be controlled from ENV vars (CFE-2994)
- cf-check: Added backup command
This command copies lmdb files to a timestamped backup directory.
(ENT-4064)
- cf-check: diagnose and backup now use state directory by default
(ENT-4064)
3.13.0:
- Add support for TLS 1.3 and its ciphersuites
- Add 'feature' hard classes for supported TLS versions
Different versions of TLS are supported depending on what version
of OpenSSL CFEngine was compiled and linked with. Newly added
feature hard classes bring that information to the
policy. Classes like these are now defined (for supported
versions of TLS):
feature_tls source=agent,hardclass
feature_tls_1 source=agent,hardclass
feature_tls_1_0 source=agent,hardclass
feature_tls_1_1 source=agent,hardclass
feature_tls_1_2 source=agent,hardclass
feature_tls_1_3 source=agent,hardclass
- Add a new variable $(sys.default_policy_path)
A new sys variable that provides the path of the default policy
file evaluated when no file is specified with the '-f' option.
- Add an option to skip the initial policy run on bootstrap
In some cases it may not be desired to run the policy as the last
step of the bootstrap. This can be done with the new
'--skip-bootstrap-policy-run' option for cf-agent. (CFE-2826)
- Trigger promises.cf as the last step of bootstrap (CFE-2826)
- Add support for overriding the package module's path (CFE-2103)
- Add support for setting package module interpreter (CFE-2880)
- Added --log-level option to all components
This allows you to specify any log level (info, verbose, debug etc.).
It is also less strict, allowing different spelling. As an example,
--log-level i, --log-level INFO, --log-level inform are all the same.
- Added a new binary: cf-check
Corrupt local databases (LMDB) continues to be a problem.
cf-check will be used to diagnose and remediate problems
with corrupt databases. It is a standalone binary, which
doesn't evaluate policy or use the local databases, thus
it can be used in situations where the other binaries
like cf-agent would hang.
cf-check replaces our lmdb database dumper, lmdump.
`cf-check lmdump` or symlinking / renaming it to lmdump
will make cf-check have the exact same behavior as lmdump.
cf-check will include much more functionality in the future
and some of the code will be added to other binaries,
for example to do health checks of databases on startup.
Ticket: (ENT-4064)
- Added function string_replace. (CFE-2850)
- Allow dots in variable identifiers with no such bundle
As described and discussed in CFE-1915, defining remote variables
(injecting variables into remote bundles) is dangerous and must
be blocked. However, using a dot-separated common prefix for
variables raises no security concerns and can be considered
valid. (CFE-1915)
- Allow requiring TLS 1.3 as the minimum version
- Apply augments after vars, classes and inputs in def.json
(CFE-2741, CFE-2844)
- Bundle name is now part of the log message when aborting a bundle
(CFE-2793)
- Class names set by module protocol are automatically canonified
(CFE-2877, CFE-2887)
- Classes failsafe_fallback and bootstrap_mode are now reported by default
- Correct log level for data_readstringarray* (CFE-2922)
- Do not iterate over JSON objects' properties in mustache (CFE-2125)
- Do not render templates when passed invalid data (CFE-2194)
- Eliminated error messages caused by attempting to kill expired processes
(CFE-2824)
- Fix cf-runalerts systemd unit conditions so the service will run
(ENT-3929)
- Fix the off-by-one error in cf-runagent background process spawning
(CFE-2873)
- Fixed OOB read / heap buffer overflow in evaluator (ENT-4136)
- Fixed a memory leak which occured when reloading RSA keys from disk
(CFE-2857)
- Fixed a memory leak which occured while loading augments files
(CFE-2913)
- Fixed an issue with splay time in cf-execd (CFE-2931)
- Fixed error handling and memory leak in cf-key (CFE-2918)
- Fixed memory leak in JSON to policy conversion (ENT-4136)
- Fixed memory leak in lmdb cleanup (CFE-2918)
- Fixed memory leaks in cf-agent during bootstrap (CFE-2918)
- Fixed memory leaks in variablesmatching() and findfiles() (CFE-2918)
- Fixed missing class with mustache templates in warn_only mode
(CFE-2600)
- Fixed small memory leak in cf-serverd (CFE-2918)
- Fixed small memory leak in cf-upgrade (ENT-4136)
- Fixed small memory leaks of environment variable strings (CFE-2918)
- LMDB database dumper, lmdump, no longer creates empty databases
(ENT-4064)
- Made variablesmatching functions treat args regexes more correctly
variablesmatching() and variablesmatching_as_data() no longer
use string comparison to find matches. The documentation is clear;
arguments should be regexes (so you have to escape special
characters).
bundle agent main
{
vars:
"myvar"
string => "example",
meta => {"os[linux]"};
"matches"
slist => variablesmatching(".*", "os\[linux\]");
reports:
"Match: $(matches)";
}
The above example is correct. If you don't escape the brackets
like above, it will no longer work. (You probably shouldn't use
brackets in tags anyway).
- Prevent the init script from managing processes inside containers
(ENT-3800)
- Read mustache-rendered files in text mode when comparing digest
(ENT-2526)
- Reload persistent classes on config reload in cf-execd and cf-serverd
(CFE-2857)
- Fixed issue with @if macro failing when it is on the first line.
(CFE-2817)
- Fixed issue with cf-agent intermittently hanging on windows
sometimes (ENT-3756)
- change GIT_BRANCH to GIT_REFSPEC and remove Design Center vars
(ENT-4023)
- os-release file is now used for hard classes and sys.flavor on all linuxes
This will improve platform detection on newer operating systems where
/etc/os-release (or /usr/lib/os-release) is present.
A hard class will be set for the value of the ID key (canonified with
underscores), if it exists. If both ID and VERSION_ID exist, multiple
hard classes will be set for all parts of the version number. The
special variable sys.flavor will also be set by determining major
version from VERSION_ID.
Example os-release file:
ID=coreos
VERSION_ID=1185.3.0
For the example above, `sys.flavor` will be `coreos_1185` and 4 hard
classes will be set; `coreos_1185_3_0`, `coreos_1185_3`, `coreos_1185`,
and `coreos`.
For backwards compatibility, older distribution specific logic is still
executed and may overwrite sys.flavor and define hard classes as before.
- refactor use of atexit to use custom cleanup function instead. On Windows
atexit() unloads DLLs before and/or during atexit functions being called
which causes bad behavior. (ENT-3756)
3.12.0b1:
New Features:
- Add a --key-type option to specify RSA key size to cf-key
- New hash_to_int policy function (CFE-2733)
- Issue a warning on ignored locking attributes (CFE-2748)
- Add IPv6 hard classes with the "ipv6_" prefix (CFE-2310)
- Introduce "missing_ok" attribute in body copy_from
This allows to ignore missing sources in file copy operations (CFE-2365)
- Enable Xen hypervisor detection on all x86 platforms (CFE-2203)
- Add sys.policy_entry variables (CFE-2572)
- Added inline_mustache template method (CFE-1846)
- New component cf-net (cf-net is a CLI for the CFEngine network protocol,
useful for debugging, testing etc) and accompanying policy variable
sys.cf_net containing path to cf-net binary
Changes:
- Load augments at the end of context discovery
This means that classes defined as part of the context discovery
(e.g. 'am_policy_hub' and 'policy_server') can be used in the
augments (CFE-2482)
- Open measurements promise type from enterprise cf-monitord
- Transform filesexist() into a collecting function (CFE-2744)
- Load process table when actually needed for a processes promise (ENT-2536)
- Ignore commented out entries in fstab when edit_fstab is true (CFE-2198)
- Do not move obstructions in warn policy mode (CFE-2740)
- Made the max bytes parameter to file reading functions optional (CFE-2656)
- Do not tag large volatile variables for inventory
sys.interfaces_data, sys.inet and sys.inet6 are commonly larger than the
maximum data size allowed to be collected by cf-hub. Data larger than 1k
is truncated. Instead of reporting truncated data this change stops
tagging the variable so that it will not be collected to the Enterprise
hub and will not be available in Mission Portal. (ENT-3483)
- cf-execd now re-parses augments on policy reload (CFE-2406)
- Improve misleading verbose message
For constraints if/ifvarclass/unless, we now print the whole rval of the constraint.
Previously the message was just "skipping variable because ifvarclass is not defined" while the variable itself was defined.
Old message example:
verbose: Skipping promise 'mailto' because 'if'/'ifvarclass' is not defined
Changed to:
verbose: Skipping promise 'mailto' because 'ifvarclass => not(isvariable("mailto"))' is not defined
(CFE-2697)
- Promise comments for file changes moved to verbose (ENT-3414)
- Suppress output from systemctl based restart of services in
bootstrap/failsafe (CFE-1459)
- Parser can now handle larger inbut buffers (CFE-1886)
- Improve logging of ACL errors (ENT-3455)
- cf-execd systemd service now only kills cf-execd itself (ENT-3395)
- Load multiple augments from "augments" string array in def.json
(CFE-2084)
- Improve support for Alpine Linux
- Set the exit value when running cf-key
When running cf-key to generate new keys, set the exit value of the
program to be 0 on success and 1 on failure. This makes it easier to
catch errors during setup of a new machine.
Change the default behavior of the program to not write anything to stdout,
opting to use the Log() function which can write to stdout and will also
allow output to be sent to syslog.
Add a --inform option to set the global log level to LOG_LEVEL_INFO.
Change the permissions of the randseed file to 600 and catch the exception
if the chmod call fails.
- Properly reverse-resolve DNS names longer than 63 chars (ENT-3379)
- Properly redirect init script to systemd on debian systems (ENT-3326)
Bug fixes:
- Disallow modifications of variables from a remote bundle (CFE-1915)
- Speedup evalution by not copying variables table when expanding a promise
(CFE-2524)
- Resolve subkey conflicts when converting to JSON
Whenever there is a conflict of array variable definitions prefer
the container subkeys over simple values when converting to JSON
(CFE-2536)
- Do not ignore meta promises in server bundles (CFE-2066)
- Add a debug log for computed class in splayclass
- Don't error when calling isexecutable on broken link (CFE-741)
- Fix segfault when no show-evaluated-vars/classes is specified
- Fix memory leak in cf-execd, triggered when sending email failed (CFE-2712)
- Fix IPv6 parsing to be un-reversed (CFE-2580)
- Fix bug preventing permission changes on Unix sockets (CFE-1782)
- Fix storage mount promise when existing mountpoint has a similar path
(CFE-1960)
- Fix segfault when cf-promises -p is called against a file with syntax
errors (CFE-2696)
- Fix rare cf-execd hang (CFE-2719)
- Fix mergedata segfault when called on a non-container (CFE-2704)
- Do not segfault if policy_server.dat only contains whitespaces and/or line breaks
- Fix segfault on JSON policy files with no bundles and bodies (CFE-2754)
3.11.0:
New Features:
- Allow function calls in promiser using universal "with" attribute
(CFE-1092)
- Add example of with attribute (CFE-1092)
- Detect Amazon Linux and set "AmazonLinux" hard class and
sys.flavour variable
- New sysctlvalue() and data_sysctlvalues() functions from /proc/sys
(CFE-2513)
- readdata() also auto-detects .yml files as YAML
- Added support for ENV and CSV file parsing (CFE-1881)
- Added vars and classes for CoreOS (ENT-3043)
- cf-agent: implement --show-evaluated-vars and --show-evaluated-classes
- Support for custom ports and host names as policy hub (CFE-953)
- cf-promises: allows --show-vars and --show-classes to take an optional filter
- Added a new tool: cf-net. cf-net is a CLI for the CFEngine
network protocol, useful for debugging, testing etc (CFE-2493)
- New policy variable: sys.cf_net contains path to cf-net binary
- Read /etc/os-release into sys.os_release (CFE-1881)
Changes:
- readintlist() now prints an error if the
file contains real numbers, not integers, and aborts; previously it was
printing an info-level error message, was half-reading an integer out of
the real, and was continuing successfully.
- "make tar-package" should create a tarball with the contents of
"make install" (ENT-3041)
- Allow opening symlinks owned by root or by the current user
(CFE-2516)
- Change warning message about depth_search on a non directory to
DEBUG level
- Ensure synchronous start and stop with systmectl (ENT-2841)
- Put logs in /var/log and PID files in /var/run when using FHS layout
(CFE-2449)
- readstringlist(), readintlist(), readreallist(): Print
verbose instead of error message if file can not be read
- cf-serverd: Do not close connection when file does not exist
(CFE-2532)
- policy_server.dat now appends a newline and supports host & port
- Allow string_head and string_tail to take negative arguments
- getvalues(inexistent_var) returns an empty list.
Restores 3.7.x and earlier behaviour. (CFE-2479)
- Partially restore old getvalues(array) behaviour
Bugfix: getvalues() now behaves correctly for old CFEngine
arrays of depth 1
Behaviour change: it always returns a list now. Even when v is a simple
string (i.e. not an iterable) it will return an slist with one element:
the value of the string variable.
Known issues: getvalues() still misbehaves with double-indexed arrays
(see CFE-2504, CFE-2536)
- The source version of CFEngine now installs binaries into
bin folder instead of sbin folder (CFE-2448)
- Don't error during dry run for proposed execution (CFE-2561)
- Print verbose instead of error message when readfile() fails (CFE-2512)
- cf-serverd: Auto configure max open files ulimit according to
maxconnections (CFE-2575)
- Made the max bytes parameter to file reading functions optional.
Affects readfile(), readenvfile(), readcsv()
Bug fixes:
- Fix insert_lines related memory corruption (CFE-2520)
- Prevent LMDB assertion on AIX by ensuring nested DB calls are
not occuring during signal handler cleanup (CFE-1996)
- Fix a bug which could cause cf-execd to believe there was
an error when sending the email report, when there really wasn't
- zendesk#3204: Fix "lastseenexpireafter" 32-bit signed int overflow
- Fix cf-execd not exiting immediately with SIGTERM on AIX (ENT-3147)
- Fix automatic service stops based on runlevel (redhat/centos)
(CFE-2611)
- Fix cf-serverd crash when reporting corrupted data (ENT-3023)
- Fix rare output truncation on Solaris 10/11 (CFE-2527)
- Fix crash on Solaris when ps ucb variant is not available (CFE-2506)
- Fix logic to detect when running under a Xen Hypervisor (CFE-1563)
- Fix "lastseenexpireafter" 32-bit signed int overflow (zendesk#3204)
- Fix IPv6 parsing to be un-reversed (CFE-2580)
3.10.0:
New features/additions:
- All new features/additions for 3.8 and 3.9 are also included in 3.10.
- Add: Classes body tailored for use with diff
- New feature: Classes promise: allow classes without an expression to default to defined.
- Support for custom ports and host names as policy hub (CFE-953)
- Add: Definition of from_cfexecd for cf-execd initiated runs
(CFE-2386)
- Add < <= > >= operators to eval().
- Add testing jUnit and TAP bundles and include them in stdlib.cf
- New function isipinsubnet() (ENT-7949)
- LogDebug(): implement module-based debug logging.
Now most DEBUG messages are *not* printed even when "-d" is in use, but
the specific debug module has to be enabled on the command line. For
example to enable all log modules, run:
cf-agent -d --log-modules=all
- Add: edit_line contains_literal_string to stdlib
- add variablesmatching_as_data() function paralleling variablesmatching()
(Redmine #7885)
- Allow specifying agent maxconnections via def.json (CFE-2461)
- Add getuserinfo() function
- Add body agent control select_end_match_eof option. (CFE-2390)
- Add class to enable post transfer verification during policy updates
- Add ability to append to bundlesequnece with def.json (CFE-2460)
- policy_server.dat now appends a newline and supports host & port
Changes:
- Rewrite iteration engine to avoid combinatorial explosion with nested variable expansions.
This speeds up enormously the execution of policies that included long
slists or JSON containers, that in the past didn't even terminate.
Change: "cf_null" string literal was changed to not be something
special, and it's now a string that can be used anywhere, like
in slists or part of bundlesequence etc.
NOTE: Old policy should be grep'ed for "cf_null" and in case such
occurences were handled specially, they should be reworked.
Change: "--empty-list--" is now never printed by format(),
an empty list is now printed as "{ }".
Change: Order of pre-evaluation was slightly changed, A new "vars" pass
at the beginning of pre-evaluation was added. It used to be
classes-vars, but it was changed to vars-classes-vars. As a
result some classes or variables might be evaluated at a
different time than before. As always try to write policy code that works no matter what the
order of execution is.
One way is to always *guard* the execution of functions to avoid
bogus function results. For example the following will avoid
running execresult() bevore the file has been created:
execresult("cmd /path/to/filename") if => fileexists("/path/to/filename");
C internals: NULL Rlist is now perfectly valid, in fact it is the only
way to denote an empty Rlist.
C internals: Since a slist variable can be NULL, API of
EvalContextVariableGet() changed: The way to detect if a
variable is found, is not to check return value for NULL,
but to check returned *type* for CF_DATA_TYPE_NONE.
Fixed what I could find as wrong API uses. (CFE-2162)
- Allow arbitrary service policies (CFE-2402)
- Behaviour change: cf-execd: Do not append -Dfrom_cfexecd to exec_command .
(CFE-2386)
- Failsafe/Bootstrap no longer copy files starting with .git (like .gitignore) or .mailmap
(CFE-2439)
- Change: Enable strict transport security
- Change: Disable http TRACE method
- Change: Verify transfered files during policy update
- Allow getvariablemetatags() and getclassmetatags() to get a specific tag key
- Change: Use more restrictive unix socket perms (ENT-2705)
- Add sys.user_data container for user starting agent.
- Pass package promise options to underlying apt-get call (#802)
(CFE-2468)
- Change: Enable agent component management policy on systemd hosts
(CFE-2429)
- Change: Switch processes restart_class logging to verbose
- Change: Log level for keeping verbatim JSON to DEBUG (CFE-2141)
- Change: Require network before cfengine services (CFE-2435)
- Behaviour change: getvalues(inexistent_var) returns an empty list.
Restores 3.7.x and earlier behaviour. (CFE-2479)
- Behaviour change: when used with CFEngine 3.10.0 or greater,
bundles set_config_values() and set_line_based() are appending a
trailing space when inserting a configuration option with empty value.
(CFE-2466)
- Behaviour change: getvalues() always returns a list now. Even when v is a simple
string (i.e. not an iterable) it will return an slist with one element:
the value of the string variable.
- Behaviour change: readintlist() now prints an error if the
file contains real numbers, not integers, and aborts; previously it was
printing an info-level error message, was half-reading an integer out of
the real, and was continuing successfully.
- Ensure synchronous start and stop with systemctl (ENT-2841)
- Change select_region INI_section to match end of section or end of file
(CFE-2519)
Bug fixes:
- fix files promise not setting ACL properly on directories. (CFE-616)
- Upgrade CFEngine dependencies to the following versions:
- lixml2 2.9.4
- OpenSSL 1.0.2j
- LibYAML 0.1.7
- Curl 7.50.3
- Fix cumulative() to accept up to 1000 years, like it's documented.
- Fixed parsing of host name/IP and port number in cf-runagent
(CFE-546)
- Fix intermittent error message of type:
"error: Process table lacks space for last columns: " (CFE-2371)
- storage: Properly initialize the list of current mounts (CFE-1803)
- Fix 'contain' attribute 'no_output' having no effect when
the 'commands' promise is using 'module => "true"'. (CFE-2412)
- Fix bug which caused empty emails to be sent from cf-execd
if there was no previous output log and the new log was fully filtered
by email filters. (ENT-2739)
- allow ifelse(FALSE, $(x), "something else") to work. (CFE-2260)
- Fix connection cache, reuse connections when possible. (CFE-2447)
- Fix rare bug that would sometimes prevent redis-server from launching.
- Fix bug in files promise when multiple owners are promised
but first one doesn't exist, and improve logging . (CFE-2432)
- define kept outcome with action warn if edit_line is as expected
(CFE-2424)
- Example using getvariablemetatags() and getclassmetatags() to get a specific tag key
- Remove 2k limit on strings length when writing JSON policies
(CFE-2383)
- Fix ttime_range constraint to go higher than 2G as number of seconds.
- Change: cronjob bundle tolerates different spacing
- Allow editing fields in lines longer than 4k (CFE-2438)
- Don't send empty emails for logs where everything is filtered.
(ENT-2739)
- allow maplist(), maparray(), and mapdata() to evaluate function calls during iteration
(ARCHIVE-1619)
- insert_lines is no longer implicitly matching EOF as
end of the region if 'select_end' pattern is not matched . (CFE-2263)
- Change: Remove executable bit from systemd units (CFE-2436)
- cf-serverd should reload def.json when reloading policy (CFE-2406)
- Fix cf-monitord detection of usernames of the process table on AIX.
- Speed up local and remote file copying and fix spurious errors.
(ENT-2769)
- Fix occasional segfault when running getindices() on a
variable that has indices of multiple depths (e.g. both "a[x]" and
"a[x][y]"). (CFE-2397)
- When no file is provided when calling cf-promises
with cf or json output, use promises.cf by default. This restores the
previous behavior. (CFE-2375)
- Fix: Services starting or stopping unnecessarily (CFE-2421)
- Change: Split systemd units (CFE-2278)
- EOF is matched as an end of the region in edit_line
promises only if 'select_end_match_eof' parameter is true. (CFE-2263)
- Fix double logging of output_prefix, and log process name for cf-agent syslog messages.
(CFE-2225)
- Be less verbose if a network interface doesn't have a MAC address.
(CFE-1995)
- Fix: CFEngine choking on standard services (CFE-2806)
- fix insert_lines related memory corruption (CFE-2520)
- fix cf-serverd crash when reporting corrupted data. (ENT-3023)
- Fix ability to manage INI sections with metachars for
manage_variable_values_ini and set_variable_values_ini (CFE-2519)
- Fix apt_get package module incorrectly using interactive mode.
- Fix crash on Solaris when ps ucb variant is not available. (CFE-2506)
- cf-serverd: Do not close connection when file does not exist.
(CFE-2532)
- getvalues() now behaves correctly for old CFEngine arrays of depth 1.
Known issues: getvalues() still misbehaves with double-indexed arrays
(see (CFE-2504, CFE-2536)
3.9.0:
New features/additions:
- Add optional interface parameter to iprange() to match only one interface.
- Allow '=' in symbolic modes (Redmine #7826)
- Add: FreeBSD ports package module
- New package module for FreeBSD pkg package manager.
- Add support for adding/removing fifos in policy
- Add Linux parsing of /proc/net/ data.
- sys.inet
- sys.inet6
- sys.interface_data
- Data is returned as a data container.
- See documentation for more details. (Jira CFE-1991)
- sys.ip2iface: new reverse mapping variable from IP to interface name
- Namespaced classes can now be specified on the command line.
- namespaces can now be passed to cf-runagent -D and --remote-bundles
(Redmine #7856)
- Add 'cf-full' and 'json-full' to cf-promises '-p' option.
They generate output based on the entire policy. The existing 'cf'
already behaved this way, and it has now been changed to generate
output only for a single file, which the existing 'json' option
already does.
- New language functions: processexists() and findprocesses()
(Redmine #7633)
- Implement new regex_replace() function. (Redmine #7346)
- Add log rotation policy for state/classes.jsonl log. (Redmine #7951)
- Added collect_vars utility bundle to stdlib
- Intoduce report_class_log attribute to body agent control.
(Redmine #7951)
- Add standard_services service_method allowing for explicit usage
- cf-promises --show-vars can now show JSON variables.
- Add json_pipe mode to mapdata(), which allows piping a
JSON container to an external program for manipulation and receiving
JSON back. The `jq` tool is a good example where this mode can be
useful. A corresponding `$(def.jq)` variable has also been added with
a default path to this tool. See documentation for mapdata() for
more information and examples. (Jira CFE-2071)
- behaviour change: "true" is always defined and "false" is never defined in a context expression.
- Add: nimclient package module for AIX
This module provides basic functionality for using nimclient as a means
to ensure packages are either present or absent. It does not support
listing package updates available or provide any special caching.
- Add callstack_callers() and callstack_promisers() functions.
- Log variable definitions in debug output. (Redmine #7137)
- Add: Memory information to host info report (Jira CFE-1177)
- In Mustache templates, one can now use `` and
`` tags to iterate over the top level element in a
container. (Redmine #6545)
- Add network_connections() function that parses /proc/net
- Provide new -w argument to override the workdir for testing
- New feature: Emails sent by cf-execd can be filtered to get
rid of emails for unwanted log messages. The attributes
mailfilter_include and mailfilter_exclude in body executor
control control what to include. See documentation for cf-execd for
more information. (Jira CFE-2283)
- Add: file_make_mustache bundle to render mustache templates
- Add '-n' flag to cf-key to avoid host name lookups.
- cf-agent, cf-execd, cf-promises, cf-runagent and cf-serverd honor multiple -D, -N and -s arguments
(Redmine #7191)
- Add "canonify" mode to mapdata().
- Add: printfile bodies to stdlib
- Add: New `results` classes body [] (Redmine #7418, #7481)
- Implement cf-runagent --remote-bundles and cf-serverd "bundle" access promise.
(Redmine #7581)
- Add commands promise arglist attribute, augmenting args attribute.
- It's now possible to reference variables in inline JSON,
for example: `mergedata('[ thing, { "mykey": otherthing[123] } ]')`.
`thing` and `otherthing[123]` will be resolved as variables, since
they are unquoted. See the documentation for more details.
(Redmine #7871)
- Allow inline JSON to be used in the following function
calls:
- data_expand()
- difference()
- every()
- filter()
- format()
- getindices()
- getvalues()
- grep()
- intersection()
- join()
- length()
- makerule()
- mapdata()
- maplist()
- mean()
- mergedata()
- none()
- nth()
- parsejson()
- product()
- regarray()
- reglist()
- reverse()
- shuffle()
- some()
- sort()
- storejson()
- string_mustache()
- sublist()
- sum()
- unique()
- url_get()
- variance()
For example: `mergedata('[ "thing", { "mykey": "myvalue" } ]')`
See the documentation for more details. (Jira CFE-2253)
- Add: edit_line contains_literal_string to stdlib
- Add body agent control select_end_match_eof option. (Jira CFE-2390)
Changes:
- Change: classesmatching(): order of classes changed
- Change: getindices(), getvalues(), variablesmatching(), maparray():
order of variables returned has changed
- Change: set_quoted_values uses bundle scoped classes
- Change: set_config_values uses bundle scoped classes
- Change: set_variable_values uses bundle scoped classes
- Change: set_config_values_matching uses bundle scoped classes
- Change: manage_variable_values_ini uses bundle scoped classes
- Change: set_line_based should use bundle scoped classes
(Jira CFE-1959)
- getvalues() will now return a list also for data containers,
and will descend recursively into the containers. (Redmine #7116)
- Change: Improve git drop user support
- Use new package promise as default package promise
implementation. (Jira CFE-2332)
- Don't follow symbolic links when copying extended attributes.
- When a bodydefault:_ body is
defined, it will be used by all promises of type
unless another body is explicitly used.
- cf-serverd no longer appends "-I -Dcfruncommand" to
cfruncommand, this has to be done manually in masterfiles
body server control. (Redmine #7732)
- eval() function arguments mode and options are now
optional.
- sort() function argument mode is now optional.
- Change: returnszero() no longer outputs the output of a command.
The output can be seen by enabling info mode (-I).
- cfruncommand is not executed under shell. (Redmine #7409)
- Remove: Apache CGI module
- Change: Make maxbytes arg of readjson() and readyaml() optional
- Classes matching agent control's abortclasses are
now printed before exit, even if they are defined in common bundles.
Previously the regex (in abortclasses) that matched the class was
printed if the class was defined in a common bundle, but the class
itself was printed if it was defined in an agent bundle. With this
change, the defined class that caused the abort is always printed.
- Remove: Support for email settings from augments_file (Redmine #7682)
- Change: set_variable_values_ini uses bundle scoped classes
- findfiles() now skips relative paths. (Redmine #7981)
- Clients connections using non TLS protocol are rejected
by default. . (Jira CFE-2339)
- Change: Policy files specified in the "inputs" section of
def.json will no longer be auto-loaded. One has to refer to the
which are using the "inputs" field inside def.json. (Redmine #7961)
- Change: Separate binary details from policy update (Redmine #7662)
- Add guard for binary upgrade during bootstrap (Redmine #7861)
- Change: Modernize pkg module and package_method
- Remove: Userdir apache module
- filestat(path, "linktarget") now follows non-absolute links and returns full path of target.
This introduces a change in behaviour. Here is an example:
$ ls -l /bin/sh
lrwxrwxrwx 1 root root 4 Jun 4 2015 /bin/sh -> dash
Previously the filestat function would return "dash", and would also log
an error that the file can not be read. Now it will return "/bin/dash"
(or the final destination if it happens that /bin/dash is also a
symlink).
You can still get the previous behaviour by using
filestat(path, "linktarget_shallow"). (Redmine #7404)
- Define (bootstrap|failsafe)_mode during update.cf when triggerd from failsafe.cf
(Redmine #7861)
- Behavior change: The promise string of a processes
promise now matches just the command line of each process instead of
the line that is output by ps. This was done to reduce fragmentation
between platforms, since ps is a very nonstandardized tool.
(Jira CFE-2161)
- Allowed namespace names made more strict, to disallow
namespaces that cannot be reached. (Redmine #7903)
- Behavior change: When using readintlist(), readreallist()
or readstringlist(), parsing an empty file will no longer result in a
failed function call, but instead an empty list. Failure to open the
file will still result in a failed function call.
- insert_lines is no longer implicitly matching EOF as
end of the region if 'select_end' pattern is not matched .
(Jira CFE-2263)
- EOF is matched as an end of the region in edit_line
promises only if 'select_end_match_eof' parameter is true.
(Jira CFE-2263)
Bug fixes:
- Upgrade CFEngine dependencies to the following versions:
- Curl 7.48.0
- libxml2 2.9.4
- LMDB 0.9.18
- OpenLDAP 2.4.44
- OpenSSL 1.0.2h
- PCRE 8.38
(Jira ENT-2720)
- Upgrade dependencies to latest minor versions.
For Community / Enterprise:
For Enterprise:
- Fix bug which sometimes misparses user names in ps output.
- Fix: Problem with git not dropping privileges soon enough
- Allow def.json up to 5MB instead of 4K.
- It is possible to edit the same value in multiple regions
of one file. (Redmine #7460)
- CFEngine on Windows no longer truncates log messages if the
program in question is killed halfway through.
- Fixed a bug which caused def.json not being able to define
classes based on other hard classes. (Jira CFE-2333)
- Change: Tighten Enterprise hub permissions (Jira ENT-2708)
- Fix a regression which would sometimes cause "Permission
denied" errors on files inside directories with very restricted
permissions. (Redmine #7808)
- Fix use-after-free in ArrayMap and HashMap (Redmine #7952)
- Package repositories are no more hit every time package promise
is evaluated on SUSE.
- Fix a bug which sometimes caused package promises to be
skipped with "XX Another cf-agent seems to have done this since I
started" messages in the log, most notably in long running cf-agent
runs (longer than one minute). (Redmine #7933)
- TTY detection should be more reliable. (Redmine #7606)
- cf-promises -p cf now produces valid cfengine code (Redmine #7956)
- Fix ps options for FreeBSD to check processes only in current host and not in jails
- cf-runagent now properly supports multiple -D or -s arguments
(Redmine #7191)
- Fix: Work around impaired class definition from augments
(Jira CFE-2333)
- Fix "No such file or directory" LMDB error on heavily loaded hosts.
(Jira CFE-2300)
- Check for empty server response in RemoteDirList after decryption
(Redmine #7908)
- Small performance optimization when cf-execd scans emails before sending.
- Fix handling of closed connections during transactions
(Redmine #7926)
- The core ps parsing engine used for processes promises
has been rewritten from scratch, and should be more robust than
before. (Jira CFE-2161)
- Fix the lexer which could not handle empty newline(s)
before a ```@endif```.
- groupexists() no longer fails to detect a group name
starting with a digit. (Jira CFE-2351)
- Fix HP-UX specific bug that caused a lot of log output to disappear.
- Fix unresolved variable (Redmine #7931)
- Change: Suppress standard services noise on SUSE (Redmine #6968)
- Reduce verbosity of yum package module (Redmine #7485)
- cf-runagent: Allow connections to localhost instead of failing silently.
- Show errors regarding failure to copy extended attributes
when doing a local file copy. Errors could happen when copying
across two different mount points where the support for extended
attributes is different between the mount points.
- Fix classes being set because of hash collision in the implementation.
(Redmine #7912)
- fix build failure on FreeBSD 7.1 (Redmine #7415)
- Improve logging when managing setuid/setgid
- Reduce verbosity of apt_get package module (Redmine #7485)
- packagesmatching() and packageupdatesmatching() should work
when new package promise is used. (Jira CFE-2246)
- Fix bug which could render host unable to recover from a
syntax error, even if failsafe.cf was utilized. This could happen if
the file containing the syntax error was specified in the def.json
special file. (Redmine #7961)
- Prevent crash in cf-execd email code when policy server is not set.
- In case of networking error, assume checksum is wrong
- Fix two cases where action_policy warn still produces errors
(Redmine #7274)
- Fix bad option nlwp to vzps on Proxmox / OpenVZ. (Redmine #6961)
- @if minimum_version now correctly ignores lines starting with '@'
(Redmine #7862)
- No longer hang when changing permissions/ownership on fifos
(Redmine #7030)
- readfile() and read*list() should print an error if they fail to read file.
(Redmine #7702)
- The isvariable() function call now correctly accepts all
array variables when specified inline. Previously it would not accept
certain special characters, even though they could be specified
indirectly by using a variable to hold it. (Redmine #7088)
- Fix file descriptor leak when there are network errors.
- Improve robustness of process table parsing on Solaris.
(Jira CFE-2161)
- Installing packages containing version numbers using yum
now works correctly. (Redmine #7825)
- Parse def.json vars, classes and inputs from the C
code. This fixes a bug where certain entries in this file would be
parsed too late to have any effect on the evaluation.
(Redmine #7453, #7615)
- Change package modules permissions on hub package so that
hub can execute package promises. (Redmine #7602)
- Fix: CFEngine choking on standard services (Jira CFE-2086)
- Fix: cf-upgrade on SUSE
- Fix: Stop cfengine choking on systemctl output (Jira CFE-2806)
- storage: Properly initialize the list of current mounts
(Jira CFE-1803)
- Fix bug which caused empty emails to be sent from cf-execd
if there was no previous output log and the new log was fully filtered
by email filters. (Jira ENT-2739)
- Don't send empty emails for logs where everything is filtered.
(Jira ENT-2739)
- Fix intermittent error message of type:
"error: Process table lacks space for last columns: "
(Jira CFE-2371)
- Be less verbose if a network interface doesn't have a MAC address.
(Jira CFE-1995)
3.8.2:
Fixes:
- Update library dependencies to latest version.
Libraries upgraded:
- curl 7.47.0
- LMDB 0.9.18
- MySQL 5.1.72
- OpenLDAP 2.4.44
- OpenSSL 1.0.2g
- PostgreSQL 9.3.11
- Redis 3.0.7
- rsync 3.1.2
PHP was kept at 5.6.17 because of problems with the 5.6.19 version.
- Reduce verbosity of apt_get package module (Redmine #7485)
- Reduce verbosity of yum package module (Redmine #7485)
- The isvariable() function call now correctly accepts all
array variables when specified inline. Previously it would not accept
certain special characters, even though they could be specified
indirectly by using a variable to hold it. (Redmine #7088)
- Don't follow symbolic links when copying extended attributes.
- Fix a bug which sometimes caused package promises to be
skipped with "XX Another cf-agent seems to have done this since I
started" messages in the log, most notably in long running cf-agent
runs (longer than one minute). (Redmine #7933)
- Fix bug which could render host unable to recover from a
syntax error, even if failsafe.cf was utilized. This could happen if
the file containing the syntax error was specified in the def.json
special file. (Redmine #7961)
- Change: Policy files specified in the "inputs" section of
def.json will no longer be auto-loaded. One has to refer to the
$(def.augments_inputs) variable in the policy (the standard
masterfiles policies include this by default). This only affects
installations which are not based on the standard masterfiles, and
which are using the "inputs" field inside def.json. (Redmine #7961)
- Fix file descriptor leak when there are network errors.
- Fix cf-serverd error messages with classic protocol clients
(Redmine #7818)
- Installing packages containing version numbers using yum
now works correctly. (Redmine #7825)
- Fix ps options for FreeBSD to check processes only in current host and not in jails
- fix build failure on FreeBSD 7.1 (Redmine #7415)
- Show errors regarding failure to copy extended attributes
when doing a local file copy. Errors could happen when copying
across two different mount points where the support for extended
attributes is different between the mount points.
- Fix classes being set because of hash collision in the implementation.
(Redmine #7912)
- Allow def.json up to 5MB instead of 4K.
- Fix a regression which would sometimes cause "Permission
denied" errors on files inside directories with very restricted
permissions. (Redmine #7808)
- Change: Suppress standard services noise on SUSE (Redmine #6968)
Changes:
- Change: classesmatching(): order of classes changed
3.8.1:
Changes:
- Upgrade CFEngine dependencies to the following versions:
- OpenSSL 1.0.2e
- PCRE 8.38
- libxml2 2.9.3
- OpenLDAP 2.4.43
- libcurl 7.46.0
- Upgrade LMDB to version 0.9.17. (Redmine #7879)
Bug fixes:
- @if minimum_version now correctly ignores lines starting with '@'
(Redmine #7862)
- Add guard for binary upgrade during bootstrap (Redmine #7861)
- Namespaced classes can now be specified on the command line.
- Fix bad option nlwp to vzps on Proxmox / OpenVZ. (Redmine #6961)
- Fix two cases where action_policy warn still produces errors
(Redmine #7274)
- Parse def.json vars, classes and inputs from the C
code. This fixes a bug where certain entries in this file would be
parsed too late to have any effect on the evaluation.
(Redmine #7453, #7615)
- Fix HP-UX specific bug that caused a lot of log output to disappear.
- Check for empty server response in RemoteDirList after decryption
(Redmine #7908)
- getvalues() will now return a list also for data containers,
and will descend recursively into the containers. (Redmine #7116)
- Define (bootstrap|failsafe)_mode during update.cf when triggerd from failsafe.cf
(Redmine #7861)
3.8.0:
New features/additions:
- New feature: Bodies can now inherit attribute values from
other bodies by specifying "inherit_from" with the name of the body to
inherit from, plus any arguments it accepts. For example:
body classes myclasses
{
inherit_from => classes_generic("myname");
}
(Redmine #4309)
- Add url_get() function. (Redmine #6480)
- Add @if feature() syntax
@if feature work like @if minimum_version but allows distinguishing
between features chosen at compile time.
- Extend module protocol to create persistent classes.
To use it, have the module print a line with "^persistence="
before printing any class names. "persistence=0" goes back to non-
persistent classes. (Redmine #7302)
- Add: New `results` classes body (Redmine #7418)
- Add: Debug reports in cfe_internal_cleanup_agent_reports
- Add: Path to svcprop in stdlib
- Add: masterfiles-stage script to contrib
- Whitespace is now allowed in class expressions for
readability, between class names and operators. (Redmine #7152)
Changes:
- Change: Clarify bootstrap/failsafe reports
- Change: Improve in-line docs for internal log maintenance
- Change: Improve efficiency and debug reports (Redmine #7527)
- Remove: 3.5 support from masterfiles policy framework
- Long promiser strings with multiple lines are now
abbreviated in logs. (Redmine #3964)
- Change: Reunify Version based policy split
- Change: Separate binary details from policy update (Redmine #7662)
- Remove /var/cfengine/cf3..runlog. (Redmine #6957)
- Change: sys.libdir and sys.local_libdir to non version specific path
- sys.libdir now resolves to $(sys.inputdir)/lib
- sys.local_libdir now resolves to lib (Redmine #7559)
- Moved the following files to /var/cfengine/log/:
- /var/cfengine/promise_summary.log
- /var/cfengine/cfagent..log
- Change: Separate binary details from policy update (Redmine #7662)
- Remove: Support for email settings from augments_file (Redmine #7682)
Bug fixes:
- It is possible to edit the same value in multiple regions
of one file. (Redmine #7460)
- Change package modules permissions on hub package so that
hub can execute package promises. (Rednime #7602) (Redmine #7602)
- Fix exporting CSV reports through HTTPS. (Redmine #7267)
- cf-agent, cf-execd, cf-promises, cf-runagent and cf-serverd honor
multiple -D, -N and -s arguments (Redmine #7191)
- readfile() and read*list() should print an error if they fail to read file.
(Redmine #7702)
- No longer hang when changing permissions/ownership on fifos
(Redmine #7030)
- Fix broken HA policy for 3rd disaster-recovery node.
- Fix: Policy errors for 3.5 and 3.6
- Mustache templates: Fix key when value is not a
primitive. The old behavior, when iterating across a map or array of
maps, was to abort if the key was requested with . The new
behavior is to always replace with either the key name or the
iteration position in the array. An error is printed if is used
outside of a Mustache iteration section.
- Fix build with musl libc. (Redmine #7455)
- Fixed a bug which could cause daemons to not to be killed
correctly when upgrading or manually running "service cfengine3 stop".
(Redmine #7193)
- Fix daemons not restarting correctly on upgrade on AIX.
- Package promise: Fix inability to install certain packages
with numbers. (Redmine #7421)
- Redmine #6027 Directories should no more be changed randomly
into files. (Redmine #6027)
- Improve cf-serverd's lock contention because of getpwnam()
call. (Redmine #7643) (Redmine #7643)
- action_policy "warn" now correctly produces warnings instead
of various other verbosity levels. (Redmine #7274)
- If there is an error saving a mustache template file
it is now logged with log-level error (was inform).
- The JSON parser now supports unquoted strings as keys.
- Reduce malloc() thread contention on heavily loaded
cf-serverd, by not exiting early in the logging function, if no message
is to be printed. (Redmine #7624) (Redmine #7624)
- Fix a bug which caused daemons not to be restarted on
upgrade. (Redmine #7528)
- Include latest security updates for dependencies.
- Fixed bug which would cause bff and depot packages not to
run package scripts on removal. (Redmine #7193)
- Fix upgrade causing error message under systemd because of open ports.
- Fixed several bugs which prevented CFEngine from loading
libraries from the correct location. This affected several platforms.
(Redmine #6708)
- Legacy package promise: Result classes are now defined if
the package being promised is already up to date. (Redmine #7399)
- failsafe.cf will be created when needed. (Redmine #7634)
(Redmine #7634)
- If file_select.file_types is set to symlink and there
are regular files in the scanned directory, CFEngine no longer
produces an unnecessary error message. (Redmine #6996)
- Fix 'AIX_PREINSTALL_ALREADY_DONE.txt: cannot create' error
message on AIX.
- Fix package promise not removing dependent packages. (Redmine #7424)
- Fix: Solaris packages no longer contain duplicate library
files, but instead symlinks to them. (Redmine #7591)
- Fix select_class not setting class when used in common bundle with slist.
(Redmine #7482)
- Fix "@endif" keyword sometimes being improperly processed
by policy parser. (Redmine #7413)
- Fix noise from internal policy to upgrade windows agents
(Redmine #7456)
- cfruncommand now works if it contains spaces, with the TLS protocol.
(Redmine #7405)
- Fix warning "Failed to parse csv file entry" with certain very long
commands promises. (Redmine #7400)
- CFEngine no longer erronously passes -M to useradd on HP-UX. (Redmine #6734)
- cf-monitord no longer complains about missing thermal zone files.
(Redmine #7238)
- systemd is now detected correctly if it is a symlink (Redmine #7297)
- TTY detection should be more reliable. (Redmine #7606)
(Redmine #7606)
3.7.3
Fixes:
- Reduce verbosity of yum package module (Redmine #7485)
- Reduce verbosity of apt_get package module (Redmine #7485)
- Upgrade dependencies to latest patch versions.
Upgraded libraries:
- curl 7.47.0
- libxml2 2.9.3
- LMDB 0.9.18
- MySQL 5.1.72
- OpenLDAP 2.4.44
- OpenSSL 1.0.2g
- PCRE 8.38
- PostgreSQL 9.3.11
- Redis 2.8.24
- rsync 3.1.2
PHP was kept at 5.6.17 because of problems with the 5.6.19 version.
- parse def.json vars, classes, and inputs in C (Redmine #7453)
- Namespaced classes can now be specified on the command line.
- getvalues() will now return a list also for data containers,
and will descend recursively into the containers. (Redmine #7116)
- @if minimum_version now correctly ignores lines starting with '@'
(Redmine #7862)
- Fix definition of classes from augments file
- Don't follow symbolic links when copying extended attributes.
- Fix ps options for FreeBSD to check processes only in current host and not in jails
- Fix cf-serverd error messages with classic protocol clients
(Redmine #7818)
- Change: Suppress standard services noise on SUSE (Redmine #6968)
- The isvariable() function call now correctly accepts all
array variables when specified inline. Previously it would not accept
certain special characters, even though they could be specified
indirectly by using a variable to hold it. (Redmine #7088)
- Show errors regarding failure to copy extended attributes
when doing a local file copy. Errors could happen when copying
across two different mount points where the support for extended
attributes is different between the mount points.
- Fix bad option nlwp to vzps on Proxmox / OpenVZ. (Redmine #6961)
- Fix file descriptor leak when there are network errors.
- Fix a regression which would sometimes cause "Permission
denied" errors on files inside directories with very restricted
permissions. (Redmine #7808)
- Check for empty server response in RemoteDirList after decryption
(Redmine #7908)
- Allow def.json up to 5MB instead of 4K.
- Add guard for binary upgrade during bootstrap (Redmine #7861)
- Fix HP-UX specific bug that caused a lot of log output to disappear.
- Fix a bug which sometimes caused package promises to be
skipped with "XX Another cf-agent seems to have done this since I
started" messages in the log, most notably in long running cf-agent
runs (longer than one minute). (Redmine #7933)
- Define (bootstrap|failsafe)_mode during update.cf when triggerd from failsafe.cf
(Redmine #7861)
- Fix two cases where action_policy warn still produces errors
(Redmine #7274)
- Fix classes being set because of hash collision in the implementation.
(Redmine #7912)
- fix build failure on FreeBSD 7.1 (Redmine #7415)
- Installing packages containing version numbers using yum
now works correctly. (Redmine #7825)
Changes:
- Change: classesmatching(): order of classes changed
3.7.3
Fixes:
- Reduce verbosity of yum package module (Redmine #7485)
- Reduce verbosity of apt_get package module (Redmine #7485)
- Upgrade dependencies to latest patch versions.
Upgraded libraries:
- curl 7.47.0
- libxml2 2.9.3
- LMDB 0.9.18
- MySQL 5.1.72
- OpenLDAP 2.4.44
- OpenSSL 1.0.2g
- PCRE 8.38
- PostgreSQL 9.3.11
- Redis 2.8.24
- rsync 3.1.2
PHP was kept at 5.6.17 because of problems with the 5.6.19 version.
- parse def.json vars, classes, and inputs in C (Redmine #7453)
- Namespaced classes can now be specified on the command line.
- getvalues() will now return a list also for data containers,
and will descend recursively into the containers. (Redmine #7116)
- @if minimum_version now correctly ignores lines starting with '@'
(Redmine #7862)
- Fix definition of classes from augments file
- Don't follow symbolic links when copying extended attributes.
- Fix ps options for FreeBSD to check processes only in current host and not in jails
- Fix cf-serverd error messages with classic protocol clients
(Redmine #7818)
- Change: Suppress standard services noise on SUSE (Redmine #6968)
- The isvariable() function call now correctly accepts all
array variables when specified inline. Previously it would not accept
certain special characters, even though they could be specified
indirectly by using a variable to hold it. (Redmine #7088)
- Show errors regarding failure to copy extended attributes
when doing a local file copy. Errors could happen when copying
across two different mount points where the support for extended
attributes is different between the mount points.
- Fix bad option nlwp to vzps on Proxmox / OpenVZ. (Redmine #6961)
- Fix file descriptor leak when there are network errors.
- Fix a regression which would sometimes cause "Permission
denied" errors on files inside directories with very restricted
permissions. (Redmine #7808)
- Check for empty server response in RemoteDirList after decryption
(Redmine #7908)
- Allow def.json up to 5MB instead of 4K.
- Add guard for binary upgrade during bootstrap (Redmine #7861)
- Fix HP-UX specific bug that caused a lot of log output to disappear.
- Fix a bug which sometimes caused package promises to be
skipped with "XX Another cf-agent seems to have done this since I
started" messages in the log, most notably in long running cf-agent
runs (longer than one minute). (Redmine #7933)
- Define (bootstrap|failsafe)_mode during update.cf when triggerd from failsafe.cf
(Redmine #7861)
- Fix two cases where action_policy warn still produces errors
(Redmine #7274)
- Fix classes being set because of hash collision in the implementation.
(Redmine #7912)
- fix build failure on FreeBSD 7.1 (Redmine #7415)
- Installing packages containing version numbers using yum
now works correctly. (Redmine #7825)
Changes:
- Change: classesmatching(): order of classes changed
3.7.2:
Bug fixes:
- readfile() and read*list() should print an error if they fail to read file.
(Redmine #7702)
- Fix 'AIX_PREINSTALL_ALREADY_DONE.txt: cannot create' error
message on AIX.
- If there is an error saving a mustache template file
it is now logged with log-level error (was inform).
- Change: Clarify bootstrap/failsafe reports
- Fixed several bugs which prevented CFEngine from loading
libraries from the correct location. This affected several platforms.
(Redmine #6708)
- If file_select.file_types is set to symlink and there
are regular files in the scanned directory, CFEngine no longer
produces an unnecessary error message. (Redmine #6996)
- Fix: Solaris packages no longer contain duplicate library
files, but instead symlinks to them. (Redmine #7591)
- cf-agent, cf-execd, cf-promises, cf-runagent and cf-serverd honor
multiple -D, -N and -s arguments (Redmine #7191)
- Fix "@endif" keyword sometimes being improperly processed
by policy parser. (Redmine #7413)
- It is possible to edit the same value in multiple regions
of one file. (Redmine #7460)
- Fix select_class not setting class when used in common bundle with slist.
(Redmine #7482)
- Fix broken HA policy for 3rd disaster-recovery node.
- Directories should no more be changed randomly
into files. (Redmine #6027)
- Include latest security updates for 3.7.
- Reduce malloc() thread contention on heavily loaded
cf-serverd, by not exiting early in the logging function, if no message
is to be printed. (Redmine #7624)
- Improve cf-serverd's lock contention because of getpwnam()
call. (Redmine #7643)
- action_policy "warn" now correctly produces warnings instead
of various other verbosity levels. (Redmine #7274)
- Change: Improve efficiency and debug reports (Redmine #7527)
- Change package modules permissions on hub package so that
hub can execute package promises. (Redmine #7602)
- No longer hang when changing permissions/ownership on fifos
(Redmine #7030)
- Fix exporting CSV reports through HTTPS. (Redmine #7267)
- failsafe.cf will be created when needed. (Redmine #7634)
- Mustache templates: Fix key when value is not a
primitive. The old behavior, when iterating across a map or array of
maps, was to abort if the key was requested with . The new
behavior is to always replace with either the key name or the
iteration position in the array. An error is printed if is used
outside of a Mustache iteration section.
- Legacy package promise: Result classes are now defined if
the package being promised is already up to date. (Redmine #7399)
- TTY detection should be more reliable. (Redmine #7606)
Masterfiles:
- Add: Path to svcprop in stdlib
- Add: New `results` classes body [] (Redmine #7418, #7481)
- Remove: Support for email settings from augments_file (Redmine #7682)
3.7.1:
Bug fixes:
- Fix daemons not restarting correctly on upgrade on AIX. (Redmine #7550)
- Fix upgrade causing error message under systemd because of open ports.
- Fix build with musl libc. (Redmine #7455)
- Long promiser strings with multiple lines are now
abbreviated in logs. (Redmine #3964)
- Fixed a bug which could cause daemons to not to be killed
correctly when upgrading or manually running "service cfengine3 stop".
(Redmine #7193)
- Package promise: Fix inability to install certain packages
with numbers.
- Fix package promise not removing dependent packages. (Redmine #7424)
- Fix warning "Failed to parse csv file entry" with certain very long
commands promises. (Redmine #7400)
- Fix misaligned help output in cf-hub. (Redmine #7273)
- Augmenting inputs from the augments_file (Redmine #7420)
- Add support for failover to 3rd HA node located outside cluster.
- Upgrade all dependencies for patch release.
- Fix a bug which caused daemons not to be restarted on
upgrade. (Redmine #7528)
3.7.0:
New features:
- New package promise implementation.
The syntax is much simpler, to try it out, check out the syntax:
packages:
"mypackage"
policy => "absent/present",
# Optional, default taken from common control
package_module => apt_get,
# Optional, will only match exact version. May be
# "latest".
version => "32.0",
# Optional.
architecture => "x86_64";
- Full systemd support for all relevant platforms
- New classes to determine whether certain features are enabled:
* feature_yaml
* feature_xml
For the official CFEngine packages, these are always enabled, but
packages from other sources may be built without the support.
- New readdata() support for generic data input (CSV, YAML, JSON, or auto)
- YAML support: new readyaml() function and in readdata()
- CSV support: new readcsv() function and in readdata()
- New string_mustache() function
- New data_regextract() function
- eval() can now be called with "class" as the "mode" argument, which
will cause it to return true ("any") if the calculated result is
non-zero, and false ("!any") if it is zero.
- New list_ifelse() function
- New mapdata() function as well as JSON support in maparray().
- filestat() function now supports "xattr" argument for extended
attributes.
- "ifvarclass" now has "if" as an alias, and "unless" as an inverse
alias.
- Ability to expand JSON variables directory in Mustache templates:
Prefix the name with '%' for multiline expansion, '$' for compact
expansion.
- Ability to expand the iteration *key* in Mustache templates with @
- Canonical JSON output: JSON output has reliably sorted keys so the
same data structure will produce the same JSON every time.
- New "@if minimum_version(x.x)" syntax in order to hide future language
improvements from versions that don't understand them.
- compile time option (--with-statedir) to
override the default state/ directory path.
- Fix error messages/ handling in process signalling which no longer
allowed any signals to fail silently
- Also enable shortcut keyword for cf-serverd classic protocol, eg to
simplify the bootstrap process for clients that have different
sys.masterdir settings (Redmine #3697)
- methods promises now accepts the bundle name in the promiser string,
as long as it doesn't have any parameters.
- In a services promise, if the service_method bundle is not specified,
it defaults to the promiser string (canonified) with "service_" as a
prefix. The bundle must be in the same namespace as the promise.
- inline JSON in policy files: surrounding with parsejson() is now
optional *when creating a new data container*.
- New data_expand() function to interpolate variables in a data container.
- Add configurable network bandwidth limit for all outgoing
connections ("bwlimit" attribute in "body common control") . To
enforce it in both directions, make sure the attribute is set on both
sides of the connection.
- Secure bootstrap has been facilitated by use of
"cf-agent --boostrap HUB_ADDRESS --trust-server=no"
- Implement new TLS-relevant options (Redmine #6883):
- body common control: tls_min_version
- body server control: allowtlsversion
- body common control: tls_ciphers
- body server control: allowciphers (preexisting)
Changes:
- Improved output format, less verbose, and messages are grouped.
- cf-execd: agent_expireafter default was changed to 120 minutes
(Redmine #7113)
- All embedded databases are now rooted in the state/ directory.
- TLS used as default for all outgoing connections.
- process promise now reports kept status instead of repaired if a
signal is not sent, even if the restart_class is set. The old
behavior was to set the repaired status whenever the process was not
running. (Redmine#7216).
- Bootstrapping requires keys to be generated in advance using cf-key.
- Disable class set on reverse lookup of interfaces IP addresses.
(Redmine #3993, Redmine #6870)
- Define a hard class with just the OS major version on FreeBSD.
- Abort cf-agent if OpenSSL's random number generator can't
be seeded securely.
- Masterfiles source tarball now installs using the usual commands
"./configure; make install".
- Updated Emacs syntax highlighting template to support the latest
syntax enhancements in 3.7.
Deprecations:
- Arbitrary arguments to cfruncommand (using "cf-runagent -o") are
not acceptable any more. (Redmine #6978)
- 3.4 is no longer supported in masterfiles.
Bug fixes:
- Fix server common bundles evaluation order (Redmine#7211).
- Limit LMDB disk usage by preserving sparse areas in LMDB files
(Redmine#7242).
- Fixed LMDB corruption on HP-UX 11.23. (Redmine #6994)
- Fixed insert_lines failing to converge if preserve_block was used.
(Redmine #7094)
- Fixed init script failing to stop/restart daemons on openvz/lxc
hosts. (Redmine #3394)
- rm_rf_depth now deletes base directory as advertised. (Redmine #7009)
- Refactored cf-agent's connection cache to properly differentiate
hosts using all needed attributes like host and port.
(Redmine #4646)
- Refactored lastseen database handling to avoid inconsistencies.
(Redmine #6660)
- cf-key --trust-key now supports new syntax to also update the
lastseen database, so that clients using old protocol will trust
the server correctly.
- Fixed a bug which sometimes caused an agent or daemon to kill or stop
itself. (Redmine #7075, #7244)
- Fixed a bug which made it difficult to kill CFEngine daemons,
particularly cf-execd. (Redmine #6659, #7193)
- Fixed a bug causing systemd not to be detected correctly on Debian.
(Redmine #7297)
- "cf-promises -T" will now correctly report the checked out commit,
even if you haven't checked out a Git branch. (Redmine #7332)
- Reduce verbosity of harmless errors related to socket timeouts and
missing thermal zone files. (Redmine #6486 and #7238)
Masterfiles:
Added:
- Support for user specified overring of framework defaults without
modifying policy supplied by the framework itself (see
example_def.json)
- Support for def.json class augmentation in update policy
- Run vacuum operation on postgresql every night as a part of
maintenance.
- Add measure_promise_time action body to lib (3.5, 3.6, 3.7, 3.8)
- New negative class guard `cfengine_internal_disable_agent_email` so
that agent email can be easily disabled by augmenting def.json
Changed:
- Relocate def.cf to controls/VER/
- Relocate update_def to controls/VER
- Relocate all controls to controls/VER
- Only load cf_hub and reports.cf on CFEngine Enterprise installs
- Relocate acls related to report collection from bundle server
access_rules to controls/VER/reports.cf into bundle server
report_access_rules
- Re-organize cfe_internal splitting core from enterprise specific
policies and loading the appropriate inputs only when necessary
- Moved update directory into cfe_internal as it is not generally
intended to be modified
- services/autorun.cf moved to lib/VER/ as it is not generally intended
to be modified
- To improve predictibility autorun bundles are activated in
lexicographical order
- Relocate services/file_change.cf to cfe_internal/enterprise. This
policy is most useful for a good OOTB experience with CFEngine
Enterprise Mission Portal.
- Relocate service_catalogue from promsies.cf to services/main.cf. It is
intended to be a user entry. This name change correlates with the main
bundle being activated by default if there is no bundlesequence
specified.
- Reduce benchmarks sample history to 1 day.
- Update policy no longer generates a keypair if one is not found.
(Redmine: #7167)
- Relocate cfe_internal_postgresql_maintenance bundle to lib/VER/
- Set postgresql_monitoring_maintenance only for versions 3.6.0 and
3.6.1
- Move hub specific bundles from lib/VER/cfe_internal.cf into
lib/VER/cfe_internal_hub.cf and load them only if policy_server policy
if set.
- Re-organize lib/VER/stdlib.cf from lists into classic array for use
with getvalues
Removed:
- Diff reporting on /etc/shadow (Enterprise)
- Update policy from promise.cf inputs. There is no reason to include
the update policy into promsies.cf, update.cf is the entry for the
update policy
- _not_repaired outcome from classes_generic and scoped_classes generic
(Redmine: # 7022)
Fixes:
- standard_services now restarts the service if it was not already
running when using service_policy => restart with chkconfig (Redmine
#7258)
3.6.5:
Features:
- Introduced "systemd" hard class. (Redmine #6995)
- Added paths to dtrace, zfs and zpool on FreeBSD in masterfiles.
Bug fixes:
- Fixed build error on certain RHEL5 and SLES10 setups. (Redmine #6841)
- Fixed a bug which caused dangling symlinks not to be removed.
(Redmine #6582)
- Fixed data_readstringarrayidx function not preserving the order of the
array it's producing. (Redmine #6920)
- Fixed a bug which sometimes caused CFEngine to kill the wrong daemon
if both the host and a container inside the host were running
CFEngine. (Redmine #6906)
- Made sure the rm_rf_depth bundle also deletes the base directory.
(Redmine #7009)
- Fixed monitord reporting wrongly on open ports. (Redmine #6926)
- Skip adding the class when its name is longer than 1024 characters.
Fixed core dump when the name is too large. (Redmine #7013)
- Fixed detection of stopped process on Solaris. (Redmine #6946)
- Fixed infinite loop (Redmine #6992) plus a couple more minor
bugs in edit_xml promises.
3.6.4:
Features:
- Introduced users promises support on HP-UX platform.
- Introduced process promises support on HP-UX platform.
Bug fixes:
- Fixed bug on FreeBSD which sometimes led to the wrong process being
killed (Redmine #2330)
- Fixed package version comparison sometimes failing with rpm package
manager (Redmine #6807)
- Fixed a bug in users promises which would sometimes set the wrong
password hash if the user would also be unlocked at the same time.
- Fixed a bug on AIX which would occasionally kill the wrong process.
- Improved error message for functions that require an absolute path.
(Redmine #6877)
- Fixed some spelling errors in examples.
- Fixed error in out-of-tree builds when building cf-upgrade.
- Fixed a bug which would make cf-agent exit with an error if it was
built with a custom log directory, and that directory did not exist.
- Fixed ordering of evaluating promises when depends_on is used.
(Redmine #6484, Redmine #5462)
- Skip non-empty directories silently when recursively deleting.
(Redmine #6331)
- Fix memory exhaustion with list larger than 4994 items.
(Redmine # 6672)
- Fix cf-execd segfault on IP address detection (Redmine #6905).
- Fix hard class detection of RHEL6 ComputeNode (Redmine #3148).
3.6.3
New features:
- support for HP-UX 11.23 and later
- experimental support for Red Hat Enterprise Linux 7
Bug fixes:
- fix getindices on multi-dimensional arrays (Redmine #6779)
- fix mustache template method to run in dryrun mode (Redmine #6739)
- set mailto and mailfrom settings for execd in def.cf (Redmine #6702)
- fix conflation of multi-index entries in arrays (Redmine #6674)
- fix promise locking when transferring using update.cf (Redmine #6623)
- update JSON parser to return an error on truncation (Redmine #6608)
- fix sys.hardware_addresses not expanded (Redmine #6603)
- fix opening database txn /var/cfengine/cf_lastseen.lmdb:
MDB_READERS_FULL when running cf-keys --show-hosts (Redmine #6602)
- fix segfault (Null pointer dereference) when select_end in
delete_lines never matches (Redmine #6589)
- fix max_file_size => "0" not disabling or allowing any size
(Redmine #6588)
- fix ifvarclass, with iteration over list, failing when deleting
files with time condition (Redmine #6577)
- fix classes defined with "or" constraint are never set if any value
doesn't evaluate to a scalar (Redmine #6569)
- update "mailfrom" default in default policy (Redmine #6567)
- fix logrotate ambiguity of filename (Redmine #6563)
- fix parsing JSON files (Redmine #6549)
- reduce write count activity to /var partition (Redmine #6523)
- fix files delete attribute incorrectly triggering promise_kept
(Redmine #6509)
- update services bundle output related to chkconfig when run in
inform mode. (Redmine #6492)
- fix Solaris serverd tests (Redmine #6406)
- fix broken bechaviour of merging arrays with readstringarray
(Redmine #6369)
- fix ifelapsed bug with bundle nesting (Redmine #6334)
- fix handling cf_null in bundlesequence (Redmine #6119)
- fix maparray reading whole input array when using subarray
(Redmine #6033)
- fix directories being randomly changed to files (Redmine #6027)
- update defaults promise type to work with classes (Redmine #5748)
- systemd integration in services promises (Redmine #5415)
- fix touch attribute ignoring action = warn_only (Redmine #3172)
- fix 4KB string limit in functions readfile, string_downcase,
string_head, string_reverse, string_length, string_tail,
string_upcase (Redmine #2912)
3.6.2
Bug fixes:
- don't regenerate software_packages.csv every time (Redmine #6441)
- improve verbose message for package_list_command
- fix missing log output on AIX (Redmine #6434)
- assorted fixes to dirname() esp on Windows (Redmine #4716)
- fix package manager detection
- fix build issues on FreeBSD
- allow copying of dead symbolic links (Redmine #6175)
- preserve order in readstringarrayidx (Redmine #6466)
- fix passing of unexpanded variable references to arrays
(Redmine #5893)
- use entries for new {admin,deny}_{ips,hostnames} constraints in
the relevant legacy lists (Redmine #6542)
- cope with ps's numeric fields overflowing to the right
- interpret failing function calls in ifvarclass as class not set
(Redmine #6327)
- remove unexpanded lists when extending lists (Redmine #6541)
- infer start-time of a process from elapsed when needed
(Redmine #4094)
- fix input range definition for laterthan() function (Redmine #6530)
- don't add trailing delimiter when join()'ing lists ending with a
null-value (Redmine #6552)
- 9999999999 (ten 9s) or higher has been historically used as an upper
bound in CFEngine code and policy but because of overflow on 32-bit
platforms it caused problems with big numbers. Fixed in two ways:
first change all existing policy uses to 999999999 (nine 9s instead
of eleven 9s), second fix the C code to not wrap-around in case of
overflow, but use the LONG_MAX value (Redmine #6531).
- cf-serverd and other daemons no longer reload their configuration
every minute if CFEngine is built with an inputs directory outside
of the work directory (not the default). (Redmine #6551)
3.6.1
New features:
- Introduced Solaris and AIX support into the 3.6 series, with many associated build and
bug fixes.
Changes:
- Short-circuit evaluation of classes promises if class is already set (Redmine #5241)
- fix to assume all non-specified return codes are failed in commands promises (Redmine #5986)
- cf-serverd logs reconfiguration message to NOTICE (was INFO) so that it's always logged in syslog
Bug fixes:
- File monitoring has been completely rewritten (changes attribute in files promise), which
eliminates many bugs, particularly regarding files that are deleted. Upgrading will keep
all monitoring data, but downgrading again will reinitialize the DB, so all files will be
reported as if they were new. (Redmine #2917)
- $(this.promiser) expands in files promises for 'transformer', 'edit_template',
'copy_from.source', 'file_select.exec_program', 'classes' and 'action' bodies
(Redmine #1554, #1496, #3530, #1563)
- 'body changes' notifies about disappeared files in file monitoring (Redmine #2917)
- Fixed CFEngine template producing a zero sized file (Redmine #6088)
- Add 0-9 A-Z _ to allowed context of module protocol (Redmine #6063)
- Extend ps command column width and prepend zone name on Solaris
- Fixed strftime() function on Solaris when called with certain specifiers.
- Fixed users promise bug regarding password hashes in a NIS/NSS setup.
- Fixed $(sys.uptime), $(sys.systime) and $(sys.sysday) in AIX. (Redmine #5148, #5206)
- Fixed processes_select complaining about "Unacceptable model uncertainty examining processes" (Redmine #6337)
- ps command for linux has been changed to cope with big rss values (Redmine #6337)
- Address ps -axo shift on FreeBSD 10 and later (Redmine #5667)
- methods and services promises respect action_policy => "warn" (Redmine #5924)
- LMDB should no longer deadlock if an agent is killed on the hub while holding the DB lock.
Note that the change only affects binary packages shipped by CFEngine, since the upstream
LMDB project has not yet integrated the change. (Redmine #6013)
3.6.0
Changes:
- Changes to logging output
- add process name and pid in syslog message (GitHub #789)
- cf-serverd logging levels are now more standardised:
- INFO logs only failures
- VERBOSE logs successful requests as well
- DEBUG logs actual protocol traffic.
- cf-serverd now logs the relevant client IP address on
each message.
- Logging contexts to local database (cf_classes.tcdb) has been deprecated.
- 'usebundle' promisees are logged for all the bundle promises
- output from 'reports' promises has nothing prefixed except 'R: '
- a log line with stack path is generated when the promise type evaluated changes
- LMDB (symas.com/mdb) is the default database for local data storage : use version 0.9.9 or later
cf-agent --self-diagnostics (-x) is only implemented for TCDB, not for LMDB
- port argument in readtcp() and selectservers() may be a
service name (e.g. "http", "pop3").
- Enable source file in agent copy_from promises to be a relative path.
- file "changes" reporting now reports with log level "notice", instead of "error".
- process_results default to AND'ing of set attributes if not specified (Redmine #3224)
- interface is now canonified in sys.hardware_mac[interface] to align with
sys.ipv4[interface] (Redmine #3418)
- cf-promises no longer errors on missing bodies when run without --full-check (-c)
- Linux flavor "SUSE" now correctly spelled with all uppercase in variables and class names
(Redmine #3734). The "suse" lowercase version is also provided for convenience (Redmine #5417).
- $(this.promise_filename) and $(..._dirname) variables are now absolute paths. (Redmine #3839)
- including the same file multiple times in 'body control inputs' is not an error
- portnumber in body copy_from now supports service names like
"cfengine", "pop3" etc, check /etc/services for more.
- The failsafe.cf policy, run on bootstrap and in some other
unusual cases, has been extracted from C code into libpromises/failsafe.cf
- masterfiles
- cf_promises_validated is now in JSON format
- timestamp key is timestamp (sec since unix epoch) of last time validated
- the masterfiles now come from https://github.com/cfengine/masterfiles and are
not in the core repository
- cf-serverd calls cf-agent with -Dcfruncommand when executing cf-runagent requests
- Mark as removed: promise_notkept_log_include, promise_notkept_log_exclude, promise_repaired_log_include,
promise_repaired_log_exclude, classes_include, classes_exclude, variables_include,
variables_exclude attributes from report_data_select body (syntax is valid but not functional).
They have been replaced by the following attributes: promise_handle_include,
promise_handle_exclude, metatags_include, metatags_exclude.
New features:
- New promise type "users" for managing local user accounts.
- TLS authentication and fully encrypted network protocol.
Additions specific to the new type of connections:
- New attribute "allowlegacyconnects" in body server control,
which enables serving policy via non-latest cfengine protocol,
to the given list of hosts. If the option is absent, it
defaults to allow all hosts. To refuse non-TLS connections,
specify an empty list.
- New attribute "protocol_version" in body copy_from, and body
common control, which defines the preferred protocol for
outgoing connections.. Allowed values at the moment: "0" or
"undefined", "classic" or "1", "latest" or "2". By leaving the
copy_from option as undefined the common control option is
used, and if both are undefined then classic protocol is used
by default.
- The new networking protocol uses TLS for authentication,
after which all dialog is encrypted within the established
TLS session. cf-serverd is still able to speak the legacy
protocol with old agents.
- The 'skipverify' option in 'body server control' is
deprecated and only left for compatibility; it does
nothing
- cf-serverd does not hang up the connection if some request
fails, so that the client can add more requests.
- For the connections using the new protocol, all of the
paths in bundle server access_rules now differentiate
between a directory and a file using the trailing
slash. If the path exists then this is auto-detected and
trailing slash appended automatically. You have to append
a trailing slash manually to an inexistent or symbolic
path (e.g. "/path/to/$(connection.ip)/") to force
recursive access.
- New in 'access' promises for 'bundle server access_rules'
- Attributes "admit_ips", "admit_hostnames", "admit_keys",
"deny_ips", "deny_hostnames", "deny_keys"
- "admit_keys" and "deny_keys" add the new functionality
of controlling access according to host identity,
regardless of the connecting IP.
- For these new attributes, regular expressions
are not allowed, only CIDR notation for "admit/deny_ips", exact
"SHA=..." strings for "admit/deny_keys", and exact hostnames
(e.g. "cfengine.com") or subdomains (starting with dot,
e.g. ".cfengine.com") for "admit/deny"_hostnames. Same rules
apply to 'deny_*' attributes.
- These new constraints and the paths in access_rules, can contain
special variables "$(connection.ip)", "$(connection.hostname)",
"$(connection.key)", which are expanded dynamically for every
received connection.
- For connections using the new protocol, "admit" and "deny"
constraints in bundle server access_rules are being phased
out, preferred attributes are now "admit_ips", "deny_ips",
"admit_hostnames", "deny_hostnames", "admit_keys",
"deny_keys".
- New "shortcut" attribute in bundle server access_rules used to
dynamically expand non-absolute request paths.
- masterfiles
- standard library split: lib/3.5 (compatibility) and lib/3.6 (mainline)
- many standard library bundles and bodies, especially packages- and file-related,
were revised and fixed
- supports both Community and Enterprise
- new 'inventory/' structure to provide OS, dmidecode, LSB, etc. system inventory
(configured mainly in def.cf)
- cf_promises_release_id contains the policy release ID which is the GIT HEAD SHA
if available or hash of tree
- a bunch'o'bundles to make starting with CFEngine easier:
- file-related: file_mustache, file_mustache_jsonstring, file_tidy, dir_sync, file_copy,
file_link, file_hardlink, file_empty, file_make
- packages-related: package_absent, package_present, package_latest,
package_specific_present, package_specific_absent, package_specific_latest, package_specific
- XML-related: xml_insert_tree_nopath, xml_insert_tree, xml_set_value, xml_set_attribute
- VCS-related: git_init, git_add, git_checkout, git_checkout_new_branch,
git_clean, git_stash, git_stash_and_clean, git_commit, git
- process-related: process_kill
- other: cmerge, url_ping, logrotate, prunedir
- New command line options for agent binaries
- New options to cf-promises
- '--show-classes' and '--show-vars'
- '--eval-functions' controls whether cf-promises should evaluate functions
- Colorized output for agent binaries with command line option '--color'
(auto-enabled if you set CFENGINE_COLOR=1)
- New language features
- New variable type 'data' for handling of structured data (ie JSON),
including supporting functions:
- 'data_readstringarray' - read a delimited file into a data map
- 'data_readstringarrayidx' - read a delimited file into a data array
- 'datastate' - create a data variable with currently set classes and variables
- 'datatype' - determine the type of the top element of a container
- 'format' - %S can be used to serialize 'data' containers into a string
- 'mergedata' - merge two data containers, slists/ilists/rlists, or "classic"
arrays into a data container
- 'parsejson' - create a data container from a JSON string
- 'readjson' - create a data container from a file that contains JSON
- 'storejson' - serialize a data container into a string
- Most functions operating on lists can also operate on data containers
- pass a data container to a bundle with the @(container) notation
- the module protocol accepts JSON for data containers with the '%' sigil
- Tagging of classes and variables allows annotating of language construct with
meta data; supporting functionality:
- The module protocol in 'commands' promises has been extended to allow setting
of tags of created variables and classes, and the context of created variables
- 'getclassmetatags' - returns list of meta tags for a class
- 'getvariablemetatags' - returns list of meta tags for a variable
- 'body file control' has an 'inputs' attribute to include library files and other
dependencies
- bundlesequences can be built with bundlesmatching() based on bundle name and tags
- New attributes in existing promise types and bodies
- New option 'preserve_all_lines' for insert_type in insert_lines promises
- Caching of expensive system functions to avoid multiple executions of
execresult() etc, can be controlled via cache_system_functions attribute in
body common control
- New option 'mailsubject' in body executor control allows defining the subject
in emails sent by CFEngine
- Support for Mustache templates in 'files' promises; use 'template_method' and
'template_data' attributes. Without 'template_data' specified, uses datastate().
- New and improved functions
- 'bundlesmatching' - returns list of defined bundles matching a regex and tags
- 'canonifyuniquely' - converts a string into a unique, legal class name
- 'classesmatching' - returns list of set classes matching a regex and tags
- 'eval' - evaluates mathematical expressions; knows SI k, m, g quantifiers, e.g. "100k"
- 'findfiles' - list files matching a search pattern; use "**" for recursive searches
- 'makerule' - evaluates whether a target file needs to be rebuilt from sources
- 'max', 'min' - returns maximum and minimum of the numbers in a container or list
(sorted by a 'sort' method)
- 'mean' - returns the mean of the numbers in a container or list
- 'nth' - learned to look up by key in a data container holding a map
- 'packagesmatching' - returns a filtered list of installed packages.
- 'readfile' - learned to read system files of unknown size like those in /proc
- 'sort' - can sort lexicographically, numerically (int or real), by IP, or by MAC
- 'string_downcase', 'string_upcase' - returns the lower-/upper-case version of a
string
- 'string_head', 'string_tail' - returns the beginning/end of a string
- 'string_length' - returns the length of a string
- 'string_reverse' - reverses a string
- 'string_split' - improved implementation, deprecates 'splitstring'
- 'variablesmatching' - returns a list of variables matching a regex and tags
- 'variance' - returns the variance of numbers in a list or container
- New hard classes
- Introduced alias 'policy_server' for context 'am_policy_hub' (the latter will
be deprecated)
- all the time-based classes have GMT equivalents
- New variables
- 'sys.bindir' - the location of the CFEngine binaries
- 'sys.failsafe_policy_path' - the location of the failsafe policy file
- 'sys.inputdir' - the directory where CFEngine searches for policy files
- 'sys.key_digest' - the digest of the host's cryptographic key
- 'sys.libdir', 'sys.local_libdir' - the location of the CFEngine libraries
- 'sys.logdir' - the directory where the CFEngine log files are saved
- 'sys.masterdir' - the location of masterfiles on the policy server
- 'sys.piddir' - the directory where the daemon pid files are saved
- 'sys.sysday' - the number of days since the beginning of the UNIX epoch
- 'sys.systime' - the number of seconds since the beginning of the UNIX epoch
- 'sys.update_policy_path' - the name of the update policy file
- 'sys.uptime' - the number of minutes the host has been online
- 'this.promise_dirname' - the name of the file in which the current promise
is defined
- 'this.promiser_uid' - the ID of the user running cf-agent
- 'this.promiser_gid' - the group ID of the user running cf-agent
- 'this.promiser_ppid' - the ID of the parent process running cf-agent
Deprecations:
- 'splitstring' - deprecated by 'string_split'
- 'track_value'
- 'skipverify'
Bug fixes: for a complete list of fixed bugs, see Redmine at https://cfengine.com/dev
- various fixes in evaluation and variable resolution
- Improve performance of list iteration (Redmine #1875)
- Removed limitation of input length to internal buffer sizes
- directories ending with "/" are not ignored
- lsdir() always return a list now, never a scalar
- 'abortclasses' fixed to work in common bundles and other cases
- namespaced 'edit_line' bundles now work (Redmine#3781)
- lists are interpolated in correct order (Redmine#3122)
- cf-serverd reloads policies properly when they change
- lots of leaks (memory and file descriptor) fixed
3.5.3
Changes:
- Improved security checks of symlink ownership. A symlink created by a user pointing
to resources owned by a different user will no longer be followed.
- Changed the way package versions are compared in package promises. (Redmine #3314)
In previous versions the comparison was inconsistent. This has been fixed, but may
also lead to behavior changes in certain cases. In CFEngine 3.5.3, the comparison
works as follows:
For instance:
apache-2.2.31 ">=" "2.2.0"
will result in the package being installed.
Bug fixes:
- fix cf-monitord crash due to incorrect array initialization (Redmine #3180)
- fix cf-serverd stat()'ing the file tree every second (Redmine #3479)
- correctly populate sys.hardware_addresses variable (Redmine #2936)
- add support for Debian's GNU/kfreebsd to build system (Redmine #3500)
- fix possible stack corruption in guest_environments promises (Redmine #3552)
- work-around hostname trunctation in HP-UX's uname (Redmine #3517)
- fix body copy purging of empty directories (Redmine #3429)
- make discovery and loading of avahi libraries more robust
- compile and packaging fixes for HP-UX, AIX and Solaris
- fix fatal error in lsdir() when directory doesn't exist (Redmine #3273)
- fix epoch calculation for stime inrange calculation (Redmine #2921)
3.5.2
Bug fixes:
- fix delayed abortclasses checking (Redmine #2316, #3114, #3003)
- fix maplist arguments bug (Redmine #3256)
- fix segfaults in cf-pomises (Redmine #3173, 3194)
- fix build on Solaris 10/SmartOS (Redmine #3097)
- sanitize characters from /etc/issue in sys.flavor for Debian (Redmine #2988)
- Fix segfault when dealing with files or data > 4K (Redmine #2912, 2698)
- Don't truncate keys to 126 characters in getindices (Redmine #2626)
- files created via log_* actions now have mode 600 (Redmine #1578)
- fix wrong log message when a promise is ignored due to 'ifvarclass' not matching
- fix lifetime of persistent classes (Redmine #3259)
- fix segfault when process_select body had no process_result attribute
Default to AND'ed expression of all specified attributes (Redmine #3224)
- include system message in output when acl promises fail
- fix invocation of standard_services bundle and corresponding promise compliance (Redmine #2869)
3.5.1
Changes:
- file changes are logged with log level Notice, not Error
- the CFEngine Standard Library in masterfiles/libraries is now split into
promise-type specific policy files, and lives in a version-specific directory.
This should have no impact on current code, but allows more granular include of
needed stdlib elements (Redmine #3044)
Bug fixes:
- fix recursive copying of files (Redmine #2965)
- respect classes in templates (Redmine ##2928)
- fix timestamps on Windows (Redmine #2933)
- fix non-root cf-agent flooding syslog (Redmine #2980)
- fix email flood from cf-execd due to timestamps in agent output (Redmine #3011)
- Preserve security context when editing or copying local files (Redmine #2728)
- fix path for sys.crontab on redhat systems (Redmine #2553)
- prevent incorrect "insert_lines promise uses the same select_line_matching anchor" warning (Redmine #2778)
- Fix regression of setting VIPADDRESS to 127.0.0.1 (Redmine #3010)
- Fix "changes" promise not receiving status when file is missing (Redmine #2820)
- Fix symlinks being destroyed when editing them (Redmine #2363)
- Fix missing "promise kept" status for the last line in a file (Redmine #2943)
3.5.0
New features:
- classes promises now take an optional scope constraint.
- new built-in functions: every, none, some, nth, sublist, uniq, filter
- every
- none
- some
- nth
- sublist
- uniq
- filter
- classesmatching
- strftime
- filestat
- ifelse
- maparray
- format
- cf-promises flag --parse-tree is replaced by --policy-output-format=, requiring the
user to specify the output format (none, cf, json)
- cf-promises allows partial check of policy (without body common control) without integrity check;
--full-check enforces integrity check
- agent binaries support JSON input format (.json file as generated by cf-promises)
- cf-key: new options --trust-key/-t and --print-digest/-p
- Class "failsafe_fallback" is defined in failsafe.cf when main policy contains errors and
failsafe is run because of this
- add scope attribute for body classes (Redmine #2013)
- Better diagnostics of parsing errors
- Error messages from parser now show the context of error
- new cf-agent option: --self-diagnostics
- new output format, and --legacy-output
- warnings for cf-promises.
- Enable zeroconf-discovery of policy hubs for automatic bootstrapping
if Avahi is present
- Support for sys.cpus on more platforms than Linux & HPUX
Changes:
- parser no longer allows ',' after promiser or promisee. must be either ';' or lval
- Make parser output in GCC compatible format the only supported format
(remove --gcc-brief-format flag)
- Silence license warnings in Enterprise Free25 installations
- action_policy => "warn" causes not_kept classes to be set on promise needing repair.
- command line option version (-V) now prints a shorter parsable version without graphic
- implicit execution of server and common bundles taking arguments is skipped in cf-serverd.
- WARNING: option --policy-server removed, require option to --bootstrap instead
- process promises don't log if processes are out of range unless you
run in verbose mode
- reports promises are now allowed in any context (Redmine #2005)
- cf-report has been removed
- cf-execd: --once implies --no-fork
- Version info removed from mail subject in the emails sent by cf-execd.
The subject will only contain "[fqname/ipaddress]" instead of "communnity/nova [fqname/ipaddress]"
Please change your email filters accordingly if necessary.
- "outputs" promise type is retired. Their semantics was not clear, and the functionality
is better suited for control body setting, not a promise.
- Tokyo Cabinet databases are now automatically checked for
correctness during opening. It should prevent a number of issues
with corrupted TC databases causing binaries to hang.
- Improved ACL handling on Windows, which led to some syntax changes. We now consistently
use the term "default" to describe ACLs that can be inherited by child objects. These
keywords have received new names:
acl_directory_inherit -> acl_default
specify_inherit_aces -> specify_default_aces
The old keywords are deprecated, but still valid. In addition, a new keyword
"acl_inherit" controls inheritance behavior on Windows. This feature does not exist on
Unix platforms. (Redmine #1832)
- Networking code is moved from libpromises to its own library,
libcfnet. Work has begun on making the API more sane and thread-safe.
Lots of legacy code was removed.
- Add getaddrinfo() replacement in libcompat (borrowed from PostgreSQL).
- Replace old deprecated and non thread-safe resolver calls with
getaddrinfo() and getnameinfo().
- Hostname2IPString(), IPString2Hostname() are now thread-safe, and are
returning error when resolution fails.
- Running cf-execd --once now implies --no-fork, and also does not wait
for splaytime to pass.
- execresult(), returnszero() and commands promises no longer requires the first word
word to be an absolute path when using the shell. (Part of Redmine #2143)
- commands promises useshell attribute now accepts "noshell" and "useshell" values. Boolean
values are accepted but deprecated. (Part of Redmine #2143)
- returnszero() now correctly sets the class name in this scenario (Part of
Redmine #2143):
classes:
"commandfailed" not => returnszero("/bin/nosuchcommand", "noshell");
Bugfixes:
- bundles are allowed to be empty (Redmine #2411)
- Fixed '.' and '-' not being accepted by a commands module. (Redmine #2384)
- Correct parsing of list variables by a command module. (Redmine #2239)
- Fixed issue with package management and warn. (Redmine #1831)
- Fixed JSON crash. (Redmine #2151)
- Improved error checking when using fgets(). (Redmine #2451)
- Fixed error message when deleting nonexistent files. (Redmine #2448)
- Honor warn-only when purging from local directory. (Redmine #2162)
- Make sure "restart" and "reload" are recognized keywords in packages. (Redmine #2468)
- Allocate memory dynamically to avoid out-of-buffer or out-of-hash
situations
- fix edit_xml update of existing attributes (Redmine #2034)
- use failsafe policy from compile-time specified workdir (Redmine #1991)
- ifvarclass checked from classes promises in common bundles
- do not wait for splaytime when executing only once
- disable xml editing functionality when libxml2 doesn't provide necessary APIs (Redmine #1937)
- Out-of-tree builds should work again, fixed a bunch of related bugs.
- Fixed race condition in file editing. (Redmine #2545)
- Fixed memory leak in cf-serverd and others (Redmine #1758)
3.4.5 (Bugfix and Stability release)
Bugfixes:
- make qualified arrays expand correcty (Redmine #1998, Mantis #1128)
- correct possible errors in tcdb files when opening
- avoid possible db corruption when mixing read/write and cursor operations
- Allow umask value of 002 (Redmine #2496)
3.4.4 (Bugfix and Stability release)
Bugfixes:
- prevent possible crash when archiving files (GitHub #316)
- don't create symlinks to cf-know in update policy
- don't enable xml support if libxml2 is too old (Redmine #1937)
3.4.3 (Bugfix and Stability release)
Bugfixes:
- Don't flood error messages when processes are out of defined range
- prevent segmentation fault in cf-monitord -x (Redmine #2021)
- when copying files, use same file mode as source file, rather than 0600 (Redmine #1804)
- include xpath in messages generated by edit_xml operations (Redmine #2057)
3.4.2 (Bugfix and Stability release)
Bugfixes:
- Fixes to policies in masterfiles (see masterfiles/Changelog for details)
- Fixes for OpenBSD (GitHub #278)
- Do not canonify values specified in abortbundleclasses/abortclasses (Redmine #1786)
- Fix build issues on NetBSD, SLES 12.2
- Improve error message when libxml2 support is not compiled (Redmine #1799)
- fix potential segmentation fault when trimming network socket data (GitHub #233)
- fix potential segmentation fault when address-lookups in lastseen db failed (GitHub #233)
- execute background promise serially when max_children was reached, rather
than skipping them (GitHub #233)
- fix segmentation fault in cf-promises when invoked with --reports (Redmine #1931)
- fix compilation with Sun Studio 12 (Redmine #1901)
- silence type-pun warning when building on HP-UX (GitHub #287)
3.4.1 (Bugfix and Stability release)
New feature/behavior:
- cf-execd terminates agent processes that are not responsive
for a configurable amount of time (see agent_expireafter in body
executor control), defaulting to 1 week
Bugfixes:
- fix regression of classmatch() failing with hard classes (Redmine #1834)
- create promise-defined and persistent classes in correct
namespace (Redmine #1836)
- several fixes to namespace support
- fix several crash bugs caused by buffer overflow and race
conditions in cf-serverd
- regenerate time classes in cf-execd for each run (Redmine #1838)
- edit_xml: fix select_xpath implementation and update documentation
NOTE: code that uses select_xpath_region needs to be changed to
select_xpath
- edit_xml: make sure that text-modification functions don't overwrite
child nodes
- edit_xml: improve error logging
3.4.0
New features:
- Added rpmvercmp utility to compare versions of RPM packages for
accurate sorting of RPM packages for packages promises.
- Implement network timeout on server side to avoid keeping stale
connections for hours.
- XML editing capabilities. See the documentation for edit_xml
body. Note the new dependency: libxml2.
- Implement inheritance of local classes by bundles called using
"usebundle". By default classes are not inherited. See the
examples/unit_inherit.cf for an example.
- Moved from Nova/Enterprise:
- POSIX ACL support,
- "outputs" promise type,
- remote syslog support.
- packages_default_arch_command hook in packages promises, to
specify default architecture of the packages on the system.
- packages_version_less_command / packages_version_equal_command hooks
in packages promises, to specify external command for native package
manager versions comparison
- agent_expireafter in body executor control allows you to set a
timeout on all cf-agent runs, to enforce a threshold on the
number of concurrent agents
- Running in Solaris zone is now detected and classes "zone" and
"zone_" are created in this case.
- VirtualBox support added to guest_environment promises.
- guest_environment promises are supported under OS X.
- The "depends_on" attribute is now active, for the partal ordering
of promises. If a promise depends on another (referred by handle)
it will only be considered if the depends_on list is either kept
or repaired already.
** WARNING: When upgrading, make sure that any existing use
of depends_on does not make some promises being
unintentionally ignored. This can happen if you are
currently referring to non-existent or never-run handles
in depends_on attributes.
- methods return values, initial implementation
- New format for cf-key -s, includes timestamp of last connection
- cf-promises --parse-tree option to parse policy file and dump it
in JSON format
- Namespaces support for bundles and bodies. See the
examples/unit_namespace*.cf for the usage.
- Default arguments for bundles. See the examples/unit_defaults.cf
- Metadata promise type. See the examples/unit_meta.cf
New semantics:
- Methods promises now return the status of promises
kept within them. If any promise was not kept, the method is not
kept, else if any promise is repaired, the method was repaired
else it was kept.
- Remote variable access in namespaces by $(namespace:bundle.variable)
Changed functionality:
- cf-execd -F switch no longer implies 'run once'. New -O/--once
option is added to achieve this behaviour. This makes cf-execd
easier to run from systemd, launchd and other supervision
systems.
Misc:
- Support for the following outdated platforms and corresponding
classes has been removed. De facto those platforms were
unsupported for a long time, as CFEngine codebase uses C99
language features unavailable on old platforms:
- SunOS 3.x (sun3)
- SunOS 4.x (sun4)
- Ultrix (ultrix)
- DEC OSF/1 AXP (osf)
- Digital UNIX (digital)
- Sony NEWS (newsos)
- 4.3BSD (bsd4_3)
- IRIX (irix, irix4, irix64)
- IBM Academic Operating System (aos)
- BSD/OS / BSDi / BSD/386 (bsdos)
- NeXTSTEP (nextstep)
- GNU Hurd (gnu)
- NEC UX/4800 (ux4800)
- (Old news) Since 3.3.0 the layout of CFEngine Community packages
has changed slightly.
cf-* binaries have been moved to /var/cfengine/bin, due to the
following reasons:
- cf-* binaries are linked to libraries installed to
/var/cfengine/lib, so placing binaries in /usr/local/sbin does not
increase reliability of the CFEngine,
- keeping whole CFEngine under single prefix (/var/cfengine)
makes packaging simpler,
- it matches the layout of CFEngine Enterprise packages.
Please adjust your policies (the recommended ways to deal with
the move are either to adjust $PATH to include /var/cfengine or to
create symlinks in /usr/local/sbin in case you are relying on
binaries to be available in $PATH).
- Workdir location is properly changed if --prefix or --enable-fhs
options are supplied to configure (Mantis #1195).
- Added check for broken libmysqlclient implementations (Mantis #1217).
- Standard library is updated from COPBL repository.
- cf-know is no longer built in Community releases. The only
functionality useful in Community, namely the reference manual
generation, is provided by new compile-time cf-gendoc tool.
- Filename (for storing filechanges) changed
from file_change.log -> file_changes.log (in /var/cfengine/state)
New format for storing file changes introduced:
[timestamp,filename,,Message]
N = New file found
C = Content Changed
S = Stats changed
R = File removed
- Acceptance test suite passes on Mac OS X.
- Changed some port numbers to replace old services with imap(s)
- archlinux hard class on Arch Linux.
- Detect BSD Make and automatically switch to GNU Make during build.
Bugfixes:
- cfruncommand for cf-execd is an arbitrary shell command now (Mantis #1268).
- Fixed broken "daily" splayclasses (Mantis #1307).
- Allow filenames up to 4096 bytes in network transfers (Redmine #1199).
- Fix stale state preserved during cf-serverd reload (Redmine #1487).
- Free disk space calculation is fixed (Mantis #1120).
- Numerous portability bugfixes (especially OpenBSD, Solaris, AIX-related).
- Compatibility fixes for AIX, HP-UX, Solaris (Mantis #1185, Mantis #1177, Mantis #1109).
- Fixed broken socklen_t configure check under OpenBSD (Mantis #1168).
- Fixed hang in cf-promises under OpenBSD (Mantis #1113).
- Fixed endless loop in evaluating "$()" construct (Mantis #1023).
- Fixed check for old PCRE versions (Mantis #1262).
- Fixed insertion of multi-line blocks at the start of file (Mantis #809).
- Fixed numerous memory leaks.
- Fixes for metadata that were not resolvable
- Fixes for namespaces that would not support metadata and variable expansion
- Point-to-point network interfaces are detected and reported by CFEngine (Mantis #1246)
- Partial non-GNU userspace support in acceptance testsuite (Mantis #1255)
Full list of issues fixed is available on
https://cfengine.com/bugtracker/changelog_page.php (old bug tracker)
and https://cfengine.com/dev/projects/core/versions/34 (new bug tracker)
3.3.9 (Bugfix and Stability release)
Bugfixes:
- Do not lose hard classes in cf-serverd during policy reload
(Mantis #1218).
- Implement receive network timeout in cf-serverd. Prevents
overloading cf-serverd with stale connections.
3.3.8 (Bugfix and Stability release)
Versions 3.3.6, 3.3.7 were internal and weren't released.
Bugfixes:
- Propery set sys.domain variable if hostname is fully-qualified.
- Fixed several small memory leaks.
- Make network timeout for network reads configurable. Previously
it was hardcoded to be 30 seconds, which was not enough for
cf-runagent invoking cf-agent on big policies (Mantis #1028).
3.3.5 (Bugfix and Stability release)
Bugfixes:
- Fixed cf-execd memory leak on hosts with cf-monitord running.
- Robustify against wrongly-sized entires in embedded databases.
Standard library:
- Bugfixes from upstream COPBL repository.
- standard_services bundle from upstream COPBL repository.
3.3.4 (Bugfix and Stability release)
Evaluation of policies:
- Fix wrong classes set after installation of several packages
using packages promises (Mantis #829).
- Fix segfault using edit_template on existing file (Mantis #1155).
Misc:
- Fix memory leak during re-read of network interfaces'
information in cf-execd/cf-serverd.
3.3.3 (Bugfix and Stability release)
Evaluation of policies:
- Zero-length files are valid for readfile() and similar functions
(Mantis #1136).
- Unchoke agent in case it encounters symlinks in form ./foo
(Similar to Mantis #1117).
Misc:
- Fix generation of reference manual on machines with umask more
relaxed than 022.
- Use statvfs(3) on OpenBSD to obtain filesystem information
(Mantis #1135).
3.3.2 (Bugfix and Stability release)
Evaluation of policies:
- Do not segfault if file copy was interrupted due to network
connectivity or server going away (Mantis #1089).
- Do not segfault if log_failed attribute is present in body, but
log_kept is not (Mantis #1107).
- Do not mangle relative paths in symlinks during file copy
Previously symlink a -> b was mangled to a -> ./b.
(Mantis #1117)
- Properly compare 1.0 and 1.0.1 in packages promises. Previously
only versions with equal amount of "segments" were comparable
(Mantis #890, #1066).
Base policy:
- Properly set permissions on files for /var/cfengine/lib on HP-UX
(Mantis #1114).
- Standard library (cfengine_stdlib.cf) is synced with COPBL
repository.
Misc:
- Do not create huge file in case corrupted TokyoCabinet database
is detected (Mantis #1106).
- Fix file descriptor leak on error paths, may have caused crashes
of cf-execd and cf-serverd (Issue #1096).
- Fix intermittent segfault in cf-execd (Mantis #1116).
- Impose an upper limit on amount of listening sockets reported by
cf-monitord. Huge amounts of listening sockets caused cf-agent to
segfault on next run (Mantis #1098).
- Add missing function prototypes caused errors during compilation
on HP-UX (Mantis #1109).
- Fix compilation on Solaris 11 (Mantis #1091).
3.3.1 (Bugfix and Stability release)
Evaluation of policies:
- Do not cut off name of bundle in variables interpolation (Mantis #975).
- Do not segfault in function evaluation guarded by ifvaclass clause (Mantis #1084, #864).
- Do not segfault if "classes" promise does not declare any value to be evaluated (Mantis #1074).
- Do not segfault in database promises if there is no
database_operation provided (Mantis #1046).
Built-in functions:
- Fix countclassesmatching() function which was misbehaving trying
to match classes starting with alphanumeric symbol (Mantis #1073).
- Fix diskfree() to return kilobytes, as described in documentation (Mantis #980, #955).
- Fix hostsseen() function to avoid treating all hosts as not
being seen since 1970 (Mantis #886).
- Do not output misleading error message if readtcp() is unable to connect (Mantis #1085).
Command-line interface:
- -d option previously reqired an argument, though help message disagreed (Mantis #1053).
- Disable --parse-tree option, not ready for the release (Mantis #1063).
- Acept -h as a --help option.
- Ensure that cf-execd might be started right after being shut down.
Misc:
- Plug file descriptor leak after failed file copy (Mantis #990).
- Fix unsafe admit rules in default promises.cf (Mantis #1040).
- Fix splaytime to match documentation: it is specified in minutes, not seconds (Mantis #1099).
Packaging:
- Fix owner/group of initscript and profile.d snippet in RPM builds (Mantis #1061, #1058).
- Fix location of libvirt socket CFEngine uses to connect to libvirtd (Mantis #1072).
- Install CoreBase to /var/cfengine/masterfiles during installation (Mantis #1075).
- Do not leave old cf-twin around after upgrade (Mantis #1068)
- Do not leave rcS.d symlinks after purging .deb package (Mantis #1092).
3.3.0
New promise types:
- Guest environments promises, which allow to manipulate virtual
machines using libvirt.
- Database promises, which allow to maintain schema of MySQL and
PostgreSQL databases. Database promises are in "technical preview"
status: this promise type is subject to change in future.
- Services promises for Unix, allows abstraction of details
on managing any service
New built-in functions:
- dirname() to complement lastnode()
- lsdir()
- maplist() to apply functions over lists
New features:
- Allow defining arrays from modules.
- Allow both `process_stop' and `signals' constraints in
`processes' promises at the same time.
- cf-promises --gcc-brief-format option to output warnings and
errors in gcc-compatible syntax which to ease use "go to next
error" feature of text editors.
- Iteration over lists is now allowed for qualified (non-local) lists.
New built-in variables and classes (Linux):
- Number of CPUs: $(sys.cpus), 1_cpu, 2_cpus etc
New built-in variables and classes (Unices):
- $(sys.last_policy_update) - timestamp when last policy change was seen by host
- $(sys.hardware_addresses) - list of MAC adresses
- $(sys.ip_addresses) - list of IP addresses
- $(sys.interfaces) - list of network interfaces
- $(sys.hardware_mac[$iface]) - MAC address for network interface
- mac_:: - discovered MAC addresses
Changes:
- Major cleanup of database handling code. Should radically decrease
amount of database issues experienced under heavy load.
*WARNING*: Berkeley DB and SQLite backends are *removed*, use
Tokyo Cabinet or QDBM instead. Both Tokyo Cabinet and QDBM are
faster than Berkeley DB in typical CFEngine workloads.
Tokyo Cabinet requires C99 environment, so it should be
available on every contemporary operating system.
For the older systems QDBM, which relies only on C89, is a
better replacement, and deemed to be as portable, as Berkeley DB.
- Change of lastseen database schema. Should radically decrease
I/O contention on lasteen database.
- Automatic reload of policies by cf-execd.
- Documentation is generated during build, PDF and HTML files are
retired from repository.
- Rarely used feature retired: peer connectivity intermittency calculation.
- Memory and CPU usage improvements.
- Testsuite now uses 'make check' convention and does not need root
privileges anymore.
- cf_promises_validated now filled with timestamp, allows digest-copy
for policy instead of mtime copy which is safer when clocks are unsynchronised
- The bundled failsafe.cf policy now has trustkey=false to avoid IP spoofing
attacks in default policy
- See the full list of bugfixes at
https://cfengine.com/bugtracker/changelog_page.php
3.2.4 (Bugfix and Stability release)
Fixed failure in network transfer in case of misbehaving peer
A few tiny memory leaks on error paths fixed
3.2.3 (Bugfix and Stability release)
A few tiny memory leaks fixed
Improved performance of cf-serverd under heavy load with
TokyoCabinet database
Full list of issues fixed is available on
https://cfengine.com/bugtracker/changelog_page.php
3.2.2 (Bugfix and Stability release)
Enabled compilation in "large files" mode under AIX
Alleviated problem with broken file transfers over unstable
Internet links.
Full list of issues fixed is available on
https://cfengine.com/bugtracker/changelog_page.php
3.2.1 (Bugfix and Stability release)
Fixed compilation under HP-UX and Solaris
Enabled compilation using HP ANSI C compiler
Full list of issues fixed is available on
https://cfengine.com/bugtracker/changelog_page.php
3.2.0
New bootstrap method with single-command bootstrapping:
- cf-agent --bootstrap --policy-server 123.456.789.123
- Associated policy template files are added, partially maintained
by CFEngine
Bug fixes for file-editing, package versioning, and embedded
database corruption (We recommend using TokyoCabinet instead of
BerkeleyDB if building from source).
Improved upgrade path for Nova.
Patches for improved run-agent concurrency
Reorganization of documentation and community resources
100% on regression test suite on 3 operating systems
(Ubuntu, Debian, SuSE on x86-64 hardware)
Support for multiple release environments
package_policy update and addupdate now check if user-supplied
version is larger than currently installed - updates only if so
Help text of cf-report -r corrected - a list of key hashes is
required, not ip addresses.
New Emacs mode for CFEngine policy files (thanks to Ted Zlatanov!)
Warnings are on edit_line changes can now give greater degree of information
without spamming promise logs
Class expressions parser accepts '||' as an alias for '|' again.
Invalidation of package list cache on installation/removal of
packages.
New option cf-key -r to remove host key by IP or hostname.
Added detection of network interfaces which belong to BSD jails.
Improve robustness of multi-threaded code, in particular fix
problems with spurious access denials in server and losing of
authentication rules after policy reload.
cf-promises accepts option -b matching cf-agent, which causes it
to do not complain about missing bundlesequence.
New functions and(), not(), or() and concat() to ease use of
ifvarclass() clause.
Full list of issues fixed is available on
https://cfengine.com/bugtracker/changelog_page.php
3.1.5
New class parser, '||' is no longer allowed in expressions (use '|').
Class setting in the promise types insert_lines, delete_lines,
replace_patterns, field_edits, vars, classes is restored.
suspiciousnames implemented.
New function getvalues().
New functions parse{read,int,string}array to match read{read,int,string}array.
Testsuite added to check for core functionality.
Syslog prefix is fixed to say 'cf3' instead of 'community'.
3.1.4 (Bugfix and Stability release)
Some urgent patches to 3.1.3.
Class validation parse bug fixed.
Global zone handling error for solaris fixed.
Package architectures handled correctly (bug #456).
Reading and writing of key name "root-.pub" eliminated (bug #442, #453).
cf-serverd crash because of race condition on SERVER_KEYSEEN fixed.
Lock purging to avoid remnant complexity explosion (bug #430).
Some copyright notices added that got lost.
3.1.3 (Stability release)
Major memory leaks in cf-monitord, cf-execd, cf-serverd fixed (bug #427).
The daemons now show no growth even with very complex policies.
cf-serverd crash due to race condition in DeleteScope() fixed (bug #406).
Added 30 second timeout on recv() on Linux.
package_noverify_returncode implemented (bug #256).
A flexible mechanism for setting classes based on return codes of
commands has been introduced. Allows for setting promise kept,
repaired or failed based on any return codes. This is currently
implemented for commands-promises, package-manager commands and
transformer in files. In classes body, see attributes
kept_returncodes, repaired_returncodes, failed_returncodes (bug
#248, #329).
New function ip2host - reverse DNS lookup (bug #146).
3.1.2 (Scalability/efficiency release)
Big efficiency improvements by caching output from
cf-promises. Can also be used for much more efficient policy
deployment (only pull if changed).
Caching state of ps command for greater efficiency. Reloaded for each bundle.
Index class lookup improves efficiency of class evaluation for huge configurations.
Fixed issue where certain promiser strings got corrupted.
Minor memory access issues fixed.
Iterator bug introduced in 3.1.0 fixed
3.1.1 (Bugfix release)
Memory leaks in server tracked down and fixed.
List expansion bug (one list items not executed) fixed.
Security issue introduced by change of runcommand shell policy fixed. If users defined a runcommand for cf-runagent/cf-serverd communication, possible to execute commands.
cf-key -s command for showing key hash/IP address identity pairs
3.1.0
Change in storage of public keys. Cfengine now hashes the public key and uses this
as the keyname. Keys will be converted automatically.
The old dynamic addresses lists are deprecated.
Caching of dns and key information for greater server speed.
Change in last-seen format reflects the public key usage.
New package policy addupdate - installs package if not there and
updates it otherwise.
Support for package_changes => "bulk" in file repository as well.
New special function readstringarrayidx, similar to readstringarray,
but uses integer indices. Very useful if first row elements are
not good identifiers (e.g. contains spaces, non-unique, etc.).
Change two log formats to use time() instead of date()
- filechanges
- total compliance
Change from using md5 to sha256 as default digest for commercial version,
community retains md5 for compat.
Commands not returning 0 in commands-promises are flagged
as repair_failed.
Adjustable timeout on connect(). Defaults to 10 seconds, adjustable
with default_timeout in agent control.
Redesign of the knowledge map infrastructure.
Now possible to use variables to call methods, e.g
methods:
"name $(list)" usebundle => $(list)("abc");
See reference manual notes
Changes to normal ordering to optimize execution.
Increased stability by always initializing Attribute and Promise
structures.
When running cf-promises in dry-run mode (-n), the user does not need
to put binaries in WORKDIR/bin. For example, non-privileged users can verify root
policies.
Source control revision added in version string if run in verbose mode
(e.g. "cf-promises -vV"). This needs some refining, uses revision of a header now.
New semantics in return values of list functions. Null values are now allowed
and there is no iteration over empty lists. The value "cf_null" is reserved for
use as a null iterator.
3.0.5p1
Showing paths allowed/denied access to when cf-serverd is run in verbose mode.
Bug in server fixed for dynamic addresses.
File handle closure bugfix - too many open databases.
Seg fault in mount files fix.
Twin used in cf-execd without checking.
Check_root set wrong directory permissions at source not destination.
Error message degraded in body definition.
Undefined body not warned as error.
Various build enahncements.
Package_list_update called only once per manager, and fixed crash.
Version number bug in packages.
3.0.5
Encryption problems fixed - client key buffer was uninitialized.
Classes-promisers are now automatically canonified when class
strings are defined, to simplifying the use of variables in classes.
New scalars sys.cf_version and sys.nova_version that hold Cfengine version information.
Attribute package_delete_convention added, to allow customizable
package name in delete command during update.
package_list_update_ifelapsed limit added.
Private variable $(firstrepo) is available in package_name_convention
and package_delete_convention in order to expand the full path to
a package, which is required by some managers.
Some of the threading code is rewritten and made more robust. This includes
synchronizing access to the lastseen database from the server.
Bad initialization of BSD flags fixed
Multiple variable expansion issues in control fixed for server and agent
Allow ignore_missing_bundles to affect methods: bundles too
Run agent trust dialogue fixed
Bug in CPU monitoring, increasing time scale caused linear decay
of CPU measurement.
Bug in Setuid log storage, fix.
Hooks added for new Nova virtualization promises.
Multithreading mutex failed to collide during cfservd leading to dropped authentication under heavy load.
3.0.4
Class cancellation in promises to create better class feedback,
allows emulation of switch/case semantics etc
Value of SA measurement promises
Special function getenv() which returns the contents of an
environment variable (on all platforms).
New function translatepath for generic Windows
New function escape() to escape literals as regular expressions (like SQL)
New function host2ip for caching IP address lookup
New function regextract for setting variables with backreferences
New variables for the components $(sys.cf_agent), $(sys.cf_know) etc
pointing to the binaries.
More robust integrated database implementation; closing all
handles when receiving signals, self-healing on corruption.
Package installation on localhost without a manager like yum completed,
multiple repositories searched, and universal methods.
Numerous bugfixes
3.0.3
sha256 .. new hashes in openssl included in syntax tree.
End of line autocropping in readfile (hopefully intelligent)
hashmatch function incorrectly implemented - old debugging code left behind. Fix.
sys.crontab variable
Unknown user is now interpretated as "same user", so that we give cfengine a chance to
fix
Unregistered addresses no longer report "(Non registered IP)", but return as the address
itself when doing reverse lookups.
3.0.2
IMPORTANT: Change in normal ordering of editing. replace comes
after insert lines Much testing and minor bug fixing
Memory leaks fixed
Many hooks added for Nova enterprise extensions.
promise_output reports now placed in WORKDIR/reports directory
Initialization correction and self-correx in monitord
Many new body constraints added.
Code readied for enterprise version Nova.
-b option can override the bundlesequence (must not contain parameters yet)
collapse_destination_dir option added to copy so that files can be
aggregated from subdirectories into a single destination.
Preparation for release:
unit_accessed_before.cf x
unit_accumulated_time.cf x
unit_acl.cf x
unit_acl_generic.cf x
unit_ago.cf x
unit_arrays.cf x
unit_backreferences_files.cf x
unit_badpromise.cf x
unit_badtype.cf x
unit_bsdflags.cf x
unit_cf2_integration.cf x
unit_changedbefore.cf x
unit_change_detect.cf x
unit_chdir.cf x
unit_classes_global.cf x
unit_classmatch.cf x
unit_classvar_convergence.cf x
unit_compare.cf x
unit_controlclasses.cf x
unit_control_expand.cf x
unit_copy.cf x
unit_copy_edit.cf x
unit_copylinks.cf x
unit_createdb.cf x
unit_create_filedir.cf x
unit_definitions.cf x
unit_deletelines.cf x
unit_disable_and_rotate_files.cf x
unit_dollar.cf x
unit_edit_column_files.cf x
unit_edit_comment_lines.cf x
unit_edit_deletenotmatch.cf x
unit_edit_insert_lines.cf x
unit_edit_insert_lines_silly.cf x
unit_edit_replace_string.cf x
unit_edit_sectioned_file.cf x
unit_edit_setvar.cf x
unit_edit_triggerclass.cf x
unit-env.cf x
unit_epimenides.cf x
unit_exec_args.cf x
unit_execd.cf x
unit_exec_in_sequence.cf x
unit_execresult.cf x
unit_expand.cf x
unit_failsafe.cf x
unit_file_change_detection.cf x
unit_fileexists.cf x
unit_file_owner_list_template.cf x
unit_fileperms.cf x
unit_filesexist2.cf x
unit_filesexist.cf x
unit_getgid.cf x
unit_getindices.cf x
unit_getregistry.cf x
unit_getuid.cf x
unit_global_list_expansion_2.cf x
unit_global_list_expansion.cf x
unit_groupexists.cf x
unit_hash.cf x
unit_hashcomment.cf x
unit_hashmatch.cf x
unit_helloworld.cf x
unit_hostrange.cf x
unit_intarray.cf x
unit_iprange.cf x
unit_irange.cf x
unit_isdir.cf x
unit_islink.cf x
unit_isnewerthan.cf x
unit_isplain.cf x
unit_isvariable.cf x
unit_iteration.cf x
unit_knowledge_txt.cf x
unit_lastnode.cf x
unit_ldap.cf x
unit_linking.cf x
unit_literal_server.cf x
unit_locate_files_and_compress.cf x
unit_log_private.cf x
unit_loops.cf x
unit_measurements.cf x
unit_method.cf x
unit_method_validate.cf x
unit_module_exec_2.cf
unit_module_exec.cf
unit_mount_fs.cf x
unit_neighbourhood_watch.cf x
unit_null_config.cf x
unit_occurrences.cf x
unit_ordering.cf x
unit_package_apt.cf x
unit_package_hash.cf x
unit_package_rpm.cf x
unit_package_yum.cf x
unit_package_zypper.cf x
unit_parallel_exec.cf x
unit_pathtype.cf x
unit_pattern_and_edit.cf x
unit_peers.cf x
unit_postfix.cf x
unit_process_kill.cf x
unit_process_matching2.cf x
unit_process_matching.cf x
unit_process_signalling.cf x
unit_readlist.cf x
unit_readtcp.cf x
unit_regarray.cf x
unit_registry.cf x
unit_regline.cf x
unit_reglist.cf x
unit_remove_deadlinks.cf x
unit_rename.cf x
unit_report_state.cf x
unit_reporttofile.cf x
unit_returnszero.cf x
unit_select_mode.cf x
unit_select_region.cf x
unit_selectservers.cf x
unit_select_size.cf x
unit_server_copy_localhost.cf x
unit_server_copy_remote.cf x
unit_server_copy_purge.cf x
unit_splitstring.cf x
unit_sql.cf x
unit_storage.cf x
unit_strcmp.cf x
unit_stringarray.cf x
unit_syslog.cf x
unit_template.cf x
unit_tidy_all_files.cf x
unit_user_edit.cf x
unit_user_edit_method.cf x
unit_userexists.cf x
unit_varclass.cf x
unit_vars.cf x
unit_warnifline.cf x
unit_webserver.cf x
3.0.1
First standalone release, independent of cfengine 2
Purge old definitions and check consistency.
NB: changed search_mode to be a list of matching values
Reporting rationalized in cf-promises with -r only to avoid
leaving output files everywhere.
Hooks added for upcoming commercial additions to cfengine.
Added classify() and hostinnetgroup() functions
Added additional change management options for change detection
Package management added - generic mechanisms.
Limits on backgrounding added to avoid resource contention during cfengine runs.
Image type added to cf-know.
New classes for quartly shifts: Morning,Afternoon,Evening,Night
Bug fixes in editfiles - line insertion for multiple line objects
Change the name of the variables and context from the monitord for
better separation of data, and shorter names. sys -> mon
average -> av, stddev -> dev
canonical name for windows changed from "nt" to "windows", also version names
added "vista","xp" etc..
License notices updated for dual license editions.
3.0.0
First release of cfengine 3. Known omissions:
- no support for ACLs
- no support for packages
- no support for interface configuration
These will be added in the next release.