Up to [cvs.NetBSD.org] / pkgsrc / sysutils / syslog-ng-redis
Request diff between arbitrary revisions
Keyword substitution: kv
Default branch: MAIN
syslog-ng*: update to 3.38.1
snmp module untested since net-snmp doesn't currently build on NetBSD-current
Changes:
3.38.1
======
## Highlights
### Sneak peek into syslog-ng v4.0
syslog-ng v4.0 is right around the corner.
This release (v3.38.1) contains all major changes, however, they are
currently all hidden behind a feature flag.
To enable and try those features, you need to specify `@version: 4.0` at the
top of the configuration file.
You can find out more about the 4.0 changes and features
[here](https://github.com/syslog-ng/syslog-ng/blob/master/NEWS-4.0.md).
Read our practical introduction to typing at
[syslog-ng-future.blog](https://syslog-ng-future.blog/syslog-ng-4-progress-3-38-1-release/).
## Features
* `grouping-by()`: added `inject-mode(aggregate-only)`
This inject mode will drop individual messages that make up the correlation
context (`key()` groups) and would only yield the aggregate messages
(e.g. the results of the correlation).
([#3998](https://github.com/syslog-ng/syslog-ng/pull/3998))
* `add-contextual-data()`: add support for type propagation, e.g. set the
type of name-value pairs as they are created/updated to the value returned
by the template expression that we use to set the value.
The 3rd column in the CSV file (e.g. the template expression) now supports
specifying a type-hint, in the format of "type-hint(template-expr)".
Example line in the CSV database:
selector-value,name-value-pair-to-be-created,list(foo,bar,baz)
([#4051](https://github.com/syslog-ng/syslog-ng/pull/4051))
* `$(format-json)`: add --key-delimiter option to reconstruct JSON objects
using an alternative structure separator, that was created using the
key-delimiter() option of json-parser().
([#4093](https://github.com/syslog-ng/syslog-ng/pull/4093))
* `json-parser()`: add key-delimiter() option to extract JSON structure
members into name-value pairs, so that the names are flattened using the
character specified, instead of dot.
Example:
Input: {"foo":{"key":"value"}}
Using json-parser() without key-delimiter() this is extracted to:
foo.key="value"
Using json-parser(key-delimiter("~")) this is extracted to:
foo~key="value"
This feature is useful in case the JSON keys contain dots themselves, in
those cases the syslog-ng representation is ambigious.
([#4093](https://github.com/syslog-ng/syslog-ng/pull/4093))
## Bugfixes
* Fixed buffer handling of syslog and timestamp parsers
Multiple buffer out-of-bounds issues have been fixed, which could cause
hangs, high CPU usage, or other undefined behavior.
([#4110](https://github.com/syslog-ng/syslog-ng/pull/4110))
* Fixed building with LibreSSL
([#4081](https://github.com/syslog-ng/syslog-ng/pull/4081))
* `network()`: Fixed a bug, where syslog-ng halted the input instead of skipping a character
in case of a character conversion error.
([#4084](https://github.com/syslog-ng/syslog-ng/pull/4084))
* `redis()`: Fixed bug where using redis driver without the `batch-lines` option caused program crash.
([#4114](https://github.com/syslog-ng/syslog-ng/pull/4114))
* `pdbtool`: fix a SIGABRT on FreeBSD that was triggered right before pdbtool
exits. Apart from being an ugly crash that produces a core file,
functionally the tool behaved correctly and this case does not affect
syslog-ng itself.
([#4037](https://github.com/syslog-ng/syslog-ng/pull/4037))
* `regexp-parser()`: due to a change introduced in 3.37, named capture groups
are stored indirectly in the LogMessage to avoid copying of the value. In
this case the name-value pair created with the regexp is only stored as a
reference (name + length of the original value), which improves performance
and makes such name-value pairs use less memory. One omission in the
original change in 3.37 is that syslog-ng does not allow builtin values to
be stored indirectly (e.g. $MESSAGE and a few of others) and this case
causes an assertion to fail and syslog-ng to crash with a SIGABRT. This
abort is now fixed. Here's a sample config that reproduces the issue:
regexp-parser(patterns('(?<MESSAGE>.*)'));
([#4043](https://github.com/syslog-ng/syslog-ng/pull/4043))
* set-tag: fix cloning issue when string literal were used (see #4062)
([#4065](https://github.com/syslog-ng/syslog-ng/pull/4065))
* `add-contextual-data()`: fix high memory usage when using large CSV files
([#4067](https://github.com/syslog-ng/syslog-ng/pull/4067))
## Other changes
* The `json-c` library is no longer bundled in the syslog-ng source tarball
Since all known OS package managers provide json-c packages nowadays, the json-c
submodule has been removed from the source tarball.
The `--with-jsonc=internal` option of the `configure` script has been removed
accordingly, system libraries will be used instead. For special cases, the JSON
support can be disabled by specifying `--with-jsonc=no`.
([#4078](https://github.com/syslog-ng/syslog-ng/pull/4078))
* platforms: Dropped support for ubuntu-impish as it became EOL
([#4088](https://github.com/syslog-ng/syslog-ng/pull/4088))
revbump for icu and libffi
syslog-ng: update to 3.34.1 Based on package provided by Peter Czanik in PR 56406. Changes below are for 3.34.1 only, the other 16 releases before that are not summarized. 3.34.1 ====== ## Highlights * `regexp-parser()`: new parser that can parse messages with regular expressions * `redis()`: `workers()` and batching support The Redis driver now support the `workers()` option, which specifies the number of parallel workers, and the `batch-lines()` option. This could drastically increase the throughput of the Redis destination driver. * `mqtt()`: TLS and WebSocket Secure support The MQTT destination now supports TLS and WSS. ## Features * `system()` source: added support for NetBSD * `stats`: new statistics counter * `mqtt()`: username/password authentication * `mqtt()`: new option `http-proxy()` for specifying HTTP/HTTPS proxy for WebSocket connections ([#3747](https://github.com/syslog-ng/syslog-ng/pull/3747)) * `syslog-ng-ctl`: new flag for pruning statistics * `disk-buffer()`: added a new option to reliable disk-buffer: `qout-size()`. ## Bugfixes * `network(), syslog()`: fixed network sources on NetBSD * `disk-buffer()`: fixed a very rare case, where the reliable disk-buffer never resumed after triggering `flow-control`. * `disk-buffer()`: fixed a rare memory leak that occurred when `mem-buf-length()` or `mem-buf-size()` was configured incorrectly * `redis()`: fixed command errors that were not detected and marked as successful delivery
Update syslog-ng to the latest stable 3.7.2 version. Rework and clean up the package, split off various bindings into separate packages. Add SMF support. Major features and improvements introduced in major releases since 3.2. 3.7 - OpenSSL is now a required dependency for syslog-ng. - Java-destination driver ported from syslog-ng-incubator. - Python language support is ported from syslog-ng incubator. - New Java destination drivers - New Parsers 3.6 - PCRE is now a required dependency of syslog-ng. - Threaded mode is now enabled by default. 3.5 - Multi-line support - STOMP destination - Redis destination - Template type hinting - Template options honored everywhere - Support for unit suffixes in the configuration - The Incubator project 3.4 - New plugins: AMQP & SMTP destinations, JSON parser. - New parsers for patterndb: HOSTNAME, EMAIL, PCRE and LLADDR. - It is now possible to control what db-parser() sees as its input via it's new template() option. - value-pairs() gained support for programmatically rewriting key names in bulk, via the rekey() method. - The network() driver is introduced, unifying and extending tcp(), udp(), syslog(), unix-dgram() and unix-stream(). The old drivers are still available, but - Support for junctions & channels were added, which improve the flexibility of the syslog-ng configuration language. 3.3 - multi-core/CPU scaling: the new multi-threaded architecture allows syslog-ng to scale into the 800k msg/sec region. - MongoDB support: using MongoDB instead of SQL is faster and allows better representation of log data. - JSON support: using the $(format-json) template function it is now possible to construct JSON (JavaScript Object Notation) output for log messages. - A number of enhancements all over the place: SQL, patterndb. - The default ports have changed. syslog-ng is using the standard