![[BACK]](/icons/back.gif){kind=icon}
Up to [cvs.NetBSD.org] / pkgsrc / devel / librdkafka
Request diff between arbitrary revisions
Default branch: MAIN
Revision 1.15 / (download) - annotate - [select for diffs], Tue Oct 24 22:08:52 2023 UTC (5 months, 3 weeks ago) by wiz
Branch: MAIN
CVS Tags: pkgsrc-2024Q1-base,
pkgsrc-2024Q1,
pkgsrc-2023Q4-base,
pkgsrc-2023Q4,
HEAD
Changes since 1.14: +2 -2
lines
Diff to previous 1.14 (colored)
*: bump for openssl 3
Revision 1.14 / (download) - annotate - [select for diffs], Mon Aug 14 05:24:12 2023 UTC (8 months ago) by wiz
Branch: MAIN
CVS Tags: pkgsrc-2023Q3-base,
pkgsrc-2023Q3
Changes since 1.13: +2 -1
lines
Diff to previous 1.13 (colored)
*: recursive bump for Python 3.11 as new default
Revision 1.13 / (download) - annotate - [select for diffs], Fri Aug 11 09:05:24 2023 UTC (8 months, 1 week ago) by nia
Branch: MAIN
Changes since 1.12: +3 -1
lines
Diff to previous 1.12 (colored)
librdkafka: Eliminate -Werror usage
Revision 1.12 / (download) - annotate - [select for diffs], Wed May 27 19:37:39 2020 UTC (3 years, 10 months ago) by wiz
Branch: MAIN
CVS Tags: pkgsrc-2023Q2-base,
pkgsrc-2023Q2,
pkgsrc-2023Q1-base,
pkgsrc-2023Q1,
pkgsrc-2022Q4-base,
pkgsrc-2022Q4,
pkgsrc-2022Q3-base,
pkgsrc-2022Q3,
pkgsrc-2022Q2-base,
pkgsrc-2022Q2,
pkgsrc-2022Q1-base,
pkgsrc-2022Q1,
pkgsrc-2021Q4-base,
pkgsrc-2021Q4,
pkgsrc-2021Q3-base,
pkgsrc-2021Q3,
pkgsrc-2021Q2-base,
pkgsrc-2021Q2,
pkgsrc-2021Q1-base,
pkgsrc-2021Q1,
pkgsrc-2020Q4-base,
pkgsrc-2020Q4,
pkgsrc-2020Q3-base,
pkgsrc-2020Q3,
pkgsrc-2020Q2-base,
pkgsrc-2020Q2
Changes since 1.11: +2 -2
lines
Diff to previous 1.11 (colored)
*: reset MAINTAINER for fhajny on his request
Revision 1.11 / (download) - annotate - [select for diffs], Sat Feb 1 20:49:43 2020 UTC (4 years, 2 months ago) by adam
Branch: MAIN
CVS Tags: pkgsrc-2020Q1-base,
pkgsrc-2020Q1
Changes since 1.10: +8 -7
lines
Diff to previous 1.10 (colored)
librdkafka: updated to 1.3.0 librdkafka v1.3.0 release This is a feature release adding support for KIP-392 Fetch from follower, allowing a consumer to fetch messages from the closest replica to increase throughput and reduce cost. Features KIP-392 - Fetch messages from closest replica / follower (by @mhowlett) Added experimental (subject to change or removal) mock broker to make application and librdkafka development testing easier. Fixes Fix consumer_lag in stats when consuming from broker versions <0.11.0.0 (regression in librdkafka v1.2.0). librdkafka v1.2.2 release v1.2.2 fixes the producer performance regression introduced in v1.2.1 which may affect high-throughput producer applications. Fixes Fix producer insert msgq regression in v1.2.1. Upgrade builtin lz4 to 1.9.2. Don't trigger error when broker hostname changes. Less strict message.max.bytes check for individual messages. Don't call timespec_get() on OSX (since it was removed in recent XCode) by @maparent . configure: add --runstatedir for compatibility with autoconf. LZ4 is available from ProduceRequest 0, not 3. Address 12 code issues identified by Coverity static code analysis. Enhancements Add warnings for inconsistent security configuration. Optimizations to hdr histogram (stats) rollover. Reorganized examples and added a cleaner consumer example, added minimal C++ producer example. Print compression type per message-set when debug=msg librdkafka v1.2.1 release Warning: v1.2.1 has a producer performance regression which may affect high-throughput producer applications. We recommend such users to stay on v1.2.0. v1.2.1 is a maintenance release: Properly handle new Kafka-framed SASL GSSAPI frame semantics on Windows. This bug was introduced in v1.2.0 and broke GSSAPI authentication on Windows. Fix msgq (re)insertion code to avoid O(N^2) insert sort operations on retry The msgq insert code now properly handles interleaved and overlapping message range inserts, which may occur during Producer retries for high-throughput applications. configure: added --disable-c11threads to avoid using libc-provided C11 threads. configure: added more autoconf compatibility options to ignore librdkafka v1.2.0 release WARNING: There is an issue with SASL GSSAPI authentication on Windows with this release. Upgrade directly to v1.2.1 which fixes the issue. v1.2.0 is a feature release making the consumer transaction aware. Transaction aware consumer (isolation.level=read_committed) implemented by @mhowlett. Sub-millisecond buffering (linger.ms) on the producer. Improved authentication errors (KIP-152) Consumer-side transaction support This release adds consumer-side support for transactions. In previous releases, the consumer always delivered all messages to the application, even those in aborted or not yet committed transactions. In this release, the consumer will by default skip messages in aborted transactions. This is controlled through the new isolation.level configuration property which defaults to read_committed (only read committed messages, filter out aborted and not-yet committed transactions), to consume all messages, including for aborted transactions, you may set this property to read_uncommitted to get the behaviour of previous releases. For consumers in read_committed mode, the end of a partition is now defined to be the offset of the last message of a successfully committed transaction (referred to as the 'Last Stable Offset'). For non-transactional messages there is no change from previous releases, they will always be read, but a consumer will not advance into a not yet committed transaction on the partition. Upgrade considerations linger.ms default was changed from 0 to 0.5 ms to promote some level of batching even with default settings. New configuration properties Consumer property isolation.level=read_committed ensures the consumer will only read messages from successfully committed producer transactions. Default is read_committed. To get the previous behaviour, set the property to read_uncommitted, which will read all messages produced to a topic, regardless if the message was part of an aborted or not yet committed transaction. Enhancements Offset commit metadata (arbitrary application-specified data) is now returned by rd_kafka_committed() and rd_kafka_offsets_for_times() C++: Added Conf::c_ptr*() to retrieve the underlying C config object. Added on_thread_start() and on_thread_exit() interceptors. Increase queue.buffering.max.kbytes max to INT_MAX. Optimize varint decoding, increasing consume performance by ~15%. Fixes General: Rate limit IO-based queue wakeups to linger.ms, this reduces CPU load and lock contention for high throughput producer applications. Reduce memory allocations done by rd_kafka_topic_partition_list_new(). Fix socket recv error handling on MSVC (by Jinsu Lee). Avoid 1s stalls in some scenarios when broker wakeup-fd is triggered. SSL: Use only hostname (not port) when valid broker hostname (by Hunter Jacksson) SSL: Ignore OpenSSL cert verification results if enable.ssl.certificate.verification=false rdkafka_example_cpp: fix metadata listing mode (@njzcx) SASL Kerberos/GSSAPI: don't treat kinit ECHILD errors as errors Fix compare overflows configure: Add option to disable automagic dependency on zstd (by Thomas Deutschmann) Documentation updates and fixes by Cedric Cellier and @ngrandem Set thread name on MacOS X (by Nikhil Benesch) C++: Fix memory leak in Headers (by Vladimir Sakharuk) Fix UBSan (undefined behaviour errors) CONFIGURATION.md: escape || inside markdown table Refresh broker list metadata even if no topics to refresh Consumer: Make rd_kafka_pause|resume_partitions() synchronous, making sure that a subsequent consumer_poll() will not return messages for the paused partitions. Fix incorrect toppar destroy in OffsetRequest Fix message version 1 offset calculation Defer commit in transport error to avoid consumer_close hang. Producer: Messages were not timed out for leader-less partitions. Improve message timeout granularity to millisecond precision (the smallest ffective message timeout will still be 1000ms). message.timeout.ms=0 is now accepted even if linger.ms > 0 (by Jeff Snyder) Don't track max.poll.interval.ms unless in Consumer mode, this saves quite a few memory barries for high-performance Producers. Optimization: avoid atomic fatal error code check when idempotence is disabled.
Revision 1.10 / (download) - annotate - [select for diffs], Sat Jan 18 21:48:07 2020 UTC (4 years, 3 months ago) by jperkin
Branch: MAIN
Changes since 1.9: +2 -1
lines
Diff to previous 1.9 (colored)
*: Recursive revision bump for openssl 1.1.1.
Revision 1.9 / (download) - annotate - [select for diffs], Wed Jul 10 01:31:21 2019 UTC (4 years, 9 months ago) by minskim
Branch: MAIN
CVS Tags: pkgsrc-2019Q4-base,
pkgsrc-2019Q4,
pkgsrc-2019Q3-base,
pkgsrc-2019Q3
Changes since 1.8: +2 -3
lines
Diff to previous 1.8 (colored)
devel/librdkafka: Update to 1.0.1 Notable changes in 1.0: - Idempotent producer - guaranteed ordering, exactly-once producing. - Sparse/on-demand connections - connections are no longer maintained to all brokers in the cluster. - KIP-62 - max.poll.interval.ms for high-level consumers.
Revision 1.8 / (download) - annotate - [select for diffs], Thu Apr 25 07:32:48 2019 UTC (4 years, 11 months ago) by maya
Branch: MAIN
CVS Tags: pkgsrc-2019Q2-base,
pkgsrc-2019Q2
Changes since 1.7: +2 -1
lines
Diff to previous 1.7 (colored)
PKGREVISION bump for anything using python without a PYPKGPREFIX. This is a semi-manual PKGREVISION bump.
Revision 1.7 / (download) - annotate - [select for diffs], Sat Dec 15 21:12:21 2018 UTC (5 years, 4 months ago) by wiz
Branch: MAIN
CVS Tags: pkgsrc-2019Q1-base,
pkgsrc-2019Q1,
pkgsrc-2018Q4-base,
pkgsrc-2018Q4
Changes since 1.6: +2 -2
lines
Diff to previous 1.6 (colored)
*: update email for fhajny
Revision 1.6 / (download) - annotate - [select for diffs], Thu Apr 5 09:01:16 2018 UTC (6 years ago) by fhajny
Branch: MAIN
CVS Tags: pkgsrc-2018Q3-base,
pkgsrc-2018Q3,
pkgsrc-2018Q2-base,
pkgsrc-2018Q2
Changes since 1.5: +7 -2
lines
Diff to previous 1.5 (colored)
devel/librdkafka: Update to 0.11.4. Default changes - socket.max.fails changed to 1 to provide same functionality (fail request immediately on error) now when retries are working properly again. - fetch.max.bytes (new config property) is automatically adjusted to be >= message.max.bytes, and receive.message.max.bytes is automatically adjusted to be > fetch.max.bytes. New features - Message Headers support - Java-compatible Murmur2 partitioners - Add PKCS#12 Keystore support - ssl.keystore.location Noteworthy fixes - Formalise and fix Producer retries and retry-ordering - Ordering is now retained despite retries if max.in.flight=1. - Behaviour is now documented - Fix timeouts for retried requests and properly handle retries for all request types - Add and use fetch.max.bytes to limit total Fetch response size. Fixes "Invalid response size" issues. Enhancements - Added sasl.mechanism and compression.type configuration property aliases for conformance with Java client. - Improved Producer performance - C++: add c_ptr() to Handle,Topic,Message classes to expose underlying librdkafka object - Honour per-message partition in produce_batch() if MSG_F_PARTITION set - Added on_request_sent() interceptor - Added experimental flexible producer queuing.strategy=fifo|lifo - Broker address DNS record round-robin: try to maintain round-robin position across resolve calls. - Set system thread name for internal librdkafka threads - Added more concise and user-friendly 'consumer' debug context - Add partitioner (string) topic configuration property to set the builtin partitioners - Generate rdkafka-static.pc (pkg-config) for static linking Fixes - Fix producer memory leak on <0.11 brokers when compressed messageset is below copy threshold - CRC32C - fix unaligned access on ARM - Fix read after free in buf_write_seek - Fix broker wake up - Fix consumer hang when rebalancing during commit - CMake fixes for Windows - LeaveGroup was not sent on close when doing final offset commits - Fix for consumer slowdown/stall on compacted topics where actual last offset < MsgSet.LastOffset - Fix global->topic conf fallthru in C++ API - Fix infinite loop on LeaveGroup failure - Fix possible crash on OffsetFetch retry - Incorporate compressed message count when deciding on fetch backoff - Fix debug-only crash on Solaris (%s NULL) - Drain broker ops queue on termination to avoid hang - cmake: Allow build static library - Don't store invalid offset as next one when pausing - use #if instead of #ifdef / defined() for atomics - fixed .lib paths in nuget packaging - Fixes strerror_r crash on alpine - Allow arbitrary lengthed (>255) SASL PLAIN user/pass - Trigger ApiVersionRequest on reconnect if broker.version.fallback supports it - Read Fetch MsgAttributes as int8 - Portability: stop using typeof in rdkafka_transport.c - Portability: replace use of #pragma once with header guards - mklove: add LIBS in reverse order to maintain dependency order - Fix build when python is not available #1358
Revision 1.5 / (download) - annotate - [select for diffs], Sat Dec 9 16:57:04 2017 UTC (6 years, 4 months ago) by fhajny
Branch: MAIN
CVS Tags: pkgsrc-2018Q1-base,
pkgsrc-2018Q1,
pkgsrc-2017Q4-base,
pkgsrc-2017Q4
Changes since 1.4: +2 -2
lines
Diff to previous 1.4 (colored)
Update devel/librdkafka to 0.11.3. Default changes - Change default queue.buffering.max.kbytes and queued.max.message.kbytes to 1GB - win32: Use sasl.kerberos.service.name for broker principal, not sasl.kerberos.principal Enhancements - Default producer message offsets to OFFSET_INVALID rather than 0 - new nuget package layout + debian9 librdkafka build - Allow for calling rd_kafka_queue_io_event_enable() from the C++ world - rdkafka_performance: allow testing latency with different size messages Fixes - Improved stability on termination - offsets_for_times() return ERR__TIMED_OUT if brokers did not respond in time - Let list_groups() return ERR__PARTIAL with a partial group list - Properly handle infinite - Fix offsets_store() return value when at least one valid partition - portability: rdendian: add le64toh() alias for older glibc - Add MIPS build and fix CRC32 to work on big endian CPUs - osx: fix endian checking for software crc32c - Fix comparison in rd_list_remove_cmp - stop calling cnd_timedwait() with a timeout of 0h - Fix DNS cache logic broker.address.ttl - Fix broker thread "hang" in CONNECT state - Reset rkb_blocking_max_ms on broker DOWN to avoid busy-loop during CONNECT - Fix memory leak when producev() fails - Raise cmake minimum version to 3.2 - Do not assume LZ4 worst - Fix ALL_BROKERS_DOWN re-generation - rdkafka-performance: busy wait to wait short periods of time
Revision 1.4 / (download) - annotate - [select for diffs], Wed Oct 18 10:00:04 2017 UTC (6 years, 6 months ago) by fhajny
Branch: MAIN
Changes since 1.3: +2 -2
lines
Diff to previous 1.3 (colored)
Update devel/librdkafka to 0.11.1. Noteworthy critical bug fixes - Fix OpenSSL instability on Windows (fix thread id callback) - the bug resulted in SSL connections being torn down for no apparent reason. - Fetch response fix: read all MessageSets, not just the first one. Enhancements - Add api.version.request.timeout.ms - Point users to documentation when attempting to use Java security properties Fixes: - Adjust log level for partial message reads when debug is enabled - Allow app metadata() requests regardless of outstanding metadata requests - Proper size calculation from flags2str to include nul - Thread-safe rd_strerror() - Proper re-sends of ProduceRequests in-transit on connection down - Producer: invalid offsets reported back when produce.offset.report=false - Consumer: Message Null Key/Value were messed up - Fix metadata querying for topics with LEADER_UNAVAIL set - Treat request __TIMED_OUT as a retryable error - Let timed out ProduceRequests result in MSG_TIMED_OUT error code for messages - Fix crash on leader rejoin when outstanding assignor metadata - sasl_cyrus: Fix dangling stack ptr to sasl_callback_t
Revision 1.3 / (download) - annotate - [select for diffs], Wed Aug 2 15:49:24 2017 UTC (6 years, 8 months ago) by fhajny
Branch: MAIN
CVS Tags: pkgsrc-2017Q3-base,
pkgsrc-2017Q3
Changes since 1.2: +2 -2
lines
Diff to previous 1.2 (colored)
Update devel/librdkafka to 0.11.0. Features - Added support for MsgVersion v2 (message format of KIP-98) - message format compatible with EOS clients - Added support for client interceptors - Added support for dynamically loaded plugins (plugin.library.paths, for use with interceptors) - Added SASL SCRAM support (KIP-84) - Added builtin SASL PLAIN provider (for Win32, #982) Enhancements - Deprecate errno usage, use rd_kafka_last_error() instead. - Deprecate rd_kafka_wait_destroyed(). - Implemented per-partition Fetch backoffs, previously all partitions for the given broker were backed off. - Added updated Kafka protocol and error enums - Added rd_kafka_message_latency() - Added rd_kafka_clusterid() and rd_kafka_type() - SSL: set default CA verify locations if ssl.ca.location is not specified - C++: add yield() method - Added support for stats as events (#1171) - Build with system liblz4 if available, else fall back on built-in lz4, for improved portability. - Use SNI when connecting through SSL (@vincentbernat) - Improve broker thread responsiveness, decreasing internal latency - Improve OpenSSL config error propagation (#1119) - Prioritize all relevant user-facing ops (callbacks) over messages on poll queue (#1088) - Added global->topic config fallthru: default topic config properties - can now be set effortlessly on global config object. - Log offset commit failures when there is no offset_commit_cb (closes #1043) - Add CRC checking support to consumer (#1056) - C++: Added seek() support to KafkaConsumer - Added rd_kafka_conf_dup_filter() to selectively copy a config object. Fixes: - Avoid _ALIGN re-definition on BSD (#1225) - rdkafka_performance: exit with code 1 if not all messages were delivered - Fix endianism issues that were causing snappy to compress incorrectly (#1219, @rthalley) - Fix stability on AIX (#1211) - Document that rd_kafka_message_errstr() must not be used on producer - Add support for re-queuing half-processed ops to honour yield() - Handle null Protocol in JoinGroupResponse (#1193) - Consumer: Proper relative offset handling (#1192, @rthalley) - OSX: silence libsasl deprecated warnings - partition count should be per topic in offset request buffer (closes #1199, @matthew-d-jones) - fix build on SmartOS (#1186 by @misterdjules) - ERR_remove_thread_state OpenSSL version checking - Don't emit TIMED_OUT_QUEUE for timed out messages (revert) - producev() default partition should UA, not 0 (#1153) - Fix SaslHandshakeRequest timeout to 10s - SASL: fix memory leak: received SASL auth frames were not freed - Clean up OpenSSL per-thread memory on broker thread exit - Properly auto-set metadata.max.age.ms when metadata.refresh.interval.ms is disabled (closes #1149) - Fix memory alignment issues (#1150) - configure: auto add brew openssl pkg-config path - Fix consumer_lag calculation (don't use cached hi_offset) - rdkafka_example: fix message_errstr usage, not allowed on producer - Avoid use of partially destroyed topic object (#1125) - Improve reconnect delay handling (#1089) - C++: fix conf->get() allocation (closes #1118) - Use app_offset to calculate consumer_lag (closes #1112) - Fix retrybuf memory leak on termination when broker is down - Fix small memory leak in metadata_leader_query - Fix use-after-free when log.queue and debug was used - consumer_example: fix crash on -X dump (closes #841) - Added rd_kafka_offsets_store() (KafkaConsumer::offsets_store) (closes #826) - Optimize broker id lookups (closes #523) - Don't log broker failures when an error_cb is registered (closes #1055) - Properly log SSL connection close (closes #1081) - Win32 SASL GSSAPI: protection level and message size were not sent - C++: improved error reporting from Conf::set() - Flush partition fetch buffer on seek (from decide()) - Fix recursive locking on periodic refresh of leader-less partition (closes #1311)
Revision 1.2 / (download) - annotate - [select for diffs], Mon Apr 24 12:20:37 2017 UTC (6 years, 11 months ago) by fhajny
Branch: MAIN
CVS Tags: pkgsrc-2017Q2-base,
pkgsrc-2017Q2
Changes since 1.1: +8 -1
lines
Diff to previous 1.1 (colored)
* Use bash for CONFIG_SHELL * Fix build on NetBSD * Uses python to build * Enable tests
Revision 1.1 / (download) - annotate - [select for diffs], Thu Apr 20 08:01:18 2017 UTC (7 years ago) by fhajny
Branch: MAIN
Import librdkafka 0.9.5 as devel/librdkafka. librdkafka is a C library implementation of the Apache Kafka protocol, containing both Producer and Consumer support.