Up to [cvs.NetBSD.org] / pkgsrc / net / libtrace
Request diff between arbitrary revisions
Keyword substitution: kv
Default branch: MAIN
net: Replace RMD160 checksums with BLAKE2s checksums All checksums have been double-checked against existing RMD160 and SHA512 hashes Not committed (merge conflicts...): net/radsecproxy/distinfo The following distfiles could not be fetched (fetched conditionally?): ./net/citrix_ica/distinfo citrix_ica-10.6.115659/en.linuxx86.tar.gz ./net/djbdns/distinfo dnscache-1.05-multiple-ip.patch ./net/djbdns/distinfo djbdns-1.05-test28.diff.xz ./net/djbdns/distinfo djbdns-1.05-ignoreip2.patch ./net/djbdns/distinfo djbdns-1.05-multiip.diff ./net/djbdns/distinfo djbdns-cachestats.patch
net: Remove SHA1 hashes for distfiles
libtrace: updated to 4.0.10 libtrace 4.0.10: New features Added new API function (trace_get_errstr()) which will map a given libtrace error number to a printable error message. Bug fixes Fixed SIOCGSTAMP undeclared error when building against newer Linux kernels. Fixed corruption bug when running multiple concurrent etsilive: input processes. Improvements Bumped TTL of nDAG multicast group joining messages to 4, so they can be routed outside of the immediate subnet (i.e. through the host when libtrace is run within a container). libtrace 4.0.9: Bug fixes Fixed traceanon build error on systems that did not have libcrypto installed. Fixed DPDK detection in configure when the DPDK package was installed on either Debian buster and Ubuntu disco. Updated DPDK code to compile against more recent DPDK releases, such as 18.11. Fixed segmentation fault when failing to open a DAG device. Fixed issue where a pcapng packet that does not match any of our known data types ends up having an uninitialised data type. Fix some compilation errors when using DPDK on FreeBSD (may still be linking problems if you have built DPDK using the ports tree, though). Fix infinite decoding loop if libpacketdump sees an SCTP option with a length of zero. libtrace 4.0.8: New features traceanon is now capable of anonymising RADIUS traffic within packet traces. The anonymisation will obfuscate the data within AVPs that can be considered 'sensitive', including user names, IP addresses and password hashes. Counter fields such as byte and packet counters are by default untouched, but traceanon can be configured to anonymise those as well if required. traceanon can now be configured using a YAML configuration file, instead of CLI arguments. This change is due to the increased number of configuration options introduced by the RADIUS anonymisation feature. Instructions on how to write a configuration file can be found on the traceanon manpage, as well as on this wiki page. Bug fixes Fixed bug where ndag multicast sockets would bind to all addresses on an interface, rather than just the address of the multicast group. Fixed segfault that can occur when pausing a trace input that has not been able to create its per packet processing threads for some reason. libtrace 4.0.7: New features Added new API functions for exploring meta-data that is either attached to a specific packet or included in a trace as separate records (e.g. ERF provenance or pcap-ng meta-data). Many meta-data fields have a specific accessor function that can be called directly (e.g. trace_get_interface_fcslen()). You can also use trace_get_section() to get an array containing all meta-data items within a particular section, which will allow you to get access to any fields for which we have not implemented direct access functions. Added new API functions to instant decoding all of the post-layer 2, pre-layer 3 headers in a packet so you can now easily explore any / all VLAN, MPLS, etc. headers in a packet without having to effectively re-implement trace_get_layer3() in your own code. See trace_get_layer2_headers() for more details. Added support for both reading and writing TZSP sniffing streams. Bug fixes Fixed uninitialised bytes in message structure sent via trace_post_reporter -- thanks to Mark Weiman for fixing this. Fixed build errors caused by attempting to #include pcap-int.h. Fixed bug where a corrupt ERF record could cause a libtrace program to become un-haltable. Fixed bug in error tracking when creating a fanout socket for the ring and int formats. Fixed potential segfault when halting a libtrace program that was reading from a ring: input. Fixed uninitialised mutex when copying a packet. Improvements Improved parallel performance by skipping some needless per-packet sanity checks. libtrace 4.0.6: New features Added write support for pcapng: format. Bug fixes Fixed incorrect counting of input sources when using etsilive: for reading packets. Fixed bug where trace_event() API was ignoring all received packets. Fixed bug where tracereplay would segfault. Fixed packet corruption bug in tracesplit when using the "jump to IP header" mode. Fixed bug where we could end up trying to close a NULL pcap output. Fixed build problems when building with dpdk enabled. Fixed bug that was causing recvmmsg detection to fail at configure time. Fixed bug where ETSI live sockets created later on are uninitialised. Fixed memory leak when using BPF filters with ring: inputs. Fixed a variety of potential crashes and buffer overflows revealed by Perry's fuzzing experiments Improvements Replaced numerous internal assertion checks with error return values instead, i.e. instead of a libtrace function assert failing and crashing your program, it will now return an error (or set the error status on the trace) and allow the user to deal with the error however they want. Similarly, tidied up some of the error messaging to be clearer about what has gone wrong and added a variety of new error types. Improved ring: read performance when used with the parallel API by reading multiple packets per function call. Added option to report numbers of dropped and missing packets (cumulative) in tracertstats. Ported traceends and tracetopends to use the parallel API. Improvements to ndag packet reading performance. libtrace 4.0.5: Bug fixes Fixed bug where clients would obtain an exclusive lock on an nDAG multicast group. Fixed bogus payload length calculations on outgoing packets when the IP length field is filled in by the NIC prior to sending. Fixed bug where any non-negative return value other than zero from a pstart callback would be treated as an error. Fixed bug where packets that have been invalidated by a call to trace_ppause() are still treated as valid. Fixed bug where parallel ring: inputs would assert fail when the input is halted. Reduced likelihood of dropping packets on an ndag: input during initialisation phase. Fixed build error for DPDK format due to missing header file. Fix race condition that can occur when two threads attempt to call trace_create() or trace_create_dead() at the same time. Improvements Improved etsilive: decoding performance. Avoid invalidating packets received via ring: following a pause until the trace is restarted. Added caching for packet framing length. libtrace 4.0.4: NOTE: libwandio 4.0.0 is required to build this version of libtrace. Older versions of libwandio will not work. New Features Added trace_increment_packet_refcount() and trace_decrement_packet_refcount() functions to the parallel API. These functions can be used to track references to a libtrace packet across multiple threads, so that a shared packet can be released once all threads have finished with it. Packets where the reference count is decremented to zero are automatically released. Add new built-in data structure: simple circular buffer. Added new format for receiving and decoding packets encapsulated in the ETSI Lawful Intercept streaming format (requires libwandder). Added support for decoding ETSI Lawful Intercept records to libpacketdump (requires libwandder). Add trace_flush_output() API function to force a libtrace output to dump any buffered output to disk. Flushed files may still not be properly readable afterwards, but this will help give the appearance that the output file is growing in situations where the output rate is slow. Bug Fixes Fixed bug in ndag: which was causing the stream to be treated as inactive when there are buffered records available. Fixed build errors caused by pthread_attr_setaffinity_np() being a glibc-only extension -- thanks to Tim Dawson for contributing this patch. Fixed bug where uninitialised internal message queues were being destroyed -- thanks to EaseTheWorld for reporting this. Fixed lack of error being returned when a user tries to change the number of perpkt threads on a paused trace. Fixed problems in tracereplay caused by trying to replay packets with no contents (e.g. meta-data records). Fix bug where packets received via a ring: interface were being released twice. Fix rounding error in trace_event_trace() which would cause sleep intervals to be rounded down to zero. Fix rounding error in pcapng_get_timespec() which would cause all packet timestamps to be truncated to the previous second. Fix deadlock when calling trace_pstop() on a trace that has already been stopped. Fix bug where two concurrent ring: inputs would be assigned to the same fanout group, causing the second input to fail to start. Fixed errors in manpages for tracesplit, traceanon and tracemerge (regarding the correct names for the various compression methods) -- thanks to Hendrik Leppelsack for reporting this problem. Fixed some uninitialised memory errors when valgrinding a parallel libtrace program. Fixed potential buffer overruns in pcapng reading code. Fixed bug that was preventing trace_pstop() from working as intended on pcapint:. Fixed potential build errors relating to the absence (or not) of strndup(), strncasecmp() and snprintf(). Improvements Updated DPDK support to be able to compile against DPDK 18.02.1 tracereplay is now able to reduce inter-packet gaps in the replayed stream by a user-specified speedup factor, so the trace can be replayed faster but with the same relative gaps between packets. libtrace 4.0.3: New Features Added new API function: trace_get_perpkt_thread_id(), which allows callers to get the ID number of the packet processing thread that they are currently in. Message Queue data structure API is now publicly exported. Toeplitz Hash API is now publicly exported. Added dpdkndag: capture format, which allows a libtrace program to capture and parse nDAG records that are intercepted on a DPDK-capable interface. Moved trace_prepare_packet() into the external API. Bug Fixes Fixed bug where captures from GRE tunnel interfaces would fail due to unknown ARPHRD type. Fixed problems when reading ERF provenance records from a DAG or ERF source -- thanks to Anthony Coddington at Endace for resolving this issue. Fixed bug where nDAG packets could be corrupted if all of the receive buffers are full. Fixed assertion failure when libwandio fails but does not set errno to a useful value -- thanks to Robert Zeh for patching this bug. Fixed minor memory leak when a user does not provide a hash function when calling trace_set_hasher(). Fixed missing pthread_spinlock.h error that occurred whenever a user tried to include message_queue.h or ring_buffer.h. Fixed bug where some key data structures were not initialised when doing DPDK output. Fixed bug where DPDK memory buffers were too small to hold a full packet, causing payload to be truncated. Fixed uninitialised write index in format_ndag, which could cause some nDAG captures to appear corrupt. Improvements Updated dag: format to use the 64 bit API -- this means that we can support capture on DAG streams that have large amounts of memory attached. Improved nDAG performance by avoiding unnecessary calls to recvmmsg when there is no data available on the socket. Improved nDAG performance by caching the byteswapped versions of some frequently accessed fields. tracertstats will now handle SIGINT and SIGTERM signals cleanly. libtrace 4.0.2: New Features Added ability to read pcapng trace files (and convert them into other formats). Added input format for receiving and processing packets emitted by an nDAG multicaster. Bug Fixes Fixed bug that would cause the IPv6 fragment offset to be calculated incorrectly. Fixed return value bug with pcap_write_packet(). Fixed bad assertion failure when halting parallel programs with SIGINT. Fixed compilation issues caused by mismatched BPF presence macros when pcap-bpf.h is missing. Fixed libpacketdump bug where it was reading past the end of captured IPv6 headers. Fixed several issues in the libpacketdump parser for SCTP. Fixed assertion failure in traceanon if the cryptopan key is too short. Fixed compilation error with traceanon if libssl version >= 1.1.0. Fix bug where the wrong parallel read function would be used in some specific configurations. DPDK shared libraries are now correctly detected by the configure script. libtrace 4.0.1: New Features DPDK support has been extended to cover the most recent stable release. Added ability to parse SIT (IPv6 within IPv4) packets inside SLL. Added trace_clear_statistics() API function. Added support for IPv6 in PPP. Added native support for bidirectional and balanced hashing to DPDK inputs. Bug Fixes Fixed bug where ring: and int: parallel inputs would not respond to trace_pstop() on older kernels. Fixed bug where trace_interrupt() would not trigger on busy inputs (including files). Fixed bug where DPDK inputs would cause the event API to hang. Fixed bug where ring: and int: parallel formats could end up repeatedly polling. Fixed performance issue with tracertstats when used on live formats. Fixed bug where libtrace's default hasher was always sending packets to the same thread. Fixed race conditions when using parallel API to read from a file format. Fixed bug where the ordered combiner would appear to send packets to the reporter thread out of order, due to the packet ordering being based on a non-monotonic clock. Fixed bug where trace_get_payload_from_gre() would not correctly parse PPTP GRE. Improvements Received packet counters are now valid for pcap inputs. Improved performance by removing mutex from packet reading code. Don't install extra header files directly into /usr/local/include; these are now installed into a libtrace-specific directory. This should resolve some namespace collision issues with some of our poorly-named header files. libtrace 4.0.0: New Features New licensing -- Libtrace now uses the LGPL v3 rather than GPL v2, so it is now possible for people to link against libtrace without having to make their own code available under the GPL. All new parallel API, written by Richard Sanger, that makes it easy to split packet processing tasks over multiple threads. If a capture format has support for native parallelism, e.g. DPDK, DAG streams, parallel libtrace will take advantage of these. The parallel API is contained and documented in "libtrace_parallel.h" -- include this header file to access the parallel API. The previous single-threaded API is still supported, so all of your old libtrace programs should compile and run against libtrace 4 without modification. Libwandio is no longer built in to libtrace and is now its own separate library. You can download libwandio from http://research.wand.net.nz/software/libwandio.php . Thanks to Alistair King for helping remove libwandio from libtrace. New API function: trace_strip_packet(), which attempts to remove any VLAN, MPLS or other layer 2.5 headers from a captured packet. Converted traceanon, tracertstats and tracestats tools to use the new parallel API. Bug Fixes Fixed bug where libpacketdump would print ICMP checksums in the wrong place. Fixed inability to correctly parse ERF records that contained extension headers. Fixed problem where traceanon wouldn't handle keyboard interrupts nicely. Fixed memory leak if we fail to guess the format for an input trace (Thanks to Vincenzo Caruso for reporting this bug). Fixed double free when destroying a DAG input. Bugs squashed since the beta release: Fixed bug that prevented multiple ring: or int: parallel inputs from being used on a single host concurrently. Fixed memory leak when using a heavily filtered RT input. Fixed bug where the ordered combiner would emit packets out of order. Fixed bug where thread message queues were not being destroyed when the parent trace was destroyed. Fixed race condition when modifying BPF headers on FreeBSD 9 systems. Use default DPDK device driver thresholds instead of our previously hard-coded values. Fixed potential infinite loop when parsing extended RadioTap headers. Fixed bad decoding of RadioTap headers with extended presence. Fixed bug where pausing a pcap: trace file would cause any resumption to return to the start of the file rather than resuming from where it left off. Fixed segfault when destroying a packet associated with a trace has reached EOF. Fixed memory management in trace_construct_packet (Thanks to Perry Lorier for submitting code to do this). Fixed bug where pcap file descriptors were being leaked (Thanks to Tomas Konir for reporting this bug). Fixed bug where trace_create_packet() would segfault if the system runs out of memory. Improvements Added BPF filtering option to traceanon. Use libcrypto for traceanon IP address encryption rather than our own rijndael implementation. This adds a dependency on libcrypto, but should result in faster encryption operations. Added a --jump option to tracesplit which can be used to strip any headers preceding the Nth layer 3 header; useful for decapsulating tunnelled IP traffic (Thanks to Perry Lorier for adding this feature).
Add SHA512 digests for distfiles for net category Problems found with existing digests: Package haproxy distfile haproxy-1.5.14.tar.gz 159f5beb8fdc6b8059ae51b53dc935d91c0fb51f [recorded] da39a3ee5e6b4b0d3255bfef95601890afd80709 [calculated] Problems found locating distfiles: Package bsddip: missing distfile bsddip-1.02.tar.Z Package citrix_ica: missing distfile citrix_ica-10.6.115659/en.linuxx86.tar.gz Package djbdns: missing distfile djbdns-1.05-test25.diff.bz2 Package djbdns: missing distfile djbdns-cachestats.patch Package djbdns: missing distfile 0002-dnscache-cache-soa-records.patch Package gated: missing distfile gated-3-5-11.tar.gz Package owncloudclient: missing distfile owncloudclient-2.0.2.tar.xz Package poink: missing distfile poink-1.6.tar.gz Package ra-rtsp-proxy: missing distfile rtspd-src-1.0.0.0.tar.gz Package ucspi-ssl: missing distfile ucspi-ssl-0.70-ucspitls-0.1.patch Package waste: missing distfile waste-source.tar.gz Otherwise, existing SHA1 digests verified and found to be the same on the machine holding the existing distfiles (morden). All existing SHA1 digests retained for now as an audit trail.
Update libtrace to 3.0.10 libtrace 3.0.10 (2011-03-11) Bug Fixes Improvements * Significantly improved performance of libtrace event API * Transport headers and payload length are now cached for each packet, saving time on subsequent lookups libtrace 3.0.9 (2011-01-25) Bug Fixes Improvements * tracesplit can now accept multiple input URIs which are read in turn libtrace 3.0.8 (2010-12-03) Bug Fixes New Features * Added a new API function called trace_get_payload_length() that returns the length of the original payload content (i.e. the size of the post-transport header payload prior to any snapping) Improvements * Added IPv6 and IPv6 fragmentation header decoders to libpacketdump * traceanon can now read cryptopan keys from a file
New Features: * Replaced IO subsystem with wandio abstraction * IO / compression / decompression is now performed in a separate thread, resulting in improved performance * Modular design makes it easy to add support for new compression formats * Added native support for reading and writing bzip files * Added native support for writing lzo files * JITing of BPF bytecode using LLVM, leading to faster BPF filtering * Added enums for post-IP protocols and Ethertypes * Write support added for DAG cards - thanks to Daniel Lawson * Added new trace tool: tracetop. Shows the top N flows each second * Added new trace tool: tracereplay. Attempts to replay trace files in trace time * Added new trace tool: tracediff. Displays packets that differ between two trace files * Added trace_get_timespec() function * If the format is not specified as part of the URI, libtrace can now attempt to guess the trace format * Libpacketdump can now decode CHDLC and PPP/HDLC headers * Added all the code examples from the libtrace tutorial to the examples directory Bug Fixes: * Fixed bug where packets read from a DAG card that did not match the filter were causing lengthy sleep events under the event API * Fixed various tools that were not reporting the occurrence of a read error * Fixed segfault caused by malformed URIs * Fixed bug where reading a zero-length payload from a PCAP trace would result in an EOF being incorrectly reported * Fixed bug where filtered packet count was not initialised to zero * trace_get_payload_from_ip() now returns NULL when the IP version is incorrect rather than asserting * Fixed segfault when writing packets to a Linux native socket, caused by byte ordering issue * Fixed bug where custom pcap event function was not being used * Fixed misplaced assertion in the pcap file reading code * Fixed bug where trace_event would never get a packet event under recent versions of libpcap * Fixed assertion failure when an unknown linktype is encountered by libpacketdump * Fixed error caused by LCP packets that are common in some trace sets, e.g. Leipzig * Increased size of RT packet buffer to fix problems caused by jumbograms * Fixed errors caused by 32- and 64-bit incompatibility when sending Linux Native packets using the RT protocol * trace_get_*_port() functions now always return 0 for ICMP packets * Fixed problems with decoding HDLC and CHDLC headers * Fixed segfault when reading PCAP packets that had no packet content * Fixed bug where PCAP packets would be written with a larger capture length than the wire length * Fixed segfault in the TCP segment report in tracereport caused by segments larger than 1500 bytes * Fixed bug with restarting a PCAP trace file * Fixed bugs relating to the size of the TSH packet records * Fixed bug where we were not accounting for the FCS in legacy Ethernet captures * Fixed bug where libpacketdump could not decode Linux SLL properly due to using an "undefined" function * Fixed bug where libpacketdump was not skipping IP options before attempting to decode the next header * Fixed bug where padding was being treated as part of a truncated header * Fixed assertion when converting a packet with a corrupt wire length to PCAP * More fixes for missing #includes Improvements: * trace_get_source_address() and trace_get_destination_address() now return link layer addresses in the absence of an IP header wherever possible * trace_get_<protocol> short-cut functions now return NULL if the entire header (minus options) is not present in the packet * Added missing set_capture_length() functionality for Linux Native * traceanon can now write compressed traces * traceanon now replaces checksums with zeroes * traceanon, tracesplit and tracemerge now support all libtrace compression types for output * tracereport no longer does the flow report by default * Added support for new ERF types * Added linktype for Experimental Ethernet * Added --count option to tracereport * Added --merge-inputs option to tracertstats * Added support for ARPHRD_NONE * Added a libpacketdump decoder for ubiquity headers * Improved libpacketdump's method of searching for decoders * More efficient arrangement of internal structures * Tidied up exported symbols * General code maintenance * Tidied up manpages * Improved documentation
Update to the latest version in order to fix build breakage. libtrace 3.0.6 (2008-11-27) * Fixed compilation errors caused by missing #includes (r1382) * Added trace_get_payload_from_pppoe() to external API (r1383) * autoconf now correctly detects libgdc properly for tracertstats (r1384) * Fixed some warnings on recent versions of gcc (r1385)
Update libtrace to 3.0.4 libtrace 3.0.4 (2008-01-02) Deprecate wtf:/wag: format. These traces no longer exist. Cleanup bpf: capture format add LINUX_SLL header support to get_source_mac() deprecate trace_get_link() and replace it with the newer trace_get_packet_buffer()/trace_get_layer2() Bug: Don't crash when destroying an output trace that failed to initialise Use Linux's in kernel BPF filter if available Add support for Cisco HDLC over PoS Allow BPF bytecode to be used to construct a filter Code cleanups Fix libtrace_ip's bitfields Fix pcapfile output bug Documentation cleanups Discard RT packets when writing pcapfile: files Add a new "stats" example program Build system cleanups Avoid using assert() to report errors RT packet issues Properly deal with the packet parsing/length cache when using the event system Add a new loss counter framework Bug: Event framework not generating sleep events when reading traces from disk Be more strict about returning NULL from trace_get_payload_from_X() functions libtrace 3.0.3 (2007-09-05) Code cleanups w.r.t warnings tracesplit_dir now provides a warning of the number of packets that had an unknown direction at the end of the trace Fix a segfault in tracereport with rxerrors, non ip Add support for decoding 802.2 LLC/SNAP and Ethernet II in 802.11 frames Documentation fixes and clarifications Fix bug with trace_get_payload_from_80211() and 3 vs 4 frame formats Deal correctly with uri's with parse errors causing segfaults on cleanup Minor tidyups to protocol decoders Add more information to libtracepktdump Correctly deal with PPP captures Cache trace_get_capture_length() and trace_get_l3() which are both heavily used internally Build system cleanups Add a GRE tracepktdump decoder Add a preliminary PPPoE tracepktdump decoder Add more information to tracereport Fix bug in legacy decoder with wire lengths Fix bug in trace_ether_ntoa Add legacynzix: trace format Don't assert() on bad packets (instead return BADPACKET) for erf traces Add TRACE_OPTION_EVENT_REALTIME to allow the event framework to playback traces in realtime Rename TRACE_META_FREQ to TRACE_OPTION_META_FREQ to follow naming convention Correctly deal with errors when using trace_set_option Deal better with signals when writing packets to files Add support for dag 3.x Improved dag 2.5+ support dag2.5+ supports setting the snaplen from libtrace Add support for setting direction on linux int: formats Consider loopback packets outgoing, not incoming Fix trace_get_source_mac() for wireless frames Add support for interfaces_per_input to tracemerge Fix tracereport direction report Deprecated wag: and wtf formats libtrace 3.0.2 (2007-04-27) Fixed make install for libpacketdump Add support for tsh: and rf+ tracefiles. Update support for radiotap Add a new tool traceflow(1) More correctly deal with pcap LINKTYPE's vs DLT's Major cleanups of tracereport libpacketdump Decoder cleanups trace_event() memory leak fixes Fix segfaults with bad arguments in tracesplit(1) Don't suffix a number if we are only generating one file Support snapping packets Minor bugs in libtrace error handling Misc cleanups and bug fixes libtrace 3.0.1 (2007-03-26) Added missing manpages to release tarball Update manpages Fixed TCP option length calculation in libpacketdump tracereport has had a massive tidy up getopt support for disabling/enabling reports. Documentation cleanups Fix 0 byte gzwrite(3)'s were causing the compressed file checksum to fail RT closing issues Metadata available tracedump renamed tracepktdump due to naming conflict in debian Implement better PoS decoding in libtrace for erf, legacypos. Build fixes for MacOS
Initial import of libtrace 3.0.0 libtrace is a library for trace processing. It supports multiple input methods, including device capture, raw and gz-compressed trace, and sockets; and multiple input formats, including pcap and DAG. Features * Understands PCAP, ERF, DAG, legacy POS, ATM and Ethernet and preliminary WAG formats * Read from tracefile, gz-compressed tracefile * Native DAG read support * BPF filter support on all input formats * Format conversion into ERF and PCAP formats * Write to tracefile for all formats * Write to interface via PCAP or Natively under Linux * libpacketdump, a packet dumping library useful for diagnosis * Various tools for trace manipulation
Initial revision