CVS log for src/external/bsd/file/dist/src/apprentice.c

Up to [cvs.NetBSD.org] / src / external / bsd / file / dist / src

Request diff between arbitrary revisions

Keyword substitution: kv
Default branch: MAIN

break strength ties so that qsort is deterministic

merge conflicts between file-5.43 and file-5.45

Update to file-5.45 (Last was file-5.44)

2023-07-27  15:45  Christos Zoulas <christos@zoulas.com>

	* release 5.45

2023-07-17  11:53  Christos Zoulas <christos@zoulas.com>

	* PR/465: psrok1: Avoid muslc asctime_r crash

2023-05-21  13:05  Christos Zoulas <christos@zoulas.com>

	* add SIMH tape format support

2023-02-09  12:50  Christos Zoulas <christos@zoulas.com>

	* bump the max size of the elf section notes to be read to 128K
	  and make it configurable

2023-01-08   1:08  Christos Zoulas <christos@zoulas.com>

	* PR/415: Fix decompression with program returning empty

2022-12-26   1:47  Christos Zoulas <christos@zoulas.com>

	* PR/408: fix -p with seccomp
	* PR/412: fix MinGW compilation

merge changes between 5.40 and 5.43

Import file-5.43+; last was file-5.40

2022-09-20  17:12  Christos Zoulas <christos@zoulas.com>

	* fixed various clustefuzz issues

2022-09-19  15:54  Christos Zoulas <christos@zoulas.com>

	* Fix error detection for decompression code (Vincent Mihalkovic)

2022-09-15  13:50  Christos Zoulas <christos@zoulas.com>

	* Add MAGIC_NO_COMPRESS_FORK and use it to produce a more
	  meaningful error message if we are sandboxing.

2022-09-15  10:45  Christos Zoulas <christos@zoulas.com>

	* Add built-in lzip decompression support (Michal Gorny)

2022-09-14  10:35  Christos Zoulas <christos@zoulas.com>

	* Add built-in zstd decompression support (Martin Rodriguez Reboredo)

2022-09-13  14:55  Christos Zoulas <christos@zoulas.com>

	* release 5.43

2022-09-10   9:17  Christos Zoulas <christos@zoulas.com>

	* Add octal indirect magic (Michal Gorny)

2022-08-17  11:43  Christos Zoulas <christos@zoulas.com>

	* PR/374: avoid infinite loop in non-wide code (piru)
	* PR/373: Obey MAGIC_CONTINUE with multiple magic files (vismarli)

2022-07-26  11:10  Christos Zoulas <christos@zoulas.com>

	* Fix bug with large flist (Florian Weimer)

2022-07-07  13:21  Christos Zoulas <christos@zoulas.com>

        * PR/364: Detect non-nul-terminated core filenames from QEMU
	  (mam-ableton)

2022-07-04  15:45  Christos Zoulas <christos@zoulas.com>

	* PR/359: Add support for http://ndjson.org/ (darose)
	* PR/362: Fix wide printing (ro-ee)
	* PR/358: Fix width for -f - (jpalus)
	* PR/356: Fix JSON constant parsing (davewhite)

2022-06-10   9:40  Christos Zoulas <christos@zoulas.com>

	* release 5.42

2022-05-31  14:50   Christos Zoulas <christos@zoulas.com>

	* PR/348: add missing cases to prevent file from aborting on
	  random magic files.

2022-05-27  21:05   Christos Zoulas <christos@zoulas.com>

	* PR/351: octalify filenames when not raw before printing.

2022-04-18  17:51   Christos Zoulas <christos@zoulas.com>

	* fix regex cacheing bug (Dirk Mueller)
	* merge file_regcomp and file_regerror() to simplify the code
	  and reduce memory requirements for storing regexes (Dirk Mueller)

2022-03-19  12:56   Christos Zoulas <christos@zoulas.com>

	* cache regex (Dirk Mueller)
	* detect filesystem full by flushing output (Dirk Mueller)

2021-11-19  12:36   Christos Zoulas <christos@zoulas.com>

	* implement running decompressor programs using
	  posix_spawnp(2) instead of vfork(2)

2021-10-24  11:51   Christos Zoulas <christos@zoulas.com>

	* Add support for msdos dates and times

2021-10-20   9:55   Christos Zoulas <christos@zoulas.com>

	* use the system byte swapping functions if available (Werner Fink)

2021-10-18  11:57  Christos Zoulas <christos@zoulas.com>

	* release 5.41

2021-09-23  03:51  Christos Zoulas <christos@zoulas.com>

	* Avinash Sonawane: Fix tzname detection

2021-09-03  09:17  Christos Zoulas <christos@zoulas.com>

	* Fix relationship tests with "search" magic, don't short circuit
	  logic

2021-07-13  01:06  Christos Zoulas <christos@zoulas.com>

	* Fix memory leak in compile mode

2021-07-01  03:51  Christos Zoulas <christos@zoulas.com>

	* PR/272: kiefermat: Only set returnval = 1 when we printed something
	  (in all cases print or !print). This simplifies the logic and fixes
	  the issue in the PR with -k and --mime-type there was no continuation
	  printed before the default case.

2021-06-30  13:07  Christos Zoulas <christos@zoulas.com>

	* PR/270: Don't translate unprintable characters in %s magic formats
	  when -r
	* PR/269: Avoid undefined behavior with clang (adding offset to NULL)

2021-05-09  18:38  Christos Zoulas <christos@zoulas.com>

	* Add a new flag (f) that requires that the match is a full word,
	   not a partial word match.
	* Add varint types (unused)

2021-04-19  17:17  Christos Zoulas <christos@zoulas.com>

	* PR/256: mutableVoid: If the file is less than 3 bytes, use the file
	  length to determine type
	* PR/259: aleksandr.v.novichkov: mime printing through indirect magic
	  is not taken into account, use match directly so that it does.

2021-04-04  17:02  Christos Zoulas <christos@zoulas.com>

	* count the total bytes found not the total byte positions
	  in order to determine encoding (Anatol Belski)

merge local changes between 5.39 and 5.40 and add magic entries from HEAD.

2021-03-30  20:21  Christos Zoulas <christos@zoulas.com>

	* release 5.40

2021-02-05  16:31  Christos Zoulas <christos@zoulas.com>

	* PR/234: Add limit to the number of bytes to scan for encoding
	* PR/230: Fix /T (trim flag) for regex

2021-02-01  12:31  Christos Zoulas <christos@zoulas.com>
	* PR/77: Trim trailing separator.

2020-12-17  15:44  Christos Zoulas <christos@zoulas.com>

	* PR/211: Convert system read errors from corrupt ELF
	  files into human readable error messages

2020-12-08  16:24  Christos Zoulas <christos@zoulas.com>

	* fix multithreaded decompression file descriptor issue
	  by using close-on-exec (Denys Vlasenko)

2020-06-27  11:58  Christos Zoulas <christos@zoulas.com>

	* Exclude surrogate pairs from utf-8 detection (Michael Liu)

2020-06-25  12:53  Christos Zoulas <christos@zoulas.com>

	* Include # to the list of ignored format chars (Werner Fink)

merge conflicts

Import 5.39:

   * Remove unused subtype_mime (Steve Grubb)
   * Remove unused check in okstat (Steve Grubb)
   * Fix mime-type in elf binaries by making sure $x is set
   * Fix indirect negative offsets broken by OFFNEGATIVE
   * Fix GUID equality check
   * PR/165: Handle empty array and strings in JSON
   * PR/162: Add --exclude-quiet
   * Fix memory leak in ascmagic (Steve Grubb)
   * Fix string comparison length with ignore whitespace
   * Fix mingwin 64 compilation
   * PR/159: whitelist getpid needed for file_pipe2file()
   * Indicate negative offsets with a flag OFFNEGATIVE
     so that -0 works.
   * Introduce "offset" magic type that can be used to
     detect the file size, and bail on short files.
   * document DER better in the magic man page.
   * fix memory leaks (SonarQube)
   * rewrite confusing loops (SonarQube)
   * fix bogus test (SonarQube)
   * pass a sized buffer to file_fmttime() (SonarQube)
   * Don't allow * in printf formats, or the code itself (Christoph Biedl)
   * Introduce a printf output size checker to avoid DoS attacks
   * Avoid memory leak on error (oss-fuzz)
   * Check length of string on DER before derefercing and add new types
   * Add missing DER string (oss-fuzz)
   * Add missing DER types, and debugging
   * PR/140: Avoid abort with hand-crafted magic file (gockelhahn)
   * PR/139:  Avoid DoS in printf with hand-crafted magic file (gockelhahn)
   * PR/138: Avoid crash with hand-crafted magic file (gockelhahn)
   * PR/136: Fix static build by adding a libmagic.pc (Fabrice Fontaine)
   * add guid support native support via the "guid" type.

Merge changes from current as of 20200406

merge conflicts

Import 5.38:
	- Always accept -S (no sandbox) even if we don't support sandboxing
	- More syscalls elided for sandboxiing
	- For ELF dynamic means having an interpreter not just PT_DYNAMIC
	- Check for large ELF session header offset
	- When saving and restoring a locale, keep the locale name in our
	  own storage.
	- Add a flag to disable CSV file detection.
	- Don't pass NULL/0 to memset to appease sanitizers.
	- Avoid spurious prints when looks for extensions or apple strings
	  in fsmagic.
	- Add builtin decompressors for xz and and bzip.
	- Add a limit for the number of CDF elements.
	- More checks for overflow in CDF.

Sync with HEAD

merge conflicts

2019-05-14  22:26  Christos Zoulas <christos@zoulas.com>

	* release 5.37

2019-05-09  22:27  Christos Zoulas <christos@zoulas.com>

	* Make sure that continuation separators are printed
	  with -k within softmagic

2019-05-06  22:27  Christos Zoulas <christos@zoulas.com>

	* Change SIGPIPE saving and restoring during compression to use
	  sigaction(2) instead of signal(3) and cache it. (Denys Vlasenko)
	* Cache stat(2) calls more to reduce number of calls (Denys Vlasenko)

2019-05-06  17:25  Christos Zoulas <christos@zoulas.com>

	* PR/77: Handle --mime-type and -k correctly.

2019-05-03  15:26  Christos Zoulas <christos@zoulas.com>

	* Switch decompression code to use vfork() because
	  tools like rpmdiff and rpmbuild call libmagic
	  with large process footprints (Denys Vlasenko)

2019-04-07  14:05  Christos Zoulas <christos@zoulas.com>

	* PR/75: --enable-zlib, did not work.

2019-02-27  11:54  Christos Zoulas <christos@zoulas.com>

	* Improve regex efficiency (Michael Schroeder) by:
		1. Prefixing regex searches with regular search
		   for keywords where possible
		2. Using memmem(3) where available

Sync with head

lint fixes

Merge conflicts

2018-10-18  19:32  Christos Zoulas <christos@zoulas.com>

	* release 5.35

2018-09-10  20:38  Christos Zoulas <christos@zoulas.com>

	* Add FreeBSD ELF core file support (John Baldwin)

2018-08-20  18:40  Christos Zoulas <christos@zoulas.com>

	* PR/30: Allow all parameter values to be set (don't treat 0 specially)
	* handle default annotations on the softmagic match instead at the
	  end.

2018-07-25  10:17  Christos Zoulas <christos@zoulas.com>

	* PR/23: Recognize JSON files

2018-07-25  10:17  Christos Zoulas <christos@zoulas.com>

	* PR/18: file --mime-encoding should not print mime-type

2018-07-25   8:50  Christos Zoulas <christos@zoulas.com>

	* release 5.34

2018-06-22  16:38  Christos Zoulas <christos@zoulas.com>

	* Add Quad indirect offsets

2018-05-24  14:10  Christos Zoulas <christos@zoulas.com>

	* Enable parsing of ELF dynamic sections to handle PIE better

Sync with HEAD

merge conflicts for file-5.33

2018-04-15  14:52  Christos Zoulas <christos@zoulas.com>

	* release 5.33

2018-02-24  14:50  Christos Zoulas <christos@zoulas.com>

	* extend the support for ${x?:} expansions for magic descriptions

2018-02-21  16:25  Christos Zoulas <christos@zoulas.com>

	* add support for ${x?:} in mime types to handle
	  pie binaries.

2017-11-03   9:23  Christos Zoulas <christos@zoulas.com>

	* add support for negative offsets (offsets from the end of file)

2017-09-26   8:22  Christos Zoulas <christos@zoulas.com>

	* close the file on error when writing magic (Steve Grubb)

2017-09-24  12:02  Christos Zoulas <christos@zoulas.com>

	* seccomp support (Paul Moore)

2017-09-02  11:53  Christos Zoulas <christos@zoulas.com>

	* release 5.32

2017-08-28  16:37  Christos Zoulas <christos@zoulas.com>

	* Always reset state in {file,buffer}_apprentice (Krzysztof Wilczynski)

2017-08-27  03:55  Christos Zoulas <christos@zoulas.com>

	* Fix always true condition (Thomas Jarosch)

2017-05-24  17:30  Christos Zoulas <christos@zoulas.com>

	* pickier parsing of numeric values in magic files.

2017-05-23  17:55  Christos Zoulas <christos@zoulas.com>

	* PR/615 add magic_getflags()

merge file-5.32

2017-09-02  11:53  Christos Zoulas <christos@zoulas.com>

	* release 5.32

2017-08-28  16:37  Christos Zoulas <christos@zoulas.com>

	* Always reset state in {file,buffer}_apprentice (Krzysztof Wilczynski)

2017-08-27  03:55  Christos Zoulas <christos@zoulas.com>

	* Fix always true condition (Thomas Jarosch)

2017-05-24  17:30  Christos Zoulas <christos@zoulas.com>

	* pickier parsing of numeric values in magic files.

2017-05-23  17:55  Christos Zoulas <christos@zoulas.com>

	* PR/615 add magic_getflags()

2017-05-23  13:55  Christos Zoulas <christos@zoulas.com>

	* release 5.31

2017-03-17  20:32  Christos Zoulas <christos@zoulas.com>

	* remove trailing spaces from magic files
	* refactor is_tar
	* better bounds checks for cdf

merge 5.31

Import file-5.31; mostly oss-fuzz found bugs.

Sync with HEAD

Sync with HEAD

Ignore files that start with .

pacify lint.

merge conflicts

2017-02-10  12:24  Christos Zoulas <christos@zoulas.com>

	* release 5.30

2017-02-07  23:27  Christos Zoulas <christos@zoulas.com>

	* If we exceeded the offset in a search return no match
	  (Christoph Biedl)
	* Be more lenient on corrupt CDF files (Christoph Biedl)

2017-02-04  16:46  Christos Zoulas <christos@zoulas.com>

	* pacify ubsan sign extension (oss-fuzz/524)

2017-02-01  12:42  Christos Zoulas <christos@zoulas.com>

	* off by one in cdf parsing (PR/593)
	* report debugging sections in elf (PR/591)

2016-11-06  10:52  Christos Zoulas <christos@zoulas.com>

	* Allow @@@ in extensions
	* Add missing overflow check in der magic (Jonas Wagner)

2016-10-25  10:40  Christos Zoulas <christos@zoulas.com>

	* release 5.29

2016-10-24  11:20  Christos Zoulas <christos@zoulas.com>

	* der getlength overflow (Jonas Wagner)
	* multiple magic file load failure (Christoph Biedl)

2016-10-17  11:26  Christos Zoulas <christos@zoulas.com>

	* CDF parsing improvements (Guy Helmer)

2016-07-20   7:26  Christos Zoulas <christos@zoulas.com>

	* Add support for signed indirect offsets

2016-07-18   7:41  Christos Zoulas <christos@zoulas.com>

	* cat /dev/null | file - should print empty (Christoph Biedl)

2016-07-05  15:20  Christos Zoulas <christos@zoulas.com>

	* Bump string size from 64 to 96.

2016-06-13  20:20  Christos Zoulas <christos@zoulas.com>

	* PR/556: Fix separators on annotations.

2016-06-13  19:40  Christos Zoulas <christos@zoulas.com>

	* release 5.28
	* fix leak on allocation failure

2016-06-01   1:20  Christos Zoulas <christos@zoulas.com>

	* PR/555: Avoid overflow for offset > nbytes
	* PR/550: Segv on DER parsing:
	    - use the correct variable for length
	    - set offset to 0 on failure.

2016-05-13  12:00  Christos Zoulas <christos@zoulas.com>

	* release 5.27

2016-04-18   9:35  Christos Zoulas <christos@zoulas.com>

	* Errors comparing DER entries or computing offsets
	  are just indications of malformed non-DER files.
	  Don't print them.
	* Offset comparison was off-by-one.
	* Fix compression code (Werner Fink)
	* Put new bytes constant in the right file (not the generated one)

2016-04-16  18:34  Christos Zoulas <christos@zoulas.com>

	* release 5.26

2016-03-31  13:50  Christos Zoulas <christos@zoulas.com>

	* make the number of bytes read from files configurable.

2016-03-21  13:40  Christos Zoulas <christos@zoulas.com>

	* Add bounds checks for DER code (discovered by Thomas Jarosch)
	* Change indirect recursion limit to indirect use count and
	  bump from 15 to 50 to prevent abuse.

2016-03-13  20:39  Christos Zoulas <christos@zoulas.com>

	* Add -00 which prints filename\0description\0

2016-03-01  13:28  Christos Zoulas <christos@zoulas.com>

	* Fix ID3 indirect parsing

2016-01-19  10:18  Christos Zoulas <christos@zoulas.com>

	* add DER parsing capability

2015-11-13  10:35  Christos Zoulas <christos@zoulas.com>

	* provide dprintf(3) for the OS's that don't have it.

2015-11-11  16:25  Christos Zoulas <christos@zoulas.com>

	* redo the compression code report decompression errors

2015-11-10  23:25  Christos Zoulas <christos@zoulas.com>

	* REG_STARTEND code is not working as expected, delete it.

2015-11-09  16:05  Christos Zoulas <christos@zoulas.com>

	* Add zlib support if we have it.

2015-11-05  11:22  Christos Zoulas <christos@zoulas.com>

	* PR/492: compression forking was broken with magic_buffer.

2015-09-16   9:50  Christos Zoulas <christos@zoulas.com>

	* release 5.25

2015-09-11  13:25  Christos Zoulas <christos@zoulas.com>

	* add a limit to the length of regex searches

2015-09-08   9:50  Christos Zoulas <christos@zoulas.com>

	* fix problems with --parameter (Christoph Biedl)

2015-07-11  10:35  Christos Zoulas <christos@zoulas.com>

	* Windows fixes PR/466 (Jason Hood)

2015-07-09  10:35  Christos Zoulas <christos@zoulas.com>

	* release 5.24

2015-06-11   8:52  Christos Zoulas <christos@zoulas.com>

	* redo long option encoding to fix off-by-one in 5.23

2015-06-10  13:50  Christos Zoulas <christos@zoulas.com>

	* release 5.23

2015-06-09  16:10  Christos Zoulas <christos@zoulas.com>

	* Fix issue with regex range for magic with offset
	* Always return true from mget with USE (success to mget not match
	  indication). Fixes mime evaluation after USE magic
	* PR/459: Don't insert magic entries to the list if there are parsing
	  errors for them.

2015-06-03  16:00  Christos Zoulas <christos@zoulas.com>

	* PR/455: Add utf-7 encoding

2015-06-03  14:30  Christos Zoulas <christos@zoulas.com>

	* PR/455: Implement -Z, look inside, but don't report on compression
	* PR/454: Fix allocation error on bad magic.

2015-05-29  10:30  Christos Zoulas <christos@zoulas.com>

	* handle MAGIC_CONTINUE everywhere, not just in softmagic

2015-05-21  14:30  Christos Zoulas <christos@zoulas.com>

	* don't print descriptions for NAME types when mime.

2015-04-09  15:59  Christos Zoulas <christos@zoulas.com>

	* Add --extension to list the known extensions for this file type
	  Idea by Andrew J Roazen

2015-02-14  12:23  Christos Zoulas <christos@zoulas.com>

	* Bump file search buffer size to 1M.

2015-01-09  14:35  Christos Zoulas <christos@zoulas.com>

	* Fix multiple issues with date formats reported by Christoph Biedl:
		- T_LOCAL meaning was reversed
		- Arithmetic did not work
	  Also stop adjusting daylight savings for gmt printing.

2015-01-05  13:00  Christos Zoulas <christos@zoulas.com>

	* PR/411: Fix memory corruption from corrupt cdf file.

merge conflicts

Import file-5.22

sync with head of upstream to fix MacOS/X compilation and more.

welcome to file-5.20

import file-5.20; bug fixes and better image magic descriptions.

Rebase to HEAD as of a few days ago.

Rebase.

merge conflicts

import file-5.19

2014-06-09   9:04  Christos Zoulas <christos@zoulas.com>

	* Misc buffer overruns and missing buffer size tests in cdf parsing
	  (Francisco Alonso, Jan Kaluza)

2014-06-02  14:50  Christos Zoulas <christos@zoulas.com>

	* Enforce limit of 8K on regex searches that have no limits
	* Allow the l modifier for regex to mean line count. Default
	  to byte count. If line count is specified, assume a max
	  of 80 characters per line to limit the byte count.
	* Don't allow conversions to be used for dates, allowing
	  the mask field to be used as an offset.

2014-05-30  12:51  Christos Zoulas <christos@zoulas.com>

	* Make the range operator limit the length of the
	  regex search.

2014-05-14  19:23  Christos Zoulas <christos@zoulas.com>

	* PR/347: Windows fixes
	* PR/352: Hangul word processor recognition
	* PR/354: Encoding irregularities in text files

2014-05-06  6:12  Christos Zoulas <christos@zoulas.com>

	* Fix uninitialized title in CDF files (Jan Kaluza)

2014-05-04  14:55  Christos Zoulas <christos@zoulas.com>

	* PR/351: Fix compilation of empty files

2014-04-30  17:39  Christos Zoulas <christos@zoulas.com>

	* Fix integer formats: We don't specify 'l' or
	  'h' and 'hh' specifiers anymore, only 'll' for
	  quads and nothing for the rest. This is so that
	  magic writing is simpler.

2014-04-01  15:25  Christos Zoulas <christos@zoulas.com>

	* PR/341: Jan Kaluza, fix memory leak
	* PR/342: Jan Kaluza, fix out of bounds read

2014-03-28  15:25  Christos Zoulas <christos@zoulas.com>

	* Fix issue with long formats not matching fmtcheck

sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs.  ("Protocol error: too many arguments")

merge conflicts

from ftp.astron.com, this is a bug fix release:
       * always leave magic file loaded, don't unload for magic_check, etc.
       * fix default encoding to binary instead of unknown which broke recently
       * handle empty and one byte files, less specially so that
         --mime-encoding does not break completely.
       * fix erroneous non-zero exit code from non-existant file and message
       * add CDF MSI file detection (Guy Helmer)

resync from head

merge new file

import file-5.14
changes are "name" + "use" keyword features, bug fixes

resync with head

sync with head

merge changes

from ftp.astron.com

sync with head

bring in changes from head.

Pull up following revision(s) (requested by christos in ticket #30):
	Update file to 5.11 (CDF security fixes)

merge conflicts

from ftp.astron.com, CDF security fixes

merge changes

from ftp.astron.com:
- elf seeking fixes
- improvements in cdf parsing
- waitpid only for our child
- magic fixes and additions

Remove a noisy debugging printf that triggers on bigendian platforms.
(Reported upstream as well)

resolve conflicts

from ftp.astron.com.
- many security related fixes
- no MAXPATHLEN limits
- fixed missing text specification on ascii magic
- new ``pascal'' style string formats
- whitespace comparison fix
- more magic

Sync with HEAD.

Second commit. See http://mail-index.netbsd.org/source-changes/2009/05/13/msg221222.html

file apprentice.c was added on branch jym-xensuspend on 2009-05-13 18:52:02 +0000

build glue and lint fixes

from ftp.astron.com

Initial revision