CVS log for src/usr.bin/mklocale/mklocale.1

Up to [cvs.NetBSD.org] / src / usr.bin / mklocale

Request diff between arbitrary revisions

Keyword substitution: kv
Default branch: MAIN

Pull up following revision(s) (requested by rin in ticket #538):

	usr.bin/mklocale/yacc.y: revision 1.35
	usr.bin/mklocale/yacc.y: revision 1.36
	usr.bin/mklocale/mklocale.1: revision 1.18
	usr.bin/mklocale/mklocale.1: revision 1.19

mklocale: XXX: Neglect TODIGIT at the moment
PR lib/57798

It was implemented with an assumption that all digit characters
can be mapped to numerical values <= 255.
This is no longer true for Unicode, and results in, e.g., wrong
return values of wcwidth(3) for U+5146 or U+16B60.

As a workaround, neglect TODIGIT for now, as done for OpenBSD:
https://github.com/OpenBSD/src/commit/4efe9bdeb34
XXX

At least netbsd-10 should be fixed, but it requires some tests.


mklocale(1): Add range check for TODIGIT, rather than disabling it
PR lib/57798

Digit value specified by TODIGIT is storaged as lowest 8 bits of
_RuneType, see lib/libc/locale/runetype_file.h:
https://nxr.netbsd.org/xref/src/lib/libc/locale/runetype_file.h#56

The symptom reported in the PR is due to missing range check for
this value; values of 256 and above were mistakenly treated as
other flag bits in _RuneType.

For example, U+5146 has numerical value 1000,000,000,000 ==
0xe8d4a51000 where __BITS(30, 31) == _RUNETYPE_SW3 are turned on.

This is why wcwidth(3) returned 3 for this character.

This apparently affected not only character width, but also other
attributes storaged in _RuneType.

IIUC, digit value attributes in _RuneType have never been utilized
until now, but preserve these if digit fits within (0, 256). This
should be safer for pulling this up into netbsd-10. Also, these
attributes may be useful to implement some I18N features as
suggested by uwe@ in the PR.

netbsd-[98] is not affected as these use old UTF-8 ctype definitions.

mklocale(1): Add range check for TODIGIT, rather than disabling it

PR lib/57798

Digit value specified by TODIGIT is storaged as lowest 8 bits of
_RuneType, see lib/libc/locale/runetype_file.h:

https://nxr.netbsd.org/xref/src/lib/libc/locale/runetype_file.h#56

The symptom reported in the PR is due to missing range check for
this value; values of 256 and above were mistakenly treated as
other flag bits in _RuneType.

For example, U+5146 has numerical value 1000,000,000,000 ==
0xe8d4a51000 where __BITS(30, 31) == _RUNETYPE_SW3 are turned on.
This is why wcwidth(3) returned 3 for this character.

This apparently affected not only character width, but also other
attributes storaged in _RuneType.

IIUC, digit value attributes in _RuneType have never been utilized
until now, but preserve these if digit fits within (0, 256). This
should be safer for pulling this up into netbsd-10. Also, these
attributes may be useful to implement some I18N features as
suggested by uwe@ in the PR.

netbsd-[98] is not affected as these use old UTF-8 ctype definitions.

mklocale: XXX: Neglect TODIGIT at the moment

PR lib/57798

It was implemented with an assumption that all digit characters
can be mapped to numerical values <= 255.

This is no longer true for Unicode, and results in, e.g., wrong
return values of wcwidth(3) for U+5146 or U+16B60.

As a workaround, neglect TODIGIT for now, as done for OpenBSD:
https://github.com/OpenBSD/src/commit/4efe9bdeb34

XXX
At least netbsd-10 should be fixed, but it requires some tests.

Remove workaround for ancient HTML generation code.

Rebase to HEAD as of a few days ago.

sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs.  ("Protocol error: too many arguments")

Fix description of hex encoding.

Escape punctuation. Use logical markup for indentation.

Pull up following revision(s) (requested by wiz in ticket #225):
	usr.bin/mklocale/mklocale.1: revision 1.14
Document new -t flag, bump date.

Document new -t flag, bump date.

Some fixes from jmc@openbsd.

Move UCB-licensed code from 4-clause to 3-clause licence.

Patches provided by Joel Baker in PR 22365, verified by myself.

Comment out another euc(4) reference.
Prefer mdoc to man plain roff.
Use \*[Lt] and \*[Gt] instead of < and > for HTML output, but prefer
.Aq if possible.
"a, b, and c" instead of "a, b and c".

Remove references to non-existent pages.  Addresses PR#12544.

Debugging is usually spelled with three gs.

New sentence, new line.

Generate <>& symbolically. I'm avoiding .../dist/... directories for now.

Sort sections.

Comment out colldef xref.

Remove references to rune(3), mbrune(3), and utf2(4).
Fix some markup, and remove a sentence.
OK'd by soda.

noted that we should switch to localedef.

populate citrus xpg4dl locale definitions into /usr/share.
if there are any mistakes, please report (i only speak English/Japanese).
currently, only singlebyte locales are usable.

TODO: enable multibyte handler in libc - there still are few design
discussions, but we are almost ready.

Import of most recent XPG4DL dated 2000.06.23.

Citrus Project XPG4DL, an implementation of I18N (locale) framework,
is imported.

file mklocale.1 was initially added on branch minoura-xpg4dl.