The NetBSD Project

CVS log for src/usr.sbin/makemandb/Makefile

[BACK] Up to [cvs.NetBSD.org] / src / usr.sbin / makemandb

Request diff between arbitrary revisions


Default branch: MAIN


Revision 1.9 / (download) - annotate - [select for diffs], Sun Jun 18 16:24:10 2017 UTC (3 months ago) by abhinav
Branch: MAIN
CVS Tags: perseant-stdc-iso10646-base, perseant-stdc-iso10646, HEAD
Changes since 1.8: +13 -6 lines
Diff to previous 1.8 (colored)

Add a custom tokenizer which does not stem certain keywords.

Which keywords should not be stemmed is specified in the nostem.txt file.
(Right now I have taken all the man page names, split them if they had
underscores, removed common English words and converted everything to
lowercase.)

The tokenizer itself is based on the Porter stemming tokenizer shipped with
Sqlite. The code in custom_apropos_tokenizer.c is copy of that code with
some modifications to prevent stemming keywords specified in nostem.txt.

Additionally, it now uses underscore `_' also as a token delimiter. Therefore,
now it's possible to do query for `lwp' and all `_lwp_*' man page names
will be matched. Or the query can be `unconst' and `__UNCONST' will be matched.
This was not possible earlier, because underscore was not a delimiter and therefore
the index would have __UNCONST as a key rather than UNCONST.

The tokenizer needs fts3_tokenizer.h file, which is not shipped with the
amalgamation build of Sqlite, therefore it needs to be added here (unless
we decide there is a better place for it).

To enforce using the new tokenizer, a schema version bump is needed

Since the tokenization is done both at the indexing time (via makemandb) and
also while query time (via apropos or whatis), it will be needed to bump
the schema version everytime nostem.txt is modified. Otherwise the
index will consist of old tokens and desired changes will not be seen with
apropos.

This should also fix the issue reported in PR bin/46255. Similar suggestion was
also made on tech-userlevel@ recently:
<http://mail-index.netbsd.org/tech-userlevel/2017/06/08/msg010620.html>

Thanks to christos@ for multiple rounds of reviews of the tokenizer code.

Revision 1.8 / (download) - annotate - [select for diffs], Sun May 21 15:28:43 2017 UTC (3 months, 4 weeks ago) by riastradh
Branch: MAIN
CVS Tags: netbsd-8-base, netbsd-8
Changes since 1.7: +1 -3 lines
Diff to previous 1.7 (colored)

Remove MKCRYPTO option.

Originally, MKCRYPTO was introduced because the United States
classified cryptography as a munition and restricted its export.  The
export controls were substantially relaxed fifteen years ago, and are
essentially irrelevant for software with published source code.

In the intervening time, nobody bothered to remove the option after
its motivation -- the US export restriction -- was eliminated.  I'm
not aware of any other operating system that has a similar option; I
expect it is mainly out of apathy for churn that we still have it.
Today, cryptography is an essential part of modern computing -- you
can't use the internet responsibly without cryptography.

The position of the TNF board of directors is that TNF makes no
representation that MKCRYPTO=no satisfies any country's cryptography
regulations.

My personal position is that the availability of cryptography is a
basic human right; that any local laws restricting it to a privileged
few are fundamentally immoral; and that it is wrong for developers to
spend effort crippling cryptography to work around such laws.

As proposed on tech-crypto, tech-security, and tech-userlevel to no
objections:

https://mail-index.netbsd.org/tech-crypto/2017/05/06/msg000719.html
https://mail-index.netbsd.org/tech-security/2017/05/06/msg000928.html
https://mail-index.netbsd.org/tech-userlevel/2017/05/06/msg010547.html

P.S.  Reviewing all the uses of MKCRYPTO in src revealed a lot of
*bad* crypto that was conditional on it, e.g. DES in telnet...  That
should probably be removed too, but on the grounds that it is bad,
not on the grounds that it is (nominally) crypto.

Revision 1.5.12.2 / (download) - annotate - [select for diffs], Wed Apr 26 02:53:36 2017 UTC (4 months, 3 weeks ago) by pgoyette
Branch: pgoyette-localcount
Changes since 1.5.12.1: +6 -1 lines
Diff to previous 1.5.12.1 (colored) to branchpoint 1.5 (colored) next main 1.6 (colored)

Sync with HEAD

Revision 1.7 / (download) - annotate - [select for diffs], Fri Apr 21 23:07:45 2017 UTC (4 months, 4 weeks ago) by christos
Branch: MAIN
CVS Tags: prg-localcount2-base3, prg-localcount2-base2, prg-localcount2-base1, prg-localcount2-base, prg-localcount2, pgoyette-localcount-20170426
Changes since 1.6: +6 -1 lines
Diff to previous 1.6 (colored)

libarchive now needs crypto

Revision 1.5.12.1 / (download) - annotate - [select for diffs], Tue Jul 26 03:24:24 2016 UTC (13 months, 3 weeks ago) by pgoyette
Branch: pgoyette-localcount
Changes since 1.5: +3 -3 lines
Diff to previous 1.5 (colored)

Sync with HEAD

Revision 1.6 / (download) - annotate - [select for diffs], Thu Jul 21 12:24:54 2016 UTC (13 months, 4 weeks ago) by christos
Branch: MAIN
CVS Tags: pgoyette-localcount-20170320, pgoyette-localcount-20170107, pgoyette-localcount-20161104, pgoyette-localcount-20160806, pgoyette-localcount-20160726, localcount-20160914, bouyer-socketcan-base1, bouyer-socketcan-base, bouyer-socketcan
Changes since 1.5: +3 -3 lines
Diff to previous 1.5 (colored)

Add -lz to makefile to fix the build.

Revision 1.2.4.2 / (download) - annotate - [select for diffs], Mon Feb 25 00:30:45 2013 UTC (4 years, 6 months ago) by tls
Branch: tls-maxphys
Changes since 1.2.4.1: +4 -5 lines
Diff to previous 1.2.4.1 (colored) to branchpoint 1.2 (colored) next main 1.3 (colored)

resync with head

Revision 1.2.2.4 / (download) - annotate - [select for diffs], Wed Jan 23 00:06:43 2013 UTC (4 years, 7 months ago) by yamt
Branch: yamt-pagecache
CVS Tags: yamt-pagecache-tag8
Changes since 1.2.2.3: +4 -5 lines
Diff to previous 1.2.2.3 (colored) to branchpoint 1.2 (colored) next main 1.3 (colored)

sync with head

Revision 1.5 / (download) - annotate - [select for diffs], Mon Jan 14 18:04:58 2013 UTC (4 years, 8 months ago) by christos
Branch: MAIN
CVS Tags: yamt-pagecache-base9, yamt-pagecache-base8, tls-maxphys-base, tls-earlyentropy-base, tls-earlyentropy, riastradh-xf86-video-intel-2-7-1-pre-2-21-15, riastradh-drm2-base3, riastradh-drm2-base2, riastradh-drm2-base1, riastradh-drm2-base, riastradh-drm2, pgoyette-localcount-base, netbsd-7-nhusb-base-20170116, netbsd-7-nhusb-base, netbsd-7-nhusb, netbsd-7-base, netbsd-7-1-RELEASE, netbsd-7-1-RC2, netbsd-7-1-RC1, netbsd-7-1, netbsd-7-0-RELEASE, netbsd-7-0-RC3, netbsd-7-0-RC2, netbsd-7-0-RC1, netbsd-7-0-2-RELEASE, netbsd-7-0-1-RELEASE, netbsd-7-0, netbsd-7, agc-symver-base, agc-symver
Branch point for: pgoyette-localcount
Changes since 1.4: +3 -3 lines
Diff to previous 1.4 (colored)

Fix the #ifdef NOTYET, namely use the terminal standout mode to highlight the
search terms.

Revision 1.4 / (download) - annotate - [select for diffs], Mon Jan 14 18:01:59 2013 UTC (4 years, 8 months ago) by christos
Branch: MAIN
Changes since 1.3: +2 -3 lines
Diff to previous 1.3 (colored)

Since mdocml decided to name headers that conflict with system ones (term.h)
move the header inclusion one up.

Revision 1.2.4.1 / (download) - annotate - [select for diffs], Tue Nov 20 03:03:02 2012 UTC (4 years, 10 months ago) by tls
Branch: tls-maxphys
Changes since 1.2: +8 -5 lines
Diff to previous 1.2 (colored)

Resync to 2012-11-19 00:00:00 UTC

Revision 1.2.2.3 / (download) - annotate - [select for diffs], Tue Oct 30 19:00:37 2012 UTC (4 years, 10 months ago) by yamt
Branch: yamt-pagecache
Changes since 1.2.2.2: +8 -5 lines
Diff to previous 1.2.2.2 (colored) to branchpoint 1.2 (colored)

sync with head

Revision 1.3 / (download) - annotate - [select for diffs], Sat Oct 6 15:33:59 2012 UTC (4 years, 11 months ago) by wiz
Branch: MAIN
CVS Tags: yamt-pagecache-base7, yamt-pagecache-base6
Changes since 1.2: +8 -5 lines
Diff to previous 1.2 (colored)

Make mandb path configurable. makemandb (and related tools) use
the path from the _mandb variable from man.conf now.

Set _mandb in man.conf to same value as was used before.

From Abhinav Upadhyay <er.abhinav.upadhyay@gmail.com>.

Revision 1.2.2.2 / (download) - annotate - [select for diffs], Tue Apr 17 00:09:49 2012 UTC (5 years, 5 months ago) by yamt
Branch: yamt-pagecache
Changes since 1.2.2.1: +40 -0 lines
Diff to previous 1.2.2.1 (colored) to branchpoint 1.2 (colored)

sync with head

Revision 1.1.2.1 / (download) - annotate - [select for diffs], Sat Feb 18 18:03:26 2012 UTC (5 years, 7 months ago) by riz
Branch: netbsd-6
CVS Tags: netbsd-6-1-RELEASE, netbsd-6-1-RC4, netbsd-6-1-RC3, netbsd-6-1-RC2, netbsd-6-1-RC1, netbsd-6-1-5-RELEASE, netbsd-6-1-4-RELEASE, netbsd-6-1-3-RELEASE, netbsd-6-1-2-RELEASE, netbsd-6-1-1-RELEASE, netbsd-6-1, netbsd-6-0-RELEASE, netbsd-6-0-RC2, netbsd-6-0-RC1, netbsd-6-0-6-RELEASE, netbsd-6-0-5-RELEASE, netbsd-6-0-4-RELEASE, netbsd-6-0-3-RELEASE, netbsd-6-0-2-RELEASE, netbsd-6-0-1-RELEASE, netbsd-6-0, matt-nb6-plus-nbase, matt-nb6-plus-base, matt-nb6-plus
Changes since 1.1: +3 -3 lines
Diff to previous 1.1 (colored) next main 1.2 (colored)

Pull up following revision(s) (requested by joerg in ticket #4):
	usr.sbin/makemandb/apropos.c: revision 1.5
	external/bsd/mdocml/dist/read.c: revision 1.6
	external/bsd/mdocml/dist/mandoc.h: revision 1.2
	usr.sbin/makemandb/makemandb.c: revision 1.3
	usr.sbin/makemandb/makemandb.c: revision 1.4
	usr.sbin/makemandb/makemandb.c: revision 1.5
	usr.sbin/makemandb/Makefile: revision 1.2
	etc/man.conf: revision 1.30
	usr.sbin/makemandb/apropos.1: revision 1.3
	external/bsd/mdocml/Makefile.inc: revision 1.15
Be a bit more friendly to man pages using the roff .so command by
changing the current directory to the parent of the man -p entry, e.g.
/usr/share/man for /usr/share/man1.
Also handle hyphen replacement if it was used as plain input and no
backslash sequence was used at all in the line.
Add support for apropos -s like in the old apropos. Fix capitalisation.
Add note about optional quotation. From Abhinav Upadhyay.
Merge mandoc.h r1.99 and read.c 1.28 from upstream:
Add mparse_readmem, which allows application to run the parser directly
from memory, e.g. after de-compressing a document.
Add support for compressed man pages in all the usual formats.
Use mmap when dealing with plain files.
Add entries for bzip2 and xz.

Revision 1.2.2.1, Thu Feb 16 20:58:55 2012 UTC (5 years, 7 months ago) by yamt
Branch: yamt-pagecache
Changes since 1.2: +0 -40 lines
FILE REMOVED

file Makefile was added on branch yamt-pagecache on 2012-04-17 00:09:49 +0000

Revision 1.2 / (download) - annotate - [select for diffs], Thu Feb 16 20:58:55 2012 UTC (5 years, 7 months ago) by joerg
Branch: MAIN
CVS Tags: yamt-pagecache-base5, yamt-pagecache-base4
Branch point for: yamt-pagecache, tls-maxphys
Changes since 1.1: +3 -3 lines
Diff to previous 1.1 (colored)

Add support for compressed man pages in all the usual formats.

Revision 1.1 / (download) - annotate - [select for diffs], Tue Feb 7 19:13:32 2012 UTC (5 years, 7 months ago) by joerg
Branch: MAIN
CVS Tags: netbsd-6-base
Branch point for: netbsd-6

Import the new apropos/whatis.

This code has been developed by Abhinav Upadhyay as part of Google's Summer
of Code 2011. It uses libmandoc to parse man pages and builds a Full
Text Index in a SQLite database. The combination of indexing the full
manual page, filtering out stop words and ranking individual matches
based on the section gives a much improved user experience.

The old makewhatis and friends are kept under MKMAKEMANDB=no for now.

This form allows you to request diff's between any two revisions of a file. You may select a symbolic revision name using the selection box or you may type in a numeric name using the type-in text box.




CVSweb <webmaster@jp.NetBSD.org>