The NetBSD Project

CVS log for src/usr.sbin/makemandb/apropos-utils.h

[BACK] Up to [cvs.NetBSD.org] / src / usr.sbin / makemandb

Request diff between arbitrary revisions


Default branch: MAIN


Revision 1.14 / (download) - annotate - [select for diffs], Sat Nov 25 14:29:38 2017 UTC (3 weeks ago) by abhinav
Branch: MAIN
CVS Tags: HEAD
Changes since 1.13: +13 -3 lines
Diff to previous 1.13 (colored)

Encapsulate all the arguments required by the query callback function in a struct.

If we want to add or remove arguments from the callback functions, it requires
changing the callback interface all over the place. By letting the callback simply
expect a single struct argument, it would clean things up a bit.

ok christos

Revision 1.13 / (download) - annotate - [select for diffs], Sun Jun 18 16:24:10 2017 UTC (5 months, 4 weeks ago) by abhinav
Branch: MAIN
CVS Tags: perseant-stdc-iso10646-base, perseant-stdc-iso10646
Changes since 1.12: +2 -2 lines
Diff to previous 1.12 (colored)

Add a custom tokenizer which does not stem certain keywords.

Which keywords should not be stemmed is specified in the nostem.txt file.
(Right now I have taken all the man page names, split them if they had
underscores, removed common English words and converted everything to
lowercase.)

The tokenizer itself is based on the Porter stemming tokenizer shipped with
Sqlite. The code in custom_apropos_tokenizer.c is copy of that code with
some modifications to prevent stemming keywords specified in nostem.txt.

Additionally, it now uses underscore `_' also as a token delimiter. Therefore,
now it's possible to do query for `lwp' and all `_lwp_*' man page names
will be matched. Or the query can be `unconst' and `__UNCONST' will be matched.
This was not possible earlier, because underscore was not a delimiter and therefore
the index would have __UNCONST as a key rather than UNCONST.

The tokenizer needs fts3_tokenizer.h file, which is not shipped with the
amalgamation build of Sqlite, therefore it needs to be added here (unless
we decide there is a better place for it).

To enforce using the new tokenizer, a schema version bump is needed

Since the tokenization is done both at the indexing time (via makemandb) and
also while query time (via apropos or whatis), it will be needed to bump
the schema version everytime nostem.txt is modified. Otherwise the
index will consist of old tokens and desired changes will not be seen with
apropos.

This should also fix the issue reported in PR bin/46255. Similar suggestion was
also made on tech-userlevel@ recently:
<http://mail-index.netbsd.org/tech-userlevel/2017/06/08/msg010620.html>

Thanks to christos@ for multiple rounds of reviews of the tokenizer code.

Revision 1.11.6.1 / (download) - annotate - [select for diffs], Tue May 2 03:19:23 2017 UTC (7 months, 2 weeks ago) by pgoyette
Branch: prg-localcount2
Changes since 1.11: +2 -2 lines
Diff to previous 1.11 (colored) next main 1.12 (colored)

Sync with HEAD - tag prg-localcount2-base1

Revision 1.12 / (download) - annotate - [select for diffs], Mon May 1 05:28:00 2017 UTC (7 months, 2 weeks ago) by abhinav
Branch: MAIN
CVS Tags: prg-localcount2-base3, prg-localcount2-base2, prg-localcount2-base1, netbsd-8-base, netbsd-8, matt-nb8-mediatek-base, matt-nb8-mediatek
Changes since 1.11: +2 -2 lines
Diff to previous 1.11 (colored)

Simplify handling of the section arguments in apropos(1).

Earlier, a white space separated string was generated containing all the section
numbers passed through command line arguments. Later on that would have to be
tokenized and processed. Instead of that, use a NULL terminated array of strings.

Thanks to christos@ for reviewing and suggesting further improvements.

Revision 1.11 / (download) - annotate - [select for diffs], Wed Apr 13 11:48:29 2016 UTC (20 months ago) by christos
Branch: MAIN
CVS Tags: prg-localcount2-base, pgoyette-localcount-base, pgoyette-localcount-20170426, pgoyette-localcount-20170320, pgoyette-localcount-20170107, pgoyette-localcount-20161104, pgoyette-localcount-20160806, pgoyette-localcount-20160726, pgoyette-localcount, localcount-20160914, bouyer-socketcan-base1, bouyer-socketcan-base, bouyer-socketcan
Branch point for: prg-localcount2
Changes since 1.10: +2 -3 lines
Diff to previous 1.10 (colored)

PR/51062: Abhinav Upadhyay: Allow non numeric sections to be indexed and
searched by apropos(1).
Fold long lines.

Revision 1.10 / (download) - annotate - [select for diffs], Wed Apr 13 01:37:50 2016 UTC (20 months ago) by christos
Branch: MAIN
Changes since 1.9: +8 -5 lines
Diff to previous 1.9 (colored)

PR/51038: Abhinav Upadhyay: check for access permissions to the sqlite database

Revision 1.2.4.6 / (download) - annotate - [select for diffs], Thu May 22 11:43:06 2014 UTC (3 years, 6 months ago) by yamt
Branch: yamt-pagecache
Changes since 1.2.4.5: +11 -6 lines
Diff to previous 1.2.4.5 (colored) to branchpoint 1.2 (colored) next main 1.3 (colored)

sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs.  ("Protocol error: too many arguments")

Revision 1.3.2.3 / (download) - annotate - [select for diffs], Sun Jun 23 06:29:05 2013 UTC (4 years, 5 months ago) by tls
Branch: tls-maxphys
Changes since 1.3.2.2: +10 -7 lines
Diff to previous 1.3.2.2 (colored) to branchpoint 1.3 (colored) next main 1.4 (colored)

resync from head

Revision 1.9 / (download) - annotate - [select for diffs], Tue Apr 2 17:16:50 2013 UTC (4 years, 8 months ago) by christos
Branch: MAIN
CVS Tags: yamt-pagecache-base9, tls-maxphys-base, tls-earlyentropy-base, tls-earlyentropy, riastradh-xf86-video-intel-2-7-1-pre-2-21-15, riastradh-drm2-base3, riastradh-drm2-base2, riastradh-drm2-base1, riastradh-drm2-base, riastradh-drm2, netbsd-7-nhusb-base-20170116, netbsd-7-nhusb-base, netbsd-7-nhusb, netbsd-7-base, netbsd-7-1-RELEASE, netbsd-7-1-RC2, netbsd-7-1-RC1, netbsd-7-1, netbsd-7-0-RELEASE, netbsd-7-0-RC3, netbsd-7-0-RC2, netbsd-7-0-RC1, netbsd-7-0-2-RELEASE, netbsd-7-0-1-RELEASE, netbsd-7-0, netbsd-7
Changes since 1.8: +9 -7 lines
Diff to previous 1.8 (colored)

instead of having a format and no format flag, and exposing various formatters,
provide a format enum and expose html formatting too.

Revision 1.8 / (download) - annotate - [select for diffs], Fri Mar 29 20:07:31 2013 UTC (4 years, 8 months ago) by christos
Branch: MAIN
Changes since 1.7: +2 -1 lines
Diff to previous 1.7 (colored)

- If the stdout is not a tty, prevent formatting unless forced with -i
- Don't ever page unless asked for with -p
- Introduce "legacy mode" (-l)
  1. searches only name and name_desc, prints name(section) - name_description
  2. turns off escape formatting (can be forced on with -i)
  3. turns off context printing (can be forced on with -c)
- Parse the environment $APROPOS variable as an argument vector.

With these changes one can simply 'export APROPOS=-l' and get the old apropos
behavior.

Revision 1.3.2.2 / (download) - annotate - [select for diffs], Mon Feb 25 00:30:45 2013 UTC (4 years, 9 months ago) by tls
Branch: tls-maxphys
Changes since 1.3.2.1: +5 -2 lines
Diff to previous 1.3.2.1 (colored) to branchpoint 1.3 (colored)

resync with head

Revision 1.7 / (download) - annotate - [select for diffs], Sun Feb 10 23:58:27 2013 UTC (4 years, 10 months ago) by christos
Branch: MAIN
CVS Tags: agc-symver-base, agc-symver
Changes since 1.6: +3 -1 lines
Diff to previous 1.6 (colored)

add -r flag to elide tty formatting

Revision 1.6 / (download) - annotate - [select for diffs], Sun Feb 10 23:24:18 2013 UTC (4 years, 10 months ago) by christos
Branch: MAIN
Changes since 1.5: +2 -2 lines
Diff to previous 1.5 (colored)

remove trailing whitespace

Revision 1.2.4.5 / (download) - annotate - [select for diffs], Wed Jan 23 00:06:43 2013 UTC (4 years, 10 months ago) by yamt
Branch: yamt-pagecache
CVS Tags: yamt-pagecache-tag8
Changes since 1.2.4.4: +2 -1 lines
Diff to previous 1.2.4.4 (colored) to branchpoint 1.2 (colored)

sync with head

Revision 1.5 / (download) - annotate - [select for diffs], Mon Jan 14 21:26:25 2013 UTC (4 years, 11 months ago) by christos
Branch: MAIN
CVS Tags: yamt-pagecache-base8
Changes since 1.4: +2 -1 lines
Diff to previous 1.4 (colored)

- move the terminal handling in apropos-utils.c since htmp and pager are also
  handled there.
- underline the name, section, and description so that it is prettier.
- change to bold terminal the terminal highlighting to match with less

Revision 1.3.2.1 / (download) - annotate - [select for diffs], Tue Nov 20 03:03:02 2012 UTC (5 years ago) by tls
Branch: tls-maxphys
Changes since 1.3: +4 -3 lines
Diff to previous 1.3 (colored)

Resync to 2012-11-19 00:00:00 UTC

Revision 1.2.4.4 / (download) - annotate - [select for diffs], Tue Oct 30 19:00:38 2012 UTC (5 years, 1 month ago) by yamt
Branch: yamt-pagecache
Changes since 1.2.4.3: +4 -3 lines
Diff to previous 1.2.4.3 (colored) to branchpoint 1.2 (colored)

sync with head

Revision 1.4 / (download) - annotate - [select for diffs], Sat Oct 6 15:33:59 2012 UTC (5 years, 2 months ago) by wiz
Branch: MAIN
CVS Tags: yamt-pagecache-base7, yamt-pagecache-base6
Changes since 1.3: +4 -3 lines
Diff to previous 1.3 (colored)

Make mandb path configurable. makemandb (and related tools) use
the path from the _mandb variable from man.conf now.

Set _mandb in man.conf to same value as was used before.

From Abhinav Upadhyay <er.abhinav.upadhyay@gmail.com>.

Revision 1.2.4.3 / (download) - annotate - [select for diffs], Wed May 23 10:08:29 2012 UTC (5 years, 6 months ago) by yamt
Branch: yamt-pagecache
Changes since 1.2.4.2: +2 -2 lines
Diff to previous 1.2.4.2 (colored) to branchpoint 1.2 (colored)

sync with head.

Revision 1.2.2.1 / (download) - annotate - [select for diffs], Wed May 9 03:41:00 2012 UTC (5 years, 7 months ago) by riz
Branch: netbsd-6
CVS Tags: netbsd-6-1-RELEASE, netbsd-6-1-RC4, netbsd-6-1-RC3, netbsd-6-1-RC2, netbsd-6-1-RC1, netbsd-6-1-5-RELEASE, netbsd-6-1-4-RELEASE, netbsd-6-1-3-RELEASE, netbsd-6-1-2-RELEASE, netbsd-6-1-1-RELEASE, netbsd-6-1, netbsd-6-0-RELEASE, netbsd-6-0-RC2, netbsd-6-0-RC1, netbsd-6-0-6-RELEASE, netbsd-6-0-5-RELEASE, netbsd-6-0-4-RELEASE, netbsd-6-0-3-RELEASE, netbsd-6-0-2-RELEASE, netbsd-6-0-1-RELEASE, netbsd-6-0, matt-nb6-plus-nbase, matt-nb6-plus-base, matt-nb6-plus
Changes since 1.2: +1 -1 lines
Diff to previous 1.2 (colored) next main 1.3 (colored)

Pull up following revision(s) (requested by wiz in ticket #229):
	usr.sbin/makemandb/makemandb.c: revision 1.9
	usr.sbin/makemandb/DBSCHEMA: revision 1.2
	usr.sbin/makemandb/apropos-utils.c: revision 1.5
	usr.sbin/makemandb/apropos-utils.h: revision 1.3
PR 46419 by Abhinav Upadhyay using his updated patch:
Clean up after removing man page aliases.

Revision 1.3 / (download) - annotate - [select for diffs], Mon May 7 11:18:16 2012 UTC (5 years, 7 months ago) by wiz
Branch: MAIN
CVS Tags: yamt-pagecache-base5
Branch point for: tls-maxphys
Changes since 1.2: +2 -2 lines
Diff to previous 1.2 (colored)

PR 46419 by Abhinav Upadhyay using his updated patch:
Clean up after removing man page aliases.

Revision 1.2.4.2 / (download) - annotate - [select for diffs], Tue Apr 17 00:09:49 2012 UTC (5 years, 8 months ago) by yamt
Branch: yamt-pagecache
Changes since 1.2.4.1: +92 -0 lines
Diff to previous 1.2.4.1 (colored) to branchpoint 1.2 (colored)

sync with head

Revision 1.2.4.1, Tue Feb 7 19:17:16 2012 UTC (5 years, 10 months ago) by yamt
Branch: yamt-pagecache
Changes since 1.2: +0 -92 lines
FILE REMOVED

file apropos-utils.h was added on branch yamt-pagecache on 2012-04-17 00:09:49 +0000

Revision 1.2 / (download) - annotate - [select for diffs], Tue Feb 7 19:17:16 2012 UTC (5 years, 10 months ago) by joerg
Branch: MAIN
CVS Tags: yamt-pagecache-base4, netbsd-6-base
Branch point for: yamt-pagecache, netbsd-6
Changes since 1.1: +1 -1 lines
Diff to previous 1.1 (colored)

Fix C&P error with $NetBSD$

Revision 1.1 / (download) - annotate - [select for diffs], Tue Feb 7 19:13:32 2012 UTC (5 years, 10 months ago) by joerg
Branch: MAIN

Import the new apropos/whatis.

This code has been developed by Abhinav Upadhyay as part of Google's Summer
of Code 2011. It uses libmandoc to parse man pages and builds a Full
Text Index in a SQLite database. The combination of indexing the full
manual page, filtering out stop words and ranking individual matches
based on the section gives a much improved user experience.

The old makewhatis and friends are kept under MKMAKEMANDB=no for now.

This form allows you to request diff's between any two revisions of a file. You may select a symbolic revision name using the selection box or you may type in a numeric name using the type-in text box.




CVSweb <webmaster@jp.NetBSD.org>