![[BACK]](/icons/back.gif){kind=icon}
Up to [cvs.NetBSD.org] / pkgsrc / devel / kyua-cli
Request diff between arbitrary revisions
Default branch: MAIN
Revision 1.23, Thu Aug 14 14:14:52 2014 UTC (9 years, 8 months ago) by jmmv
Branch: MAIN
CVS Tags: HEAD
Changes since 1.22: +1 -1
lines
FILE REMOVED
Import kyua-0.10. This is a "new" import that unifies all the previous Kyua subpackages (kyua-atf-compat, kyua-cli and kyua-testers) under a single package. The change has been made upstream mainly for simplicity reasons; the old modular scheme was unnecessary. Given that this new import should be really considered to be an update, the following are the changes since kyua-cli-0.9 and kyua-testers-0.3, both released last week: Experimental version released on August 14th, 2014. * Merged 'kyua-cli' and 'kyua-testers' into a single 'kyua' package. * Dropped the 'kyua-atf-compat' package. * Issue 100: Do not try to drop privileges to unprivileged_user when we are already running as an unprivileged user. Doing so is not possible and thus causes spurious test failures when the current user is not root and the current user and unprivileged_user do not match. * Issue 79: Mention kyua.conf(5) in the "see also" section of kyua(1). * Issue 75: Change the rewrite__expected_signal__bad_arg test in testers/atf_result_test to use a different signal value. This is to prevent triggering a core dump that made the test fail in some platforms.
Revision 1.22 / (download) - annotate - [select for diffs], Fri Aug 8 14:30:24 2014 UTC (9 years, 8 months ago) by jmmv
Branch: MAIN
Changes since 1.21: +4 -5
lines
Diff to previous 1.21 (colored)
Update kyua-cli to 0.9: Experimental version released on August 8th, 2014. Major changes: The internal architecture of Kyua to record the results of test suite runs has completely changed in this release. Kyua no longer stores all the different test suite run results as different "actions" within the single store.db database. Instead, Kyua now generates a separate results file inside ~/.kyua/store/ for every test suite run. Due to the complexity involved in the migration process and the little need for it, this is probably going to be the only release where the 'db-migrate' command is able to convert an old store.db file to the new scheme. Changes in more detail: * Added the 'report-junit' command to generate JUnit XML result files. The output has been verified to work within Jenkins. * Switched to results files specific to their corresponding test suite run. The unified store.db file is now gone: 'kyua test' creates a new results file for every invocation under ~/.kyua/store/ and the 'kyua report*' commands are able to locate the latest file for a corresponding test suite automatically. * The 'db-migrate' command takes an old store.db file and generates one results file for every previously-recorded action, later deleting the store.db file. * The '--action' flag has been removed from all commands that accepted it. This has been superseded by the tests results files. * The '--store' flag that many commands took has been renamed to '--results-file' in line with the semantical changes. * The 'db-exec' command no longer creates an empty database when none is found. This command is now intended to run only over existing files.
Revision 1.21 / (download) - annotate - [select for diffs], Sat May 3 13:14:37 2014 UTC (9 years, 11 months ago) by alnsn
Branch: MAIN
CVS Tags: pkgsrc-2014Q2-base,
pkgsrc-2014Q2
Changes since 1.20: +2 -2
lines
Diff to previous 1.20 (colored)
Revbump for Lua multiversion support.
Revision 1.20 / (download) - annotate - [select for diffs], Wed Apr 9 07:27:06 2014 UTC (10 years ago) by obache
Branch: MAIN
Changes since 1.19: +2 -2
lines
Diff to previous 1.19 (colored)
recursive bump from icu shlib major bump.
Revision 1.19 / (download) - annotate - [select for diffs], Sat Feb 8 15:59:30 2014 UTC (10 years, 2 months ago) by jmmv
Branch: MAIN
CVS Tags: pkgsrc-2014Q1-base,
pkgsrc-2014Q1
Changes since 1.18: +3 -2
lines
Diff to previous 1.18 (colored)
Depend on atf 0.20 (not atf-libs) and bump PKGREVISION.
Revision 1.18 / (download) - annotate - [select for diffs], Sun Dec 8 01:21:59 2013 UTC (10 years, 4 months ago) by jmmv
Branch: MAIN
CVS Tags: pkgsrc-2013Q4-base,
pkgsrc-2013Q4
Changes since 1.17: +2 -3
lines
Diff to previous 1.17 (colored)
Update to 0.8: Experimental version released on December 7th, 2013. * Added support for Lutok 0.4. * Issue 24: Plug the bootstrap tests back into the test suite. Fixes in kyua-testers-0.2 to isolate test cases into their own sessions should allow these to run fine. * Issue 74: Changed the kyuafile(5) parser to automatically discover existing tester interfaces. The various *_test_program() functions will now exist (or not) based on tester availability, which simplifies the addition of new testers or the selective installation of them.
Revision 1.17 / (download) - annotate - [select for diffs], Wed Oct 30 06:49:53 2013 UTC (10 years, 5 months ago) by dholland
Branch: MAIN
Changes since 1.16: +2 -1
lines
Diff to previous 1.16 (colored)
Bump PKGREVISION of packages whose Lua depends changed form, but whose own PKGNAME is unchanged.
Revision 1.16 / (download) - annotate - [select for diffs], Fri Oct 18 21:02:02 2013 UTC (10 years, 6 months ago) by jmmv
Branch: MAIN
Changes since 1.15: +2 -3
lines
Diff to previous 1.15 (colored)
Update to 0.7: Experimental version released on October 18th, 2013. * Made failures from testers more resilent. If a tester fails, the corresponding test case will be marked as broken instead of causing kyua to exit. * Added the '--results-filter' option to the 'report-html' command and set its default value to skip passed results from HTML reports. This is to keep these reports more succint and to avoid generating tons of detail files that will be, in general, useless. * Switched to use Lutok 0.3 to gain compatibility with Lua 5.2. * Issue 69: Cope with the lack of AM_PROG_AR in configure.ac, which first appeared in Automake 1.11.2. Fixes a problem in Ubuntu 10.04 LTS, which appears stuck in 1.11.1.
Revision 1.15 / (download) - annotate - [select for diffs], Thu Jul 4 21:27:56 2013 UTC (10 years, 9 months ago) by adam
Branch: MAIN
CVS Tags: pkgsrc-2013Q3-base,
pkgsrc-2013Q3
Changes since 1.14: +2 -2
lines
Diff to previous 1.14 (colored)
Revbump after updating lang/lua to 5.2.2.
Revision 1.14 / (download) - annotate - [select for diffs], Sat Jun 15 00:31:34 2013 UTC (10 years, 10 months ago) by jmmv
Branch: MAIN
CVS Tags: pkgsrc-2013Q2-base,
pkgsrc-2013Q2
Changes since 1.13: +2 -3
lines
Diff to previous 1.13 (colored)
Switch to use Lutok 0.3. Bump PKGREVISION to 2.
Revision 1.13 / (download) - annotate - [select for diffs], Thu May 9 07:39:36 2013 UTC (10 years, 11 months ago) by adam
Branch: MAIN
Changes since 1.12: +2 -1
lines
Diff to previous 1.12 (colored)
Massive revbump after updating graphics/ilmbase, graphics/openexr, textproc/icu.
Revision 1.12 / (download) - annotate - [select for diffs], Sat Feb 23 03:37:40 2013 UTC (11 years, 1 month ago) by jmmv
Branch: MAIN
CVS Tags: pkgsrc-2013Q1-base,
pkgsrc-2013Q1
Changes since 1.11: +4 -5
lines
Diff to previous 1.11 (colored)
Update to 0.6: Experimental version released on February 22nd, 2013. * Issue 36: Changed 'kyua help' to not fail when the configuration file is bogus. Help should always work. * Issue 37: Simplified the syntax() calls in configuration and Kyuafile files to only specify the requested version instead of also the format name. The format name is implied by the file being loaded, so there is no use in the caller having to specify it. The version number of these file formats has been bumped to 2. * Issue 39: Added per-test-case metadata values to the HTML reports. * Issue 40: Rewrote the documentation as manual pages and removed the previous GNU Info document. * Issue 47: Started using the independent testers in the kyua-testers package to run the test cases. Kyua does not implement the logic to invoke test cases any more, which provides for better modularity, extensibility and robustness. * Issue 57: Added support to specify arbitrary metadata properties for test programs right from the Kyuafile. This is to make plain test programs more versatile, by allowing them to specify any of the requirements (allowed architectures, required files, etc.) supported by Kyua. * Reduced automatic screen line wrapping of messages to the 'help' command and the output of tables by 'db-exec'. Wrapping any other messages (specially anything going to stderr) was very annoying because it prevented natural copy/pasting of text. * Increased the granularity of the error codes returned by kyua(1) to denote different error conditions. This avoids the overload of '1' to indicate both "expected" errors from specific subcommands and unexpected errors caused by the internals of the code. The manual now correctly explain how the exit codes behave on a command basis. * Optimized the database schema to make report generation almost instantaneous. * Bumped the database schema to 2. The database now records the metadata of both test programs and test cases generically, without knowledge of their interface. * Added the 'db-migrate' command to provide a mechanism to upgrade a database with an old schema to the current schema. * Removed the GDB build-time configuration variable. This is now part of the kyua-testers package. * Rewrote the Kyuafile parsing code in C++, which results in a much simpler implementation. As a side-effect, this gets rid of the external Lua files required by 'kyua', which in turn make the tool self-contained. * Added caching of various configure test results (particularly in those tests that need to execute a test program) so that cross-compilers can predefine the results of the tests without having to run the executables.
Revision 1.11 / (download) - annotate - [select for diffs], Sat Jan 26 21:37:22 2013 UTC (11 years, 2 months ago) by adam
Branch: MAIN
Changes since 1.10: +2 -1
lines
Diff to previous 1.10 (colored)
Revbump after graphics/jpeg and textproc/icu
Revision 1.10 / (download) - annotate - [select for diffs], Wed Oct 31 11:17:09 2012 UTC (11 years, 5 months ago) by asau
Branch: MAIN
CVS Tags: pkgsrc-2012Q4-base,
pkgsrc-2012Q4
Changes since 1.9: +1 -3
lines
Diff to previous 1.9 (colored)
Drop superfluous PKG_DESTDIR_SUPPORT, "user-destdir" is default these days.
Revision 1.9 / (download) - annotate - [select for diffs], Thu Jul 19 18:30:18 2012 UTC (11 years, 9 months ago) by jmmv
Branch: MAIN
CVS Tags: pkgsrc-2012Q3-base,
pkgsrc-2012Q3
Changes since 1.8: +2 -2
lines
Diff to previous 1.8 (colored)
These packages are all modified-bsd, not 2-clause-bsd.
Revision 1.8 / (download) - annotate - [select for diffs], Tue Jul 10 22:09:32 2012 UTC (11 years, 9 months ago) by jmmv
Branch: MAIN
Changes since 1.7: +2 -4
lines
Diff to previous 1.7 (colored)
Update to 0.5: Experimental version released on July 10th, 2012. * Issue 15: Added automatic stacktrace gathering of crashing test cases. This relies on GDB and is a best-effort operation. * Issue 32: Added the '--build-root' option to the debug, list and test commands. This allows executing test programs from a different directory than where the Kyuafile scripts live. See the 'Build roots' section in the manual for more details. * Issue 33: Removed the kyuaify.sh script. This has been renamed to atf2kyua and moved to the kyua-atf-compat module, where it ships as a first-class utility (with a manual page and tests). * Issue 34: Changed the HTML reports to include the stdout and stderr of every test case. * Fixed the build when using a "build directory" and a clean source tree from the repository.
Revision 1.7 / (download) - annotate - [select for diffs], Sun Jul 1 15:29:08 2012 UTC (11 years, 9 months ago) by jmmv
Branch: MAIN
Changes since 1.6: +4 -3
lines
Diff to previous 1.6 (colored)
Depend on atf-libs instead of atf. Bump PKGREVISION.
Revision 1.6 / (download) - annotate - [select for diffs], Thu Jun 7 01:13:58 2012 UTC (11 years, 10 months ago) by jmmv
Branch: MAIN
CVS Tags: pkgsrc-2012Q2-base,
pkgsrc-2012Q2
Changes since 1.5: +20 -4
lines
Diff to previous 1.5 (colored)
Update to 0.4. Other than the upgrade itself, this also adds a new 'tests' option to the package to indicate whether the Kyua-specific tests should be built or not. I do not recommend disabling their build, but not specifiying the 'tests' option cuts down the build time significantly. The major changes since 0.3 are: * Added the 'report-html' command to generate HTML reports of the execution of any recorded action. * Changed the '--output' flag of the 'report' command to only take a path to the target file, not its format. Different formats are better supported by implementing different subcommands, as the options they may receive will vary from format to format. * Added a '--with-atf' flag to the configure script to control whether the ATF tests get built or not. May be useful for packaging systems that do not have ATF in them yet. Disabling ATF also cuts down the build time of Kyua significantly, but with the obvious drawbacks. * Grouped 'kyua' subcommands by topic both in the output of 'help' and in the documentation. In general, the user needs to be aware of commands that rely on a current project and those commands that rely purely on the database to generate reports. * Made 'help' print the descriptions of options and commands properly tabulated. * Changed most informational messages to automatically wrap on screen boundaries. * Rewrote the configuration file parsing module for extensibility. This will allow future versions of Kyua to provide additional user-facing options in the configuration file. No syntax changes have been made, so existing configuration files (version 1) will continue to be parsed without problems. There is one little exception though: all variables under the top-level 'test_suites' tree must be declared as strings. Similarly, the '-v' and '--variable' flags to the command line must now carry a 'test_suites.' prefix when referencing any variables under such tree.
Revision 1.5 / (download) - annotate - [select for diffs], Fri Apr 27 12:31:42 2012 UTC (11 years, 11 months ago) by obache
Branch: MAIN
Changes since 1.4: +2 -1
lines
Diff to previous 1.4 (colored)
Recursive bump from icu shlib major bumped to 49.
Revision 1.4 / (download) - annotate - [select for diffs], Fri Feb 24 21:07:41 2012 UTC (12 years, 1 month ago) by jmmv
Branch: MAIN
CVS Tags: pkgsrc-2012Q1-base,
pkgsrc-2012Q1
Changes since 1.3: +4 -3
lines
Diff to previous 1.3 (colored)
Update kyua-cli to 0.3: Experimental version released on February 24th, 2012. * Made the 'test' command record the results of the executed test cases into a SQLite database. As a side effect, 'test' now supports a '--store' option to indicate where the database lives. * Added the 'report' command to generate plain-text reports of the test results stored in the database. The interface of this command is certainly subject to change at this point. * Added the 'db-exec' command to directly interact with the store database. * Issue 28: Added support for the 'require.memory' test case property introduced in ATF 0.15. * Renamed the user-specific configuration file from ~/.kyuarc to ~/.kyua/kyua.conf for consistency with other files stored in the ~/.kyua/ subdirectory. * Switched to use Lutok instead of our own wrappers over the Lua C library. Lutok is just what used to be our own utils::lua module, but is now distributed separately. * Removed the 'Atffile's from the source tree. Kyua is stable enough to generate trustworthy reports, and we do not want to give the impression that atf-run / atf-report are still supported. * Enabled logging to stderr for our own test programs. This makes it slightly easier to debug problems in our own code when we get a failing test.
Revision 1.3 / (download) - annotate - [select for diffs], Thu Sep 29 19:15:09 2011 UTC (12 years, 6 months ago) by jmmv
Branch: MAIN
CVS Tags: pkgsrc-2011Q4-base,
pkgsrc-2011Q4,
pkgsrc-2011Q3-base,
pkgsrc-2011Q3
Changes since 1.2: +2 -1
lines
Diff to previous 1.2 (colored)
Add a NetBSD-specific message pointing to the instructions that describe how to run the NetBSD test suite using Kyua.
Revision 1.2 / (download) - annotate - [select for diffs], Wed Aug 24 21:04:42 2011 UTC (12 years, 7 months ago) by jmmv
Branch: MAIN
Changes since 1.1: +2 -2
lines
Diff to previous 1.1 (colored)
Update to 0.2: Experimental version released on August 24th, 2011. The biggest change in this release is the ability for Kyua to run test programs implemented using different frameworks. What this means is that, now, a Kyua test suite can include not only ATF-based test programs, but also "legacy" (aka plain) test programs that do not use any framework. I.e. if you have tests that are simple programs that exit with 0 on success and 1 on failure, you can plug them in into a Kyua test suite. Other than this, there have been several user-visible changes. The most important are the addition of the new 'config' and 'debug' subcommands to the 'kyua' binary. The former can be used to inspect the runtime configuration of Kyua after parsing, and the latter is useful to interact with failing tests cases in order to get more data about the failure itself. Without further ado, here comes the itemized list of changes: * Generalized the run-time engine to support executing test programs that implement different interfaces. Test programs that use the ATF libraries are just a special case of this. (Issue 18.) * Added support to the engine to run "plain" test programs: i.e. test programs that do not use any framework and report their pass/fail status as an exit code. This is to simplify the integration of legacy test programs into a test suite, and also to demonstrate that the run-time engine is generic enough to support different test interfaces. (Issue 18.) * Added the 'debug' subcommand. This command allows end users to tweak the execution of a specific test case and to poke into the behavior of its execution. At the moment, all this command allows is to view the stdout and stderr of the command in real time (which the 'test' command currently completely hides). * Added the 'config' subcommand. This command allows the end user to inspect the current configuration variables after evaluation, without having to read through configuration files. (Issue 11.) * Removed the test_suites_var function from configuration files. This was used to set the value of test-suite-sepecific variables, but it was ugly-looking. It is now possible to use the more natural syntax 'test_suites.<test-suite-name>.<variable> = <value>'. (Issue 11.) * Added a mechanism to disable the loading of configuration files altogether. Needed for testing purposes and for scriptability. Available by passing the '--config=none' flag. * Enabled detection of unused parameters and variables in the code and fixed all warnings. (Issue 23.) * Changed the behavior of "developer mode". Compiler warnings are now enabled unconditionally regardless of whether we are in developer mode or not; developer mode is now only used to perform strict warning checks and to enable assertions. Additionally, developer mode is now only automatically enabled when building from the repository, not for formal releases. (Issue 22.) * Fixed many build and portability problems to Debian sid with GCC 4.6.3 and Ubuntu 10.04.1 LTS. (Issues 20, 21, 26.)
Revision 1.1.1.1 / (download) - annotate - [select for diffs] (vendor branch), Fri Jul 8 09:49:11 2011 UTC (12 years, 9 months ago) by jmmv
Branch: TNF
CVS Tags: pkgsrc-base
Changes since 1.1: +0 -0
lines
Diff to previous 1.1 (colored)
Initial import of kyua-cli, version 0.1: Kyua (pronounced Q.A.) is a testing framework for both developers and users. Kyua is different from most other testing frameworks in that it puts the end user experience before anything else. There are multiple reasons for users to run the tests themselves, and Kyua ensures that they can do so in the most convenient way. At the moment, Kyua is focused on implementing a solid foundation and a powerful command-line tool to run tests implemented with the Automated Testing Framework (ATF). Later on, Kyua will also provide a set of language bindings (C, C++ and shell, at the least) to ease the implementation of test cases in a variety of programming languages. In effect, Kyua is intended to be a replacement for ATF.
Revision 1.1 / (download) - annotate - [select for diffs], Fri Jul 8 09:49:11 2011 UTC (12 years, 9 months ago) by jmmv
Branch: MAIN
Initial revision