[LLVMdev] Testing documentation and terminology
sabre at nondot.org
Fri May 23 19:12:53 CDT 2008
On May 23, 2008, at 5:06 AM, Matthijs Kooijman wrote:
> Hi all,
> as you might have seen, I just did some restructuring on the testing
> documentation. Since the changes are only documentation and mostly
> movement, I took the liberty of committing without posting a patch
> first. Was
> that appropriate?.
Yep, if you consider it to be obvious, it can be committed without
> The incentive for this commit was that I've spent the last week
> trying to get
> a grip on the test suite with moderate success. I think this
> will help new users to get started with the test suite faster. Or,
> directly, I wanted to add some documentation on comparing test
> results, but
> couldn't find a decent spot in the testing documentation structure.
Awesome, we welcome improvements, obviously there are many to make
> There is still one issue I'd like to discuss here. The test suite
> documentation is a bit confusing in terminology. There are three
> concepts: All of the testing stuff in llvm, the llvm/test DejaGNU
> part, and
> the test-suite part.
> Currently, the first is called the "LLVM test suite" (which is the
> subject of
> the testing documentation). The second is called the DejaGNU tests,
> fragment tests or llvm/test. The last one is the most confusing,
> it's called
> the whole program tests, test-suite (named after the svn directory
> it is in,
> easily confused with the entire test suite), llvm-test (named after
> directory in which you should place it, easily confused with llvm/
llvm-test was the name of the CVS module. When we moved to SVN, it
got renamed to test-suite, which I consider to be very confusing. We
should probably recommend that people check it out into test-suite,
and refer to it as that. Alternatively, change the svn module name to
something more useful.
> Furthermore, I would no longer refer to all of llvm's testing stuff
> as the
> "LLVM test suite", but use the term "LLVM test architecture" or
> "framework" or
> something similar (suggestions?).
LLVM testing infrastructure?
> Since the testing stuff is composed of two
> distinct parts, I'm not sure the term "suite" is appropriate here.
> If people
> think that "test suite" is still very much appropriate here, I think
> renaming the "test-suite" svn module would be in order.
Right. It is somewhat complicated by the fact that "llvm-test" is
really just a collection of programs with some makefile drivers. How
about calling it "program-tests" or "llvm-program-tests" or something
> Lastly, I would refer to the "whole program tests" as the "Test
> suite", since
> it is a suite of testing programs that can be run as a whole. This
> will also
> make sure that the name properly matches the svn module that the
> program tests" are housing in.
Yeah, we definitely need a better name :)
> After these changes, I think the testing framework should be a lot
> easier to
> describe and document and more accessible to new users.
Sounds great to me,
More information about the LLVMdev