The commercial
Parasoft
Jtest
Java testing tool won numerous awards and has been
adopted by
thousands of development teams worldwide. Its clients
include IBM, HP,
and over 10,000 companies worldwide. On 26 March 2002, Parasoft
Co.
announced the commercial
Jtest
4.5 Java testing tool, which has been popularly used
in
industry. Jtest 4.5 allows the user to specify
method-call
lengths of generated tests from one up to three. When
setting
method-call lengths as three, the test generation time is
long and the
number of generated tests is large for relatively large
programs. On
January 2004, we proposed a novel definition of
redundant
tests based on method inputs (different from
traditional
definitions of redundant tests based on structural coverage)
and
published our preliminary results (in a technical report) on
detecting
a high percentage (about 90%) of redundant tests among
tests
generated by Jtest
4.5 and later published the more comprehensive results
in our
ASE 04 paper; these identified redundant tests increase
the
testing time without increasing the ability to detect
faults or
increase confidence. Parasoft
announced Jtest
5.0 on 3 February 2004 and Jtest
5.1 on 28 June 2004; neither of these new versions
allows the
user to configure the method-call lengths for generating a
large number
of test inputs any more. We expect that the new techniques
in our ASE
04 paper can be ultimately incorporated by the existing test
generation
tools to improve testing performance greatly, benefiting
software
practitioners.
Updates: Mar 17, 2005:
Parasoft Jtest 5.1 won Software Development Magazine's 15th
Annual
Productivity Award.
Nov 5, 2004: Upon Parasoft's
invitation, Tao Xie visited
Parasoft and gave a presentation on "Strategic
Automated Software Testing in the Absence of
Specifications".
Sept 16, 2004: We were notified by Parasoft
Co. that
Parasoft Jtest 6.0 (internal version, not released yet) has
fixed the
test redundancy issue identified by us and added back the
option to
generate long method-call sequences.]
Tao Xie, Darko Marinov, and David Notkin. Rostra: A
Framework
for Detecting Redundant Object-Oriented Unit Tests.
In
Proceedings of the 19th
IEEE International Conference on
Automated Software Engineering (ASE
2004),
Linz, Austria, pp. 196-205, September 2004. [PDF][BibTex]
©2004 IEEE
Within
our knowledge, our ASE 03 paper
is the first proposed approach of using inferred
program semantic properties (other than structural
coverage)
for helping test generation and test selection for
inspection.
Automated test generation tools are able to produce and
execute a large
number of test inputs that extensively exercise the unit
under test.
However, without a priori specifications, developers need to
manually
verify the outputs of these test executions, which is
generally
impractical. To reduce this cost, we developed unit test
selection
techniques to select a valuable subset of automatically
generated test
inputs. Then developers can verify their outputs, equip them
with test
oracles, and put them into the existing test suite. In
particular, we
feed dynamically inferred program behaviors in the form of
specifications (e.g., the ones inferred by Daikon)
to
a specification-based test generation tool (e.g., Parasoft
Jtest) so that some benefits of specification-based
testing
can be achieved without requiring specifications. On 2 March
2004, Agitar Software Inc. announced
the
commercial Agitator
testing tool for Java. The tool automatically generates
initial tests,
infers Daikon-invariant-like
observations,
lets developers confirm these observations to assertions,
and generates more tests to violate these
inferred&confirmed
observations.
Updates: July 17, 2006: Agitar
people published an ISSTA
06
paper on Agitator.
Feb
7, 2006: Agitar
CTO Alberto Savoia gave a
guest lecture at Tao Xie's CSC
591T
course at North Carolina State University.
April
7, 2005: Agitar CTO Alberto Savoia
gave a guest
lecture at UC Berkeley on Unit/Developer Testing,
describing
Agitator's philosophy.
Mar
17, 2005: Agitar’s Agitator and
Dashboard 2.0 won Software Development Magazine's 15th
Annual
Jolt Product Excellence Award.
Nov
17, 2004: Kent Beck gave a presentation on
"Developer Testing" at Developer
Testing Forum. Listen
to
it!
Aug 19, 2004: Agitar CTO
Alberto Savoia presented “Beyond
JUnit:
The Future of Developer Testing for Java” at
JavaOne 2004. He attracted a standing-room only crowd of
more than 600
interested developers.
Aug
2, 2004: Kent
Beck, the creator of JUnit and pioneer of eXtreme
programming
joined Agitar, becoming the first Agitar
Software
Fellow.
July 23, 2004: Upon Agitar's
invitation, Tao Xie visited
Agitar and gave a presentation on "Automated Software
Testing with
Inferred Program Properties".
June
29, 2004: Agitar’s Agitator won 2004
Duke’s
Choice Award at JavaOne Developer Conference.
Dec 6, 2003: The
tools in our ASE 03 paper were highlighted in the testdriven.com
newsletter
#1.
Tao Xie and David Notkin. Tool-Assisted Unit Test
Selection
Based on Operational
Violations. In Proceedings of the 18th IEEE
International Conference on Automated Software Engineering
(ASE
2003),
Montreal, Canada, pp. 40-48, Oct. 2003. [PDF][BibTex][Slides]
(Nominated
for
the Best Paper Award) ©2003
IEEE.
An extended version to appear in a special issue of Automated
Software
Engineering Journal. [PDF][BibTex]
Generating
tests
for
an object-oriented program typically involves
two tasks: generating relevant receiver-object states and
generating
relevant method arguments. Existing specification-based test
generation
tools such as Korat
and TestEra
rely on specifications such as class invariants to determine
whether
their directly constructed receiver-object states are valid.
However,
specifications often don't exist in practice. In our TACAS
05 paper, we
have developed new techniques and a supporting tool, called
Symstra,
for indirectly constructing valid and relevant
receiver-object states
through method sequences, without requiring any
specifications. To
address the issues of state explosion and relevant-argument
generation,
we have defined symbolic states of object-oriented programs
(produced
by symbolic execution) and novel comparisons of symbolic
states.
Experimental results showed that the tool generates tests
that achieve
higher structural coverage faster than the existing tools.
The Automated Software
Engineering group at NASA Ames Research Center
implemented a
modified version of our Symstra techniques on top of Java
PathFinder,
a model checker and test-input generator for Java, and found
the
techniques to be effective in Java Pathfinder’s
test-input generation. The Foundations
of
Software Engineering group at Microsoft Research
has also implemented the Symstra techniques in their test-generation
tools for C#.
Tao
Xie, Darko Marinov, Wolfram
Schulte, and David Notkin. Symstra:
A
Framework for
Generating Object-Oriented Unit Tests using Symbolic
Execution.
In Proceedings
of
the 11th International Conference on Tools
and Algorithms for the Construction and Analysis of
Systems
(TACAS
2005),
Edinburgh,
U.K., pp. 365-381, April 2005. [PDF][BibTex]
How
is
our research work related to software industry?
How is our research
work related to software industry?