Contents:
Java annotations are meta-data about Java program elements, as in
“@Deprecated class Date
{ ... }
” or “List<@NonNull String>
”.
Ordinarily, Java annotations are
written in the
source code of a .java
Java source file. When
javac
compiles the source code, it inserts the annotations in
the resulting .class
file (as
“attributes”).
Sometimes, it is convenient to specify
the annotations outside the source code or the .class
file.
The document
“Annotation File Format Specification” (PDF, HTML)
defines a textual format for annotations,
and it also motivates reasons why such a file format is necessary in
addition to the .java
and .class
formats. The
file format supports both the declaration annotations and type annotations.
An annotation file
conventionally has the extension .jaif
(for Java Annotation Index
File).
The scene-lib
sub-project provides
API methods for building and manipulating annotation files.
Programmers need to be able to transfer annotations between the three possible locations for annotations — source code, class files, and annotation files. Programmers will want to extract annotations from source and class files to an annotation file in order to easily read annotations, while various tools will only read annotations from source and class files. The Annotation File Utilities provide three tools to read and write annotation files.
insert-annotations
reads annotations from an annotation file
and inserts them into a class fileextract-annotations
reads annotations from a class file
and writes them out to an annotation fileinsert-annotations-to-source
reads annotations from an
annotation file and inserts them into a Java source fileA diagram showing the interactions betweeen these tools is below.
There is no
extract-annotations-from-source
tool: one can
compile the source code and then use
extract-annotations
to read the annotations from the class
file.
The following instructions assume either a Linux or Windows system using a command-line environment.
The current release is Annotation File Utilities version 3.6.45, 05 Sep 2017.
annotation-tools
by
unpacking the distribution zipfile (a standard place to do this is in a
directory ~/jsr308/
):
unzip annotation-tools-3.6.45.zip
The annotation-tools
directory must be a sibling of the jsr308-langtools
directory (available at https://checkerframework.org/jsr308/).
Alternatively, Unix (including Linux and MacOS) users may set the LANGTOOLS
environment variable to
the location of the jsr308-langtools
directory:
export LANGTOOLS=/path/to/jsr308-langtools
annotation-file-utilities
directory to your path.
~/.bashrc
or ~/.bash_profile
file:
export PATH=${PATH}:/path/to/annotation-tools/annotation-file-utilities/scripts
PATH
system
variable by going to
Control Panel -> System -> Advanced -> Environment Variables
From there, find the PATH
variable under “System variables”
and append to it the directory path\to\annotatation-tools\annotation-file-utilities\scripts
.
The annotation file utilities are pre-compiled (a jar file is included in the distribution), so most users do not need to compile it themselves.
There are two ways to obtain the source code. Source code is provided in the distribution. Alternately, see the source code repository at https://github.com/typetools/annotation-tools.
To compile, run
ant jarfile
from the annotation-file-utilities
subdirectory. If the javac
and java
programs are
not on your execution path, see the notes near the top of
annotation-file-utilities/tests/Makefile
.
To use the tools, simply run them from the command-line with the
appropriate arguments. The following instructions are for running the
tools on a Linux/Unix/MacOS machine.
The tools work identically on Windows, except
the extension .bat
needs to be appended to the tool name (for
example, Windows users would execute insert-annotations.bat
instead of insert-annotations
).
For all the tools, arguments starting with a single
‘@
’ are recognized as argument files
(argfiles
), the contents of which get expanded into the
command line. (Initial @@
represents a literal
@
in the argument text.) For additional details of argfile
processing, refer to Oracle's
javac
documentation.
To insert annotations specified by an annotation file into a class file, use the insert-annotations tool. Running:
insert-annotations mypackage.MyClass indexFile.jaif
will read in all the annotations from the annotation file
indexFile.jaif
and insert those annotations pertaining to
mypackage.myClass
into the class file for
mypackage.MyClass
, outputting the final class file to
mypackage.MyClass.class
in the present working directory.
Note that the class file for mypackage.MyClass
must be located
on your classpath.
Multiple pairs of class and index files (in that order) can be specified on a single command line; if the program exits normally, the results are the same as if the program were run once for each pair of arguments in sequence. Run:
insert-annotations --help
for full usage information.
To extract annotations from a class file and write them to an annotation file, use the extract-annotations tool. Running:
extract-annotations mypackage.MyClass
will locate the class file for mypackage.MyClass
, read all
annotations from it, and write the results in annotation file format to
mypackage.MyClass.jaif
in the present working directory. Note
that mypackage.MyClass
must be located on your classpath.
Multiple classes can be specified on a single command line; if the program exits normally, the results are the same as if the program were run once for each class in sequence. Run:
extract-annotations --help
for full usage information.
To insert annotations specified by an annotation file into a Java source file, use the insert-annotations-to-source tool. Running:
insert-annotations-to-source index1.jaif index2.jaif mypackage/MyClass.java yourpackage/YourClass.java
will read all the annotations from index1.jaif
and
index2.jaif
, insert them (when applicable) into their
appropriate locations in mypackage/MyClass.java
and
yourpackage/YourClass.java
, and write the results to
annotated/mypackage/MyClass.java
and
annotated/mypackage/MyClass.java
, respectively.
Index and source files can be specified in any order, mixing the two file types freely; if the source files have no overlapping definitions and the program exits normally, the results are the same as if the program were run once for each source file, with all JAIFs given for each run. Run:
insert-annotations-to-source --help
for full usage information.
If you wish to insert annotations into method bodies, you must have the
associated class mypackage.MyClass.class
on your classpath.
You can insert annotations on class/field/method declarations and
signatures without the class on your classpath.
In general (but see below for exceptions), your source code needs to
contain the locations that your
.jaif
file references. In particular, if the
.jaif
file contains annotations for a type parameter, but the
source code uses a raw type, then you will get an error such as
Found class Edge, but unable to insert @checkers.nullness.quals.Nullable: @checkers.nullness.quals.Nullable (nl=true) @ [GenericArrayLocationCriterion at ( [TYPE_ARGUMENT(0)] ), ...
In this case, you should add type arguments, such as changing
public void pushNonezeroRing(Stack stack, Hashtable seen) {
to
public void pushNonezeroRing(Stack<Edge> stack, Hashtable<Edge, ?> seen) {
In a few cases, insert-annotations-to-source will generate "default" code to provide a location for an annotation. These include
extends Object
);{"a", "b"}
becomes new String[] {"a", "b"}
).The command-line options are as follows:
Note that if the user runs the annotator with --in-place, makes edits, and then re-runs the annotator with this --in-place option, those edits are lost. Similarly, if the user runs the annotator twice in a row with --in-place, only the last set of annotations will appear in the codebase at the end.
To preserve changes when using the --in-place option, first remove the
backup files. Or, use the -d .
option, which makes (and
reads) no backup, instead of --in-place. [default false]
This section describes some high level-design and implementation details of the Annotation File Utilities, including the different components of the Annotation File Utilities and how they fit together. It is intended for someone who is beginning work on the Annotation File Utilities or is curious about how the Annotation File Utilities work.
The Annotation File Utilities is composed of three sub-projects:
asmx
, scene-lib
, and
annotation-file-utilities
. The asmx
sub-project provides methods for reading and writing class files. The
scene-lib
sub-project represents a .jaif
file
and inserts and extracts annotations to/from bytecode. The
annotation-file-utilities
sub-project inserts annotations
into source code.
The asmx
sub-project provides methods for reading and
writing class files. asmx
is based on an old version
(2.2.2) of the ASM Framework. It has been modified to
allow it to read and write JSR 308
annotations in bytecode. However, it is far behind the current ASM
version (5.0), which provides support for type annotations. We should
discard the custom asmx
in the Annotation File Utilities
and adapt the Annotation File Utilities to use the official, supported
version of ASM.
scene-lib
is an interface to a .jaif
file.
It reads in and writes out .jaif
files and provides an
internal representation of a .jaif
file to access and
manipulate.
Internally, a .jaif
file is represented by the
scene-lib/src/annotations/el/AScene.java
class. The
AScene
class (or “annotated scene”) roughly
parallels the root of an abstract syntax tree. An AScene
has a number of classes (AClass
) as children. Each class
has a number of methods (AMethod
), fields
(AElement
), etc. as children. All of these classes are
related in the type hierarchy shown below.
Each class in the type hierarchy has one or more fields to hold
annotations for the different components of the class. For example, the
AMethod
class has the following fields: bounds, return
type, receiver parameters, and throws clause. Each of these fields
holds the annotations stored on that part of the method. For details on
the remainder of the classes in the type hierarchy, and their
respective fields, see the documentation for each file in
scene-lib/src/annotations/el/
.
An AScene
instance can be created in two ways. An empty
AScene
can be created by calling the AScene
constructor, or an AScene
can be created by parsing an
existing .jaif
file. Once an AScene
is
created, annotations can be added to it by adding them to the correct
fields of the children. An AScene
can also be output to
create a new .jaif
file.
Annotations can be inserted into bytecode by executing the
annotation-file-utilities/scripts/insert-annotations
script. This script takes one or more ⟨class name,
.jaif
file⟩ pairs as arguments. The annotations
specified in the .jaif
file are inserted into the
classfile directly before the .jaif
file in the argument
list.
First, each .jaif
file is parsed into an
AScene
(as described in
Scene-lib). Then,
asmx/src/org/objectweb/asm/ClassReader.java
parses the
classfile. As it is parsing the classfile, it passes the parsed
bytecode off to the
scene-lib/src/annotations/io/classfile/ClassAnnotationSceneWriter.java
class. This class has a reference to the AScene
parsed
from the .jaif
file. As this class receives the parsed
bytecode it inserts the relevant annotations from the
AScene
in the bytecode and then writes the bytecode back
out.
Annotations can be extracted from bytecode by executing the
annotation-file-utilities/scripts/extract-annotations
script. This script takes one or more class names as arguments and
outputs the annotations found in those classes to .jaif
files.
First, an empty AScene
is constructed to store
the annotations.
asmx/src/org/objectweb/asm/ClassReader.java
parses the
classfile and passes the parsed bytecode off to the
scene-lib/src/annotations/io/classfile/ClassAnnotationSceneReader.java
class. This class filters out the annotations in the bytecode and adds
them to the correct part of the AScene
. After this, the
AScene
is output to a .jaif
file.
The annotation-file-utilities
sub-project inserts annotations into source
code. It can be run by executing the
annotation-file-utilities/scripts/insert-annotations-to-source
script. The script takes one or more .jaif
files, followed by one or more .java
source files as arguments. The annotations in the .jaif
files are inserted into the .java
source files.
First, an instance of
annotation-file-utilities/src/annotator/specification/IndexFileSpecification.java
is created. Its parse
method parses the .jaif
file into an AScene
(as described in
Scene-lib). The parse
method
calls the parseScene
method, which traverses through the
AScene
and creates an
annotation-file-utilities/src/annotator/specification/CriterionList.java.
A CriterionList
identifies a unique AST node that is the
location of an insertion. It contains objects that implement the
annotation-file-utilities/src/annotator/find/Criterion.java
interface.
Each Criterion
has an isSatisifiedBy
method —
a predicate that takes an AST node and returns true
if the
AST node satisfies the Criterion
and false
otherwise. To determine if a given node matches a
CriterionList
, the node is passed to all of the
Criterion
s in the CriteriaList
. If every
Criterion
returns true
then it is match. If
one or more Criterion
s return false
then it is
not a match. The various Criterion
classes are in the
annotation-file-utilities/src/annotator/find/
directory.
For example, take the following source code:
package afu.example; public class Test { public void m(boolean b, int i) { // ... } }
The CriterionList
to specify the location of the
i
parameter contains the following
Criterion
s:
InPackageCriterion("afu.example")
InClassCriterion("Test")
InMethodCriterion("m(ZI)V")
ParamCriterion(1)
After this CriterionList
is built up an
annotation-file-utilities/src/annotator/find/Insertion.java
is created. An Insertion
stores an
annotation-file-utilities/src/annotator/find/Criteria.java
(which is created from a CriterionList
) and the text to be
inserted. All of these Insertion
s are then added to a
list. The Java compiler then is called to parse the Java source into an
abstract syntax tree. This is followed by a call to the
getPositions
method of
annotation-file-utilities/src/annotator/find/TreeFinder.java
,
which scans through each node of the abstract syntax trees. For each node,
it runs through the Criteria
for each un-matched
Insertion
. If at least one of the Criteria
does not match, then this is not the correct place for the
Insertion
and the Insertion
will be checked
at the remaining nodes of the tree. If all of the Criteria
match, then this node is the correct place for the
Insertion
. It is removed from the list of un-matched
Insertion
s and the position where to insert the
Insertion
is determined. This position is the integer
index in the file where the Insertion
should be inserted.
After the positions are found for all of the Insertion
s,
the Insertion
text is inserted into the file. This happens
backwards, with Insertion
s at the end of the file (i.e.
with higher positions) being inserted first. If Insertion
s
were instead inserted from the beginning of the file then a single
Insertion
would invalidate all of the positions for the
following Insertion
s.
If there are remaining Insertion
s that were not matched
to a node in the abstract syntax tree then an error message is
displayed.
To submit a bug report or request a new feature, use the issue tracker. When reporting a bug, please include exact instructions in how to reproduce it, and please also attach relevant input files. This will let us resolve the issue quickly.
You can also reach the developers at annotation-tools-dev@googlegroups.com. But please use the issue tracker for bug reports and feature requests.
The changelog describes what is new in each release.