Livedoc seamlessly integrates source-code statistics provided by Code Recommenders into the HTML output of Javadoc via a new Doclet and several Providers using the headless API of Code Recommenders 2.0 in an OSGi environment.

Additionally, integration with online services like bugtrackers or mailing lists is planned.

Disclaimer

The Livedoc plugin (respectively the standalone command-line application) is an incubator project developed as part of the Google Summer of Code. Note that it clearly has some rough edges. It’s not production ready yet.

Latest Online Demo

Sample output of Livedoc can be viewed here: http://download.codetrails.com/livedoc/published/latest/

Disclaimer: The online demo may include third party providers that do not come with Livedoc.

Release Planning

Version Release Date Features

0.7

2013-12-21

"Call Also" Provider, Recursive subpackages

0.8

2014-04-07

New Provider API, Package Summary View, "Method Self Call" Provider

0.9

2014-05-14

"Class Self Call" Provider, Better model resolution, Debug mode

0.10

2014-06-11

Diff. source/model artifact coordinates, Google Analytics Provider

0.11

2014-07-09 (←- Current)

JDK 8 Compatibility, DocLint of JDK 8 Javadoc

0.12

(August 2014)

Interconnected web of Javadoc documentation for different APIs

System requirements

Livedoc needs a Java 8 JDK installed and is not compatible to prior versions of Java.

Since the release of version 0.11, Livedoc is Java 8 compatible and makes use of the extended JDK 8 Javadoc API. This allows us to support the DocLint feature of Javadoc (see below) and clears the way for multiple Javadoc runs on whole source repositories with just one command line call. Furthermore, JDK 8 Javadoc fixes some bugs, which were sometimes critical to Javadoc runs.

However, since we cannot afford a simultaneous release plan, Javadoc is not compatible with JDK 7 any more, thus you need an installed JDK 8 (and not just a JRE!) to use Livedoc.

Installation

Livedoc comes as an Eclipse product and can be downloaded from milestones product download site.

Usage Guide

For a quick start, here is an out-of-the-box working command line for Livedoc:

./livedoc org.eclipse.jgit:org.eclipse.jgit:3.4.1.201406201815-r -d ./output -highlight

Livedoc will start generating Javadoc for the org.eclipse.jgit package with annotated Code Recommenders statistics, stored in the folder ./output.

This is possible because the org.eclipse.jgit coordinate above is both legal in the Maven Central repository (http://repo1.maven.org/maven2/) as well as in the Code Recommenders model repository (http://download.eclipse.org/recommenders/models/luna/)

For a usage synopsis, just run ./livedoc without any parameters.

In Detail

Livedoc requires three inputs from you:

  1. A Maven repository for source files (packaging: jar, classifier: sources)

  2. A Maven repository for Code Recommenders models

  3. An artifact coordinate identifying the JAR file you wish to create Livedoc documentation for

Source Repository

The sources JAR files need to be available in a maven. This repository can reside locally on your harddisk or remotely, for instance on a Nexus repository.

A basic testing repository is available from Codetrails. It provides a source code repository with some artifacts from Eclipse projects for testing purposes (you can look up the contents with your internet browser): http://download.codetrails.com/livedoc/sources/

If you wish to use a local repository, you can deploy existing JAR files using the mvn deploy command. For example, to deploy your JRE jar to a local Maven repository, just grab your JRE’s source archive (src.zip in your JDK installation folder for Oracle JDKs) and type the following (for Java version 1.7):

mvn deploy:deploy-file -Durl=file:/some/folder/repo -Dfile=src.zip -DgroupId=jre -DartifactId=jre -Dversion=1.7.0 -Dpackaging=jar -Dclassifier=sources

After that, you can use /some/folder/repo as your source code repository for Livedoc (on JRE runs).

Model Repository

The location of the Eclipse Code Recommenders models (for Eclipse Luna) is http://download.eclipse.org/recommenders/models/luna-m3.

For a list of all available Eclipse releases, go to http://download.eclipse.org/recommenders/models and click on Show Directory Contents.

If you want to livedoc your own source code, you’ll have to mine models for it first. Take a look at http://www.codetrails.com/ctrlflow for further information.

Artifact Coordinate

Livedoc uses the maven artifact coordinate notation:

GROUP_ID:ARTIFACT_ID:VERSION

You can get the GROUP_ID and ARTIFACT_ID from your Maven repository. Note that on VERSION, you’ll have to always provide the ‘full’ version (x.y.z), e.g. for JRE 1.7 its 1.7.0.

Command line

Putting these informations together, the Livedoc command line has the following syntax:

./livedoc GROUP_ID:ARTIFACT_ID:VERSION [-sr URL] [-mr URL ...] [-Xdoclint (all|none|[-]<group>) ...] [-Xresume] [-d DIRECTORY] [-debugging] [-encoding ENCODING] [-highlight] [-jar] [-mc GROUP_ID:ARTIFACT_ID:VERSION] [-noDependencies] [-p PROVIDER_ID [PROVIDER_ARGUMENTS] ...] [-splitIndex] [-subpackages "org.eclipse org.apache ..."] [-transitive] [-u URL] [-verbose]

with the following options:

Parameter Description

GROUP_ID:ARTIFACT_ID:VERSION

The Maven coordinate of the source artifact for which you want to generate Javadoc.

-sr URL

Maven repository containing sources JAR.
Defaults to http://repo1.maven.org/maven2/

-mr URL

Maven repositories containing Code Recommenders models.
Defaults to http://download.eclipse.org/recommenders/models/luna/

-p PROVIDER_ID [PROVIDER_ARGUMENTS]

Choose Livedoc Providers and configure filter thresholds for recommendations, each Provider respectively.

-mc GROUP_ID:ARTIFACT_ID:VERSION

Optional Maven coordinate for the model artifact used to enhance the Javadoc.
If not set, the source coordinate is used for the model.

-u URL

Maven repository to upload Javadoc JAR to

-d DIRECTORY

Destination directory for the Javadoc HTML output files

-debugging

Debug output messages, includes full stack traces

-encoding ENCODING

Overwrite auto-guessed encoding of source files

-highlight

Shall Livedoc highlight its output with a yellow background?

-jar

Want to have a zipped JAR output for Javadoc?
Warning: Requires -d <DIRECTORY>

-noDependencies

Disable download of additional classpath entries via direct maven dependencies

-splitIndex

Splits the index file into multiple files, alphabetically, one file per letter, plus a file for any index entries that start with non-alphabetical symbols.

-subpackages "org.eclipse org.apache …"

A list of packages to Javadoc (full recursively), otherwise all found packages in the maven coordinate will be used.

-transitive

Shall Livedoc try to also resolve transitive dependencies instead of only direct ones?

-verbose

Output messages about what Livedoc is doing

-Xdoclint (all|none|[-]<group>)

Enable or disable specific checks for problems in javadoc comments, where <group> is one of the following:
accessibility, html, missing, reference, or syntax.

-Xresume

Fast mode, no unpacking, no temp directory cleaning
WARNING: For testing purposes only.

Providers

Livedoc comes with five Providers (more coming soon!), which enhance the HTML output with several code statistics:

Name Id Description

Method Overrides

ovrm

Methods frequently overridden by subclasses

Method Override Patterns

ovrp

Groups of methods which are frequently overriden together

"Call Also"

call-also

Subsequent method calls

Method "Self Calls"

selfm

Methods called frequently by implementors of a method (from subclasses)

Class "Self Calls"

selfc

Methods called frequently by subclasses

Google Analytics Provider

analytics

Adds Google Analytics to your Javadoc web page

You can decide which Providers you want to use with the -p flag, e.g. with

./livedoc org.eclipse.jface:org.eclipse.jface:3.8.102 -p ovrm -p selfm

only the Providers Method Overrides and Method "Self Calls" are being used. If you want to simply use all available Providers (dependent on the plug-ins loaded), you can also skip the flag

./livedoc org.eclipse.jface:org.eclipse.jface:3.8.102

and all Providers will be used, except for the Google Analytics Provider, which is a special case: This Provider will add the necessary Javascript Code for the Google Analytics API to your web page, including your own Tracking-ID. (see also: "Provider Configuration")

The following code statistic Providers are currently available:

Method Overrides

The Method Overrides Provider shows recommendations for methods which are frequently overridden by subclasses of a Class:

ovrm jface dialog

Method Override Patterns

The Method Override Patterns Provider shows recommendations for groups of methods which are frequently overridden together by subclasses of a Class:

ovrp jface dialog

"Call Also"

The "Call Also" Provider shows recommendations of the form "People who called Foo also called Bar":

callAlso jface dialog

Method "Self Calls"

Method "Self Calls" are calls to methods which are made frequently by implementors of a super method:

selfm jface dialog

Class "Self Calls"

Class "Self Calls" are methods called frequently by subclasses:

selfc jface dialog

Provider Configuration

Regardless of whether you’re using all providers or just one, you may want to filter their output. Otherwise you may end with hundreds of recommendations for a class, but with extremly low percentage values. Additionally, some Providers may need additional arguments to fit your special needs. For this, each Provider can be configured on its own, using command-line arguments.

Let’s start with filtering the output:

Livedoc provides three thresholds for this, configurable for each provider:

Threshold Flag Description ovrm ovrp call-also selfm selfc

Number

-top

Show only the top N recommendations

X

X

X

X

X

Times observed

-times

Show only recommendations with N or more times observed

X

X

X

Percentage

-percent

Show only recommendations with N or more percent

X

X

X

X

However, not every threshold can be used with every Provider, only the ones with "X" marked in the table above. Using a threshold with an unsupported Provider won’t have any consequences to filtering.

To specify a threshold for a Provider, you’ll have to pass it as a Provider option, this happens with square brackets, following the syntax -p PROVIDER_ID [-FLAG VALUE], e.g.

./livedoc org.eclipse.jface:org.eclipse.jface:3.8.102 -p ovrm [-percent 15] -p ovrp [-top 5]

Now, only method override recommendations with 10% or more will be shown, plus the top 5 method override patterns.

Additional Provider Arguments:

The three thresholds for filtering are optional, Providers can also work without these filter values. However, there are Providers which need additional arguments to work, thus they won’t be loaded if these arguments aren’t present.

An example (and currently the only one) for this is the Google Analytics Provider. With this Provider, you can let Livedoc easily insert the Tracking-Code for Google Analytics into your Javadoc HTML code.

To achieve this, you will have to provide your Google Analytics ID as an Argument:

./livedoc org.eclipse.jface:org.eclipse.jface:3.8.102 -p analytics [ANALYTICS_TRACKING_ID]

If you select the Google Analytics Provider (with "-p analytics"), but don’t provide an ID, the following error will be printed:

15:22:47.865 [main] ERROR o.e.r.i.l.javadoc.RecommendersTaglet - Provider "AnalyticsProvider" couldn’t be set up, skipping it.
    Message: org.kohsuke.args4j.CmdLineException: Argument "WEB_PROPERTY_ID" is required
    Exception: org.eclipse.recommenders.livedoc.providers.LiveDocProviderException

and the Provider won’t be loaded!

DocLint feature support

Livedoc supports the new DocLint feature of JDK 8 Javadoc. However, a slight syntax modification to the original Javadoc -Xdoclint is needed to configure the option in Livedoc:

./livedoc ... [-Xdoclint (all|none|[-]<group>) ...]

E.g., with

./livedoc ... -Xdoclint all -Xdoclint -accessibility

you would enable all groups of checks except the accessibility checks. For a reference of the group checks, see the JDK 8 Javadoc Documentation: http://docs.oracle.com/javase/8/docs/technotes/tools/unix/javadoc.html#BEJEFABE

NOTE: The DocLint feature in Livedoc is an opt-in instead of an opt-out like in original JDK 8 Javadoc. Meaning by default, the -Xdoclint option is disabled and no type of group checks will be made. See also this blog post by Stephen Colebourne why we made this decision.

Direct and transitive dependency resolution

Let’s take a look at this Livedoc command line:

./livedoc org.eclipse.jface:org.eclipse.jface:3.8.102 -sr http://download.codetrails.com/livedoc/sources/ -mr http://download.eclipse.org/recommenders/models/luna-m3/ -d ./output -highlight

Livedoc can handle the org.eclipse.jface package, however it has one problem: During the Livedoc run, you’ll see command line outputs like the following:

/tmp/livedoc/sources/org.eclipse.jface-3.8.102-sources/org/eclipse/jface/window/ApplicationWindow.java:37:
error: package org.eclipse.swt.widgets does not exist
import org.eclipse.swt.widgets.Control;
                              ^
…
…
4396 warnings
Done.

This means that the artifact org.eclipse.jface:org.eclipse.jface:3.8.102 has dependencies on the package org.eclipse.swt.widgets. Without these dependencies, Javadoc won’t be able to use fully qualified names for these Classes and thus, Livedoc can’t include Code Recommenders statistics for them.

Livedoc tries to resolve (download) these dependencies from your source code repository using the source artifact’s meta information. However, Livedoc will only try to resolve direct dependencies. Transitive dependencies won’t be resolved by default, as this could take some time (and bandwidth). You can force Livedoc to resolve transitive dependencies using the -transitive flag:

./livedoc org.eclipse.jface:org.eclipse.jface:3.8.102 -sr http://download.codetrails.com/livedoc/sources/ -mr http://download.eclipse.org/recommenders/models/luna-m3/ -d ./output -highlight -transitive

The result will be less warnings , fully qualified names in the HTML output and more statistics from Code Recommenders! Of course, Livedoc can only use dependencies which are included in your source code repository.