Interface - Reports

Custom Analyzers


Tools - Path Analyzer

Custom Analyzers

To this point, we have looked at what Deep Dive can do with its built-in analyzer functionality. However, Deep Dive is extensible, in that you can build your own custom analyzers. Unfortunately, building custom analyzers that can examine classes requires a good understanding of byte code and how to manipulate it with libraries like ASM or Javassist. The Deep Dive help content only covers how to use Deep Dive, so you will have to learn about byte code engineering on your own (but don't be discouraged - there are plenty of resources on the web and its much easier than you might think).

Custom Analyzer Development

All Deep Dive analyzers must implement the DeploymentVisitor interface:

package ca.discotek.deepdive.security.visitor;

...

public interface DeploymentVisitor {
    
    public void visitStart() throws AnalyzerException;
    
    public void visitDeployment(File file) throws AnalyzerException;
    
    public void visitEar(Ear ear, Location location) throws AnalyzerException;
    public void visitEarLib(Jar jar, Location location) throws AnalyzerException;
    public void visitWar(War war, Location location) throws AnalyzerException;
    public void visitEjbJar(Jar jar, Location location) throws AnalyzerException;
    public void visitClass(ClassFile classFile, Location location) throws AnalyzerException;
    public void visitWebInfClass(ClassFile classFile, Location location) throws AnalyzerException;
    public void visitWebInfLibJar(Jar jar, Location location) throws AnalyzerException;
    public void visitUnknownJar(Jar jar, Location location) throws AnalyzerException;
    public void visitJsp(JspFile jsp, Location location) throws AnalyzerException;
    public void visitImage(ImageFile image, Location location) throws AnalyzerException;
    public void visitJavascript(JavascriptFile javascript, Location location) throws AnalyzerException;
    public void visitCss(CssFile css, Location location) throws AnalyzerException;
    public void visitHtml(HtmlFile html, Location location) throws AnalyzerException;
    public void visitUnknownFile(UnknownFile unknownFile, Location location) throws AnalyzerException;
    public void visitManifest(Manifest manifest, Location location) throws AnalyzerException;
    public void visitEnd() throws AnalyzerException;
    
    public String getTitle();
    public void setTitle(String title);
    public String getSummary();
    public String getDescription();
    public String[] getLinks();
    
    public void outputToHtml(StringBuilder buffer, boolean hyperlinkPolicy, Pattern fileFilter) throws OutputException;
}

Deep Dive uses a visitor pattern to examine various deployment unit constructs. The method names in this interface are mostly self documenting. However, you should note that most common deployment unit constructs are provided to you as types (e.g. Ear, War, ClassFile etc). This should be of great advantage to you in that very little parsing is required, other than the parsing required by your customizations.

The outputToHtml(...) method warrants some further explanation. This method is invoked when it is time to output your results. All results are specified as HTML and you will need to append your marked up results to the StringBuilder buffer parameter. The hyperlinkPolicy parameter indicates whether the output targets memory (i.e. to be displayed within Deep Dive) or the disk (i.e. the HTML report to later be viewed in a browser). This parameter is to be used with the ca.discotek.deepdive.grep.classmatcher.iterator.OutputIterator class. This class was originally designed to house the code to generate analyzer output generically. However, creating the hierarchical HTML (lists containing lists) was particularly error prone. Another class, ca.discotek.deepdive.security.misc.HtmlList eventually replaced much of that functionality. It is recommended that you use HtmlList to generate your custom output. However, OutputIterator still has some useful methods, such as generating the links to resources like locations, classes, and other files. To use OutputIterator for this purpose, in your DeploymentVisitor's outputToHtml(...) method, instantiate your own OutputIterator, using the hyperlinkPolicy value as a parameter (i.e. OutputIterator it = new OutputIterator(hyperlinkPolicy);) and then call the following methods for outputing links:

Unfortunately, these methods are not well documented, so you will probably have to experiment a little to make sure they work the way you would like. In the future, these methods will be better documented, but they will probably be in some other class (most likely OutputIterator will be replaced by HtmlList's functionality, but may System analyzers still use it).

You should also be aware there is an abstract implementation of the DeploymentVisitor interface called ca.discotek.deepdive.security.visitor.assess.AbstractDeploymentVisitor, which will make implementing custom analyzers a little easier. Here is the code:

public abstract class AbstractDeploymentVisitor implements DeploymentVisitor {

    private String title;
    public final String summary;
    public final String description;
    public final String links[];
    
    public AbstractDeploymentVisitor(String title, String summary, String description, String links[]) {
        this.title = title;
        this.summary = summary;
        this.description = description;
        this.links = links;
    }
    
    public void visitStart() throws AnalyzerException {}
    public void visitDeployment(File file) throws AnalyzerException {}
    public void visitClass(ClassFile classFile, Location location) throws AnalyzerException {}
    public void visitEar(Ear ear, Location location) throws AnalyzerException {}
    public void visitEarLib(Jar jar, Location location) throws AnalyzerException {}
    public void visitWar(War war, Location location) throws AnalyzerException {}
    public void visitEjbJar(Jar jar, Location location) throws AnalyzerException {}
    public void visitWebInfClass(ClassFile classFile, Location location) throws AnalyzerException {}
    public void visitWebInfLibJar(Jar jar, Location location) throws AnalyzerException {}
    public void visitJsp(JspFile jsp, Location location) throws AnalyzerException {}
    public void visitImage(ImageFile image, Location location) throws AnalyzerException {}
    public void visitJavascript(JavascriptFile javascript, Location location) throws AnalyzerException {}
    public void visitCss(CssFile css, Location location) throws AnalyzerException {}
    public void visitHtml(HtmlFile html, Location location) throws AnalyzerException {}
    public void visitUnknownFile(UnknownFile unknownFile, Location location) throws AnalyzerException {}
    public void visitManifest(Manifest manifest, Location location) throws AnalyzerException {}
    public void visitEnd() throws AnalyzerException {}
    
    public String getTitle() {
        return title;
    }
    
    public void setTitle(String title) {
        this.title = title;
    }
    
    public String getDescription() {
        return description;
    }
    
    public String getSummary() {
        return summary;
    }
    
    public String[] getLinks() {
        return links;
    }
    
    public abstract void outputToHtml(StringBuilder buffer, boolean hyperlinkPolicy, Pattern fileFilterPattern) throws OutputException;
}

Custom Analyzer Configuration

Once you have one or more custom analyzers, you must package them in a jar file. Further, the jar file must contain one or more XML configuration files which tells Deep Dive about the analyzers. You should have a separate XML file for each Category of analyzers. Categories are described in the Interface help page. These XML files must be located in the /META-INF/analyzers directory of the jar file. The following is an example of the XML format for a third party category XML file:


<analyzers category="Deep Dive Example">

    <custom-analyzer class="ca.discotek.deepdive.thirdparty.example.CountFileTypesAnalyzer">
        <title>Example Analyzer - Counts File Type Analyzer</title>
        <summary>Counts Number of each File Type Found in Deployment Unit.</summary>
        <description>The number of file types in a deployment unit isn't necessarily a measure of
        quality or security. This analyzer really just demonstrates how an analyzer could be built.</description>
        <links>
            <link>http://example.com/why_type_counts_are_important.html</link>
        </links>
    </custom-analyzer>
    
    <custom-analyzer class="ca.discotek.deepdive.thirdparty.example.FormMethodAnalyzer">
        <title>Example Analyzer - Form Method Analyzer</title>
        <summary>Finds References to Form Method Attribute</summary>
        <description>
        This analyzer might be helpful in assessing the security of an application. For instance,
        if you wanted to ensure that all forms use the <i>POST</i>s and not <i>GET</i>s. However,
        it is really just for demonstration purposes.
        </description>
        <links>
            <link>http://example.com/why_reviewing_form_method_types_are_important.html</link>
        </links>
    </custom-analyzer>
    
</analyzers>

The format of these XML files is the same as those generated by creating User Defined categories and analyzers. Deep Dive stores these files in <user-home>/project-manager/deepdive/<project-name>. Further, you can use the Class Grep, File Finder, and Class Matcher analyzers in your third party jar of analyzers. Here is an example of each type (taken from Deep Dive's built-in System/Quality analyzers):


    <class-match-analyzer>

      <title>printStackTrace / dumpStack</title>
      <summary>Indicates poor logging.</summary>
       <description>Throwable.printStackTrace(...) and Thread.dumpStack() methods should not be used to record diagnostic information. These methods generally go to stdout and may not
        be recorded at all. A standardized logging utility like Log4J, Java Logging API, SLF4J should be used.</description>

       <class result-type="method">
           <method>
               <instructions>
                   <method-instruction>
                       <match-expression name="name" type="regex" case-sensitive="false">printStackTrace|dumpStack</match-expression>
                   </method-instruction>
               </instructions>
           </method>
       </class>
    </class-match-analyzer>
    
    <grep-analyzer>
        <title>Reflection references.</title>
        <summary>Reflection belongs in framework and/or libraries and its use in applications should be exceptional.</summary>
        <description>Reflection is expensive and adds complexity. It is rarely needed in application code. Any use reflection in your 
        application code should be eliminated unless deemed essential.</description>
        <match-expression name="grep-pattern" type="contains" case-sensitive="true">java.lang.reflect</match-expression>
    </grep-analyzer>
    
    <file-finder-analyzer>
        <title>All EJB References</title>
        <summary>EJBs are an integration point and should represent your business tier logic.</summary>
        <description>Finds all references to EntityBeans, SessionBeans, and MessageDrivenBeans. These results do not indicate any specific issue. They are to help
        the assessor understand the application and inspect these integration points for related issues.</description>

        <match-expression name="file-pattern" type="regex">.*\.class|.*\.xml</match-expression>
        <match-expression name="contains-pattern" type="regex">EntityBean|SessionBean|MessageDrivenBean</match-expression>
    </file-finder-analyzer>
    

If you are planning on using these types of built-in analyzers, it is recommended you generate the XML by creating a User Defined analyzers, then copying it to your third-party XML.

Custom Analyzer Deep Dive Integration

Once you have a third-party Deep Dive jar, you integrate it with Deep Dive by using the third-party-path system property. The value of this property is a list of paths to Deep Dive third-party jars. This list is delimited by the platform's path separator (i.e. ';' for windows and ':' for *nix). Additionally, you must add the same jar to Deep Dive's application classpath. An example, might look like this:

java -Dthird-party-path=C:/deep-dive/deepdive-<deep-dive-version>/third-party-project-example/dist/thirdparty-example-deepdive.jar -classpath C:/deep-dive/deepdive-<deep-dive-version>/dist/discotek.deepdive-<deep-dive-version>.jar;C:/deep-dive/deepdive-<deep-dive-version>/lib/discotek.deepdive-engine-<deep-dive-version>.jar;C:/deep-dive/deepdive-<deep-dive-version>/third-party-project-example/dist/thirdparty-example-deepdive.jar ca.discotek.deepdive.security.gui.DeepDiveGui

Lastly, you can find an example project in the Deep Dive distribution zip file directory deepdive-<version>/third-party-example-project. This project contains an ANT script in the root directory, which demonstrates how to build third-party Deep Dive jar.


Interface - Reports

Custom Analyzers


Tools - Path Analyzer