Value Types or: Let’s add another type system to Java

Java has two major type systems, primitives types(eg. int, byte, etc) and class/object types(eg. java.lang.String). Primitive types provide concise storage of data with no attach methods, while class types provide the ability to extend the functionality of an object while sacrificing space. These type systems are on different ends of the space/performance spectrum and Value Types aim to get the best of both worlds.

Value Types are a JDK Enhancement Proposals(JEP) that will add a change to the JVM instruction set allowing “support [for] small immutable, identityless value types”. The “identityless” notion is the big difference here. In Java, the identity is there to support mutability, synchronization, and few other features. The identity comes with a cost to performance and increases the object’s footprint.

There are a number of features that could be gained from Value Types. Tuples (esp. nice when trying to return multiple value from a method), numeric (supporting more than the eight default Java primitives), native types (ie taking advantage of specific processor types), algebraic data types (see jADT for some good examples/rational) and iterator/cursor simplification.

Below is some examples showing two of the possible gains from value types: Reduction in footprint, and increase in performance.

Performance – Reductions in Opcodes

Take a look at this simple example.

public class Example {
    public static void main(String[] args) {
        Example example = new Example();
        int intPrimitive = example.intMethod(1, 2);
        Integer intObject = example.integerMethod(1, 2);
    }
    public int intMethod(int a, int b) {
        return a + b;
    }
    public Integer integerMethod(Integer a, Integer b) {
        return a + b;
    }
}

The code is adding the same two numbers together. The first time using the primitive int type and the second time using the Integer object. Lets first take a look at the byte code for the primitive method.

  public int intMethod(int, int);
    descriptor: (II)I
    flags: ACC_PUBLIC
    Code:
      stack=2, locals=3, args_size=3
         0: iload_1
         1: iload_2
         2: iadd
         3: ireturn

Pretty straight forward. The method loads the value 1 (iload_1), then loads the value 2 (iload_2) and returns them after adding.
And now, the Integer version.

  public java.lang.Integer integerMethod(java.lang.Integer, java.lang.Integer);
    descriptor: (Ljava/lang/Integer;Ljava/lang/Integer;)Ljava/lang/Integer;
    flags: ACC_PUBLIC
    Code:
      stack=2, locals=3, args_size=3
         0: aload_1
         1: invokevirtual #10   // Method java/lang/Integer.intValue:()I
         4: aload_2
         5: invokevirtual #10   // Method java/lang/Integer.intValue:()I
         8: iadd
         9: invokestatic  #5    // Method java/lang/Integer.valueOf:(I)Ljava/lang/Integer;
        12: areturn

It takes three more commands and the bytecode array is 4x larger. What is killing us here is the un/boxing of the Integer object. (Note: In this code we’re not using any Object features.) This is pretty basic knowledge but it’s neat to see in the bytecode.

Footprint – Header Size

Let skip past the size of an int (4-bytes) and go straight to Integer. Using jol–a tool to help analyze object layout–we can see the internals of a  workings object.

Running 64-bit HotSpot VM.
java.lang.Integer object internals:
 OFFSET  SIZE  TYPE DESCRIPTION                    VALUE
      0    12       (object header)                N/A
     12     4   int Integer.value                  N/A
Instance size: 16 bytes (estimated, the sample instance is not available)
Space losses: 0 bytes internal + 0 bytes external = 0 bytes total

Notice that we waste 12-bytes on the header. That means that an Integer is 4 times the size of an int.

These little excesses add up after a while. In fact, this is how some algorithm classes make sure that when you do the programming assignments that you actually implement the algorithm and don’t use a library (or several libraries) to bypass learning how an algorithm is implemented. Think about it, if you have to sort an array of 100 ints with bubble sort and you instead use Collection.sort() then your program is going to be much larger than if you used the primitive int, not to mention slower.


Most of my understanding for this post comes from this blog and from the JEP description, which is long but interesting.

Refactoring : Logical Operators Instead of Conditional Flow

I’ve recently been reading Working Effectively With Legacy Code by Feathers (an oldie, but a goodies). The book is good for enumerating techniques of refactoring but more importantly, it is a reminder to be more observant when looking through old code, and that is exactly what happened to me this weekend. I was trying to familiarize myself with how our application used OSGi inside of a web container when I came across the following code. It looks as if it had been written in stages. Something akin to: “Try it this way. Oh, that worked here but not there. Add another thing…oh, and there is another use case.” At any point, when the programmer realized they needed to add another test case, they should have use the Sprout technique. After all, it doesn’t take much to create a method and put the code down a few lines. There are several other concerning thing about the code. I’ll enumerate them below and rewrite the method a couple of times.

public boolean Startup() {
    boolean started = true;
    // first try
    String osgiServletURL = URLUtils.getUrl() + BRIDGE_URL;
    started = callBridgeServlet(osgiServletURL);

    // recovery part 1  
    if (!started) {
        InetAddress thisIp = null;
        try {
            thisIp = InetAddress.getLocalHost();
        } catch (UnknownHostException e) {
            started = false;
        }
        if (thisIp != null && thisIp.getHostAddress() != null) {
           String thisIpAddress = thisIp.getHostAddress().toString();
           osgiServletURL = createOSGIServletURL(thisIpAddress);
           started = callBridgeServlet(osgiServletURL);
        } else
           started = false;
    }
    // recovery part 2 
    if (!started) {
        osgiServletURL = createOSGIServletURL(LOCALHOST_NAME);
        started = callBridgeServlet(osgiServletURL);
    }

    return started;
}
  • Boolean success variables: The started variable is hard to keep track of especially with all those conditional branches. (Should this variable be initialized to true‽)
  • Nested conditional statements: Not only that, but there are missing brackets around the last else statement, making it harder to read.
  • The method is making three attempts to connect to the servlet and in one case it’s checking for an exception but not in the others, making the code asymmetrical?
  • A variable that changes meaning: Notice how osgiServletURL holds values that mean different things as it goes through the method. At one point it’s a URL at another it’s an IP address.

Let’s do a little clean up in our first pass. Eliminate unnecessary comments (the code will be obvious when we get done), pull out method-wide variables, and if you have to use a boolean success variable make sure you initialize it to the false, otherwise you’ll constantly be setting the value and you’re likely to miss a case.

public boolean startup() {
    boolean started = false;
    String osgiServletURL = URLUtils.getUrl()+BRIDGE_URL;
    started = callBridgeServlet(osgiServletURL);

    if (!started) {
        InetAddress thisIp = null;
        try {
            thisIp = InetAddress.getLocalHost();
            if (thisIp != null && thisIp.getHostAddress() != null) {
                String thisIpAddress = thisIp.getHostAddress().toString();
                osgiServletURL = createOSGIServletURL(thisIpAddress);
                started = callBridgeServlet(osgiServletURL);
            }

        } catch (UnknownHostException e) {
        }

    }
    if (!started) {
        osgiServletURL = createOSGIServletURL(LOCALHOST_NAME);
        started = callBridgeServlet(osgiServletURL);
    }

    return started;
}

Looking a little bit better. Let’s create some sprout method to clean up that ugly looking nested exception(canConnectWithUrl Line 2-4). We’re also going to create two more sprout methods so that they will have similar names(canConnectWithIpLine 7-17 and canConnectWithLocalHostNameLine 21-22). Ultimately, just a readability thing.

public boolean startup() {
    boolean started = false;

    started = canConnectWithUrl();
    if (!started) {
        started = canConnectWithIp();
    }else if (!started) {
        started = canConnectWithLocalHostName();
    }

    return started;
}

And, finally, to get rid of the if statements and the boolean success variable, let’s use logical operators:

public boolean startup() {
    return canConnectWithUrl() 
        || canConnectWithIp() 
        || canConnectWithLocalHostName();
}

Java will call each method in turn and return as soon as the first one is true. So we have code that reads like this: “Can it connect using the url or can it connect with an ip address or can it connect with the localhost name”. If the code were written this way yesterday, I would have been able to read it, understand it, and move on to the next step within a minute. (Not to mention that the refactored code will be easier to test.)

This is not a complex case study, more similar to the type of pointers that I would give during a code review. Hopefully it will stand as a reminder that taking a second look at code can go a long way in readability and maintainability.

Annotation Processing in Java

Java 8 introduced Type Annotations. This means you can do something like the following:

public void processChange(@NonNull Boolean set){}

Java points out that it doesn’t supply a type checking framework but this got me to thinking. How can we as a company use annotations to make better internal code? Of course we all use annotations on a regular basis but how can we indicate to our coworkers our intentions and reduce bugs. This is one of the thought behind Design by Contract. I didn’t want to get into writing a full blown type checking framework but did want to understand the basic built in Java annotation mechanism. And so the following represents a tech demo project that I used to understand annotations better.

Documenter

Say you have a project—an open source project, a tutorial project, or even a new framework—that you’d like to make more sense out of for users. It would be nice if you could annotate pieces of the code and generate a report (or more preferably an IDE plugin) that would point to and describe that piece of code. So I created a library that allows you to supply a message and a priority of that message to parts of the code. The idea being that you could run you annotations on a project and it would create an ordered list of key points in the code..

Below is a trivial example of how one would use Documenter.

static void addTask(Project project){
    @Document(key="Must declare extensions in RunSimpleExtensions class", priority=1)
    project.extensions.create("runSimple", RunSimpleExtension)

    project.task("runSimple", type: JavaExec ) {
        project.afterEvaluate{
            @Document(key="First use of extensions", priority=2)
            main = project.runSimple.mainClass
            classpath = project.sourceSets.main.runtimeClasspath
            @Document(key="Second use of extensions", priority=3)            
            args = project.runSimple.args
        }    
    }
}

Document

Creating the annotation is trivial. Just use the @interface annotation type definition. The two things to consider when creating an annotation are (1)where you want to use it (the target) and (2) how long you want it to stay around (the retention policy). In the Document example I use a RetentionPolicy of SOURCE, which means the annotation will stay around for the shortest amount of time. The most useful thing would be for the annotations to be use in all possible places. If you don’t specify the Target or the Retention Java assumes that you want to keep the annotation and that you can use the annotation on any allowed type.

import static java.lang.annotation.ElementType.*;

@Retention(RetentionPolicy.SOURCE)
@Target({ANNOTATION_TYPE, CONSTRUCTOR, FIELD, METHOD, PARAMETER, PACKAGE, TYPE, LOCAL_VARIABLE, TYPE_PARAMETER, TYPE_USE })
public @interface Document{
    int priority();
    String key();
}

Service Provider

Let’s take a detour and talk about the Service Loader framework in Java. The Service Provider Interface was designed to allow a third-party to add functionality to an application. To do this, a user would implement an interface and put it on the classpath, then point to it in the META-INF file. It was the responsibility of the Service Provider to use the Service Loader to grab the classes from Java. Here is an example:

private ServiceLoader<Documenter> loader;

private DocumeterService() {
    loader = ServiceLoader.load(Documenter.class);
    Iterator<Document> documents = loader.iterator();
    while(documents.hasNext()) {
        Document d = documents.next();
    }
} 

(This can be seen as a basic implementation of Dependency Injection.) Annotations are handle in a similar way, but instead of using the Service Loader, Java will pass the annotations to a class that you create that extends the javax.annotation.processing.Processor interface.

Abstract Processor

If you have created a javax.annotation.processing.Processor file and put it in your the META-INF/service/ folder then Java will call the class/es that you list in that file. The following is part of the AbstractProcessor for the Documenter project. (It is standard to extend the AbstractProcessor over directly implementing the Processor interface.)

@SupportedAnnotationTypes({ "com.scuilion.documenter.Document" })
public class AnnotationProcessor extends AbstractProcessor {

    public boolean process(Set<? extends TypeElement> annotations, RoundEnvironment roundEnv) {
        Map<String, Note> documents;
        documents = new HashMap<>();
        if (!roundEnv.processingOver()) {
            Scanner scanner = new Scanner();
            scanner.scan(roundEnv.getElementsAnnotatedWith(com.scuilion.documenter.Document.class), documents);
        }
        return true;
    }
}
  • Line 1. Java will filter out all other annotations other than the one you specify. If you really want to handle all annotations in the system, then don’t declare a SupportedAnnotationTypes
  • Line 4. Round Environments hold the annotations that are available for processing.
  • Line 9. Scanner is an extension of ElementScanner class. Calling scan will pass the annotated elements to the appropriate visitXYZ method. For instance, visitPackage will be called when a package level annotation is used. (Have you ever scene a package annotated?)

The process is explicitely using the visitor pattern. I won’t got into the ElementScanner you can see my example on github. The result of what was done is to process each annotated position and output the type and the class the element was located in.

Weld CDI: User Injected Functionality

Here is the scenario: Suppose you are creating a library that will aggregate data from a system and send that information to standard out. A user may want to use your library to aggregate the same information but wants to send the data elsewhere (eg, a db, a file, etc).  How do you provide these feature to you library without the user having to explicitly create your object and pass in a writer? (Similar to creating an extensible application or plug-in.) You could use Java’s Service Provider but then you would have to programmatically exclude default behavior when users implement their own.

Recently, I’ve been working on an creating an annotation library. In order to create an annotation library in Java, you have to extend an AbstractProcessor and declare that class as a javax.annotation.processing.Processor in META-INF/services. Java will pick this class up automatically, so I need a way inject a users implementation class without changing the code and I want to disable my default implementation when this occurs. In comes Dependency Injection (DI) in the form of JEE’s CDI service.

The following is an example of how to replace one bean implementation with another using CDI. I’m using the JEE reference implementation of CDI, Weld, in an SE fashion. So there is no need to run in a container. The example uses Alternatives to replace a default implementation with a user created implementation. You can find the working example on github.

https://github.com/Scuilion/weldit

The first thing is initializing Weld. Weld requires the base beans.xml file to be in the META-INF folder and because I’m not using an EE container, the container has to be generated manually (Note how the container is created and destroyed before the injected class is used.)

public class Producer {
    @Inject
    Writer writer;
    public someLibraryMethod() {
        Weld weld = new Weld(); 
        WeldContainer container = weld.initialize(); 
        writer = container.instance().select(Writer.class).get(); 
        weld.shutdown(); 
        writer.process();
    }
}

Writer is the interface that you and the consumer of you library will implement. You’re library will come with a default implementation (WriterImpl.java). If the consumer does not create their own implementation then Weld will load up this default.

public class WriterImpl implements Writer {
    @Override
    public void process() {
        System.out.println("in default writer implementation");
    }
}

And here is and example users implementation. All we have to do is use the @Alternative annotation to tell weld that we want to use this class as oppose to the default class to be injected.

@Alternative 
public class ReplacementWriterImpl implements Writer {
    @Override
    public void process() {
        System.out.println("************************");
        System.out.println("in alternative");
        System.out.println("************************");
    }
}

Notice that the only thing different is some extra print statements. Let’s see how this works when running. I’ve set up test for the base and the consumer under com.BaseTest and com.ConsumerTest, respectively. If I run the base test, again with only the default implementation I get the following.

kmb-us-master109:weldit kevin.oneal$ ./gradlew :base:test
:base:compileJava
:base:processResources
:base:classes
:base:compileTestJava
:base:processTestResources
:base:testClasses
:base:test

com.BaseTest > testSomeLibraryMethod STANDARD_OUT
    in default writer implementation

BUILD SUCCESSFUL

Total time: 4.089 secs

And when we run the consumer test, we get the following.

kmb-us-master109:weldit kevin.oneal$ ./gradlew :consumer:test
:base:compileJava UP-TO-DATE
:base:processResources UP-TO-DATE
:base:classes UP-TO-DATE
:base:jar
:consumer:compileJava
:consumer:processResources
:consumer:classes
:consumer:compileTestJava
:consumer:processTestResources UP-TO-DATE
:consumer:testClasses
:consumer:test

com.ConsumerTest > testSomeLibraryMethod STANDARD_OUT
    in consumer
    before called to producer 
    ************************
    in alternative
    ************************
    after called to producer 

BUILD SUCCESSFUL

Total time: 3.971 secs

You can get this project from github under the tag v0.1.

Ultimately, the difference between using the Service Provider or Weld’s interpretation of CDI is that with DI I can eliminate the very trivial default implementation. Although I haven’t tried it, Weld is suppose to allow you to mimic the feature of Service Providers where Java will pick up all implementation of an interface/abstract, just by changing the @Inject variable to take a list.

First Experience with Bintray

When I started to learn Gradle, I wrote a simple plugin. It was a fairly useless adapter for JavaExec. It automatically set up the classpath and created an extension for pointing to the main class. This was a exercise.

project.extensions.create("runSimple", RunSimpleExtension)

project.task('runSimple', type: JavaExec ) {
    project.afterEvaluate{
        main = project.runSimple.mainClass
        classpath = project.sourceSets.main.runtimeClasspath
    }
}

Recently, I’ve been beefing up my development process in Vim and installed Syntastic. This plugin provides syntax checking by running external checkers, two of which I needed–JSHint and javac. Out-of-the-box, Syntastic works great with Java, until you start adding external libraries. Fortunately, I use Gradle on all of my projects and Gradle makes it easy to determine you dependencies.

project.sourceSets.each { srcSet ->
    srcSet.java.srcDirs.each { dir ->
        classpathFiles.add(dir.absolutePath)
    }
}

So I added this functionality to my original plugin and called it gradle-utils. The problem was the hassle of using the plugin from one computer to the next. I’d have to pull the project from GitHub and publish it locally (using the maven-publish plugin). Not to mention if I made changes the whole process would start over.

In Walks jCenter

This was a perfect opportunity to try out BinTray. I’d had an account, but other than signing up, it sat dormant. Here are a list of the things learned while uploading my first artifact.

  • Don’t forget you have to push up your source as well as the complied classes if you want to attach you package to the public jCenter repo. I’m using the gradle maven-publish plugin and accomplish that like so:
    task sourceJar(type: Jar) {
        from sourceSets.main.groovy
        from sourceSets.main.resources
    }
    artifacts {
        archives jar, sourceJar
    }
    publishing {
        publications {
            maven(MavenPublication) {
                from components.java
                artifact sourceJar {
                    classifier "sources"
                }
            }
        }
    }
    
  • Gradle 2.1’s new developer plugin index makes include the Bintray plugin a snap. (Example of this below.)
  • In order to include your package in the the publicly accessible jCenter you have to ask. It took me longer than I would like to admit to find how to do this. I assumed that the option would be located somewhere within the package you were attempting to release, but it actually on the home page of jCenterbintray

A Personal Plugin for Personal Use

This plugin is very “me” centric, but it’s really easy to get it setup, assuming you already have the Syntastic plugin working in Vim. There are two things you need, 1) set Syntastic so that it creates a config file, and 2) add the gradle-utils plugin to your build.gradle file.

1) .vimrc

let g:syntastic_java_checkers=['checkstyle', 'javac']
let g:syntastic_java_javac_config_file_enabled = 1

2) build.gradle

buildscript {
    repositories {
      jcenter()
    }
    dependencies {
       classpath group: 'com.scuilion.gradle', name: 'utils', version: '0.2'
    }
}
apply plugin: 'utils'

Note: This is a post process plugin and should be applied at the end of your build file.

Screenshot from 2014-07-19 17:18:02
When junit is commented out of the build file, Syntastic shows that it can’t compile this particular file.

An aside: I used Gradle 2.1’s developer index to include the BinTray plugin. So instead of:

buildscript {
    repositories {
        jcenter()
    }
    dependencies {
        classpath "com.jfrog.bintray.gradle:gradle-bintray-plugin:0.5"
    }
}
apply plugin: "com.jfrog.bintray"
plugins {
    id "com.jfrog.bintray" version "0.5"
}

Pretty cool!

Takeaways from UberConf 2014

As usual, the NFJS guys put on a very informative event. I figured I would write down my thought on the week before they get stale. This is my fifth event to attend but only my first UberConf. The added workshops was definitely the hook that got me to pester my boss to pay for this. It has to be a tough task to come up with a simple yet helpful/doable set a exercises, especially if the workshop is only half a day. Most of the workshops could have been twice as long; By the time the presenting part of the session was done there was little time to do actual work. (System setup was usually not handled in the best way.) In one class that I attended, the “workshop” title was just an excuse to talk longer. Fortunately, the long discussion was not bothersome. The following is a list in no order, just a data dump.

  • Neo4J – Six months ago I finished a small one-off project that would have benefited if I had used Neo4J. I wish that I had thought to use a graph database. Here in the next week I will rewrite that program. (Future post to follow on this.)
  • Apache Camel – This pops up just about everywhere. A month ago I consulted on how to setup an Apache ServiceMix project and several of the presenters were using Camel as part of their examples. I need to come up with a project, so I can get more hands on.
  • Continuous Delivery vs Continuous Deployment – This reminded me of the Shippable vs Saleable argument (or minimum viable product). My company’s product (at least the one I work on) is such a behemoth legacy app that even the discussion of Continuous Deployment is far down the road.
  • NGINX – came up in discussion several times. I don’t remember it being so popular before.
  • Clojure – I’ve deciding on which new language to delve into and plan on spending a good amount of time learning. For a while it was a toss up between Clojure and Scala. Clojure has officially won for me. The thought is that with Java 8 and it’s added functional features that I would go full on and learn Lisp as opposed to some middle ground. I’m using the following resources (along with the obvious Clojure references and cheat sheet maintained on clojure.org): Koans, exercism.io, and Seven Languages in Seven Weeks.
  • OSGi – For several months now I’ve been dealing with a rewrite of our OSGi implementation, which was fragile to begin with and is out-of-date (It’s stuck on Java 6 and buckminster). Talking with one of the presenters I found out that I’d missed OSGi DevCon by only a couple of weeks. This would have been a tremendous resource at figuring out the last few missing pieces. It also appears that the DevCon people didn’t put up any videos, which would have been helpful. Guess I find out what the slides give me.
  • Pomodoro Technique – On Wednesday the conference went late–up until 10. I decided that I would take a session that was less technical since I had been stuffing my brain all day. A friend of mine had encouraged me to try out the Pomodoro Technique. It seemed like a good idea but I didn’t read his blog post about it and eventually forgot. After debugging my way through this session, I decided I would give it a try. In fact, my first two pomodoros were done the next night after the conference and hit the mark for time. An abnormality that I chalk up to beginners luck, as my next 5 were staggeringly underestimated.
  • Docker – Another thing that the aforementioned friend put me onto. I’d mostly let it go without further research because I’m currently stuck on Windows at work. The session was enlightening and the flexibility of Docker was reiterated in a preceding workshop where environment setup could have been drastically reduced with the help of Docker.
  • Bintray – I was reminded about my stale account. Hopefully I’ll be using jcenter as soon as I get the Neo4J project going.

Books

  • Scalability Rules
  • The Design of Everyday Things – I’m halfway through this book. It’s my airplane book, and seeing as I don’t go on a lot of trips, it is taking a while for me to get through it.
  • Presentation Matters – This is co-authored by a couple of regular NFJS presenters. This is a soft skills topic that techy people seem to try and avoid, especially me. The last presentation I gave in front of my organization was about Java 8’s added annotations to type and repetitive annotations. Afterwards my boss commended me on my  presentation but suggested that I tell people who I am before jumping in.
  • How To Win Friends And Influence People – You hear about a certain book and after you hear about it a number of times, it just automatically gets put on your reading list.
  • Understanding Cryptography

Two soft skill books and two technical books added to my reading list. Plus two pages of bullet point task of personal work that I need to get done.