Feeds:
Posts
Comments

Devoxx US 2017 Recap

 

Devoxx US 2017 was back in March 21-23 of this year, and I’m only now getting around to posting an article about it.

The conference was in the San Jose McEnery Convention Center, which is quite a convenient venue for me. It’s only a little bit farther from home than my office. The session rooms and exhibition space were pretty nice too.

Unfortunately, the attendance seemed fairly light, which might have had something to do with the postponement of the next Devoxx US until 2019, skipping 2018.

An uncrowded conference meant there was more time for conversations with other speakers and other conference attendees. This was really great. I remember one conversation in particular with Trisha Gee where we had time to talk about nulls and Optional in detail. Some of the ideas from this conversation wound up in an article Code Smells: Null that she wrote recently.

As is typical, I had several sessions at the conference.

Ten Simple Rules for Writing Great Test Cases
– conference session with Steve Poole | slides | video

This is somewhat refreshed and updated version of the BOF Ten Things You Should Know When Writing Good Unit Test Cases in Java that Paul Thwaite (Steve’s colleague at IBM) and I had at JavaOne 2013. We didn’t actually update it all that much; I think most of the advice here is quite broadly applicable and doesn’t go obsolete. Actually, we did update it – “now with added cloud.”

Streams in JDK 8: The Good, The Bad, and the Ugly
– BOF with Simon Ritter | slides

This was a reprise of the BOF that Simon gave at Devoxx BE 2016 where he pulled me up front and asked me to provide some extemporaneous commentary. This worked so well that we decided to have me as an official co-speaker for the BOF this time.

Collections Refueled – conference session | slides | video

This is my talk about the new stuff in the Collections Framework in Java 8 and 9. Unfortunately, I didn’t prepare for this very well, and I had 60 minutes of material but only 45 minutes to present it. I ended up having to skip a bunch of the Java 9 material towards the end. (My JavaOne 2016 version of this talk is probably better.)

Optional: The Mother of all Bikesheds – conference session | slides | video

I’m happy to say that this was the second-highest rated talk at Devoxx US, according to the ratings shown by the Java Posse during the closing keynote:

JavaPosse-TopRatedTalks

Hm, these are Devoxx alternative facts, so maybe they’re alternative ratings as well.

There is a YouTube playlist of all Devoxx US 2017 sessions, so if you missed anything you can always go back and replay it.

Advertisements

This evening, I presented Collections Refueled at the Silicon Valley JUG. Thanks to the JUG for having me, and to the attendees for all the interesting questions!

Here are the slides for my presentation: CollectionsRefueled.pdf

 

The first segment of Episode 23 of the Java Off-Heap podcast covered the deprecation of Object.finalize in Java 9 and deprecation and finalization in general. Deprecation is a subject near and dear to my heart. The hosts even mentioned me by name. Thanks for the shout-out, guys!

I wanted to clarify a few points and to answer some of the questions that weren’t resolved in that segment of the show.

Java Finalizers vs. C++ Destructors

The role of Java’s finalizers differs from C++ destructors. In C++ (prior to the introduction of mechanisms like shared_ptr) anytime you created something with new in a constructor, you were required to call delete on it in the destructor. People mistakenly carried this thinking over to Java, and they thought that it was necessary to write finalize methods to null out references to other objects. (This was never necessary, and the fortunately the practice seems to have died out long ago.) In Java, the garbage collector cleans up anything that resides on the heap, so it’s rarely necessary to write a finalizer.

Finalizers are useful if an object creates resources that aren’t managed by the garbage collector. Examples of this are things like file descriptors or natively allocated (“off-heap”) memory. The garbage collector doesn’t clean these up, so something else has to. In the early days of Java, finalization was the only mechanism available for cleaning up non-heap resources.

Phantom References

The point of finalization is that it allows one last chance at cleanup after an object becomes unreachable, but before it’s actually collected. One of the problems with finalization is that it allows “resurrection” of an object. When an object’s finalize method is called, it has a reference to this — the object about to be collected. It can hook the this reference back into the object graph, preventing the object from being collected. As a result, the object can’t simply be collected after the finalize method returns. Instead, the garbage collector must run again in order to determine whether the object is truly unreachable and can therefore be collected.

The reference package java.lang.ref was introduced all the way back in JDK 1.2. This package includes several different reference types, including PhantomReference. The salient feature of PhantomReference is that it doesn’t allow the object to be “resurrected.” It does this by making the contained reference be inaccessible. A holder of a phantom reference gets notified that the referent has become unreachable (strictly speaking, phantom-reachable) but there’s no way to get the referent out and hook it back into the object graph. This makes the garbage collector’s job easier.

Another advantage of a PhantomReference is that, like the other reference types, it can be cleared explicitly. Suppose there’s an object that holds some external resource like a file descriptor. Typically, such objects have a close method the application should call in order to release the descriptor. Prior to the introduction of the reference types, such objects also need a finalize method in order to clean up if the application had failed to call close. The problem is, even if the application has called close, the collector needs to do finalization processing and then run again, as described above, in order to collect the object.

PhantomReference and the other reference types have a clear method that explicitly clears the contained reference. An object that has released its native resources via an explicit call to a close method would call PhantomReference.clear. This avoids a subsequent reference processing step, allowing the object to be collected immediately when it becomes unreachable.

Why Deprecate Object.finalize Now?

A couple of things have changed. First, JEP 277 has clarified the meaning of deprecation in Java 9 so that it doesn’t imply that the API will be removed unless forRemoval=true is specified. The deprecation of Object.finalize is an “ordinary” deprecation in that it’s not being deprecated for removal. (At least, not yet.)

A second thing that’s changed in Java 9 is the introduction of a class java.lang.ref.Cleaner. Reference processing is often fairly subtle, and there’s a lot of work to be done to create a reference queue and a thread to process references from that queue. Cleaner is basically a wrapper around ReferenceQueue and PhantomReference that make reference handling easier.

What hasn’t changed is that for years, it’s been part of Java lore that using finalization is discouraged. It’s time to make a formal declaration, and the way to do this is to deprecate it.

Has Anything Ever Been Removed from Java SE?

The podcast episode mentioned a Quora answer by Cameron Purdy written in 2014, where he said that nothing had ever been removed from Java. When he wrote it, the statement was correct. Various features of the JDK had been removed (such as apt, the annotation processing tool), but public APIs had never been removed.

However, the following six APIs were deprecated in Java SE 8, and they have been removed from Java SE 9:

  1. java.util.jar.Pack200.Packer.addPropertyChangeListener
  2. java.util.jar.Pack200.Unpacker.addPropertyChangeListener
  3. java.util.logging.LogManager.addPropertyChangeListener
  4. java.util.jar.Pack200.Packer.removePropertyChangeListener
  5. java.util.jar.Pack200.Unpacker.removePropertyChangeListener
  6. java.util.logging.LogManager.removePropertyChangeListener

In addition, in Java SE 9, about 20 methods and six modules have been deprecated with forRemoval=true, indicating our intent to remove them from the next major Java SE release. Some of the classes and methods to be removed include:

  • java.lang.Compiler
  • Thread.destroy
  • System.runFinalizersOnExit
  • Thread.stop(Throwable)

The modules deprecated for removal are the following:

  1. java.activation
  2. java.corba
  3. java.transaction
  4. java.xml.bind
  5. java.xml.ws
  6. java.xml.ws.annotation

So yes, we are getting serious about removing stuff!

Will Finalization Be Removed?

As mentioned earlier, Object.finalize is not being deprecated for removal at this time. As such, its deprecation is merely a recommendation that developers consider migrating to alternative cleanup mechanisms. The recommended replacements are PhantomReference and the new Cleaner class.

That said, we do eventually want to get rid of finalization. It adds extra complexity to the garbage collector, and there are recurring cases where it causes performance problems.

Before we can get rid of it, though, we need to remove uses of it from the JDK. That’s more than just removing the overrides of finalize and rewriting the code to use Cleaner instead. The problem is that there are some public API classes in the JDK that override finalize and specify its behavior. In turn, their subclasses might override finalize and rely on the existing behavior of super.finalize(). Removing the finalize method would expose these subclasses to a potentially incompatible behavior change. This will need to be investigated carefully.

There might also be a transition period where calling of the finalize method is controlled by a command-line option. This would allow testing of applications to see if they can cope without finalization. Only after a transition period would we consider removing the finalization mechanism entirely. We might even leave the finalize method declaration around for binary compatibility purposes, even after the mechanism for calling it has been removed.

As you can see, removing finalization would require a long transition period spanning several JDK releases, taking several years. That’s all the more reason to start with deprecation now.

Now that Devoxx US is imminent, it’s about time for me to post about Devoxx BE 2016, which took place in November 2016 in Antwerp. That was several months ago, which was ages in conference time, so this post is mainly a placeholder to host slides and links to the videos.

Array Linked to a List, the Full Story! – José Paumard (video)

I was surprised to find that I was mentioned by name in the abstract for this university session. José Paumard took a tweet of mine from a year earlier (actually one by my alter ego, Dr Deprecator) and turned it into an entire university session. José was happy to have me attend the session, and he was gracious enough to invite me on stage for a few comments and questions.

Streams in JDK 8: The Good, The Bad and the Ugly – Simon Ritter (BOF)

This was another of my impromptu appearances. Simon had submitted this session, and he asked me to join him in presenting it. I said that I wasn’t sure what I would speak about. He said to me, “I’ll put up a slide and say a few words about it. I’m sure you’ll have an opinion.” (He was right.) This was a BOF, so it was pretty informal, but Simon came up with some really interesting examples, and we had a good discussion and covered a lot of issues.

Simon and I will be repeating this BOF at Devoxx US this coming week.

Ask the JDK Architects – panel session (video)

This was a panel session featuring Mark Reinhold and Brian Goetz (the actual JDK architects) along with Alan Bateman and myself (JDK Core Libraries engineers). This session consisted entirely of answering questions from the audience.

Optional: The Mother of All Bikesheds – conference session (slides, video)

This was a conference session about a single Java 8 API, java.util.Optional. Some were were skeptical that I could talk for an entire hour about a single API. I proved them wrong. Credit for the title goes to my übermanager at Oracle, Jeannette Hung. It refers to the many protracted mailing list discussions (“centithreads”) about the design of Optional.

Thinking in Parallel – joint conference session with Brian Goetz (slides, video)

This was an amazing experience because the auditorium was so full that people were sitting on the steps. Brian Goetz was the big draw here, but I also think it was packed because there were fewer sessions running at the same time.

* * *

I was pleased to learn that both of my conference sessions were in the top 20 talks for the conference. Thanks for your support!

I just finished a vJUG24 session entitled Optional: The Mother of All Bikesheds.

Video: YouTube

Slide deck: PDF

This slide deck has a few minor updates relative to what I presented in the vJUG24 session:

  • slides 21-22: clarify problem statement (the before and after code is correct)
  • slide 26: mention flatMap() for completeness
  • slide 31: add link to Stack Overflow question
  • slide 36: clarify reason for not deprecating Optional.get()
  • slide 42: new slide describing new methods in Java 9

For convenience, here are the six seven style rules I proposed in the session:

  1. Never, ever, use null for an Optional variable or return value.
  2. Never use Optional.get() unless you can prove that the Optional is present.
  3. Prefer alternative APIs over Optional.isPresent() and Optional.get().
  4. It’s generally a bad idea to create an Optional for the specific purpose of chaining methods from it to get a value.
  5. If an Optional chain has a nested Optional chain, or has an intermediate result of Optional, it’s probably too complex.
  6. Avoid using Optional in fields, method parameters, and collections.On a related note, I thought of another rule after I presented the session:
  7. Don’t use an Optional to wrap any collection type (List, Set, Map). Instead, use an empty collection to represent the absence of values.

JavaOne 2016 Sessions

Here are links to the JavaOne pages, slide decks, and videos for the sessions I presented at JavaOne 2016.

Collections Refueled

Enhanced Deprecation

Thinking in Parallel (with Brian Goetz)

Occasionally I see mention of a “fail-safe” Iterator in Java. What does this mean?

The context here is what happens when a collection is modified while it’s being iterated, or “concurrent modification” for short. Each collection defines its own policy for handling concurrent modification. Note that the issue is when the collection is modified directly, from “outside” the iterator. Modifying the collection via the iterator is generally permitted, with some specific exceptions.

The term “fail-safe” is not used anywhere in the Java SE specification. Instead, the specification describes four different policies for concurrent modification: fail-fast, weakly consistent, snapshot, and undefined.

1. fail-fast

Most of the non-concurrent collections in java.util have fail-fast iterators. See the ArrayList specification for example:

The iterators returned by this class’s iterator and listIterator methods are fail-fast: if the list is structurally modified at any time after the iterator is created, in any way except through the iterator’s own remove or add methods, the iterator will throw a ConcurrentModificationException. Thus, in the face of concurrent modification, the iterator fails quickly and cleanly, rather than risking arbitrary, non-deterministic behavior at an undetermined time in the future.

2. weakly consistent

Most of the concurrent collections in java.util.concurrent, such as ConcurrentHashMap, are weakly consistent. The definition from the java.util.concurrent package specification is:

Most concurrent Collection implementations (including most Queues) also differ from the usual java.util conventions in that their Iterators and Spliterators provide weakly consistent rather than fast-fail traversal:

  • they may proceed concurrently with other operations
  • they will never throw ConcurrentModificationException
  • they are guaranteed to traverse elements as they existed upon construction exactly once, and may (but are not guaranteed to) reflect any modifications subsequent to construction.

3. snapshot

A third policy is snapshot semantics. The main example of this is CopyOnWriteArrayList:

All mutative operations (add, set, and so on) are implemented by making a fresh copy of the underlying array.

The “snapshot” style iterator method uses a reference to the state of the array at the point that the iterator was created. This array never changes during the lifetime of the iterator, so interference is impossible and the iterator is guaranteed not to throw ConcurrentModificationException. The iterator will not reflect additions, removals, or changes to the list since the iterator was created.

4. undefined

The fourth policy, if it can be called a policy, is that nothing is specified at all. The results of modifying a collection during iteration are undefined and may result in inconsistencies. The examples here are the legacy collections Vector and Hashtable and their methods that return Enumeration, including Vector.elements, Hashtable.elements, and Hashtable.keys.

If you iterate an Enumeration from the Vector.elements method, it’s not difficult to get odd behaviors such as elements being skipped, elements appearing twice in the iteration, or getting an exception unexpectedly.

As an aside, when the Collections Framework was introduced in JDK 1.2, Vector and Hashtable were retrofitted to implement the new interfaces. For example, if you have a Vector, you can get an Enumeration using the Vector.elements method, or you can get an Iterator using the Vector.iterator method. The Iterator has a fail-fast policy, while the Enumeration does not. Hashtable also provides both an Enumeration and an Iterator, and only the latter has the fail-fast policy. What’s interesting is that Hashtable‘s Iterator and Enumeration are implemented by the same class which has a flag that determines whether it should behave as an Iterator or Enumeration.

After all this, where does “fail-safe” come into the picture? Answer: it doesn’t. The words “fail-safe” are never used in the Java SE specifications that describe the concurrent modification policy of a collection. As such, there is no reliable, consistent definition of “fail-safe” for an Iterator. One can attempt to apply the general concept of “fail-safety” to an Iterator, but this is open to varying, misleading, and even contradictory interpretations.

Don’t use “fail-safe” to describe a Java Iterator. Instead, use one of the documented policies listed above.