Java 8 : Imperative vs Declarative style of coding

The focus of this article is to compare the imperative style of solving a problem vs declarative style using a couple of examples.It assumes basic knowledge about Lambdas and Streams introduced in Java 8.

Example 1:   I do not wish to give you the problem statement on purpose. Look at the code below and try to figure out the intention of the code.


List<Player> validPlayers = new ArrayList<>();
FootballLeague premierLeague = null;
for(FootballLeague league:leagues)
{
if(league.getLeagueName() == LeagueName.PREMIER_LEAGUE )
{
premierLeague = league;
break;
}
}
if(null != premierLeague)
{
List<Player> players = premierLeague.getPlayers();
for(Player player: players)
{
if(player.getAge() <= 25)
{
validPlayers.add(player);
}
}
}
Collections.sort(validPlayers, new Comparator<Player>() {
@Override
public int compare(Player p1, Player p2 ) {
return p1.getName().compareTo(p2.getName());
}
});
StringBuilder builder = new StringBuilder();
for(Player player : validPlayers)
{
builder.append( player.getName()+ ",");
}
return builder.substring(0,builder.length()-1);

I am sure you read that code very carefully and possibly a couple of times. What does it do? You might read it again ! Most developers have a short memory. You know that you might have to revisit this code at some point and hence you will probably add comments in the code above like sort by name , remove last comma etc.

From a list of football leagues, the code checks if the league is the English Premier league, selects all the players who play for the English premier league where their age is <= 25 years. It then sorts them by their names and then finally returns a comma separated list of their names.

So our problem statement:  Given a list of football leagues, select all the players who play for the English Premier League and are <=25 years of age. Then sort their names and return them as comma separated.  Let us break this problem statement about what we want to do:

  1. Select all the players who play in the Premier league
  2. Remove all the players who are above 25 years of age
  3. Sort the names of the players
  4. Get their name and make them comma separated

Let us code this now using declarative style.

Declarative style:


return leagues.stream()
.filter(league -> league.getLeagueName() == LeagueName.PREMIER_LEAGUE)
.flatMap(league ->league.getPlayers().stream())
.filter(player -> player.getAge() <=25)
.sorted(Comparator.comparing(Player::getName))
.map(player ->player.getName())
.collect(Collectors.joining(","));

Look at the code above and the problem statements from 1-4. Don’t you think that the code above reads more like the problem statement? Not only does it read like the problem statement, the code does not have any garbage variables, no for loops instructing how we want to iterate and select. We are focusing more on what we want rather how we want to do it. Hence it is also easier to make any changes to the code in case the requirements change which means the maintenance is easy.  

We could have used a .sorted(Comparator.naturalOrder()) above and done the mapping to get the name first before sorting.


return leagues.stream()
.filter(league -> league.getLeagueName() == LeagueName.PREMIER_LEAGUE)
.flatMap(league ->league.getPlayers().stream())
.filter(player -> player.getAge() <=25)
.map(player ->player.getName())
.sorted(Comparator.naturalOrder())
.collect(Collectors.joining(","));

Example 2:  Given a list of strings,we need to output the strings based on their length. If the input string is “I”, ”ate”, ”food”, ”slept”. The output should be

1 – I

3-ate

4-food,slept

Imperative style:


Map<Integer, List<String>> lengthMap = new HashMap<>();
for(String string : strings)
{
List<String> sameLength = null;
Integer length = string.length();
if(lengthMap.get(length) == null)
{
sameLength = new ArrayList<>();
sameLength.add(string);
lengthMap.put(length, sameLength);
}else
{
List<String> sameLengthString = lengthMap.get(length);
sameLengthString.add(string);
}
}
return lengthMap;

Hmm, one thing is clear that we need a Hashmap. The key will be the length of string and the value will be the actual string. So what are we doing here

  1. Create a new hashmap
  2. Iterate through the list of strings
  3. Create  a new list
  4. If the current string has length x and x is already in the map then get the list corresponding to the length and add the current string to that list, else
  5. Create a new list and add the length of string and the string length as key , value as actual string.
  6. Done !

In short, we are grouping strings by their length.

Declarative style:


return strings.stream()
.collect(Collectors.groupingBy(String::length)));

What are we doing here:

  1. Group all the strings by length
  2. Done !

Again, the code is expressive and it translates to the problem statement very clearly !

The Collectors.groupingBy method returns a Map of Integer, List<String> or K, V where

K- the type that we are grouping by

V – The elements that we are operating on ( in this case, String)

Note that we have not considered the case where strings can repeat. This can be solved easily by using a Set instead of a List which would be pretty easy in both imperative and declarative style.

Declarative style does make your code more readable and concise.Declarative code might not be the magical solution to all problems, if you find the declarative style getting too complicated, stick to the old imperative style of programming.

Sorting a list in Java with null values

I am going to tell you a story with 2 characters, the developer and the quality analyst(QA). The developer is working on  a very simple application which deals with teenagers and the brand of cell phone they use.

I think the relation between developers and QA is like a one night stand, well, not the typical one! One fine day the QA finds bugs and thanks to your manager (who leaves home early), the developer and the QA are made to stay back and work all night until the bug is fixed and the code is completely bug free.(Yeah, right !)

Our developer friend has written a Teenager class which has an age field and a phoneBrand field.


public class Teenager {
private int age;
private String phoneBrand;
//getters,setters
}

view raw

Teenager.java

hosted with ❤ by GitHub

Manager: We have a new requirement,there is a need for a survey and hence the application needs to return the list of teenagers by increasing age. If two teenagers have the same age, then we need to return the list considering the brand of the cell phones in alphabetical order.

Developer: This will take time. I need to make sure that I write the code using Object Oriented Principles.

Manager: Alright, I give you a days time.( You know he does not understand Object oriented principles !)

You bring all your Object oriented skills to the table, create a separate class for the comparator as the sorting is done using 2 different fields, the age field and then the phoneBrand field. The comparator class looks like this:


public class TeenageSorter implements Comparator<Teenager>{
@Override
public int compare(Teenager t1, Teenager t2)
{
if( t1.getAge() == t2.getAge())
{
return (t1.getPhoneBrand().compareTo(t2.getPhoneBrand()));
}
return t1.getAge() > t2.getAge()? 1 : –1;
}
}

The input to your sort function is a List of Teenagers. You call the sort method like this:

Collections.sort(teens, new TeenageSorter());

Everything works fine.Given an input like this:


List<Teenager> teens = new ArrayList<>();
teens.add(new Teenager(15,"Apple"));
teens.add(new Teenager(16,"Samsung"));
teens.add(new Teenager(14,"Xiaomi"));
teens.add(new Teenager(16,"Apple"));

Output:

[Age:14 |Phone:Xiaomi, Age:15 |Phone:Apple, Age:16 |Phone:Apple, Age:16 |Phone:Samsung]

Sorted by increasing age and as there are 2 teenagers aged 16, the one with the Apple phone is followed by Samsung using alphabetical order. You check in the code and inform your manager that you are done with your task.

A few hours later, at around 6 pm when you are just about to leave, the QA guy comes to your desk.

QA : Your code is not working .

Developer : That is not possible ![Unpleasant stare]

QA : I am not sure, I added a teenager but did not give him a phone !

You think about it for a second, your coding skills (spelt  “ego”) are at stake, you have to come up with a reply.You know you left that part in the code but…

Developer: How can a teenager not have a phone ! [ You know in your heart that you never had a cell phone as a teenager]

QA : Well, yes, all teenagers do but currently our system allows me to do that, I followed the steps, kept the brand name of the phone empty, it crashed !

Well, you take a deep breath.You know you assumed that the data will never be null. You got what you deserved , a null pointer exception !

You go to your manager and tell him that it is possible that a teenager might not have a phone, so the requirement has to be discussed now. You involve the business analyst as well and explain the situation. The analyst says, well, if a teenager does not have a cell phone and two teenagers have the same age, we need to put them at the end of the list.

You have been asked to fix this in the next few hours.You start thinking about this problem. If the teenager does not have a phone, it means the phoneBrand field is null. Hence the sorting has to now deal with nulls.You look at your Comparator and this is what you come up with:


public class TeenageSorter implements Comparator<Teenager>{
@Override
public int compare(Teenager t1, Teenager t2) {
if( t1.getAge() == t2.getAge()){
return sortByPhoneBrand(t1.getPhoneBrand(),t2.getPhoneBrand());
}
return t1.getAge() > t2.getAge()? 1 : –1;
}
private int sortByPhoneBrand(String phoneBrand1,String phoneBrand2){
if(phoneBrand1 == null && phoneBrand2 == null){
return 0;
}
if(phoneBrand1 == null && phoneBrand2 != null){
return 1;
}
if(phoneBrand1 != null && phoneBrand2 == null){
return1;
}
return phoneBrand1.compareTo(phoneBrand2);
}
}

A shiny new method to sort the phoneBrand. This time when you test your code, you supply the following input:


List<Teenager> teens = new ArrayList<>();
teens.add(new Teenager(15,"Apple"));
teens.add(new Teenager(16,"Samsung"));
teens.add(new Teenager(14,"Xiaomi"));
teens.add(new Teenager(14,"Samsung"));
teens.add(new Teenager(16,"Apple"));
teens.add(new Teenager(16,null));

Output:

[Age:14 |Phone:Samsung, Age:14 |Phone:Xiaomi, Age:15 |Phone:Apple, Age:16 |Phone:Apple, Age:16 |Phone:Samsung, Age:16 |Phone:null]

The list above is sorted by increasing age. When the age is the same,16, the teenager without the phone gets pushed to the end of the list as if the teenager is not wanted. You check in the code and leave for the day (night?)  and you are still feeling bad that the teenager does not have a phone.

The next morning, the QA does his testing, things are working fine!

QA : Everything is fine now.

Developer : Yes, I know !  [Arrogance is the key, isn’t it ! ]

Is there a chance to refactor your code ? You look at your code and ask yourself if you can refactor that code somehow. The comparator methods do not really look that clean. It is kind of buggy. It is not that clear when nulls starting popping up, should you return a 1 or a -1 ? It is more like trial and error at times. Imagine what happens if you are asked to sort by another field in addition to the 2 fields above? What if we add another field, earphoneBrand. If two teens are aged the same, have the same phone then sort by the brand of the ear phones. What if someone does not have earphones ?

You remember reading about Lambdas in Java 8 and start exploring if they made any changes to the Comparator interface. On further reading, you actually discover a lot and come up with the following analysis:

  1. Comparator present in the java.util.package is a functional interface and hence..
  2. Usage of the anonymous inner class can be replaced with usage of Lambdas.
  3. There are default methods in the Comparator interface now which can also sort objects if they are null.

So let us revisit the problem statement:

Given a list of Teenagers:

a) sort the list by age first

b) if age is the same then sort by phone brand name and

c) if the teenager does not have a phone(null), then it should be kept at the end for that particular age.

Declarative style:


Collections.sort(teens,
Comparator.comparing(Teenager::getAge)
.thenComparing(Teenager::getPhoneBrand,Comparator.nullsLast(Comparator.naturalOrder())));

What ? What is that ? Let us break that down a little bit, following a, b and c above.


Comparator<Teenager> phoneBrandPossibleNulls =
Comparator.comparing(Teenager::getPhoneBrand,Comparator.nullsLast(Comparator.naturalOrder()));
Collections.sort(teens, Comparator.comparing(Teenager::getAge)
.thenComparing(phoneBrandPossibleNulls));

The code on lines 1 and 2 creates a comparator to sort by the phone brand and instructs how to deal with null values. Line 4 combines the two, if the age is the same then compare with the phone brand. The part, Comparator.nullsLast(Comparator.naturalOrder()) means that when we get 2 phoneBrand objects, we can have various combinations. Both null, one of them null and none of them null. The code says, use natural order for sorting phoneBrand fields and if you encounter a null, then put them at the end of the list.

I must admit the code does need a little bit of explanation this time, but it is still expressive enough and probably takes getting used to the function composition part. But the code flows well, any kind of changes to it are quite easy rather than the imperative style. The focus is more on what you want. If you get a new requirement of putting the teenager without a phone at the beginning of the list, all we need to do is modify the code to use Comparator.nullsFirst above. If we need to sort by another field, add another thenComparing().

You are so enthusiastic about Java 8 but then it dawns upon you that your manager has not yet approved that you switch to Java 8 and you are still coding with an old version. Well, send him an email and tell him that you feel like this:

outdated1

 

Well, if you are still stuck with prior versions of Java , there are other libraries to do that. One solution is to use the Google Guava libraries. The Google Guava library has something similar to the Java 8 style of composing functions.It uses chaining of comparators like above.


Collections.sort(teens,new Comparator<Teenager>() {
@Override
public int compare(Teenager t1, Teenager t2)
{
return
ComparisonChain.start()
.compare(t1.getAge(), t2.getAge())
.compare(t1.getPhoneBrand(), t2.getPhoneBrand(),Ordering.natural().nullsLast())
.result();
}
});

This chaining is not very different from the Java 8 style, in fact a few things in the Google Guava library have served as an inspiration to the designers of the JDK library. So if you cannot switch to Java 8 yet, you could use the google guava version, it is much cleaner that writing a comparator of your own with all null checks and other checks.

It has been a couple of days since you checked in the code, things seem quiet, the QA, Business Analyst and the Manager, nobody has turned up at your desk. It means everything is working fine !

Introduction to JShell : The Java Shell

In this article we will be taking a look at the new tool, JShell, which is a part of the Java 9 release.

What is JShell ?

It is an interactive tool or shell also referred to as REPL ( Read- Evaluate-Print loop) that is packaged as part of  Java 9 release. REPL is basically Reading input or data typed by the user, Evaluating the input , Printing the result that was evaluated and then going back to read the next input(Looping).

Why JShell ?

It is a very handy tool for developers both old and new wanting to explore the Java language by trying or experimenting snippets of code.

First time Java developers who wish to get started with the Java language can use JShell to write small pieces of code and later on move to IDE’s like Eclipse, STS, IntelliJ to set up full fledged projects.

For the experienced Java developer who wants to explore a new API or try out short  examples, the JShell can be quite useful. Let us say you are in the midst of fixing a bug or implementing a new functionality in an existing class using an API you have not tried before. Typical way to do this would be to create a new class with a main method in your code repository, type in the sample code for the API, right-click and run. Finally, we would copy the code and paste it somewhere in the existing code. As we age, we tend to forget a few things like deleting the sample file with the main method or sometimes we might even create a main method in an existing class where the new code is needed. While checking in, we might just check in the file with the main method only for your lead to do a code review which will obviously be rejected. This is assuming your lead does a proper code review !

Starting JShell

Open a command prompt, go to your  <Jdk 9  Installation>/bin directory and type jshell. You can also add it to your environment variables so that it can be started from any directory on the command prompt.

Some of the things we can do with JShell

  • Additionaddition

You don’t need JShell to check that 2+3=5. However make note of the variable created above , $1. This is a temporary variable. But, did you realize that we did not write a class with a main method with a System.out.println to print the result of 2+3 ?

  • Exploring Lambdas, the shiny new feature – Wait a minute, these are more like shiny old features introduced in Java 8!Consumer_Lambda

Did you notice the nice lambda created above ?

  • Can’t remember the method in the Consumer interface?Consumer_Lambda_Tab

Type in the name of the variable(s) created above , followed by a dot and press tab ! You get a bunch of methods that can be called on the Consumer interface. For the Consumer interface, it is the accept method that we are interested in.

  • Let’s try something with Streamsstreams

The moment you create a Stream which is referenced by the strings variable above, notice the type of strings. It is a ReferencePipeLine, JShell returns the type of the variable too!

  • Create a class, declare methods and variablesclass

Notice that after typing class Test { and pressing enter, JShell is intelligent enough to figure out that you are not yet done with the current statement and hence it waits for the next line of code.

  • Listing out variables declared so farvars

Get a list of all the variables using the /var command. To get a list of all the methods, use the /methods command.

  • Listing all the code typed in so farlist

Use the /list command. We performed an addition, created a Lambda , a Stream , created a class, then an object and called the getValue method. The /list command, simply lists them.

  • Typed in some code, time for a tea break, worried about machine crash ?save

You can save your code snippets using the /save command. Make sure you have the right permissions to save/create a file. You can follow that up with a /open command which will simply execute the code in the file.

  • Made a mistake, want to edit something ?editTyping the /edit command as shown above opens up a window where you can go and edit the code you typed. Using our knowledge of method reference, we can change the lambda expression from (String s) -> System.out.println(s) to System.out::println. On clicking the accept button, the change is reflected.

editpost_edit

Notice that the /vars command now lists the updated reference to the Lambda expression.

  • Create a package

Come on , JShell is not for that. The entire thing is just in a package. Now, don’t whine that JShell does not allow you to create packages. It is not meant to do that !

  • Want to quit JShell ?

Inform JShell by typing  /exit. JShell politely says GoodBye to you. Well, you could be a little rude and press Ctrl-D to exit JShell, in that case don’t expect JShell to say Goodbye!

Conclusion

JShell is a great addition to the JDK 9 release. Use it to experiment snippets of code and get immediate feedback. This article was just a quick introduction. JShell also has an API which can be used to integrate it into an IDE or a tool.

JDK 9 reached GA on 21st September 2017, to take a look at all the features, visit the openjdk site here.

Java 8 map and flatMap

In this post, we will be taking a look at the map and flatMap methods introduced in Java 8. We will be looking at a scenario where the map method does not produce the required output and why we really need the flatMap method. I will be using multiple examples to illustrate the same and also show you imperative and declarative styles of coding.

map: Let us consider the scenario where we have a collection of Person objects. Given a list of person objects, what if we wanted to return only the names of the people who have UK citizenship.

The imperative style of writing code would give us the following:


public List<String> getNamesOfUKCitizens(List<Person> persons)
{
List<String> allUKCitizens = new ArrayList<>();
for(Person p : persons)
{
if(p.getCitizenship() == Citizenship.UK){
allUKCitizens.add(p.getName());
}
}
return allUKCitizens;
}

To get this, we have to initialize an empty list, iterate through the loop, filter the people who have UK citizenship and then add the names to the empty list we create.Let’s try and solve this the  declarative way, using the map method.

The map method basically takes an object of one type and gives us an object of another type.


public List<String> getNamesOfUKCitizens()
{
return persons.stream()
.filter( p -> p.getCitizenship() == Citizenship.UK)
.map(p -> p.getName())
.collect(Collectors.toList());
}

Map_Flatmap_Pic1
filter,map and collect – Once the knob is turned on, everything happens !

The signature of the map method:

<R> Stream<R> map(Function<? super T,? extends R> mapper)

The signature looks complicated but it is easy to understand. The R is the output type of the stream and T is the input type.

Remember how the filter method (explained here) took a Predicate as a parameter? The map method takes a Function. This is also a functional interface. The function gets applied to each element. The p -> p.getName is a lambda expression, the function that gets applied to the Person object.  To understand this better, we could write the same thing as follows:


public List<String> getNamesOfUKCitizens()
{
Function<Person,String> mapper = p -> p.getName();
return persons.stream()
.filter( p -> p.getCitizenship() == Citizenship.UK)
.map(mapper)
.collect(Collectors.toList());
}

Remember that the map method takes one object and returns exactly one object. This is 1-1.

Few more examples of map API:

1.Given a list of numbers, we want to generate the square of each number:

We have input as a list of numbers of 1 type and we want to transform it:

Input: List<Integer> numbers = Arrays.asList(1,2,3,4,5);


numbers.stream()
.map( number -> number * 2)
.collect(Collectors.toList());

 Output:

[2, 4, 6, 8, 10]

2. Given a list of string, convert each string to upper case:

Transformation of one type to another, this time String to String – use map function

Input:

List<String> strings = Arrays.asList(“abc”,”pqr”,”xyz”);


strings.stream()
.map(s -> s.toUpperCase())
.collect(Collectors.toList()));

Output:  [ABC, PQR, XYZ]

3.Given a list of string, find the length of each string:

Input:

List<String> strings = Arrays.asList(“abc”,”pqr”,”xyz”);

Input to the map is a string and output is the length of each string.


strings.stream()
.map(s -> s.length())
.collect(Collectors.toList()));

Output: [3, 3, 3]

FlatMap: To understand the flatMap, let us consider a different example. Let us consider there are 3 systems and each system returns a list of strings. There is an application which combines these lists and sends the data back to us. Our system needs to take this input and generate all the strings as a single output.

List<String> system1 = Arrays.asList(“AB”,”cd”,”EF”);

List<String> system2 = Arrays.asList(“GH”,”ij”);

List<String> system3 = Arrays.asList(“kl”,”MN”,”op”);

//Combination

List<List<String>> input = Arrays.asList(system1,system2,system3);

Attempt 1: The input type is a List of List<String>.  We want to get all the strings from this input. We know that the map function helps us to transform an object. Will it help us here?  Let us take this step by step.


input.stream()
.map(list -> list)
.forEach(System.out::println)

The call to the input.stream()  returns a Stream<List<String>>

This gives an output:

[AB, CD, EF]

[GH, ij]

[kl, MN, op]

When we apply the map function, each time we are getting a list. But we need the individual elements in that list as a combined result. How do we get that?

When we have a single list as shown below and we applied the stream() to method to it, what happened?

List<String> strings = Arrays.asList(“A”,”b”,”C”);

strings.stream()

              .forEach(System.out::println);

This gave us the individual elements in that stream. So will applying the stream method to the list above solve the issue? Let’s try

Attempt 2:  Applying a stream to the list and using a map


input.stream()
.map(list -> list.stream())
.forEach(System.out::println)

This gives a weird output like this:

java.util.stream.ReferencePipeline$Head@87aac27

java.util.stream.ReferencePipeline$Head@3e3abc88

java.util.stream.ReferencePipeline$Head@6ce253f1

This gives us a stream of objects. So the usage of the map method in this scenario is not right. This is because the map method as mentioned earlier takes an input and produces one output. But in this case, the map method takes a list and we want the individual elements of that list to be combined together. This is not what the map function does. We need to use a different function.

Attempt 3: Using a flatMap


input.stream()
.flatMap(list -> list.stream())
.forEach(System.out::println);

This gives us the required output:

AB

cd

EF

GH

ij

kl

MN

Op

Let us break flatMap up into 2 operations:

map:

[ [AB, CD, EF]      [GH, ij]           [kl, MN, op] ]

  Stream 1             Stream 2           Stream 3

flatten it:

AB, cd, EF             GH, ij               kl ,MN ,op

The flatMap() does a map + flat. Flattening here is of the individual streams from each item in the list to a single stream. The flatMap() methods needs a stream and hence the list->list.stream().The flatMap method takes an input and can give either 0 or more elements.

Let us consider another example to understand this well. Let us consider 3 different football leagues. The English Premier league, the LIGA BBVA or the Spanish League and the Bundesliga. Each league has a list of teams. We can represent this as:


public class League {
private String leagueName;
private List<Team> teams;
}

view raw

League.java

hosted with ❤ by GitHub


public class Team
{
private String name;
public Team(String teamName){
this.name=teamName;
}
}

view raw

Team.java

hosted with ❤ by GitHub

Problem: Given a list of leagues, we need to return the names of the all the teams in the leagues.

Imperative style:


private static List<String> displayAllTeamNamesImperative(List<League> leagues){
List<String> teamNames = new ArrayList<>();
for(League league : leagues)
{
List<Team> teams = league.getTeams();
for(Team team : teams)
{
teamNames.add(team.getName());
}
}
return teamNames;
}

Declarative style using map:

We have a list of leagues. When we call stream() on it, it will operate on a single League object. But a league has multiple teams in it.  If we try solving this using map() then we land up getting this:


leagues.stream()
.map(league -> league.getTeams().stream())
.forEach(System.out::println);

Output:

java.util.stream.ReferencePipeline$Head@87aac27

java.util.stream.ReferencePipeline$Head@3e3abc88

java.util.stream.ReferencePipeline$Head@6ce253f1

We land up getting 3 streams. This means that the map operation is not the right fit here. We need a function that can take these streams and combine each of these elements in these streams to a unified output.

This is the scenario for a flatMap() as we have a collection of collections. So the input is  a collection of leagues and each league has another collection which is team.


private static List<String> displayAllTeamNamesFunctional(List<League> leagues)
{
return leagues.stream()
.flatMap(league ->league.getTeams().stream())
.map( team ->team.getName())
.collect(Collectors.toList());
}

When you have a collection of collections and want a unified output, use the flatMap.

I hope you have understood the basics of both map and flatMap methods and the scenarios in which they should be applied.

 

Unspecified behavior of the Java Iterator

In this article we will take a look at how the behavior of the Iterator produces some surprising results if not used in the right away.

In my previous article, we saw different ways of  removing elements from an ArrayList. When we used the remove(Object o) method from the ArrayList while we traverse through the ArrayList, we get a ConcurrentModificationException. But is this always the case ?

Well, let’s consider the same example of removing a person below the age of 18 from an ArrayList but with a different data set.


package blog;
import java.util.ArrayList;
import java.util.List;
public class RemoveItemsEnhancedForModified {
private static List<Person> persons = new ArrayList<>();
private static void initialize() {
persons.add(new Person("Rahul", 25, Gender.MALE, Citizenship.INDIA));
persons.add(new Person("Sally", 21, Gender.FEMALE, Citizenship.USA));
persons.add(new Person("Ben", 35, Gender.MALE, Citizenship.CANADA));
persons.add(new Person("Wayne", 30, Gender.MALE, Citizenship.UK));
persons.add(new Person("ShaneYoung", 18, Gender.MALE, Citizenship.AUSTRALIA));
persons.add(new Person("Kimmo", 17, Gender.MALE, Citizenship.FINLAND));
persons.add(new Person("Simo", 17, Gender.MALE, Citizenship.FINLAND));
}
public static void main(String[] args) {
initialize();
removeIllegalUsingEnhancedForLoop();
System.out.println(persons);
}
private static void removeIllegalUsingEnhancedForLoop()
{
for (Person p : persons)
{
if (p.getAge() < 18)
{
persons.remove(p);
}
}
}
}

Most of the code remains the same, the only thing that has changed is the data. Notice the Person data on the 2nd last and last elements in the ArrayList.

persons.add(new Person("Kimmo", 17, Gender.MALE, Citizenship.FINLAND));
persons.add(new Person("Simo", 17, Gender.MALE, Citizenship.FINLAND));

Well, anyways, we should get a ConcurrentModificationException! Right ?

On running this the output :

[Rahul, Sally, Ben, Wayne, ShaneYoung, Simo]

Shocked ? No exception ! Not just that , Simo should have been filtered. Well, lucky Simo! He can still have beer even though he is 17.

So what is happening  ?  Why is an exception not thrown and why is the last element returned ?

The 2nd last element satisfies the filtering criteria , it gets deleted. When it gets deleted, the size of the ArrayList is reduced by 1. But, the combination of Iterating and calling remove(Object o) makes the Iterator to behave in a weird way.  When remove(Object o) is called, it makes a call to the fastRemove method which reduces the size of the list. Then there is a call to the hasNext() on the iterator to check if there are more elements to to traversed. The hasNext() looks as follows:

public boolean hasNext() {
    return cursor != size;
 }

The cursor is an index that points to the index of the next element that will be processed. It starts from 0. In our case when we delete the 2nd last element, the cursor value is 6.

Before deleting: (size =7 , cursor =6 at Simo)

Rahul
21
Sally
21
Ben
35
Wayne
30
ShaneYoung
30
Kimmo
17
Simo
  17

After deleting: (size =6 , cursor = 6 at Simo)

Rahul
21
Sally
21
Ben
35
Wayne
30
ShaneYoung
30
Simo
17

Original value of size in our list is 7. When 2nd last element(Kimmo) is deleted, size =6 , cursor = 6. The Iterator thinks all the elements have been visited. So after deleting, when hasNext() is called ( remember that an enhanced for loop still means that the Iterator is used for moving around), the above condition returns false and hence the next() is not called. The last element is completely skipped and included in the output ! Since the hasNext returns false, next() is not called and the exception is not thrown. It is the call to next() which checks for concurrent modification but it is never called.

final void checkForComodification() {
     if (modCount != expectedModCount)
    throw new ConcurrentModificationException();
 }

The Javadoc for Iterator remove method clearly states that the behavior of the iterator is unspecified if the underlying collection is modified while the iteration is in progress in any other way than by calling Iterator remove(). So the above behavior is pretty much unspecified but can catch you by surprise.

Conclusion

Read the Javadocs carefully!  I know you are religiously going to do that 🙂 !

If you are still using Java 7 or any version below that, don’t by pass the gate keeper, “The Iterator “, when simultaneously looping and deleting elements from an ArrayList. Use the remove() method from the Iterator.

Removing elements from a Java ArrayList

In this article we are going to look at the different ways of removing elements from a Java ArrayList. We will be looking at why some ways work, some might appear to work and some don’t. Finally we will look at the recommended way to remove elements from an ArrayList.

Let us assume that we have a list of Persons with name, age , gender and citizenship. Drinking below the legal age is considered a crime in most places. Let us say we want to remove these persons from the list assuming the legal age for drinking is > 18.

First Attempt – The Exceptional way

We can remove the desired elements from an ArrayList using the code below.


package blog;
import java.util.ArrayList;
import java.util.List;
public class RemoveItemsEnhancedFor
{
private static List<Person> persons = new ArrayList<>();
private static void initialize()
{
persons.add(new Person("Rahul", 25, Gender.MALE, Citizenship.INDIA));
persons.add(new Person("Sally", 21, Gender.FEMALE, Citizenship.USA));
persons.add(new Person("Ben", 35, Gender.MALE, Citizenship.CANADA));
persons.add(new Person("Wayne", 30, Gender.MALE, Citizenship.UK));
persons.add(new Person("Kimmo", 45, Gender.MALE, Citizenship.FINLAND));
persons.add(new Person("ShaneYoung", 17, Gender.MALE, Citizenship.AUSTRALIA));
}
public static void main(String[] args)
{
initialize();
removeIllegalUsingEnhancedForLoop();
System.out.println(persons);
}
private static void removeIllegalUsingEnhancedForLoop()
{
for (Person p : persons)
{
if (p.getAge() < 18)
{
persons.remove(p);
}
}
}
}

Take a look at the removeIllegalUsingEnhancedForLoop  method which uses the enhanced for loop. The initialize method adds some data to the ArrayList. Well, the code looks quite straightforward. But on running this, we get a ConcurrentModificationException.

Exception in thread "main" java.util.ConcurrentModificationException
 at java.util.ArrayList$Itr.checkForComodification(ArrayList.java:901)
 at java.util.ArrayList$Itr.next(ArrayList.java:851)
 at blog.RemoveItems.removeIllegalUsingEnhancedForLoop(RemoveItems.java:52)
 at blog.RemoveItems.main(RemoveItems.java:27)

The moment you see the word Concurrent , you are thinking, why I am getting this exception, I am not using threads ! Note that while iterating using the enhanced for loop we are also trying to remove elements, which means we are modifying the structure of the list while we are iterating. Keep in mind that we are using the <arrayList>.remove(Object o). 

But from a technical perspective, what is happening behind the scenes?

Even though we are using an enhanced for loop, there is an Iterator involved behind the scenes. The Iterator is an interface. ArrayList implements the Iterator interface via a private inner class, Itr. When we use the enhanced for loop, there is a call to a method iterator() in the ArrayList when the for loop is going to be executed.

public Iterator<E> iterator() {
 return new Itr();
 }

This Itr is the one that maintains the state of the ArrayList. It does this using 2 variables, one is the modCount and the other one is the expectedModCount. modCount is an instance variable which keeps track of the number of times the ArrayList is modified. So far we only added elements, 6 to be precise, so the modCount is 6. The Itr class maintains the expectedModCount. Since this Itr class implements Iterator interface, it provides implementation for next(), hasNext(), remove().  Note that the remove method here does not take any arguments.

When we say persons.remove(p), the ArrayList remove method which takes an object as a parameter is called and not the Itr remove(). This remove(Object o) method makes a call to fastRemove() which does increase the value of the modCount but does not touch the expectedModCount. When the element is deleted  in our case, the modCount becomes 7 but the expectedModCount is still 6. Even if you use an enhanced for loop, the elements in the ArrayList are iterated using the next, hasNext method. The next method is called as the iteration moves ahead. The next method has this piece of code :

public E next() 
 {
    checkForComodification();
    ......
 
 }
final void checkForComodification() {
 if (modCount != expectedModCount)
 throw new ConcurrentModificationException();
 }

The method next checks for modification by evaluating if modCount and expectedModCount are not equal. Since the expectedModCount has not been touched in the remove(Object o) of the ArrayList, we get this ConcurrentModificationException. In short, the ArrayList was changed without use of the Iterator.remove. It is the Iterator which maintains the internal state. We removed the element using <arrayList>.remove(Object o) leaving the Iterator puzzled, so the Iterator complains about ConcurrentModificationException. It complains that –  “Hey, you did not remove elements through me, I am the gate keeper and since you did not get your gate pass, I am going to throw an exception”.

So if this is not the right way of removing elements,  can we remove elements using an Iterator directly ?

Second Attempt – The Heavy duty Iterator way of removing elements

The Iterator has a remove method.We need to use this remove method. This is how it is done :


private static void removeIllegalUsingIterator()
{
Iterator<Person> personIterator = persons.iterator();
while (personIterator.hasNext())
{
if (personIterator.next().getAge() < 18)
{
personIterator.remove();
}
}
}

This works fine and we do not get any Exceptions. We are making use of personIterator.remove().

Why does this work ?

Itr inner class in the ArrayList implements Iterator interface.  It also maintains the correct state by increamenting modCount but also assigning expectedModCount to modCount. When we call remove() on the Iterator, not only does it increase the value of the modCount by 1 but it also does the following :

expectedModCount = modCount;

So the call to next() as shown above calls :

final void checkForComodification() {
 if (modCount != expectedModCount)
 throw new ConcurrentModificationException();
 }

This does not throw a exception as they are equal now. You got your gate pass and now the guardian of the ArrayList, the gatekeeper, the Iterator, is happy !

So obviously we must use the Iterator to remove the elements from an ArrayList while we are iterating. Well, true, if you are still stuck with Java 7. This brings us to the 3rd way of implementing removal.

Third attempt – Java 8 cute way of removing elements from an ArrayList


private static void removeIllegalUsingJava8()
{
persons.removeIf(person -> person.getAge() < 18);
}

Did you stare at the code a couple of times ? Have I missed out on showing you some code ? That is the code , yes it is ! Trust me!  You don’t ? Well, then go ahead and try it out.

The removeIf is a method added in Java 8. It has been added and has an implementation in the Collection interface. Yes, you read that right, Collection interface. It is a default method.

default boolean removeIf(Predicate<? super E> filter) {

}

This method is however overridden in the ArrayList class. It takes a Predicate which is a Functional interface. If you refer to the removeIllegalUsingJava8 method, we are passing a Lambda, person -> person.getAge() < 18. This is passed to the Predicate interface and then in the removeIf method, the test method is called. If the condition is satisfised, the element is removed. Finally the modCount is incremented to indicate that the ArrayList has gone through a modification.

So should we use this method now because it looks cute and concise ? Well that is a start. The code is short and concise, it clearly tells us it’s intention – Remove an element that satisfies this condition.

Not convinced about the cuteness – There is more to it !

The removeIf method is also more performant. What ? Cute and Efficient ? Hard to believe ? Well, the removeIf method removes and manages the elements in the ArrayList in a different was as compared to the Iterator way of removing and adjusting the elements in the ArrayList. The remove method using an Iterator uses System.arrayCopy and shifts the elements each time an element is removed from the list. This can considered as O(n^2).

The removeIf method take a different approach. It uses a BitSet class and does not use a System.arrayCopy. It creates a BitSet to maintain the index of the elements that need to be removed. It finally shifts the elements once rather than shifting them each time. So it also performs better than the Iterator remove and the performance is O(n).

Conclusion

The Java 8 removeIf way of implementing removal is not only cute but performs better. Beauty with Brains ? That is a lethal combination, I urge you to start using it!