Removing elements from a Java ArrayList

In this article we are going to look at the different ways of removing elements from a Java ArrayList. We will be looking at why some ways work, some might appear to work and some don’t. Finally we will look at the recommended way to remove elements from an ArrayList.

Let us assume that we have a list of Persons with name, age , gender and citizenship. Drinking below the legal age is considered a crime in most places. Let us say we want to remove these persons from the list assuming the legal age for drinking is > 18.

First Attempt – The Exceptional way

We can remove the desired elements from an ArrayList using the code below.

Take a look at the removeIllegalUsingEnhancedForLoop  method which uses the enhanced for loop. The initialize method adds some data to the ArrayList. Well, the code looks quite straightforward. But on running this, we get a ConcurrentModificationException.

Exception in thread "main" java.util.ConcurrentModificationException
 at java.util.ArrayList$Itr.checkForComodification(ArrayList.java:901)
 at java.util.ArrayList$Itr.next(ArrayList.java:851)
 at blog.RemoveItems.removeIllegalUsingEnhancedForLoop(RemoveItems.java:52)
 at blog.RemoveItems.main(RemoveItems.java:27)

The moment you see the word Concurrent , you are thinking, why I am getting this exception, I am not using threads ! Note that while iterating using the enhanced for loop we are also trying to remove elements, which means we are modifying the structure of the list while we are iterating. Keep in mind that we are using the <arrayList>.remove(Object o). 

But from a technical perspective, what is happening behind the scenes?

Even though we are using an enhanced for loop, there is an Iterator involved behind the scenes. The Iterator is an interface. ArrayList implements the Iterator interface via a private inner class, Itr. When we use the enhanced for loop, there is a call to a method iterator() in the ArrayList when the for loop is going to be executed.

public Iterator<E> iterator() {
 return new Itr();
 }

This Itr is the one that maintains the state of the ArrayList. It does this using 2 variables, one is the modCount and the other one is the expectedModCount. modCount is an instance variable which keeps track of the number of times the ArrayList is modified. So far we only added elements, 6 to be precise, so the modCount is 6. The Itr class maintains the expectedModCount. Since this Itr class implements Iterator interface, it provides implementation for next(), hasNext(), remove().  Note that the remove method here does not take any arguments.

When we say persons.remove(p), the ArrayList remove method which takes an object as a parameter is called and not the Itr remove(). This remove(Object o) method makes a call to fastRemove() which does increase the value of the modCount but does not touch the expectedModCount. When the element is deleted  in our case, the modCount becomes 7 but the expectedModCount is still 6. Even if you use an enhanced for loop, the elements in the ArrayList are iterated using the next, hasNext method. The next method is called as the iteration moves ahead. The next method has this piece of code :

public E next() 
 {
    checkForComodification();
    ......
 
 }
final void checkForComodification() {
 if (modCount != expectedModCount)
 throw new ConcurrentModificationException();
 }

The method next checks for modification by evaluating if modCount and expectedModCount are not equal. Since the expectedModCount has not been touched in the remove(Object o) of the ArrayList, we get this ConcurrentModificationException. In short, the ArrayList was changed without use of the Iterator.remove. It is the Iterator which maintains the internal state. We removed the element using <arrayList>.remove(Object o) leaving the Iterator puzzled, so the Iterator complains about ConcurrentModificationException. It complains that –  “Hey, you did not remove elements through me, I am the gate keeper and since you did not get your gate pass, I am going to throw an exception”.

So if this is not the right way of removing elements,  can we remove elements using an Iterator directly ?

Second Attempt – The Heavy duty Iterator way of removing elements

The Iterator has a remove method.We need to use this remove method. This is how it is done :

This works fine and we do not get any Exceptions. We are making use of personIterator.remove().

Why does this work ?

Itr inner class in the ArrayList implements Iterator interface.  It also maintains the correct state by increamenting modCount but also assigning expectedModCount to modCount. When we call remove() on the Iterator, not only does it increase the value of the modCount by 1 but it also does the following :

expectedModCount = modCount;

So the call to next() as shown above calls :

final void checkForComodification() {
 if (modCount != expectedModCount)
 throw new ConcurrentModificationException();
 }

This does not throw a exception as they are equal now. You got your gate pass and now the guardian of the ArrayList, the gatekeeper, the Iterator, is happy !

So obviously we must use the Iterator to remove the elements from an ArrayList while we are iterating. Well, true, if you are still stuck with Java 7. This brings us to the 3rd way of implementing removal.

Third attempt – Java 8 cute way of removing elements from an ArrayList

Did you stare at the code a couple of times ? Have I missed out on showing you some code ? That is the code , yes it is ! Trust me!  You don’t ? Well, then go ahead and try it out.

The removeIf is a method added in Java 8. It has been added and has an implementation in the Collection interface. Yes, you read that right, Collection interface. It is a default method.

default boolean removeIf(Predicate<? super E> filter) {

}

This method is however overridden in the ArrayList class. It takes a Predicate which is a Functional interface. If you refer to the removeIllegalUsingJava8 method, we are passing a Lambda, person -> person.getAge() < 18. This is passed to the Predicate interface and then in the removeIf method, the test method is called. If the condition is satisfised, the element is removed. Finally the modCount is incremented to indicate that the ArrayList has gone through a modification.

So should we use this method now because it looks cute and concise ? Well that is a start. The code is short and concise, it clearly tells us it’s intention – Remove an element that satisfies this condition.

Not convinced about the cuteness – There is more to it !

The removeIf method is also more performant. What ? Cute and Efficient ? Hard to believe ? Well, the removeIf method removes and manages the elements in the ArrayList in a different was as compared to the Iterator way of removing and adjusting the elements in the ArrayList. The remove method using an Iterator uses System.arrayCopy and shifts the elements each time an element is removed from the list. This can considered as O(n^2).

The removeIf method take a different approach. It uses a BitSet class and does not use a System.arrayCopy. It creates a BitSet to maintain the index of the elements that need to be removed. It finally shifts the elements once rather than shifting them each time. So it also performs better than the Iterator remove and the performance is O(n).

Conclusion

The Java 8 removeIf way of implementing removal is not only cute but performs better. Beauty with Brains ? That is a lethal combination, I urge you to start using it!

 

Command Pattern using Java 8 Lambdas

I have been looking at how to improve my knowledge of Design Patterns and at the same time sharpen my Java 8 skills too. I have begun to realize that one can combine the two and also realized that Java 8 Lambdas are extremely powerful.

In this post, we are going to take a look at how the implementation of the Command Design Pattern can be simplified using Java 8 Lambdas.

I have written another blog where you can understand the essence of the Command Design Pattern.

The example we are going to refactor using Java 8 Lambdas is from here. This article on the tutorialspoint site refers to the typical way of implementing the pattern. You must take a look at it to get a basic understanding of the domain/problem at hand. The following is not a UML but I have shown the main components for the example mentioned.

 

CommandPatternWithoutClient

In short , BuyStock and SellStock are 2 commands which implement the Order interface. Stock is the recipient and finally the Broker is the invoker. The Broker class deals only with the Order interface and has no idea about the concrete commands.

Let us implement the same using Java 8 Lambdas. The Order interface is a Functional interface and each ConcreteCommand above is really a function. Do we really a class to represent each Command ? Well, take a look at the client code now using Lambdas:

The Broker,Order interface remain the same and so does the Stock class. We can get rid of the individual Concrete Commands (BuyStock, SellStock) classes.

What we have done is actually passed behavior to the takeOrder method. This works as the Order interface has a method execute which takes no parameters and returns void. And hence () -> abcStock.buy() is a lambda expression which takes no parameters and returns void.

To understand this pattern better,  let us extend our problem. Imagine that in the same system, we had to sell Bonds too. The process of buying/selling stocks is different from bonds. With the Broker and the Order interface, the system can easily be extended with minimal changes.

NewCommandPattern

We added 2 more classes BuyBond and SellBond which implement the Order interface. These commands would invoke the Bond class( Receiver) which would perform the task of either buying or selling bonds. The Broker class remains untouched as it deals with only the Order Interface. Well, using Java 8 Lambdas, we don’t need the BuyBond or SellBond classes either.

The Command pattern can be implemented easily using Lambdas. One could argue that the Order interface has just 1 method and we haven’t considered the undo and redo commands in the Command Design pattern. We can always fall back on the typical implementation of the pattern in case we need an undo and redo in addition to the execute functionality.

Having said that, do we always need an Order interface which acts as the Command interface ? How about using the Runnable interface instead  to run all the commands?  It could affect the readability of the code a little bit but otherwise the use of Java 8 Lambdas really makes the implementation easy and powerful at the same time.

Understanding the Command Design Pattern

Let us forget all design patterns for a moment and let us try to solve a simple problem using basic Java programming knowledge.

Problem: I would like to collect tasks to be executed later.

Let us break this down:

  1. The first part is collecting tasks –  this would mean using some kind of a storage which would add tasks together in a data structure. In Java, this can be done using  a Collection. So we need to maintain a Collection<Something>.
  2. Well a better idea would be a List<Something>. A List would be more specific.
  3. What is this Something ? It is a Task ! Okay,  a List<Task> ?
  4. That gives birth to a class Task, now we have a List<Task> to which we can add a task.
  5. Let us get more specific, let us begin with an ArrayList, this would be List<Task> tasks = new ArrayList<>()
  6. But this Task could be different types of tasks. Each “type” of Task… Each “type” of Task…List is generic, it is an interface,  ArrayList, LinkList are different “types” of List. So what next ?
  7. Refactor Task to an interface, and let each “type” of Task be a class which implements the interface Task.
  8. With this in place, we can add any type of Task to the tasks arraylist above. Task1,Task2….Taskn implements Task.
  9. As we are now done with collecting tasks, let us turn to the second part of running the tasks. The interface Task should have a method, let us call it executeTask(). Each type of Task will now override the executeTask() and perform it’s own task.
  10. To run each task – loop through each task and simply call the executeTask method.

for(Task task : tasks) {
task.executeTask();
}

In fact, the above for loop can be executed by using the Runnable interface/Thread class.

Let us turn our attention to the UML of actual Command Design Pattern :

 

Command_Pattern

Each type of Task is really a ConcreteCommand. The Command interface is really the same Task interface. The Invoker is a class which executes the for loop above and maintains a list of tasks/command. The only thing we have not seen is the Receiver, well when each Task/Command is executed, the Receiver will be invoked from each task. It is the work horse of the Command, the object that really does the work.

Advantages of using the Command Pattern:

  1. We can add ‘n’ different commands to the system later but the Invoker remains the same as it only deals with the Interface(Command/Task).
  2. Invoker is not aware of the work horse or the Receiver of each Command. So our system is extensible and loosely coupled.

This is really the essence of the Command Design Pattern.

I have written another blog where I dive into the implementation of the pattern.

Function composition using Java 8

The objective of this blog post is to give you a basic idea of functional composition in Java 8. We will look at 2 methods, andThen and compose which are part of the functional interface, Function.  We will then build on it further to show how we can apply a stream to a variable number of functions to combine all the functions.

Let us take a very simple example of 2 functions, one function adds 0.2 to an incoming value and another one adds 0.5 to an input value. So the input is a double and the output is also a double. This can represented as a Function<Double,Double>. Function is the functional interface introduced in Java 8.

Function<Double, Double> func1  =  d -> d + 0.2;
Function<Double, Double> func2  = d -> d + 0.5;

Both andThen and compose are default methods.

The andThen method:

To compose these 2 functions together , we can use andThen method from the Function interface. What do we get now ?

func1.andThen(func2)

Let us start with an initial value, 1.0.

Hence :

func1.andThen(func2).apply(1.0)

This gives us an output of 1.7.

How ?

  1. The call to the andThen method first applies the func1 to 1.0 which is the input. This gives us an output of 1.2 as behavior of func1 is d -> d+0.2, here d = 1.0
  2. The output from above , 1.2 , serves as an input to func2 and hence we get 1.2+0.5 = 1.7.

The compose method:

To understand this let us slightly modify the 2 functions:

Function<Double, Double> func1 = d -> d + 0.2;
Function<Double, Double> func2 = d -> d * 10.0;

The second function multiplies the incoming value by 10.

If we were to do:

func1.compose(func2).apply(1.0)

The output is 10.2

How?

  1. The call to the compose method first applies the func2 to 1.0 which is the input. This gives us an output of 10.0 as behavior of func2 is d -> d * 10.0, here d = 1.0
  2. The output from above , 10.0 , serves as an input to func1 and hence we get 10.0 + 0.2 = 10.2

What if we applied andThen to the above?  Well, the output is 12.0 . Surprised ?

Why?

  1. The call to the andThen method first applies the func1 to 1.0 which is the input. This gives us an output of 1.2 as behavior of func1 is d -> d+0.2, here d = 1.0
  2. The output from above , 1.2 , serves as an input to func2 and hence we get 1.2 * 10.0 = 12.0.

Remember:

The andThen method applies the first function, func1 to the input, that is the parameter to the apply method but the compose method operates by applying func2 to the value in the apply and then passes the value to func1.

andThen:

func1.andThen(func2).apply(1.0)

compose:

func1.compose(func2).apply(1.0)

More fun with functional composition:

Let us go back to :

Function<Double, Double> func1 = d -> d + 0.2;
Function<Double, Double> func2 = d -> d + 0.5;

When we did the following, we got the output as 1.7 :

func1.andThen(func2).apply(1.0)

Let us apply a Stream.of to func1 and func2.

The output from this is also 1.7

The Stream.of(…) returns a stream of Function<Double,Double>. The reduce method takes a Function.identity(), this is just saying, hey whatever is the next element in the stream, return it to me as it is. The Function::andThen specifies that combine each element in the stream with  andThen(). So <element_from_the stream>.andThen<next_element_from_stream>. The combination of stream and reduce method combines or chains all the functions together and gives us a single Function<Double,Double> which is a chain of functions.

Now how is this useful ? We can create a function which accepts  a variable number of functions and then they can all be chained in one place.

Output from all 3 types:

1.7
1.7
1.7

We used a very simple Function<Double,Double> which just added a double value but we could easily pass and chain functions which do a lot more.Function composition is a powerful mechanism which can be used to solve a variety of problems.Think about the decorator pattern or how consecutive filters are applied , can’t we use functional composition instead ?

How I met Spring Boot…

The title of this blog is inspired from the famous sitcom “How I met your Mother”. Well, this is a short story of How I met Spring Boot.

We are a team of 5 developers among whom there is a so-called “Software Architect”. In this part of the world that I live, the 2 words, “software architect” really mean different things to different people. More on that later.

There was a new requirement to rewrite a system currently maintained using mainframe to a more modern AngularJS/RESTful based application.The back-end would have REST based services which would communicate with an Oracle database using JPA . On the server side, the first task was really to set up the basic back end application and then the developers on the team would start with their use cases. Our architect decided to take up the responsibility of setting up the basic back end and mentioned that it was a tough task. By basic I mean, just the bare bones with all the basic dependencies, configuration against an in memory database to begin with, simple rest service to help the team get up and running.

When the scrum master asked him an estimate , he said he would take around 2 weeks to set up the basic backend code.

Well during the daily stand up meeting, the architect would say that he is still setting up the project and that went on for 2 weeks.At the end of 2 weeks when he checked in the code, what I saw was really shocking. What I saw was strange dependencies, outdated versions of jars, unused code,dependencies. I realized later that he had copy pasted the basic setup from another project.

Well, I said to myself, “Is setting up a project that hard ? I do remember shying away from trying a few things on my own assuming it would be complicated to bring in all the right dependencies, set up the right Spring configuration.I remember I wanted to integrate Spring with Hibernate and never really understood spring-orm library and everything else that came along with it. I have sometimes wondered how to get all configuration right and well sometimes been a little lazy.

But this time,  I decided to investigate this further and decided to set up the project  and of course nobody in the team knew this. This is when I discovered Spring Boot.

Well, as I was browsing through the official Spring framework website, I read something about bootstrapping your Spring Boot application and this is when I stumbled upon the Spring Initializr. I switched to the full version there and chose the following:

  1. Maven project
  2. Java 1.8
  3. Left the group and artifact id as it is as I was playing around.
  4. JPA support
  5. H2 as in memory database and finally
  6. Full stack web development with Tomcat and Spring MVC

And well on clicking the Generate Project button,  I had a full-fledged maven project. That took a few days…Correction – seconds!

I then wrote the following simple classes, one end point for the rest service, one for the repository layer, one JPA entity and finally the class which starts everything.

EmployeeRestController.java

EmployeeRepository.java

Employee.java

DemoApplication.java

Snapshot of the pom.xml

Then I did the following:

  1. Ran mvn clean install
  2. java -jar ./target\demo-0.0.1-SNAPSHOT.jar ( Yes, that is right, a jar file)

That’s it, the application was up and running ! When I hit the URL :

http://localhost:8080/employees

I got the following output:

[{"firstName":"Barney","lastName":"Stinson"},{"firstName":"Alex","lastName":"Hitch"},{"firstName":"Douglas","lastName":"Ramsey"}]

Note The DemoApplication implements a CommandLineRunner and hence the run method. This is being done only to populate some mock data into the H2 database which the rest service can access .

The entire thing took me only a few minutes. We were able to set up and run a Spring REST, JPA based maven project in a matter of few minutes ! Not just that , we were able to get an executable jar file which ran in an embedded server. Now, how cool is that !

Well, I wonder what the architect did during those 2 weeks what we could do in a few minutes! Not just that, we were able to configure the project with all the latest and relevant dependencies without worrying about compatibility and version related issues.

We did not write a single configuration file, there are no xml’s,  there is absolutely no code generation behind the scenes !

But what really happened ?

This is all handled by Spring Boot ! The starter pom that we added brings in the required jars to the classpath. As an example ,with the spring-boot-starter-parent and spring-boot-starter-data-jpa dependency , Sprint Boot will bring all the latest versions of the jars(Spring data, Hibernate). In our case  it pulls in version 5.0.9 of Hibernate.

Then through the usage of the @SpringBootApplication annotation (see DemoApplication.java)  Spring Boot configures  some defaults.

Some of the defaults in this case:

  1. Spring Boot configures an embedded Tomcat server if we haven’t configured it to run on anything else. This is done since we added the spring-boot-starter-web in  then pom.xml
  2. As we did not specify any other database specific parameters, Sprint boot decided to use an in memory H2 database which we added in the pom.xml
  3. Spring boot configured JPA with Hibernate for us as we added the spring-boot-starter-data-jpa dependency

Of course we can add other specific configurations that we want and in that case Spring Boot will honor that and move out of the way or if we wanted to bring an older version of Hibernate, we just need to modify the pom.xml and add the following:

<hibernate.version>Required_Version</hibernate.version>

We were able to run the application using the java -jar command. This is enabled through the spring-boot-maven-plugin dependency we added in the pom.xml.

We have only scratched the surface of how simple it is get a project up and running with a rest based service along with JPA but Spring Boot can do a lot more for us. Not only did we manage to get it up and running quickly but with all the right dependencies and the setup looks clean !

We can integrate spring boot with a nosql database, jms , apache camel, jersey, cloud foundry, service discovery and even use it to monitor the application in production. Take a look at Spring Initializr for all the cool things it can do.

This article was only meant to introduce you to Spring Boot. Well, Spring Boot is just perfect for a lazy developer like me.

Well Barney Stinson from the sitcom “How I met your mother” always wore a suit and one of his famous quote is “Suit Up”.  Having met Spring Boot, I must say that we must all Boot up…Spring Boot Up !

Strategy design pattern using Lambdas

I was recently reviewing some code and was thinking of solving a code smell that I came across. It seemed to me that the Strategy pattern would be a good fit to solve the code smell. Well, honestly , I am not a big fan of design patterns in every code smell but I feel they can surely be used in cases where it is appropriate. I consider design patterns as a common vocabulary to solve problems.

I don’t remember the exact implementation of most of the design patterns at the back of my head and so I started reading about the Strategy pattern on wiki :  Strategy pattern on wiki. If you are not aware of the pattern at all, I would recommend that you read  about the pattern first before looking at the Java code on wiki.

The problem the code sample tries to solve is using a billing strategy at runtime to generate  the Customers bill. Customer class can use a NormalStrategy or use a HappyHourStrategy. The code is open for extension in case we decide to use a third strategy to bill the customer.

At  the  end of the Java code sample, the wiki page mentions that there is a much simpler example in modern java using Lambdas. When I clicked that link, it showed an altogether different example. Well, I got a little curious and decided to solve the same Customer bill generation problem using Lambdas.

Thought process:

The strategy pattern has an interface and the example above uses 2 concrete implementations of the strategy. If I could replace it with a lambda, what do I need to focus on ? The 2 implementations of the strategy are quite simple, what do they do ? They take a double as a parameter and return a double. Can I represent this using a lambda ? Yes, I can. There is a functional interface in Java 8 called Function, it takes one type as input and returns/produces another type:

public interface Function<T, R>

And so we can do this :

private Function<Double, Double> billingStrategy;

Refer to the code on wiki, Customer constructor takes a BillingStrategy:

public Customer(BillingStrategy billingStrategy){
             this.billingStrategy = billingStrategy;
}

This can now be replaced :

public Customer(Function<Double, Double> billingStrategy){
             this.billingStrategy=billingStrategy;
}

The add method in the Customer class is:

public void add(double price, int quantity){
     drinks.add(billingStrategy.getActualPrice(price * quantity));
}

Instead of the implementation of the billing strategy, we now have a billingStrategy which is a Functional interface(Function)

public void add(double price, int quantity){
       drinks.add(billingStrategy.apply(price * quantity));

And so now the full code:

Customer.java

StrategyPatternWiki.java

What has changed:

  1. The billing strategy has been declared using a Function and lambda.
    • The class NormalStrategy takes a rawPrice and returns the rawPrice and hence we can do this:                                                                                                              Function<Double, Double> normalStrategy =  (Double d) -> d;
    • The HappyHourStrategy takes a rawPrice and gives a 50 % discount and hence we can do this :                                                                                            Function<Double, Double> happyHourStrategy =  (Double d) -> d * 0.5;
  2. This is being passed to the Customer constructor.
  3. The setStrategy method now takes a Function<Double,Double>

Notice that we have no reference to the BillingStrategy interface or the 2 classes which implement the BillingStrategy interface.

Using lambdas and the functional interface helped us replacing the Strategy pattern which defines an interface and different classes which implement the interface. We simply used functions, not normal functions but higher order functions. The alternative way of solving the same problem using lambdas and functional interfaces is quite powerful.

We could do better:

BillingStrategy.java

Notice that this is a class with static methods. It has behaviour for the 2 strategies

StrategyPatternTest.java

The code change is  the usage of the method references.

Function<Double, Double> normalStrategy = BillingStrategy::getNormalStrategy;
Function<Double, Double> happyHourStrategy = BillingStrategy::getHappyHourStrategy;

Well it looks like Java 8 lambdas, methods references and the functional style of programming have opened up new and really interesting ways of solving recurring problems in our code.The next time you encounter a Strategy pattern, think of lambdas and the Functional Interface, Function.

Java 8 Streams

The objective of this post is to introduce you to Streams in Java 8 by answering some very basic questions.

What is a stream?

Let us take a slightly abstract view first. Imagine a pipe, a pipe with 2 open ends. One end, the input end, is connected to a source, and the other end is connected to the destination. Let us assume that the source here contains data.The data from the source will then flow through the pipe and possibly undergo transformation. The desired data then flows out from the other end.

Data from the source does not start flowing through the pipe the moment the input end of the pipe is connected to the data source. It only starts flowing and reaches the destination when a knob is turned on.

A stream is nothing but a pipe. The pipe here is an abstraction and so is the stream. A stream enables the flow of data from the source to the destination and it could undergo transformation on its way. A stream does not hold any data.

How do we create a stream/pipe?

The stream API is part of the java.util.stream package. It can be obtained in many ways, some of the common ways include the following:

  • From a collection : The stream method is a default method that has been added to the Collection  interface.A stream can be attached to any class that implements the Collection interface.The collection of persons here acts as a source of the data.
List<Person> persons =  // initialize list of people

Stream<Person> persons = persons.stream()

  • From static factory methods like Stream.of() :
Stream<Integer> numbers = Stream.of(1,2,3,4,5);
  • Files.lines() which returns each line of a file as a stream:
Stream<String> lines = Files.lines(Paths.get("C:\\TestFile.txt"));

What do we do with a Stream?

We saw a few ways above to get a reference to a stream/pipe. But what do we do with a stream? To get an answer to that, let’s think about what do we do with collections in java? We iterate through collection and pick up the relevant data that we need. The relevant data is obtained by applying some conditions, this is nothing but filtering. So in short, as we iterate, we filter it and then collect the data. Pay attention to these 3 words in italics. Iterate, Filter, Collect. These are some of the common operations that we would usually perform on a stream.

  • Iterate through a collection/data source:

 This is actually done implicitly for us using the stream method we discussed above.  Remember that there needs to be a trigger for the data to flow through the pipe. We  will discuss this soon but we did refer to this as the knob that makes the data flow  through the pipe. Once the knob is turned on, the stream method will push the data  down the pipe. 

Streams_Basic
Calling the stream() does not result in data flow

Example:

List<Person> persons =   // list of people

persons.stream()

  • Filter:

Once we have a stream/pipe and data begins to flow through it, we usually filter the data that moves through the pipe. This is done by using some conditional logic as we move through the data pipe. In Java 8, we use the filter method on the stream API to do this. The filter method looks like this:

The signature of the filter method:

Stream<Tfilter(Predicate<? super T> predicate)

The return type is a stream. The parameter to the filter method is a Predicate which is a functional interface.The data element will be evaluated using this filter, if it passes the criteria, it moves ahead in the pipe else it gets dropped out.

Example:

persons.stream()

                .filter(person -> person.getCitizenship() == Citizenship.USA)

Filter_Basic

 

Here the person-> person.getCitizenship == Citizenship.USA is a lambda expression that is mapped to the Predicate above.

  • collect:

So we created a stream and wrote a filter. Assuming that the data now flows through the pipe and passes through the filter and reaches the end of the pipe, what do we do now? We collect the data that we want. This is done using the collect method. The collect () API does exactly what it means, it collects the data. We can specify the final data structure into which the data needs to be collected.

Remember the knob we spoke about earlier. Unless the knob is turned on, nothing passes through the pipe and hence nothing gets filtered. The collect method is one such example which acts as the knob, it is called the terminal operation. Unless we call a terminal operation nothing happens! This makes the streams extremely lazy.

lazy_garfield

To summarize, using steps 1, 2 and 3 above, we created a stream, pushed the data down the stream, filtered it and then collected it.

Example:

The Collectors is a utility class which helps in accumulating elements into a collection as shown below.

Filter_Collect

 

 

Other simple examples to understand stream, filter, collect:

Get a list of all men from a collection of Person objects

Get a list of all women who are Canadian citizens:

 

Why Streams?

This topic deserves more attention and will be covered in another article but a short answer to the same would be that they bring functional style of programming to Java. There are many more operations like map, flatMap, sort etc which can be chained together to transform and sort objects. Streams do not create any temporary data structures, it is all 1 pass.  So the filter operation does not create any temporary data structure.Streams also open up the gates for parallel processing by exploiting the underlying hardware. This can be done by calling the parallelStream() instead of the stream method above.

There are many other operations than be called on the stream and the same will be covered in other posts.

 

Java 8 Predicate Interface using Lambdas

I am going to tell you a short programming story. Let us assume that we have a system which deals with a list of Person objects. Person class has name, age, citizenship and gender.

One fine day your manager comes to your desk.

Manager: The current system should be able to give us a list of all the men.

Developer: That is easy.

Manager:  OK, this is a very critical requirement, check in the code in the next 1 hour.

You are brimming with happiness, you have done this so many times and then you say to yourself, that’s  easy!  I just need to initialize an empty collection, iterate over the list of collection, add an if condition and that’s it. This is what you come up with:

Before checking in the code, you look through the code again, notice the new List of men objects, the iteration and if condition. You feel a great sense of accomplishment and send an email to your manager and possibly your colleagues that your task is completed. You then hop away for lunch.

After a heavy lunch, you come back on your desk and notice the manager walk towards you, it looks like he has a very serious and difficult requirement.

Manager: Good coding, I have a new requirement for you. Now the system needs to fetch the list of all the women from the database.

You: Okay, consider it done.

Manager: Alright, I need you to do this in 45 minutes.

What do you do?  You know the class name, you open it, look at the getAllMen() and use the most favorite operations,  “copy” , “paste” and then rename the method name and change the condition. This is what you come up with:

There is a big smile on your face as you look at your watch, you accomplished this code in 5 full minutes. You look at your code and say to yourself, create new list, iterate, if condition. It looks good and then check in the code, but send the email after 40 minutes just to make sure that your manager notices the 5 minutes that you saved.

Then you suddenly look at the code again, create new empty list, iterate, if condition…… But then you realize that you have forgotten to rename the variable. There is a little bit of panic but you are really fast, you rename that variable, men to women, check in that code so quickly that even Vin Diesel from the Fast and the Furious fame would be ashamed. Now, it’s time for a cup of tea. 

The modified version

It’s 5:30 pm already, you know it is time to go home. But alas, the manager again walking towards you. You do not want to look at him this time, you stare at your PC as if you are really busy and have no time for anyone.

Manager: Good job, great coding, you are really fast but I need you to retrieve the list of all men who are citizens of USA.

Hmm, so, you look at your watch, he looks back at you..

Manager: I would like you to check in the code before you leave for the day.

How do you feel ? Frustrated. You promised to take your girlfriend on that coffee date at 7 pm. The manager leaves your table. You roll up your sleeves and start copy, pasting and profusely start typing.  You look at your code and say to yourself, create new list, iterate, change if condition and this time…..change variable name too!

This time you are more careful to rename the variables, you go through the code once, this time everything looks flawless. Time to check in!

You feel a great sense of accomplishment, 3 full methods in the entire day and pray that there are no bugs. It’s 6 pm, you send an email to your manager and everyone is now happy. Suddenly you look at all the 3 methods and how do you feel?  WASTED!  You ask yourself , can I change something here? You look at the 3 methods one more time and realize that the if condition that checks if the gender is male or female seems repetitive. You look at your watch once again, it is already 6:15! You decide to leave.

On your way to meet your girlfriend, as you are riding your bike, you feel that the air around is polluted but you know deep down inside about polluting the code base. You meet your girl friend, there is a big smile on her face but a lot of dirt on your face caused due to the code pollution.You have also committed another sin, you have forgotten to pick up flowers for her!

You wish you could have at least written a separate method to check if the gender is male. The meeting with your girlfriend does not go that good, well the code…..we all know about that.

The next day, you wake up early, have a shower, and think about the new method you are going to write. You feel really good and go early to office. You bring all your experience to the table and start writing a new method which can be reused.

 As you are belting out that code, your chest swells up as you see that method name:  isMan(). You refactor your old code and change all occurrences of the condition  

             if(person.getGender() == Gender.MALE) to isMan()

You look at your code and say to yourself, new list, iterate, if condition, add to list. You give yourself a pat on your back but this time you do not send an email to your manager.  Well, you don’t want him to know that you made a change to your own code. Time for a morning tea break!

You are back on your desk, the manager has not come to your desk for a long time. You wonder if there are bugs in your code. But just then, there is a ping from the manager……

Manager: Good morning, got a new requirement. The system needs to fetch all women who are American citizens.

You (say this to yourself ☺) : Easy! Copy, Paste, rename, change if!

This time, you look at your code a few more times and realize that you want to refactor anything that repeats. You keep looking at your code and realize that you could write a method isWoman and change all if conditions to use that. You quickly do that. How? Copy, Paste, rename!

You stare at your 2 methods, getAllMenUSA and getAllWomenUSA(). You keep staring at it for a few minutes, you think about today’s meeting at 7 pm and of course, it is with your girlfriend. Now, you really want to refactor something. You realize that you can actually combine 2 methods into one.

You then remove the 2 methods written earlier. You replace the calls to the 2 methods by 1 method.

Earlier:  getAllMenUSA(persons)

                 getAllWomenUSA(persons);

Now:      getAllPersonByGenderAndCitizenship(persons,gender,citizenship)

Time for review, new list, iterate, if condition, check in. Time for a long lunch break!

You are back… Of course, it is now time for a new requirement. No free lunch!

Manager: Hey, need a list of men who are above the age of 35 but citizens of Canada.

You: Silent….

You look at your getAllPersonByGenderAndCitizenhip and say, okay, hmm, let’s add a new parameter to that function and create a new method by copy, pasting and renaming the old method.

getAllPersonByGenderAndCitizenshipAge(persons,Gender.MALE,Citizenship.CANADA, age);

New list, iterate, if condition(s), add to new list. Done! You let out a burp! Heavy lunch, a lot of coding, of course the food had to digest that quickly! Check in……This time, you get up, walk past your manager, and give him a smile. That look on your face for writing really good code!

It is 4 pm now. To kill the time, you read your code, all the methods, and realize that you are really writing the same code all over, but if one is right, they are all right and hence decide not to change anything.

Out of boredom, you then look at your colleague’s machine, he is reading about DRY – Don’t repeat yourself. You ask him what that is. He explains it you and you say oh, that’s easy. The look on your face, well, it’s as if you have been doing it since you were 5. You wonder if you can apply it your code!

Well, what the developer does not realize is the fact that even after refactoring some of the methods, these requirements can spiral out of control. There could be many more methods like:

private List<Person> getPersonByGender(List<Person> persons, Gender gender)

private List<Person> getPersonByGenderCitizenship(List<Person> persons,Gender gender, Citizenship citizenship)

private List<Person> getAllPersonByGenderCitizenshipAge (List<Person> persons,Gender gender, Citizenship citizenship, Age age)

All this is only leading to copy, paste, and repeat. If you look at all the methods above, what is really changing is the “if” condition. Everything else remains the same and yet we are duplicating code. How do we solve this? How do we isolate the “if” condition? Java 8 Predicates to the rescue! The Predicate interface is a functional interface. It has a method called the test method as shown below:

@FunctionalInterface
public interface Predicate<T> {

    boolean test(T t)

}

where t is the parameter that needs to pass a test. If it passes then it returns true else false

How do we use it?

This is done using lambda expressions. Right side represents the condition which in a lambda expression and then assign the condition to a Predicate reference.

How does it help?

Concentrate on the if conditions used in the code above and use a lambda expression instead.

All men: 

Predicate<Person> allMen = person -> person.getGender() == Gender.MALE;

getPeopleWithFilter(persons,allMen);

All women:

Predicate<Person> allWomen = allMen.negate();

getPeopleWithFilter(persons, allWomen);

All US Citizens:

Predicate<Person> usaCitizenship = p -> p.getCitizenship() == Citizenship.USA;

Get all men who are US citizens, done by combining predicates:

getPeopleWithFilter (persons, allMen.and(usaCitizenship));

Age above 35 years:

Predicate<Person> ageAbove35 = p -> p.getAge()  > 35;

Get all men who are US citizens and above the age of 35:

getPeopleWithFilter (persons, allMen.and(usaCitizenship).and(ageAbove35));

Now, all the methods written above can simply be deleted and replaced with:

You say to yourself, create new list, iterate, single if condition using predicate. Hmm, can we do something more? Can we get rid of the creation of the new list, the for loop ?

More refactoring:

The filter method on the stream accepts a Predicate which internally returns true or false. If it returns true, it gets collected into the list else it gets filtered out.

This code has no temporary creation of a new list during the stream or filter calls, has no explicit iteration, no repetitive if conditions!

The code looks cleaner, more expressive and maintenance of this code is easier. Using Lambdas and the Predicate interface we can do powerful things.The next time your manager walks up to you with a new requirement, write a simple Predicate and call the method above. It is 6 pm already, time to check in and leave for the coffee date, this time make sure you reach before time, carry some flowers or may be a card too which says “You look beautiful like my code”!

Moral of the story: Love thy code!

Java 8 Lambdas – A brief introduction

Let us say that you are in a hurry, you do not know the local language and you need directions to travel from point A to point B. You finally see just one stranger on the road who can speak in your language and this is how he instructs you to get to point B:

  1. Walk straight for about a 100 metres and around the corner take a right. Hey, on your way don’t  forget to notice my favorite restaurant which serves the best pasta.
  2. Once you take a right, walk straight down for another 100 metres, and take a left. During the 100 m walk, you will see a gymnasium, that is where I work out.
  3. Then your destination will be on the right side but before you cross over to the other side, don’t forget to look at the beautiful tower clock.

If you are really in a hurry and not hungry, do you really care about the pasta and the tower clock ? Well anonymous classes in Java are a bit like this. You setup a lot of instructions before the actual set of instructions that we are really interested in.

When you sit in a plane, do you go to the pilot and tell him how to get to the destination?  You don’t !  You sit in the plane and let the pilot do the job for you.  And well, pray that he does it well☺ !

Why am I talking about Anonymous classes ? You must be thinking  – Do I have to learn that now to understand lambdas? Aren’t they kind of ugly? Yes, you are right, they are! But sometimes, things in life can get ugly and if I may get a little philosophical, ugly things in life make you stronger and probably motivate you to do the same things in a better way.

Let us take a simple example:

Take a look at the code above and you will probably feel like this :

Confused_Lambdas

If you asked an experienced developer on your team (so called expert), they might say : That is nothing but an anonymous inner class which implements the Runnable interface. Once you implement the Runnable interface, you override the run method. The computation that we need to perform in another thread is written in that run method.

Lambdas_Funny

We had to do so much to print that line in the run method so that it executes in another thread. Is there a better way to do that? Let us give it a try, let us break that down a little bit.

What are the parameters to the run method? None. What is the run method doing? Just that one single line which prints a message. Our main focus is the run method which takes no parameters and performs some computation. So we could represent the parameters to the run method as () indicating empty parameters. The body of the method is System.out.println(“Do something in the thread”) and now let’s combine the two.

What do we get ?

() - System.out.println(“Do something in the thread”);

Let us a use a different separator, the ->. The -> is just a separator between the parameters to the run method and the body of the method.

() -> System.out.println(“Do something in the thread”);

And well, that is a lambda expression!

So we can actually replace the code using anonymous class as:

So we have been able to refactor our code from:

To:

That surely looks a lot better. This code is quite clear in conveying what we really intend to achieve in the thread, unlike the stranger who gave us a lot of instructions which was not really useful.

Are lambdas just a replacement for the inner class syntax above? Is the compiler doing the same thing behind the scenes?

No, it isn’t! When we write an anonymous class using the anonymous class syntax and compile our code, we actually get 2 classes. <ClassName>.class and <ClassName$1>.class. When we use the lambda expression, we get just 1 .class file. That is one of the most important differences between an anonymous inner class and a lambda expression. How this really works behind the scenes will be covered in another article. But to give you a heads up, this is done using  a byte code instruction called the invokedynamic that was introduced in Java 7.

How did we manage put a lambda in place of an anonymous class?

Instance of a Thread class takes a Runnable:

Thread t = new Thread( Runnable target);

Runnable is an interface. We started with an anonymous class which was then replaced with a lambda.

Thread t = new Thread( () -> System.out.println(“Do something in the thread…”));

So it means, we actually managed to do this:

Runnable target = () -> System.out.println(“Do something in the thread…”);

Well, the Runnable interface has been there in the JDK library right from the beginning.How did we just manage to replace all of that code with a lambda ? This is because Runnable is an interface, it has just one method, the run method. This is now called a functional interface – An interface with 1 abstract method. Runnable has just one – run().

@FunctionalInterface
public interface Runnable{
void run();
}

Thread class takes Runnable and we can assign a lambda to Runnable since it is a Functional interface. The signature of the run method gives us an insight on the lambda expression.The return type is automatically inferred by the compiler.

A lambda expression can take multiple parameters:

(int a, int b) -> a+b;

A lambda expression can also have multiple statements:

() ->{
System.out.println(“One”);
System.out.println(“Two”);
};

Lambdas that  have multiple statements need to be enclosed in curly braces. We must try and avoid having too many statements in the curly braces above as this will defeat one purpose of lambdas – clarity in code. If you see the body of the lambda getting too big, replace it with a function.

To solidify our understanding , let us go through another example.

Let us sort a list of strings by their decreasing length using a Comparator.

The 2nd parameter to the sort method is an instance of Comparator. What do we really want to do in the code above ? We want to sort strings by their length but landed up doing a lot more.

Is this a good candidate for Lambdas ?  Let us find out by asking the following questions

  1. Is there an instance of an anonymous inner class- Yes
  2. Does Comparator have only 1 abstract method – Yes

Let us go ahead and replace it with a lambda like we did it with Runnable.

  1. What parameters does the compare method take – 2 Strings.
  2. What is the method doing – comparing the length

(String s1, String s2) -> s1.length() < s2.length() ? 1 : -1

Let us replace the code:

In fact, the compiler is smart to know that we are sorting a list of Strings. Hence the same code can be written without mentioning the type of s1 and s2.

We assigned the behavior of comparing strings by length to type Comparator.

To summarize, lambdas:

  1. Help us concentrate on what we want to do rather that how to do it
  2. Pass behavior to function which can be a very powerful mechanism. 
  3. Are anonymous functions that map to functional interfaces.

Finally, I hope we have been able to move our state….

Confused_Lambdas Happy_Lambdas