Java Streams Overview, Part II

In my previous article, I wrote about the fundamentals of streams in Java 8. Now, let’s augment our skills with some additional information about streams, like how we can chain them, and we can use them to access files.

Chaining Streams

When working with streams, they are often chained together.

Let us see what are the advantages of using chained streams:

  • One stream instance leverages another stream instance.
  • This creates a higher level of functionality. We can have one stream accessing the data, then we have another stream that takes the results of that and processes more complex functionality.
  • This simplifies reusability because you can organize your streams in a way that allows each of them performs a specific job. In this way, they do not need to know each other’s inner workings.

We perform chaining using a constructor. We construct a higher level instance of the stream and then pass an instance of a lower level stream.

A good example of a chained stream is the InputStreamReader class, which is what we talked about in my previous article.

This class leverages chaining by providing reader behavior over an  InputStream. It translates the binary response to character behavior.

Let us see how it works.

void doChain(InputStream in) throws IOException{
int length;
char[] buffer = new char[128];
try(InputStreamReader rdr = new InputStreamReader(in)) {
while((length = >= 0) {
//do something

As you can see, we do not need to care about how the  InputStream works. Whether it is backed by a file or network, it does not matter.

The only thing we know that it gives us binary data, we will pass it to our InputStreamReader and it converts it and can work with it as a character data.

Notice that we use try-with-resources here as well. If we close the InputStreamReader, it automatically closes theInputStream as well. This a very powerful concept, that you should know about.

File and Buffered Streams

We often use streams for accessing files.

There are several classes for that in the package to use, like:

  • FileReader
  • FileWriter 
  • FileInputStream 
  • FileOutputStream 

The real thing is that these file streams are deprecated now. Despite that, they are still widely used in codes. So, you probably will face them in the near future, so it’s worth a notation.

Let us look at new ways to interact with files.

Buffered Streams

Buffered Streams are introduced to replace the FileStream classes in the package. These new Streams are placed under the java.nio package.

It was necessary because direct file access can be inefficient and buffered streams can significantly improve efficiency with the following:

  • Buffer content in memory
  • Perform reads/writes in large chunks
  • Reduce underlying stream interaction

Buffering available for all four stream types:

  • BufferReader 
  • BufferWriter 
  • BufferedInputStream 
  • BufferedOutputStream 

Using them is very straightforward.

try(BufferedReader br = new BufferedReader(new FileReader("file.txt"))){
int value;
while((value = >= 0) {
char charValue = (char)value;
//do something

Additional benefits to using BufferedStreams includes:

  • It handles linebreaks for various platforms like Windows or Unix
  • Uses correct value for the current platform
  • The BufferedWriter has a method:newLine(). It will create a new line with the appropriate character.
  • TheBufferedReader has a method for line based read: readLine().

Let us see how they work.


void writeData(String[] data) throws IOException {
try(BufferedWriter bw = new BufferedWriter(new FileWriter("file.txt"))){
int value;
for(String str : data) {


void readData(String[] data) throws IOException {
try(BufferedReader br = new BufferedReader(new FileReader("file.txt"))){
String inValue;
while((inValue = br.readLine()) != null) {

The code above will write out the file’s content line by line.

Accessing Files With the java.nio.file package

In Java 8, the streams are deprecated. There is a new package to handle file streams called the java.nio.file package.

The package has several benefits over

  • Better exception reporting
  • Greater scalability, they work much better with large files
  • More file system feature support
  • Simplifies common tasks

Bellow, we will talk about the most fundamental Types In this new package.

Paths and Path Types


  • Used to locate a file system item
  • It can be a file or directory


  • Used to get the Path objects through static Path factory methods
  • It translates a string-based hierarchical path or URI to Path.

Example: Path p = Paths.get(“\\documents\\foo.txt”)

Files Type

  • Static methods for interacting with files
  • Create, copy, delete, etc…
  • Open files streams
    • newBufferedReader 
    • newBufferedWriter 
    • newInputStream 
    • newOutputStream 
  • Read/Write file contents
    • readAllLines
    • write 

Reading Lines With BufferedReader

Let us see some quick example of how you can use it.

void readData(String[] data) throws IOException {
try(BufferedReader br = Files.newBufferedReader(Paths.get("data.txt")){
String inValue;
while((inValue = br.readLine()) != null) {

Read All lines

void readAllLines(String[] data) throws IOException {
List<String> lines = Files.readAllLines(Paths.get("data.txt"));
for(String line:lines) {

File Systems

When we work with files from a Java program, those files are contained within a file system. Most commonly, we use the computer’s default file system.

Java also supports specialized file systems, such as the Zip file system.

Path instances are tied to a file system and thePath class works only for the default one. So, we need another solution. Fortunately, in the Java.nio package, we have the opportunity to deal with this.

File System Types


  • Represents an individual file system
  • Factory for Path instances


  • Used to get the FileSystem objects through static FileSystem factory methods
  • Open or create a file system
    •  newFileSystem

Accessing File Systems

File systems identified by URIs

  • Specifics of URI vary greatly among the file systems
  • Zip file system uses “jar:file” scheme
    • jar:file:/documents/

File systems support custom properties

  • Different for each file system type
  • Examples: String encoding, whether to create if it does not exist

Creating a Zip Filesystem

public static void main(String[] args) throws FileNotFoundException, IOException {
try (FileSystem zipFileSystem = openZip(Paths.get(""))){ //pass the Path where we would like to create our FileSystem
}catch (Exception e) {
System.out.println(e.getClass().getSimpleName() + " - " + e.getLocalizedMessage());;
private static FileSystem openZip(Path path) throws URISyntaxException, IOException {
Map<String, String> properties = new HashMap<>();
properties.put("create", "true"); //set the property to allow creating
URI zipUri = new URI("jar:file", path.toUri().getPath(), null); //make a new URI from the path
FileSystem zipFileSystem = FileSystems.newFileSystem(zipUri, properties); //create the filesystem
return zipFileSystem;

After the code above, you should see your file in your directory.

Copying Files to Zip Filesystem

Let us augment the above example with a File copy operation.

In this example, I created a file called file.txt in my project library. We will copy this file to our Filesystem.

Streams in Java 8
public static void main(String[] args) throws FileNotFoundException, IOException {
try (FileSystem zipFileSystem = openZip(Paths.get(""))){
copyFileToZip(zipFileSystem); //Here we call the file copy
}catch (Exception e) {
System.out.println(e.getClass().getSimpleName() + " - " + e.getLocalizedMessage());;
private static FileSystem openZip(Path path) throws URISyntaxException, IOException {
Map<String, String> properties = new HashMap<>();
properties.put("create", "true");
URI zipUri = new URI("jar:file", path.toUri().getPath(), null);
FileSystem zipFileSystem = FileSystems.newFileSystem(zipUri, properties);
return zipFileSystem;
static void copyFileToZip(FileSystem zipFileSystem) throws IOException{
Path sourceFile = FileSystems.getDefault().getPath("file.txt"); //Read the file to copy
Path destFile = zipFileSystem.getPath("/fileCopied.txt"); //get the path of the new file
Files.copy(sourceFile, destFile);//Copy the file to our zip FileSystem

After you run the code, you should see the fileCopied.txt int our zip-file. Its context should be the same as in our file.txt.


In this article, we went further into streams in Java 8. I demonstrated how stream chaining works, as well as how you can deal with files through the new java.nio package. We also touched on why you should use more up-to-date, buffered versions of the Filestreams.

Hope you enjoyed!

Using Cache in Spring Boot

Let’s imagine a web application, where for each request received, it must read some configuration data of a database. That data doesn’t change usually, but the application, in each request, must connect, execute the correct instructions to read the data, pick it up from the network, etc. Imagine also that the database is very busy or the connection is slow. What would happen? We would have a slow application because it is reading continuously data that hardly changes.

A solution to that problem could be using a cache, but how do you implement it? In that article, I explain how to use a basic cache in Spring Boot.

A Little Theory

The cache is replicated over functions, where for the same entry value, we are waiting for the same return value. That’s why we always have at least one parameter for entry and exit.

A typical example will be this:

public int cachedFunction(int value){
..... complicated and difficult calculations ....
  return N;

And now, let’s suppose we have the next code for calling that function:

int value=cachedFunction(1);
int otherValue=cachedFunction(2);
int thirdValue=cachedFunction(1);

When executing the program, in the first line, Spring will execute the function and save the result that returns. In the second line, if it doesn’t know the value it must return for the input “2.” Nevertheless, in the third line, Spring will detect that a function tagged as @Cacheable   with the name “headers” was already called with the value “1.” It won’t execute the function, it will only return the value that in the first call it saved.

The cache’s name is important because, among other things, it permits us to have different independent caches, which we could clean to instruct Spring Boot to execute the functions again.

So, the idea is that in each call to a function tagged as @Cacheable it will save the return values for each call in an internal table, in such a way that if it already has a return value for one entry, it doesn’t call to the function.

The Practice

And now, let’s get to the practice.

An example project can be found here.

First, we must include the following dependency in our project.


Now, we can use the tags that will allow us to use Cache  in our application.

The first tag set is  @EnableCaching. With this label, we tell Spring that it must prepare the support to use Cache. If we do not put it, it will simply not use Cache, regardless of whether we then mark the functions with cache tags.

public class CacheExampleApplication {
    public static void main(String[] args) {
, args);

In this example, we read the data of a database using REST requests.

Data  in the class which is in the package com.profesorp.cacheexample.impl

The function that reads the data is the following:

@Cacheable(cacheNames="headers", condition="#id > 1")
public DtoResponse getDataCache(int id) {         
    try {
    } catch (InterruptedException e) {
    DtoResponse requestResponse=new DtoResponse();                     
    Optional<Invoiceheader> invoice=invoiceHeaderRepository.findById(id);

As can be seen, we have the tag  @Cacheable(cacheNames="headers", condition="#id > 1") 

With this, we told Spring two things:

  1. We want to cache the result of this function.
  2. We put it as a condition that it must store the results in cache if the input is greater than one.

Later, in the function flushCache we put the tag @CacheEvict that cleans the indicated cache. Also, in this case, we tell it to delete all the entries that it has in cache.

@CacheEvict(cacheNames="headers", allEntries=true)
public void flushCache() { }

In the function update we update the database and with the label @CachePut, we inform Spring that it updates the data for the existing value in

Of course, this function must return an object equal to the function labeled with the tag @Cacheable , and we must indicate the input value on which we want to update the data


To understand the application better, we will execute it and give it a request .

The application at the beginning has four invoices in the invoiceHeader table. You can see how it fills the table in the data.sql file

Let’s run the get function of the PrincipalController class. For this we write this:

> curl -s http://localhost:8080/2

The application will return the following:


The field interval is the time in milliseconds that has takes the application making the request. As can be seen, it has taken more than half a second, because in the getDataCachefunction of we have a sleep 500instruction.

Now, we execute the call again:

> curl -s http://localhost:8080/2

Now the time the call has taken is 1, because Spring hasn’t executed the code of the function, and it has simply returned the value that it had cached.

However, if we request the id as 1, we have indicated that you should not cache this value, always execute the function and therefore we will have a time exceeding 500 milliseconds:

>curl -s http://localhost:8080/1
>curl -s http://localhost:8080/1
>curl -s http://localhost:8080/1

If we call to the flushcache function, we’ll clean the cache and therefore, the next call to the function will execute the code in it.

> curl -s http://localhost:8080/flushcache
Cache Flushed!
> curl -s http://localhost:8080/2
> curl -s http://localhost:8080/2

Finally, we will see as if we change the value of the field activo to N, since the function that makes the change is labeled with @CacheEvict, it will update the value of the cache, but the getDataCache function won’t execute in the next call.

> curl -X PUT   http://localhost:8080/   -H "Content-Type: application/json"   -d "{\"id\": 2, \"active\": \"N\"}"
>curl -s http://localhost:8080/2


Spring without any difficulty allows us to cache the results of the functions. However, you have to take into account that cache is very basic and it is realized in memory. Spring Boot permits us to use external libraries that will allow us to save the data in disc or database.

In the documentation, you can find the different implementations of cache that Spring Boot supports, one of which is EhCache with which you will can different kinds of backend for the data, as well as specify validity times for the data, and more.

How Much Memory Does a Java Thread Take?

A memory, which is taken by all Java threads, is a significant part of the total memory consumption of your application. There are a few techniques on how to limit the number of created threads, depending on whether your application is CPU-bound or IO-bound. If your application is rather IO-bound, you will very likely need to create a thread pool with a significant number of threads which can be bound to some IO operations (in blocked/waiting state, reading from DB, sending HTTP request).

However, if your app rather spends time on some computing task, you can, for instance, use HTTP server (e.g. Netty) with a lower number of threads and save a lot of memory. Let’s look at an example of how much memory we need to sacrifice to create a new thread.

Thread memory contains stack frames, local variables, method parameters, … and a thread size can is configured with defaults this way (in kilobytes):

$ java -XX:+PrintFlagsFinal -version | grep ThreadStackSize 
intx CompilerThreadStackSize    = 1024  {pd product} {default}
intx ThreadStackSize            = 1024  {pd product} {default}
intx VMThreadStackSize          = 1024  {pd product} {default}

Thread Memory Consumption on Java 8

$ java -XX:+UnlockDiagnosticVMOptions -XX:NativeMemoryTracking=summary /
-XX:+PrintNMTStatistics -version
openjdk version "1.8.0_212"
OpenJDK Runtime Environment (AdoptOpenJDK)(build 1.8.0_212-b03)
OpenJDK 64-Bit Server VM (AdoptOpenJDK)(build 25.212-b03, mixed mode)
Native Memory Tracking:
Total: reserved=6621614KB, committed=545166KB
- Java Heap (reserved=5079040KB, committed=317440KB)
  (mmap: reserved=5079040KB, committed=317440KB) 
-  Class (reserved=1066074KB, committed=13786KB)
    (classes #345)
    (malloc=9306KB #126) 
    (mmap: reserved=1056768KB, committed=4480KB) 
-  Thread (reserved=19553KB, committed=19553KB)
   (thread #19)
    (stack: reserved=19472KB, committed=19472KB)
    (malloc=59KB #105) 
    (arena=22KB #34)

We can see two types of memory:

  • Reserved — the size which is guaranteed to be available by a host’s OS (but still not allocated and cannot be accessed by JVM) — it’s just a promise
  • Committed — already taken, accessible, and allocated by JVM

In a section Thread, we can spot the same number in Reserved and Committed memory, which is very close to a number of threads * 1MB. The reason is that JVM aggressively allocates the maximum available memory for threads from the very beginning.

Thread Memory Consumption on Java 11

$ java -XX:+UnlockDiagnosticVMOptions -XX:NativeMemoryTracking=summary /-XX:+PrintNMTStatistics -version
openjdk version "11.0.2" 2019-01-15
OpenJDK Runtime Environment AdoptOpenJDK (build 11.0.2+9)
OpenJDK 64-Bit Server VM AdoptOpenJDK (build 11.0.2+9, mixed mode)
Native Memory Tracking:Total: reserved=6643041KB, committed=397465KB
-   Java Heap (reserved=5079040KB, committed=317440KB)
(mmap: reserved=5079040KB, committed=317440KB) 
-   Class (reserved=1056864KB, committed=4576KB)
(classes #426)(  instance classes #364, array classes #62)
(malloc=96KB #455) 
(mmap: reserved=1056768KB, committed=4480KB) 
(  Metadata:   )
(    reserved=8192KB, committed=4096KB)
(    used=2849KB)
(    free=1247KB)
(    waste=0KB =0,00%)
(  Class space:)
(    reserved=1048576KB, committed=384KB)
(    used=270K
(    free=114KB)
(    waste=0KB =0,00%)
-  Thread (reserved=15461KB, committed=613KB)
(thread #15)
(stack: reserved=15392KB, committed=544KB
(malloc=52KB #84) 
(arena=18KB #28)

You may notice that we are saving a lot of memory just because we are using Java 11, which no longer aggressively allocates up to Reserved Memory at the time of thread creation. Of course, this is just java -version command, but if you try it out, you will definitely notice a big improvement.


Java Streams Overview

For a long time, I had a gap in my knowledge of Java streams. On a basic level, I could use them, but I did not a deep understanding of them. So, I decided to do a quick overview of Java streams.

In this article, I will deal with many everything from the fundamentals to chaining and clean-up.

Hope this helps you broaden your knowledge of Java streams.

What Are Streams?

A stream is an ordered sequence of data that…

  • Provides a common I/O model
  • Abstracts details from an underlying source or destination

Whether you use streams to take data from memory, storage, or your network, it will hide the implementation details from you. The details are abstracted away, so in every situation, you can look at it as an ordered sequence of data.

  • Stream types are uni-directional

This means that if you create an instance of a Java stream, you decide whether you would like to write to it or read from it. You can’t do both at the given time on a single stream.

Read / Write

We can divide the streams into two categories:

  • Byte streams – Interacts as binary data
  • Text streams – Interacts as unicode characters

The general interaction is the same for both Java stream types

Reading With Streams

As we mentioned, each stream is used either to read from or write to.

Firstly, let us see how we can read data from Java streams.

Java streams

The base class to read binary data in Java is the InputStreamAnd the base class to read text data is called the Readerclass.

Both classes almost have the same two methods:

  •  int read()
  •  int read(byte/char[] buff)

Notice that in both scenarios, they give back an Integer value. These are interpreted values. An integer is a 32-bit container, so it will work in both cases.

The difference between the two:

  • The InputStream works with bytes, which are 8-bits.
  • The Reader works with unicode characters, which are 16-bits.

Read Bytes With InputStream

InputStream input = // create input stream
int result;
while(result = >= 0) //Indicates the end-of-stream with a return value of -1
byte byteVal = (byte)result;
// do something with byteVal

Read Text with Reader

Reader reader = // create reader
int result;
while(result = >= 0)//Indicates the end-of-stream with a return value of -1
char charVal = (char)result;
// do something with charVal

Note that if you would like to retrieve the value, you simply need to cast the result to the appropriate type — in this case, byte or char.

Writing With Streams

To write data, there are two base classes similar to the read streams.

  •  OutputStream (for bytes)
  •  Writer (for text)
Java Text streams
To write with Java streams is more straightforward than reading them. Both classes have a few   write  methods with the  void return type.

Writing Bytes With OutputStream

To write with OutputStream, you can pass a single byte the write method, or you can pass a byte array as well.

OutputStream output // create output stream;
Byte byteVal = 100;
byte[] byteBuff = {0, 10, 20};

Writing Characters With Writer

To write with the Writer class, you can pass a simple character, character array, or String to its  write method.

Writer writer // create output stream;
char charVal = 'c';
char[] charArray = {'c', 'h', 'a', 'r'};
String stringVal = "String";

As you can see, you need much less code to write than to read.

Common Java Stream Classes

Above, I wrote about the base stream classes. Now, let us go a bit deeper and talk about the different implementations.

Common Input/OutputStream Derived Classes

Java streams
  •  ByteArrayInputStream /  ByteArrayOutputStream – Enables us to create a stream over a byte array
  •  PipedInputStream PipedOutputStream – This is much like a producer-consumer concept. A piped output stream can be connected to a piped input stream to create a communications pipe. The piped output stream is the sending end of the pipe. Typically, data is written to a PipedOutputStream object by one thread and data is read from the connected PipedInputStream by some other thread.
  •  FileInputStream /  FileOutputStream – These allow us to create streams over files.

Common Reader/Writer Derived Classes

Java Reader/ Writer streams
Above are examples of Reader/Writer stream implementations.
  •  CharArrayReader / Writer – Allows creating streams over characters
  •  StringReader / Writer – Allows creating stream over Strings
  •  PipedReader / Writer – Allows creating a stream in a Producer/Consumer relationship over text. Similarly to the PipedOutput InputStream 
  •  InputStreamReader / Writer – Allows us to create a stream over an Input OutputStream 
  •  FileReader / Writer – These are delivered from the least mentioned above. It allows us to make a stream over text files.

Stream Errors and Clean-Up

So far, we looked at the general features of streams, but we have not considered all the realities of working with them.

Stream Realities

Java streams

Let us see the two main groups here.



  • Streams are backed by physical storage, which often exists outside the Java runtime, like files or network connections.
  • Hence, Java may not reliably clean up, so we need to do our own reliable clean-up. We need to close the Streams when we are done with them.


  • Streams implement the Closeable interface, which implements one single close method. So, this is our responsibility to call it.

Let us see a simple solution:

Reader reader;
reader = // create output stream;
// do something with reader;
}catch (IOException e) {
//handle exception
}finally {
if(reader != null)

The problem with the above example is that you always need to implement it. Usually, we use Streams frequently, so it should be done automatically. Let us see how we can achieve it.

Automating Clean-Up

  •  AutoClosable interface
    • One method: close
    • The base interface of the Closable interface, so every Stream supports it.
    • Provides support for try-with-resources


  • Automates the clean-up of one or more resources
    • A “resource” is any type that implements  AutoClosable
  • Syntax similar to traditional try statement
  • Optionally includes catch block(s)
    • Handle try body
    • Handle close method call

Working With Try-With-Resources

Here, I provided a simple example of how we can use the automatic close of streams with try-with-resources block.

I will use it through a FileInputStream. We will talk about this specific stream later.

 try(FileInputStream input = new FileInputStream("file1.txt")) {
        int data =;
        while(data != -1){
            System.out.print((char) data);
            data =;

With the above approach, you do not need to investigate further work to close your streams.


In this article, we talked about the fundamentals of Java streams, what are they, how they work, and how you can use them. In my next article, I will dig a bit deeper, and I will write about more advanced Stream topics like chaining, buffered streams, and how to use file systems with it. So stay tuned!

See more about streams in the official Java documentation here.

RESTful API Design Principle: Deciding Levels of Granularity

Granularity is an essential principle of REST API design. As we understand, business functions divided into many small actions are fine-grained, and business functions divided into large operations are coarse-grained.

However, discussions about what level of granularity that needs to be in APIs may vary, we will get distinct suggestions, and even end up in debates. Regardless, its best to decide based upon business functions and its use cases, as granularity decisions would undoubtedly vary on a case by case basis.

This article discusses a few points on how API designers would need to choose their RESTful service granularity levels.

Image title

Coarse-Grained and Fine-Grained APIs

In some cases, calls across the network may be expensive, so to minimize them, coarse-grained APIs may be the best fit, as each request from the client forces lot of work at the server side, and in fine-grained, many calls are required to do the same amount of work at the client side.

Example: Consider a service returns customer orders in a single call. In case of fine-grained, it returns only the customer IDs, and for each customer id, the client needs to make an additional request to get details, so n+1 calls need to be made by the clients. It may be expensive round trips regarding its performance and response times over the network.

In a few other cases, APIs should be designed at the lowest practical level of granularity, because combining them is possible and allowed in ways that they suit the customer needs.

Example: An electronic form submission may need to collect addresses as well as, say, tax information. In this case, there are two functions: one is a collection of applicant’s whereabouts, and another is a collection of tax details. Each task needs to be addressed with a distinct API and requires a separate service because an address change is logically a different event and not related to tax time reporting, i.e., why one needs to submit the tax information (again) for an address change.

Levels of Granularity

Level of granularity should satisfy the specific needs of business functions or use cases. While the goal is to minimize calls across the network and for better performance, we must understand the set of operations that API consumers require and how they would give a better idea of the “correctly-grained” APIs in our designs.

At times, it may be appropriate that the API design supports both coarse-grained as well as fine-grained to give the flexibility for the API developers to choose the right APIs for their use cases.


The following points may serve as some basic guidelines for the readers to decide their APIs granularity levels in their API modeling.

  • In general, consider that the services may be coarse-grained, and APIs are fine-grained.
  • Maintain a balance between the amount of response data and the number of resources required to provide that data. It will help decide the granularity.
  • The types of performed operations on the data should also be considered as part of the design when defining the granularity.
  • Read requests are normally coarse-grained. Returning all information as required to render the page; it won’t hurt as much as two separate API calls in some cases.
  • On the other hand, write requests must be fine-grained. Find out everyday operations clients needs, and provide a specific API for that use case.
  • At times, you should use medium grained, i.e., neither fine-grained or coarse-grained. An example could be as seen in the following sample where the nested resources are within two levels deep.Image title

While the above guideline may understandably lead to more API deployment units, this can cause annoyances down the line. There are patterns, especially the API Gateway, that bring a better orchestration with those numerous APIs. Orchestrating the APIs with optimized endpoints, request collapsing, and much more helps in addressing the granularity challenges.

Sidecar Design Pattern in Your Microservices Ecosystem

The sidecar design pattern is gaining popularity and wider adoption within the community. Building a microservice architecture that is highly scalable, resilient, secure, and observable is challenging. The evolution of service mesh architecture has been a game changer. It shifts the complexity associated with microservice architecture to a separate infrastructure layer and provides a lot of functionalities like load balancing, service discovery, traffic management, circuit breaking, telemetry, fault injection, and more.

Read my last post to understand the concepts behind service mesh, why it is needed for your cloud native applications, and the reasons for its popularity: The Rise of Service Mesh Architecture.

What Is a Sidecar Pattern?

Segregating the functionalities of an application into a separate process can be viewed as a Sidecar pattern. The sidecar design pattern allows you to add a number of capabilities to your application without additional configuration code for third-party components.

As a sidecar is attached to a motorcycle, similarly in software architecture a sidecar is attached to a parent application and extends/enhances its functionalities. A sidecar is loosely coupled with the main application.

Let me explain this with an example. Imagine that you have six microservices talking with each other in order to determine the cost of a package.

Each microservice needs to have functionalities like observability, monitoring, logging, configuration, circuit breakers, and more. All these functionalities are implemented inside each of these microservices using some industry standard third-party libraries.

But, is this not redundant? Does it not increase the overall complexity of your application? What happens if your applications are written in different languages — how do you incorporate the third-party libraries which are generally specific to a language like .NET, Java, Python, etc.?

Benefits of Using a Sidecar Pattern:

  • Reduces the complexity in the microservice code by abstracting the common infrastructure-related functionalities to a different layer.
  • Reduces code duplication in a microservice architecture since you do not need to write configuration code inside each microservice.
  • Provide loose coupling between application code and the underlying platform.

How Does the Sidecar Pattern Work?

The service mesh layer can live in a sidecar container that runs alongside your application. Multiple copies of the same sidecar are attached alongside each of your applications.

All the incoming and outgoing network traffic from an individual service flows through the sidecar proxy. As a result, the sidecar manages the traffic flow between microservices, gathers telemetry data, and enforces policies. In a sense, the service is not aware of the network and knows only about the attached sidecar proxy. This is really the essence of how the sidecar pattern works — it abstracts away the network dependency to the sidecar.

Sidecar Design Pattern

Inside a service mesh, we have the concept of a data plane and a control plane.

  • The data plane’s responsibility is to handle the communication between the services inside the mesh and take care of the functionalities like service discovery, load balancing, traffic management, health checks, etc.
  • The control plane’s responsibility is to manage and configure the sidecar proxies to enforce policies and collect telemetry.

In the Kubernetes and Istio world, you can inject the sidecars inside a pod. Istio uses the sidecar model with Envoy as the proxy.

Envoy from Lyft is the most popular open source proxy designed for cloud native applications. Envoy runs along side every service and provides the necessary features in a platform agnostic manner. All traffic to your service flows through the Envoy proxy.

The shift from monolith to microservices has enabled organizations to deploy applications independently and at scale. In a container and Kubernetes world, the sidecar design pattern is more compatible. The sidecars abstract away complexity from the application and handle functionalities like service discovery, traffic management, load balancing, circuit breaking, etc.

You can learn more about the sidecar pattern here:

Final Keyword and JVM Memory Impact

For a majority of developers, the keyword final is a well-known concept when we talk about composing a Java class. In general, it forbids the overriding of classes and methods, as well as changes a variable that has been already initialized. However, this is only one part (and I think the smaller part) of the final keyword definition. The other part is the connection to JVM memory, and that’s actually what this post is about!

I ‘d like to discuss this topic further by providing some examples, but if you want to really dive deeper, then I recommend Chapter 17 from Java Language Specification. Let’s dive right in!

Why Should I Bother Adding the Final Keyword?

I know that I’m not going to change the initialized value in my very small class, so why should care about this keyword; isn’t it just blowing up my concise Java source file?

NO! There is other cool stuff from the final keyword.

  • Guarantees visibility in a multi-threaded application
  • Safe initialization for objects, arrays, and collections

And please, don’t argue that you don’t use threads in your application — it is very likely that you use a framework that actually does, just under the hood.

How I Actually Reason With Object Initialization

Let’s step back a bit from the final keyword and demonstrate what object initialization might look like. It helps us understand where racing between threads may appear.

Image title

The picture above shows a small Java class and a description of how the initialization of that class could be done in reality. There is no concept of classes in a native language, which means the program needs to be expressed by instructions, one after another.

In this case, we can see that the object itself was created in the first line, and then, we populated the fields and published the reference to make it available and accessible. Everything looks fine, but there are several important considerations:

  • To make your program faster, the JVM and CPU can reorder the instructions to delay store instruction on some variable as late as possible. A very simple explanation would be: JVM and CPU can do everything with your code until the PROGRAM ORDER is satisfied. This means that the JVM can move store and read instructions but the behavior of your program must remain unchanged (we can go deeper into that in a different blog post).

So far, so good, but why am I supposed to be scared when the behavior of my program cannot be changed?

  • PROGRAM ORDER is followed only in a single thread. This means that if you run your application in a multi-threaded environment, then you have to inform the JVM that there is a possibility to complete a race condition when the JVM starts optimizing your code.
  • In a nutshell, it means that your second thread can see a partially initialized object, let’s say, because a step “publish reference (the last step from the picture) can be reordered with the field assignment “temp.y,” and then, the second thread will see a reference to the object with null value instead of the value “2.”

Let’s Introduce the Term: Safe Publication

The question is: How can we publish the shared object (a global variable) and be sure that all threads in our program will see the properly initialized object?

  • Initialize an object reference from a static initializer
    • This is a typical singleton holder pattern
    • The reason why this is safe is that because the JVM always performs object initialization on a single thread and ensures that the initialization happens before everything that uses that class (no other synchronization needed)
  • Use volatile or java.util.concurrent.atomic.Reference
  • Use locks to guard the global variable
  • Or … use the final field and never leak THIS from the constructor of the given class!

How Specification Ensures Safe Publication Using a Final Field

We already mentioned that the JVM or CPU is able to reorder everything and make your program faster. This means that we have to have some mechanism to inform the entire environment — please, don’t optimize this part.

  • JVM adds a synthetic freeze action (yes, this is an official name) before a constructor completes
    • Freeze action is added at runtime, and there is no bytecode instruction for this
  • Freeze ensures that the second thread sees null or an object reference with a fully constructed field, which was marked as final
  • On a CPU level, the freeze is implemented as a store barrier (sfence instruction on x86)
  • Other operations after the freeze are racing between two threads and must be properly synchronized

The JVM ensures that the second thread sees the state of the final field corresponding to the freeze action — no more, no less.

Image title

First, let’s agree on how I described steps of the execution:

  • Variabletemp is the given object that we want to publish.
  • Using  nfv = temp, we expose the publication of a reference of our object.
  •  <freeze value> means the point of the execution where the JVM inserts the freeze action.

On the picture above, we can see a very simple example of what can go wrong. We’ve already mentioned exactly this case which fails because of reordering the linetemp.value = 1 after the object publication nfv = temp.

On the contrary, the example on the right side shows fully and properly exposed our object to all threads. Freeze ensures that there is no reordering, which means that the publication of our object can be executed only when our field is properly pushed to a memory where it can be shared with other threads (Main Memory, or shared via Cache Coherence).

Image title

These examples show that the final field can help us correctly expose, even collections and arrays:

  • Collections and arrays are properly exposed, even with all the values inside.
  • The values inside the collection are properly exposed even if they don’t have final fields
  • BUT if we add a new value after the construction of our object, then this item is not supposed to be exposed properly, and Thread 2 can see the collection in both possible states.

Image title

The  Composition class shows a simple fact: Even if an object with non-final fields is exposed using a final field, then the inner object itself becomes properly exposed.

Image title

This example is very tricky. Don’t count on the behavior of an implementation, always code against the specification!

OpenJDK Hotspot behaves in a way that even if only one field is marked as final, it ensures that the freeze action is added “at the end of our constructor,” and flushing the CPU write buffers ensures that all other fields become visible to other threads. It works; you can check out other examples on my GitHub (link below) with JStress and give it a shot.
Image title

ConcurrentHashMap ensures that all values added to our Map are visible immediately to threads (internally synchronized).

However, it does not mean that the field holding the map, which is not marked as final, is visible to other threads after ConcurrentUtil construction. In this case, we can see the object with null value instead of initialized  ConcurrentHashMap.

What If I Cannot Use Final Fields…

We sometimes reach the situation where we are not able to modify a class and change all of its fields to final. So, is there any other way to expose the class properly with all the non-final fields?

We can use approaches that ensure we get safe publication for free:

  • Static Initializer (Singleton Holder pattern)
  • Thread-confinement
    • Run a block of code (task, job, HTTP request, …) only on one dedicated thread
    • Frameworks with a matured and well-document thread model, e.g. Event Loops
  • Stack-confinement
    • An object is reachable only through local variables
  • ThreadLocal
    • Maintains a per-thread value

Or, we can use some synchronization tools:

  • Synchronization block (implicit lock)
  • Explicit locks, volatile

The java.util.concurrent package, which does not ensure that a non-final field itself will be properly visible, contains objects that are used by those classes and makes sure they are properly exposed and visible to other threads. The classes below passes the object from one thread to another via some internal memory barrier (e.g. unsafe, or volatile field) that means that the object, which is created and written to any structure below by Thread A and then is read by Thread B, is fully constructed even if the object contains non-final fields.

Image title

  • CAS Operations
    •  Atomic*,  *AdderAtomic*FieldUpdater
  • java.util.concurrent package
    •  SynchronizedMapConcurrentHashMap
    •  CopyOnWriteArray(List|Set) 
    •  Synchronized(List|Set) 
    •  BlockingQueueConcurrentLinkedQueue
  •  VarHandlesUnsafe?

Thank you for reading my article and please leave comments below. If you would like to be notified about new posts, then start following me on Twitter: @p_bouda.