Ultimate Guide for Reading Files in Java

reading file line by line in Java with BufferedReader

Reading files in Java is the cause for a lot of confusion. There are multiple ways of accomplishing the same task and it’s often not clear which file reading method is best to use. Something that’s quick and dirty for a small example file might not be the best method to use when you need to read a very large file. Something that worked in an earlier Java version, might not be the preferred method anymore.

This article aims to be the definitive guide for reading files in Java 7, 8 and 9. I’m going to cover all the ways you can read files in Java. Too often, you’ll read an article that tells you one way to read a file, only to discover later there are other ways to do that. I’m actually going to cover 15 different ways to read a file in Java. I’m going to cover reading files in multiple ways with the core Java libraries as well as two third party libraries.

But that’s not all – what good is knowing how to do something in multiple ways if you don’t know which way is best for your situation?

I also put each of these methods to a real performance test and document the results. That way, you will have some hard data to know the performance metrics of each method.

Methodology

JDK Versions

Java code samples don’t live in isolation, especially when it comes to Java I/O, as the API keeps evolving. All code for this article has been tested on:

  • Java SE 7 (jdk1.7.0_80)
  • Java SE 8 (jdk1.8.0_162)
  • Java SE 9 (jdk-9.0.4)

When there is an incompatibility, it will be stated in that section. Otherwise, the code works unaltered for different Java versions. The main incompatibility is the use of lambda expressions which was introduced in Java 8.

Java File Reading Libraries

There are multiple ways of reading from files in Java. This article aims to be a comprehensive collection of all the different methods. I will cover:

Closing File Resources

Prior to JDK7, when opening a file in Java, all file resources would need to be manually closed using a try-catch-finally block. JDK7 introduced the try-with-resources statement, which simplifies the process of closing streams. You no longer need to write explicit code to close streams because the JVM will automatically close the stream for you, whether an exception occurred or not. All examples used in this article use the try-with-resources statement for importing, loading, parsing and closing files.

File Location

All examples will read test files from C:\temp.

Encoding

Character encoding is not explicitly saved with text files so Java makes assumptions about the encoding when reading files. Usually, the assumption is correct but sometimes you want to be explicit when instructing your programs to read from files. When encoding isn’t correct, you’ll see funny characters appear when reading files.

All examples for reading text files use two encoding variations:
Default system encoding where no encoding is specified and explicitly setting the encoding to UTF-8.

Download Code

All code files are available from Github.

Code Quality and Code Encapsulation

There is a difference between writing code for your personal or work project and writing code to explain and teach concepts.

If I was writing this code for my own project, I would use proper object-oriented principles like encapsulation, abstraction, polymorphism, etc. But I wanted to make each example stand alone and easily understood, which meant that some of the code has been copied from one example to the next. I did this on purpose because I didn’t want the reader to have to figure out all the encapsulation and object structures I so cleverly created. That would take away from the examples.

For the same reason, I chose NOT to write these example with a unit testing framework like JUnit or TestNG because that’s not the purpose of this article. That would add another library for the reader to understand that has nothing to do with reading files in Java. That’s why all the example are written inline inside the main method, without extra methods or classes.

My main purpose is to make the examples as easy to understand as possible and I believe that having extra unit testing and encapsulation code will not help with this. That doesn’t mean that’s how I would encourage you to write your own personal code. It’s just the way I chose to write the examples in this article to make them easier to understand.

Exception Handling

All examples declare any checked exceptions in the throwing method declaration.

The purpose of this article is to show all the different ways to read from files in Java – it’s not meant to show how to handle exceptions, which will be very specific to your situation.

So instead of creating unhelpful try catch blocks that just print exception stack traces and clutter up the code, all example will declare any checked exception in the calling method. This will make the code cleaner and easier to understand without sacrificing any functionality.

Future Updates

As Java file reading evolves, I will be updating this article with any required changes.

File Reading Methods

I organized the file reading methods into three groups:

  • Classic I/O classes that have been part of Java since before JDK 1.7. This includes the java.io and java.util packages.
  • New Java I/O classes that have been part of Java since JDK1.7. This covers the java.nio.file.Files class.
  • Third party I/O classes from the Apache Commons and Google Guava projects.

Classic I/O – Reading Text

1a) FileReader – Default Encoding

FileReader reads in one character at a time, without any buffering. It’s meant for reading text files. It uses the default character encoding on your system, so I have provided examples for both the default case, as well as specifying the encoding explicitly.


1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
import java.io.FileReader;
import java.io.IOException;

public class ReadFile_FileReader_Read {
  public static void main(String [] pArgs) throws IOException {
    String fileName = "c:\\temp\\sample-10KB.txt";
   
    try (FileReader fileReader = new FileReader(fileName)) {
      int singleCharInt;
      char singleChar;
      while((singleCharInt = fileReader.read()) != -1) {
        singleChar = (char) singleCharInt;
       
        //display one character at a time
        System.out.print(singleChar);
      }
    }
  }
}

1b) FileReader – Explicit Encoding (InputStreamReader)

It’s actually not possible to set the encoding explicitly on a FileReader so you have to use the parent class, InputStreamReader and wrap it around a FileInputStream:


1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
import java.io.FileInputStream;
import java.io.IOException;
import java.io.InputStreamReader;

public class ReadFile_FileReader_Read_Encoding {
  public static void main(String [] pArgs) throws IOException {
    String fileName = "c:\\temp\\sample-10KB.txt";
    FileInputStream fileInputStream = new FileInputStream(fileName);
   
    //specify UTF-8 encoding explicitly
    try (InputStreamReader inputStreamReader =
      new InputStreamReader(fileInputStream, "UTF-8")) {

      int singleCharInt;
      char singleChar;
      while((singleCharInt = inputStreamReader.read()) != -1) {
        singleChar = (char) singleCharInt;
        System.out.print(singleChar); //display one character at a time
      }
    }
  }
}

2a) BufferedReader – Default Encoding

BufferedReader reads an entire line at a time, instead of one character at a time like FileReader. It’s meant for reading text files.


1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
import java.io.BufferedReader;
import java.io.FileReader;
import java.io.IOException;

public class ReadFile_BufferedReader_ReadLine {
  public static void main(String [] args) throws IOException {
    String fileName = "c:\\temp\\sample-10KB.txt";
    FileReader fileReader = new FileReader(fileName);
   
    try (BufferedReader bufferedReader = new BufferedReader(fileReader)) {
      String line;
      while((line = bufferedReader.readLine()) != null) {
        System.out.println(line);
      }
    }
  }
}

2b) BufferedReader – Explicit Encoding

In a similar way to how we set encoding explicitly for FileReader, we need to create FileInputStream, wrap it inside InputStreamReader with an explicit encoding and pass that to BufferedReader:


1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
import java.io.BufferedReader;
import java.io.FileInputStream;
import java.io.IOException;
import java.io.InputStreamReader;

public class ReadFile_BufferedReader_ReadLine_Encoding {
  public static void main(String [] args) throws IOException {
    String fileName = "c:\\temp\\sample-10KB.txt";

    FileInputStream fileInputStream = new FileInputStream(fileName);
   
    //specify UTF-8 encoding explicitly
    InputStreamReader inputStreamReader = new InputStreamReader(fileInputStream, "UTF-8");
   
    try (BufferedReader bufferedReader = new BufferedReader(inputStreamReader)) {
      String line;
      while((line = bufferedReader.readLine()) != null) {
        System.out.println(line);
      }
    }
  }
}

Classic I/O – Reading Bytes

1) FileInputStream

FileInputStream reads in one byte at a time, without any buffering. While it’s meant for reading binary files such as images or audio files, it can still be used to read text file. It’s similar to reading with FileReader in that you’re reading one character at a time as an integer and you need to cast that int to a char to see the ASCII value.

By default, it uses the default character encoding on your system, so I have provided examples for both the default case, as well as specifying the encoding explicitly.


1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
import java.io.File;
import java.io.FileInputStream;
import java.io.FileNotFoundException;
import java.io.IOException;

public class ReadFile_FileInputStream_Read {
  public static void main(String [] pArgs) throws FileNotFoundException, IOException {
    String fileName = "c:\\temp\\sample-10KB.txt";
    File file = new File(fileName);
   
    try (FileInputStream fileInputStream = new FileInputStream(file)) {
      int singleCharInt;
      char singleChar;

      while((singleCharInt = fileInputStream.read()) != -1) {
        singleChar = (char) singleCharInt;
        System.out.print(singleChar);
      }
    }
  }
}

2) BufferedInputStream

BufferedInputStream reads a set of bytes all at once into an internal byte array buffer. The buffer size can be set explicitly or use the default, which is what we’ll demonstrate in our example. The default buffer size appears to be 8KB but I have not explicitly verified this. All performance tests used the default buffer size so it will automatically re-size the buffer when it needs to.


1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
import java.io.BufferedInputStream;
import java.io.File;
import java.io.FileInputStream;
import java.io.FileNotFoundException;
import java.io.IOException;

public class ReadFile_BufferedInputStream_Read {
  public static void main(String [] pArgs) throws FileNotFoundException, IOException {
    String fileName = "c:\\temp\\sample-10KB.txt";
    File file = new File(fileName);
    FileInputStream fileInputStream = new FileInputStream(file);
   
    try (BufferedInputStream bufferedInputStream = new BufferedInputStream(fileInputStream)) {
      int singleCharInt;
      char singleChar;
      while((singleCharInt = bufferedInputStream.read()) != -1) {
        singleChar = (char) singleCharInt;
        System.out.print(singleChar);
      }
    }
  }
}

New I/O – Reading Text

1a) Files.readAllLines() – Default Encoding

The Files class is part of the new Java I/O classes introduced in jdk1.7. It only has static utility methods for working with files and directories.

The readAllLines() method that uses the default character encoding was introduced in jdk1.8 so this example will not work in Java 7.


1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
import java.io.File;
import java.io.IOException;
import java.nio.file.Files;
import java.util.List;

public class ReadFile_Files_ReadAllLines {
  public static void main(String [] pArgs) throws IOException {
    String fileName = "c:\\temp\\sample-10KB.txt";
    File file = new File(fileName);

    List  fileLinesList = Files.readAllLines(file.toPath());

    for(String line : fileLinesList) {
      System.out.println(line);
    }
  }
}

1b) Files.readAllLines() – Explicit Encoding


1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
import java.io.File;
import java.io.IOException;
import java.nio.charset.StandardCharsets;
import java.nio.file.Files;
import java.util.List;

public class ReadFile_Files_ReadAllLines_Encoding {
  public static void main(String [] pArgs) throws IOException {
    String fileName = "c:\\temp\\sample-10KB.txt";
    File file = new File(fileName);

    //use UTF-8 encoding
    List  fileLinesList = Files.readAllLines(file.toPath(), StandardCharsets.UTF_8);

    for(String line : fileLinesList) {
      System.out.println(line);
    }
  }
}

2a) Files.lines() – Default Encoding

This code was tested to work in Java 8 and 9. Java 7 didn’t run because of the lack of support for lambda expressions.


1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
import java.io.File;
import java.io.IOException;
import java.nio.file.Files;
import java.util.stream.Stream;

public class ReadFile_Files_Lines {
  public static void main(String[] pArgs) throws IOException {
    String fileName = "c:\\temp\\sample-10KB.txt";
    File file = new File(fileName);

    try (Stream linesStream = Files.lines(file.toPath())) {
      linesStream.forEach(line -> {
        System.out.println(line);
      });
    }
  }
}

2b) Files.lines() – Explicit Encoding

Just like in the previous example, this code was tested and works in Java 8 and 9 but not in Java 7.


1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
import java.io.File;
import java.io.IOException;
import java.nio.charset.StandardCharsets;
import java.nio.file.Files;
import java.util.stream.Stream;

public class ReadFile_Files_Lines_Encoding {
  public static void main(String[] pArgs) throws IOException {
    String fileName = "c:\\temp\\sample-10KB.txt";
    File file = new File(fileName);

    try (Stream linesStream = Files.lines(file.toPath(), StandardCharsets.UTF_8)) {
      linesStream.forEach(line -> {
        System.out.println(line);
      });
    }
  }
}

3a) Scanner – Default Encoding

The Scanner class was introduced in jdk1.7 and can be used to read from files or from the console (user input).


1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
import java.io.File;
import java.io.FileNotFoundException;
import java.util.Scanner;

public class ReadFile_Scanner_NextLine {
  public static void main(String [] pArgs) throws FileNotFoundException {
    String fileName = "c:\\temp\\sample-10KB.txt";
    File file = new File(fileName);
   
    try (Scanner scanner = new Scanner(file)) {
      String line;
      boolean hasNextLine = false;
      while(hasNextLine = scanner.hasNextLine()) {
        line = scanner.nextLine();
        System.out.println(line);
      }
    }
  }
}

3b) Scanner – Explicit Encoding


1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
import java.io.File;
import java.io.FileNotFoundException;
import java.util.Scanner;

public class ReadFile_Scanner_NextLine_Encoding {
  public static void main(String [] pArgs) throws FileNotFoundException {
    String fileName = "c:\\temp\\sample-10KB.txt";
    File file = new File(fileName);
   
    //use UTF-8 encoding
    try (Scanner scanner = new Scanner(file, "UTF-8")) {
      String line;
      boolean hasNextLine = false;
      while(hasNextLine = scanner.hasNextLine()) {
        line = scanner.nextLine();
        System.out.println(line);
      }
    }
  }
}

New I/O – Reading Bytes

Files.readAllBytes()

Even though the documentation for this method states that “it is not intended for reading in large files” I found this to be the absolute best performing file reading method, even on files as large as 1GB.


1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
import java.io.File;
import java.io.IOException;
import java.nio.file.Files;

public class ReadFile_Files_ReadAllBytes {
  public static void main(String [] pArgs) throws IOException {
    String fileName = "c:\\temp\\sample-10KB.txt";
    File file = new File(fileName);

    byte [] fileBytes = Files.readAllBytes(file.toPath());
    char singleChar;
    for(byte b : fileBytes) {
      singleChar = (char) b;
      System.out.print(singleChar);
    }
  }
}

3rd Party I/O – Reading Text

Commons – FileUtils.readLines()

Apache Commons IO is an open source Java library that comes with utility classes for reading and writing text and binary files. I listed it in this article because it can be used instead of the built in Java libraries. The class we’re using is FileUtils.

For this article, version 2.6 was used which is compatible with JDK 1.7+

Note that you need to explicitly specify the encoding and that method for using the default encoding has been deprecated.


1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
import java.io.File;
import java.io.IOException;
import java.util.List;

import org.apache.commons.io.FileUtils;

public class ReadFile_Commons_FileUtils_ReadLines {
  public static void main(String [] pArgs) throws IOException {
    String fileName = "c:\\temp\\sample-10KB.txt";
    File file = new File(fileName);
 
    List fileLinesList = FileUtils.readLines(file, "UTF-8");

    for(String line : fileLinesList) {
      System.out.println(line);
    }
  }
}

Guava – Files.readLines()

Google Guava is an open source library that comes with utility classes for common tasks like collections handling, cache management, IO operations, string processing.

I listed it in this article because it can be used instead of the built in Java libraries and I wanted to compare its performance with the Java built in libraries.

For this article, version 23.0 was used.

I’m not going to examine all the different ways to read files with Guava, since this article is not meant for that. For a more detailed look at all the different ways to read and write files with Guava, have a look at Baeldung’s in depth article.

When reading a file, Guava requires that the character encoding be set explicitly, just like Apache Commons.

Compatibility note: This code was tested successfully on Java 8 and 9. I couldn’t get it to work on Java 7 and kept getting “Unsupported major.minor version 52.0” error. Guava has a separate API doc for Java 7 which uses a slightly different version of the Files.readLine() method. I thought I could get it to work but I kept getting that error.


1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
import java.io.File;
import java.io.IOException;
import java.util.List;

import com.google.common.base.Charsets;
import com.google.common.io.Files;

public class ReadFile_Guava_Files_ReadLines {
  public static void main(String[] args) throws IOException {
    String fileName = "c:\\temp\\sample-10KB.txt";
    File file = new File(fileName);

    List fileLinesList = Files.readLines(file, Charsets.UTF_8);
   
    for(String line : fileLinesList) {
      System.out.println(line);
    }
  }
}

Performance Testing

Since there are so many ways to read from a file in Java, a natural question is “What file reading method is the best for my situation?” So I decided to test each of these methods against each other using sample data files of different sizes and timing the results.

Each code sample from this article displays the contents of the file to a string and then to the console (System.out). However, during the performance tests the System.out line was commented out since it would seriously slow down the performance of each method.

Each performance test measures the time it takes to read in the file – line by line, character by character, or byte by byte without displaying anything to the console. I ran each test 5-10 times and took the average so as not to let any outliers influence each test. I also ran the default encoding version of each file reading method – i.e. I didn’t specify the encoding explicitly.

Dev Setup

The dev environment used for these tests:

  • Intel Core i7-3615 QM @2.3 GHz, 8GB RAM
  • Windows 8 x64
  • Eclipse IDE for Java Developers, Oxygen.2 Release (4.7.2)
  • Java SE 9 (jdk-9.0.4)

Data Files

GitHub doesn’t allow pushing files larger than 100 MB, so I couldn’t find a practical way to store my large test files to allow others to replicate my tests. So instead of storing them, I’m providing the tools I used to generate them so you can create test files that are similar in size to mine. Obviously they won’t be the same, but you’ll generate files that are similar in size as I used in my performance tests.

Random String Generator was used to generate sample text and then I simply copy-pasted to create larger versions of the file. When the file started getting too large to manage inside a text editor, I had to use the command line to merge multiple text files into a larger text file:

copy *.txt sample-1GB.txt

I created the following 7 data file sizes to test each file reading method across a range of file sizes:

  • 1KB
  • 10KB
  • 100KB
  • 1MB
  • 10MB
  • 100MB
  • 1GB

Performance Summary

There were some surprises and some expected results from the performance tests.

As expected, the worst performers were the methods that read in a file character by character or byte by byte. But what surprised me was that the native Java IO libraries outperformed both 3rd party libraries – Apache Commons IO and Google Guava.

What’s more – both Google Guava and Apache Commons IO threw a java.lang.OutOfMemoryError when trying to read in the 1 GB test file. This also happened with the Files.readAllLines(Path) method but the remaining 7 methods were able to read in all test files, including the 1GB test file.

The following table summarizes the average time (in milliseconds) each file reading method took to complete. I highlighted the top three methods in green, the average performing methods in yellow and the worst performing methods in red:

The following chart summarizes the above table but with the following changes:

I removed java.io.FileInputStream.read() from the chart because its performance was so bad it would skew the entire chart and you wouldn’t see the other lines properly
I summarized the data from 1KB to 1MB because after that, the chart would get too skewed with so many under performers and also some methods threw a java.lang.OutOfMemoryError at 1GB

 

The Winners

The new Java I/O libraries (java.nio) had the best overall winner (java.nio.Files.readAllBytes()) but it was followed closely behind by BufferedReader.readLine() which was also a proven top performer across the board. The other excellent performer was java.nio.Files.lines(Path) which had slightly worse numbers for smaller test files but really excelled with the larger test files.

The absolute fastest file reader across all data tests was java.nio.Files.readAllBytes(Path). It was consistently the fastest and even reading a 1GB file only took about 1 second.

The following chart compares performance for a 100KB test file:

You can see that the lowest times were for Files.readAllBytes(), BufferedInputStream.read() and BufferedReader.readLine().

The following chart compares performance for reading a 10MB file. I didn’t bother including the bar for FileInputStream.Read() because the performance was so bad it would skew the entire chart and you couldn’t tell how the other methods performed relative to each other:

Files.readAllBytes() really outperforms all other methods and BufferedReader.readLine() is a distant second.

The Losers

As expected, the absolute worst performer was java.io.FileInputStream.read() which was orders of magnitude slower than its rivals for most tests. FileReader.read() was also a poor performer for the same reason – reading files byte by byte (or character by character) instead of with buffers drastically degrades performance.

Both the Apache Commons IO FileUtils.readLines() and Guava Files.readLines() crashed with an OutOfMemoryError when trying to read the 1GB test file and they were about average in performance for the remaining test files.

java.nio.Files.readAllLines() also crashed when trying to read the 1GB test file but it performed quite well for smaller file sizes.

Performance Rankings

Here’s a ranked list of how well each file reading method did, in terms of speed and handling of large files, as well as compatibility with different Java versions.

RankFile Reading Method
1java.nio.file.Files.readAllBytes()
2java.io.BufferedFileReader.readLine()
3java.nio.file.Files.lines()
4java.io.BufferedInputStream.read()
5java.util.Scanner.nextLine()
6java.nio.file.Files.readAllLines()
7org.apache.commons.io.FileUtils.readLines()
8com.google.common.io.Files.readLines()
9java.io.FileReader.read()
10java.io.FileInputStream.Read()

Conclusion

I tried to present a comprehensive set of methods for reading files in Java, both text and binary. We looked at 15 different ways of reading files in Java and we ran performance tests to see which methods are the fastest.

The new Java IO library (java.nio) proved to be a great performer but so was the classic BufferedReader.