Extracting PDF text using Apache Tika

One of the most difficult file types for parsing and extracting data is PDF. Some PDFs are not even possible to parse because they are password-protected, while some others contain scanned texts and images. This dynamic file type, therefore, sometimes becomes the worst nightmare for data scientists. This recipe demonstrates how to extract text from PDF files using Apache Tika, given that the file is not encrypted or password-protected and contains text that is not scanned.

Getting ready

In order to perform this recipe we will require the following:

1. Download Apache Tika 1.10 JAR file from http://archive.apache.org/dist/tika/tika-app-1.10.jar, and include it in your Eclipse project as an external Java library.
2. Have any unlocked PDF file saved as testPDF.pdf on your C: drive.

How to do it...

1. Create a method named convertPdf(String), which takes the name of the PDF file to be converted as parameter:

           public void convertPDF(String fileName){ 
   

2. Create an input stream that will contain the PDF data as a stream of bytes:

           InputStream stream = null; 
   

3. Create a try block as follows:

           try{ 
   

4. Assign the file to the stream you have just created:

           stream = new FileInputStream(fileName); 
   

5. There are many different parsers offered in the Apache Tika package. If you do not know which parser you are going to use, or say you have not only PDFs but also other types of documents to get converted, you should use an AutoDetectParser as follows:

           AutoDetectParser parser = new AutoDetectParser(); 
   

6. Create a handler to handle the body content of the file. Note the -1 as the parameter of the constructor. Usually, Apache Tika is limited to handling files with at most 100,000 characters. The -1 value ensures that this limitation is overlooked by the body handler:

           BodyContentHandler handler = new BodyContentHandler(-1); 
   

7. Create a metadata object:

           Metadata metadata = new Metadata(); 
   

8. Call the parser() method of the parser object with all these objects you just created:

           parser.parse(stream, handler, metadata, new ParseContext()); 
   

9. Use the tostring() method of the handler object to get the body text extracted from the file:

           System.out.println(handler.toString()); 
   

10. Close the try block and complement it with a catch block and finally block, and close the method as follows:

            }catch (Exception e) { 
                          e.printStackTrace(); 
                     }finally { 
                         if (stream != null) 
                              try { 
                                   stream.close(); 
                              } catch (IOException e) { 
                                System.out.println("Error closing stream"); 
                              } 
                       } 
            } 
    

    The full method with the driver method in a class will be as follows. The method you have just created can be called by sending it the path and the name of the PDF file you need to convert, which is in your C: drive saved as testPDF.pdf:

    import java.io.FileInputStream; 
    import java.io.IOException; 
    import java.io.InputStream; 
    import org.apache.tika.metadata.Metadata; 
    import org.apache.tika.parser.AutoDetectParser; 
    import org.apache.tika.parser.ParseContext; 
    import org.apache.tika.sax.BodyContentHandler; 
     
    public class TestTika { 
         public static void main(String args[]) throws Exception { 
              TestTika tika = new TestTika(); 
              tika.convertPdf("C:/testPDF.pdf"); 
         } 
         public void convertPdf(String fileName){ 
              InputStream stream = null; 
              try { 
                  stream = new FileInputStream(fileName); 
                  AutoDetectParser parser = new AutoDetectParser(); 
                  BodyContentHandler handler = new BodyContentHandler(-1); 
                  Metadata metadata = new Metadata(); 
                  parser.parse(stream, handler, metadata, new 
                      ParseContext()); 
                  System.out.println(handler.toString()); 
              }catch (Exception e) { 
                  e.printStackTrace(); 
              }finally { 
                  if (stream != null) 
                       try { 
                            stream.close(); 
                       } catch (IOException e) { 
                            System.out.println("Error closing stream"); 
                       } 
              } 
         } 
    }