Java Read URL to String

In my current project, I had a requirement to read the WSDL file from the URL and store it into the database as CLOB.

There was no validation required, so it was kind of reading URL content to String and then storing it into the database table.

Java Read URL to String

Here is the program I wrote in java to read URL to String.

package com.journaldev.java;

import java.io.BufferedReader;
import java.io.InputStreamReader;
import java.net.URL;
import java.net.URLConnection;

public class ReadURLToString {
	public static void main(String[] args) throws Exception {
		URL test = new URL("https://journaldev.com");
		URLConnection uc = test.openConnection();
		uc.addRequestProperty("User-Agent", "Mozilla/4.0");
		BufferedReader in = new BufferedReader(new InputStreamReader(uc
				.getInputStream()));
		String inputLine;
		StringBuilder sb = new StringBuilder();
		while ((inputLine = in.readLine()) != null) {
			sb.append(inputLine);
			System.out.println(inputLine);
		}

		in.close();
		System.out.println("HTML Data:" + sb.toString());
	}
}

When we run the above program, it produces the following output.

Most of the code is self-understood except setting the user agent. For some websites, if you don’t set User-Agent header, you might get 403 error code. Just comment it in above program and you will get below error.

Exception in thread "main" java.io.IOException: Server returned HTTP response code: 403 for URL: https://www.journaldev.com/
	at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1876)
	at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1474)
	at ReadURLToString.main(ReadURLToString.java:12)

If you have landed here and looked for something similar, feel free to use the above code and don’t forget to comment or share with others. That’s all for reading URL content in java program.

Reference: Java URLConnection API Doc