Custom Search API

Google Custom Search enables you to create a search engine for your website, your blog, or a collection of websites. You can configure your search engine to search both web pages and images. You can fine-tune the ranking, customize the look and feel of the search results.

The Custom Search API has a number of features available but we are going to use only one that is searching across the web for a specific collection of images. The images are going to be stored locally and there are going to be processed in the main application.

The Custom Search API with JSON / Atom lets you develop applications to retrieve and display results from the Google Custom Search programmatically. With this we can search the web and retrieve images in a JSON like format and save the images in a local server.

To create a custom search it is necessary to create a Custom Search Engine in the google web API. The engine provides 100 search queries per day for free. This is only to obtain the API key from the Search Engine and use it to create queries in Google Cloud Console.

Once having the authentication key the the next step is to make the searches with the API. The searches can be obtained using a HTTP GET Request with a URL. The output of the request is JSON format.

https://www.googleapis.com/customsearch/v1?parameters

To make a search, you only need three parameters, this information can be obtained from the Custom Search Engine and the Google Cloud Console.

• Search Engine ID (cx)
• A Public API access from the cloud Console (key)
• Something to Search (q)

A web search for “Picasso” would look like something like this

https://www.googleapis.com/customsearch/v1?key="YOUR_KEY"&cx="ENGINE_API"&q="picasso&alt=json

More information about all the query parameters:

https://developers.google.com/custom-search/json-api/v1/using_rest

So, to search only images with file type .jpg and .png we need the following code.

https://www.googleapis.com/customsearch/v1?key="YOUR_KEY"&cx="ENGINE_API"&q="picasso"&searchType="image"&fileType="png,jpg"&alt=json

more about the Source Custom Search here.

Google Custom Search API lets you search the web programmatically via HTTP requests. Once making a request it is possible to obtain the information in a JSON format.

We are going to use Processing and some external Java libraries to achieve our goal of mining custom images from the web. Java provides a set of libraries that lets you easily connect via HTTP protocol.

We start by making a simple Processing sketch.

First we import all the Java libraries like network communication, exception handling and input/output files.

import java.net.*;
import java.net.MalformedURLException;
import java.net.URISyntaxException;
import java.io.Reader.*;

The information from the search result is being saved in class called GResults, more about the class check the gitHub link.

void Search()  throws MalformedURLException, URISyntaxException, IOException {

  String key = "";
  String qry = "nebulas"; 
  String cx  = "";
  String fileType = "png,jpg";
  String searchType = "image";

  URL url = new URL ("https://www.googleapis.com/customsearch/v1?key=" +key+ "&cx=" +cx+ "&q=" +qry+"&fileType="+fileType+"&searchType="+searchType+"&alt=json");

  HttpURLConnection conn = (HttpURLConnection) url.openConnection();
  conn.setRequestMethod("GET");
  conn.setRequestProperty("Accept", "application/json");
  BufferedReader br = new BufferedReader(new InputStreamReader ( ( conn.getInputStream() ) ) );
  GResults results = new Gson().fromJson(br, GResults.class);

  conn.disconnect();
}

We need to parse the JSON obtained from the search query to Java objects. To do this, we are going to use GSon, a open source java library. With this library it is possible to convert JSON to java objects and vice-versa. More about Gson [here](https://code.google.com/p/google-gson/)

We use the following code just to obtain the data from the search result.

  for (int i=0; i < 10; i++) {
    String path  = results.getLink(i)
    loadImage(path);
  }

The string `path` will contain the path of the image found from the query, we only need to retrieve the `path` of the image, this is because we can use a easy built-in function from processing called `loadImage(path)`. This function loads images from any given path, it could be a local path or a URL path.

The whole project is located on github Processing-Sketch/CustomSearchJSON