Search Result Scraping with Zenserp API using PHP

PHPZAG Team API, PHP, Tutorials

SERP (Search Engine Results Pages) data is the search result pages displayed by search Engines. Many businesses looking for large search data with multiple keywords and its not possible with manual keyword search. In that scenario, scraping system developed to scrape search result data.

But it’s a challenging task for developers to develop reliable system to scrape SERP data because it involves captcha-solving, managing proxy servers and parsing of the constantly changing markup of the search results.

So if you’re running a business or a developers and looking for reliable system to scrape SERP data, then you’re here at the right place. In this tutorial you will learn how to scrape SERP data using Zenserp API using PHP.

The Zenserp API is a super fast real-time Google search result scraping API. You don’t need to worry about captcha-solving, managing proxy servers etc. Just need to use interface directly from Zenserp API or you can integrate Zenserp API at your end to scrape the SERP data into your desired format like raw HTML or CSV data.

Also, read:


We will cover this tutorial in easy steps to integrate the Zenserp API with example using PHP.

Step1: Get Zenserp API Key

To access the Zenserp API, we need to sign up on Zenserp API to create account to access the user dashboard where we will get the API Key.

After creating account on Zenserp, we need to go the Dashboard and get API Key. We will use that API key to make HTTP request to Zenserp API URL like below.

https://app.zenserp.com/api/v2/search?apikey=APIKEY

Step2: Create Zenserp API Search Query

There parameters needs to pass with values to make request to Zenserp API to scrape the SERP data. Here we will pass q parameter with keyword to search for. This is a required parameter. We will also optional pass parameters such as strongcode>location parameter for the Geo-location used in the search query, pass search_engine parameter with url of the search engine fro making search query.

We can also pass other optional parameters like output parameter for result output (html or csv), gl for country code, hl for search interface language. For more parameters and details, you can gone through the documentation to use in your search query. Here we will pass following parameters. 


<?php 

$searchParameters = [
	'q' => '"Pied Piper',
	'location' => 'United States',
	'search_engine' => 'google.com',
	'gl' => 'US',
	'hl' => 'en'
];

?>

Step3: Make HTTP Request to Zenserp API

After storing search parameters data into an array, we will make HTTP request to Zenserp API with PHP Curl library. We will pass $searchParameters data with the request and also pass the API Key for the API authentication for the successful search request. The SERP data will be returned as HTML or CSV data, depending on what type of output data needed. We will store scrapped response data into $response variable.

<?php 

$apiKey = 'YOUR_API_KEY';

$ch = curl_init();

curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_HEADER, false);

curl_setopt($ch, CURLOPT_URL, "https://app.zenserp.com/api/v2/search?" . http_build_query($searchParameters));

curl_setopt($ch, CURLOPT_HTTPHEADER, array(
	"Content-Type: application/json",
	"apikey: $apiKey",
));

$response = curl_exec($ch);
curl_close($ch);

?>

Step4: Decode Response Data

Here we will decode the scrapped SERP response data into JSON data using PHP json_decode() and store into $resultData variable and print the entire JSON data.

<?php

$resultData = json_decode($response);
var_dump($resultData);

?>

Step5: Complete Code

Here is the complete example code to make HTTP request to Zenserp API with search query to get SERP data from Google search engine.

<?php

$apiKey = 'YOUR_API_KEY';

$searchParameters = [
	'q' => '"Pied Piper',
	'location' => 'United States',
	'search_engine' => 'google.com',
	'gl' => 'US',
	'hl' => 'en'
];

$ch = curl_init();

curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_HEADER, false);

curl_setopt($ch, CURLOPT_URL, "https://app.zenserp.com/api/v2/search?" . http_build_query($searchParameters));

curl_setopt($ch, CURLOPT_HTTPHEADER, array(
	"Content-Type: application/json",
	"apikey: $apiKey",
));

$response = curl_exec($ch);
curl_close($ch);

$resultData = json_decode($response);
var_dump($resultData);

?>

Below is the response JSON data after running above example code.

{
	"query": {
		"q": "Pied Piper",
		"hl": "English",
		"gl": "US",
		"location": "United States",
		"domain": "google.com",
		"device": "desktop",
		"url": "https://google.com/search?
q=Pied+Piper&uule=
w+CAIQICINVW5pdGVkIFN0YXRlcw&gl=US"
	},
	"organic": [
		{
			"position": 1,
			"title": "Pied Piper of Hamelin - Wikipedia",
			"url": "https://en.wikipedia.org/wiki/Pied_Piper_of_Hamelin",
			"destination": "https://en.wikipedia.org/
wiki/Pied_Piper_of_Hamelin",
			"description": "The Pied Piper of Hamelin is
 the titular character of a legend from the 
			town of Hamelin (Hameln), Lower Saxony, 
Germany, as well as the title of the fairy tale that ..."
		},
		{
			"position": 2,
			"questions": [
				{
					"question": "What is the meaning of the Pied Piper?"
				},
				{
					"question": "What is the legend of the Pied Piper?"
				},
				{
					"question": "What is the moral of the story pied piper?"
				}
			]
		},
		{
			"position": 3,
			"videos": [
				{
					"title": "Walt Disney's Fables - Pied Piper",
					"url": "https://www.youtube.com
/watch?v=UXOCHn7Vfec"
				},
				{
					"title": "The True History of The
 Pied Piper of Hamelin | Fairy Tales With Jen",
					"url": "https://www.youtube.com/
watch?v=obIAwAAcAAg"
				}
			]
		},
		{
			"position": 4,
			"title": "The Chilling True Story Behind the 
Pied Piper of Hamelin - The Portalist",
			"url": "https://theportalist.com/
the-chilling-true-story-behind-the-pied-piper-of-hamelin",
			"destination": "https://theportalist.com › 

The Chilling True Story Behind the Pied Piper of Hamelin",
			"description": "Aug 2, 2017 - 
Is there a true story behind the legend of the Pied Piper of Hamelin? 
			The myth may be based on a real-life tragedy."
		},
		{
			"position": 5,
			"localPack": [
				{
					"title": "Pied Piper Pub & Inn and The Cat House Lounge",
					"url": "http://www.cathouselounge.com/",
					"street": "Eureka Springs, AR",
					"stars": 4.3,
					"review": 482,
					"phone": null,
					"description": "Biker-friendly hangout with live music",
					"price": "$$",
					"type": "Pub",
					"extensions": null
				},
				{
					"title": "Pied Piper Pest, Termite & Lawn Service",
					"url": "http://thepiedpiper.biz/",
					"street": "Eufaula, OK",
					"stars": 5,
					"review": 130,
					"phone": "(918) 689-3000",
					"description": "Open ⋅ Closes 11:30PM",
					"price": "",
					"type": "Pest control service",
					"extensions": null
				},
				{
					"title": "The Pied Piper Bar",
					"url": "http://piedpipersf.com/",
					"street": "San Francisco, CA",
					"stars": 4.2,
					"review": 174,
					"phone": null,
					"description": "Posh American eatery in historic hotel",
					"price": "$$",
					"type": "Bar & grill",
					"extensions": null
				}
			],
			"mapUrl": "https://google.com/search?q=Pied+Piper&uule=w+CAIQICINVW5pdGVkIFN0YXRlcw&gl=US/search?
			gl=US&q=Pied+Piper
&npsic=0&rflfq=1&rldoc=1&rlha=
0&rllag=36538514%2C-108071483%2C1282005&tbm=
lcl&sa=X&ved=2ahUKEwjk0MGd9-biAhUSmxQKHSmmDwMQtgN6BAgJEAQ"
		},
		...
		{
			"position": 10,
			"title": "The Disturbing True Story of the Pied Piper
 of Hamelin | Ancient Origins",
			"url": "https://www.ancient-origins.net/myths-legends/
disturbing-true-story-pied-piper-hamelin-001969",
			"destination": "https://www.ancient-origins.net/.../
disturbing-true-story-pied-piper-hamelin-001969",
			"description": "Aug 14, 2014 - Many are familia
r with the story of the Pied Piper of Hamelin.
			Few realise however, that the story is based on
 real events, which evolved over ..."
		}
	],
	"related_searches": [
		{
			"title": "pied piper nursery rhyme",
			"url": "https://google.com/search?q=Pied
+Piper&uule=
w+CAIQICINVW5pdGVkIFN0YXRlcw&
			gl=US/search?gl=
US&q=pied+piper+nursery+
rhyme&sa=
X&ved=2ahUKEwjk0MGd9-biAhUSmxQKHSmmDwMQ1QIoAHoECAsQAQ"
		},
		{
			"title": "pied piper poem",
			"url": "https://google.com/search?q=Pied+Piper&uule=w+CAIQICINVW5pdGVkIFN0YXRlcw&gl=
			US/search?
gl=US&q=pied+piper+
poem&sa=
X&ved=2ahUKEwjk0MGd9-biAhUSmxQKHSmmDwMQ1QIoAXoECAsQAg"
		},
		{
			"title": "pied piper music",
			"url": "https://google.com/search?q=Pied+Piper&uule=
w+CAIQICINVW5pdGVkIFN0YXRlcw&gl=
			US/search?
gl=US&q=pied+piper
+music&
sa=X&ved=2ahUKEwjk0MGd9-biAhUSmxQKHSmmDwMQ1QIoB3oECAsQCA"
		}
	],
	"number_of_results": 16200000
}

Step6: Conclusion

Here in this tutorial we have explained how to consume Zenserp API with PHP to scrape the Google search result data. You can also use more advance options by going through the documentation in your Zenserp API request.


You may also like: