Update: I feel really sad to inform that google has discontinued the Google Search Console API Explorer. Due to this, this method doesnât work anymore.
The Google Search Console is of great help to Webmasters with several useful features like sitemap submission, search analytics, links to your site, crawl errors report. With the GSC, the SEO specialists have a powerful tool at their fingertips to direct the course of their SEO requests.
However, the Google Search Console also has several aspects that are somewhat frustrating. For eg; the âLinks to your siteâ feature only shows a maximum of 1000 domains. Now, Imagine how many domains link to a site like Moz or Buzzfeed! Do you think the Google Search Console serves them well?
The sitemap tool shows how many URLs were indexed but does not distinguish between the non-indexed and indexed URLs. And well there are other similar annoyances. (Update: Google is testing a beta feature called Index Coverage report that will show indexed pages count and reasons why some pages could not be indexed. Read more on Google Webmaster Blog.)
Another point that annoys Google Search Console (GSC) users is the Crawl Errors report. The Search Console shows us the crawl errors, the HTTP status code of the error (404, 503 etc), and also the source of the error. The bothersome part is you have to click on each URL to view where the error originates from.
Another problem is the 1,000 errors limit. Youâll have to fix the existing ones to view newer ones.
The Google API explorer is a savvy tool to communicate with numerous Google APIs. But we need to concern ourselves only with the Search Console API.
Youâll need Full Access to use the API and Search Console account, so make sure you have them beforehand.
Once logged in, youâll notice the Search Console API offers several Services, 13 services to be exact. But our focus here is the one labeled webmasters.urlcrawlerrorssamples.list. So, go ahead and click on it.
The next screen will look like the image below. Beyond this point itâs a simple 3 step process:
The fields you notice on the next screen read as follows:
Use the fields editor to select required fields. The available ones are:
I prefer selecting all the data and filtering it down later. You are free to choose your path.
Oh! Do turn on the Authorize requests using OAuth 2.0 button on top right (marked with an orange rectangle in the image)
or
Once all the fields are set, just click the Authorize and execute button.
Youâd be greeted with a popup before the report runs, like in the image below:
Check both options and hit the Authorize and execute button.
If everything goes well, you should see a status 200 OK message with a JSON output similar to the image below
Copy the output after the first curly bracket { till the last } curly bracket in the end.
Now, head over to a JSON to CSV converter. I prefer https://konklone.io/json/ for the simple interface, a preview window, and the ability to download the CSV.
Note: I tried another converter too but the report wasnât 100% accurate. Test your converter before you finalize on a report.
Paste the JSON output and convert it into a CSV.
This is pretty much self-explanatory! Download your CSV and start fixing those 404 errors.
Word to the wise: Google Search Console may not list sources or linking domains for all errors. So don’t worry if some rows returned are empty.
404 errors may not seem critical but fixing them helps saves crawl budget, improves user experience, and also helps retain link value.
You may use the same process to find and fix other errors. The API includes 9 types of errors in the Google Search Console that you can fix. You donât have to wreck your brain to search them out; we have listed all error types below.
I hope you find this useful! Now head to your laptop and start fixing those 404 errors.
Introduction CueBlocks Technologies is an eCommerce design, development, and marketing agency with extensive experience working on Magento, Shopify, Woo Commerce, …
Continue reading “CueBlocks Delivers Exceptional eCcommerce Design & ROI: GoodFirms”
Shopify 1.0 vs 2.0 E-stores around the globe prefer Shopify due to its many features. 29th June 2021 brought one …
Continue reading “The New Shopify version 2.0 – An Update To Look Into”
Google Discover is a curated content feed inside the Google app. The app has more than 800 million monthly active …
Continue reading “How to Optimize & Track Google Discover in Google Analytics”
Colours of Rangoli, Flowers, Lord Ganesha and Women of CB lighting diyas Diyas for enlightenment, spectrum of colour emitting harmony …
Continue reading “Sparking The Joy Within: Diwali 2022 at CueBlocks”
I am sharing this post to announce a happy news with all of you â We have a new WORKSPACE! …
At CueBlocks, we love fresh beginnings! Thatâs why weâre proud to display our work on Top Design Firms, a new …
Continue reading “CueBlocks Proud to be Featured on Top Design Firms!”
Thanks for this post, works excellent! Helped a lot and prevented me from writing my own script ;).
Thanks very much for this excellent post Rajiv! Just tried it and its very handy! I was just wondering when you think you’ll find a method to extract more than 1,000 URL’s as that would be ideal for larger websites! Thanks again and all the best! Alex
Thank you, Alex!
I am currently working on it. It appears a lot of us run into 404 errors in bulk. Will be posting the solution for 1000 plus URLs soon, hopefully.
Cheers!
Rajiv
Hi Rajiv,
Great walk-through of the API.
Using the Search Console interface I was able to download 1,325/5,929 errors while the API only returned 1,000. Any idea what I’ve missed?
Thanks again – JR
Hi John,
Thank you! Unfortunately, the API explorer seems to be limited to 1000 results only. I am working on a way around this. I’ll make it another blog post once it is complete.
Rajiv
Thanks Rajiv.
This worked a treat, first time.
Glad to know you found it useful, Ewan
Hi,
Its working!
Thanks for showing the easy way to download it.
Great stuff.
Regards,
Malisa
Thank you, Malisa!