Where can I obtain a list of User Agents for SEO bots

0 votes
i am planing on implementing a simplistic filter on how much of my site unregistered users can acces. need to compile a list of names of user agents I will allow, for this, I need a list of the names of the bots, starting with Googlebot
Feb 28 in Digital Marketing by Kichu
• 19,040 points
116 views

1 answer to this question.

0 votes

add crawler user agents and that would help in the major part and avoid viewing through search engine cache 

try this list http://www.useragentstring.com/pages/useragentstring.php?typ=Crawler

answered Mar 1 by narikkadan
• 37,660 points

Related Questions In Digital Marketing

0 votes
0 answers

SEO: A list of photographers, one photo plus caption: <ul> or <dl>?

<ul> <li title='John Doe'>     <div class='item' id='123'>         <span class='left'>John Doe</span>         <span ...READ MORE

Mar 8 in Digital Marketing by Kichu
• 19,040 points
52 views
0 votes
0 answers

How can I view organic SEO traffic keywords in the new Google Analytics 4?

I created a Google Analytics account with ...READ MORE

Mar 12 in Digital Marketing by Kichu
• 19,040 points
95 views
0 votes
0 answers

Design and SEO for a single page dynamic website with AJAX

I designed a website in which the ...READ MORE

Mar 12 in Digital Marketing by Kichu
• 19,040 points
145 views
0 votes
0 answers

WP All in one SEO, does it have a database?

I am using wordpress for my website ...READ MORE

Mar 2 in Digital Marketing by Kichu
• 19,040 points
59 views
0 votes
1 answer

SEO With Angular and ASP.NET MVC (not SPA)

the best thing you can do is ...READ MORE

answered Feb 17 in Others by narikkadan
• 37,660 points
71 views
0 votes
1 answer

Crawling after login in Python

You missed a few login data forms, ...READ MORE

answered Sep 7, 2018 in Python by Priyaj
• 58,100 points
1,136 views
0 votes
1 answer

Crawling after login in Python

You missed a few login data forms, ...READ MORE

answered Sep 14, 2018 in Python by Priyaj
• 58,100 points
396 views
0 votes
1 answer

How to run Nutch in Hadoop installed in pseudo-distributed mode

Make sure you have built Nutch from ...READ MORE

answered Jan 24, 2019 in Big Data Hadoop by Frankie
• 9,830 points
519 views
0 votes
1 answer
0 votes
1 answer

SEO - List of links - Farmlinking?

Link farming is a really bad SEO ...READ MORE

answered Mar 11 in Digital Marketing by narikkadan
• 37,660 points
102 views
webinar REGISTER FOR FREE WEBINAR X
REGISTER NOW
webinar_success Thank you for registering Join Edureka Meetup community for 100+ Free Webinars each month JOIN MEETUP GROUP