How to scrape AJAX based website data?

0 votes

I am unable to get data from https://www.propertyfinder.ae/en/search?bf=1&bt=2&c=2&fu=1&l=39&ob=pa&rp=y&t=1 

It will take only 25 first full records and prices for all records. After 25 records it miss the names and URL only prices will extract. Need your expert opinion on this matter. I am using UiPath Version 2019.10.0. There are 3 pages record it only take all details which I selected for first 25 records only. After that it will pick only prices. Here is the screenshot of csv file.

Also share some ajax based web scraping. And what it I want to save all image files from scraping?

Thanks.

Nov 12 in RPA by Adnan
• 140 points
71 views
Hey @Adnan, could you share the script that you have used that is fetching these records?

1 answer to this question.

+1 vote

Hi Adnan,

Before you scrape any website/webpage, you must check if the administrator of that site has blocked scrapping Administrators block it because crawling affects the performance of the website

. In case you continue to scrape such a website, your ISP may block (blacklist)you.  I checked the website you mentioned and  they have blocked scrapping. Hence trying any method might just end up your ISP to block you(blacklist) or worse legal action for scrapping.

read this link for further reference: https://www.scrapehero.com/how-to-prevent-getting-blacklisted-while-scraping/

https://www.propertyfinder.ae/robots.txt as per this link, ajax scrapping is not allowed. Please talk to the respective service provider to seek permission to scrape/crawl their content

Once you seek their permission, you can use once of the methods mentioned in this link. 

https://www.codementor.io/codementorteam/how-to-scrape-an-ajax-website-using-python-qw8fuitvi

Regards,

RRR

PS:

If my post has answered your question, please mark it as answer. Feel free to post your thoughts. !!

answered 6 days ago by Raveendiran
• 860 points

Related Questions In RPA

+3 votes
1 answer

How to extract data from a website and save it to excel using UiPath?

Hi Piyush, if you want to extract ...READ MORE

answered Mar 8 in RPA by Anvi
• 13,960 points
6,351 views
0 votes
1 answer

How to scrape data using Get Text from UI Automation tools in UiPath?

Hi Roshni, scraping any text from a web page ...READ MORE

answered Mar 20 in RPA by Pratibha
• 3,680 points
278 views
+2 votes
1 answer

How to extract data from website and save it to excel using Blue Prism?

Hey, you can use following steps to ...READ MORE

answered Aug 19 in RPA by Abha
• 27,180 points
532 views
0 votes
1 answer

How to read data from a excel, based on that data, check in the corresponding checkbox in browser appln? Uipath

Hi, to achieve this, you can perform ...READ MORE

answered Oct 10 in RPA by Abha
• 27,180 points
82 views
0 votes
1 answer

How to assign a data in a variable in to excel file?

System -> File -> Workbook -> Append ...READ MORE

answered May 22, 2018 in RPA by wrecker
• 3,110 points
112 views
0 votes
1 answer

UiPath - Citirx Automation

best way to do that is to ...READ MORE

answered Apr 13, 2018 in RPA by wrecker
• 3,110 points
258 views
0 votes
1 answer

UiPath - Web Scraping

Multiple solutions are listed below: Check your Selector Use ...READ MORE

answered Apr 13, 2018 in RPA by wrecker
• 3,110 points
303 views
0 votes
1 answer

Robot Manager in UiPath

the robot.exe file is located in: C:\Users\USER_ACCOUNT\AppData\Local\UiPath\app-17.1.6523\Ui-Robot.exe Hope it ...READ MORE

answered Apr 17, 2018 in RPA by wrecker
• 3,110 points
478 views
0 votes
1 answer

Automation in Uipath

you can use the Database package for connecting  ...READ MORE

answered Apr 17, 2018 in RPA by wrecker
• 3,110 points
395 views
+2 votes
1 answer