how can i extact all the links from a website using python and save it in a csv file ?

0 votes
Jun 15 in Python by Shubham
• 150 points
353 views

1 answer to this question.

0 votes

Hi, @Shubham,

Web scraping is the technique to extract data from a website.

The module BeautifulSoup is designed for web scraping. The BeautifulSoup module can handle HTML and XML. You can refer to the code below:

from BeautifulSoup import BeautifulSoup
import urllib2
import re

html_page = urllib2.urlopen("https://abc.com")
soup = BeautifulSoup(html_page)
for link in soup.findAll('a', attrs={'href': re.compile("^http://")}):
    print link.get('href')

It downloads the raw HTML code with the line

html_page = urllib2.urlopen("https://abc.com")

A BeautifulSoup object is created and we use this object to find all links:

soup = BeautifulSoup(html_page)
for link in soup.findAll('a', attrs={'href': re.compile("^http://")}):
    print link.get('href')

I hope this will be helpful for you.

answered Jun 16 by Gitika
• 41,360 points

Related Questions In Python

0 votes
2 answers
+2 votes
3 answers

How can I play an audio file in the background using Python?

down voteacceptedFor windows: you could use  winsound.SND_ASYNC to play them ...READ MORE

answered Apr 3, 2018 in Python by charlie_brown
• 7,780 points
6,407 views
0 votes
1 answer

How can I find out the index of an element from row and column in Python?

You probably want to use np.ravel_multi_index: [code] import numpy ...READ MORE

answered Apr 16, 2018 in Python by charlie_brown
• 7,780 points
346 views
0 votes
1 answer

How can I convert a list of dictionaries from a CSV into a JSON object in Python?

You could try using the AST module. ...READ MORE

answered Apr 17, 2018 in Python by anonymous
1,965 views
0 votes
1 answer

How can I read numbers in Python from a custom file?

Hi, good question. Let us first assume that ...READ MORE

answered Feb 6, 2019 in Python by Nymeria
• 3,520 points
234 views
0 votes
1 answer

How can I lookup hostname using the IP address with a timeout in Python?

Good question. I actually was stuck with ...READ MORE

answered Feb 6, 2019 in Python by Nymeria
• 3,520 points
482 views
0 votes
1 answer