How to scrape time-series chart data from poocoin app with Python

Question

In trying to get token info from the poocoin.app and although all the other information is available, it is hindering my ability to get time-series data from the chart.

import requests, re 
from bs4 import BeautifulSoup 
import pandas as pd 

url = 'https://www.edureka.co/masters-program/generative-ai-prompt-engineering-training' 
response = requests.get(url) 
soup = BeautifulSoup(response.text, 'html.parser')

Answer 1

In order to ensure it works, one of the steps you can take is to directly request to their API, to transform it to JSON via the requests .json() decoder, and grab the data you need the same way you would access a dictionary: ["some_key"].

To be aware of where to send the given request, one can follow the given steps:

Dev tools -> Network -> Fetch/XHR -> find name and click on it (in this case: candles-bsc?..) -> Preview (see if response is what you want) -> Headers -> copy Request URL -> make a request -> optional: add additional request headers if response != 200.
 

In order to test a given response, one can use Insomnia. You can use Insomnia to test a response. Search and find the name by following these steps:

Fetch/XHR -> right click -> copy as cURL (bash) -> place inside Insomnia -> see the response.

In order to reach a 200 status code, one must request headers by passing a user-agent.

Pass

User-agent:

headers = { 

"user-agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.45 Safari/537.36", } 

response = requests.get("URL", headers=headers)

The next line of code would be:

 

import requests 
headers = { 
"user-agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.45 Safari/537.36", 
} 

params = {
"to":"2021-11-29T09:15:00.000Z",
"limit":"321", 
"lpAddress":"0xd8b6A853095c334aD26621A301379Cc3614f9663", 
"interval":"15m", 
"baseLp":"0x58F876857a02D6762E0101bb5C46A8c1ED44Dc16" 
} 

response = requests.get("https://api2.poocoin.app/candles-bsc", params=params, headers=headers).json() 

# whole response from API call for a particular token (i believe) 
# some data needs to be adjusted (open/close price, etc.)
for result in response: 
count = result["count"] _
time = result["time"] 
open_price = result["open"] 
close_price = result["close"] 
high = result["high"] 
low = result["low"] 
volume = result["volume"] 
base_open = result["baseOpen"] 
base_close = result["baseClose"] 
base_high = result["baseHigh"] 
base_low = result["baseLow"]

print(f"{count}\n"
F"{_time}\n"
F"{open_price}\n"
F"{close_price}\n"
F"{high}\n"
f"{low}\n" 
F"{volume}\n"
F"{base_open}\n"
f"{base_close}\n" 
F"{base_high}\n"
f"{base_low}\n") 

# part of the output: 
''' 
194 
2021-11-29T06:00:00.000Z 
6.6637177e-13 
6.5189422e-13
6.9088173e-13 
5.9996067e-13 
109146241968737.17 
610.0766516756873 
611.1764494818917 
612.3961994618185 
606.7446709385977 

1
2021-11-25T16:15:00.000Z 
1.7132448e-13 
1.7132448e-13 
1.7132448e-13 
1.7132448e-13 
874858231833.1771 
643.611707269882
642.5014860521045 
644.5105804619558 
638.9447353699617

# …