Python is not working when I try to read an excel file from S3 inside of an AI flow dag.
It is quite strange because it works when I read it using pd.read excel(s3 excel path) from outside airflow.
How I acted:
Create an AWS account in Airflow (this works well as I can list my s3 bucket)
In my Docker environment, where I run Airflow, instal pandas and s3fs.
Pd.read excel(s3 excel path) should be used to attempt to read the file.
I've tried it outside of Airflow, and it functions as stated.
Furthermore, even after waiting 20 minutes, nothing happens; the dag just keeps running indefinitely (at the stage where it is meant to read the file).
(I am attempting to read the file ...READ MORE
Jan 3, 2023
in AWS
by
Tejashwini
• 3,820 points
•
1,554 views
