how to import numpy and pandas inside aws lambda function

Question

i need to process csv file using numpy and pandas , i could do it in local machine its running as a expected manner then i am trying to do the same thing in aws lambda function with the same code i was used in my local machine.i am trying to import numpy inside my aws lambda function but error was thrown as Unable to import module 'aws_lambda_demo/lambda_function': No module named 'numpy'. then i create a zip file with aws lambda function along with numpy package after that also facing same issue Unable to import module 'aws_lambda_demo/lambda_function': No module named 'numpy'. Notes: i kept my csv files in s3 storage .any help will be appreciated . thanks in advance.

Answer 1

The problem is that your local numpy and pandas are compiled for the local machine's architecture. Since AWS Lambda uses custom Linux, they are probably not compatible.

So if you want to use them, you have two choices:

• Compile dependencies on EC2 instance which uses the same Amazon Linux version as AWS Lambda and create a deployment package.

• Use one of the precompiled packages from here

Comments

Hey @Sunil, follow these instructions:

upload your deployment package to the following repo:

git clone https://github.com/pbegle/aws-lambda-py3.6-pandas-numpy.git

Add your lambda_function.py to the zip file by running:

zip -ur lambda.zip lambda_function.py

Upload to S3 and source to lambda.

Answer 2

I had good luck using pre-made pandas and numpy layers that allowed me to skip a lot of the docker and pip solutions I saw elsewhere. This is what worked for me: https://medium.com/@melissa_89553/how-to-import-python-packages-in-aws-lambda-pandas-scipy-numpy-bb2c98c974e9