How to pipe (/stream) big file (video,pdf,any…) from aws s3 to node.js express res?

0 votes

Currently i have tried many ways like

const S3FS = require('s3fs');

s3fsImpl = new S3FS(bucketName, {
  accessKeyId,
  secretAccessKey,
})

var range = 'undefined' !== typeof req.headers.range ? req.headers.range : 'bytes=0-';
var positions = range.replace(/bytes=/, "").split("-");
var start = parseInt(positions[0], 10);
var total = details.ContentLength;
var end = positions[1] ? parseInt(positions[1], 10) : total - 1;
var chunksize = (end - start) + 1;

res.setHeader("Content-Type", contentType);
res.status(206);
res.setHeader("Content-Range", "bytes " + start + "-" + end + "/" + total);
res.setHeader("Accept-Ranges", "bytes");
res.setHeader("Content-Length", chunksize);

s3fsImpl.createReadStream(key,{ start: start, end: end }).pipe(res);

( same with const AWS = require('aws-sdk') ) It just hangs up and no video occured. Currently i just do pipe without chunks, but it take video too long to occur, and seems i need to implement chunks stream from s3. So how can i pipe video, pddf, or other big files from the aws s3?

Sep 24, 2018 in AWS by bug_seeker
• 14,970 points
512 views

1 answer to this question.

Your answer

Your name to display (optional):
Privacy: Your email address will only be used for sending these notifications.
0 votes

As far as I remember the limit for Amazon chubk size is somewhere around 5g's. And anyone sane would not implement this as a 5g chunk in node (not because it wouldn't make sense, but because it would be hard and wouldn't make sense). So unless you are over the size the issue doesn't seem to be in the code you posted

answered Sep 24, 2018 by Priyaj
• 56,120 points

Related Questions In AWS

0 votes
1 answer

How to skip headers when reading a CSV file in S3 and creating a table in AWS Athena?

This is a known deficiency. The best ...READ MORE

answered Sep 4, 2018 in AWS by Archana
• 3,770 points
478 views
0 votes
1 answer

How to download the latest file in a S3 bucket using AWS CLI?

You can use the below command $ aws ...READ MORE

answered Sep 6, 2018 in AWS by Archana
• 3,770 points
1,530 views
0 votes
1 answer

How to transfer file from ec2 to s3?

I'm using s3cmd to store nightly exported database backup ...READ MORE

answered Jan 14 in AWS by neha
69 views
0 votes
1 answer

AWS S3 uploading hidden files by default

versioning is enabled in your bucket. docs.aws.amazon.com/AmazonS3/latest/user-guide/….... the ...READ MORE

answered Oct 4, 2018 in AWS by Priyaj
• 56,120 points
84 views
0 votes
1 answer

How to decrypt the encrypted S3 file using aws-encryption-cli --decrypt

Use command : aws s3 presign s3://mybucket/abc_count.png you get ...READ MORE

answered Oct 22, 2018 in AWS by Priyaj
• 56,120 points
181 views
0 votes
1 answer

Import my AWS credentials using python script

Using AWS Cli  Configure your IAM user then ...READ MORE

answered Nov 16, 2018 in AWS by Jino
• 5,520 points
98 views
0 votes
2 answers
0 votes
1 answer

How to copy .csv file from Amazon S3 bucket?

Boto3 is the library to use for ...READ MORE

answered Jul 6, 2018 in AWS by Priyaj
• 56,120 points
260 views
0 votes
1 answer

© 2018 Brain4ce Education Solutions Pvt. Ltd. All rights Reserved.
"PMP®","PMI®", "PMI-ACP®" and "PMBOK®" are registered marks of the Project Management Institute, Inc. MongoDB®, Mongo and the leaf logo are the registered trademarks of MongoDB, Inc.