Defining multiple schedules in same data pipeline

0 votes

Can you define multiple schedules for different activities in the same pipeline? If yes, how? I couldn't find a proper solution for this anywhere.

Nov 16, 2018 in AWS by datageek
• 3,090 points
514 views

1 answer to this question.

0 votes

Yes, simply define multiple schedule objects in your pipeline definition file and associate the desired schedule to the correct activity via its schedule field. This allows you to define a pipeline in which, for example, log files are stored in Amazon S3 each hour to drive generation of an aggregate report one time per day.

answered Nov 16, 2018 by Archana
• 5,640 points

Related Questions In AWS

+1 vote
1 answer
0 votes
1 answer

Trying to upload data stream in Amazon S3

Try adding the ContentType & ACL in ...READ MORE

answered Jun 28, 2018 in AWS by Cloud gunner
• 4,650 points
1,542 views
0 votes
1 answer

In Amazon Data Pipeline, how to make sure only once instance of a pipeline is running at any time?

On the CopyTablesActivity, you could set a lateAfterTimeout attribute ...READ MORE

answered Sep 19, 2018 in AWS by Priyaj
• 58,140 points
1,320 views
0 votes
1 answer
0 votes
1 answer
0 votes
1 answer

On what resources does AWS Data Pipeline activities run?

AWS Data Pipeline activities are run on ...READ MORE

answered Nov 16, 2018 in AWS by Archana
• 5,640 points
243 views