Incremental data copy using azure datafactory

Question

I've created a pipeline to copy data from one blob storage to a different blob storage.  I would like to use incremental copy if it's possible, but haven't found how to specify it. the reason is i would like to run this on a schedule and only copy any new data since last run.

Answer 1

• If your blob is well named with a timestamp, you could follow this doc to copy partitioned data. You could use copy data tool to setup the pipeline. You could select tumbling window and then in file path filed input {year}/{month}/{day}/fileName and choose the right pattern. It will help you construct the parameters.

• In case If your blob is not well named with timestamp, you could use get metadata activity to check the last modified time. 

Event trigger is just one way to control when the pipeline should run. You could also use tumbling window trigger or schedule trigger in your scenarios.