Question: 1 / 155

What should a consultant ensure when ingesting daily store transaction data from Amazon S3 that gets deleted after seven days?

Ensure that the deletion of old files is enabled.

Ensure the refresh mode is set to 'Upsert'.

When ingesting daily store transaction data that is stored on Amazon S3 and subject to deletion after a set period, it is important for a consultant to consider how to best update the data they are working with. Setting the refresh mode to 'Upsert' indicates that the process will update existing records while also adding new records for the data being ingested. This is particularly useful in environments where data is frequently changing or being deleted, as it allows for incremental updates without the need to reprocess already ingested data. Choosing this approach ensures that the most current transactional data is reflected in the system while maintaining efficiency in processing. As old files get deleted after seven days, having an 'Upsert' operation allows for seamless integration of new data as it comes in without losing important historical context that may still exist in the dataset. In contrast, other options do not align as effectively with the scenario presented. Ensuring that the deletion of old files is enabled may not be relevant since the deletion of files is already part of the data management strategy in S3. Using a wildcard in the filename could complicate the ingestion process without addressing the need for regular updates. Setting the refresh mode to 'Full Refresh' would require reloading all the data every time, which

Ensure the filename contains a wildcard to accommodate the timestamp.

Ensure the refresh mode is set to 'Full Refresh'.

Next

Report this question