How should this be accomplished?
Use the AWS CLI to schedule a manual snapshot of the DB cluster. Restore the snapshot to a new DB cluster using the AWS CLI.
Create a dump file from the DB cluster. Load the dump file into a new DB cluster.
Schedule a job to create a clone of the DB cluster at the end of the overnight batch process.
Set up a new daily AWS DMS task that will use cloning and change data capture (CDC) on the DB cluster to copy the data to a new DB cluster. Set up a time for the AWS DMS stream to stop when the new cluster is current.
Explanations:
While creating a manual snapshot and restoring it to a new DB cluster might work, it is time-consuming for a 20 TB database. The snapshot and restore process could take a significant amount of time, which doesn’t meet the requirement for the shortest possible time after the batch job completes.
Creating a dump file and loading it into a new DB cluster is also a slow process, especially for large databases like 20 TB. This method is inefficient and does not provide the required speed for making the most up-to-date data available quickly.
Aurora supports DB cluster cloning, which is a fast way to create a copy of the database. Aurora clones provide a nearly instantaneous, space-efficient copy of the database, which is ideal for making the data available quickly for testing after the batch job completes.
Setting up a daily AWS DMS task with change data capture (CDC) is designed for ongoing replication and not for immediate availability of data right after a batch job. It introduces unnecessary complexity and latency for the specific requirement of quickly making data available for testing after the batch job.
I surmise that the answer is:
Schedule a job to create a clone of the DB cluster at the end of the overnight batch process.
I assess that the answer is:
Schedule a job to create a clone of the DB cluster at the end of the overnight batch process.