Which additional data preparation steps should the company take before uploading the files to Amazon S3?

By:
In: MLS-C01
Tagged:
With: 2 Comments
A social media company wants to develop a machine learning (ML) model to detect inappropriate or offensive content in images.The company has collected a large dataset of labeled images and plans to use the built-in Amazon SageMaker image classification algorithm to train the model.The company also intends to use SageMaker pipe mode to speed up the training.The company splits the dataset into training, validation, and testing datasets.The company stores the training and validation images in folders that are named Training and Validation, respectively.The folders contain subfolders that correspond to the names of the dataset classes.The company resizes the images to the same size and generates two input manifest files named training.lst and validation.lst, for the training dataset and the validation dataset, respectively.Finally, the company creates two separate Amazon S3 buckets for uploads of the training dataset and the validation dataset.

Which additional data preparation steps should the company take before uploading the files to Amazon S3?

Generate two Apache Parquet files, training.parquet and validation.parquet, by reading the images into a Pandas data frame and storing the data frame as a Parquet file. Upload the Parquet files to the training S3 bucket.

Compress the training and validation directories by using the Snappy compression library. Upload the manifest and compressed files to the training S3 bucket.

Compress the training and validation directories by using the gzip compression library. Upload the manifest and compressed files to the training S3 bucket.

Generate two RecordIO files, training.rec and validation.rec, from the manifest files by using the im2rec Apache MXNet utility tool. Upload the RecordIO files to the training S3 bucket.

Explanations:

While generating Parquet files can improve data loading performance for certain types of data, the SageMaker image classification algorithm does not directly support Parquet files as input format. It primarily requires image data in formats such as RecordIO or the specified manifest files.

Compressing the directories using the Snappy compression library is not a supported format for SageMaker. SageMaker requires images to be in specific formats for training, and using unsupported compression would likely lead to errors during training or data loading.

Similar to option B, using gzip compression for the training and validation directories is not compatible with the SageMaker image classification algorithm. The algorithm expects images in specific formats and not as compressed directories.

Generating RecordIO files from the manifest files using the im2rec utility is the appropriate preparation step. RecordIO is a preferred input format for SageMaker image classification, as it efficiently packages image data for training, allowing faster data loading and improved performance during model training.

2026-03-29

2 Comments

  1. Christopher
    Author

    I organize that the answer is:
    Generate two RecordIO files, training.rec and validation.rec, from the manifest files by using the im2rec Apache MXNet utility tool. Upload the RecordIO files to the training S3 bucket.

  2. Andrea
    Author

    I plan that the answer is:
    Generate two RecordIO files, training.rec and validation.rec, from the manifest files by using the im2rec Apache MXNet utility tool. Upload the RecordIO files to the training S3 bucket.

Leave a Reply

Your email address will not be published. Required fields are marked *

four + twenty =