How to load custom datasets in Keras?

10 months ago

Noah Thompson

2 minutes

In Keras, loading a custom dataset typically involves the following steps:

Prepare the dataset: First, make sure to have your custom dataset ready, including data files, label files, and so on.
Create a data generator: In Keras, it is common to use the ImageDataGenerator class to create a data generator, which generates batches of data from a dataset during model training.
Load data: Use the flow_from_directory() method of the data generator to load the dataset, which loads data from a specified directory and allows for setting parameters such as batch size and image size.
Data preprocessing: Perform necessary operations such as normalization and scaling on the data being loaded.

Here is an example code demonstrating how to load a custom dataset in Keras.

from keras.preprocessing.image import ImageDataGenerator

# 创建数据生成器
data_gen = ImageDataGenerator(rescale=1./255)  # 对数据进行归一化

# 加载数据集
train_generator = data_gen.flow_from_directory(
    'path/to/train_data',
    target_size=(224, 224),  # 图像大小
    batch_size=32,
    class_mode='categorical'  # 分类标签
)

# 创建模型
model = some_model()

# 编译模型
model.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['accuracy'])

# 训练模型
model.fit(train_generator, epochs=10)

In the example above, we start by creating an ImageDataGenerator object and configuring data normalization. We then load the training dataset using the flow_from_directory() method, setting image size, batch size, and classification labels. Next, we create a model and compile it. Finally, we train the model using the fit() method, passing in the data generator as the source of training data.