kreas入门实例(猫狗大战)

下载数据集

从kaggle中找到dogs-vs-cats比赛，下载数据集压缩包并解压得到一个大数据集train文件夹。

这里我们不用这么多张图片，我们自己创建一个小型数据集进行学习，提升自我能力。

先给数据集路径做个YAML文件:

data.yml文件的代码

# 原始数据集解压目录
original_dir: "train"
# 保存较小的数据集目录
new_base_dir: "cats_vs_dogs_small"

createdataset.py的代码

import os, shutil, pathlib
import yaml

# 读取YAML文件
with open('data.yml', 'r', encoding='utf-8') as file:
    data = yaml.safe_load(file)

# 原始数据集解压目录
original_dir = pathlib.Path(data['original_dir'])
# 保存较小的数据集目录
new_base_dir = pathlib.Path(data['new_base_dir'])

# 将索引从start_index到end_index的猫/狗图像复制到子目录new_base_dir/{subset_name}/cat(或者dog)
# subset_name可以是"train","validation","test"
def make_subset(subset_name, start_index, end_index):
    for category in ("cat", "dog"):
        dir = new_base_dir / subset_name / category
        os.makedirs(dir)
        fnames = [f"{category}.{i}.jpg"
                  for i in range(start_index, end_index)]
        for fname in fnames:
            shutil.copyfile(src=original_dir / fname, dst=dir / fname)

make_subset("train", start_index=0, end_index=1000)
make_subset("validation", start_index=1000, end_index=1500)
make_subset("test", start_index=1500, end_index=2000)

print("数据集制作完毕")

这样下来，子数据集目录如下：

cat_vs_dogs_small/
...train/
......cat/   # 1000张
......dog/   # 1000张
...validation/
......cat/   # 500张
......dog/   # 500张
...test/
......cat/   # 500张
......dog/   # 500张

数据预处理

init.py的代码

import yaml
import pathlib
from keras.utils import image_dataset_from_directory
from tensorflow import keras
from keras import layers

# 读取YAML文件
with open('data.yml', 'r', encoding='utf-8') as file:
    data = yaml.safe_load(file)

# 读取较小的数据集目录
new_base_dir = pathlib.Path(data['new_base_dir'])

train_dataset = image_dataset_from_directory(
    new_base_dir / "train",
    image_size=(180, 180),
    batch_size=32)

validation_dataset = image_dataset_from_directory(
    new_base_dir / "validation",
    image_size=(180, 180),
    batch_size=32)

data_augmentation = keras.Sequential(
    [
        layers.RandomFlip("horizontal"), # 将水平翻转运用于50%的图片
        layers.RandomRotation(0.1),      # 随机旋转10%
        layers.RandomZoom(0.2)           # 放大缩小图像20%
    ]
)

查看增强的数据

show.py的代码

import matplotlib.pyplot as plt

from init import train_dataset, data_augmentation

plt.figure(figsize=(10, 10))

for images,_ in train_dataset.take(1):
    for i in range(9):
        augmented_images = data_augmentation(images)
        ax = plt.subplot(3, 3, i + 1)
        plt.imshow(augmented_images[0].numpy().astype("uint8"))
        plt.axis("off")

plt.show()

随机显示几张增强后的训练图片

显示批量图片的第一张，在9次迭代中，每一张图片都是对同一张图片的不同增强

训练

train.py的代码

import matplotlib.pyplot as plt

from keras import layers
from tensorflow import keras
from init import train_dataset, validation_dataset, data_augmentation

inputs = keras.Input(shape=(180, 180, 3))
x = data_augmentation(inputs) # 数据增强
x = layers.Rescaling(1./255)(x) # 将输入除以255使其缩放至[0,1]

x = layers.Conv2D(filters=32, kernel_size=3, activation="relu")(x)
x = layers.MaxPool2D(pool_size=2)(x)

x = layers.Conv2D(filters=64, kernel_size=3, activation="relu")(x)
x = layers.MaxPool2D(pool_size=2)(x)

x = layers.Conv2D(filters=128, kernel_size=3, activation="relu")(x)
x = layers.MaxPool2D(pool_size=2)(x)

x = layers.Conv2D(filters=256, kernel_size=3, activation="relu")(x)
x = layers.MaxPool2D(pool_size=2)(x)

x = layers.Conv2D(filters=256, kernel_size=3, activation="relu")(x)
x = layers.Flatten()(x)

x = layers.Dropout(0.5)(x)
outputs = layers.Dense(1, activation="sigmoid")(x)
model = keras.Model(inputs=inputs, outputs=outputs)

model.summary()
# 模型的最后一层是单一的sigmoid单元，所以使用二元交叉熵作为损失函数

model.compile(loss="binary_crossentropy", optimizer="rmsprop", metrics=["accuracy"])

callbacks = [
    keras.callbacks.ModelCheckpoint(
        filepath="convent_from_scratch.keras",
        save_best_only=True,
        monitor="val_loss")
]

history = model.fit(train_dataset, epochs=80, validation_data=validation_dataset, callbacks=callbacks)

model.save('model.h5') # 将模型保存到本地

# 训练准确度(已知图片)和验证准确度(未知图片)曲线
acc = history.history['accuracy']
val_acc = history.history['val_accuracy']
loss = history.history['loss']
val_loss = history.history['val_loss']

# 精度曲线
epochs = range(len(acc))
plt.plot(epochs, acc, 'r', label='Training accuracy')
plt.plot(epochs, val_acc, 'b', label='validation accuracy')
plt.title('Training and validation accuracy')
plt.legend(loc=0)
plt.savefig('accuracy_plot.png')  # 保存为本地文件

# 创建一个新的Figure对象
plt.figure()

# 损失曲线
epochs = range(len(loss))
plt.plot(epochs, loss, 'r', label='Training loss')
plt.plot(epochs, val_loss, 'b', label='validation loss')
plt.title('Training and validation loss')
plt.legend(loc=0)
plt.savefig('loss_plot.png')  # 保存为本地文件

为了进一步降低过拟合，我们还会在模型密集连接分类器之前添加一个Dropout层

我们根据验证损失这个指标来保存最优的模型convent_from_scratch.keras

训练结果

其实我还有没有经过数据增强的结果曲线，对比后可以看出数据增强后效果相当不错

测试

如果是在Colab上训练的，记得把这个convent_from_scratch.keras训练模型下载下来，别搞丢了

test.py的代码

import yaml
import pathlib
from keras.utils import image_dataset_from_directory
from keras.models import load_model

# 读取YAML文件
with open('data.yml', 'r', encoding='utf-8') as file:
    data = yaml.safe_load(file)

# 读取较小的数据集目录
new_base_dir = pathlib.Path(data['new_base_dir'])

test_dataset = image_dataset_from_directory(
    new_base_dir / "test",
    image_size=(180, 180),
    batch_size=32)

# 评估模型
test_model = load_model('convent_from_scratch.keras')
loss, accuracy = test_model.evaluate(test_dataset)

print("Test Loss:", loss)
print("Test Accuracy:", accuracy)

测试结果

Found 1000 files belonging to 2 classes.
32/32 [==============================] - 6s 141ms/step - loss: 0.5211 - accuracy: 0.8490
Test Loss: 0.5210882425308228
Test Accuracy: 0.8489999771118164

测试精度达到84.9%

---

参考：python深度学习