优化器

• 优化器演算法

  • 在反向传播的过程中，优化器的用途在於最小化损失函数的loss值，期望找到全局的最佳解。

  • 有可能会遇到昨天我们所说的鞍点，这时搭配好的学习率，是有机会避开鞍点的。

  • 没有一个优化器是最好也最厉害的，所以选择优化器的方式，小弟认为是多尝试，等到经验够多，可能就可以知道哪些的资料集种类适合那些优化器。

  • 不同优化器的收敛轨迹动态图
    
    图片来源：https://lonepatient.top/2018/09/25/Cyclical_Learning_Rate

比较各种优化器事项

1. 模型选定：用轻量型的预训练模型densenet201做训练，训练时间也较快。

2. 资料集：因中文手写字图片过多，决定节省时间，选定之前介绍过的验证码辨识的图片共12000张训练集，测试集3000张，验证集1000张。

   • dataset建立

   import os
    from PIL import Image
    import torch
    from torch.utils.data import Dataset
    import pandas as pd
   
   
    alphabet = 'ABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789'
   
   
    def img_loader(img_path):
        img = Image.open(img_path)
        return img.convert("RGB")
    # data_path = r"C:/Users/Frank/PycharmProjects/practice/captcha_recognition/picture/"
    # ans_path = r'C:\Users\Frank\PycharmProjects\practice\captcha_recognition\answer\answer2.csv'
    def make_dataset(data_path,ans_path,alphabet):
        img_names = os.listdir(data_path)
        img_names.sort(key=lambda x: int(x.split(".")[0]))
        df_ans = pd.read_csv(ans_path)
        ans_list = list(df_ans["code"].values)
        samples = []
   
        for ans, img_name in zip(ans_list, img_names):
            if len(str(ans)) == 5  :#num_char:
                img_path = os.path.join(data_path, img_name)
                target = []
                for char in str(ans):
                    vec = [0] * 36 # num_class
                    vec[alphabet.find(char)] = 1
                    target += vec
                samples.append((img_path, target))
            else:
                print(img_name)
        return samples
   
   
    class CaptchaData(Dataset):
        def __init__(self, data_path,ans_path,
                     transform=None, target_transform=None, alphabet=alphabet):
            super(Dataset, self).__init__()
            self.data_path = data_path
            self.ans_path = ans_path
            # self.num_class = num_class
            # self.num_char = num_char
            self.transform = transform
            self.target_transform = target_transform
            self.alphabet = alphabet
            self.samples = make_dataset(self.data_path,self.ans_path,self.alphabet
                                        )
   
        def __len__(self):
            return len(self.samples)
   
        def __getitem__(self, index):
            img_path, target = self.samples[index]
            img = img_loader(img_path)
            if self.transform is not None:
                img = self.transform(img)
            if self.target_transform is not None:
                target = self.target_transform(target)
            return img, torch.Tensor(target)