想将算法进⼀步开发吗?⼿把⼿教你搭建基于CNN模型的
FlaskWeb应⽤
对于机器学习和⼈⼯智能研究⼈员⽽⾔,好多⼈都只是构建好模型后就没有进⼀步处理了,停留在⼀个⽐较粗糙的模型上⾯,没有将其变成⼀个产品,其实好多创业型⼈⼯智能公司都是设计好模型后,将其转化成产品,之后再推向市场。每⼀个深度学习研究者⼼中或多或少都想成为⼀名创业者,但不知道超哪个⽅向发展。那么,本⽂将从最简单的⽹页应⽤开始,⼀步⼀步带领你使⽤TensorFlow创建⼀个卷积神经⽹络(CNN)模型后,使⽤Flash RESTful API将模型变成⼀个⽹页应⽤产品。
本⽂使⽤TensorFlow NN模块构建CNN模型,并在CIFAR-10数据集上进⾏训练和测试。为了使模型可以远程访问,使⽤Python创建Flask web应⽤来接收上传的图像,并使⽤HTTP返回其分类标签。
1.安装Python、TensorFlow、PyCharm和Flask API
孔⼦云:⼯欲善其事,⽐先利其器。程序员亦如此,在进⾏开发前,需要准备好开发环境并基本掌握开发⼯具。Python是第⼀个需要安装的⼯具,因为整个环境都依赖于它。如果你已经配置好了开发环境,那么可以跳过第⼀步。
1.1 安装Anaconda/Python
虽然可以安装传统的⽅法安装Python,但是建议使⽤类似于Anaconda这样完整的包,因为⾥⾯已经安装了⼀些好的库可供你直接调⽤。本⽂中使⽤的是Anaconda3版本,对于Windows系统,。
为了确保Anaconda3是否安装成功,在CMD命令⾏中输⼊(where Python),如果结果类似于下图,则表明安装成功。
1.2 安装TensorFlow
在上⼀步Anaconda3安装完毕后,接下来是安装TensorFlow(TF)。本⽂使⽤的是Windows系统下CPU版本的TF,。
TF的安装步骤如下:
1)使⽤下⾯代码创建conda环境:
C:> conda create -n tensorflow pip python=3.5
这为TF安装创建了⼀个空的⽂件以保持虚拟环境(virtual environment, venv),vevn的位置在Anaconda3安装的⽬录⽂件下:(Anaconda3envstensorflow)。
2)使⽤下⾏命令激活venv
C:> activate tensorflow
上⾏命令告诉我们venv和所需安装的所有库,输⼊这⾏命令后,命令⾏将变(tensorflow)C:>,接下来是安装TensorFlow包。
3)在激活venv后,Window下CPU版本的TensorFlow可以直接使⽤pip安装:
代码
为了测试TF是否安装成功,可以导⼊TensorFlow,若结果与下图⼀样,则表明安装成功。但是在导⼊TF之前,请确保venv被激活。flask下载
1.3安装PyCharm Python IDE
相较于在CMD命令⾏中输⼊代码,本⽂更倾向于使⽤Python IDE。本⽂选择PyCharm,。此外,下载安装完毕后,需要设置Python 编译器,操作如下图所⽰,选择之前安装的作为IDE的编译器。
1.4 安装Flask
最后的⼀个⼯具是安装Flask RESTful API,安装命令如下:
C:> pip install Flask-API
在全部安装完毕后,接下来要开始新建⼯程了。
2.下载并预处理CIFAR-10数据集
,该数据集包含60,000张图像,并将其划分为训练集和测试集。其中训练集有5个⽂件夹,分别命名为data_batch_1、
data_,data_batch_5,每个⽂件夹中包含10,000张,每张都是32x32x3的RGB图像。测试集只有⼀个⽂件夹,命名为a,包含10,000张图像。训练集和测试集中包含的图像类别为飞机(airplane)、⼿机(automobile)、鸟(bird)、猫(cat)、⿅(deer)、狗(dog)、青蛙(frog)、马(horse)、船(ship)以及truck(卡车)。
由于数据集中的每个⽂件都是⼆进制⽂件,因此应该对其进⾏解码以检索实际的图像数据。基于此,创建unpickle_patch函数来执⾏如下操作:
def unpickle_patch(file):
"""
Decoding the binary file.
:param file:File to decode it data.
:return:Dictionary of the file holding details including input data and output labels.
"""
patch_bin_file = open(file, 'rb')#Reading the binary file.
patch_dict = pickle.load(patch_bin_file, encoding='bytes')#Loading the details of the binary file into a dictionary.
return patch_dict#Returning the dictionary.
该⽅法接收⼆进制⽂件并返回⼀个包含有关此⽂件详细信息的字典。该字典除了标签之外还包含⽂件中所有的10,000个样本的数据。 为了解码整个训练集,创建get_dataset_images函数。该函数接收数据集路径并仅对训练数据起作⽤。因此,它会过滤⼀些⽂件并只返回以data_batch_开头的⽂件。测试数据在模型训练好后再进⾏处理。
对于每⼀个训练⽂件夹,使⽤unpickle_patch函数解码,该函数输出⼀个字典。之后使⽤get_dataset_images函数获取图像数据以及其对应的类别标签。图像数据是从“data”键中检索,类别标签从“labels”键中检索。
由于数据图形是以⼀维向量的形式保存,此外TensorFlow接收的是三维形式,因此应对其进⾏变换处理。基于
此,get_dataset_images函数接收的参数为图像的⽂件路径、⾏/列数以及图像的通道数。
def get_dataset_images(dataset_path, im_dim=32, num_channels=3):
"""
This function accepts the dataset path, reads the data, and returns it after being reshaped to match the requierments of the CNN.
:param dataset_path:Path of the CIFAR10 dataset binary files.
:param im_dim:Number of rows and columns in each image. The image is expected to be rectangular.
:param num_channels:Number of color channels in the image.
:return:Returns the input data after being reshaped and output labels.
"""
num_files = 5#Number of training binary files in the CIFAR10 dataset.
images_per_file = 10000#Number of samples withing each binary file.
files_names = os.listdir(patches_dir)#Listing the binary files in the dataset path.
"""
Creating an empty array to hold the entire training data after being reshaped.
The dataset has 5 binary files holding the data. Each binary file has 10,000 samples. Total number of samples in the dataset is 5*10,000=50,000.
Each sample has a total of 3,072 pixels. These pixels are reshaped to form a RGB image of shape 32x32x3.
Finally, the entire dataset has 50,000 samples and each sample of shape 32x32x3 (50,000x32x32x3).
"""
dataset_array = s(shape=(num_files * images_per_file, im_dim, im_dim, num_channels))
#Creating an empty array to hold the labels of each input sample. Its size is 50,000 to hold the label of each sample in the dataset.
dataset_labels = s(shape=(num_files * images_per_file), dtype=numpy.uint8)
index = 0#Index variable to count number of training binary files being processed.
for file_name in files_names:
"""
Because the CIFAR10 directory does not only contain the desired training files and has some other files, it is required to filter the required files.
Training files start by 'data_batch_' which is used to test whether the file is for training or not.
"""
if file_name[0:len(file_name) - 1] == "data_batch_":
print("Working on : ", file_name)
"""
Appending the path of the binary files to the name of the current file.
Then the complete path of the binary file is used to decoded the file and return the actual pixels values.
"""
data_dict = unpickle_patch(dataset_path+file_name)
"""
Returning the data using its key 'data' in the dictionary.
Character b is used before the key to tell it is binary string.
"""
images_data = data_dict[b"data"]
#Reshaping all samples in the current binary file to be of 32x32x3 shape.
images_data_reshaped = shape(images_data, newshape=(len(images_data), im_dim, im_dim, num_channels))
#Appending the data of the current file after being reshaped.
dataset_array[index * images_per_file:(index + 1) * images_per_file, :, :, :] = images_data_reshaped
#Appening the labels of the current file.
dataset_labels[index * images_per_file:(index + 1) * images_per_file] = data_dict[b"labels"]
index = index + 1#Incrementing the counter of the processed training files by 1 to accept new file.
return dataset_array, dataset_labels#Returning the training input data and output labels.
处理好训练集后,下⼀步构建CNN模型并进⾏训练。
3.使⽤TensorFlow构建CNN模型
使⽤creat_CNN函数创建CNN模型,该函数创建卷积层(conv)、ReLU激活函数、最⼤池化(max pooling)、dropout以及全连接层(full connection,FC),最后⼀层全连接层输出结果。每⼀层的输出
都是下⼀层的输⼊,这就要求相邻两层之间的特征图尺⼨⼤⼩要⼀致。此外,对于每个conv、ReLU以及最⼤池化层等,都有⼀些超参数需要设置,⽐如卷积或池化时候设置的步长等。
由于卷积层将输⼊数据与设置的卷积核进⾏卷积运算,因此create_CNN函数将输⼊数据作为输⼊参数,这些数据是由
get_dataset_images函数返回的数据。create_conv_layer函数接收输⼊数据、过滤器⼤⼩和过滤器数量,并返回输⼊数据与过滤器集合进⾏卷积的结果。这组滤波器的⼤⼩根据输⼊图像的深度⽽设置。 create_conv_layer的定义如下:
对于dropout层,接收⼀个保持神经元的概率参数,它表明会有多少神经元在dropout层被丢弃。 dropout层是使⽤
dropout_flatten_layer函数实现,如下所⽰:
ef dropout_flatten_layer(previous_layer, keep_prop):
"""
Applying the dropout layer.
:param previous_layer: Result of the previous layer to the dropout layer.
:param keep_prop: Probability of keeping neurons.
:return: flattened array.
"""
dropout = dropout(x=previous_layer, keep_prob=keep_prop)
num_features = _shape()[1:].num_elements()
layer = shape(previous_layer, shape=(-1, num_features))#Flattening the results.
return layer
由于最后⼀个FC层的输出神经元数应等于数据集类别数量,因此数据集类的数量将⽤作create_CNN函数的另⼀个输⼊参数。全连接层是使⽤fc_layer函数创建,该函数接收dropout层输出结果,输出结果中的特征数量以及来⾃此FC层的输出神经元的数量。根据输⼊和输出的数量,创建⼀个权重张量,然后乘以flattened_layer得到FC层的返回结果。
版权声明:本站内容均来自互联网,仅供演示用,请勿用于商业和其他非法用途。如果侵犯了您的权益请与我们联系QQ:729038198,我们将在24小时内删除。
发表评论