2_fullyconnected.ipynb (ud730)

Migrate your TensorFlow 1 code to TensorFlow 2  |  TensorFlow Coreを参考にexamples/2_fullyconnected.ipynb at master · tensorflow/examples · GitHubについて 2.1.0-rc1 の時点では一応以下で動作させられた:

try:
  # Use the %tensorflow_version magic if in colab.
  %tensorflow_version 2.x
except Exception:
  pass

import tensorflow.compat.v1 as tf
tf.disable_v2_behavior()

まぁ、いつ動作しなくなるか分かったものではないが。(なので考古学的な興味以上にこの書き方に固執しても意味はないんだろうけど)
以下のような感じで・・・いいのかな。Test accuracy: 89.0% へと改善はした。

batch_size = 128
hidden_nodes = 1024

graph = tf.Graph()
with graph.as_default():

  # Input data. For the training data, we use a placeholder that will be fed
  # at run time with a training minibatch.
  tf_train_dataset = tf.placeholder(tf.float32,
                                    shape=(batch_size, image_size * image_size))
  tf_train_labels = tf.placeholder(tf.float32, shape=(batch_size, num_labels))
  tf_valid_dataset = tf.constant(valid_dataset)
  tf_test_dataset = tf.constant(test_dataset)

  # Variables.
  weights_1 = tf.Variable(
    tf.truncated_normal([image_size * image_size, hidden_nodes]))
  biases_1 = tf.Variable(tf.zeros([hidden_nodes]))

  weights_2 = tf.Variable(
    tf.truncated_normal([hidden_nodes, num_labels]))
  biases_2 = tf.Variable(tf.zeros([num_labels]))

  # Training computation.
  inter_output = tf.matmul(tf_train_dataset, weights_1) + biases_1
  inter_output = tf.nn.relu(inter_output)
  logits = tf.matmul(inter_output, weights_2) + biases_2

  loss = tf.reduce_mean(
    tf.nn.softmax_cross_entropy_with_logits(labels=tf_train_labels, logits=logits))

  # Optimizer.
  optimizer = tf.train.GradientDescentOptimizer(0.5).minimize(loss)

  # Predictions for the training, validation, and test data.
  train_prediction = tf.nn.softmax(logits)
  valid_inter_output = tf.matmul(tf_valid_dataset, weights_1) + biases_1
  valid_inter_output = tf.nn.relu(valid_inter_output)
  valid_prediction = tf.nn.softmax(
    tf.matmul(valid_inter_output, weights_2) + biases_2)
  test_inter_output = tf.matmul(tf_test_dataset, weights_1) + biases_1
  test_inter_output = tf.nn.relu(test_inter_output)
  test_prediction = tf.nn.softmax(tf.matmul(test_inter_output, weights_2) + biases_2)

たぶんコレ、TensorFlow 2.x で書いたら以下で済むんだろうな・・・

model = tf.keras.Sequential([
    tf.keras.layers.Dense(1024, bias_initializer='zeros', input_shape=(784,), activation=tf.nn.relu),
    tf.keras.layers.Dense(10, bias_initializer='zeros', activation=tf.nn.softmax)
])

model.compile(optimizer='sgd', 
              loss='sparse_categorical_crossentropy',
              metrics=['accuracy'])

...

test_prediction = model.predict(tf_test_dataset)

計算グラフをベタベタと書く TenforFlow 1.x と、Keras で済ませる TensorFlow 2.x とどっちが良いか分からないけど、真ん中をとるような感じで？PyTorch でベタベタ実装を書くのも分かりやすいかも？

from torch import nn, optim
import torch.nn.functional as F

class Classifier(nn.Module):
    def __init__(self):
        super().__init__()
        self.fc1 = nn.Linear(784, 1024)
        self.fc2 = nn.Linear(1024, 10)

    def forward(self, x):
        x = F.relu(self.fc1(x))
        x = F.log_softmax(self.fc2(x), dim=1)

        return x

model = Classifier()
criterion = nn.NLLLoss()
optimizer = optim.SGD(model.parameters(), lr=0.5)