XOR Problem Tensorflow NN - Ipynb

{"cells":[{"cell_type":"code","execution_count":1,"metadata":
{"id":"c7DYJJ52a__o","executionInfo":
{"status":"ok","timestamp":1669121162970,"user_tz":0,"elapsed":3772,"user":
{"displayName":"Samuel Jularbal","userId":"00348146265806993811"}}},"outputs":
[],"source":["import tensorflow as tf\n","import matplotlib.pyplot as plt"]},
{"cell_type":"markdown","metadata":{"id":"ndjRB767a__p"},"source":["# XOR Problem\
n","\n","## What is the XOR operation?\n","\n","* XOR is a boolean logic operation
that compares two input bits and generates one output bit. \n","* IF the bits are
the same (or both are true) the result is 0 (or false).\n","* IF the bits are
different the result is 1 (or true).\n","\n","## What is the XOR problem?\n","\
n","* The XOR can be seen as a classification problem. \n","* You receive two
inputs and you want to predict true or false, depending if the inputs are different
or the same\n","* However, the inputs are not linearly separable. That means that
there is no way to use a straight line to separate the possible options an make an
automated prediction correctly.\n","* In the next figure we can se that it is not
possible to separate the red dots and the green dots with a straight line. "]},
{"cell_type":"code","execution_count":2,"metadata":{"colab":{"base_uri":"https://
localhost:8080/","height":265},"id":"IcwfNdLGa__r","executionInfo":
{"displayName":"Samuel
Jularbal","userId":"00348146265806993811"}},"outputId":"77c5f589-7667-4fd2-f696-
001e1889b067"},"outputs":[{"output_type":"display_data","data":{"text/plain":
["<Figure size 432x288 with 1
Axes>"],"image/png":"iVBORw0KGgoAAAANSUhEUgAAAXQAAAD4CAYAAAD8Zh1EAAAABHNCSVQICAgIfA
hkiAAAAAlwSFlzAAALEgAACxIB0t1+/
AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9y
Zy+WH4yJAAAQvklEQVR4nO3df4xdZZ3H8fe3nU6hUmXXjou0xdZQIrW7qXjF7pKsrD+WgthqcN3WsAsbYsX
d6ib+SNi4sgb9A1fFLKGudl3DqpFS2UgmWiX+qKG6VDsIIi2BlAp0KtgBapuW/qDy3T/
u1QzTae8ZeubeztP3K5n0nuc8Pc/
36Z359Mz5cU9kJpKkiW9StwuQJNXDQJekQhjoklQIA12SCmGgS1Ihero18IwZM3LOnDndGl6SJqS77777yc
zsG21d1wJ9zpw5DAwMdGt4SZqQIuLRo63zkIskFcJAl6RCGOiSVAgDXZIKMeEC/
aGnHmLDoxvYc3BPt0uRpLHZtQs2bICHHx6XzbcN9Ij4ckTsjIj7j7I+IuLGiNgaEfdFxHn1lwlD+4ZY9KVF
LPzCQi695VLO+MwZXP/j68djKEmqVyZ87GNw5pnwtrfBggVw4YXw29/WOkyVPfSbgcXHWH8xMK/
1tQL4z+Mv60iXrb2Mnz/+c/Yf3s+eg3vYf3g/
n7zzk3zroW+Nx3CSVJ81a+Bzn4MDB2D37uafd90Fl19e6zBtAz0z7wSePkaXpcBXsmkjcHpEvLyuAgEe2/0
Ym369iWefe/Z57fue3cdn/++zdQ4lSfX7zGdg377ntx06BN//
Pjz1VG3D1HEMfSawfdjyYKvtCBGxIiIGImJgaGio8gBP73+aKZOmjLpu5zM7x1CqJHXBk0+O3t7TU+thl46
eFM3M1ZnZyMxGX9+od66O6twZ5xIRR7T3Tu7lrfPeWmeJklS/
xYub4T3StGlQ40eg1BHoO4DZw5ZntdpqM7VnKjcuvpFpU6YRNIN96uSpvPTUl/KRv/
hInUNJUv2uvRZOPx16e5vLEc0w//znYfLk2oap47Nc+oGVEbEGeD2wOzMfr2G7z3PFwis4+4/
P5oaNNzC4e5CLzr6ID7z+A8yYNqPuoSSpXjNnwv33N0+M/
vCHMHcufPjD8LrX1TpMtHumaETcAlwIzAB+A/
wbMAUgM78QzWMhN9G8EuYZ4B8ys+2nbjUajfTDuSRpbCLi7sxsjLau7R56Zi5vsz6Bf3qBtUmSajLh7hSVJ
I3OQJekQhjoklQIA12SCmGgS1IhDHRJKoSBLkmFMNAlqRAGuiQVwkCXpEIY6JJUCANdkgphoEtSIQx0SSqE
gS5JhTDQJakQBrokFcJAl6RCGOiSVAgDXZIKYaBLUiEMdEkqhIEuSYUw0CWpEAa6JBXCQJekQhjoklQIA12
SCmGgS1IhDHRJKoSBLkmFqBToEbE4Ih6MiK0Rcc0o68+KiPURcU9E3BcRl9RfqiTpWNoGekRMBlYBFwPzge
URMX9Et38F1mbma4BlwOfrLlSSdGxV9tDPB7Zm5rbMPASsAZaO6JPAi1uvXwL8ur4SJUlVVAn0mcD2YcuDr
bbhPg5cHhGDwDrg/
aNtKCJWRMRARAwMDQ29gHIlSUdT10nR5cDNmTkLuAT4akQcse3MXJ2Zjcxs9PX11TS0JAmqBfoOYPaw5Vmt
tuGuAtYCZOZdwCnAjDoKlCRVUyXQNwHzImJuRPTSPOnZP6LPY8CbACLiXJqB7jEVSeqgtoGemYeBlcAdwAM
0r2bZHBHXRcSSVrcPAe+JiF8AtwBXZmaOV9GSpCP1VOmUmetonuwc3nbtsNdbgAvqLU2SNBbeKSpJhTDQJa
kQBrokFcJAl6RCGOiSVAgDXZIKYaBLUiEMdEkqhIEuSYUw0CWpEAa6JBXCQJekQhjoklQIA12SCmGgS1IhD
HRJKoSBLkmFMNAlqRAGuiQVwkCXpEIY6JJUCANdkgphoEtSIQx0SSqEgS5JhTDQJakQBrokFcJAl6RCGOiS
VAgDXZIKUSnQI2JxRDwYEVsj4pqj9HlXRGyJiM0R8fV6y5QktdPTrkNETAZWAW8BBoFNEdGfmVuG9ZkH/
AtwQWbuioiXjVfBkqTRVdlDPx/
YmpnbMvMQsAZYOqLPe4BVmbkLIDN31lumJKmdKoE+E9g+bHmw1TbcOcA5EfGTiNgYEYtH21BErIiIgYgYGB
oaemEVS5JGVddJ0R5gHnAhsBz4r4g4fWSnzFydmY3MbPT19dU0tCQJqgX6DmD2sOVZrbbhBoH+zHw2M38FP
EQz4CVJHVIl0DcB8yJibkT0AsuA/
hF9bqe5d05EzKB5CGZbjXVKktpoG+iZeRhYCdwBPACszczNEXFdRCxpdbsDeCoitgDrgY9k5lPjVbQk6UiR
mV0ZuNFo5MDAQFfGlqSJKiLuzszGaOu8U1SSCmGgS1IhDHRJKoSBLkmFMNAlqRAGuiQVwkCXpEIY6JJUCAN
dkgphoEtSIQx0SSqEgS5JhTDQJakQBrokFcJAl6RCGOiSVAgDXZIKYaBLUiEMdEkqhIEuSYUw0CWpEAa6JB
XCQJekQhjoklQIA12SCmGgS1IhDHRJKoSBLkmFMNAlqRAGuiQVwkCXpEJUCvSIWBwRD0bE1oi45hj9LouIj
IhGfSVKkqpoG+gRMRlYBVwMzAeWR8T8UfpNB/
4Z+GndRUqS2quyh34+sDUzt2XmIWANsHSUfp8APgUcqLE+SVJFVQJ9JrB92PJgq+0PIuI8YHZmfvtYG4qIF
RExEBEDQ0NDYy5WknR0x31SNCImATcAH2rXNzNXZ2YjMxt9fX3HO7QkaZgqgb4DmD1seVar7femAwuAH0XE
I8AioN8To5LUWVUCfRMwLyLmRkQvsAzo//
3KzNydmTMyc05mzgE2Aksyc2BcKpYkjaptoGfmYWAlcAfwALA2MzdHxHURsWS8C5QkVdNTpVNmrgPWjWi79
ih9Lzz+siRJY+WdopJUCANdkgphoEtSIQx0SSqEgS5JhTDQJakQBrokFcJAl6RCGOiSVAgDXZIKYaBLUiEM
dEkqhIEuSYUw0CWpEAa6JBXCQJekQhjoklQIA12SCmGgS1IhDHRJKoSBLkmFMNAlqRAGuiQVwkCXpEIY6JJ
UCANdkgphoEtSIQx0SSqEgS5JhTDQJakQlQI9IhZHxIMRsTUirhll/
QcjYktE3BcRP4iIV9RfqiTpWNoGekRMBlYBFwPzgeURMX9Et3uARmb+GXAb8O91FypJOrYqe+jnA1szc1tm
HgLWAEuHd8jM9Zn5TGtxIzCr3jIlSe1UCfSZwPZhy4OttqO5CvjOaCsiYkVEDETEwNDQUPUqJUlt1XpSNCI
uBxrAp0dbn5mrM7ORmY2+vr46h5akk15PhT47gNnDlme12p4nIt4MfBR4Q2YerKc8SVJVVfbQNwHzImJuRP
QCy4D+4R0i4jXAF4Elmbmz/
jIlSe20DfTMPAysBO4AHgDWZubmiLguIpa0un0aOA34RkTcGxH9R9mcJGmcVDnkQmauA9aNaLt22Os311yX
JGmMvFNUkgphoEtSIQx0SSqEgS5JhTDQJakQBrokFcJAl6RCGOiSVAgDXZIKYaBLUiEMdEkqhIEuSYUw0CW
pEAa6JBXCQJekQhjoklQIA12SCmGgS1IhDHRJKoSBLkmFMNAlqRAGuiQVwkCXpEIY6JJUCANdkgphoEtSIQ
x0SSqEgS5JhTDQJakQBrokFaKn2wWMyeAg3Hxz8883vhHe8Q6YMqXbVUlSWwcPH+S2Lbex4bENzDl9Dlcuv
JIzTjuj1jEiM9t3ilgM/AcwGfhSZl4/Yv1U4CvAa4GngL/NzEeOtc1Go5EDAwPVK12/
Hi69FH73Ozh4EE47Dc45BzZsgGnTqm9Hkjpsz8E9LPrSIrbv2c7eQ3s5pecUeib18L2/+x6LZi0a07Yi4u7
MbIy2ru0hl4iYDKwCLgbmA8sjYv6IblcBuzLzbOBzwKfGVGE7zz0Hy5fDM880wxxg71544AG46aZah5Kkul
3/4+vZtmsbew/tBeDA4QPsPbSXd//vu6myU11VlWPo5wNbM3NbZh4C1gBLR/RZCvxP6/
VtwJsiImqrcvNm2LfvyPb9+
+FrX6ttGEkaD7fefysHf3fwiPYn9j7Bo7sfrW2cKoE+E9g+bHmw1TZqn8w8DOwGXjpyQxGxIiIGImJgaGio
epVTpzb30kdz6qnVtyNJXdDb0ztqe2YydfLU2sbp6FUumbk6MxuZ2ejr66v+F+fNg7POgpE7/
S96EVx9db1FSlLN3vva9zJtyvPP9U2KSSz4kwW8fPrLaxunSqDvAGYPW57Vahu1T0T0AC+heXK0HhHwzW9C
Xx9Mn948CXrqqfD2t8MVV9Q2jCSNh5Xnr+Qtr3wL06ZMY9qUaUzvnc6Zp53J2neurXWcKpctbgLmRcRcmsG
9DHj3iD79wBXAXcA7gR9mnUf6AV71Kti+Hb77XXjiCbjgAnj1q2sdQpLGQ8+kHm5fdjv3PnEvP9vxM2ZOn8
lFZ19Ez6R6rxxvu7XMPBwRK4E7aF62+OXM3BwR1wEDmdkP/Dfw1YjYCjxNM/
Tr19sLS5aMy6YlabwtPGMhC89YOG7br/TfQ2auA9aNaLt22OsDwN/
UW5okaSy89V+SCmGgS1IhDHRJKoSBLkmFqPThXOMycMQQ8ELveZ0BPFljOROBcz45OOeTw/
HM+RWZOeqdmV0L9OMREQNH+7SxUjnnk4NzPjmM15w95CJJhTDQJakQEzXQV3e7gC5wzicH53xyGJc5T8hj6
JKkI03UPXRJ0ggGuiQV4oQO9IhYHBEPRsTWiLhmlPVTI+LW1vqfRsSczldZrwpz/
mBEbImI+yLiBxHxim7UWad2cx7W77KIyIiY8Je4VZlzRLyr9V5vjoivd7rGulX43j4rItZHxD2t7+9LulFn
XSLiyxGxMyLuP8r6iIgbW/8e90XEecc9aGaekF80P6r3YeCVQC/wC2D+iD7/
CHyh9XoZcGu36+7AnP8KmNZ6/b6TYc6tftOBO4GNQKPbdXfgfZ4H3AP8UWv5Zd2uuwNzXg28r/
V6PvBIt+s+zjn/JXAecP9R1l8CfAcIYBHw0+Md80TeQ+/
+w6k7r+2cM3N9Zj7TWtxI8wlSE1mV9xngE8CngAOdLG6cVJnze4BVmbkLIDN3drjGulWZcwIvbr1+CfDrDt
ZXu8y8k+bzIY5mKfCVbNoInB4Rx/U8uhM50Gt7OPUEUmXOw11F83/4iaztnFu/
is7OzG93srBxVOV9Pgc4JyJ+EhEbI2Jxx6obH1Xm/
HHg8ogYpPn8hfd3prSuGevPe1v1Pv9IHRMRlwMN4A3drmU8RcQk4Abgyi6X0mk9NA+7XEjzt7A7I+JPM/
O3Xa1qfC0Hbs7Mz0bEn9N8CtqCzHyu24VNFCfyHnr3H07deVXmTES8GfgosCQzD3aotvHSbs7TgQXAjyLiE
ZrHGvsn+InRKu/zINCfmc9m5q+Ah2gG/ERVZc5XAWsBMvMu4BSaH2JVqko/72NxIgf6Hx5OHRG9NE969o/
o8/
uHU8N4PZy6s9rOOSJeA3yRZphP9OOq0GbOmbk7M2dk5pzMnEPzvMGSzBzoTrm1qPK9fTvNvXMiYgbNQzDbO
llkzarM+THgTQARcS7NQB/qaJWd1Q/8fetql0XA7sx8/Li22O0zwW3OEl9Cc8/kYeCjrbbraP5AQ/MN/
wawFfgZ8Mpu19yBOX8f+A1wb+urv9s1j/ecR/
T9ERP8KpeK73PQPNS0BfglsKzbNXdgzvOBn9C8AuZe4K+7XfNxzvcW4HHgWZq/cV0FXA1cPew9XtX69/
hlHd/X3vovSYU4kQ+5SJLGwECXpEIY6JJUCANdkgphoEtSIQx0SSqEgS5Jhfh/
5ZPA+04p4mYAAAAASUVORK5CYII=\n"},"metadata":
{"needs_background":"light"}}],"source":["plt.scatter([0, 0, 1, 1], [0,1,1,0],
c=['red', 'green', 'red', 'green'])\n","plt.show()"]},
{"cell_type":"markdown","metadata":{"id":"0O0AH1haa__s"},"source":["# Using a
Multilayer Perceptron (MLP) architecture\n","\n","* A MLP with an input and an
output layer is not capable of solving the problem since it the result are linear
solutions.\n","* However, if we add a hidden layer to the MLP, the network will be
capable of achieving non-linear separation and correctly predict the results.\n","\
n","# Training a MLP with a single hidden layer to solve the XOR problem\n"]},
{"cell_type":"code","execution_count":3,"metadata":
{"id":"3P39Z6yUa__s","executionInfo":
[],"source":["# First we define the input data (X) and the output data (Y) that we
are going to use for training\n","X = [[0,0], [0,1], [1,0], [1,1]]\n","Y = [[0],
[1],[1],[0]]"]},{"cell_type":"code","execution_count":4,"metadata":
{"id":"dpyYXcvNa__s","executionInfo":
[],"source":["# Here we define the architecture of our network. \n","# The number
of neurons in the input layer is defined by the number of input values in our
dataset, in this case 2. \n","# For this architecture we are going to use 2
neurons in our hidden layer.\n","# The number of neurons in the output layer is
defined by the number of expected values, in this case we expect a single value 1
or 0.\n","model = tf.keras.models.Sequential() # we create a Neural Network model
that is Sequential, which means each layer we add will be connected to the previous
one. \n","model.add(tf.keras.Input(shape=[None, 2])) # Input layer\
n","model.add(tf.keras.layers.Dense(2, activation='sigmoid')) # Hidden Layer\
n","model.add(tf.keras.layers.Dense(1, activation='sigmoid')) # Output Layer"]},
{"id":"NOl36W5ma__t","executionInfo":
[],"source":["#
We define which optimizer and which error loss are we using to update the weights
of the network.\n","sgd = tf.keras.optimizers.SGD(learning_rate=0.8,momentum=0.3)\
n","mse = tf.keras.losses.MeanSquaredError()"]},
{"id":"ig1rJv9ka__t","executionInfo":
[],"source":["# we compile our model \n","model.compile(optimizer=sgd, loss=mse,
metrics=['accuracy'])"]},{"cell_type":"code","execution_count":7,"metadata":
{"colab":{"base_uri":"https://localhost:8080/"},"id":"8PXOb-
Bya__t","executionInfo":
Jularbal","userId":"00348146265806993811"}},"outputId":"520be800-c2a3-4a23-9a31-
74979e0322ef"},"outputs":[{"output_type":"stream","name":"stdout","text":["Epoch
1/1000\n"]},{"output_type":"stream","name":"stderr","text":
["WARNING:tensorflow:Model was constructed with shape (None, None, 2) for input
KerasTensor(type_spec=TensorSpec(shape=(None, None, 2), dtype=tf.float32,
name='input_1'), name='input_1', description=\"created by layer 'input_1'\"), but
it was called on an input with incompatible shape (None, 2).\
n","WARNING:tensorflow:Model was constructed with shape (None, None, 2) for input
it was called on an input with incompatible shape (None, 2).\n"]},
{"output_type":"stream","name":"stdout","text":["1/1
[==============================] - 1s 1s/step - loss: 0.2644 - accuracy: 0.5000\
n","Epoch 2/1000\n","1/1 [==============================] - 0s 18ms/step - loss:
0.2609 - accuracy: 0.5000\n","Epoch 3/1000\n","1/1 [==============================]
- 0s 20ms/step - loss: 0.2574 - accuracy: 0.5000\n","Epoch 4/1000\n","1/1
[==============================] - 0s 14ms/step - loss: 0.2548 - accuracy: 0.5000\
0.2502 - accuracy: 0.7500\n","Epoch 12/1000\n","1/1
n","Epoch 77/1000\n","1/1 [==============================] - 0s 34ms/step
- loss: 0.2500 - accuracy: 0.5000\n","Epoch 78/1000\n","1/1
[==============================] - 0s 10ms/step - loss:
[==============================] - 0s 10ms/step
[==============================] - 0s 11ms/step
[==============================] - 0s
12ms/step - loss: 0.2489 - accuracy: 0.5000\n","Epoch 442/1000\n","1/1
[==============================]
[==============================]
[==============================]
[==============================]
[==============================]
[==============================]
n"]},{"output_type":"execute_result","data":{"text/plain":
["<keras.callbacks.History at 0x7f952abfeed0>"]},"metadata":
{},"execution_count":7}],"source":["# After compiling the model we are able to
start the training using the 'fit' function. \n","# We need to make use of our
training dataset (X and Y).\n","# We also specify the number of epochs (or
iterations) the network is going to train for.\n","model.fit(X, Y, epochs=1000)"]},
{"cell_type":"code","execution_count":8,"metadata":{"colab":{"base_uri":"https://
localhost:8080/"},"id":"NBC2rm5ca__u","executionInfo":
Jularbal","userId":"00348146265806993811"}},"outputId":"4640a5c1-e84c-4ab8-a0ef-
6d3d8f5d59fb"},"outputs":[{"output_type":"stream","name":"stderr","text":
["WARNING:tensorflow:Model was constructed with shape (None, None, 2) for input
it was called on an input with incompatible shape (None, 2).\n"]},
{"output_type":"stream","name":"stdout","text":["1/1
[==============================] - 0s 137ms/step\n"]},
{"output_type":"execute_result","data":{"text/plain":["array([[0.18238835],\n","
[0.85430956],\n"," [0.8319951 ],\n"," [0.15787144]],
dtype=float32)"]},"metadata":{},"execution_count":8}],"source":["# After our model
finished training we can test it and see how well it predicts \
n","model.predict(X)"]},{"cell_type":"code","execution_count":8,"metadata":
{"id":"N_SzB8zma__u","executionInfo":
[],"source":[]}],"metadata":{"interpreter":
{"hash":"4abaeb3a57bf4244bbca625b90f7d8472cb60582770265c9058212119fa7bfad"},"kernel
spec":{"display_name":"Python 3.7.9 64-bit ('tensorflow':
venv)","name":"python3"},"language_info":{"codemirror_mode":
{"name":"ipython","version":3},"file_extension":".py","mimetype":"text/x-
python","name":"python","nbconvert_exporter":"python","pygments_lexer":"ipython3","
version":"3.7.9"},"orig_nbformat":4,"colab":{"provenance":
[]}},"nbformat":4,"nbformat_minor":0}

XOR Problem Tensorflow NN - Ipynb

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

XOR Problem Tensorflow NN - Ipynb

Uploaded by

Copyright:

Available Formats

{"cells":[{"cell_type":"code","execution_count":1,"metadata":

You might also like