Dynamic Programming

You can consider "dynamic programming" as "careful/controlled brute force". What we usually
do is that we basically solve the problem using brute force and add memoization after we have
solved the problem. So you are given a problem, what you should do is NOT think about
memoization at first but instead you should think about solving it using brute force then add
memoization. For optimization problems, we want to write a function that returns the optimal
solution according to some variables that are sent to the function.

1. First of all, you need to identify problem variables. This can be, for example, the health of a
monster as in the lab or the current location (i,j) in the grid problem as in the assignment. These
variables are used as inputs to the function and will be used for memoization later on. What I mean
by variables are the things that change from one sub-problem to another, for example if we have
an input to a function which is the power of each weapon. The power of each weapon does not
change from one sub-problem to another so it's not considered as of the problem variables (though
they should be sent to the function) the health of a monster changes from one sub-problem to
another, so this is considered a variable.

2. Then you should define sub-problems. Assuming you're trying to draw a recursion tree for the
monster killing problem, if we think about each monster individually, the root node of the tree will
contain the starting health of the monster. What I mean by defining sub-problems is what will be
the children of the root node. If we have 3 weapons, we want to try all of them since we want to
try all possibilities since it's brute force. So the root node will have 3 children, one for each weapon
and the node will have value = current health - weapon power.

3. The next step is to relate sub-problems solutions. What I mean by that is, if we have 3 weapons,
this means we have 3 sub-problems, but how can we relate these sub-problems and return the
optimal solution? This is usually done for optimization problems where we try to find a minimum
or maximum value. So if we have 3 sub-problems, we relate them by a function (for example MIN
or MAX) or (MAX + something). You should assume that these sub-problems return the optimal
solution and you think what is the optimal solution for the original problem given these sub-
problems. So, you relate these sub-problems by a function that returns the optimal solution for the
problem.

Steps 2 and 3 try to characterize the structure of an optimal solution.

4. Identify the base cases.

5. Add memoization (last step). We said in step 1 that we have variables for each problem and for
each problem there's an optimal solution that is returned according to these variables. Given a
problem, we now want to see if we have already solved this problem or not, if we have solved it
then return the solution without re-computing it. The variables mentioned in step 1 are used to
identify problems. So, if F(30) = 20. This means that 30 is the variable used to identify the problem
and 20 is the optimal solution. In the case of monster killing, we have the variable health as stated
before, we can use that health to identify if we have solved a problem previously with the same
health or not. The DP storage used in memoization is used by taking the variables of the problem
as input and returns the optimal solution as output. Before this step, we identified the optimal
structure for the solution and the base cases, so we solved the problem using brute force. In this
step we add one line at the beginning of our brute force solution and one at the end and that is it.
For example, at the beginning, we say "if(memo[variable]) return memo[variable];".
And at the end of the brute force function, we say "memo[variable] = optimal solution; return
memo[variable];"

Steps 1 to 4 is what we used to do throughout the whole semester, which is brute force. Step 5 is
the new part added for dynamic programming.