This week’s task involved modifying the Layer interface parameters to make them more user friendly and readable. Along with it, the unnecessary feature of order of execution was also removed.

Where does this fit in the project?

The more user-friendly and readable our project is, the more people are going to use it! By involving more strings and less numbers, the interface becomes more user friendly. Also, features like the order of execution(user has to specify in which order to perform the computations of the layers) make it more difficult for the user to completely understand the working of the API. Excluding this feature, makes the computer work more in the background and the user less in the front.

After all, don’t work for the computer, instead make the computer work for you!(This quote doesn’t belong to me!)

Robert Kiyosaki

Outcome of this Week

The library is now easier to understand and use. The computer does more work than the human to create the neural network and the computational graph behind the scenes. The user enters more strings than numbers and in any order, that may or may not be correct. The computer works to understand the strings and generate the correct connections.

Logic of the Code

Although some silly mistakes like incomplete explanations of comments and documentation are very frustrating. Yet, the end product is always worth working for! There were 2 main objectives for the week, let’s discuss them one by one.

Providing a better user interface was easy to acheive. The lower level functions remained almost the same, only some upper level functions in the Layer interface class had to be changed to map for the type of layer, sensor input and list of output connections strings.

Introducing TensorFlow to the project was the real deal! In order to remove the feature of order of execution and introduce the feature of delayed connections for all the recurrent connections, the following algorithm is working behind the scenes:

  1. (Assuming the user has provided the input) Iterate the layers in a breadth first search manner from input towards output. In essence, generate the network layer by layer.

  2. For each layer, generate a TFFunction that takes sensor input from the user and an input vector from the previous layers. The sensor input is a placeholder that is provided by the user in the form of feed dictionary. The input vector is generated by concatenating the output of previous layers.

  3. For layers that serve as input layer, the input vector is taken as zero and the sensor input is taken from the user fed input dictionary.

  4. For other layers, the input vector is generated in two ways. For inputs that are not delayed, they are taken from the current outputs of the layers. For delayed layers, they are taken from the previous ouput of the layers. However, taking the current output may be a problem because the layer should be declared in order to use it’s output. Therefore, layers that create this problem are kept in an error queue.

  5. The error queue is iterated again and again till it’s size reduces to zeros, removing the layers from the queue that are generated. If the size of the error queue reduces with each iteration, then we are on the correct track. If the size remains the same, then there is some problem in specification of delays by the user. In order to deal with it, the layers that are creating a problem during initialization are instead taken from the previous output. Hence, one by one, all the errors will be resolved and the error queue will soon be empty.

A simple case analysis shows that this algorithm, would always result in an empty queue, and would not be stuck in an infinte loop(Discrete Mathematics in action!!). A very wonderful result of this algorithm is that this even allows the detection of a Static or Dynamic Network(just observe the size of the error queue). Hence, the network is generated for any order of intialization provided by the user. It would be correct every time the user enters correct delayed connections. Even if some connections are not specified correctly, the API, will most of the time generate the correct network.

Problems and their Solutions

This week was filled with little setbacks and excitement. However, all the problems approached were solved in some way or the other.

  • Complete Understanding: This is one big problem that I have faced over the course of 4 weeks. For many parts of the code, I did not completely understand, what the input parameters were going to be and what was to be expected as output. Only after making mistakes in these parts, I was able to improve the overall code. In order to avoid this problem from next time onwards, I would work to ask more doubts to keep a clear understanding of the overall project. So, only important mistakes and improvements are highlighted and not silly ones!

  • Improvements and Iterations: No big thing was made in a single day in a single iteration. It took months of prototyping and iterations to make the perfect product. Just a thought though!