A Single Layer Perceptron for Regression: Part 2

 

Functions Edited:

  • initialize_weights: Earlier, this function returned an numpy array of shape (1,n). It has now been edited to return the transpose, i.e., an numpy array of shape (n,1). This change was made to make matrix multiplication in the function forward_pass easy.

  • forward_pass: The main change is that now this function uses matrix multiplication to multiply all inputs with the weights. This will enable batch processing. 

  • activate: This function now activates the input sums for every row of predictor values.


    New Functions created:

    • backward_pass: Finding the change in weights required with respect to the Error. Based on the derivation, for a neuron that is on the output layer, delta w = learning_rate * eeta * input_attribute_ij wherein eeta = predicted_output_i*( 1 - predicted_output_i)(actual_output_i - predicted_output_i). 
      Parameters:  (1) input (numpy array (m,n))
                           (2) predicted_output(numpy array (m,1)) 
                           (3) actual_output(numpy array (m,1))
                           (4) learning_rate(float): the rate at which the user wants the learning to happen so                                                                  as to make the perceptron learn fast enough while not                                                                        overshooting the minima.
                           (5)error_type(string) : Type of error used so as to determine how to calculate delta                                                              w.

    • change_weights: This function is use to change the values of the current weights based on the result of the backward pass.
      Parameters:(1) delw((numpy array (n,1)): changes required in each of the weights.
                         (2) weights(numpy array of shape (n,1)): Current weights
    • epochs: This function creates epochs/iterations of forward and backward passes until the difference between the errors of 2 consecutive epochs is within a certain threshold.
      Parameters: (1)input(numpy array (m,n))
                          (2) actual_output(numpy array ()
                          (3) threshold(float)

    Functions in development:

    -  normalize
    - denormalize
    - predict


    Code

    Comments

    Post a Comment

    Popular posts from this blog

    A single layer Perceptron for Regression: Part 1

    Image
     The next agenda: to create the code for a single layer perceptron. What is a single layer perceptron? A neural network with 1 layer that has a single neuron. This means: 1) Input/Predictor values enter this neuron. 2) Each input has some weight connected to it. 3) Inputs are multiplied to their respective weights and summed up. 4) This sum is passed through an activation function. 5) The activated value of the sum/net is the predicted/output value of the neuron. 6) The error is calculated and the weights are changed with the help of a backward pass. image created by Hridaya Annuncio on Kleki.com Functions created till now: initialize_weights : The initial random weights assigned. Parameters :  (1) row (string): a single row of predictor values. This is used to find the number of                            weights that have to be initialized.            activate : Based on the activation function desired to be used, the activated net value of all the inputs times their weights is calcu
    Read more

    What Data Structure Do I use?

    Hello Everyone! Creating a Neural Network from Scratch has been on my mind for the last few months. So, I have finally converted my thoughts to action! I am going to try to create neural networks from scratch, i.e., without using libraries like Tensorflow and PyTorch. While creating the basic roadmap, I came across the following: 1) What kind of data structure should I use for each neuron? After  a bit of looking around: The answer is numpy arrays.  Why Numpy arrays over Lists?  - Each item of a list is a pointer to a memory location that actually contains the value. (And this is why lists can store items of different data types) - Each item in a numpy array is stored in contiguous memory locations (they are written in C++). -This means that each time a list value is retrieved there are two visits to memory locations, but when a numpy array value is retrieved, there is only one memory location to go to. I hope to understand more about numpy arrays - the way they use memory. 2) How to u
    Read more