nn.h File Reference

A simple neural network library header. More...

#include "la.h"
#include "calc.h"
#include <assert.h>
#include <math.h>

Include dependency graph for nn.h:

Go to the source code of this file.

Data Structures
struct	nn_neural_network
	Structure representing a neural network. More...

Macros
#define	nn_output(nn)   (nn).as[(nn).layer_count]
	Macro to access the output of the neural network.

Typedefs
typedef struct nn_neural_network	nn_neural_network
	Structure representing a neural network.

Functions
nn_neural_network	nn_alloc (size_t *arch, size_t layer_count, calc_function activation, double learning_rate)
	Allocate and initialize a neural network.
void	nn_backpropagation (nn_neural_network nn, la_matrix y)
	Perform backpropagation to compute gradients.
double	nn_calculate_cost (la_matrix y, la_matrix y_pred, la_matrix cost)
	Calculate the cost of the neural network's predictions.
void	nn_fit (nn_neural_network nn, la_matrix X, la_matrix y, la_matrix y_pred)
	Train the neural network using the given dataset.
void	nn_forward (nn_neural_network nn, la_matrix input)
	Perform a forward pass through the neural network.
void	nn_fprint (nn_neural_network nn, FILE *fp)
	Print the details of the neural network to a file.
void	nn_update_weights (nn_neural_network nn)
	Update the weights and biases of the neural network.

Detailed Description

A simple neural network library header.

This file contains the declarations of functions and structures used for basic neural network operations such as initialization, forward pass, backpropagation, and weight updates. The implementations of these functions are included in the compiled library lib/libnn.so.

Definition in file nn.h.

Macro Definition Documentation

nn_output

#define nn_output

(

nn

)

(nn).as[(nn).layer_count]

Macro to access the output of the neural network.

This macro provides a convenient way to access the output of the neural network after a forward pass.

Parameters

nn	The neural network.

Returns

The output activation matrix of the neural network.

Definition at line 88 of file nn.h.

Typedef Documentation

nn_neural_network

typedef struct nn_neural_network nn_neural_network

Structure representing a neural network.

This structure holds all the parameters and intermediate values of a neural network.

Function Documentation

nn_alloc()

nn_neural_network nn_alloc	(	size_t *	arch,
		size_t	layer_count,
		calc_function	activation,
		double	learning_rate )

Allocate and initialize a neural network.

This function allocates and initializes the neural network with the given architecture, activation function, and learning rate.

Parameters

arch	An array representing the architecture of the neural network. Each element in the array corresponds to the number of neurons in that layer.
layer_count	The number of layers in the neural network.
activation	The activation function to be used in the neural network.
learning_rate	The learning rate for training the neural network.

Returns

An initialized neural network.

Example:

size_t arch[] = {2, 3, 1};
nn_neural_network nn = nn_alloc(arch, 3, CALC_FUNCTION_SIGMOID, 0.01);

nn_backpropagation()

void nn_backpropagation	(	nn_neural_network	nn,
		la_matrix	y )

Perform backpropagation to compute gradients.

This function performs backpropagation on the neural network to compute the gradients of the cost function with respect to the weights and biases.

Parameters

nn	The neural network.
y	The target output matrix.

\[ C = \left(a^\left(L\right) - y\right)^2 \]

\[ a^\left(L\right) = \sigma\left( z^\left(L-1\right)\right) \]

\[ z^\left(L-1\right) = w^\left(L-1\right) a^\left(L-1\right) + b^\left(L-1\right) \]

\[ \frac{\partial{C}}{\partial{w^\left(L-1\right)}} = \frac{\partial{C}}{\partial{a^\left(L\right)}} \frac{\partial{a^\left(L\right)}}{\partial{z^\left(L-1\right)}} \frac{\partial{z^\left(L-1\right)}}{\partial{w^\left(L-1\right)}} \]

\[ \frac{\partial{C}}{\partial{b^\left(L-1\right)}} = \frac{\partial{C}}{\partial{a^\left(L\right)}} \frac{\partial{a^\left(L\right)}}{\partial{z^\left(L-1\right)}} \frac{\partial{z^\left(L-1\right)}}{\partial{b^\left(L-1\right)}} \]

\[ \frac{\partial{C}}{\partial{a^\left(L\right)}} = 2\left(a^\left(L\right) - y\right) \]

\[ \frac{\partial{a^\left(L\right)}}{\partial{z^\left(L-1\right)}} = \sigma'\left(z^\left(L-1\right)\right) = \]

\[ = \sigma\left(z^\left(L-1\right)\right) \left(1 - \sigma\left(z^\left(L-1\right)\right)\right) = \]

\[ = a^\left(L\right) \left(1 - a^\left(L\right)\right) \]

\[ \frac{\partial{z^\left(L-1\right)}}{\partial{w^\left(L-1\right)}} = a^\left(L-1\right) \]

\[ \frac{\partial{z^\left(L-1\right)}}{\partial{b^\left(L-1\right)}} = 1 \]

\[ \frac{\partial{C}}{\partial{w^\left(L-1\right)}} = 2\left(a^\left(L\right)-y\right) a^\left(L\right) \left(1 - a^\left(L\right)\right) a^\left(L-1\right) \]

\[ \frac{\partial{C}}{\partial{b^\left(L-1\right)}} = 2\left(a^\left(L\right)-y\right) a^\left(L\right) \left(1 - a^\left(L\right)\right) \]

---

\[ C_{k,0} = \left(a^\left(L\right)_{k,0} - y_{k,0}\right)^2 \]

\[ a^\left(L\right)_{k,0} = \sigma\left( z^\left(L-1\right)_{k,0}\right) \]

\[ z^\left(L-1\right)_{k,0} = \sum_{j=0}^{n-1} \left( w^\left(L-1\right)_{k,j} a^\left(L-1\right)_{j,0} \right) + b^\left(L-1\right)_{k,0} \]

\[ \frac{\partial{C_{k,0}}}{\partial{w^\left(L-1\right)_{k,j}}} = \frac{\partial{C_{k,0}}}{\partial{a^\left(L\right)_{k,0}}} \frac{\partial{a^\left(L\right)_{k,0}}}{\partial{z^\left(L-1\right)_{k,0}}} \frac{\partial{z^\left(L-1\right)_{k,0}}}{\partial{w^\left(L-1\right)_{k,j}}} \]

\[ \frac{\partial{C_{k,0}}}{\partial{b^\left(L-1\right)_{k,0}}} = \frac{\partial{C_{k,0}}}{\partial{a^\left(L\right)_{k,0}}} \frac{\partial{a^\left(L\right)_{k,0}}}{\partial{z^\left(L-1\right)_{k,0}}} \frac{\partial{z^\left(L-1\right)_{k,0}}}{\partial{b^\left(L-1\right)_{k,0}}} \]

\[ \frac{\partial{C_{k,0}}}{\partial{a^\left(L\right)_{k,0}}} = 2\left(a^\left(L\right)_{k,0}-y_{k,0}\right) \]

\[ \frac{\partial{a^\left(L\right)_{k,0}}}{\partial{z^\left(L-1\right)_{k,0}}} = a^\left(L\right)_{k,0} \left(1 - a^\left(L\right)_{k,0}\right) \]

\[ \frac{\partial{z^\left(L-1\right)_{k,0}}}{\partial{w^\left(L-1\right)_{k,j}}} = a^\left(L-1\right)_{j,0} \]

\[ \frac{\partial{z^\left(L-1\right)_{k,0}}}{\partial{b^\left(L-1\right)_{k,0}}} = 1 \]

\[ \frac{\partial{C_{k,0}}}{\partial{w^\left(L-1\right)_{k,j}}} = 2\left(a^\left(L\right)_{k,0}-y_{k,0}\right) a^\left(L\right)_{k,0} \left(1 - a^\left(L\right)_{k,0}\right) a^\left(L-1\right)_{j,0} \]

\[ \frac{\partial{C_{k,0}}}{\partial{b^\left(L-1\right)_{k,0}}} = 2\left(a^\left(L\right)_{k,0}-y_{k,0}\right) a^\left(L\right)_{k,0} \left(1 - a^\left(L\right)_{k,0}\right) \]

Todo

Review

\[ \frac{\partial{C_{k,0}}}{\partial{a^\left(L\right)_{k,0}}} \]

Example:

la_matrix y = la_matrix_alloc(1, 1);
la_matrix_fill(y, 1.0);
nn_backpropagation(nn, y);

nn_calculate_cost()

double nn_calculate_cost	(	la_matrix	y,
		la_matrix	y_pred,
		la_matrix	cost )

Calculate the cost of the neural network's predictions.

This function calculates the cost (loss) of the neural network's predictions using the mean squared error cost function.

Parameters

y	The target output matrix.
y_pred	The predicted output matrix.
cost	The cost matrix to store the calculated cost.

Returns

The total cost.

Example:

la_matrix y = la_matrix_alloc(1, 1);
la_matrix y_pred = la_matrix_alloc(1, 1);
la_matrix cost = la_matrix_alloc(1, 1);
la_matrix_fill(y, 1.0);
la_matrix_fill(y_pred, 0.5);
double cost_value = nn_calculate_cost(y, y_pred, cost);

nn_fit()

void nn_fit	(	nn_neural_network	nn,
		la_matrix	X,
		la_matrix	y,
		la_matrix	y_pred )

Train the neural network using the given dataset.

This function trains the neural network using the given input and target output matrices. It performs forward and backward passes and updates the network's weights and biases.

Parameters

nn	The neural network.
X	The input matrix for training.
y	The target output matrix for training.
y_pred	The predicted output matrix.

Example:

la_matrix X = la_matrix_alloc(4, 2);
la_matrix y = la_matrix_alloc(4, 1);
la_matrix y_pred = la_matrix_alloc(4, 1);
nn_fit(nn, X, y, y_pred);

nn_forward()

void nn_forward	(	nn_neural_network	nn,
		la_matrix	input )

Perform a forward pass through the neural network.

Parameters

nn	The neural network.
input	The input data.

Example:

la_matrix input = la_matrix_alloc(2, 1);
la_matrix_fill(input, 1.0);
nn_forward(nn, input);

nn_fprint()

void nn_fprint	(	nn_neural_network	nn,
		FILE *	fp )

Print the details of the neural network to a file.

This function prints the details of the neural network, including weights, biases, and activations, to the specified file.

Parameters

nn	The neural network to be printed.
fp	The file pointer to which the details will be printed.

Example:

FILE* fp = fopen("nn_details.txt", "w");
nn_fprint(nn, fp);
fclose(fp);

nn_update_weights()

void nn_update_weights

(

nn_neural_network

nn

)

Update the weights and biases of the neural network.

This function updates the weights and biases of the neural network using the computed gradients and the learning rate.

Parameters

nn	The neural network.

Example:

nn_update_weights(nn);