Solved – Neural network architecture that takes in matrix as input and outputs matrix

I am trying to build a neural network that takes in matrix (read: m x n) data at a particular timestep and then outputs the prediction for the next timestep. While retaining spacial relationships. For instance, given sea surface temperatures at one month, predict the next month.

My current idea is to use a few layers of CNN, reshape into vector, feed it through fully connected layers, then reshape it back into the original m x n shape. However, I do not know if this is best practice, as it might be difficult for the network to understand the relationship between the vector and reshaping into map shape.

Additionally, I would like to feed in other inputs, such as a month index.

How should I proceed?

Neural networks in computer vision usually operate with inputs of shape (m, n)— images—so there is nothing unusual about that.

I would avoid using fully connected layers in this case and go for a fully-convolutional network. Have a look at U-Net, which has become quite a standard architecture for image-to-image tasks:

Feeding it additional data of could be tricky, but fusing them in as additional channels at the bottom part of the U shape could work. Proper scaling might be necessary.

Similar Posts:

Rate this post

Leave a Comment