This is my very first attempt to work with neural networks after reading some theory about them.
What I am trying to do is to classify long lists of numbers (600+ of them per training session, consisting of the numbers 1-10, typically). Each such list indicates a certain activity, so I’ve written a C++ program that utilizes the OpenCV DNN libraries to create a MLP network to get the job done.
To train the MLP, I label each row in the matrix of training data I have based upon the most prevalent item per row (even though it is not necessarily a strong indication regarding the activity it represents, but that will do for now), standardize the matrix and perform a PCA. The MLP consists of 3 layers (3, 5, 1 perceptrons) and I employ a sigmoid activation function.
To predict, I take a vector (=one row), standardize it, run PCA on it and do the call to OpenCV libraries.
Unfortunately the results are not good even after several training sessions - the resulting prediction seems to be biased towards a constant value that is within range, but does not reflect the actual activity.
I am using OpenCV C++ on Linux.
My questions are:
Is a MLP network suitable to address the problem I am trying to solve?
Is it worth fiddling with the MLP layout to get better results?
Are there any other relevant details you can think of that may help me?
As I pointed out above this is my very first DNN program ever. Any comment would be greatly appreciated!
Thanks for your reply. Here is the logic to load the model and perform training / prediction:
void DeepLearningModel::loadModel(cv::Mat &training_data)
// load training data for later PCA discarded here...
// training data is available "as is" for train(), this is only
// needed for PCA in predict()
cv::FileStorage dnn_file(m_model_path, cv::FileStorage::READ);
m_neural_network = ANN_MLP::load(m_model_path);
m_neural_network = ml::ANN_MLP::create();
void DeepLearningModel::train(cv::Mat &training_data)
std::cout << "train" << std::endl;
// Train a neural network to identify patterns in a list of numbers
// Every row gets its own label, representing activity demostrated by row
auto number_of_samples =
Mat labels(number_of_samples, 1, CV_32FC1);
// calculate standardized values before PCA
// Perform PCA on the training data to reduce its dimensionality
PCA pca(training_data, Mat(), PCA::DATA_AS_ROW);
Mat layers_sizes = Mat(5, 1, CV_32S);
layers_sizes.row(0) = Scalar(
.cols); // must be equal to the number of colmuns of 'trainingData'
layers_sizes.row(1) = Scalar(5);
layers_sizes.row(2) = Scalar(8);
layers_sizes.row(3) = Scalar(5);
layers_sizes.row(4) = Scalar(1);
// Set activation function and termination criteria
TermCriteria(TermCriteria::MAX_ITER, 10000, 0.0001));
CV_32F); // convert labels to floating-point matrix
// Train the neural network
m_neural_network->train(reduced_data, ml::ROW_SAMPLE, outputData);
std::cout << "predict" << std::endl;
size_t num_features = training_data.size();
Mat test_data = Mat_<float>(1, num_features, CV_32F);
for (auto const &logical_thread_id : vector_prepared_for_prediction)
test_data.at<float>(0, index++) = logical_thread_id;
PCA pca(loaded_model, Mat(), PCA::DATA_AS_ROW);
std::cout << "Activity predicted to be " << prediction << std::endl;
Thank. You can think of this data as identifiers of program parts being executed. I suppose you can consider this as categorical data, then? Ok, I will need to switch to a RNN instead - I was suspecting this, but was not sure…