- This is a good way to study the algorithm
- Not all of the deep learning library developed by c++ community are easy to build, more precisely, it is a pain to build them on some major platform(ex : windows), c and c++ community do not have a standard build system really is a big problem.
The first algorithm I am trying to build is softmax regression based on the tutorial of UFLDL and softmax regression with opencv(I implement it with Eigen and opencv rather than matlab build in function).In this post I want to record how do I implement the softmax regression and show some results. I pick Eigen to help me implement the algorithms because it is
- Portable and very easy to compile
- Api are clean, easy to use and expressive
- Performance is quite good
- Well maintain, nice document
- Expression template rock
- Find a good matrix library, like Eigen or Armadillo
- Study the algorithm carefully, make sure you understand every steps of it
- Write down the matrix operations on a white paper, check the dimensions, the operations results of each step is reasonable or not, if you can not persuade yourself this result is meaningful, go back to step 2, do not bet on luck
- Implement the algorithms
- Run the gradient checking algorithms to check the result, if error, go back to step 2 or step 3
- Run on clean examples like MNIST, those examples already do preprocess for you
- If the result are poor, try to tune the parameters or go back to step 2
using namespace ocv::ml; namespace{ using EMat = ocv::ml::softmax<>::EigenMat; EMat read_data(std::string const &file) { std::ifstream in(file); std::cout<<"read file\n"; if(in.is_open()){ std::cout<<"is open\n"; EMat output(30, 284); for(size_t col = 0; col != output.cols(); ++col){ for(size_t row = 0; row != output.rows(); ++row){ in>>output(row, col); } } return output; }else{ std::cout<<"cannot open file : "<<file<<"\n"; } return softmax<>::EigenMat(); } std::vector<int> read_label(std::string const &file) { std::ifstream in(file); std::vector<double> output; //not the most efficient way, but easier to write std::copy(std::istream_iterator(in), std::istream_iterator (), std::back_inserter(output)); return std::vector<int>(output.begin(), output.end()); } } void softmax_test() { ocv::ml::softmax<> sm; sm.train(read_data("softmax_train_data.txt"), read_label("softmax_train_label.txt")); auto const TestData = read_data("softmax_test_data.txt"); auto const TestLabel = read_label("softmax_test_label.txt"); double correct = 0; for(size_t i = 0; i != TestLabel.size(); ++i){ auto const Result = sm.predict(TestData.block(0, i, TestData.rows(), 1)); if(Result == TestLabel[i]){ ++correct; } } std::cout<<"true positive pro : "<< (correct/TestLabel.size())<<"\n"; }
This class use mini-batch to train the data, the results should within 89%~94%.The example use the ocv_libs of v1.1, the test data is located at here(I download from eric yuan). The test example(softmax_test) is located at here(v1.0).
Next post of deep learning will talk about the implementation details of this softmax class.
No comments:
Post a Comment