Age and gender classification by opencv, dlib and mxnet

    In this post, I will show you how to build an age gender classification application with the infrastructures I created in the last post. Almost everything are same as before, except the part of parsing the NDArray in the forward function.

    Before we dive into the source codes, let us have some examples. The images are predicted by two networks and concatenate, left side predicted by light model, right side predicted by heavy model based on resnet50.

   

    The results do not looks bad for both of the models if we don't know their ages :), let us use the model to predict the ages of famous person, like trumps(with resnet50).

 

    Unfortunately, results of age classification are not that good under different angles and expressions, this is because age classification from an image is very difficult, even human cannot accurately predict ages of persons by looking at a single image.    

1. Difference of face recognition and age gender classification

    The codes of parsing face recognition features are

std::vector<insight_face_key> result;
size_t constexpr feature_size = 512;
Shape const shape(1, feature_size);
for(size_t i = 0; i != batch_size; ++i){
    NDArray feature(features.GetData() + i * feature_size, shape, Context(kCPU, 0));
    result.emplace_back(std::move(feature));
}

    The codes of parsing age and gender classification are

std::vector<insight_age_gender_info> result;
int constexpr features_size = 202;
for(size_t i = 0; i != batch_size; ++i){
    auto const *ptr = features.GetData() + i * features_size;
    insight_age_gender_info info;
    info.gender_ = ptr[0] > ptr[1] ? gender_info::female_ : gender_info::male_;
    for(int i = 2; i < features_size; i += 2){
        if(ptr[i + 1] > ptr[i]){
                info.age_ += 1;
        }
    }
    result.emplace_back(info);
}

    Except of these part, everything are the same as before.

2. Make codes easier to reuse

    It is a pain to maintain similar codes with minor difference, in order to alleviate the prices of maintenance, I create a generic predictor as a template class with three policies, implement the face recognition and age/gender classification with this generic predictor.

template<typename Return, typename ProcessFeature, typename ImageConvert = dlib_mat_to_separate_rgb>
class generic_predictor
{
/*please check the details on github*/
}

    We could use it like to create age gender predictor as following

struct predict_age_gender_functor
{
    std::vector<insight_age_gender_info>
    operator()(const mxnet::cpp::NDArray &features, size_t batch_size) const
    {
        std::vector<insight_age_gender_info> result;
        int constexpr features_size = 202;
        for(size_t i = 0; i != batch_size; ++i){
            auto const *ptr = features.GetData() + i * features_size;
            insight_age_gender_info info;
            info.gender_ = ptr[0] > ptr[1] ? gender_info::female_ : gender_info::male_;
            for(int i = 2; i < features_size; i += 2){
                if(ptr[i + 1] > ptr[i]){
                    info.age_ += 1;
                }
            }
            result.emplace_back(info);
        }
        return result;
    }
 };
 
using insight_age_gender_predict = mxnet_aux::generic_predictor<insight_age_gender_info, predict_age_gender_functor>;

    Please check github if you want to know the implementation details.

3. Summary

    Gender prediction works very well, unfortunately age predictions is far from ideal. If we could obtain huge data set, which contain the face of the same person with different range of ages, expression, angles and the number of races are not super imbalance, accuracy of age accuracy may improve very much, but a huge data set like this is very hard to collect. 

    The source codes could find on github.