Rather than build a machine learning libraries by myself, prefer to improve an already exist, well written machine learning libraries looks like a more reasonable solution for me. Recently I begin to contribute some codes to mlpack, a fast, modular and scalable c++ machine learning library.
I implement a fine tune algorithms of deep network on top of mlpack and try to open a pull request, but I guess it will take a long time before it could be merged, so I clone the whole project and do my customization on it, the implementation details of the fne tune class is place at github.
I learn a lot from contribute to open source community, the experiences like code reviews are precious and hardly happen in my daily jobs. In Asia, many software companies do not care about the quality of codes, they over underestimate how important the quality of codes could affect the maintenance fees, always rush for "fast coding" and generate extremely crappy codes(most of the programmers/leaders/managers cannot understand their own codes within a few months, this suck). Almost no one has a will to keep studying after graduate, that is why so many of them hardly improve even they have 20 years experiences.
If you are same as me, cannot find a good company really care about codes quality yet, give open source a try, you should have a higher chance to gain real world experiences of "what is a good software project looks like" rather than just study the "story" or "theorem" from the books. mlpack is not perfect, but the quality surpass any legacy codes I maintained for.