# Announcing DeepLearningKit — Open Source Deep Learning Framework for Apple's iOS, OS X and tvOS

- Canonical URL: https://amund.blog/archive-medium-announcing-deeplearningkit/
- Author: Amund Tveit (https://amund.blog/about/)
- Published: 2015-12-28
- Tags: Apple Silicon, Deep Learning, Archive
- Original Source: medium.com (https://medium.com/@atveit/announcing-deeplearningkit-open-source-deep-learning-framework-for-apple-s-ios-os-x-and-tvos-be656a90d763)

---

Happy to announce that a (early) version of [DeepLearningKit](http://deeplearningkit.org/) is available on:

- [http://deeplearningkit.org](http://deeplearningkit.org/)
- [https://github.com/deeplearningkit/](https://github.com/deeplearningkit)
- [http://deeplearningkit.org/tutorials-for-ios-os-x-and-tvos/tutorial-using-deeplearningkit-with-ios-for-iphone-and-ipad/](http://deeplearningkit.org/tutorials-for-ios-os-x-and-tvos/tutorial-using-deeplearningkit-with-ios-for-iphone-and-ipad/)

**FAQ**

**0. What does DeepLearningKit do?**

It currently allows using deep convolutional neural networks model trained in [Caffe](http://caffe.berkeleyvision.org/) on Apple’s iOS, OS X and tvOS *(transformed from protobuffers into json with the tool at*[*https://github.com/DeepLearningKit/caffemodel2json*](https://github.com/DeepLearningKit/caffemodel2json)*— a*[*tutorial*](http://deeplearningkit.org/tutorials)*about this will come later)*.

**1. Open Source Licence?**

Apache 2.0

**2. How to get started?**

Have a look at [Tutorial — Using DeepLearningKit with iOS for iPhone and iPad](http://deeplearningkit.org/tutorials-for-ios-os-x-and-tvos/tutorial-using-deeplearningkit-with-ios-for-iphone-and-ipad/) — it is about using a pre-trained CIFAR-10 Network in Network example.

**3. What is DeepLearningKit developed in?**

It is developed in Metal (for GPU Acceleration) and Swift (for app integration). I believe DeepLearningKit is the first (public) Deep Learning tool that is using the Metal compute API for GPUs (Metal is Apple’s recommended way to program GPUs)

**4. More documentation**

More tutorials and a paper describing DeepLearningKit will be made available on [http://deeplearningkit.org](http://deeplearningkit.org) (+ [http://arxiv.org](http://arxiv.org) for the paper)

**5. I love developing for Apple’s [iOS,OS X or tvOS] and would like to contribute to this project, how?**

Here are a few thoughts:

1. Fork [github.com/deeplearningkit](https://github.com/deeplearningkit) repo(s), play with it/them and provide feedback or fixes.
2. Create apps that use DeepLearningKit (*disclaimer: still very early version*) and tell us about them.
3. Try (and perhaps adapt) different types of deep neural networks to DeepLearningKit, e.g.
4. Microsoft Research’s ImageNet 2015 winning approach described in the paper [Deep Residual Learning for Image Recognition](http://arxiv.org/abs/1512.03385), or
5. DeepMind’s (Google) AI for Atari games described in the papers [Human-level control through deep reinforcement learning](http://www.nature.com/nature/journal/v518/n7540/full/nature14236.html), [Deep Reinforcement Learning with Double Q-Learning](http://arxiv.org/abs/1509.06461) and [Playing Atari with Deep Reinforcement Learning](http://arxiv.org/abs/1312.5602)
6. Other types of Deep Learning, check out [http://DeepLearning.University](http://deeplearning.university/) for inspiration
7. **Performance Optimization wrt Metal (GPU):** Metal is a very new API (in particular for GPGPU non-graphical processing), and there is probably ways of improve usage of it.
8. **Performance Optimization wrt algorithms (e.g. shader functions for comvolution)**: see our paper for roadmap
9. **Importers**: develop model importers (in Swift) for convolutional neural networks) from other tools than Caffe, e.g. Torch, TensorFlow, Theano, Nervana Systems, DeepLearning4J or Pylearn.[HDF5 is an interesting format](http://www.kdnuggets.com/2015/10/blocks-fuel-deep-learning-frameworks.html).
10. **Training Support:** our goal was to primarily support using already trained Deep Learning models (*since in the long run people will probably not train their own DL models but rather pick them from a Deep Learning Model store or similar, see our paper for why*), but it would still be great to train convolutional neural networks in DeepLearningKit itself.
11. **Image Handling Support:** DeepLearningKit is missing basic conversion from e.g. UIImage to RGB (the example network supports 32x32x3 [CIFAR RGB Image Format](https://www.cs.toronto.edu/~kriz/cifar.html), but has no conversion from UIImage to it). Check out e.g. [Drawing Images From Pixel Data — In Swift](http://blog.human-friendly.com/drawing-images-from-pixel-data-in-swift) and [Image Processing in iOS Part 1: Raw Bitmap Modification](http://www.raywenderlich.com/69855/image-processing-in-ios-part-1-raw-bitmap-modification) for inspiration.

**6. Is DeepLearningKit production ready for my mission critical app?**

most likely not, but that doesn’t stop you from testing it out?

**7. DeepLearningKit reminds me more about CUDA/GPU Libraries such as **[** Nvidia’s cuDNN**](https://developer.nvidia.com/cudnn)** or **[** Facebook’s fbCunn**](https://github.com/facebook/fbcunn)** rather than larger tools such as Torch, TensorFlow and Caffe, is that right?**

You’re right, DeepLearningKit can be roughly seen as an early “metalDNN” with Swift packaging for loading and running models. *(It currently doesn’t support Fast Fourier based convolution such as Facebook’s fbcunn)*

**8. Who Developed and Open Sourced DeepLearningKit**

DeepLearningKit was developed and opensourced by the company [Memkite](http://memkite.com/) — check out the [About page](http://deeplearningkit.org/about/) for details

Best regards,

[Amund Tveit](https://no.linkedin.com/in/amundtveit)
