Tuesday 27 December 2016

Deep learning as new electronics


It is hard to imagine a modern life without electronics: radios, TVs, microwaves, mobile phones and many more gadgets. Dump or smart, they are all based on the principles of semi-conducting and electromagnetism. Now we are using these devices for granted without worrying about these underlying laws of physics.  Most people do not care about circuits that run in chips and carry out most functions of the devices.

For the past 5 years, a new breed of human-like functionalities has emerged through advances of a new field called deep learning: self-driving cars, voice command in mobile phone, translation in hundreds of language pairs and a new kind of art. In 2016, ten years after its revival, deep learning has taken over the Internet. People have used deep learning-powered products in daily life without worrying about how the underlying neural nets work.

These two fields free us from many physical and psychological constraints:

  • Electronic devices give us freedom of communication over distance, a new kind of experiences with augmented reality and many more.
  • Deep learning enables freedom from having to make tedious and incorrect decisions (e.g., driving a car), freedom of information access (personalization), of hand (e.g., voice command), of finance (automated trading), of feature extraction (through representation learning), and many more.
It is worth noting that electronics and deep learning are different in principles.
  • Electronics devices are designed with great precision for specific functions in mind. Imprecision comes from the quantum uncertainty principle and thermal fluctuations.
  • Neural nets on the other hand, are designed to learn to perform a function of its own, where data (and sometimes model) uncertainty is built in.
However, it is also striking that they are quite similar in many ways.

Super-city of interconnected simple parts

Modern electronic devices are truly super-cities built out of just few kinds of primitive building blocks. The same holds for deep neural nets:
  • Electronic primitives: resistor, capacitor, transistor, coil, diode, logic gate and switch.
  • Neural net primitives: integrate-and-fire neuron, multiplicative gating, differentiable logic gate, switch and attention module. Interestingly, one of the most recent idea is called "Highway networks", borrowing the idea that highway traffic is free of traffic lights.
These primitives are connected in graphs:
  • Electronic devices works by moving electrons in correct order and number. The force that makes them move is potential difference. A design circuit captures all necessary information.
  • In neural nets, the activation function is like the electronic current. The main difference is that the magnitude of "current" in neural nets can be learnt. A computational graph is what is needed for model execution.
Not just analogy: A two-way relationship
  • Electronics deep learning: At present, advances in electronics have given huge boost in efficiency of deep learning with GPU, TPU and other initiatives. It is interesting to see if we can learn from electronics in designing deep nets? For example, will something analogous to integrated-circuits in deep architectures?
  • Deep learning  electronics: I predict that soon the reverse will hold true: deep learning will play a great role in improving efficiency and functionalities of electronic devices. Stay tuned.

66 comments:

  1. This comment has been removed by a blog administrator.

    ReplyDelete
  2. Great post.Its very useful and thanks for sharing this article with us.
    Electronic goods Manufacturing companies in chennai

    ReplyDelete
  3. I must appreciate you for providing such a valuable content for us.

    ReplyDelete
  4. I think things like this are really interesting. I absolutely love to find unique places like this. It really looks super creepy though!! Best Machine Learning Training | machine learning certification training in chennai | Machine learning course fees in chennai

    ReplyDelete
  5. Electronics plays a very important role in our daily life. There are so many types of electronics that we use in our daily life. If you want to buy the best electronics at the affordable price then you can visit our site. Best consumer electronics WA

    ReplyDelete
  6. Great post.Thanks for sharing this article with us.Keep posting!!
    Electronic goods Manufacturing Companies in chennai

    ReplyDelete
  7. Thanks For Sharing Excellent Blog. Machine Learning is steadily moving away from abstractions and engaging more in business problem solving with support from AI and Deep Learning. With Big Data making its way back to mainstream business activities, now smart (ML) algorithms can simply use massive loads of both static and dynamic data to continuously learn and improve for enhanced performance. Pridesys IT Ltd

    ReplyDelete
  8. This comment has been removed by the author.

    ReplyDelete
  9. Awesome post. Thanks for sharing this post with us.The heart of a man-made Intelligence based mostly system is it's model. A model is nothing however a program that improves its data through a learning method by creating observations concerning its atmosphere.Machine learning course this sort of learning-based model is sorted underneath supervised Learning.

    ReplyDelete
  10. nice post, thank you for sharing the post.


    https://excelr.com.my/course/machine-learning-using-r/

    ReplyDelete
  11. great blog, thank you.




    https://excelr.com.my/course/certification-on-industrial-revolution-4-0/

    ReplyDelete
  12. your info is very much useful, thank you for sharing.

    pmp certification
    pmp certification malaysia
    project management certification
    project management courses
    project management course malaysia
    pmp course malaysia




    https://excelr.com.my/course/project-management-professional-pmp/

    ReplyDelete
  13. very nice blog, thank you for sharing the information.



    https://excelr.com.my/course/project-management-professional-pmp/

    ReplyDelete
  14. This is very educational content and written well for a change. It's nice to see that some people still understand how to write a quality post! OMC parts

    ReplyDelete
  15. Awesome post. You Post is very informative. Thanks for Sharing.
    Machine Learning Course in Noida

    ReplyDelete
  16. Nice to see this article , it’s so more informative Information. Thanks for sharing.
    best java training institute in chennai

    ReplyDelete
  17. Awesome post. You Post is very informative. Thanks for Sharing.
    ORACLE TRAINING IN CHENNAI

    ReplyDelete
  18. This comment has been removed by the author.

    ReplyDelete
  19. Interesting topic for a blog. I have been searching the Internet for fun and came upon your website. Fabulous post. Thanks a ton for sharing your knowledge! It is great to see that some people still put in an effort into managing their websites. I'll be sure to check back again real soon. best 2 way radios

    ReplyDelete
  20. This comment has been removed by the author.

    ReplyDelete
  21. Google brain is working in the Big data solutions companies to make it a huge success for the world. We hope that society will soon use AI devices at a reasonable cost.

    ReplyDelete
  22. We are tied directly into the sate’s renewal database which allows us to process your request almost instantly. https://360digitmg.com/course/data-visualization-using-tableauhttps://360digitmg.com/course/certification-program-in-data-scienceFor more info visit website
    https://360digitmg.com/course/data-analytics-using-python-r

    ReplyDelete
  23. Now a days the demand on Datascience, machine Learning, Deep learning has been increased and also has very good job opportunities. I felt this article is very much useful to any one who wanna peruse there carrier in such fields, as it has very good useful information

    data science training in aurangabad
    data science course in aurangabad

    ReplyDelete
  24. The development of artificial intelligence (AI) has propelled more programming architects, information scientists, and different experts to investigate the plausibility of a vocation in machine learning. Notwithstanding, a few newcomers will in general spotlight a lot on hypothesis and insufficient on commonsense application. machine learning projects for final year In case you will succeed, you have to begin building machine learning projects in the near future.

    Projects assist you with improving your applied ML skills rapidly while allowing you to investigate an intriguing point. Furthermore, you can include projects into your portfolio, making it simpler to get a vocation, discover cool profession openings, and Final Year Project Centers in Chennai even arrange a more significant compensation.


    Data analytics is the study of dissecting crude data so as to make decisions about that data. Data analytics advances and procedures are generally utilized in business ventures to empower associations to settle on progressively Python Training in Chennai educated business choices. In the present worldwide commercial center, it isn't sufficient to assemble data and do the math; you should realize how to apply that data to genuine situations such that will affect conduct. In the program you will initially gain proficiency with the specialized skills, including R and Python dialects most usually utilized in data analytics programming and usage; Python Training in Chennai at that point center around the commonsense application, in view of genuine business issues in a scope of industry segments, for example, wellbeing, promoting and account.


    The Nodejs Training Angular Training covers a wide range of topics including Components, Angular Directives, Angular Services, Pipes, security fundamentals, Routing, and Angular programmability. The new Angular TRaining will lay the foundation you need to specialise in Single Page Application developer. Angular Training

    ReplyDelete
  25. I was really happy to find this blog. I wanted to thank you for the excellent reading!! I really enjoy every bit of it and I've got you bookmarked to find out new stuff you 're writing.360DigiTMG data science course

    ReplyDelete
  26. I 'm glad to get this blog published! Students also want to cognize the keys to writing successful literary essays. Your first-class knowledge of this good post may become a good basis for such individuals. Nice one to watchdata science course in noida

    ReplyDelete
  27. Extraordinary blog went amazed with the content they have developed in a very descriptive manner. Hope you deliver the same near the future as well. Gratitude to the blogger for the efforts.

    360DigiTMG Cloud Computing Course

    ReplyDelete
  28. Your comment is very beneficial for the future, you have written in a very beautiful way, you have an inspiration for the youth who come to your comment is really very beautiful, nowadays children do not know where they are wandering, no one can make a comment like you. The post is very different.
    Call Girls in Sector 38
    Manesar Escorts Services
    Gurugram escorts
    independent call girls in gurugram

    ReplyDelete
  29. Your comment is very beneficial for the future, you have written in a very beautiful way, you have an inspiration for the youth who come to your comment is really very beautiful, nowadays children do not know where they are wandering, no one can make a comment like you. The post is very different.
    russian escorts gurugram
    call girls sector 18
    escort service sector 77
    call girls in gurgaon

    ReplyDelete
  30. nice blog!! i hope you will share a blog on Data Science.
    data science course noida

    ReplyDelete
  31. This is my first time visit to your blog and I am very interested in the articles that you serve. Provide enough knowledge for me. Thank you for sharing useful and don't forget, keep sharing useful info: Rent to own computers

    ReplyDelete
  32. Nice work, truly valuable to me.
    I hope you keep it up.
    if you are looking for best offshore development services then,
    Visit here:
    Iyrix Technologies
    Remote Software Developers
    Software Development Services
    Web Application Development
    Big Data Development

    ReplyDelete
  33. The article was up to the point and described the information about education and learning. Thanks to blog author for wonderful and informative post. Local AA Meetings

    ReplyDelete
  34. I am really very happy to visit your blog. Directly I am found which I truly need. please visit our website for more information about Machine Learning Company

    ReplyDelete
  35. This is a great article with lots of informative resources. I appreciate your work this is really helpful for everyone. Check out our website Custom Mobile App Development Services USA for more Symentix Technologies Private Limited related info!

    ReplyDelete