Yoshua Bengio’s Deep Thoughts on Deep Thinking

May 22nd, 2018 · 11 comments

Thinking about Thinking

One of the things that surprised me when researching Deep Work was how rare it was to find examples of smart people talking coherently about the process of effective thinking.

This is why I was pleased when a reader recently pointed me toward a video interview with computer scientist Yoshua Bengio — one of the big names in machine learning.

Around an hour and ten minutes into the interview, Bengio turns his attention to the topic of productive thought:

“There is also something to be said about concentration…to really make big progress in science you also need times when I can be very focused and where the ideas about a problem, and different points of view, and all the elements sort of fill my mind. I’m completely filled with this…that’s when I can be really productive."

Filling your mind with a single problem, Bengio emphasizes, is not a fast process, noting:

“It might take a really long time before you reach that state…"

Hard problems are hard to understand. The best researchers in my field of theoretical computer science, for example, tend to be those who have the drive, brain power and flexibility to read other peoples’ papers constantly to spark a new recombination or extension that advances the field — an exhausting process.

But as Bengio emphasizes, this effort is worth it:

“[After you fill your mind with a problem is] when you can really start seeing through things and getting things to stand together and solving…now you can extend science, now when things are solid in your mind you can move forward."

Our culture likes to emphasize eureka moments, but we often miss the hard, patient work that goes into preparing the mind to generate breakthroughs.

(Hat tip: Daniel)

11 thoughts on “Yoshua Bengio’s Deep Thoughts on Deep Thinking"

  1. Ayrat says:

    You might also find interesting the book by
    Josh Waitzkin
    the Art of Learning
    where he emphasizes the importance of internalizing of the basics
    to the degree that it becomes your native language.

    1. EA says:

      Yep, that was a good book! (Great chess player)

  2. Brian J says:

    Cal — love your blog and work, but I have to admit this post really shocked me because there has been a long tradition in philosophy of trying to get at this very question, from Aristotle’s pioneering work on logic to Descartes’ Discourse on Method. And, since you’re mathematically inclined, it might delight you to know that Plato himself looked to the methods utilized by geometers for mathematical discovery as a model for articulating what we do in thinking in general. Plato was interested in how mathematical analysis lead to moments of deep comprehension (called “nous" in Greek); in turn, that deep comprehension was then re-organized into a synthesis:
    Sorry to lecture you here but philosophy is my bread and butter.

  3. Adam says:

    This book has a very useful discussion of focussed vs diffused thinking in learning science that may be worth discussing.

    The eureka moment can come when you are relaxed (diffused) after concentrated effort on a problem (the incubation period in the literature on learning). In short, I don’t that there is one way to think deeply to move a problem ahead.

  4. sri says:

    Aha!: The Moments of Insight that Shape Our World by William B Irvine is a good book on the topic of insights, how they come about, the role of subconscious mind, and the prerequisite of multiple attempts made by the conscious mind etc.

  5. Viliami says:

    Hi Cal
    In a podcast interview for Straight A in Optimise, you mentioned that you systematically did trial and error to find the best study methods that suited your situation, and it seems that you’ve done this as well when creating a system that allowed you to master certain skills in a limited time period. I’d like to ask, how do you create such systems from scratch?

  6. Daniel Palacios says:

    I couldn’t agree more with the last paragraph. Our culture see the eureka moment as something only related to “geniuses"; of course that they don’t tend to take in mind all the hard and deep work behind the moment.

    Indeed, this post is the right place to share one of my all time favourite quotes:

    “Chance favors the prepared mind."
    -Louis Pasteur

  7. Vikrant Singh says:

    Hi Cal,

    There was a post “Are You Using Social Media or Social Media Using You" & then another on “Brain Hacking and Sean Parker on building FB", I found something that adds to that. Hope you and other out here find it useful.
    This video has formed FB employees including Sean Parker telling the deep dark things about FB itself.


  8. It’s a little bit complicated but really interesting.
    Learning, and reaching a better understanding something that will lead us to real changes.
    A.I. it’s simply a new frontier will be used to really have a better life for all the people?

  9. William Pennat says:

    The best researchers in my field of theoretical computer science, for example, tend to be those who have the drive, brain power and flexibility to read other peoples’ papers constantly to spark a new recombination or extension that advances the field — an exhausting process. — From the article

    That’s precisely why I left academia. I got sick of reading poorly written, poorly reasoned tracts just so I’d have a set of references for my own. At least 90% of everything in academic journals is dull, pedestrian crap. I don’t see how it could inspire anyone to greater creative thinking. (Sorry….)

Leave a Reply

Your email address will not be published. Required fields are marked *


Name *

Email *


Save my name, email, and website in this browser for the next time I comment.

Confirm you are NOT a spammer