Question for the end of lesson 1

1. Do you need these for deep learning?

• Lots of math T / F
• Lots of data T / F
• Lots of expensive computers T / F
• A PhD T / F

Answers:

• False. Just high school math is sufficient
• False. We've seen record-breaking results with <50 items of data
• False. You can get what you need for state of the art work for free
• False. All that matters is a deep understanding of AI & ability to implement neural networks (NNs) in a way that is actually useful. Don’t care if you even graduated high school.

2. Name five areas where deep learning is now the best in the world.

Answer: Natural language processing (NLP), Computer vision, Medicine, Biology, Image generation.

Just for your FYI, here is are more areas:

• Natural language processing (NLP): Answering questions; speech recognition; summarizing documents; classifying documents; finding names, dates, etc. in documents; searching for articles mentioning a concept
• Computer vision:: Satellite and drone imagery interpretation (e.g., for disaster resilience); face recognition; image captioning; reading traffic signs; locating pedestrians and vehicles in autonomous vehicles
• Medicine: Finding anomalies in radiology images, including CT, MRI, and X-ray images; counting features in pathology slides; measuring features in ultrasounds; diagnosing diabetic retinopathy
• Biology: Folding proteins; classifying proteins; many genomics tasks, such as tumor-normal sequencing and classifying clinically actionable genetic mutations; cell classification; analyzing protein/protein interactions
• Image generation: Colorizing images; increasing image resolution; removing noise from images; converting images to art in the style of famous artists
• Recommendation systems: Web search; product recommendations; home page layout
• Playing games: Chess, Go, most Atari video games, and many real-time strategy games
• Robotics: Handling objects that are challenging to locate (e.g., transparent, shiny, lacking texture) or hard to pick up
• Other applications: Financial and logistical forecasting, text to speech, and much more…

3. What was the name of the first device that was based on the principle of the artificial neuron?

The Mark I Perceptron

4. Based on the book of the same name, what are the requirements for parallel distributed processing (PDP)?

Answer:

• A set of processing units
• A state of activation
• An output function for each unit
• A pattern of connectivity among units
• A propagation rule for propagating patterns of activities through the network of connectivities
• An activation rule for combining the inputs impinging on a unit with the current state of that unit to produce an output for the unit
• A learning rule whereby patterns of connectivity are modified by experience
• An environment within which the system must operate

5. What were the two theoretical misunderstandings that held back the field of neural networks?

1. An MIT professor named Marvin Minsky along with Seymour Papert, wrote a book called Perceptrons (MIT Press), about Rosenblatt's invention. They showed that a single layer of these devices was unable to learn some simple but critical mathematical functions (such as XOR). In the same book, they also showed that using multiple layers of the devices would allow these limitations to be addressed. Unfortunately, only the first of these insights was widely recognized.
2. Although researchers showed 30 years ago that to get practical good performance you need to use even more layers of neurons, it is only in the last decade that this principle has been more widely appreciated and applied.

6. What is a GPU?

Graphics Processing Unit (GPU): Also known as a graphics card. A special kind of processor in your computer that can handle thousands of single tasks at the same time, especially designed for displaying 3D environments on a computer for playing games. These same basic tasks are very similar to what neural networks do, such that GPUs can run neural networks hundreds of times faster than regular CPUs. All modern computers contain a GPU, but few contain the right kind of GPU necessary for deep learning.

7. Open a notebook and execute a cell containing: 1+1. What happens?

2

More Questions: I will add answeres in a few days:

8. Follow through each cell of the stripped version of the notebook for this chapter. Before executing each cell, guess what will happen.

9. Complete the Jupyter Notebook online appendix.

10. Why is it hard to use a traditional computer program to recognize images in a photo?

11. What did Samuel mean by "weight assignment"?

12. What term do we normally use in deep learning for what Samuel called "weights"?

13. Draw a picture that summarizes Samuel's view of a machine learning model.

14. Why is it hard to understand why a deep learning model makes a particular prediction?

15. What is the name of the theorem that shows that a neural network can solve any mathematical problem to any level of accuracy?

16. What do you need in order to train a model?

17. How could a feedback loop impact the rollout of a predictive policing model?

18. Do we always have to use 224×224-pixel images with the cat recognition model?

19. What is the difference between classification and regression?

20. What is a validation set? What is a test set? Why do we need them?

21. What will fastai do if you don't provide a validation set?