Lecture8: Learning 4

Autoencoders

• Want features to capture meaningful factors of variation in data.

Unsuprevised Pre-training

• Fine-tune
  • small learning rate in the encoder part

Sparse Autoencoders

• Capture the factors with the property of sparsity

Denoising Autoencoders

The denoising autoencoder (DAE) is an autoencoder that receives a corrupted data point as input and is trained to predict the original, uncorrupted data point as its output.

• Manifold Learning

Convolutional Neural Network

• Cross-Corrleation (sliding inner product)