Математические методы прогнозирования (практика, В.В. Стрижов)/Группа 574, осень 2019

Материал из MachineLearning.

(Различия между версиями)
Перейти к: навигация, поиск
(Группа)
(Seminar 12 (Bakhteev, slides))
 
(28 промежуточных версий не показаны.)
Строка 4: Строка 4:
Short link [http://bit.ly/IS_B2 bit.ly/IS_B2]
Short link [http://bit.ly/IS_B2 bit.ly/IS_B2]
-
=== Группа ===
+
This series of seminars continues the course Bayesian model selection and investigates the theoretical aspects of model selection in various application problems.
 +
 
 +
Videolectures are available [https://www.youtube.com/playlist?list=PLk4h7dmY2eYH9RtoKGzxHKji0GLiBzSlZ here].
 +
 
 +
==== Seminar 1 (Isachenko, [[Медиа:Isachenko2019DeepGenerativeModels1.pdf‎|slides]]) ====
 +
* Generative models
 +
* Applications
 +
* Autoregressive models (CharRNN, MADE, WaveNet, PixelCNN)
 +
 
 +
==== Seminar 2 (Isachenko, [[Медиа:Isachenko2019DeepGenerativeModels2.pdf‎|slides]]) ====
 +
* Generative vs discriminative
 +
* Latent variable models
 +
* Variational Inference
 +
* ELBO
 +
* Variational Autoencoder
 +
 
 +
==== Seminar 3 (Isachenko, [[Медиа:Isachenko2019DeepGenerativeModels3.pdf‎|slides]]) ====
 +
* Mean field approximation
 +
* Flow models (NICE, RealNVP)
 +
 
 +
==== Seminar 4 (Isachenko, [[Медиа:Isachenko2019DeepGenerativeModels4.pdf‎|slides]]) ====
 +
* VAE Limitations
 +
* Flows in VAE
 +
* Autoregressive flows (MAF, IAF, Parallel WaveNet)
 +
 
 +
==== Seminar 5 (Isachenko, [[Медиа:Isachenko2019DeepGenerativeModels5.pdf‎|slides]]) ====
 +
* IWAE (lower bound, posterior, inactive units)
 +
* ELBO surgery
 +
* VampPrior
 +
 
 +
==== Seminar 6 (Isachenko, [[Медиа:Isachenko2019DeepGenerativeModels6.pdf‎|slides]]) ====
 +
* Autoregressive decoder in VAE
 +
* Posterior collapse, decoder weakening
 +
* Disentangled representations
 +
* beta-VAE
 +
 
 +
==== Seminar 7 (Bakhteev, [https://github.com/bahleg/tex_slides/raw/master/oct_19/slides1_elbo.pdf slides]) ====
 +
* Model selection statement
 +
* ELBO for model selection
 +
* Early Stopping is Nonparametric Variational Inference
 +
* Langevin dynamics
 +
 
 +
==== Seminar 8 (Bakhteev, [https://github.com/bahleg/tex_slides/raw/master/oct_19/slides2_hyper.pdf slides]) ====
 +
* Hyperparameter optimization
 +
* Bi-level optimization
 +
* RMD
 +
* Gradient optimization
 +
 
 +
==== Seminar 9 (Grabovoy, [https://github.com/andriygav/EMprior/blob/master/Lecture/Grabovoy2019EMprior.pdf slides]) ====
 +
* Mixture of Models
 +
* Mixture of Experts
 +
* Priors on the local Models
 +
 
 +
==== Seminar 10 (Isachenko, [[Медиа:Isachenko2019DeepGenerativeModels7.pdf‎|slides]]) ====
 +
* Reversible Residual Networks
 +
* Glow
 +
* Neural ODE
 +
 
 +
==== Seminar 11 (Bakhteev, [https://github.com/bahleg/tex_slides/raw/master/oct_19/slides3_meta.pdf slides]) ====
 +
* Meta-optimization
 +
* Pruning
 +
* Structure sampling
 +
 
 +
==== Seminar 12 (Bakhteev, [https://github.com/bahleg/tex_slides/raw/master/oct_19/slides4_struct.pdf slides]) ====
 +
* ARD
 +
* AdaNet
 +
* NAS
 +
* Gumbel-Softmax
 +
* Variational inference with structure generation
 +
 
 +
== Группа ==
 +
 
 +
=== 5 курс ===
{|class="wikitable"
{|class="wikitable"
|-
|-
! Студент
! Студент
! Тест 1
! Тест 1
-
! HM 1
 
! Тест 2
! Тест 2
 +
! Тест 3
 +
! Тест 4
 +
! Тест 5
 +
! HW 1
 +
! HW 2
|-
|-
|Васильев Илья
|Васильев Илья
| -
| -
-
|
+
| -
-
|
+
| 0.59
 +
| -
 +
| -
 +
| -
 +
|
|-
|-
|Гадаев Тамаз Тазикоевич
|Гадаев Тамаз Тазикоевич
| 0.56
| 0.56
-
|
+
| 0.94
-
|
+
| 0.75
 +
| -
 +
| 0.88
 +
| -
 +
|
|-
|-
|Гладин Егор Леонидович
|Гладин Егор Леонидович
| -
| -
-
|
+
| -
-
|
+
| -
 +
| -
 +
| -
 +
| -
 +
|
|-
|-
|[[Участник:andriygav|Грабовой Андрей Валериевич]]
|[[Участник:andriygav|Грабовой Андрей Валериевич]]
| 0.63
| 0.63
 +
| 0.31
 +
| 0.67
 +
| 0
 +
| -
|[[Media:Grabovoy2019Course5HM1.pdf‎|Essay]]
|[[Media:Grabovoy2019Course5HM1.pdf‎|Essay]]
-
|
+
|
|-
|-
| Кислинский Вадим Геннадьевич
| Кислинский Вадим Геннадьевич
| -
| -
-
|
+
| -
-
|
+
| -
 +
| -
 +
| -
 +
| -
 +
|
|-
|-
| Козлинский Евгений Михайлович
| Козлинский Евгений Михайлович
| -
| -
-
|
+
| -
-
|
+
| -
 +
| -
 +
| -
 +
| -
 +
|
|-
|-
| Криницкий Константин Денисович
| Криницкий Константин Денисович
| -
| -
-
|
+
| 0.25
-
|
+
| -
 +
| -
 +
| -
 +
|[https://drive.google.com/file/d/1xk2uvz6FmWz0s1SCleF56pRFsTlXvQ8_/view?usp=sharing essay]
 +
|
|-
|-
| Кириллов Егор Дмитриевич
| Кириллов Егор Дмитриевич
| -
| -
-
|
+
| -
-
|
+
| -
 +
| -
 +
| -
 +
| -
 +
|
|-
|-
| Рогозина Анна Андреевна
| Рогозина Анна Андреевна
| -
| -
-
|
+
| -
-
|
+
| -
 +
| -
 +
| -
 +
| -
 +
|
|-
|-
| Плетнев Никита Вячеславович
| Плетнев Никита Вячеславович
| 0.82
| 0.82
-
|
+
| 0.25
-
[https://drive.google.com/a/phystech.edu/file/d/1kkqHZDR2ATzWtf8FRc8BjbcNAXaApvy1/view?usp=drivesdk | Essay]
+
| 0.67
-
|
+
| -
 +
| 0.63
 +
|[https://drive.google.com/a/phystech.edu/file/d/1kkqHZDR2ATzWtf8FRc8BjbcNAXaApvy1/view?usp=drivesdk Essay]
 +
|
|-
|-
| Малиновский Григорий Станиславович
| Малиновский Григорий Станиславович
| 0.82
| 0.82
-
||[[https://drive.google.com/file/d/1UkmIDJgZEDTtzSk_a20IZlTy62F2tnJB/view?usp=sharing|task_1]]
+
| 0.81
-
|
+
| 0.84
 +
| 1
 +
| 0.63
 +
||[https://drive.google.com/file/d/1UkmIDJgZEDTtzSk_a20IZlTy62F2tnJB/view?usp=sharing|task_1]
 +
|
|-
|-
| Самохина Алина Максимовна
| Самохина Алина Максимовна
| -
| -
-
|
+
| -
-
|
+
| 0.25
 +
| 1
 +
| 0.75
 +
| -
 +
|
|-
|-
| Султанов Азат Русланович
| Султанов Азат Русланович
| -
| -
-
|
+
| -
-
|
+
| -
 +
| -
 +
| -
 +
| -
 +
|
|-
|-
| Федосов Павел Андреевич
| Федосов Павел Андреевич
| -
| -
-
|
+
| -
-
|
+
| -
 +
| -
 +
| -
 +
| -
 +
|
|-
|-
| Шульгин Егор Владимирович
| Шульгин Егор Владимирович
 +
| -
 +
| -
 +
| 0.34
 +
| -
 +
| -
 +
| 0.13
 +
|
 +
|-
 +
|}
 +
 +
=== 6 курс ===
 +
{|class="wikitable"
 +
|-
 +
! Студент
 +
! HW 1
 +
! HW 2
 +
|-
 +
| Сайранов Данил
| -
| -
|
|
 +
|-
 +
| Александра Гальцева
 +
| -
 +
|
 +
|-
 +
| Фельдман Даниил
 +
| -
|
|
|-
|-
| Никитин Филипп
| Никитин Филипп
-
| 0.56
+
| -
-
|
+
|
|
|-
|-
| Фалахов И
| Фалахов И
-
| 0.5
+
| -
|
|
 +
|-
 +
| Собраков
 +
| -
|
|
|-
|-
Строка 101: Строка 259:
-
This series of seminars continues the course Bayesian model selection and investigates the theoretical aspects of model selection in various application problems.
+
* Topics
-
 
+
-
==== Seminar 1 (Isachenko, [[Медиа:Isachenko2019DeepGenerativeModels1.pdf‎|slides]]) ====
+
-
* Generative models
+
-
* Applications
+
-
* Autoregressive models (CharRNN, MADE, WaveNet, PixelCNN)
+
-
 
+
-
==== Seminar 2 (Isachenko, [[Медиа:Isachenko2019DeepGenerativeModels2.pdf‎|slides]]) ====
+
-
* Generative vs discriminative
+
-
* Latent variable models
+
-
* Variational Inference
+
-
* ELBO
+
-
* Variational Autoencoder
+
-
 
+
-
 
+
-
 
+
-
 
+
-
 
+
-
 
+
-
 
+
-
 
+
-
 
+
-
 
+
-
 
+
-
* Seminar 3 (Isachenko)
+
-
** Inference methods of approximation
+
-
** Zoo of variational autoencoders and practical examples
+
-
* Seminar 4 (Isachenko)
+
** Generative adversarial networks
** Generative adversarial networks
-
* Seminar 5 (Bakhteev)
 
** Methods of model selection
** Methods of model selection
** Generalization theorem
** Generalization theorem
-
* Seminar 6 (Bakhteev)
 
** Complexity theorems
** Complexity theorems
-
* Seminar 7 (Grabovoy?)
 
** Mixture of experts
** Mixture of experts
** Priors on the mixture
** Priors on the mixture
** Privileged learning and distilling
** Privileged learning and distilling
-
* Seminar 8 (Aduenko?)
 
** Theorem of number of experts
** Theorem of number of experts
-
* Seminar 9 (Vladimirova?)
 
** Prior propagation for deep learning networks
** Prior propagation for deep learning networks
-
* Seminar 10
 
** Directional Bayesian statistics
** Directional Bayesian statistics
-
* Seminar 11
 
** Bayesian structure learning
** Bayesian structure learning
-
* Seminar 12
 
** Probabilistic metric space construction
** Probabilistic metric space construction
-
* Seminar 13
 
** Informative prior
** Informative prior
-
* Seminar 14
 
** Bayesian programming
** Bayesian programming
-
 
+
** Informative prior with applications
-
 
+
-
 
+
-
 
+
-
 
+
-
 
+
-
 
+
-
 
+
-
 
+
-
 
+
-
* Informative prior with applications
+

Текущая версия


Short link bit.ly/IS_B2

This series of seminars continues the course Bayesian model selection and investigates the theoretical aspects of model selection in various application problems.

Videolectures are available here.

Seminar 1 (Isachenko, slides)

  • Generative models
  • Applications
  • Autoregressive models (CharRNN, MADE, WaveNet, PixelCNN)

Seminar 2 (Isachenko, slides)

  • Generative vs discriminative
  • Latent variable models
  • Variational Inference
  • ELBO
  • Variational Autoencoder

Seminar 3 (Isachenko, slides)

  • Mean field approximation
  • Flow models (NICE, RealNVP)

Seminar 4 (Isachenko, slides)

  • VAE Limitations
  • Flows in VAE
  • Autoregressive flows (MAF, IAF, Parallel WaveNet)

Seminar 5 (Isachenko, slides)

  • IWAE (lower bound, posterior, inactive units)
  • ELBO surgery
  • VampPrior

Seminar 6 (Isachenko, slides)

  • Autoregressive decoder in VAE
  • Posterior collapse, decoder weakening
  • Disentangled representations
  • beta-VAE

Seminar 7 (Bakhteev, slides)

  • Model selection statement
  • ELBO for model selection
  • Early Stopping is Nonparametric Variational Inference
  • Langevin dynamics

Seminar 8 (Bakhteev, slides)

  • Hyperparameter optimization
  • Bi-level optimization
  • RMD
  • Gradient optimization

Seminar 9 (Grabovoy, slides)

  • Mixture of Models
  • Mixture of Experts
  • Priors on the local Models

Seminar 10 (Isachenko, slides)

  • Reversible Residual Networks
  • Glow
  • Neural ODE

Seminar 11 (Bakhteev, slides)

  • Meta-optimization
  • Pruning
  • Structure sampling

Seminar 12 (Bakhteev, slides)

  • ARD
  • AdaNet
  • NAS
  • Gumbel-Softmax
  • Variational inference with structure generation

Группа

5 курс

Студент Тест 1 Тест 2 Тест 3 Тест 4 Тест 5 HW 1 HW 2
Васильев Илья - - 0.59 - - -
Гадаев Тамаз Тазикоевич 0.56 0.94 0.75 - 0.88 -
Гладин Егор Леонидович - - - - - -
Грабовой Андрей Валериевич 0.63 0.31 0.67 0 - Essay
Кислинский Вадим Геннадьевич - - - - - -
Козлинский Евгений Михайлович - - - - - -
Криницкий Константин Денисович - 0.25 - - - essay
Кириллов Егор Дмитриевич - - - - - -
Рогозина Анна Андреевна - - - - - -
Плетнев Никита Вячеславович 0.82 0.25 0.67 - 0.63 Essay
Малиновский Григорий Станиславович 0.82 0.81 0.84 1 0.63 [1]
Самохина Алина Максимовна - - 0.25 1 0.75 -
Султанов Азат Русланович - - - - - -
Федосов Павел Андреевич - - - - - -
Шульгин Егор Владимирович - - 0.34 - - 0.13

6 курс

Студент HW 1 HW 2
Сайранов Данил -
Александра Гальцева -
Фельдман Даниил -
Никитин Филипп -
Фалахов И -
Собраков -


  • Topics
    • Generative adversarial networks
    • Methods of model selection
    • Generalization theorem
    • Complexity theorems
    • Mixture of experts
    • Priors on the mixture
    • Privileged learning and distilling
    • Theorem of number of experts
    • Prior propagation for deep learning networks
    • Directional Bayesian statistics
    • Bayesian structure learning
    • Probabilistic metric space construction
    • Informative prior
    • Bayesian programming
    • Informative prior with applications
Личные инструменты