# (07강) Sequential Models - RNN

> **Further Question**
>
> * LSTM에서는 Modern CNN 내용에서 배웠던 중요한 개념이 적용되어 있습니다. 무엇일까요?
> * Pytorch LSTM 클래스에서 3dim 데이터(batch\_size, sequence length, num feature), `batch_first` 관련 argument는 중요한 역할을 합니다. `batch_first=True`인 경우는 어떻게 작동이 하게되는걸까요?

## 1. Sequentual Model

* Naive sequence model

  ![](https://3944465397-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-MjcKlzGhHYe2bvmxTwS%2Fuploads%2FwKEvgX0FyxLvEAFp48Sr%2Fimage.png?alt=media\&token=4b0d76f9-5c17-4d5e-b7a1-751ee2087e27)
* Autoregressive model

  ![](https://3944465397-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-MjcKlzGhHYe2bvmxTwS%2Fuploads%2FFAoim5dLIx9V2FhB8rvH%2Fimage.png?alt=media\&token=bc308e39-7e0e-4210-a15f-2dc1c2121907)
* Markov model (first-order autoregressive model)

  ![](https://3944465397-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-MjcKlzGhHYe2bvmxTwS%2Fuploads%2FXifHlxDKRCPnvoD5aNfp%2Fimage.png?alt=media\&token=b7e58dcd-8e40-46bf-8ea7-3d247ce2923a)
* Latent autoregressive model

  ![](https://3944465397-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-MjcKlzGhHYe2bvmxTwS%2Fuploads%2FXv6IZsYhCJArR4133h5l%2Fimage.png?alt=media\&token=294fc4f2-1dc0-49bf-888f-183d2203b60d)

## 2. RNN (Recurrent Neural Network)

![](https://3944465397-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-MjcKlzGhHYe2bvmxTwS%2Fuploads%2F0ERYaNrM5NgQNvfDVnHO%2Fimage.png?alt=media\&token=6564b437-5fff-4e80-9e23-08228d68f2c5)

* **Short-term dependencies**
* 오래된 정보가 소실되게 된다. (Gradient Vanishing / Exploding)

## 3. LSTM (Long Short Term Memory)

RNN이 가지는 소실 문제를 해소하기 위해 과거의 정보를 누적하는 cell state를 가진다.

![](https://3944465397-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-MjcKlzGhHYe2bvmxTwS%2Fuploads%2F5RVMiBvOPzIlzrjKTZpf%2Fimage.png?alt=media\&token=1d970c4d-8383-4634-899f-14acca3884cf)

![](https://3944465397-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-MjcKlzGhHYe2bvmxTwS%2Fuploads%2Fqy3q8bWKGyS9CrsPmXTn%2Fimage.png?alt=media\&token=727b4437-7f64-4567-8168-91b0b204f8c4)

![](https://3944465397-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-MjcKlzGhHYe2bvmxTwS%2Fuploads%2FvvZkMsrSZOdvaGgl2gFM%2Fimage.png?alt=media\&token=0235ae6d-f128-4306-9625-9c8daf276139)

## 4. GRU (Gated Recurrent Unit)

![](https://3944465397-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-MjcKlzGhHYe2bvmxTwS%2Fuploads%2FLoeVq3m1KTXRsv6i3naA%2Fimage.png?alt=media\&token=40f62fe6-db02-4e55-9c9b-0a338dfc7766)