---
title: What are the Bidirectional Encoder Representations from Transformers?
description: Pre-training of Deep Bidirectional Transformers (BERT) for Language Understanding for Natural Language Processing Tasks
---

[Glossary](https://www.xenonstack.com/glossary)

# [What are the Bidirectional Encoder Representations from Transformers?](https://www.xenonstack.com/glossary/bidirectional-encoder-transformers)

 Written by [Dr. Jagreet Kaur](https://www.xenonstack.com/glossary/author/dr-jagreet-kaur) | 17 Mar 2023

## Overview of Bidirectional Encoder Representations from Transformers

- The concept of transfer learning
- It is only for the models which are without RNNs.
- Give attention to a very small diagram (or subwords) vocab also gives it an advantage considering the perspective of the memory. It is built on current technologies like Transfer learning, Generative Pre-training, semi-supervised learning, ELMo, and ULMFit.

## Comparison with other Bidirectional Unsupervised Models

It is the first of its kind, a deeply bidirectional unsupervised model that is trained using any simple corpus. It is built on PyTorch.

## Bidirectional Encoder Representations from Transformers Uses

It can be used for all text analytical tasks such as sentiment detection, classification, machine translation, named entity recognition, summarization, and question answering.

[View full post](https://www.xenonstack.com/glossary/bidirectional-encoder-transformers)

```json
{
  "@context" : "http://schema.org",
  "@type" : "BlogPosting",
  "author" : {
    "@type" : "Person",
    "name" : "Dr. Jagreet Kaur"
  },
  "dateModified" : "2023-03-17T09:56:14.350Z",
  "datePublished" : "2023-03-17T09:56:14Z",
  "headline" : "Bidirectional Encoder Representations from Transformers",
  "image" : {
    "@type" : "ImageObject",
    "height" : 720,
    "url" : "https://www.xenonstack.com/hubfs/bidirectional-encoder-transformers.png",
    "width" : 1280
  },
  "mainEntityOfPage" : "https://www.xenonstack.com/glossary/bidirectional-encoder-transformers",
  "publisher" : {
    "@type" : "Organization",
    "logo" : {
      "@type" : "ImageObject",
      "height" : 60,
      "url" : "/hs/hsstatic/content_shared_assets/static-1.4092/img/default-amp-logo.png",
      "width" : 60
    },
    "name" : "Glossary"
  }
}
```