Encoder-Decoder Architecture

Encoder-decoder architecture is the original Transformer configuration from the 2017 "Attention Is All You Need" paper, where an encoder processes the input sequence into a contextualized representation that a separate decoder then consumes to generate the output sequence. The architecture excels at sequence-to-sequence tasks where input and output domains differ — machine translation, summarization, question-answering — because the encoder is unconstrained by causal masking and the decoder can attend to the full input. Encoder-decoder models include T5 (Google's text-to-text Transformer), BART (Facebook's denoising encoder-decoder), and the entire pre-GPT-3 era of NLP. Modern frontier LLMs have largely converged on decoder-only architectures because they generalize well across tasks via prompting, are simpler to train, and use compute more uniformly across input and output. Encoder-decoder remains the dominant architecture for specific domains: machine translation (NLLB, M2M-100), speech recognition (Whisper), and image captioning. AI governance teams document the architecture choice in model lineage because encoder-decoder and decoder-only models behave differently on certain tasks.

Encoder-decoder models in Centralpoint: Centralpoint operates above encoder-decoder models like T5, BART, and Whisper alongside decoder-only LLMs in a model-agnostic platform. Tokens are metered per skill, prompts stay local, supports both generative and embedded models, and deploys chatbots through one line of JavaScript on any portal.


Related Keywords:
Encoder-Decoder Architecture,,