PaLM-E

PaLM-E is a decoder-only language model that generates textual completions autoregressively given a prefix or prompt. It is an embodied multimodal model that incorporates real-world continuous sensor modalities into language models, establishing a link between words and percepts. PaLM-E can address a variety of embodied reasoning tasks from multiple observation modalities on multiple embodiments.