Vanya Cohen

I am a PhD Student at the University of Texas at Austin (currently on leave) and AI Scientist at Blackbird.AI, where I created Compass, an agentic fact-checker for multimodal social media content. My PhD advisor is Ray Mooney.

I completed my BSc and MSc at Brown University, advised by Stefanie Tellex and George Konidaris in the Humans to Robots Lab.

Fun facts: I helped create the first publically downloadable, open-source LLM OpenGPT-2 and dataset OpenWebText. My Erdős number is 3.

Email  /  Google Scholar  /  LinkedIn

Research

I'm interested in grounded natural language processing, reinforcement learning, and robotics.

CaT-Bench: Benchmarking Language Model Understanding of Causal and Temporal Dependencies in Plans

A benchmark that evaluates language models’ ability to reason about step dependencies in task plans, using causal and temporal relations. It tests LLMs ability to predicting procedure dependencies and give explanations. We find SOTA LLMs perform poorly on this task despite its simplicity.

A Survey of Robotic Language Grounding: Tradeoffs Between Symbols and Embeddings

Robotic language grounding methods can be positioned along an axis that ranges from methods that map natural language to formal symbolic representations to those that map to high-dimensional vector spaces. The survey explores the trade-offs between interpretability, generalizability, and the scalability of these methods.

CAPE: Corrective Actions from Precondition Errors using Large Language Models

CAPE resolves precondition errors in task planning for robotic agents by leveraging large language models (LLMs). The method re-prompts LLMs using error feedback, allowing robots to make corrective actions in real-world environments. We implement CAPE on a Spot robot and demonstrate improvements over previous LLM planning methods.

OpenGPT-2: We Replicated GPT-2 Because You Can Too

OpenGPT-2 is a replication of OpenAI’s 1.5B parameter GPT-2 LLM. Our model was the first open-source, publically downloadable LLM. It was trained on our OpenWebText dataset and was followed by several other prominent open-source LLM releases.

Grounding Language Attributes to Objects using Bayesian Eigenobjects

A method to recognize 3D objects from natural language descriptions and depth images. The system leverages unsupervised learning on 3D object meshes and a small amount of annotated data and generalizes to unseen and novel viewpoints.

OpenWebText: An Open Source Replication of OpenAI's WebText

OpenWebText is a replication of OpenAI’s WebText dataset. It has been downloaded over 4 million times. OpenWebText was used to train our model OpenGPT-2, among other LLMs.

Design and source code from Jon Barron's website