https://github.com/shure-dev/natural-langauge-as-policies

Represent embodied skills with natural language, not code
https://github.com/shure-dev/natural-langauge-as-policies

llm robot robotics

Last synced: about 2 months ago
JSON representation

Represent embodied skills with natural language, not code

• Host: GitHub
• URL: https://github.com/shure-dev/natural-langauge-as-policies
• Owner: shure-dev
• Created: 2024-03-21T02:22:04.000Z (10 months ago)
• Default Branch: main
• Last Pushed: 2024-03-25T06:38:20.000Z (10 months ago)
• Last Synced: 2024-11-22T06:09:37.219Z (about 2 months ago)
• Topics: llm, robot, robotics
• Homepage:
• Size: 35.2 KB
• Stars: 2
• Watchers: 1
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

        


  Code implementation will be available here. Preprint. Under review.

  
Natural Language as Policies (NLaP):  

    Reasoning for Coordinate-Level Embodied Control with LLMs

  

  


    Yusuke Mikami, Andrew Melnik, Jun Miura, Ville Hautamäki

  

  

  

  

  

  

  


  

  


  


  


  

  

  

  


    

  

  



## Abstruct

We demonstrate experimental results with LLMs that address robotics action planning problems. Recently, LLMs have been applied in robotics action planning, particularly using a code generation approach that converts complex high-level instructions into mid-level policy codes. In contrast, our approach acquires text descriptions of the task and scene objects, then formulates action planning through natural language reasoning, and outputs coordinate level control commands, thus reducing the necessity for intermediate representation code as policies. Our approach is evaluated on a multi-modal prompt simulation benchmark, demonstrating that our prompt engineering experiments with natural language reasoning significantly enhance success rates compared to its absence. Furthermore, our approach illustrates the potential for natural language descriptions to transfer robotics skills from known tasks to previously unseen tasks.