WORLDMEM: Long-term Consistent World Simulation with Memory

🚀 How to Explore WorldMem

Follow these simple steps to get started:

  1. Choose a scene.
  2. Input your action sequence.
  3. Click "Generate".
  • You can continuously click "Generate" to extend the video and observe how well the world maintains consistency over time.
  • For best performance, we recommend running locally (1s/frame on H100) instead of Spaces (5s/frame).
  • ⭐️ If you like this project, please give it a star on GitHub!
  • 💬 For questions or feedback, feel free to open an issue or email me at zeqixiao1@gmail.com.

Happy exploring! 🌍

🖼️ Start from this frame.

🎞️ Generated videos. New contents are marked in red box.

🏞️ Choose a scene and start generation.

🕹️ Input action sequences for interaction.

💡 Action Key Guide

W: Turn up      S: Turn down     A: Turn left     D: Turn right
Q: Go forward   E: Go backward   N: No-op         U: Use item

👇 Click to quickly set action sequence examples.

⚙️ Advanced Settings
10 50
2 10
4 16

👇 Click to review generated examples, and continue generation based on them.
Examples
Case History Sequences Denoising Steps Context Length Memory Length