Wishcraft
Wish It to Create It
AI-powered XR application that enables users to create and interact with 3D environments using natural gestures and voice commands.
Timeline
July-August 2024
(6 weeks)
Role
XR Design & Prototype
Video Production
Tools
Figma, Unity 3D,
After Effects
Teammate
Minzhe Zou
Instructors
Yasaman Sheri
Miller Klitsner
Technical Support
Hongming Li
Weide Zhang
People struggle to learn and
create with existing 3D software.
😩
❌ Non-Intuitive 3D modeling
Traditional mouse inputs are limiting and not natural when interacting with 3D objects.
What is we could use natural inputs like gestures and voice commands to make creation feel easier?
❌ Lack of Spatial Awareness
Designing on a flat 2D screen makes it hard to understand space and scale.
What if we could leverage XR technology to design directly in the 3D environment?
❌ Difficult to Visualize Ideas
Not everyone can visualize their ideas through drawing.
What if we could use AI to help us with visualization?
What if we could simply use
our voice and gestures in an immersive space with the assistance from AI to build our 3D world?
Introducing
Wishcraft
AI
XR
Multimodal Input
Revolutionize
3D environment
creation
Leverage voice commands and natural gestures with AI to design and interact with 3D environments, offering a seamless alternative to traditional control methods for a more intuitive creation experience.
Wishcraft is tailored for people who are not 3d modelers — like product managers and creative directors —who have innovative ideas but struggle to visualize them.
By using natural, everyday conversation—much like making a wish—they can see their concepts come to life without the need for professional 3D design skills.
Wishcraft
Immersive
Intelligent
With AI training, Wishcraft transforms ideas into 3D objects and environments.
TripoSR on ComfyUI
It is not just a tool;
it's a versatile platform applicable to numerous use case
for early stage rapid prototyping and planning
Micro
Macro
Interior
Design
Film & Stage
Set Design
Event
Planning
Accessibility
Testing
Game
Development
Urban
Planning
Theme Park
Design
Designing individual rooms or spaces within a building.
Creating sets for movies and theater productions.
Designing spatial layouts for events like weddings or conferences.
Ensuring spaces are accessible and meet diverse user needs.
Creating immersive environments for video games.
Designing larger scale developments like neighborhoods or public spaces.
Planning extensive theme park layouts including rides and attractions.
3
Interview
&
MetaSDK/3DGen Testing
We did a small interview to observe how people interact with objects in their daily lives.
Test text-to-3d workflow with TripoSR on ComfyUI — it takes 3 minute to one generation
So I decides to use pre-downloaded models to build the 3D inventory
Test VoiceSDK
Low Fidelity Prototype
Start Developing
Step by Step
Different interactions
Generate 1 in the air
Generate 3 in the air
Generate 1 in hand
Generate 1 in hand
Single selector / gallery view for different scenario
Delete by eraser
Delete by trash can
Create different interaction pattern inspiring by exist life pattern
Create different interaction pattern inspiring by exist life pattern
With Wishcraft,
we make 3D creation
as natural as wishing for it