Portals

License: MIT

Powered by ChatGPT for Code Assistance

Overview

Portals is an immersive AR experience for Snap Spectacles, designed to bring music, culture, and climate awareness to life through interactive and accessible features. Utilizing spatial audio and computer vision, Portals allows users to engage with cultural and musical experiences in a unique way, prioritizing accessibility, cultural education, joy, and climate change awareness.

This project aims to make diverse musical traditions accessible, with a focus on underrepresented communities and global musical history.

Project Goals

	•	Accessibility: Designing inclusive, voice-controlled features and audio cues for individuals with limited hand mobility or speech difficulties.
	•	Cultural Education: Enabling auditory exploration of music from various cultures, focusing on historical context and lesser-known instruments.
	•	Joy: Creating fun, collaborative AR jam sessions with friends using spatial audio.
	•	Climate Change Awareness: Highlighting music that connects to natural environments or climate themes to increase awareness through audio immersion.

Features and Methods

1. Co-Creation 🎧

	•	Spatial Audio Circles: Position friends and collaborators in a virtual jam session circle, each playing an instrument or adding to the ambiance. Friends can interact audibly to avoid visual clutter.

2. Interactive Auditory Timeline 📜

	•	Cultural Markers Recognition: Using Spectacles’ computer vision, the app identifies cultural artifacts or historical sites and launches a timeline with music, stories, and significant figures from that culture.
	•	Evolution of Genres: Users navigate a timeline showcasing how different music genres have evolved, focusing on historical and social influences.

3. Personalization and Safety Tools 🔒

	•	Voice Command Accessibility: Users can control the experience through voice, providing accessibility for individuals with limited mobility.
	•	Custom “Safe Word” Exit: Users can set a “safe word” to exit or pause the experience, prioritizing user comfort and safety.

4. Educational Layers 📚

	•	Historical Insights: Optional deep dives into the history and significance of various music elements for educational purposes.

Setup and Installation

1. Prerequisites

	•	Snap Lens Studio: (Recommended version below 5.2.0 for Spectacles compatibility)
	•	Spectacles Interaction Kit: For co-creation tools
	•	Snap Spectacles Device: For AR testing and deployment

2. Installation

git clone https://github.com/your-username/portals.git

	•	Open Lens Studio and import the project files.
	•	Follow the setup for Snap’s Connected Lenses to sync collaborative features.

3. Running the Project

	•	Use Lens Studio to run and preview the experience on Snap Spectacles.
	•	For live testing, connect your Spectacles device to Lens Studio.

Usage

1. Start a Session

Initiate a cultural or musical session by exploring a location or by selecting a genre from the auditory timeline.

2. Interactive Controls

	•	Use voice commands to navigate, adjust the experience, and activate educational layers.
	•	Control volume, instrument choice, or effects through intuitive commands.

3. Save and Share

Record and save jam sessions or cultural journeys, then share via Snapchat or export.

Target Users

	•	Differently-Abled Individuals: Seeking accessible AR experiences.
	•	Artists, Educators, and Enthusiasts: Interested in cultural contexts and immersive technology.
	•	Music and Storytelling Enthusiasts: Exploring music as a medium for cultural and climate storytelling.

Team

	•	Mike: Project Lead, SnapAR Developer
	•	Sinchana: Cultural Research and Accessibility Advocate
	•	Leila: Audio Design and UX Specialist
	•	Chris: Technical Lead, Lens Studio Integrations

Resources

	•	Snap Spectacles Audio Features
	•	Lens Studio Scripting Guide
	•	Tutorial: Build Your First Spectacles Lens

Future Enhancements

	•	Global Jam Sessions: Connect with users worldwide to create collaborative, real-time musical experiences.
	•	Expanded Instrument Library: Include more underrepresented instruments from various cultures.
	•	Environmental Interactions: Trigger unique sounds and effects based on real-world environmental data (e.g., location, climate).

Acknowledgments

Portals celebrates music, culture, and learning through immersive AR, blending sound and visual interaction to create an unforgettable auditory experience. Special thanks to ChatGPT for providing code assistance during the development of this project.

About

Portals is an MIT-licensed project aimed at enhancing cultural and musical education through augmented reality. For more information, visit our website or contact us at [email protected].

License

This project is licensed under the MIT License. See the LICENSE file for details.

Contributors

	•	Sinchana Nama, Leila Tamale, Christopher Park, Michael Chaves

Languages

	•	JavaScript, Typescript