GOING DEEP SPATIAL: Our Technical Definition of Experiential Technology
Why define experiential technology?
Over much of the past decade, extended reality technologies using the term “XR” have stood alone as a category that enabled pervasive display-based creation of virtual 3D experiences, as they were put into mass production in the form of AR.
As you are intimately aware, XR overlays virtualized content on the real world, displaying either on a flatscreen device or a head-mounted display - and VR, which creates a fully immersive view and the experience of a full alternate reality. Our event was founded on an enthusiasm for these technologies, and our commitment to them remains steadfast and unshakeable.
With that said…
The recent wave of generative AI technologies washing over the world and transforming our online experiences in new and unexpected ways has caused us as organizers to reflect on the true nature and essence of our status-quo passion and intention in working in the area of XR.
What we found is that there is a broader category of technologies, including but not limited to XR and AI, which more properly captures the technological focus of what we wish to work with as both organizers, hackers and scientists - and we believe the future is in this area.
Experiential technology is an open-ended family of technologies which incorporate:
(a) Sensory input/output interfaces. The system’s input interface affords the user expressiveness by using sensors to capture signals. The system’s output interfaces with the body directly in a way that allows “experiences" to be simulated. These may resemble other physical realities, or may constitute novel or physically impossible experiences.
(b) Content engines. These encompass computational representations of some reality or realities which are used to drive the sensory interfaces to create specific experiences.
Most would agree that head-mounted displays such as VR/AR headsets certainly count as experiential technology.
Let's consider variants of generative AI as experiential technologies. First, consider that plain old digital displays, like the ones we find on our phones and on our walls, are powerful sensory interfaces - which is evidenced by the strong emotional impact of movies.
Images and videos created by generative AI are novel in both quality and quantity - making such a system a genuine content engine. In light of these observations, the combination of digital displays and AI-based image generators constitutes an experiential technology.
If we consider language-based interfaces, words would be a sensory interface. Words have the power to evoke all of the senses in the imagination - in some ways that are inferior to digital displays, and in other ways that go far beyond them. The LLM functions as the content engine, and the combination of text-based input and output combined with an interactive LLM would therefore fit the definition of an experiential technology.
Those were two deliberately simple examples. What about processing chips supporting IoT applications? Or physiological sensors?
We have demonstrated the construction of five systems, all of which satisfy the definition of experiential technology. Therefore, VR/AR headsets, digital displays with generative AI, word-based interfaces with LLMs, chips supporting IoT applications, and HCI sensors are all valid examples of experiential technologies.
MIT Reality Hack: Promoting Inclusive Access to Tools for Creation and Making the World a Better Place Through Experiential Technologies
Our Mission: by Scott Greenwald, PhD
MISSION STATEMENT
Our mission is to propel experiential technology towards a maximally positive role in society, by supporting a creator-oriented ecosystem, and making ideas for good into reality.
Why Experiential? XR and – more broadly – experiential technology is an immensely important category of new and maturing technologies that will play a central role for the human experience in the near and further future.
Minting creators. A new generation of technology enables us to support creators, not just consumers. We pursue this by building an inclusive community focused on collaboration and hands-on learning.
Ideas for good. Prototyping offers a chance to learn and envision a better future. By centering our ideation processes on this concept, we've seen world-changing ideas emerge.
Making it real. Prototypes are a crucial first step in building ventures that positively impact the world. We leverage our events, resources, and network to support the entire venture creation process, turning ideas into reality.
A future that works for everyone. We believe a better future for humanity must include everyone, and this future can only be achieved if all are involved in its creation. As stewards of access to knowledge and resources, we play a crucial role in counteracting systemic biases that could otherwise prevent people from participating in our events and community, ensuring everyone contributes to building this better future.
Our Mission (Extended)
It all begins with an idea.
By Scott Greenwald, PhD
Our mission is to propel experiential technology towards a maximally positive role in society, by supporting a creator-oriented ecosystem, and making ideas for good into reality.
Why Experiential? XR and – more broadly – experiential technology is an immensely important category of new and maturing technologies that will play a central role for the human experience in the near and further future.
Minting creators. A new generation of technology enables us to support creators, not just consumers. We pursue this by building an inclusive community focused on collaboration and hands-on learning.
Ideas for good. Prototyping offers a chance to learn and envision a better future. By centering our ideation processes on this concept, we've seen world-changing ideas emerge.
Making it real. Prototypes are a crucial first step in building ventures that positively impact the world. We leverage our events, resources, and network to support the entire venture creation process, turning ideas into reality.
A future that works for everyone. We believe a better future for humanity must include everyone, and this future can only be achieved if all are involved in its creation. As stewards of access to knowledge and resources, we play a crucial role in counteracting systemic biases that could otherwise prevent people from participating in our events and community, ensuring everyone contributes to building this better future.
What We’ve Set Out to Change
If we look at how mobile internet technologies have developed, from the standpoint of access and economic opportunity, we can see how an ecosystem can be structured to encourage a majority of consumers and a small minority of creators.
Conversely, we believe that creating a vibrant and inclusively beneficial human experience through experiential technology requires applying a paradigm in which everyone is empowered and encouraged to be a creator. The pathway to this form of empowerment is constituted by:
Learning. Learning to use relevant tools and apply relevant concepts, and
Creator enablement network. Gaining access to resources such as a network of potential advisors, collaborators, and sources of funding.
When people are empowered to create in the spirit of making the world a better place, world-changing ideas emerge. It is our mission not only to provide a gateway to the world of creation through experiential technology, but also to provide support for the passage of world-changing ideas from earlier to later stages of development - from conception to impact.
Why Hackathons?
We use the concept of the hackathon– a group event oriented towards creating and/or learning-by-doing over a short period of time–and a collaborative approach to support this passage at many different points, in ways that evolve and grow with a focus on maximizing our impact.
It is important to note that in order to do this, we focus on initiatives that we are uniquely positioned to execute well and it is for this reason that our flagship hackathon format remains our central focus at this time.
A second critical note: we apply the idea of learning-by-doing not only to event participants, but to our organization and its members. That is, many members of our volunteer organization are occupying their role for the first time, effectively learning-by-doing under the guidance of senior members and external mentors. The spirit of collaborative learning is germain to making the world a better place, and this infuses it into everything we do through everyone who is involved.
Team Formation and The Destiny System
It all begins with an idea.
By Scott Greenwald, PhD
Team formation is a universal problem, and we have tried to make this process easier since the early days of the hack.
Last year, we planned to implement the Destiny System for team formation, which in its design incorporated both (i) mechanisms for avoiding “homophily,” where people join together not because of shared interest, but because of superficial shared attributes like ethnicity, gender, or age, and (ii) giving due importance to “vibing” with team members - identifying affinities or “chemistry” by meeting and chatting.
Unfortunately we bit off more than we could chew, and the system was not fully operational on the day of the event.
This year, going into a second version of the Destiny System design, we aim to involve a group of enthusiastic would-be, could-be, past, or future participants to examine the assumptions behind the system, and make sure that the design will hit the mark in the way that we hope.
If you are interested in creating the future of the Destiny System for team formation, please write destiny@mitrealityhack.com to express interest, and stay tuned for further details.
Sincerely,
Scott
Experiential Technology at MIT Reality Hack: A Concept Driving Our Evolution
Experiential Technology at MIT Reality Hack: A Concept Driving Our Evolution
By Scott Greenwald, PhD
Sept. 17, 2024
Starting this year, and moving forward, we use the term experiential technology to capture the focus of our hackathon.
Why define experiential technology?
Over much of the past decade, extended reality technologies have stood alone as a category that enabled pervasive display-based creation of virtual 3D experiences, as they were put into mass production in the form of AR - which overlays virtual content on the real world, displaying either on a flatscreen device or a head-mounted display - and VR, which creates a fully immersive view and experience of an alternate reality. Our event was founded on an enthusiasm for these technologies.
The recent wave of generative AI technologies washing over the world has caused us as organizers to reflect on the true nature and essence of our status quo passion and intention in working in the area of XR. What we found is that there is a broader category of technologies, including but not limited to XR and AI, which more properly captures the technological focus of what we wish to work with.
By Scott Greenwald, PhD
Dear Friends of the Hack,
It is with great anticipation that I am finally able to share the new direction the Hack will be taking this year. This initiative represents an enhancement to our current XR focus, not a replacement, and I am confident that it will inspire as much enthusiasm in you as it has in us.
Going forward, we will adopt a term that extends beyond "XR," signaling an exciting new chapter for the Hack. This shift broadens our scope and positions the Hack to remain dynamic and at the forefront of innovation, as it has been for nearly a decade.
I am very excited to tell you that the term experiential technology will be used moving forward to capture the focus of our hackathon.
What's new in experiential technology at MIT Reality Hack 2025?
To offer a glimpse of how this will shape the Hack in 2025: we are witnessing both growing interest in and new feasibility for generative AI, as well as brain-computer interfaces (BCIs). While previous Hacks incorporated basic BCIs, more advanced metrics were previously cost-prohibitive and time-consuming to develop. This year, we are significantly enhancing the presence of BCIs at the event with the introduction of new devices and sponsors, as this technology has reached the cutting edge and is now ready to play a key role in the 2025 Hack.
Additionally, we will further integrate AI technologies into the event. This includes AI as part of the development process, leveraging large language models (LLMs) and symbolic systems to power interactive systems and agents, utilizing AI to generate spatial and graphical elements, and embedded hardware that enable advanced computation.
In short, we are thrilled about this evolution of the Hack and look forward to bold new examples of why the MIT Reality Hack continues to be a vital force in the community, driving it forward into 2025 and beyond!
Sincerely,
Scott
READ: Our Technical Definition of Experiential Technology