Notes from your future.
Looking back from January 2023, it’s hard to imagine there was a time when the world’s information was flat and lifeless, just sitting there, waiting to be understood.
Five event-filled years have passed since 2018, when Artificial Intelligence made its way into our everyday lives via useful Siri suggestions, Augmented Reality really hit the mainstream through our smartphone cameras, and people saw the true potential of Spatial Computing when cheap headsets became untethered.
Back then, an immeasurable number of text-based documents and images represented the sum total of civilization’s knowledge and communication. Simply pixels on an electronic screen, all of our digital documents, books, mails, reports, data, songs, videos, and artwork just sat there, doing absolutely nothing. Why? At that point in time, our digital artifacts didn’t know how to convey their own meaning, had no contextual awareness whatsoever, and were even missing the slightest bit of onboard intelligence. Remarkable given today’s world of Smart Information, but back then it was nothing but unresponsive pixels.
Mountains of inert data and information kept growing, every day, everywhere. But more concerning than the raw pixel tonnage piling up was the fact that civilization’s combined knowledge and wisdom was held captive within these inflexible digital prison cells. Even the best of our digital authoring tools kept creating relatively primitive representations of our important thoughts within “dumb” containers. Authoring anything more than a static container was too expensive timewise given the amount of effort required. Because content didn’t ordinarily need to adapt itself to different forms to convey meaning, there was no automated way to create this capability back then.
The enlightened among us realized that for content’s true value to be unleashed from within the containers of its original form, we needed a way to automate the immense amount of manual creation tasks required to create different representations. That laborious process alone kept this from happening for decades. Surprisingly, it seemed no one thought to question why we were fine with it for so long. But the truth is we didn’t have all the right pieces in place yet. Until one day we finally realized what was possible by combining the emerging fields of AI and Spatial Computing — it was a bold experiment that could completely change the nature of communication forever.
By tackling the challenges of existing information artifacts with newly envisioned data containers, we could overcome its biggest limitations — (1) they were inert islands trapped within our most common digital artifacts, (2) they couldn’t tell their own stories, (3) had no contextual or environmental awareness to adjust for relevance, and (4) required a dreadful manual creation process to produce custom forms for use in any channel other than the original.
With those challenges in mind, we set off to build the future. And just like that, almost overnight, we started to create a new type of “smart” information that could tell its own contextually-aware story, surface the invisible insights while conveying meaning, and make the non-obvious connections for people in ways that its digital predecessors or analog artifacts never could. Not a bad start, and spelled doom for all those dead pixels.
What began as a small set of experiments to leverage AI and Spatial Computing to power next-gen information experiences soon ignited the worldwide revolution in data design. It almost immediately shifted the way we were thinking about designing information and expanded our expectations of what was possible. We started using Machine Learning to identify data insights and Augmented Reality to surface meaning in new ways. It felt back then like a unique new communication language was born. The possibilities were limitless. We wasted no time.
Given the perspective of looking back at all this from 2023, the speed at which the transformation happened was startling. We saw our documents and photos come to life seemingly overnight, tweets instantly organized and amplified, and our most useful educational materials become more understandable at 10x the speed of other major technical revolutions. People felt smarter and more empowered without having to make any major changes to their daily habits or learn any new skills. The things we consumed every day became clearer, seemingly on their own. Responding to anything became a breeze thanks to the active assistance (first seen dramatically in Gmail finishing your sentences as you typed). It felt like mindreading had finally been perfected. Perhaps older generations would have found it unsettling and been concerned, but we embraced it because it came without effort and felt natural.
Once Designers learned of the amazing new palette they could create experiences with, there was no stopping them. Artists combined forces with Data Scientists, Developers with Marketers, and Cognitive Scientists with Evangelists to work on what felt like a new way of delivering impact and meaning. This explosion of creativity, engineering, and human ingenuity must have felt good at the time, but in a larger historical perspective it resembled a mini-Renaissance period.
And just like that, in the blink of an eye, we all were living in The Age of Smart Information.
From a historical technology perspective, Smart Information wasn’t a brand new innovation with no prior art. It was the mashup of many existing technologies and approaches into a new object-oriented framework that did have some new-to-world capabilities. The presentation layer was put together with everything from simple HTML and web-based GUI libraries, to advanced XR rendering and display tech. The middleware was nothing out of the ordinary — the same services that ran large content-based operations. And the backend leveraged much of the work being done with AI in the form of Machine Learning analyzing patterns of consumption over time, and Reinforcement Learning constantly adjusting the experience to gain insight to predict future behavior. All that said, when combined together in clever new ways, these existing solutions did create a truly breakthrough experience.
Even now, it’s hard to quantify the overall impact that the Smart Information revolution had on society. Akin to the invention of moving pictures in the late 1800’s, this technological and cultural transformation become so widespread and embedded in the fabric of life, that it would have been impossible to understand it’s eventual magnitude while it was unfolding.
You can, however, easily observe the outcome — it literally changed everything about how we author and communicate. We moved quickly from a world of static artifacts and labor-intensive tasks to almost everything having the capability to tell its own story, autonomously, whether purely digital or attached to analog objects. And better yet, we didn’t have to manually create all of it ourselves, every time.
When we learned how to leverage the power and speed of AI to assist us in creating multiple forms of the same message from one source, surface previously hidden insights, and do the heavy lifting of data analysis on our behalf, it was game on. Innovators went into overdrive. They quickly created and iteratively refined experiencing our information in multiple dimensions to better understand its intended meaning and take the appropriate action. The combination was stunning in its efficiency and beautiful to behold. Information was transformed from its historically lifeless forms into a new kind of living container.
Along the way to today’s vibrant world of intelligent content, there were four major breakthroughs that jumpstarted and catalyzed the Smart Information revolution. Each provided a key part of the puzzle and formed the platform’s base.
Contextual awareness was a foundational element of those early experiences. It was obvious to the Smart Information pioneers that one of the most valuable things we could change about the existing content consumption experience was doing a better job of presenting things in the most appropriate form for the given situation. More than simply reflowing a web page for a particular device’s screen size, this contextual awareness effort looked at where people were, what time of day it was, current activities, historical behavior, and the type of devices they had access to at the moment. Constructing a more complete picture of the current context, coupled with the ability to find or generate the most logically appropriate form of the message, spurred the development of a whole new delivery and consumption platform. It could enable information to feel more appropriate all the time.
Surfacing the invisible was the catchphrase used to describe showing previously hidden (or omitted) detail and insights in everything from throwaway tweets to our most heralded tomes. It also became a key pillar of the Smart Information movement. Showing what existed just below the surface or in the background was a true revelation to most people. We inherently know all of these things exist, but it was shocking to see how much people loved discovering these buried treasures when presented in this new way. No longer would the intended meaning of something remain obscure or exist only in the ether for the insightful — everyday people could now access all of it without much effort.
Most content started to be constructed in ways that made it easy to surface those invisible aspects in truly compelling ways without the reader or audience even needing to ask (if appropriate for the given circumstances). That’s where the advanced AI came in — understanding what needed to be presented to give people the best chance of “getting it”. That applied to everything from individual words all the way up to entire works.
The moment of clarity each of us experience when we encounter new ideas and concepts is a truly magical time. It’s that exact second the proverbial light bulb goes on. We suddenly have a clear understanding of the idea or concept we’ve just been exposed to. Wow! Our brains seem to light up and our ego inflates just a bit as we’re quite pleased to have made the connection. For millennia, creators of all sorts could never accurately predict when the moment of clarity would come for most people. If we cared to try, we guessed, and steered our work to lead to it. That’s never an easy task — making the required connections, as all of us are so incredibly diverse in our thought processes, it’s a small miracle we get people there at all.
To that end, one of the most important leaps forward in the pursuit of clarity was not only the ability to communicate at the right level of detail to be understood by a particular individual, but also learning over time what they responded to. By leveraging AI’s Reinforcement Learning techniques to glean what particular aspects of information a person has responded well to in the past opened up a whole new approach to authoring for clarity and provided a playbook that worked. Once we became proficient at engineering the moment of clarity for people, amplified by our evolving toolsets, things just felt like they snapped into place. We got it. They got it. Everything was clicking.
Ambient intelligence came to be the industry term that we used to describe how our physical environments would light up to display the right info at the right time. Made it feel like the world around us was recognizing and understanding what we did and said. That was made possible thanks to AI being embedded into the very fabric of every device and surface we came into contact with in our travels. Although it’s true that AI powered it all behind-the-scenes, the real star of that experience was the form of information that appeared — whether it was as a 3D holographic traffic indicator floating above your automobile’s dashboard or a warning indicator projected onto a hot surface that appeared just as you reached anywhere near it. Our spaces were alive and paying attention to us in ways we never thought of.
Even from the earliest days of IoT and consumer-grade sensors like the Xbox Kinect, it sure felt like our future would involve interacting with “things” more often. The rise of Intelligent Assistants like Siri and Alexa cemented that into our daily routines. And now, our toasters ask us if we really meant to burn the bagel before continuing on. Our essays for class started writing themselves when we stalled out. The world seemed to know just what we needed and quietly started providing it without us needing to ask. Smart Information was the visual, auditory, and tactile layer underlying how we communicated.
Given our penchant for machine-like efficiency and productivity when creating or consuming data, it’s remarkable that emotion became the center of gravity for this new design direction. Moving past efficiency to impact, the way it felt when interacting with things dominated the industry conversation. It was a big change that warranted much discussion and analysis within multiple industries.
The following aspects of the overall Smart Information experience came out of those discussions and experiments with emotion, which helped us move beyond the rudimentary layout and interaction models of the past onto dynamic, living pieces that made a real connection with people.
Design thinking was critical in making this leap forward. Design has always been just as transformational as technology in these types of leaps, but it’s harder to spot in action. We know when something is suddenly possible (technology), yet the way it behaves, looks, feels, and acts (design) is somehow less interesting or apparent to the average person (unless it reduces friction or pain). That situation reversed itself quickly during the early days of Smart Information. Great visual and interaction design were clearly evident and noticed right away. The way these digital artifacts appeared and felt when we interacted with them was unmistakably new. It felt good. The technology was unimportant compared to the engaging design.
Suddenly, our words, images, sounds, and physical attributes went from unremarkable in their normal resting state within information instances, to embodying unmistakable energy and demanding attention. The design also leveraged a whole new gestural language and interaction model for active communication and deep exploration. Previously activated only by our intellect and imagination, highly designed narratives and revelations began to flow from our most common digital artifacts. The future of information became a hotspot of activity and reward for creators and consumers.
Behavior is recognized as being primarily responsible for whether we deeply love or absolutely can’t stand something from an experience standpoint. Visuals and first impressions will always play a huge role in satisfaction, but an object’s behavior determines our willingness to further engage or immediately bail. It is the small things in how something acts that sets them apart.
From a behavioral design standpoint, the hallmark of the Smart Information experience was its uncanny ability to communicate effortlessly at the right altitude or level of detail for a given situation. For any given audience, that behavior is what endeared them to the medium. Hands down. Whatever was being created or consumed felt dialed in for that exact moment and situation. Add to that the ability to dynamically adapt to changing conditions and circumstances, and you have a predictable pattern of behavior that’s pleasing to your audience. People ate it up. There was no going back.
Intelligence was never thought of as a property of information itself before this revolution. In fact, that would have seemed quite odd given our focus historically on integrating intelligence into our systems, not their output. Yet, much of what we consider to be normal or expected intelligence now resides within the artifact or object itself, not the system delivering it (even though it does exist within both many times). That shift was not possible before our services, systems, and toolsets were updated to use Artificial Intelligence and Machine Learning models to extract and amplify the underlying structure and value of the content or data. “Intelligence On Board” was the bumper sticker du jour.
The tasks required to identify insights and create clarity ran like invisible engines in the background of any active encounter with information. Further enhancing that feeling of intelligence on board was the use of AI’s Reinforcement Learning techniques to track what we responded to during sessions. These incredible feats of “intelligence” were regularly accomplished by processing real-time input streams. It felt nothing short of miraculous compared to the manual number crunching and analysis tasks of the first Productivity Age.
Ethics is perhaps the most overlooked and misunderstood part of the Smart Information universe. One of the biggest issues that came along with Artificial Intelligence being embedded within our common information was the notion that our systems needed to have integrity, a conscience, and moral compass. We are a fair and just society in principle, so why wouldn’t we strive to model our AI systems in that image? We did try our best to be ethical in code, yet unconscious bias permeated the systems and algorithms we created since the dawn of the computer. We taught our models poorly at first, but eventually recognized how to identify and remove bias to form a more transparent and solid ethical base to operate from. We eventually let the content make decisions on a local basis, informed by a centralized moral compass. Worked out well for most cases, but it remains an ongoing issue.
This same “local decisions informed by a central policy” approach was also used for privacy, security, and safety issues being handled by the Smart Information delivery and consumption systems. The resulting distributed code of conduct faced some difficult challenges when trying to determine how information should conduct itself. It also needed to deal with how to treat people properly given the current context and environment. No easy answers there, but we are evolving our systems to be more empathetic and less rules based.
From a system perspective, Artificial Intelligence and Mixed Reality were the two key system elements that fueled the initial growth of Smart Information implementations. Both fields of study and implementations were on fire before any of this took off, and once combined, they turned this movement into something completely different. This was the hottest place to work for awhile. Many critical system parts still needed to be envisioned, forged, and deployed one-by-one — but the evidence of major impact those had was easily apparent from the earliest experiments. Information systems were cast in a completely new light (more complementary I must say). The result was breathtaking and humbling. A new kind of reality become visible to all of us, almost at once.
In the beginning, small teams of brilliant individuals and straight up hackers created the first (relatively) crude tools and platforms to test out system-level hacks. Far from revolution igniting work, these were just clever experiments to test proof of concept pieces. That said, they immediately showed the potential leap forward we were all about to make. And little did we know it would go worldwide almost in an instant, and end up being completely transformative.
Constructs are the invisible building blocks of our digital systems that hold everything together behind-the-scenes. To system architects and programmers, these constructs are elegant packaging for organized thought. You can see their eyes light up when describing them in hushed tones and reverence. Representing structured thoughts that have become codified, constructs hold the individual parts of our Smart Information objects together and allow them to operate with real efficiency.
For the rest of us, constructs are invisible, and assuredly boring detail of a technology solution that seems mind-bogglingly complex. That said, we all owe many of the giant leaps forward we’ve made in communicating to the forethought and beauty put into those clever constructs. They are in many respects the key reason we were able to advance past the lifeless containers and frameworks of the past. Structure matters.
Technologies that enabled the rise of Smart Information were developed for very different purposes before combining to power this revolution.
The foundational element of most business and educational communication is some form of written language. The words themselves were merely ASCII characters stored in an electronic file prior to the advent of Smart Information. The key constructs developed during this period such as Information Objects quickly evolved into the multi-dimensional, connected, intelligent containers that power most of the world’s communication today in 2023. Interoperable, componentized, and modular, words are now far more than simple marks on a page thanks to the constructs that were devised to enable stunning representations of their meaning, connections, relationships, and levels of detail.
Artificial Intelligence gave us the ability to process data, look for patterns and trends, and monitor systems on our behalf. The aspect of AI that really fueled the growth of Smart Information was the ability to better understand how people wanted to be communicated to for a given situation. That contextual relevance was missing in most common forms of interaction. We were very one dimensional in our approach prior to this, no pun intended.
Augmented Reality and its related Spatial technologies provided us a way to see the invisible elements of our information and add additional data to objects in the real world. Although AR was not required for many levels of detail to function, the ability to use audio, video, tactile, and dimensional technologies to communicate things in multiple ways at once ignited a goldrush to find and monetize efficient ways to combine these disparate solutions into compact and efficient delivery vehicles.
Cloud Computing became indispensable initially by running the services that allowed us to store and share our things with each other, regardless of what type of device were using from anywhere in the world. Over time, it became apparent that we needed to add spatial permanence and landmarks to our information, so we could understand better where things came from and belonged. That prompted the advent of the AR Cloud to not only store locational aspects, but also do the computation and processing of multi-level representations of our information. We could not have the interconnectedness, relationships, and interoperability of Smart Information without Cloud Computing.
Platforms always serve as the underpinnings of our systems and services. They make rapid scalability and extreme growth possible by providing a stable foundation for developers and architects to build upon. In the case of Smart Information, the key platform work that was done enabled both a consistent frontend experience (Cloud-based presentation and interaction processing) and powerful backend processing (Cloud-based AI and ML) to leverage tons of existing work.
All the same behaviors and computing could be done locally, but using a ubiquitous online platform opened up the possibilities for everyone, not just those who installed a particular set of software. Platform as a Service (PaaS) was key in the rapid adoption of Smart Information solutions by content providers and publishers.
Tools were also very integral to the growth and adoption of this movement. Our traditional toolsets rapidly adopted the key technologies needed for Smart Information and built upon the platforms we developed along the way to provide a seamless experience for both authors and consumers. It was way smoother in many regards than any previous disruptive change to the computing landscape (perhaps we were getting pretty good at disruption by then). We seemed to have learned our lesson about creating brand new approaches vs. leveraging existing, well-understood methods. Smart Information rode in on the latter. We got the tooling part of the equation right because we built in the ability for information to appear in multiple forms within the playback platform, not requiring anything new or different.
The most significant shift in our use of tooling for Smart Information was the move away from thinking we needed to manually create every representation ourselves — the machine could now be trusted to do a good job of most operations. For decades, we always assumed if something needed to exist, we’d end up creating it ourselves. No more. Our tools could now take care of generating multiple high-quality representations of the same object automatically. It was done surprisingly well, in fact. Audio was generated by text-to-speech synthesizers using human voice (your own if you cared to train the tooling just a bit by reading a paragraph out loud and creating a “voice font”). Various visual representations were auto-generated using AI-powered design agents. Interaction models were standardized at the platform level and added automatically by the toolsets you were using to create the original version of something. Content even appeared in fully translated forms depending on the location it was consumed in. Imagine any tweet you wrote being available in dozens of languages without you knowing how to speak or read any of them. Tools were no longer thought of as something experts had to fly, but rather an approachable interaction with a trusted friend, albeit a know-it-all.
From Silicon Valley to New York City, Moscow to Hyderabad, London to Beijing, eventually every corner of the world will experience that deep emotional connection to their content that the Smart Information revolution enabled. Even traditionally dry topics will come to life in amazing new ways. The familiar will be experienced in the most appropriate way to make that connection with the person viewing or creating it. One by one, every new piece of information will become available in these new ways — and we will generally think nothing of it. In the future, as always, our children will assume it had always existed this way, because honestly, why wouldn’t it?
The Smart Information revolution will continue on in new and completely unexpected ways, which will unquestionably lead to some major societal transitions for all of us and our descendants.
Smart Data will be the next logical extension of Smart Information. Recognizing the vast potential for adding value to data itself, we will drive more context and relational information into the exabytes of discreet datapoints we collect every second of every day. This will in turn cause an exponential increase in the amount of storage, compute power, and bandwidth required to keep pace with our expanding appetite. Just when we thought the Cloud would save us, it will become clear we bet our future on something we can easily cripple through our own complacency and ambition.
Smarter People are a natural outcome of the world’s information being available in any form we desire to experience it in. Conversations get more insightful, and less superficial. Understanding goes deeper. Communication improves across the board, regardless of personality or ability to be articulate. As a species, we will further develop the ability to parallel process multiple streams of input and understand them concurrently (as we do already with more visceral input). This ability was something the teenagers of the world seemed to master a decade earlier, but it wasn’t true. They were multi-tasking, not processing multiple sources with equal quality across them — that part is new. As always, we’ll find ways to undermine our new abilities as our animal instincts stay intact, unfazed by the ability to generate and understand higher level discourse. We are humans after all.
Smartest World is the term used to talk about the explosion of ambient computing and environmental awareness woven into the spaces we inhabit. It won’t be long before children are born who didn’t realize there was a time when you couldn’t talk with anything and everything, and expect an appropriate response. Life is recorded, rewindable, and able to be summoned at will. Smart Information fuels the way we experience that world, so we need to be careful to avoid all of the dystopian info overload clichés we envisioned when we first realized the experiential vector we were headed down. Human ingenuity, morals, and love will triumph over the soulless optimization of our technology-powered assistants / guardians, creating the best world we can hope for given our human nature.
With all of that as the setup, this book takes you on a quick trip through all the topics we just touched that contributed to igniting the Smart Information revolution. You’ll see how this foundational transformation resulted in a giant leap forward in our ability to communicate and be understood.
My hope is that by talking through all this with you, we can clearly see where this work will ultimately lead us to, together.
Copyright © 2019 Mike Pell – Futuristic Design, Inc. All rights reserved
BOOK AVAILABLE NOW FROM
Learn more about the author M. Pell at Futuristic.com
CONNECT WITH M. PELL
Copyright © 2019 Mike Pell – Futuristic Design, Inc. All rights reserved