“This major generative, transmodal, and interactive installation for SIGGRAPH 2023 celebrates the 50th conference of this most famous of computer graphics conference series.
By referring to neural nets, artificial intelligence and machine learning, and cognitive neuroscience, the "synaptic concept connects the last fifty years to the next.
Imagining the past, present, and future of the SIGGRAPH archive as a dense evolving network of artificial neurons, dendrites, and synapses, and likening the spread of ideas to branching neural signals firing across countless synaptic connections, flashing like lightning, this interactive, performative, and data-driven work traverses the landmark developments of an ever-emerging ever-expanding field, tracing the propagation of influences and innovations across time, extrapolating from the past fifty years to the future fifty.
Fusing past (five decades of SIGGRAPH archival materials). present (live interactivity, generativity, and performance). and future (Al/ML extrapolations of the past fifty years into the next fifty), this transmodal project explores synaptic interconnections between pioneering individuals, landmark ideas, technical innovations, breakthrough projects, and visionary artworks that carried the computer graphics revolution from the edge of research through the mainstream of global culture and on into the restless invention of the future.
Perforated Systems: In a gesture toward holistic thought about the environment, a multi-layered structure of "perforated systems" creates a self-balancing ecosystem of sights, sounds, and behaviors, in which the overall environment has equal agency. Many authors have provided many layers, all communicating with one another in a model of open but self-balancing collaboration inspired by the biodiversity of rainforests and coral reefs.
Media Arts and Technology Program (MAT): This project originates at the MAT transLAB at UCSB, and is a collaboration between the transLAB and AlloSphere at MAT, the Alice Lab at York University, SBCAST (Santa Barbara Center for Art Science and Technology), Biopac Systems Inc., and the ACM SIGGRAPH Archive team. Many alumni and friends of MAT have also contributed to this work.”
—Marcos Novak, Director of the transLAB
As a member of the transLAB Research Group, I contributed to the transLAB Synaptic Time Tunnel, commissioned for the 50th Anniversary of the annual SIGGRAPH conference. The conception for my contribution, the Synaptic Synth, was born from the installation’s thematic focus: a reflection of the past 50 years and a vision of the next 50 years, emphasizing the non-linearity of complex preceptory systems and of memory itself.
The Synaptic Synth consists of five independent synthesizers, each gauged for a specific task. The parameter-space for the synth network as a whole is cursedly high-dimensional, and thus too complex to be controlled by direct human input. The synth network is then controlled by an accompanying network of independent multilayer perceptrons (a common neural network architecture), each receiving cross-modal input from outside and within the entire perforated system. This approach ensured that no individual network had a complete overview of the entire data set. By doing so, we aimed to mimic the human brain's perceptual system, where stimuli received by one receptor might not only be processed independently but could also intertwine or influence other receptors.
This arrangement allowed for a unique phenomenon: the transformation of cross-modal inputs into transmodal outcomes.
This approach to perforated systems was further refined for the transLAB 2023 Celebration, where I added an autoencoder to the neural network system that performs dimensionality reduction on the totality of the system’s data. After training, the autoencoder was sliced into two networks (i.e., one composed of the input-to-bottleneck layers, the other composed of the bottleneck-to-output layers), forming a kind of “split-brain” that could both produce latent encodings for incoming inputs and also generate “hallucinated” data by navigating the latent dimensions directly.
