AI for Data Art & Data Visualization: From Model to Canvas

Artificial Intelligence (AI) is increasingly shaping various fields - data art and data visualization being no exceptions. Beyond automating analysis, AI is also becoming a tool for creative expression, blurring the lines between computation and design. As an AI engineer passionate about artistic data visualizations, I love exploring how AI algorithms and architectures can be used as creative tools to generate unique visual representations of data.

In this blog post, I invite you to explore examples of artistic pieces that demonstrate the fascinating intersection of AI and visual creativity.

Figure 1) Various AI algorithms and model architectures can be used for Data Visualization and Data Art (AI-generated, Adobe Firefly).

Nested subsets of AI

Let's begin by outlining the definitions that constitute the overarching field of AI, providing a framework for further exploration.

Artificial Intelligence (AI) is a field of computer science focused on creating systems that mimic human cognitive functions like learning, reasoning, and perception [1].

A subdiscipline within AI is Machine Learning (ML), dedicated to developing statistical algorithms capable of learning from data and generalizing to unseen examples, enabling computers to perform tasks without explicit programming [2].

A subset of ML is Deep Learning (DL), utilizing multilayered neural networks inspired by the human brain's neural structure. DL models excel at extracting complex patterns and representations from large datasets, enabling them to achieve state-of-the-art performance in tasks like image recognition, natural language processing, and speech recognition [3].

Within DL, Generative AI (GenAI) encompasses models learning patterns and structures from data during training and creating original content afterword based on the user’s input, which often has the form of a natural language prompt [4].

Figure 2) Nested subsets of Artificial Intelligence (AI): Machine Learning (ML), Deep Learning (DL), and Generative AI (Gen AI) (adapted from [5]).

Bidirectional Influences between Machine Learning and Data Art &Visualization

Data visualization accompanies the development, evaluation, and monitoring of ML models. On the other hand, ML significantly enhances data visualization. For example, dimensionality reduction can reduce visual clutter and facilitate the identification of key patterns. Beyond visualization, ML algorithms and models can serve as creative tools for data art. The following sections will describe these tools, along with detailed examples of artistic data pieces.

Figure 3) Mutual Shaping – Data visualization enhances ML model development, while ML unlocks new artistic possibilities in data-driven art."

NOTE: As discussed in my previous blog post, 'Data Art vs. Data Visualization: A Deep Dive', the distinction between these fields is often blurred. Therefore, this article will consider them collectively.

Machine Learning Algorithms and Models for Data Art & Visualization

Machine Learning subset: Dimensionality reduction

Within the machine learning toolkit, dimensionality reduction plays a crucial role in visualizing and processing complex datasets, transforming high-dimensional data into a more manageable representation (usually 2D or 3D), which preserves essential patterns and relationships [6]. This lower-dimensional representation, often referred to as latent space, captures the underlying structure of the data, allowing for more straightforward analysis and interpretation. Among the most widely used algorithms are principal component analysis (PCA), t-distributed stochastic neighbor embedding (t-SNE), and uniform manifold approximation and projection (UMAP).

While PCA prioritizes the preservation of the global dataset’s structure, t-SNE is particularly good at showing nearby data points clustered together, emphasizing local structure. UMAP provides a middle ground, capturing local clusters and the broader relationships between distant groups [7]. All methods facilitate the identification of clusters and patterns that might otherwise be obscured in high-dimensional data. If you are interested in learning more about these algorithms, there is an excellent Statquest video by Josh Starmer about PCA [8], as well as articles from Google researchers with great visualizations covering t-SNE [9] and UMAP [10].

Figure 4) Dimensionality reduction methods help uncover patterns and clusters within complex datasets (AI-generated images, Adobe Firefly).

Building on the power of dimensionality reduction, researchers at Google’s Arts & Culture Experiments used t-SNE to map thousands of artworks based purely on visual similarity. The result is a stunning, interactive 3D landscape with visually alike pieces clustering together for an intuitive, immersive experience directly in the browser for anyone to explore [11]. The ability to visualize complex relationships with t-SNE extends beyond visual data, as demonstrated in another Google AI Experiment: ‘Bird Sounds’. Here, t-SNE creates a sonic landscape, clustering similar bird calls together, allowing users to explore the nuanced world of avian vocalizations through an intuitive, browser-based application [12].

Notably, UMAP's ability to capture both local and global relationships was crucial in Nadieh Brenner’s 'Small World of Words' piece, enabling the creation of a visually balanced 2D map of word associations [13]. Another compelling example of dimensionality reduction in action is Music Galaxy by Casey Primozic. This interactive 3D visualization maps the connections between over 70,000 music artists and groups based on audience listening patterns [14]. The project uses the node2vec framework [15] to generate embeddings from Spotify’s artist relationship network, followed by PCA to reduce their dimensionality, revealing a structured musical landscape for exploration [16].

Deep Learning Subset: Deep Neural Networks

Leveraging their multi-layered architecture, deep neural networks (DNNs) excel at extracting complex patterns from large datasets. Convolutional neural networks (CNNs) [17], a specialized form of DNN, excel at image classification and object detection, powering the creation of rich visualizations and artworks.

Figure 5) Schematic representation of a Convolutional Neural Network (CNN) architecture for image classification. Convolutional layers extract spatial features by applying filters to detect patterns such as edges, textures, and shapes. Pooling layers reduce dimensionality, enhancing computational efficiency while preserving essential information. Fully connected layers integrate extracted features to make final classifications (AI-generated image of the robot, Adobe Firefly).

An excellent example of DNNs artistic use is ‘CUBE: Fashion Takes Shape’ by Kirell Benzi in collaboration with Google, interactively visualizing the vast online landscape of over 1200 leading luxury and fashion brands [18]. For this data art piece, over 10 million data points from Google search results concerning the brands were gathered and processed by four DNNs for natural language processing (NLP). These DNNs performed sequential tasks: language detection, translation, sentiment analysis, and topic classification, transforming Google search results for a given brand into a unique visual signature [19].

Variable Studio's project ‘See with Optimism’ for Chloé eyewear demonstrates the power of DNNs in creating immersive customer experiences. The project utilizes TensorFlow's face landmark detection, powered by CNNs [20], to generate dynamic, real-time portraits. The moment a customer picks glasses, a mesmerizing real-time artistic picture of their face with the selected glasses appears behind a see-through mirror [21].

Generative AI subset: Generative Adversarial Networks (GANs)

Generative Adversarial Networks (GANs) are a class of deep learning models consisting of two neural networks—a generator and a discriminator—that compete against each other to create highly realistic synthetic data. The generator tries to produce fake data that mimics actual samples, while the discriminator attempts to distinguish between real and generated data, improving both networks through continuous adversarial training [22].

The generator operates within a latent space, a structured representation of the data the GAN seeks to generate. Similar to its role in dimensionality reduction, this space provides an organized way to model variations in the data. Rather than producing outputs randomly, the generator selects points from the latent space and transforms them into synthetic samples. The discriminator then evaluates these samples, providing feedback that helps the generator refine its mapping process. The GAN improves over time through this iterative feedback loop, enabling the generator to create increasingly realistic and diverse synthetic data.

Figure 6) Conceptual illustration of Generative Adversarial Network (GAN). A GAN consists of two competing neural networks: the generator, which creates synthetic data from random noise, and the discriminator, which distinguishes between real and fake data. The generator improves its outputs through iterative training until they become indistinguishable from real data. (Image sources: Adobe Firefly & Unsplash T. Turner, L. Adai, D. Jovic).

Many artworks created by Refik Anadol’s studio are built around GAN models. Each piece begins with carefully selected data for training. For ‘Machine Hallucinations — Nature Dreams’ [23], the GAN model was trained on 300 million nature photographs. By sampling from the latent space, the trained model can ‘hallucinate’ entirely new shapes, textures, and patterns inspired by nature. Refik Anadol pushed this concept even further by expanding the latent space into a quantum hyperspace, leveraging the power of quantum computing. This advancement enables a deeper exploration of intricate relationships within the data and generates visual outputs that would be impossible with classical computing alone — pushing the boundaries of digital art.

In ‘Unsupervised — Machine Hallucinations’  [24], Refik Anadol’s studio explored a different dataset, this time sourced from The Museum of Modern Art (MoMA) collection. They gathered metadata from 138,151 artworks and trained a GAN model—specifically StyleGAN2 with adaptive discriminator augmentation (ADA) [25]—to generate new artwork. A custom-built Latent Space Browser is utilized to continuously navigate the model’s latent space, producing an evolving stream of newly generated images.

In both projects, the generated visuals are further enriched with fluid dynamics algorithms, creating a mesmerizing effect of continuous, dreamlike motion—an ever-evolving AI imagination in action.

Generative AI subset: Diffusion models

Diffusion models are generative AI models that create high-quality synthetic data by gradually refining noise into structured outputs. Unlike GANs, which use adversarial training, diffusion models operate through a step-by-step denoising process inspired by physical diffusion. The model first corrupts real data by adding noise in multiple steps, then learns to reverse this process, progressively reconstructing the original structure [26]. This approach enables diffusion models to generate highly detailed and diverse outputs, making them particularly effective in text-to-image applications such as OpenAI’s DALL·E 3 [27] and Stability AI’s Stable Diffusion [28].

Figure 7) Conceptual illustration of a diffusion model. Diffusion models generate high-quality synthetic data by learning to reverse a gradual noise addition process. The forward diffusion process progressively adds noise to real data, transforming it into pure noise. The reverse diffusion process, guided by a denoising U-Net, reconstructs realistic data by iteratively removing noise (On the basis of schema by Steins@Medium, AI-generated image of the robot, Adobe Firefly).

A striking example of diffusion models in data art is Refik Anadol Studio’s project ‘Large Language Model — Living Art’, which explores AI-generated representations of Earth’s ecosystems [29]. For this artwork, Nvidia/Getty Images' foundational diffusion model was fine-tuned using a vast scientific archive containing half a billion images of flora, fauna, and fungi species. The trained model generates new digital portrayals of nature, offering an AI-driven reimagination of the planet’s biodiversity. Additionally, fluid dynamics algorithms further enhance the visuals, creating a stunning interplay of organic motion and synthetic creativity.

Machine Learning Tools for Data Visualization Designers and Artists

Figure 8) ‘To code or not to code’—a moment of reflection for data visualization designers and artists. This image symbolizes the choice between utilizing no-code platforms for quick results or embracing coding to unlock deeper possibilities with machine learning (AI-generated, Adobe Firefly.

No-code Tools

While the pieces presented in this article were created using code, you don’t need to code to start using AI creatively for your work. No-code tools are making AI experimentation accessible to all. For example, Orange offers a drag-and-drop interface for building data analysis workflows, enabling users to perform preprocessing, visualization, dimensionality reduction (PCA, t-SNE), clustering, and even simple neural network training —all by connecting modular components [30]. Similarly, ComfyUI simplifies generative AI model workflows through a node-based interface, allowing artists to experiment with open-source models and fine-tune them with their own data without writing code [31]. Platforms like RunComfy [32] extend this accessibility further by providing cloud-based access to ComfyUI, streamlining the integration of AI into creative projects. This topic will be explored in more depth in a future blog post—subscribe to get notified!

Coding Tools

While no-code tools provide a great starting point, you might eventually seek more granular control over your AI workflows. In such case, Python is an ideal solution—one of the easiest programming languages to learn and widely used in AI development. Python offers a range of powerful libraries, such as pandas [33] for data preparation, Scikit-learn for dimensionality reduction algorithms like PCA [34] and t-SNE [35], as well as Matplotlib [36] and Seaborn [37] for data visualization. For those interested in deep learning, libraries like Keras [38], TensorFlow [39], and PyTorch [40] provide tools for building and training custom models with just a few lines of code. While Python is a dominant choice, other languages like R, Java, Julia, Scala, C++, and JavaScript also offer ML capabilities, allowing for a diverse approach to integrating AI into data-driven art and visualization.

import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
import tensorflow as tf

My self-learning journey into AI engineering began with exploring Python, data science, and machine learning through DataCamp [41], where I built a strong foundation in programming concepts. To deepen my expertise, I pursued the AI Engineering Professional Certificate on Coursera [42], which solidified my understanding of AI’s core principles and practical applications. Along the way, ‘Hands-On Machine Learning with Scikit-Learn, Keras, and TensorFlow’ by A. Géron [43] proved to be an invaluable resource, helping me structure my knowledge with both theoretical insights and hands-on practice. These experiences equipped me with the skills to experiment with AI in creative data visualization.

This topic will be explored in more depth in a future blog post, where we’ll dive deeper into the specific coding tools and techniques for integrating AI into data visualization projects. Subscribe to stay updated and get notified when the post goes live!

The reverse influence: Data Visualization & Art for Machine Learning

As mentioned, data visualization is integral to every stage of a machine learning model's lifecycle. In my work as an AI Researcher, I coded custom visualizations for tasks ranging from understanding data distribution and biases during exploratory data analysis (EDA) through monitoring training progress to evaluating model performance and gaining insights into its decision-making [44], [45], [46], [47]. This practice extends to production, where data visualization is crucial for tracking key metrics and identifying potential issues before they impact users [48].

Furthermore, the power of visual representation extends beyond purely technical applications. As Google DeepMind’s Visualizing AI program demonstrated, art can be a compelling tool for popularizing AI concepts [49]. For instance, Jesper Lindborg's video, created for Google, explores the potential of AI to solve fundamental problems and unlock broader solutions. The structure unfolding in this video closely resembles a dendrogram, commonly used to illustrate hierarchical clustering in ML [50].

Figure 9) Art for Machine Learning: Jesper Lindborg's mesmerizing video for Google visually explores AI's potential to solve fundamental problems.

Ethical Considerations

As machine learning becomes an integral part of data art and visualization, ethical considerations must be at the forefront of AI-driven creativity. The foundation of any AI model lies in its training data, making it crucial to use ethically sourced datasets that respect privacy, consent, and intellectual property rights. Unauthorized use of personal data, biased datasets, or scraped content from artists without permission can lead to ethical and legal challenges. Transparency of AI workflows, from dataset provenance to model architecture, ensures accountability and allows users to make informed decisions about the technology they engage with.  Additionally, the environmental impact of AI cannot be ignored—training large models and running inference on GPUs consume vast amounts of energy, contributing to carbon emissions. As AI becomes a more prominent creative tool, balancing innovation with ethical responsibility is essential for fostering a fair, sustainable, and transparent AI ecosystem.

Figure 10) Creative AI: where potential needs to meet responsibility and sustainability (AI-generated, Adobe Firefly).

The Future

As AI continues to evolve, its role in data art and visualization will only expand, unlocking new creative possibilities while posing fresh ethical and technical challenges. Emerging models will likely become more efficient, requiring less computational power while offering greater customization and control to artists and designers. Integrating AI with real-time interactivity, augmented reality (AR), and virtual reality (VR) could transform how we experience data-driven art, making it more immersive and dynamic. At the same time, the need for ethical AI practices, transparency, and sustainable computing will shape how these tools are developed and used.

The future of AI in data art & visualization is just beginning—be part of the journey! Subscribe to stay informed about emerging trends and breakthroughs.