Publication
Self-organization and emergent behavior lie at the core of many natural phenomena, from pattern formation in biological tissues to the collective motion of flocking birds. Harnessing these principles in computational models can unlock powerful approaches for texture synthesis and interactive simulation. This thesis explores Neural Cellular Automata (NCA), a family of models that integrate the seminal concepts of Cellular Automata and Reaction-Diffusion with modern deep learning techniques. Over four main chapters, we demonstrate how NCAs can be trained to generate complex textures, adapt to diverse supervision signals, achieve continuous space-time dynamics, and operate on diverse spatial topologies.
The background chapter, provides a comprehensive overview of cellular automata and reaction-diffusion systems, laying out their formal definitions and historical significance. We then discuss how NCA inherits the locality and simplicity of these models while leveraging trainable neural network to enable more flexible and robust pattern synthesis. Finally, we highlight the emergent properties and self-organizing nature of NCA.
Our first contribution is built on the observation that standard NCA patterns often exhibit spontaneous motion. By adapting both the model architecture and the training process, we show how to supervise this emergent motion, enabling the synthesis of realistic video textures in real time. Our proposed framework provides interactive control over factors such as motion speed and direction, all while preserving computational efficiency.
Next, we investigate whether standard NCA models actually capture continuous dynamics or merely overfit their training discretization. We find that existing NCAs tend to struggle near the initial seed, overfitting to the training grid and timestep settings. We show that using uniform noise as the initial seed solves this problem, enabling continuous control over both the speed and scale of pattern formation.
Finally, we extend NCAs to the 3D domain by drawing an analogy between spherical harmonics and convolution filters. Our approach eliminates the need for UV mapping and synthesizes seamless, dynamic textures on unseen 3D meshes in real time, while only being trained on a single icosphere mesh. Our proposed model stands as a natural generalization of NCAs to non-grid structures, retaining all the emergent properties and self-organizing behaviors of its 2D counterpart.
By combining the principles of cellular automata and reaction-diffusion systems, that simple local rules governing complex global behaviors, with modern deep learning techniques, NCAs emerge as a robust, controllable, and self-organizing model for pattern formation. Collectively, the contributions in this thesis show how NCAs form a unified framework for texture synthesis across 2D images, video, and 3D meshes. By bridging of ideas from dynamical systems, computer vision, computer graphics, and artificial life, we hope to h