Members-Only
Recent Talks & Demos are for members only
You must be an AI Tinkerers active member to view these talks and demos.
Iterated Function Systems Pre-training
This talk demonstrates pre-training neural networks using procedurally generated data from Iterated Function Systems, a technique useful when large real datasets are unavailable.
Neural networks can be pre-trained with procedurally generated data [1][2] and in some cases the performance can outperform pre-training with real data [3]. I’d like to give a demo of generating procedural data with iterated function systems and pre-training Neural Networks with them.
[1] Pre-training without natural images https://arxiv.org/abs/2101.08515
[2] improving fractal Pre-training https://arxiv.org/abs/2110.03091
[3] Visual Atoms: Pre-training Vision Transformers with Sinusoidal Waves https://arxiv.org/abs/2303.01112