MayaFlux 0.3: The Shape of the Material
MayaFlux 0.3 is the first public release to cover the full stack. Audio infrastructure, offline compute, spatial entities, mesh as data, text rendering, GPU readback, networking.
Not as separate modules that communicate across boundaries but as one numerical substrate that happens to have multiple output modalities.
The demos in this document are not feature showcases in the traditional sense. Each one was designed to demonstrate a specific consequence of removing a categorical assumption that every prior tool has kept. What happens when a mesh is just two spans of bytes with a layout. What happens when an entity does not belong to a domain. What happens when a rendered surface is addressable data. What happens when granular analysis is a function you write rather than a parameter you tune.
Some of these things exist in other tools in limited forms. None of them exist together, in the same substrate, composable with each other, in a system you can extend at the source level in live C++23.
MeshNetwork : Shared State, Closed Loop
A mesh loaded from a file. Waveguide string voices, one per submesh slot. Not mapped. Not synchronised. The same state object is the audio and the geometry simultaneously
Code
auto window = MayaFlux::create_window({ "mesh network", 1920, 1080 });
window->show();
auto net = vega.read_mesh_network("res/d/Dragon 2.5_fbx.fbx") | Graphics;
if (!net || net->slot_count() == 0) {
std::cerr << "Failed to load mesh network\n";
return;
}
const size_t N = net->slot_count();
// -------------------------------------------------------------------------
// Shared state: one struct per slot. Audio and geometry are both outputs
// of this state. Neither owns it.
// -------------------------------------------------------------------------
struct SlotState {
// Audio
std::shared_ptr<Nodes::Network::WaveguideNetwork> wg;
std::atomic<double> energy { 0.0 };
std::atomic<double> fundamental { 0.0 };
std::atomic<double> original_fundamental { 0.0 };
std::atomic<float> loss { 0.982f };
std::atomic<bool> bowing { false };
// Geometry - written by audio metro, read by transform metro
std::atomic<float> rotation_angle { 0.0f };
std::atomic<float> scale_mod { 1.0f };
// Feedback - written by transform metro, read by audio metro
std::atomic<float> pickup_pos { 0.3f };
std::atomic<float> loss_fb { 0.0f };
glm::vec3 axis { 0.0f, 1.0f, 0.0f };
glm::mat4 base_transform { 1.0f };
};
auto states = make_persistent_shared<std::vector<std::shared_ptr<SlotState>>>();
states->reserve(N);
// -------------------------------------------------------------------------
// Golden ratio axis spread: no two slots share an axis.
// -------------------------------------------------------------------------
for (size_t i = 0; i < N; ++i) {
auto s = std::make_shared<SlotState>();
const float phi = glm::golden_ratio<float>() * glm::two_pi<float>() * static_cast<float>(i);
const float cos_t = 1.0f - 2.0f * static_cast<float>(i) / static_cast<float>(N);
const float sin_t = std::sqrt(std::max(0.0f, 1.0f - cos_t * cos_t));
s->axis = glm::normalize(glm::vec3(
sin_t * std::cos(phi),
cos_t,
sin_t * std::sin(phi)));
// Each voice: slightly detuned fundamental, different pluck position,
// different loss - enough character that no two voices decay identically.
const double fund = 55.0 * std::pow(2.0, static_cast<double>(i) * 0.75 / static_cast<double>(N) * 4.5);
const double pluck = 0.2 + 0.6 * (static_cast<double>(i) / static_cast<double>(N));
s->fundamental.store(fund);
s->original_fundamental.store(fund);
s->pickup_pos.store(static_cast<float>(pluck));
s->wg = vega.WaveguideNetwork(WaveguideNetwork::WaveguideType::STRING, fund) | Audio[0];
s->wg->set_loss_factor(0.9992 + static_cast<double>(i) * 0.000005);
s->wg->set_pickup_position(pluck);
s->wg->set_exciter_type(WaveguideNetwork::ExciterType::NOISE_BURST);
s->wg->set_exciter_duration(0.02);
s->wg->pluck(pluck, 0.9);
route_network(s->wg, { 0 }, 1.5f);
route_network(s->wg, { 1 }, 1.5f);
states->push_back(s);
}
// -------------------------------------------------------------------------
// Audio metro: reads waveguide output, updates energy and fundamental drift,
// then reads geometric feedback (pickup_pos, loss_fb) and writes back
// into the waveguide. Closed loop.
// -------------------------------------------------------------------------
schedule_metro(1.0 / 60.0, [states]() {
for (auto& s : *states) {
auto buf = s->wg->get_node_audio_buffer(0);
if (!buf || buf->empty()) continue;
// Energy: RMS of current buffer
double rms = 0.0;
for (double x : *buf) rms += x * x;
rms = std::sqrt(rms / static_cast<double>(buf->size()));
// Smooth energy
const double e_prev = s->energy.load(std::memory_order_relaxed);
const double e = e_prev * 0.92 + rms * 0.08;
s->energy.store(e, std::memory_order_relaxed);
// Fundamental drift: energy pulls fundamental upward slowly,
// restoring force pulls back toward original. Clamped to one octave
// either side to prevent waveguide collapse at high slot indices.
const double fund = s->fundamental.load(std::memory_order_relaxed);
const double drift = fund + e * 0.05 - (fund - s->original_fundamental.load(std::memory_order_relaxed)) * 0.01;
const double orig = s->original_fundamental.load(std::memory_order_relaxed);
const double clamped = std::clamp(drift, orig * 0.5, orig * 2.0);
s->fundamental.store(clamped, std::memory_order_relaxed);
s->wg->set_fundamental(clamped);
// Re-pluck when energy collapses below threshold. Resets fundamental
// and loss to original values so the voice re-enters cleanly.
// bowing gate prevents re-triggering until energy recovers above 0.02.
if (e < 0.004 && !s->bowing.load(std::memory_order_relaxed)) {
s->bowing.store(true, std::memory_order_relaxed);
s->fundamental.store(orig, std::memory_order_relaxed);
s->wg->set_fundamental(orig);
s->wg->set_loss_factor(0.982);
s->loss.store(0.982f, std::memory_order_relaxed);
s->wg->set_exciter_type(WaveguideNetwork::ExciterType::NOISE_BURST);
s->wg->set_exciter_duration(0.004);
s->wg->pluck(static_cast<double>(s->pickup_pos.load(std::memory_order_relaxed)), 0.9);
auto ch = (int)get_uniform_random(0, 5) % 2;
route_network(s->wg, { static_cast<uint32_t>(ch) }, 1.5f);
} else if (e > 0.02) {
s->bowing.store(false, std::memory_order_relaxed);
}
// Geometric feedback -> audio:
// pickup position read from slot transform state,
// loss nudged by loss_fb written by transform metro.
const float pp = s->pickup_pos.load(std::memory_order_relaxed);
s->wg->set_pickup_position(static_cast<double>(pp));
const float lfb = s->loss_fb.load(std::memory_order_relaxed);
if (s->bowing.load(std::memory_order_relaxed)) {
const double l = static_cast<double>(
std::clamp(s->loss.load(std::memory_order_relaxed) + lfb * 0.00001f, 0.995f, 0.99999f));
s->wg->set_loss_factor(l);
s->loss.store(static_cast<float>(l), std::memory_order_relaxed);
}
// Rotation speed driven by energy - written for transform metro.
const float rot = static_cast<float>(e * 6.0);
s->rotation_angle.store(
s->rotation_angle.load(std::memory_order_relaxed) + rot,
std::memory_order_relaxed);
// Scale modulation: driven by energy level.
const float sc = std::clamp(
static_cast<float>(0.6 + e * 8.0), 0.6f, 1.4f);
s->scale_mod.store(sc, std::memory_order_relaxed);
} }, "audio_state");
// -------------------------------------------------------------------------
// Transform metro: reads rotation_angle and scale_mod from state,
// writes slot transforms, then reads back slot position to update
// pickup_pos and loss_fb. Closed loop from geometry back to audio.
// -------------------------------------------------------------------------
schedule_metro(1.0 / 60.0, [net, states]() {
auto& slots = net->slots();
for (size_t i = 0; i < slots.size() && i < states->size(); ++i) {
auto& s = *(*states)[i];
const float angle = s.rotation_angle.load(std::memory_order_relaxed);
const float sc = s.scale_mod.load(std::memory_order_relaxed);
slots[i].local_transform =
glm::scale(
glm::rotate(s.base_transform, angle, s.axis),
glm::vec3(sc));
slots[i].dirty = true;
// Extract position from accumulated transform,
// map Y component to pickup position [0.1, 0.9].
const glm::vec3 pos = glm::vec3(slots[i].local_transform[3]);
const float mapped = std::clamp(
(pos.y + 1.5f) / 3.0f, 0.1f, 0.9f);
s.pickup_pos.store(mapped, std::memory_order_relaxed);
// Map scale deviation from 1.0 to loss feedback.
// Expanding slot tightens the string. Collapsing loosens it.
s.loss_fb.store(sc - 1.0f, std::memory_order_relaxed);
} }, "transform_state");
// -------------------------------------------------------------------------
// Staggered re-pluck sequence: voices re-enter at different times,
// restarting the pluck->bow lifecycle from the beginning.
// -------------------------------------------------------------------------
schedule_sequence(
[&]() {
std::vector<std::pair<double, std::function<void()>>> seq;
for (size_t i = 0; i < N; ++i) {
const double t = static_cast<double>(i) * 3.7;
seq.emplace_back(t, [s = (*states)[i]]() {
s->bowing.store(false);
s->loss.store(0.982f);
s->scale_mod.store(1.0f);
s->fundamental.store(s->original_fundamental.load());
s->wg->set_fundamental(s->original_fundamental.load());
s->wg->set_exciter_type(WaveguideNetwork::ExciterType::NOISE_BURST);
s->wg->set_loss_factor(0.982);
s->wg->pluck(static_cast<double>(s->pickup_pos.load()), 0.9);
});
}
return seq;
}(),
"reenter");
auto buf = vega.MeshNetworkBuffer(net) | Graphics;
buf->setup_rendering({ .target_window = window });
buf->get_render_processor()->set_view_transform(
Kinesis::look_at_perspective(
{ 0.0f, 100.0f, 300.0f }, { 0.0f, 80.0f, 0.0f },
glm::radians(45.0f), 1920.0f / 1080.0f, 1.0f, 100000.0f));
bind_viewport_preset(window, buf->get_render_processor(),
ViewportPresetMode::Fly, {}, "mesh_net");
A mesh in MayaFlux is two spans. Vertex bytes and triangle indices, described by a layout. That is the complete representation - whether it came from an FBX, from a generative algorithm, from a Yantra operator applying a field, or from audio analysis thresholds rebuilding topology from scratch.
It is not a special case with its own access conventions. It participates in the same substrate as every other data type in the system. The same operations that work on audio buffers work on vertex bytes.
The same region model that describes a transient in sample space describes a submesh boundary in index space. Once mesh is just ordered bytes with a layout, the question of what produces those bytes becomes open.
And when the thing producing them is the same state object that is also producing audio - not driving it, not receiving from it, but being it simultaneously - you have closed a loop that the two-domain model cannot even describe.
Drone Disintegration
A mesh is two spans: vertex bytes and triangle indices, with independent dirty flags. Two different data streams that happen to share a rendered output.
Code
constexpr int W = 48;
constexpr int H = 48;
constexpr int VCOUNT = W * H;
// Build a deliberately ugly initial mesh: a twisted, folded surface.
// Not a grid, not a sphere. Something that already looks wrong at rest.
auto build_verts = []() {
std::vector<MeshVertex> v;
v.reserve(static_cast<size_t>(VCOUNT));
for (int row = 0; row < H; ++row) {
for (int col = 0; col < W; ++col) {
const float u = static_cast<float>(col) / (W - 1); // 0..1
const float vf = static_cast<float>(row) / (H - 1); // 0..1
const float a = u * glm::two_pi<float>() * 2.3f;
const float b = vf * glm::pi<float>() * 1.7f;
// Twisted torus-like base with deliberate self-intersections.
const float r1 = 0.9f + 0.35f * std::cos(b);
const float fold = std::sin(a * 0.5f + b * 1.3f) * 0.4f;
glm::vec3 pos {
r1 * std::cos(a) + fold * std::sin(b * 2.1f),
std::sin(b) * 0.7f + std::sin(a * 1.4f) * 0.25f,
r1 * std::sin(a) - fold * std::cos(a * 0.8f)
};
// Normal approximation from parametric partials (good enough).
const glm::vec3 du {
-r1 * std::sin(a),
std::cos(a * 1.4f) * 0.25f * 1.4f,
r1 * std::cos(a)
};
const glm::vec3 dv {
-0.35f * std::sin(b) * std::cos(a),
std::cos(b) * 0.7f,
-0.35f * std::sin(b) * std::sin(a)
};
const glm::vec3 n = glm::normalize(glm::cross(du, dv));
MeshVertex mv;
mv.position = pos;
mv.normal = n;
mv.tangent = glm::normalize(du);
mv.color = glm::mix(
glm::vec3(0.15f, 0.05f, 0.35f),
glm::vec3(0.7f, 0.15f, 0.05f),
u * vf + std::sin(a) * 0.2f + 0.2f);
mv.weight = 0.0f;
mv.uv = { u, vf };
v.push_back(mv);
}
}
return v;
};
// Triangulation with per-band survival based on energy floor.
// Different bands tear at different thresholds so the surface
// disintegrates unevenly rather than all at once.
auto build_indices = [](float energy) {
std::vector<uint32_t> idx;
idx.reserve(static_cast<size_t>((H - 1) * W) * 6);
for (int row = 0; row < H - 1; ++row) {
// Each row has a different tear threshold: diagonal bands tear first.
const float band = static_cast<float>((row + row / 3) % (H / 4))
/ static_cast<float>(H / 4);
if (energy * band > 0.55f)
continue;
for (int col = 0; col < W; ++col) {
const int ncol = (col + 1) % W;
const uint32_t tl = static_cast<uint32_t>(row * W + col);
const uint32_t tr = static_cast<uint32_t>(row * W + ncol);
const uint32_t bl = static_cast<uint32_t>((row + 1) * W + col);
const uint32_t br = static_cast<uint32_t>((row + 1) * W + ncol);
idx.insert(idx.end(), { tl, bl, tr, tr, bl, br });
}
}
return idx;
};
auto window = MayaFlux::create_window({ "drone disintegration", 1200, 800 });
auto mesh = vega.MeshWriterNode(static_cast<size_t>(VCOUNT)) | Graphics;
{
auto v = build_verts();
auto i = build_indices(0.0f);
mesh->set_mesh(v, i);
}
auto buf = vega.GeometryBuffer(mesh) | Graphics;
buf->setup_rendering({ .target_window = window });
buf->get_render_processor()->set_view_transform(
Kinesis::look_at_perspective(
{ 3.5f, 2.0f, 3.5f }, { 0.0f, 0.0f, 0.0f },
glm::radians(55.0f), 1200.0f / 800.0f, 0.01f, 1000.0f));
window->show();
// Drone synthesis.
//
// Root: 28 Hz sub, self-FM for warmth.
auto sub_mod = vega.Sine(28.0f, 2.2) | Audio[0];
auto sub = vega.Sine(sub_mod, 28.0f, 1.0) | Audio[{ 0, 1 }];
// 2nd harmonic: 56 Hz, AM by a very slow LFO so it breathes independently.
auto lfo_slow = vega.Sine(0.07f, 1.0);
auto h2 = vega.Sine(56.0f, 0.8) | Audio[{ 0, 1 }];
h2->set_amplitude_modulator(lfo_slow);
// 3rd harmonic: 84 Hz, FM by sub for slight instability.
auto h3_mod = vega.Polynomial([sub](double x) {
return sub->get_last_output() * 3.0;
}) | Audio[0];
auto h3 = vega.Sine(h3_mod, 84.0f, 0.55) | Audio[{ 0, 1 }];
// 5th harmonic: 140 Hz, AM by a mid-rate LFO on a different cycle.
auto lfo_mid = vega.Sine(0.19f, 1.0);
auto h5 = vega.Sine(840.0f, 0.4) | Audio[{ 0, 1 }];
h5->set_amplitude_modulator(lfo_mid);
// 7th harmonic: 196 Hz, FM by h3 — cross-modulation between harmonics.
auto h7_mod = vega.Polynomial([h3](double x) {
return h3->get_last_output() * 5.0;
}) | Audio[0];
auto h7 = vega.Sine(h7_mod, 196.0f, 0.3) | Audio[{ 0, 1 }];
// Inharmonic grind: 243 Hz (not in the series), FM by h2, creates beating.
auto grind_mod = vega.Polynomial([h2](double x) {
return h2->get_last_output() * 6.5;
}) | Audio[0];
auto grind = vega.Sine(grind_mod, 243.0f, 0.25) | Audio[{ 0, 1 }];
// Gentle tanh saturation — adds upper partials without noise.
auto sat = vega.Polynomial([sub, h2, h3, h5, h7, grind](double x) {
const double sum = sub->get_last_output()
+ h2->get_last_output()
+ h3->get_last_output()
+ h5->get_last_output()
+ h7->get_last_output()
+ grind->get_last_output();
return std::tanh(sum * 1.6) * 0.65;
}) | Audio[{ 0, 1 }];
// Spatial routing: layers orbit between channels on slow independent cycles.
// Sub stays centred. Upper harmonics drift across the stereo field.
// Each route_node call fades over 8 seconds so crossings are smooth.
route_node(sub, { 0, 1 }, 8.0);
route_node(h2, { 0 }, 0.0);
route_node(h3, { 1 }, 0.0);
route_node(h5, { 0 }, 0.0);
route_node(h7, { 1 }, 0.0);
route_node(grind, { 0, 1 }, 0.0);
route_node(sat, { 0, 1 }, 0.0);
// Slow routing rotation: h2/h3/h5/h7 swap sides every ~13 seconds.
auto routing_phase = std::make_shared<std::atomic<int>>(0);
MayaFlux::schedule_metro(13.0, [h2, h3, h5, h7, routing_phase]() {
const int phase = routing_phase->fetch_add(1, std::memory_order_relaxed) % 2;
if (phase == 0) {
route_node(h2, { 1 }, 8.0);
route_node(h3, { 0 }, 8.0);
route_node(h5, { 1 }, 8.0);
route_node(h7, { 0 }, 8.0);
} else {
route_node(h2, { 0 }, 8.0);
route_node(h3, { 1 }, 8.0);
route_node(h5, { 0 }, 8.0);
route_node(h7, { 1 }, 8.0);
}
});
// Per-layer amplitude state for mesh deformation.
// Written at audio rate, read at 60 Hz graphics rate.
// Smoothed with a very slow coefficient — deformation moves glacially.
struct LayerState {
std::atomic<float> sub { 0.0f };
std::atomic<float> mid { 0.0f }; // h2+h3
std::atomic<float> upper { 0.0f }; // h5+h7
std::atomic<float> grind { 0.0f };
std::atomic<bool> rebuild { false };
};
auto ls = std::make_shared<LayerState>();
// Smooth each layer independently with different time constants.
// Sub is very slow (geological). Grind is the fastest (still slow).
auto sub_smooth = std::make_shared<std::atomic<float>>(0.0f);
auto mid_smooth = std::make_shared<std::atomic<float>>(0.0f);
auto upper_smooth = std::make_shared<std::atomic<float>>(0.0f);
auto grind_smooth = std::make_shared<std::atomic<float>>(0.0f);
MayaFlux::schedule_metro(1.0 / 60.0,
[sub, h2, h3, h5, h7, grind,
sub_smooth, mid_smooth, upper_smooth, grind_smooth, ls]() {
auto smooth = [](std::atomic<float>& s, float in, float coeff) {
const float v = s.load(std::memory_order_relaxed) * coeff
+ std::abs(in) * (1.0f - coeff);
s.store(v, std::memory_order_relaxed);
return v;
};
const float sv = smooth(*sub_smooth,
static_cast<float>(sub->get_last_output()), 0.9985f);
const float mv = smooth(*mid_smooth,
static_cast<float>(h2->get_last_output() + h3->get_last_output()), 0.997f);
const float uv = smooth(*upper_smooth,
static_cast<float>(h5->get_last_output() + h7->get_last_output()), 0.994f);
const float gv = smooth(*grind_smooth,
static_cast<float>(grind->get_last_output()), 0.991f);
ls->sub.store(sv, std::memory_order_relaxed);
ls->mid.store(mv, std::memory_order_relaxed);
ls->upper.store(uv, std::memory_order_relaxed);
ls->grind.store(gv, std::memory_order_relaxed);
// Topology rebuild when grind layer crosses a threshold.
if (gv > 0.12f)
ls->rebuild.store(true, std::memory_order_relaxed);
});
// Graphics deformation: slow, planar, cracked-stone / splattered-paint motion.
//
// Each vertex moves in a direction fixed at startup (its "fault direction"),
// scaled by whichever audio layer dominates its region of the mesh.
// Vertices do not inflate outward — they slide along their fault plane,
// producing shear, cracking, and lateral displacement rather than spherical growth.
const auto base_verts = build_verts();
// Pre-compute per-vertex fault directions: fixed vectors that determine
// which way each vertex will slide when its layer activates.
std::vector<glm::vec3> fault_dirs(static_cast<size_t>(VCOUNT));
for (int row = 0; row < H; ++row) {
for (int col = 0; col < W; ++col) {
const size_t idx = static_cast<size_t>(row * W + col);
const auto& bv = base_verts[idx];
// Fault direction: cross between normal and a fixed world axis,
// perturbed by position. Produces a field of sliding planes.
const glm::vec3 axis = glm::normalize(glm::vec3(
std::sin(bv.position.x * 2.3f + bv.position.y),
std::cos(bv.position.z * 1.7f - bv.position.x),
std::sin(bv.position.y * 3.1f + bv.position.z * 0.9f)));
fault_dirs[idx] = glm::normalize(glm::cross(bv.normal, axis));
}
}
MayaFlux::schedule_metro(1.0 / 60.0,
[mesh, ls, base_verts, fault_dirs, build_indices]() mutable {
const float sv = ls->sub.load(std::memory_order_relaxed);
const float mv = ls->mid.load(std::memory_order_relaxed);
const float uv = ls->upper.load(std::memory_order_relaxed);
const float gv = ls->grind.load(std::memory_order_relaxed);
if (ls->rebuild.exchange(false, std::memory_order_relaxed)) {
auto new_idx = build_indices(gv * 4.0f);
if (!new_idx.empty())
mesh->set_mesh_indices(new_idx);
}
auto verts = base_verts;
for (int row = 0; row < H; ++row) {
const float row_t = static_cast<float>(row) / (H - 1);
for (int col = 0; col < W; ++col) {
const size_t i = static_cast<size_t>(row * W + col);
auto& v = verts[i];
const glm::vec3& fd = fault_dirs[i];
// Region weights: sub owns bottom rows, mid the middle,
// upper the top, grind scattered diagonally.
const float sub_w = std::max(0.0f, 1.0f - row_t * 2.5f);
const float mid_w = std::max(0.0f, 1.0f - std::abs(row_t - 0.5f) * 3.5f);
const float upper_w = std::max(0.0f, row_t * 2.5f - 1.5f);
const float grind_w = std::abs(std::sin(
row_t * glm::pi<float>() * 4.7f
+ v.position.x * 3.1f));
// Displacement is entirely along the fault direction — lateral shear.
// Large scalars: when the sub moves, it moves far.
const float disp = fd.x * (sv * sub_w * 3.2f)
+ fd.y * (mv * mid_w * 2.4f)
+ fd.z * (uv * upper_w * 1.9f)
+ (fd.x + fd.z) * 0.5f * (gv * grind_w * 1.5f);
v.position = base_verts[i].position + fault_dirs[i] * disp;
v.weight = std::clamp(std::abs(disp) * 0.6f, 0.0f, 1.0f);
v.color = glm::mix(
glm::vec3(0.08f, 0.03f, 0.22f),
glm::vec3(0.9f, 0.45f, 0.05f),
v.weight);
}
}
mesh->set_mesh_vertices(verts);
});
bind_viewport_preset(window, buf->get_render_processor(), ViewportPresetMode::Fly, {}, "drone_disintegration");
The vertex buffer and the index buffer are not a mesh. They are two independent sequences of numbers that a renderer happens to interpret together. Here they are driven by two independent aspects of the same harmonic signal - continuous spectral energy for deformation, threshold crossings in the smoothed layer envelopes for structural topology change.
The surface holds its form for a long time because the smoothing is slow. When it tears, entire bands of triangulation disappear because a spectral layer has been sustaining above its threshold long enough. This is not an effect. There is no mesh deformation plugin, no geometry shader, no particle system approximating destruction. A span of indices was rebuilt with fewer entries. The renderer drew fewer triangles.
That is the complete description. And because mesh in MayaFlux is just bytes with a layout - the same representation whether it came from an FBX, a generative algorithm, or audio analysis thresholds - any system that can produce the right byte layout can drive the geometry.
The harmonic content of a 48-partial synthesis network is one such system. So is anything else.
Nexus
Nexus is a layer for relationships between things in space. Not a scene graph. Not an actor system. A positioned entity, a function that fires on a schedule, and outputs that route to wherever you point them.
Skip ahead in the video for line and triangle rendering modes.
Code
std::vector<LineVertex> cuboid_wireframe(
const glm::vec3& centre, const glm::vec3& half)
{
const glm::vec3 c = centre;
const glm::vec3 h = half;
glm::vec3 v[8] = {
c + glm::vec3(-h.x, -h.y, -h.z),
c + glm::vec3(h.x, -h.y, -h.z),
c + glm::vec3(h.x, h.y, -h.z),
c + glm::vec3(-h.x, h.y, -h.z),
c + glm::vec3(-h.x, -h.y, h.z),
c + glm::vec3(h.x, -h.y, h.z),
c + glm::vec3(h.x, h.y, h.z),
c + glm::vec3(-h.x, h.y, h.z),
};
return { { v[0] }, { v[1] }, { v[1] }, { v[2] }, { v[2] }, { v[3] }, { v[3] }, //
{ v[0] }, { v[4] }, { v[5] }, { v[5] }, { v[6] }, { v[6] }, { v[7] }, { v[7] }, { v[4] }, //
{ v[0] }, { v[4] }, { v[1] }, { v[5] }, { v[2] }, { v[6] }, { v[3] }, { v[7] },
};
}
void nexa()
{
auto window = MayaFlux::create_window({ "nexus", 1920, 1080 });
window->show();
auto tapestry = make_persistent_shared<Nexus::Tapestry>(get_scheduler(), get_event_manager());
auto& mgr = *MayaFlux::get_buffer_manager();
// auto& sched = *MayaFlux::get_scheduler();
// auto& evmgr = *MayaFlux::get_event_manager();
//
// auto fabric = make_persistent_shared<Nexus::Fabric>(sched, evmgr);
auto fabric = tapestry->create_fabric("nexa_fabric");
// -------------------------------------------------------------------------
// Particle field: the thing being influenced.
// Physics parameters are written by Sensors, not by user code.
// -------------------------------------------------------------------------
auto particles = vega.ParticleNetwork(
2000,
glm::vec3(-2.0f),
glm::vec3(2.0f),
Kinesis::SpatialDistribution::FIBONACCI_SPHERE)
| Graphics;
auto physics = particles->create_operator<PhysicsOperator>();
physics->set_gravity(glm::vec3(0.0f));
physics->set_drag(0.03f);
physics->set_interaction_radius(0.3f);
physics->set_bounds(glm::vec3(-2.0f), glm::vec3(2.0f));
physics->set_bounds_mode(PhysicsOperator::BoundsMode::BOUNCE);
// physics->enable_spatial_interactions(true);
auto particle_buf = vega.NetworkGeometryBuffer(particles) | Graphics;
particle_buf->setup_rendering({
.target_window = window,
.vertex_shader = "point_lit.vert",
.fragment_shader = "point_lit.frag",
// TOGGLE THESE FOR FUN WITH RENDERING MODES
// .vertex_shader = "line_lit.vert",
// .fragment_shader = "line_lit.frag",
// .geometry_shader = "line_lit.geom",
// .topology = Portal::Graphics::PrimitiveTopology::LINE_STRIP,
// .vertex_shader = "triangle_lit.vert",
// .fragment_shader = "triangle_lit.frag",
// .topology = Portal::Graphics::PrimitiveTopology::TRIANGLE_LIST,
});
// physics->set_point_size(2);
// -------------------------------------------------------------------------
// Three Emitters on choreographed paths.
// Each emits a tone - pitch maps from x position, amplitude from y.
// One also renders itself as a wireframe gizmo and drives the lighting UBO.
// -------------------------------------------------------------------------
auto make_phase = [](double offset) {
return std::make_shared<double>(offset);
};
auto make_emitter = [&](glm::vec3 start, double phase_offset, int audio_ch)
-> std::shared_ptr<Nexus::Emitter> {
auto phase = make_phase(phase_offset);
auto e = std::make_shared<Nexus::Emitter>([](const Nexus::InfluenceContext&) { });
e->set_position(start);
e->sink_audio(mgr, audio_ch,
[phase](const Nexus::InfluenceContext& ctx) -> Kakshya::DataVariant {
const uint32_t n = Buffers::s_preferred_buffer_size;
const double sr = static_cast<double>(Buffers::s_registered_sample_rate);
const double freq = 110.0
+ static_cast<double>(ctx.position.x + 2.0f) * 180.0
+ static_cast<double>(ctx.position.z + 2.0f) * 90.0;
const double amp = 0.12
+ static_cast<double>(std::abs(ctx.position.y)) * 0.18;
const double inc = 2.0 * M_PI * freq / sr;
std::vector<double> samples(n);
for (uint32_t i = 0; i < n; ++i) {
samples[i] = amp * std::sin(*phase);
*phase += inc;
}
*phase = std::fmod(*phase, 2.0 * M_PI);
return Kakshya::DataVariant { std::move(samples) };
});
return e;
};
// Emitter 0: orbit in XY plane, left channel.
auto e0 = make_emitter({ 1.5f, 0.0f, 0.0f }, 0.0, 0);
// Emitter 1: orbit in XZ plane, right channel.
auto e1 = make_emitter({ 0.0f, 0.0f, 1.5f }, M_PI / 3.0, 1);
// Emitter 2: figure-eight in XY, both channels.
// Also renders itself as a wireframe gizmo and drives the lighting UBO.
auto e2 = make_emitter({ 0.0f, 1.0f, 0.0f }, M_PI * 2.0 / 3.0, 0);
e2->render(mgr, {
.target_window = window,
.topology = Portal::Graphics::PrimitiveTopology::LINE_LIST,
});
e2->set_color(glm::vec3(1.0f, 0.85f, 0.5f));
e2->set_intensity(4.0f);
e2->set_radius(2.5f);
e2->set_influence_target(particle_buf->get_render_processor());
auto update_gizmo = [e2](const glm::vec3& p) {
const auto wire = cuboid_wireframe(p, { 0.08f, 0.08f, 0.08f });
e2->set_vertices<LineVertex>(wire);
};
update_gizmo(e2->position().value());
// Orbit metros: position updates fire at 60Hz.
// Audio dispatch fires at buffer rate via fabric.
// These are independent schedules on the same entity.
auto t0 = std::make_shared<float>(0.0f);
auto t1 = std::make_shared<float>(0.0f);
auto t2 = std::make_shared<float>(0.0f);
schedule_metro(1.0 / 60.0, [e0, t0, update_gizmo]() {
*t0 += 0.008f;
e0->set_position({
std::cos(*t0) * 1.5f,
std::sin(*t0 * 0.7f) * 0.6f,
std::sin(*t0) * 0.4f }); }, "orbit_e0");
schedule_metro(1.0 / 60.0, [e1, t1]() {
*t1 += 0.011f;
e1->set_position({
std::sin(*t1 * 1.3f) * 0.5f,
std::cos(*t1) * 0.4f,
std::cos(*t1) * 1.5f }); }, "orbit_e1");
schedule_metro(1.0 / 60.0, [e2, t2, update_gizmo]() {
*t2 += 0.006f;
const glm::vec3 p {
std::sin(*t2 * 2.0f) * 1.2f,
std::sin(*t2) * 1.0f,
std::cos(*t2 * 1.4f) * 0.5f };
e2->set_position(p);
update_gizmo(p); }, "orbit_e2");
// Audio dispatch: buffer-rate scheduling via fabric.
const double buf_rate = static_cast<double>(Buffers::s_preferred_buffer_size)
/ static_cast<double>(Buffers::s_registered_sample_rate);
fabric->wire(e0).every(buf_rate).finalise();
fabric->wire(e1).every(buf_rate).finalise();
fabric->wire(e2).every(buf_rate).finalise();
// Gizmo and lighting UBO update: 60Hz.
fabric->wire(e2).every(1.0 / 60.0).finalise();
// -------------------------------------------------------------------------
// Two Sensors at fixed positions.
// Each detects nearby Emitters and writes into both physics and synthesis.
// These are the only things that connect spatial state to audio and visual.
// -------------------------------------------------------------------------
// Shared synthesis state: Sensors write, audio nodes read.
auto turbulence = std::make_shared<std::atomic<float>>(0.0f);
auto repulsion = std::make_shared<std::atomic<float>>(0.0f);
auto global_pitch = std::make_shared<std::atomic<float>>(1.0f);
auto s0 = std::make_shared<Nexus::Sensor>(1.8f,
[physics, turbulence, repulsion](const Nexus::PerceptionContext& ctx) {
const float n = static_cast<float>(ctx.spatial_results.size());
// Physics: proximity drives turbulence and repulsion.
const float t = std::min(n * 0.3f, 1.5f);
const float r = std::min(n * 0.1f, 0.5f);
physics->set_turbulence_strength(t);
physics->set_repulsion_strength(r);
turbulence->store(t, std::memory_order_relaxed);
repulsion->store(r, std::memory_order_relaxed);
});
s0->set_position(glm::vec3(0.0f, 0.5f, 0.0f));
auto s1 = std::make_shared<Nexus::Sensor>(1.4f,
[physics, global_pitch](const Nexus::PerceptionContext& ctx) {
const float n = static_cast<float>(ctx.spatial_results.size());
// Synthesis: proximity shifts the global pitch multiplier.
// More Emitters nearby -> higher pitch across all voices.
const float pitch = 1.0f + n * 0.15f;
global_pitch->store(pitch, std::memory_order_relaxed);
// Physics: attraction toward sensor when Emitters are close.
if (!ctx.spatial_results.empty()) {
physics->set_attraction_point(ctx.position);
physics->set_attraction_strength(n * 0.6f);
} else {
physics->set_attraction_strength(0.0f);
}
});
s1->set_position(glm::vec3(-0.5f, -0.8f, 0.3f));
// Register Emitter positions in the spatial index so Sensors can find them.
// Fabric handles this via wire() - Emitters registered on finalise().
fabric->wire(e0).finalise();
fabric->wire(e1).finalise();
fabric->wire(s0).finalise();
fabric->wire(s1).finalise();
// Commit-driven spatial update: fires perception callbacks with query results.
schedule_metro(1.0 / 60.0, [fabric]() { fabric->commit(); }, "spatial_commit");
// -------------------------------------------------------------------------
// Interactivity: mouse click repositions s1 in real time.
// The perception radius and callbacks are unchanged - only the position moves.
// The physics and pitch response follow immediately because they read from
// whatever position the Sensor currently occupies.
// -------------------------------------------------------------------------
on_mouse_pressed(window, IO::MouseButtons::Left,
[window, s1](double x, double y) {
const glm::vec3 p = normalize_coords(x, y, window);
s1->set_position(glm::vec3(p.x, p.y, 0.3f));
});
on_mouse_pressed(window, IO::MouseButtons::Right,
[window, s0](double x, double y) {
const glm::vec3 p = normalize_coords(x, y, window);
s0->set_position(glm::vec3(p.x, p.y, 0.5f));
});
// -------------------------------------------------------------------------
// Interactivity: pulse, speed, scatter.
// All shared state is plain value types - no atomics needed on the
// scheduler thread.
// -------------------------------------------------------------------------
auto speed_scale = std::make_shared<float>(1.0f);
auto orbits_frozen = std::make_shared<bool>(false);
// Scroll: scale orbit speed. Clamped to [0.05, 4.0].
on_scroll(window, [speed_scale](double /*dx*/, double dy) { *speed_scale = std::clamp(*speed_scale + static_cast<float>(dy) * 0.08f, 0.05f, 4.0f); }, "scroll_speed");
// Patch the orbit metros to respect speed_scale and frozen flag.
// Replace the three schedule_metro lambdas above with these:
// (remove the originals and use these instead)
schedule_metro(1.0 / 60.0, [e0, t0, speed_scale, orbits_frozen]() {
if (*orbits_frozen) return;
*t0 += 0.008f * *speed_scale;
e0->set_position({
std::cos(*t0) * 1.5f,
std::sin(*t0 * 0.7f) * 0.6f,
std::sin(*t0) * 0.4f }); }, "orbit_e0");
schedule_metro(1.0 / 60.0, [e1, t1, speed_scale, orbits_frozen]() {
if (*orbits_frozen) return;
*t1 += 0.011f * *speed_scale;
e1->set_position({
std::sin(*t1 * 1.3f) * 0.5f,
std::cos(*t1) * 0.4f,
std::cos(*t1) * 1.5f }); }, "orbit_e1");
schedule_metro(1.0 / 60.0, [e2, t2, speed_scale, orbits_frozen, update_gizmo]() {
if (*orbits_frozen) return;
*t2 += 0.006f * *speed_scale;
const glm::vec3 p {
std::sin(*t2 * 2.0f) * 1.2f,
std::sin(*t2) * 1.0f,
std::cos(*t2 * 1.4f) * 0.5f };
e2->set_position(p);
update_gizmo(p); }, "orbit_e2");
// Middle click: freeze/unfreeze orbits.
on_mouse_pressed(window, IO::MouseButtons::Middle, [orbits_frozen](double, double) { *orbits_frozen = !*orbits_frozen; }, "freeze_orbits");
// Space: pulse s0 radius then decay back over ~1.5s at 60Hz.
auto pulse_t = std::make_shared<float>(-1.0f); // negative = inactive
constexpr float k_pulse_base_radius = 1.8f;
constexpr float k_pulse_peak_radius = 4.5f;
constexpr float k_pulse_decay = 4.0f; // units per second
schedule_metro(1.0 / 60.0, [s0, pulse_t]() {
if (*pulse_t < 0.0f) return;
*pulse_t += 1.0f / 60.0f;
const float r = k_pulse_peak_radius
* std::exp(-k_pulse_decay * *pulse_t);
s0->set_query_radius(k_pulse_base_radius + r);
if (r < 0.02f) {
s0->set_query_radius(k_pulse_base_radius);
*pulse_t = -1.0f;
} }, "pulse_decay");
on_key_pressed(window, IO::Keys::Space, [pulse_t, s0]() {
s0->set_query_radius(k_pulse_peak_radius);
*pulse_t = 0.0f; }, "pulse_trigger");
// R: scatter - brief high turbulence spike then clear.
auto scatter_t = std::make_shared<float>(-1.0f);
constexpr float k_scatter_duration = 0.4f;
constexpr float k_scatter_strength = 14.0f;
schedule_metro(1.0 / 60.0, [physics, scatter_t, turbulence]() {
if (*scatter_t < 0.0f) return;
*scatter_t += 1.0f / 60.0f;
if (*scatter_t >= k_scatter_duration) {
physics->set_turbulence_strength(turbulence->load(std::memory_order_relaxed));
*scatter_t = -1.0f;
} }, "scatter_decay");
on_key_pressed(window, IO::Keys::R, [physics, scatter_t]() {
physics->set_turbulence_strength(k_scatter_strength);
*scatter_t = 0.0f; }, "scatter_trigger");
// -------------------------------------------------------------------------
// View
// -------------------------------------------------------------------------
particle_buf->get_render_processor()->set_view_transform(
Kinesis::look_at_perspective(
{ 0.0f, 0.0f, 6.0f }, { 0.0f, 0.0f, 0.0f },
glm::radians(55.0f), 1920.0f / 1080.0f, 0.01f, 1000.0f));
bind_viewport_preset(window, particle_buf->get_render_processor(),
ViewportPresetMode::Fly, {}, "nexus");
auto shared_source = particle_buf->get_render_processor()->get_view_transform_source();
e2->get_render_processor(window)->set_view_transform_source(shared_source);
}
Every framework that has ever handled both audio and visuals has kept them in separate namespaces, separate update loops, separate conceptual models, connected by explicit bridges that you build and maintain.
A light type that can write into a shader. A sound emitter type that routes into an audio engine. A force type that modifies a physics simulation. These are the same computational structure: a position, a falloff function, and a value written somewhere. Nexus does not have a light type. It does not have a force type. It does not have a sound emitter type. It has a positioned entity and a function.
What domain that function writes into is entirely your decision, expressed in the influence callback. The same object that produces a tone whose pitch follows its trajectory through space is also the wireframe you see moving, and also the value the fragment shader reads to compute distance attenuation on the particle field behind it.
There is no message passing between systems. There is one entity, three output registrations, and a function. The spatial query that tells the Sensors which Emitters are nearby feeds both the particle physics and the synthesis simultaneously because both are just numbers, and the Sensor does not know or care which domain those numbers end up in.
This is not a game engine with audio bolted on. It is not an audio engine with a renderer attached. It is a substrate for spatial computation where domain is a detail of the output registration, not a property of the entity.
Portal::Text animated
A glyph is a quad. A quad is four numbers. Four numbers can go anywhere.
Code
auto window = MayaFlux::create_window({ "text", 1920, 1080 });
window->show();
Portal::Text::set_default_font("JetBrains Mono", "Medium", 72);
const std::string text = "PLACEHOLDER";
const uint32_t w = 1920;
const uint32_t h = 1080;
auto layout = Portal::Text::create_layout(text, 0.0f, 0.0f, w);
auto text_buf = Portal::Text::press(
text,
{ .color = { 1.0f, 1.0f, 1.0f, 1.0f },
.render_bounds = { w, h } })
| Graphics;
const float sx = static_cast<float>(text_buf->get_width()) / static_cast<float>(w);
const float sy = static_cast<float>(text_buf->get_height()) / static_cast<float>(h);
text_buf->set_scale(sx, sy);
text_buf->set_position(-.5f + sx, 0.5f - sy);
text_buf->setup_rendering({ .target_window = window });
// -------------------------------------------------------------------------
// Per-glyph state: each character gets independent oscillator parameters
// seeded from its index. The motion is incommensurable across glyphs -
// no two characters share a period relationship that would produce
// synchronised behaviour.
// -------------------------------------------------------------------------
const size_t glyph_count = layout->quads.size();
struct GlyphParams {
float freq_x;
float freq_y;
float freq_rot;
float phase_x;
float phase_y;
float phase_rot;
float amp_x;
float amp_y;
float amp_rot;
};
std::vector<GlyphParams> params;
params.reserve(glyph_count);
for (size_t i = 0; i < glyph_count; ++i) {
const float fi = static_cast<float>(i);
// Frequencies seeded by index through phi to guarantee incommensurability.
const float phi = 1.6180339887f;
params.push_back({
.freq_x = 0.4f + std::fmod(fi * phi * 0.17f, 1.1f),
.freq_y = 0.3f + std::fmod(fi * phi * 0.23f, 0.9f),
.freq_rot = 0.2f + std::fmod(fi * phi * 0.31f, 0.7f),
.phase_x = std::fmod(fi * phi, glm::two_pi<float>()),
.phase_y = std::fmod(fi * phi * 1.3f, glm::two_pi<float>()),
.phase_rot = std::fmod(fi * phi * 1.7f, glm::two_pi<float>()),
.amp_x = 4.0f + std::fmod(fi * phi * 0.41f, 8.0f),
.amp_y = 6.0f + std::fmod(fi * phi * 0.53f, 12.0f),
.amp_rot = 0.05f + std::fmod(fi * phi * 0.07f, 0.18f),
});
}
// -------------------------------------------------------------------------
// Color state: each glyph drifts through hue independently.
// Hue rate also seeded by phi so no two glyphs cycle together.
// -------------------------------------------------------------------------
std::vector<float> hue_phases(glyph_count);
std::vector<float> hue_rates(glyph_count);
for (size_t i = 0; i < glyph_count; ++i) {
const float fi = static_cast<float>(i);
hue_phases[i] = std::fmod(fi * 1.6180339887f * 0.9f, glm::two_pi<float>());
hue_rates[i] = 0.003f + std::fmod(fi * 1.6180339887f * 0.004f, 0.009f);
}
auto hsv_to_rgb = [](float h, float s, float v) -> glm::vec3 {
const float c = v * s;
const float x = c * (1.0f - std::abs(std::fmod(h / 60.0f, 2.0f) - 1.0f));
const float m = v - c;
glm::vec3 rgb;
if (h < 60.0f)
rgb = { c, x, 0 };
else if (h < 120.0f)
rgb = { x, c, 0 };
else if (h < 180.0f)
rgb = { 0, c, x };
else if (h < 240.0f)
rgb = { 0, x, c };
else if (h < 300.0f)
rgb = { x, 0, c };
else
rgb = { c, 0, x };
return rgb + m;
};
// -------------------------------------------------------------------------
// Time accumulator and audio input.
// Live microphone input drives the global displacement amplitude.
// When the room is quiet the glyphs drift slowly. Loud input agitates them.
// -------------------------------------------------------------------------
auto t = make_persistent(0.0f);
auto audio_in = create_input_listener_buffer(0, true);
schedule_metro(1.0 / 60.0, [text_buf, layout, params, t, hue_phases, hue_rates, audio_in, glyph_count, hsv_to_rgb]() mutable {
t += 1.0f / 60.0f;
// RMS of current audio input frame drives global agitation.
const auto& samples = audio_in->get_data();
double rms = 0.0;
for (double s : samples) rms += s * s;
rms = std::sqrt(rms / std::max<size_t>(samples.size(), 1));
const float agitation = 1.0f + static_cast<float>(rms) * 18.0f;
auto quads = layout->quads;
for (size_t i = 0; i < std::min(quads.size(), glyph_count); ++i) {
auto& q = quads[i];
const auto& p = params[i];
// Independent oscillation per glyph on X, Y, and a
// quad-corner rotation approximated by differential offset.
const float dx = p.amp_x * agitation
* std::sin(p.freq_x * t + p.phase_x);
const float dy = p.amp_y * agitation
* std::sin(p.freq_y * t + p.phase_y);
const float rot = p.amp_rot * agitation
* std::sin(p.freq_rot * t + p.phase_rot);
// Apply rotation as differential corner displacement.
const float cx = (q.x0 + q.x1) * 0.5f;
const float cy = (q.y0 + q.y1) * 0.5f;
const float hw = (q.x1 - q.x0) * 0.5f;
const float hh = (q.y1 - q.y0) * 0.5f;
q.x0 = cx - hw * std::cos(rot) + hh * std::sin(rot) + dx;
q.y0 = cy - hw * std::sin(rot) - hh * std::cos(rot) + dy;
q.x1 = cx + hw * std::cos(rot) - hh * std::sin(rot) + dx;
q.y1 = cy + hw * std::sin(rot) + hh * std::cos(rot) + dy;
// Hue drift: independent rate per glyph.
hue_phases[i] += hue_rates[i];
const float hue = std::fmod(
hue_phases[i] * (180.0f / glm::pi<float>()), 360.0f);
const glm::vec3 rgb = hsv_to_rgb(hue, 0.7f, 1.0f);
Portal::Text::ink_quads(
text_buf,
std::span<const Portal::Text::GlyphQuad>(&q, 1),
{ rgb.r, rgb.g, rgb.b, 1.0f });
} }, "text_animate");
By this point in MayaFlux this should feel inevitable. Text rendering is not a special subsystem with its own animation model. A glyph is a quad in pixel space. A quad is coordinates. Coordinates are numbers.
The same audio input that drove the sampler in the first example is here driving the spatial agitation of every character on screen simultaneously. There is no text animation API. There is ink_quads, which takes a span of quads and a color. What you put in the quads before you call it is entirely yours.
WindowContainer GPU Bridge
A rendered window surface is a SignalSourceContainer. That means it is data.
Code
auto window = MayaFlux::create_window({ "source", 1920, 1080 });
window->show();
auto cont = get_io_manager()->load_video("res/Lynch.mkv");
auto video_buf = get_io_manager()->hook_video_container_to_buffer(cont);
video_buf->setup_rendering({ .target_window = window });
auto container = std::make_shared<WindowContainer>(window);
container->create_default_processor();
auto structure = container->get_structure();
const uint32_t W = static_cast<uint32_t>(structure.get_width());
const uint32_t H = static_cast<uint32_t>(structure.get_height());
const auto fmt = container->get_image_format();
auto win2 = create_window({ "history", W, H });
win2->show();
// -------------------------------------------------------------------------
// Capture buffer: rendered frames stored as VKImages at 0.75s intervals.
// The window is not a video. It is a stream of rendered surfaces.
// Each capture is a moment in that stream, addressable by index.
// -------------------------------------------------------------------------
auto history = make_persistent_shared<std::vector<std::shared_ptr<Core::VKImage>>>();
auto display_buf = vega.TextureBuffer(W, H, fmt);
auto registered = make_persistent(false);
std::this_thread::sleep_for(std::chrono::seconds(2));
schedule_metro(0.75, [container, history, display_buf, win2, ®istered]() {
container->process_default();
auto img = container->to_image();
if (!img) return;
history->emplace_back(std::move(img));
// Register and wire the display buffer on first capture.
if (!registered) {
display_buf->set_gpu_texture(history->back());
register_graphics_buffer(
display_buf, Buffers::ProcessingToken::GRAPHICS_BACKEND);
display_buf->setup_rendering({ .target_window = win2 });
registered = true;
} }, "capture");
// -------------------------------------------------------------------------
// Keypress: load a random frame from history into the display buffer.
// Space: random. Left/Right: step through history sequentially.
// -------------------------------------------------------------------------
auto cursor = make_persistent<int>(0);
on_key_pressed(window, IO::Keys::Space, [history, display_buf, &cursor]() {
if (history->empty())
return;
cursor = static_cast<int>(
get_uniform_random(0, static_cast<double>(history->size() - 1)));
display_buf->set_gpu_texture((*history)[static_cast<size_t>(cursor)]);
});
on_key_pressed(window, IO::Keys::Left, [history, display_buf, &cursor]() {
if (history->empty())
return;
cursor = std::max(0, cursor - 1);
display_buf->set_gpu_texture((*history)[static_cast<size_t>(cursor)]);
});
on_key_pressed(window, IO::Keys::Right, [history, display_buf, &cursor]() {
if (history->empty())
return;
cursor = std::min(
static_cast<int>(history->size()) - 1, cursor + 1);
display_buf->set_gpu_texture((*history)[static_cast<size_t>(cursor)]);
});
What this actually is: a render history. Every surface that has ever been drawn to a window is now addressable as a frame in a sequence, available for random access, sequential scrubbing, or any other operation you would apply to a loaded video file.
But it is not a video file. It is the live output of any rendering pipeline - particle systems, mesh networks, text, compute shaders, external video, anything that draws to a window. You can capture from multiple windows simultaneously and combine their histories.
You can feed a captured frame back into the rendering pipeline as a texture input on the next frame, closing a visual feedback loop with no special recursion API - just a TextureBuffer receiving a VKImage that came from the same window it is about to draw into.
You can sample from render history the same way GranularWorkflow samples from audio history: not sequentially, but analytically, selecting frames by any computable property of their pixel content. The surface is not the end of the pipeline. It is a node in it.
Sampling pipelines
A stream is an addressable region of data. SamplingPipeline gives you independent cursors into that region - each with its own position, speed, and loop state - all running simultaneously from a single loaded file.
Code
auto ch = create_samplers("res/audio.wav", 48000 * 30);
const auto frames = make_persistent(ch[0]->slice_from_stream().end_frame());
const auto third = make_persistent(frames / 3);
store(ch);
ch[0]->play_continuous(0, ch[0]->slice_from_range(0, third).with_looping(true));
ch[0]->play_continuous(1, ch[0]->slice_from_range(third, third * 2).with_speed(0.75).with_looping(true));
ch[1]->play_continuous(0, ch[1]->slice_from_range(third * 2, frames).with_looping(true));
ch[1]->play_continuous(1, ch[1]->slice_from_range(0, frames).with_speed(1.5).with_looping(true));
schedule_sequence({
{ 6.0, [ch, third]() {
ch[0]->load(1, ch[0]->slice_from_range(0, third / 4).with_speed(2.0).with_looping(true));
ch[0]->play(1);
} },
{ 12.0, [ch, third, frames]() {
ch[1]->load(0, ch[1]->slice_from_range(third, third * 2).with_speed(0.5).with_looping(true));
ch[1]->play(0);
} },
{ 18.0, [ch, frames]() {
ch[0]->load(0, ch[0]->slice_from_range(0, frames).with_speed(1.25).with_looping(true));
ch[0]->play(0);
ch[1]->load(1, ch[1]->slice_from_range(0, frames).with_speed(0.66).with_looping(true));
ch[1]->play(1);
} },
}, "evolve");
Musique concrète gave composers the idea that any recorded sound is raw material - that the identity of a sound is not fixed, that a recording of a train is not a train, it is a waveform, and a waveform is numbers, and numbers can be cut, reversed, layered, transposed, destroyed and rebuilt.
That idea was radical in 1948 because the tools were physical and the transformations were irreversible. Here the source file is never touched. You have cursors. You have regions. You have speed as a continuous parameter, not a pitch shifter, not a resampler with a musical interface bolted on top. You can read the same moment in a recording from four different positions simultaneously at four different rates and recombine the results into something that has no name yet.
And then at a precise moment in time, scheduled to the sample, you can redefine what every one of those cursors is reading. This is not a sampler. This is what sampling actually is when you remove the DAW from in front of it.
Granular workflow
GranularWorkflow treats a recording not as a piece of audio but as a population of moments. You define what makes a moment interesting - its energy, its spectral brightness, its internal variance - and the system reorganises the material entirely according to that definition.
Code
auto src = get_io_manager()->load_audio("res/1.wav");
auto container = std::dynamic_pointer_cast<Kakshya::SignalSourceContainer>(src);
// --- Pass 1: RMS ascending, synchronous ---
// Quietest grains first. Plays immediately.
auto rms_out = Granular::process_to_container(
container,
AnalysisType::FEATURE,
Granular::GranularConfig {
.grain_size = 1024,
.hop_size = 512,
.feature_key = "rms",
.channel = 0,
.ascending = true,
.attribution_context = ComputationContext::SPECTRAL,
},
"rms");
std::cout << "Starting phase 1\n";
get_io_manager()->hook_audio_container_to_buffers(rms_out);
// --- Pass 2: Brightness descending, Hann taper, async ---
// Source plays underneath while reconstruction runs in the background.
// Callback is the handle - fires when done, installs and starts the sampler.
std::shared_ptr<Kriya::SamplingPipeline> bright_sampler;
store(bright_sampler);
auto bright_matrix = Granular::make_granular_matrix(
ComputationContext::SPECTRAL,
Granular::GranularOutput::STREAM_ADDITIVE,
[](std::span<double> g) { Kinesis::Discrete::apply_hann(g); });
Granular::process_to_stream_async(
bright_matrix,
container,
AnalysisType::FEATURE,
// Capture by reference or pointer
[&bright_sampler](std::shared_ptr<Kakshya::DynamicSoundStream> stream) {
if (!stream)
return;
bright_sampler = create_sampler_from_stream(stream, 0);
bright_sampler->play_continuous(0, bright_sampler->slice_from_stream().with_looping(true).with_scale(0.3F));
std::cout << "Starting phase 2\n";
},
Granular::GranularConfig {
.grain_size = 2048,
.hop_size = 512,
.feature_key = "brightness",
.channel = 0,
.ascending = false,
.attribution_context = ComputationContext::SPECTRAL,
.taper = [](std::span<double> g) { Kinesis::Discrete::apply_hann(g); },
},
"brightness",
Granular::GranularOutput::STREAM_ADDITIVE);
// --- Pass 3: Variance descending, no taper, additive ---
// Most chaotic grains first. Raw amplitude buildup in overlap regions.
// Output feeds directly into the polyphonic sampler infrastructure.
auto var_matrix = Granular::make_granular_matrix(
ComputationContext::SPECTRAL,
Granular::GranularOutput::STREAM_ADDITIVE);
auto var_stream = Granular::process_to_stream(
container,
Granular::AttributeExecutor([](std::span<const double> samples, const ExecutionContext&) -> double {
if (samples.empty())
return 0.0;
StatisticalAnalyzer<std::vector<Kakshya::DataVariant>,
std::vector<Kakshya::DataVariant>>
az;
az.set_method(StatisticalMethod::VARIANCE);
std::vector<Kakshya::DataVariant> in {
Kakshya::DataVariant(std::vector<double>(samples.begin(), samples.end()))
};
auto r = az.analyze_statistics(in);
return r.channel_statistics.empty() ? 0.0 : r.channel_statistics[0].stat_variance;
}),
Granular::GranularConfig {
.grain_size = 1536,
.hop_size = 384,
.feature_key = "variance",
.channel = 0,
.ascending = false,
.attribution_context = ComputationContext::SPECTRAL,
},
Granular::GranularOutput::STREAM_ADDITIVE);
if (var_stream) {
std::cout << "Starting phase 3\n";
auto var_ch0 = create_sampler_from_stream(var_stream, 0);
auto var_ch1 = create_sampler_from_stream(var_stream, 1);
store(var_ch0);
store(var_ch1);
var_ch0->play_continuous(0, var_ch0->slice_from_stream().with_looping(true));
var_ch1->play_continuous(0, var_ch1->slice_from_stream().with_speed(0.75).with_looping(true));
}
In Max, SuperCollider, Pure Data - granular synthesis is a playback paradigm. You have a buffer, you have a phasor scrubbing through it, you have parameters: grain size, scatter, density, position randomisation, envelope shape. You tune these numbers and you get a texture. The grains are not objects. They have no identity.
They are anonymous windows into a buffer, generated and discarded at playback rate, defined entirely by where the phasor happens to be pointing when the scheduler decides to spawn one. The composition happens in the parameter space of the engine, not in the material itself. You are not working with the recording - you are working with a set of dials that determine how the engine chews through it.
Here there are no dials. There is no scrubbing phasor. There is no real-time grain scheduler making anonymous decisions about what to play next. Instead: the recording is decomposed into a population of discrete, named, characterised moments. Each grain is an object. It has a position in the source, a duration, and any number of attributes you choose to compute - spectral centroid, RMS energy, zero crossing rate, variance, anything you can express as a function from samples to a number.
Once attributed, the population can be sorted, filtered, reordered, partitioned, passed through any algorithm that operates on sequences. The order in which grains play is not a consequence of a phasor and a scatter parameter - it is the result of an explicit analytical decision about the structure of the material.
When you sort by brightness descending, you are not applying an effect. You are making an argument about the recording - that its most spectrally dense moments should be heard first, and that the reconstruction should be built from that ordering. That argument is code. It is inspectable, reproducible, composable with anything else in the system.
You can write a sort key that depends on the output of a neural network, on the position of a cursor in a completely different stream, on the result of a Vulkan compute shader that ran over the frequency data. The grain population is just data. It lives in the same substrate as everything else.
This is not a faster or more flexible granular engine. It is a different thing entirely - one that only becomes possible when you stop treating audio as a special domain with special tools and start treating it as what it actually is: numbers, addressable, composable, and open to any algorithm you care to write.
Networking
Networking in MayaFlux is not a separate subsystem you interface with. It is two things: a coroutine that fires when data arrives, and an input node that lives in the graph like any other node.
Code
// -------------------------------------------------------------------------
// NetworkSource: lock-free awaitable broadcast stream.
// Any number of coroutines can co_await the same source simultaneously.
// Each waiter gets its own copy of every message - no queue contention.
// -------------------------------------------------------------------------
auto source = std::make_shared<Vruta::NetworkSource>(
Core::EndpointInfo {
.transport = Core::NetworkTransport::UDP,
.role = Core::EndpointRole::RECEIVE,
.local_port = 9100,
});
// --- Coroutine path 1: every message ---
// co_await suspends the coroutine on the same scheduler tick as audio.
// No polling. No thread management. The coroutine resumes when bytes arrive.
auto recv_all = Kriya::on_message(source,
[](const Core::NetworkMessage& msg) {
auto osc = Portal::Network::as_osc(msg);
if (!osc)
return;
std::cout << "[all] " << osc->address
<< " val=" << osc->get_float(0).value_or(0.0f) << "\n";
});
get_event_manager()->add_event(
std::make_shared<Vruta::Event>(std::move(recv_all)));
// --- Coroutine path 2: predicate-filtered ---
// Only messages whose first float argument exceeds a threshold.
// The predicate runs in the coroutine frame, not in user code.
auto recv_loud = Kriya::on_message_matching(
source,
[](const Core::NetworkMessage& msg) {
auto osc = Portal::Network::as_osc(msg);
if (!osc)
return false;
return osc->get_float(0).value_or(0.0f) > 0.7f;
},
[](const Core::NetworkMessage& msg) {
auto osc = Portal::Network::as_osc(msg);
std::cout << "[loud] " << osc->address
<< " val=" << osc->get_float(0).value_or(0.0f) << "\n";
});
get_event_manager()->add_event(
std::make_shared<Vruta::Event>(std::move(recv_loud)));
// --- Coroutine path 3: sender-filtered ---
// Only messages from a specific machine.
auto recv_from = Kriya::on_message_from(source, "192.168.1.10",
[](const Core::NetworkMessage& msg) {
std::cout << "[from 192.168.1.10] "
<< msg.data.size() << " bytes\n";
});
get_event_manager()->add_event(
std::make_shared<Vruta::Event>(std::move(recv_from)));
// --- OSC input node path ---
// Same source, different model: address becomes a node in the signal graph.
// Composable with any other node. Value at /sensor/pressure is
// the same kind of thing as a Sine output.
auto pressure = vega.read_osc(
OSCConfig::value(),
Core::InputBinding::osc("/sensor/pressure"));
pressure->on_input([](auto& ctx) {
std::cout << "[node] /sensor/pressure val=" << ctx.value << "\n";
});
// --- Send path ---
// NetworkSink with a REALTIME_SMALL profile: low-latency, small datagrams.
// Profile abstracts transport tuning - the caller declares intent, not socket options.
auto sink = std::make_shared<Portal::Network::NetworkSink>(
Portal::Network::StreamConfig {
.name = "out",
.endpoint = { .address = "127.0.0.1", .port = 9100 },
.profile = Portal::Network::StreamProfile::REALTIME_SMALL,
.transport = Portal::Network::NetworkTransportHint::UDP,
});
auto counter = std::make_shared<uint32_t>(0);
schedule_metro(0.5, [sink, counter]() {
if (!sink->is_open()) return;
const float val = static_cast<float>((*counter)++) / 100.0f;
auto bytes = Portal::Network::serialize_osc("/sensor/pressure", {{ val }});
sink->send({ bytes.data(), bytes.size() }); }, "sender");
OSC exists because people needed a way to connect things that did not share a codebase. A sensor to a synthesiser. A phone to a laptop. A custom controller to Max. It was designed for the boundary between systems.
In MayaFlux there is no boundary. The network receive path is a coroutine that wakes when bytes arrive and hands them to whatever function you give it. The OSC node path puts a network address directly in the signal graph - the value at /sensor/pressure is a node output, the same kind of thing as a Sine or a Random, composable with everything else.
You can modulate a waveguide fundamental from a phone accelerometer with the same code you would use to modulate it from a Polynomial node. The transport is UDP. The programming model is whatever fits - callback, node, or both simultaneously on the same port. External controllers, generative systems talking to each other, multi-machine installations, live coding environments sending code over the wire - the same two primitives cover all of it.
The website at mayaflux.org carries the full design essay - MayaFlux: Indeterminate Form, Articulated Transformations - which goes into the philosophical and technical reasoning behind the decisions you have seen demonstrated here in more depth than a demo document can. If you are coming from Max, Pure Data, SuperCollider, or p5.js, the Rosetta Stone documentation series on the website maps familiar concepts to their MayaFlux equivalents without pretending they are the same thing.
MayaFlux is open source. The repository, the issue tracker, and the contributor guidelines are all on github. If you are affiliated with an institution doing research in this space - CCRMA, IEM Graz, UPF Barcelona, Sonology - get in touch