Visual Materiality Part I

PointCollectionNode is a GeometryWriterNode that manages multiple PointVertex entries in a single vertex buffer.

class PointCollectionNode : public GeometryWriterNode {
private:
    std::vector m_points;  // CPU-side point storage
};
            

Key methods:

points->add_point(PointVertex{position, color, size}); // Append
points->set_points(std::vector<PointVertex>); // Replace all
points->update_point(size_t index, PointVertex); // Modify one
points->clear_points(); // Empty
            

Each modification sets m_vertex_data_dirty = true. Next frame, compute_frame() uploads the entire collection:

void PointCollectionNode::compute_frame() {
{
    if (m_points.empty()) {
        resize_vertex_buffer(0);
        return;
    }

    // Copy all points to flat vertex buffer
    set_vertices(std::span{m_points.data(), m_points.size()});

    // Update vertex layout metadata
    auto layout = get_vertex_layout();
    layout->vertex_count = m_points.size();
    set_vertex_layout(*layout);
}
            

Why "unstructured"?

Points have no relationships. They're just a list of positions. No topology, no connectivity, no physics.

For particle systems with relationships (springs, forces, neighborhoods), use ParticleNetwork instead (covered in a future tutorial).

When to use PointCollectionNode:

• Static data visualization (plot 10,000 data points)
• Debug markers (show algorithm execution paths)
• Procedural forms (generated geometry like this spiral)
• Any collection where points don't interact

Compare two approaches:

Inefficient (Section 1 pattern repeated):

for (int i = 0; i < 200; i++) {
    auto point = vega.PointNode(positions[i]) | Graphics;
    auto buffer = vega.GeometryBuffer(point) | Graphics;
    buffer->setup_rendering({.target_window = window});
}
            

Result:

• 200 buffers
• 200 upload operations per frame
• 200 draw calls
• Massive CPU→GPU bandwidth waste

Efficient (current section):

auto points = vega.PointCollectionNode() | Graphics;
for (int i = 0; i < 200; i++) {
    points->add_point({positions[i], colors[i], 8.0f});
}
auto buffer = vega.GeometryBuffer(points) | Graphics;
buffer->setup_rendering({.target_window = window});
            

Result:

• 1 buffer
• 1 upload operation per frame (only if dirty)
• 1 draw call: flow.draw(cmd_id, 200)
• Minimal overhead

How GeometryBindingsProcessor batches:

void GeometryBindingsProcessor::processing_function(Buffer* buffer) {
    // For EACH bound geometry node (but we only have one):
    auto vertices = binding.node->get_vertex_data();  // All 200 points

    size_t upload_size = vertices.size_bytes();  // 200 * sizeof(PointVertex)

    upload_to_gpu(
        vertices.data(),
        upload_size,
        binding.gpu_vertex_buffer,
        binding.staging_buffer  // If GPU buffer is device-local
    );
}
            

It uploads the entire vertex array as a contiguous block. GPU reads it sequentially, renders all points in one dispatch.

This is fundamental to real-time graphics: minimize state changes, batch identical operations, upload contiguous data.

You've been working with GeometryBuffer, a specialized buffer for vertex data. But there's a hidden orchestrator: RootGraphicsBuffer.

The Graphics Subsystem runs its own thread:

void GraphicsSubsystem::graphics_thread_loop() {
    while (m_running.load()) {
        m_frame_clock->tick();  // Advance to next frame (60 FPS)

        // 1. Process visual-rate nodes (PointCollectionNode::compute_frame())
        m_handle->nodes.process(1);

        // 2. Process all graphics buffers
        m_handle->buffers.process(1);  // <-- This processes RootGraphicsBuffer

        // 3. Window management
        m_handle->windows.process();

        m_frame_clock->wait_for_next_frame();
    }
}
            

Inside m_handle->buffers.process(1), RootGraphicsBuffer executes:

class RootGraphicsBuffer : public Buffer {
private:
    std::vector> m_child_buffers;  // Your GeometryBuffer is here
    std::shared_ptr m_default_processor;
    std::shared_ptr m_final_processor;
};
            

The processing sequence:

1. GraphicsBatchProcessor (default processor) runs first:
   • Iterates through all child buffers (your GeometryBuffer)
   • Calls each buffer's default processor (GeometryBindingsProcessor - uploads vertices)
   • Calls each buffer's processing chain (RenderProcessor - records draw commands)
   • Collects buffers that have render commands ready
2. PresentProcessor (final processor) runs last:
   • Receives the list of buffers ready to render
   • Groups buffers by target window
   • For each window: creates primary command buffer, executes secondary command buffers, presents frame

GraphicsBatchProcessor code (simplified):

void GraphicsBatchProcessor::processing_function(Buffer* buffer) {
    auto root_buf = std::dynamic_pointer_cast(buffer);

    // Process all child buffers
    for (auto& ch_buffer : root_buf->get_child_buffers()) {
        // Upload vertices (GeometryBindingsProcessor)
        if (ch_buffer->has_default_processor()) {
            ch_buffer->process_default();
        }

        // Run processing chain (RenderProcessor records commands)
        if (ch_buffer->has_processing_chain()) {
            ch_buffer->get_processing_chain()->process(ch_buffer);
        }

        // If buffer has render pipeline, mark it renderable
        if (auto vk_buffer = std::dynamic_pointer_cast(ch_buffer)) {
            if (vk_buffer->has_render_pipeline()) {
                    RootGraphicsBuffer::RenderableBufferInfo info;
                    info.buffer = vk_buffer;
                    info.target_window = window;
                    info.pipeline_id = id;
                    info.command_buffer_id = vk_buffer->get_pipeline_command(id);

                    root_buf->add_renderable_buffer(info);
                });
            }
        }
    }
}
            

PresentProcessor fallback renderer (simplified):

void PresentProcessor::fallback_renderer(RootGraphicsBuffer* root) {
    // Group buffers by window
    std::unordered_map> buffers_by_window;

    for (const auto& renderable : root->get_renderable_buffers()) {
        buffers_by_window[renderable.target_window].push_back(renderable);
    }

    // Render each window
    for (const auto& [window, buffer_infos] : buffers_by_window) {
        // Begin dynamic rendering
        auto primary_cmd = foundry.begin_commands(GRAPHICS);
        flow.begin_rendering(primary_cmd, window, swapchain_image);

        // Execute all secondary command buffers for this window
        std::vector secondaries;
        for (const auto& info : buffer_infos) {
            secondaries.push_back(info.command_buffer);
        }
        primary_cmd.executeCommands(secondaries);

        flow.end_rendering(primary_cmd, window);

        // Submit and present
        display_service->submit_and_present(window, primary_cmd);
    }
}
            

Why this architecture?

• Separation: Upload (GeometryBindingsProcessor) and rendering (RenderProcessor) are distinct
• Batching: All uploads happen first, then all rendering
• Multi-window: Same geometry can render to multiple windows
• No draw loop: You never write while (!should_close()) - the subsystem handles timing

MayaFlux uses Vulkan 1.3 dynamic rendering, which means no VkRenderPass objects.

Traditional Vulkan (pre-1.3):

• Create VkRenderPass ahead of time
• Specify all attachments (color, depth) statically
• Pipeline tied to render pass compatibility
• Inflexible: changing attachments requires new pipelines

Dynamic Rendering (Vulkan 1.3+):

• No VkRenderPass objects
• Call vkCmdBeginRendering() with inline attachment info
• Specify attachments per-frame dynamically
• Flexible: same pipeline works with different attachments

RenderFlow::begin_rendering() implementation:

void RenderFlow::begin_rendering(
    CommandBufferID cmd_id,
    Window* window,
    vk::Image swapchain_image,
    const std::array& clear_color)
{
    auto cmd = foundry.get_command_buffer(cmd_id);

    // Inline color attachment info
    vk::RenderingAttachmentInfo color_attachment;
    color_attachment.imageView = swapchain_image_view;
    color_attachment.imageLayout = vk::ImageLayout::eColorAttachmentOptimal;
    color_attachment.loadOp = vk::AttachmentLoadOp::eClear;
    color_attachment.storeOp = vk::AttachmentStoreOp::eStore;
    color_attachment.clearValue.color = vk::ClearColorValue(std::array{
        clear_color[0], clear_color[1], clear_color[2], clear_color[3]
    });

    // Dynamic rendering info
    vk::RenderingInfo rendering_info;
    rendering_info.renderArea = {{0, 0}, {width, height}};
    rendering_info.layerCount = 1;
    rendering_info.colorAttachmentCount = 1;
    rendering_info.pColorAttachments = &color_attachment;

    // Begin rendering - no render pass object!
    cmd.beginRendering(rendering_info);
}
            

Why this matters:

• Post-processing chains: Render to texture A, then B, then screen (no pipeline recreation)
• Multi-pass rendering: Different attachments per pass without render pass combinatorial explosion (automatic because of dynamic rendering)
• Flexibility: Attachments decided at runtime, not compile time

From your perspective: Invisible. RenderProcessor handles it. But it enables advanced techniques later without architectural changes.

Typical graphics framework (Processing, openFrameworks):

// Global state (forced pattern)
std::vector<Point> points;

void setup() {
    // Initialize once
}

void draw() {
    background(0);

    // MUST redraw everything, every frame, forever
    for (auto& p : points) {
        circle(p.x, p.y, 10);
    }

    // Update state for next frame
    updatePhysics();
}

// Hidden main loop you don't control:
while (running) {
    draw();      // Called automatically at framerate
    swap_buffers();
}
      

Problems:

1. Forced void: draw() is a mandatory callback you're trapped in
2. Global state: Everything must be accessible from both setup() and draw()
3. No separation: Update and render are coupled in draw()
4. No timing control: Can't easily say "update every 2 frames" or "render at 120 FPS, update at 30 FPS"

MayaFlux:

void compose() {
    // Setup happens once
    auto points = vega.PointCollectionNode() | Graphics;
    auto buffer = vega.GeometryBuffer(points) | Graphics;
    buffer->setup_rendering({ .target_window = window });

    // Schedule updates independently
    MayaFlux::schedule_metro(0.033, [points]() {  // 30 Hz updates
        points->add_point({/* ... */});
    });

    // Rendering happens at 60 FPS automatically in graphics thread
    // No coupling, no forced structure
}
      

compose() is not a loop. It runs once at startup. You declare structure, not repeated execution.

Updates are scheduled, not looped: You schedule tasks that run at specific intervals using the task scheduler. These run independently from rendering.

Rendering is automatic: The graphics subsystem thread runs at 60 FPS, processes visual-rate nodes, uploads geometry, issues draw calls, presents frames. You never write that loop.

Because each buffer targets a specific window through setup_rendering(), you don't need to offset or partition your geometry. Just create separate content for separate windows:

void compose() {
    auto window1 = MayaFlux::create_window({ "Spiral", 800, 600 });
    auto window2 = MayaFlux::create_window({ "Grid", 800, 600 });

    // Spiral in window 1
    auto spiral = vega.PointCollectionNode() | Graphics;
    auto spiral_buffer = vega.GeometryBuffer(spiral) | Graphics;
    spiral_buffer->setup_rendering({ .target_window = window1 });

    // Grid in window 2
    auto grid = vega.PointCollectionNode() | Graphics;
    auto grid_buffer = vega.GeometryBuffer(grid) | Graphics;
    grid_buffer->setup_rendering({ .target_window = window2 });

    window1->show();
    window2->show();

    // Update spiral
    MayaFlux::schedule_metro(0.016, [spiral]() {
        /* add spiral points */
    });

    // Update grid independently
    MayaFlux::schedule_metro(0.033, [grid]() {
        /* add grid points */
    });
}
      

What PresentProcessor does:

// Groups buffers by target window
std::unordered_map<Window*, std::vector<BufferInfo>> buffers_by_window;
buffers_by_window[window1].push_back(spiral_buffer_info);
buffers_by_window[window2].push_back(grid_buffer_info);

// Each window gets its own rendering pass
for (auto& [window, buffer_infos] : buffers_by_window) {
    begin_dynamic_rendering(window);
    for (auto& info : buffer_infos) {
        execute_secondary_command_buffer(info.cmd_buffer);
    }
    end_dynamic_rendering(window);
    present(window);
}
      

Key point: You're not rendering to one framebuffer and offsetting content. Each window is a separate rendering target. No manual viewport management, no draw call coordination.

Traditional frameworks force you to either:

• Render everything to one window and manually partition
• Manage multiple OpenGL contexts yourself
• Fight the framework's assumptions

MayaFlux: just create another window, point a buffer at it. Done.

MayaFlux gives you three ways to schedule updates, each for different use cases:

Approach 1: schedule_metro (Simplest)

MayaFlux::schedule_metro(interval_seconds, callback);
      

Schedules a callback to run at fixed intervals using the TaskScheduler:

MayaFlux::schedule_metro(0.016, [points]() {  // ~60 Hz
    points->add_point({/* ... */});
});
      

Under the hood:

// Creates a SoundRoutine coroutine
auto metro_routine = [](Vruta::TaskScheduler& scheduler) -> Vruta::SoundRoutine {
    auto& promise = co_await Kriya::GetAudioPromise{};
    uint64_t interval_samples = scheduler.seconds_to_samples(interval_seconds);

    while (!promise.should_terminate) {
        callback();  // Your function
        co_await Kriya::SampleDelay{interval_samples};
    }
};

scheduler->add_task(std::make_shared<Vruta::SoundRoutine>(metro_routine(*scheduler)));
      

When to use: Simple periodic updates. Most common case.

Approach 2: Coroutines with Sequence or EventChain

EventChain for sequential timed events:

Kriya::EventChain()
    .then([]() { points->clear_points(); })
    .then([]() {
        for (int i = 0; i < 50; i++) {
            points->add_point({/* ... */});
        }
    }, 0.5)  // After 0.5 seconds
    .then([]() { /* do something else */ }, 1.0)  // After another 1.0 seconds
    .start();
      

Under the hood:

auto coroutine_func = [](Vruta::TaskScheduler& scheduler, EventChain* chain) -> Vruta::SoundRoutine {
    for (const auto& event : chain->m_events) {
        co_await Kriya::SampleDelay{scheduler.seconds_to_samples(event.delay_seconds)};
        event.action();  // Execute callback
    }
};
      

When to use: Multi-step animations with specific timing. State machines. Choreographed sequences.

Note: Sequence is similar but less commonly used for graphics. See project knowledge for details.

Approach 3: Node on_tick Callbacks

Every node can register callbacks that fire whenever it processes a sample:

points->on_tick([points](const Nodes::NodeContext& ctx) {
    static int frame_count = 0;
    if (++frame_count % 60 == 0) {  // Every 60 calls
        points->add_point({
            glm::vec3(
                MayaFlux::get_uniform_random(-1.0f, 1.0f),
                MayaFlux::get_uniform_random(-1.0f, 1.0f),
                0.0f
            ),
            glm::vec3(1.0f),
            10.0f
        });
    }
});
      

When the callback fires:

// Inside PointCollectionNode::compute_frame() or process_sample():
void notify_tick(double value) override {
    m_last_context = create_context(value);

    // Unconditional callbacks
    for (auto& callback : m_callbacks) {
        callback(*m_last_context);
    }

    // Conditional callbacks
    for (auto& [callback, condition] : m_conditional_callbacks) {
        if (condition(*m_last_context)) {
            callback(*m_last_context);
        }
    }
}
      

NodeContext contains:

struct NodeContext {
    double value;           // Current output value
    std::string type_id;    // Node type identifier
    // Derived classes add more info (e.g., FilterContext adds history buffers)
};
      

When to use: Tight coupling with node processing. Sample-accurate reactions. Rare for graphics (more common for audio analysis driving visual changes).

Comparison:

Pattern 1: Additive Growth (Original Example)

MayaFlux::schedule_metro(0.016, [points]() {
    points->add_point({/* new point */});  // Grows indefinitely
});
      

Points accumulate until cleared. Trail effects, particle emissions, growing forms.

Pattern 2: Full Replacement

MayaFlux::schedule_metro(0.016, [points]() {
    std::vector<Nodes::GpuSync::PointVertex> new_points;

    // Generate entirely new point set
    for (int i = 0; i < 100; i++) {
        new_points.push_back({/* compute position */});
    }

    points->set_points(new_points);  // Replace all
});
      

Internally:

void PointCollectionNode::set_points(const std::vector<PointVertex>& points) {
    m_points.clear();
    m_points = points;
    m_vertex_data_dirty = true;  // Triggers upload next frame
}
      

Entire buffer replaced. Simulations that recompute all positions (physics, flocking).

Pattern 3: Selective Updates

MayaFlux::schedule_metro(0.016, [points]() {
    for (size_t i = 0; i < points->get_point_count(); i++) {
        auto p = points->get_point(i);

        // Modify position
        p.position.x += std::sin(i * 0.1f) * 0.01f;
        p.position.y += std::cos(i * 0.1f) * 0.01f;

        points->update_point(i, p);  // Updates single point
    }
});
      

Modifies existing points in place. Mesh deformations, vertex animations.

Pattern 4: Conditional Clearing

static float radius = 0.0f;

MayaFlux::schedule_metro(0.016, [points]() {
    radius += 0.01f;

    if (radius > 1.5f) {
        points->clear_points();  // Reset
        radius = 0.0f;
        return;
    }

    points->add_point({/* ... */});
});
      

Growth with periodic reset. Cyclical animations.

You've worked with individual Nodes i.e sample-by-sample processors. NodeNetworks coordinate many nodes with relationships between them.

Available networks:

• ParticleNetwork: N-body physics simulation (particles with forces, velocities, collisions)
• ModalNetwork: Physical modeling (resonant modes, coupled oscillators)

ParticleNetwork treats each particle as a PointNode (geometry) with physics state:

struct Particle {
    std::shared_ptr<PointNode> point;  // Position, color, size
    glm::vec3 velocity;
    glm::vec3 force;
    float mass;
};
      

Every frame, physics integrates: forces → velocities → positions. The PointNode positions update. GPU receives new vertex data.

When you write:

particles->map_parameter("gravity", envelope, NodeNetwork::MappingMode::BROADCAST);
      

MayaFlux stores the mapping. Each physics step, ParticleNetwork::update_mapped_parameters() reads from the node:

void ParticleNetwork::update_mapped_parameters() {
    for (const auto& mapping : m_parameter_mappings) {
        if (mapping.mode == MappingMode::BROADCAST && mapping.broadcast_source) {
            double value = mapping.broadcast_source->get_last_output();
            apply_broadcast_parameter(mapping.param_name, value);
        }
    }
}
      

The key: Nodes have get_last_output(), a single queryable value from their most recent sample. The particle network runs at visual rate (60Hz), the node runs at audio rate (48kHz). The network simply reads whatever value the node last computed.

No conversion, no special routing. The node doesn't know it's controlling particles. The particles don't know the value came from audio synthesis. Numbers flow through the same substrate.

NetworkGeometryBuffer does for particle networks what PointCollectionNode did for manual points: aggregate many vertices into one GPU upload.

auto buffer = std::make_shared<NetworkGeometryBuffer>(particles);
buffer->setup_processors(ProcessingToken::GRAPHICS_BACKEND);
buffer->setup_rendering({ .target_window = window });
      

Each frame:

1. NodeGraphManager calls particles->process_batch(1) (physics step)
2. Physics updates: forces → velocities → positions
3. NetworkGeometryProcessor extracts all 500 PointNode positions
4. Uploads to GPU as single contiguous vertex buffer
5. One draw call renders everything

Without aggregation, 500 particles would mean 500 buffers, 500 uploads, 500 draw calls. That's GPU-hostile. NetworkGeometryBuffer batches automatically.