# Multi-rank send of a complex conduit node returns corrupted data (nan / garbage)

**URL:** <https://melissa.discourse.group/t/multi-rank-send-of-a-complex-conduit-node-returns-corrupted-data-nan-garbage/65>\
**Category:** Bugs and Error Reporting\
**Created:** [October 9, 2026, 10:11am UTC](https://melissa.discourse.group/t/multi-rank-send-of-a-complex-conduit-node-returns-corrupted-data-nan-garbage/65 "2026-10-09T10:11:31Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![antoncl](https://avatars.discourse-cdn.com/v4/letter/a/8797f3/32.png) [@antoncl](https://melissa.discourse.group/u/antoncl)\
**Post date:** [October 9, 2026, 10:11am UTC](https://melissa.discourse.group/t/multi-rank-send-of-a-complex-conduit-node-returns-corrupted-data-nan-garbage/65/1 "2026-10-09T10:11:31Z")

</div>

## What I’m seeing

I’m using the C++ client API (`melissa::send(node_name, conduit::Node&)`) and sending one node per timestep. The node has three leaves:

| Leaf | Type | How it’s set |
| --- | --- | --- |
| `time` | float64 scalar | by value (`node["time"] = t;`) |
| `p` | float64 array (~N elements) | external pointer (`set_external_float64_ptr`) |
| `U` | float64 array (~3·N elements) | external pointer (`set_external_float64_ptr`) |

On a single rank, everything arrives correctly. On more than one MPI rank, the payload reaching the server is corrupted: `p` comes through as `nan`, and even `time` shows up as nonsense floats (e.g. `7.27e-321`, `4.67e-310`, `1.78e+161`).

In this case `time` is set by value, not via an external pointer, so I wouldn’t expect it to be affected. But since the whole gathered sub-node looks bad, I suspect something goes wrong during assembly.

I also checked v3.1.0 and current `develop`, and the relevant code seems unchanged, so I don’t think this is already fixed, though I could be mistaken.

## Where I think the problem might be

In `api/comm.cpp`, function `gather_and_process_payloads`. The local `gathered_payloads` is declared at line 381:

```cpp
conduit::Node gathered_payloads; // line 381

try
{
    // Use Conduit's MPI relay to pull everything into an
    // anonymous sequence on Rank 0
    conduit::relay::mpi::gather_using_schema(
        local_payload, gathered_payloads, 0, session.mpi.comm); // lines 388-389
}
catch (const conduit::Error& e)
{
    ...
}

```

and then, for a complex schema, the nesting happens at lines 462–468:

```cpp
// If layout is a complex schema, map using zero-copy set_external handles
for (int i = 0; i < session.mpi.comm_size; ++i)
{
    std::string path_key = "r" + std::to_string(i);
    payload_to_send[path_key].set_external(gathered_payloads.child(i));
}

return Result<void>::ok();

```

My reading is that `gathered_payloads` is a local variable, and `set_external` doesn’t copy, it just points into it. So when the function returns and `gathered_payloads` goes out of scope, `payload_to_send` would hold pointers to freed memory, which is then serialized later in `send_user_node`. That would explain both the garbage values and why even the by-value `time` inside each rank’s sub-node is affected.

For contrast, the single-rank branch (lines 375–379) uses `set_external` too, but there `local_payload` stays alive for the duration of the call which matches the fact that single-rank works:

```cpp
// If single ranked, then simply send that
if (session.mpi.comm_size == 1)
{
    payload_to_send.set_external(local_payload);
    return Result<void>::ok();
}

```

I’m not fully confident this is the actual cause though. Maybe `set_external` behaves differently than I assume, or the issue is elsewhere. Happy to be corrected.

## How to reproduce

- Client sends a conduit node with multiple leaves per timestep (mix of by-value scalars and external-pointer arrays).
- Run the client with \>1 MPI rank.
- On the server, read the nested `node["r0"]/...` → nan/garbage. Same code with a single rank → correct.

---

<div class="post-metadata">

**Author:** ![AbhishekP](https://avatars.discourse-cdn.com/v4/letter/a/aeb1de/32.png) [@AbhishekP](https://melissa.discourse.group/u/AbhishekP)\
**Post date:** [October 12, 2026, 11:31am UTC](https://melissa.discourse.group/t/multi-rank-send-of-a-complex-conduit-node-returns-corrupted-data-nan-garbage/65/2 "2026-10-12T11:31:38Z")

</div>

Thanks for cathing the bug!

It was indeed corrupted because of `gathered_payload` being a local variable.

The fix is merged in the `develop`.