feat: import Chinese-localized Buzz source snapshot
Docker image / Build (linux/amd64) (push) Has been cancelled
Docker image / Build (linux/arm64) (push) Has been cancelled
Docker image / Merge release multi-arch manifest (push) Has been cancelled
Docker image / Merge debug multi-arch manifest (push) Has been cancelled
Docker image / Build public push gateway (linux/amd64) (push) Has been cancelled
Docker image / Build public push gateway (linux/arm64) (push) Has been cancelled
Docker image / Publish public push gateway image (push) Has been cancelled
Sprig image / Build (linux/amd64) (push) Has been cancelled
Sprig image / Build (linux/arm64) (push) Has been cancelled
Sprig image / Merge multi-arch manifest (push) Has been cancelled
Harbor Buzz Orchestra / Python tests and lint (push) Has been cancelled
CI / Detect Changed Paths (push) Has been cancelled
CI / Rust Lint (push) Has been cancelled
CI / Unit Tests (push) Has been cancelled
CI / Desktop Core (push) Has been cancelled
CI / Desktop Smoke E2E (1) (push) Has been cancelled
CI / Desktop Smoke E2E (2) (push) Has been cancelled
CI / Desktop Smoke E2E (3) (push) Has been cancelled
CI / Desktop Smoke E2E (4) (push) Has been cancelled
CI / Desktop (push) Has been cancelled
CI / Desktop E2E Relay (push) Has been cancelled
CI / Desktop E2E Integration (1/2) (push) Has been cancelled
CI / Desktop E2E Integration (2/2) (push) Has been cancelled
CI / Desktop E2E Integration (push) Has been cancelled
CI / Backend Integration (relay e2e) (push) Has been cancelled
CI / Relay E2E (push) Has been cancelled
CI / Web (push) Has been cancelled
CI / Mobile (push) Has been cancelled
CI / Security (push) Has been cancelled
CI / Dead Token Reference Guard (push) Has been cancelled
CI / Server Cross-Compile (aarch64-unknown-linux-musl) (push) Has been cancelled
CI / Server Cross-Compile (x86_64-unknown-linux-musl) (push) Has been cancelled
CI / Windows Rust (x86_64-pc-windows-msvc) (push) Has been cancelled
CI / Desktop Build (macOS) (push) Has been cancelled
helm chart / lint + unittest + render matrix (push) Has been cancelled
helm chart / install on kind (gated) (push) Has been cancelled
helm chart / publish chart to GHCR (push) Has been cancelled
Mesh Lifecycle / Relay-Driven Mesh Lifecycle Smoke (push) Has been cancelled
Sprig / Build (aarch64-unknown-linux-musl) (push) Has been cancelled
Sprig / Build (x86_64-unknown-linux-musl) (push) Has been cancelled
Sprig / Publish rolling release (push) Has been cancelled
Sprig / Publish tagged release (push) Has been cancelled
Docker image / Build (linux/amd64) (push) Has been cancelled
Docker image / Build (linux/arm64) (push) Has been cancelled
Docker image / Merge release multi-arch manifest (push) Has been cancelled
Docker image / Merge debug multi-arch manifest (push) Has been cancelled
Docker image / Build public push gateway (linux/amd64) (push) Has been cancelled
Docker image / Build public push gateway (linux/arm64) (push) Has been cancelled
Docker image / Publish public push gateway image (push) Has been cancelled
Sprig image / Build (linux/amd64) (push) Has been cancelled
Sprig image / Build (linux/arm64) (push) Has been cancelled
Sprig image / Merge multi-arch manifest (push) Has been cancelled
Harbor Buzz Orchestra / Python tests and lint (push) Has been cancelled
CI / Detect Changed Paths (push) Has been cancelled
CI / Rust Lint (push) Has been cancelled
CI / Unit Tests (push) Has been cancelled
CI / Desktop Core (push) Has been cancelled
CI / Desktop Smoke E2E (1) (push) Has been cancelled
CI / Desktop Smoke E2E (2) (push) Has been cancelled
CI / Desktop Smoke E2E (3) (push) Has been cancelled
CI / Desktop Smoke E2E (4) (push) Has been cancelled
CI / Desktop (push) Has been cancelled
CI / Desktop E2E Relay (push) Has been cancelled
CI / Desktop E2E Integration (1/2) (push) Has been cancelled
CI / Desktop E2E Integration (2/2) (push) Has been cancelled
CI / Desktop E2E Integration (push) Has been cancelled
CI / Backend Integration (relay e2e) (push) Has been cancelled
CI / Relay E2E (push) Has been cancelled
CI / Web (push) Has been cancelled
CI / Mobile (push) Has been cancelled
CI / Security (push) Has been cancelled
CI / Dead Token Reference Guard (push) Has been cancelled
CI / Server Cross-Compile (aarch64-unknown-linux-musl) (push) Has been cancelled
CI / Server Cross-Compile (x86_64-unknown-linux-musl) (push) Has been cancelled
CI / Windows Rust (x86_64-pc-windows-msvc) (push) Has been cancelled
CI / Desktop Build (macOS) (push) Has been cancelled
helm chart / lint + unittest + render matrix (push) Has been cancelled
helm chart / install on kind (gated) (push) Has been cancelled
helm chart / publish chart to GHCR (push) Has been cancelled
Mesh Lifecycle / Relay-Driven Mesh Lifecycle Smoke (push) Has been cancelled
Sprig / Build (aarch64-unknown-linux-musl) (push) Has been cancelled
Sprig / Build (x86_64-unknown-linux-musl) (push) Has been cancelled
Sprig / Publish rolling release (push) Has been cancelled
Sprig / Publish tagged release (push) Has been cancelled
Signed-off-by: cls_宁波本机 <908705107@qq.com>
This commit is contained in:
@@ -0,0 +1,452 @@
|
||||
//! Owner-allowlist admission smoke test — three nodes, one per process.
|
||||
//!
|
||||
//! Proves the security claim of Buzz's membership-derived mesh admission:
|
||||
//! possession of a dial pointer (invite token / EndpointAddr) admits nobody;
|
||||
//! only owners on the serve node's allowlist join the mesh.
|
||||
//!
|
||||
//! 1. SERVE process: hosts a GGUF with `trust_policy(Allowlist)` +
|
||||
//! `owner_required(true)`, trusting exactly one other owner id — the
|
||||
//! same builder calls Buzz desktop makes with a resolved member roster.
|
||||
//! 2. TRUSTED process: client presenting the allowlisted owner key, joins
|
||||
//! via the invite token, must see the routed model; the orchestrator
|
||||
//! then drives a real inference through it.
|
||||
//! 3. NON-MEMBER process: client presenting a different owner key with the
|
||||
//! same connection information. It must not be admitted or route inference.
|
||||
//!
|
||||
//! One process per node is load-bearing: mesh-llm keeps process-global state
|
||||
//! (ownership attestation at ~/.mesh-llm/node-ownership.json, tracing, the
|
||||
//! output sink), so multiple owner-keyed embedded nodes in one process
|
||||
//! corrupt each other — exactly how Buzz runs it in production anyway (one
|
||||
//! desktop = one node).
|
||||
//!
|
||||
//! Hardware-gated, not CI — loads a real model. Run with:
|
||||
//! cargo run -p buzz-relay --example mesh_admission_smoke
|
||||
use std::io::BufRead;
|
||||
use std::process::{Child, Command, Stdio};
|
||||
use std::time::Duration;
|
||||
|
||||
use mesh_llm_host_runtime::crypto::{save_keystore, OwnerKeypair};
|
||||
use mesh_llm_sdk::{client, serve, MeshDiscoveryMode, TrustPolicy};
|
||||
|
||||
const DEFAULT_MODEL: &str = "jc-builds/SmolLM2-135M-Instruct-Q4_K_M-GGUF:Q4_K_M";
|
||||
|
||||
const SERVE_API_PORT: u16 = 19347;
|
||||
const SERVE_CONSOLE_PORT: u16 = 13141;
|
||||
const TRUSTED_API_PORT: u16 = 19348;
|
||||
const TRUSTED_CONSOLE_PORT: u16 = 13142;
|
||||
const STRANGER_API_PORT: u16 = 19349;
|
||||
const STRANGER_CONSOLE_PORT: u16 = 13143;
|
||||
|
||||
/// The trusted client sees the model within seconds on one box; this bounds
|
||||
/// the stranger's chance to (fail to) see it.
|
||||
const TRUSTED_WINDOW_SECS: u64 = 120;
|
||||
const STRANGER_WINDOW_SECS: u64 = 45;
|
||||
|
||||
fn main() -> anyhow::Result<()> {
|
||||
match std::env::var("MESH_ROLE").ok().as_deref() {
|
||||
Some("serve") => tokio::runtime::Runtime::new()?.block_on(role_serve()),
|
||||
Some("client") => tokio::runtime::Runtime::new()?.block_on(role_client()),
|
||||
_ => orchestrate(),
|
||||
}
|
||||
}
|
||||
|
||||
fn env(name: &str) -> anyhow::Result<String> {
|
||||
std::env::var(name).map_err(|_| anyhow::anyhow!("{name} is required for this role"))
|
||||
}
|
||||
|
||||
async fn init_native_runtime() -> anyhow::Result<()> {
|
||||
let cache = mesh_llm_sdk::native_runtime::native_runtime_cache(None)?;
|
||||
let current = mesh_llm_sdk::native_runtime::CURRENT_MESH_VERSION;
|
||||
if !cache
|
||||
.installed()?
|
||||
.iter()
|
||||
.any(|runtime| runtime.mesh_version == current)
|
||||
{
|
||||
anyhow::bail!("MeshLLM native runtime for MeshLLM {current} is not installed; run `just mesh-e2e-hardware` once to prepare it");
|
||||
}
|
||||
mesh_llm_host_runtime::initialize_host_runtime()
|
||||
.await
|
||||
.map_err(|error| anyhow::anyhow!("MeshLLM host runtime init failed: {error}"))
|
||||
}
|
||||
|
||||
/// SERVE role: allowlist serve node. Prints `INVITE:<token>` then
|
||||
/// `READY:<model>` on stdout, then parks until the orchestrator kills it.
|
||||
async fn role_serve() -> anyhow::Result<()> {
|
||||
init_native_runtime().await?;
|
||||
let model = env("MESH_SMOKE_MODEL")?;
|
||||
let owner_key = env("MESH_OWNER_KEY")?;
|
||||
let trust_owners: Vec<String> = env("MESH_TRUST_OWNERS")?
|
||||
.split(',')
|
||||
.map(str::to_string)
|
||||
.collect();
|
||||
|
||||
let cfg = serve::EmbeddedServeConfig::builder()
|
||||
.model(&model)
|
||||
.api_port(SERVE_API_PORT)
|
||||
.console_port(SERVE_CONSOLE_PORT)
|
||||
.publish(true)
|
||||
.auto_join(false)
|
||||
.discovery_mode(MeshDiscoveryMode::Mdns)
|
||||
.console_ui(true)
|
||||
.startup_timeout(Duration::from_secs(600))
|
||||
.owner_key(owner_key)
|
||||
.owner_required(true)
|
||||
.trust_policy(TrustPolicy::Allowlist)
|
||||
.trust_owners(trust_owners)
|
||||
// Require signed bootstrap tokens so owner admission is enforced from
|
||||
// the first connection attempt.
|
||||
.signed_join_tokens(true)
|
||||
.build();
|
||||
let node = serve::start(cfg).await?;
|
||||
let invite = node
|
||||
.invite_token()
|
||||
.map(str::to_string)
|
||||
.ok_or_else(|| anyhow::anyhow!("serve node produced no invite token"))?;
|
||||
println!("INVITE:{invite}");
|
||||
|
||||
let http = reqwest::Client::new();
|
||||
let base = node.api_base_url().to_string();
|
||||
match wait_for_model(&http, &base, Duration::from_secs(600)).await? {
|
||||
Some(model) => println!("READY:{model}"),
|
||||
None => anyhow::bail!("serve node never loaded the model"),
|
||||
}
|
||||
// Park; the orchestrator kills this process when the run is over.
|
||||
loop {
|
||||
tokio::time::sleep(Duration::from_secs(3600)).await;
|
||||
}
|
||||
}
|
||||
|
||||
/// CLIENT role (trusted or stranger — the key decides). Joins via the invite
|
||||
/// token, polls its own /models for the window, prints `SEEN:<model>` or
|
||||
/// `NONE`, stops the node, exits 0.
|
||||
async fn role_client() -> anyhow::Result<()> {
|
||||
init_native_runtime().await?;
|
||||
let owner_key = env("MESH_OWNER_KEY")?;
|
||||
let join_token = env("MESH_JOIN_TOKEN")?;
|
||||
let api_port: u16 = env("MESH_API_PORT")?.parse()?;
|
||||
let console_port: u16 = env("MESH_CONSOLE_PORT")?.parse()?;
|
||||
let window_secs: u64 = env("MESH_WINDOW_SECS")?.parse()?;
|
||||
|
||||
let cfg = client::EmbeddedClientConfig::builder()
|
||||
.api_port(api_port)
|
||||
.console_port(console_port)
|
||||
.publish(false)
|
||||
.auto_join(false)
|
||||
.discovery_mode(MeshDiscoveryMode::Mdns)
|
||||
.join_token(&join_token)
|
||||
.startup_timeout(Duration::from_secs(180))
|
||||
.console_ui(true)
|
||||
// Present the owner attestation; owner_required makes a key-load
|
||||
// failure abort loudly instead of silently starting unattested
|
||||
// (which the allowlist serve would then reject as NoAttestation).
|
||||
.owner_key(owner_key)
|
||||
.owner_required(true)
|
||||
.build();
|
||||
let node = client::start(cfg).await?;
|
||||
let http = reqwest::Client::new();
|
||||
let base = node.api_base_url().to_string();
|
||||
let seen = wait_for_model(&http, &base, Duration::from_secs(window_secs)).await?;
|
||||
match &seen {
|
||||
Some(model) => {
|
||||
println!("SEEN:{model}");
|
||||
// Visibility is gossip; admission is routing. The decisive probe
|
||||
// is whether an inference actually routes through the mesh.
|
||||
match try_completion(&http, &base, model).await {
|
||||
Ok(content) => println!("INFER_OK:{content}"),
|
||||
Err(error) => println!("INFER_FAIL:{error}"),
|
||||
}
|
||||
}
|
||||
None => println!("NONE"),
|
||||
}
|
||||
let _ = node.stop().await;
|
||||
// Skip C++ static destructors (ggml Metal aborts in global teardown).
|
||||
std::process::exit(0);
|
||||
}
|
||||
|
||||
/// Orchestrator: keystores, three child processes, assertions, inference.
|
||||
fn orchestrate() -> anyhow::Result<()> {
|
||||
let model = std::env::var("MESH_SMOKE_MODEL").unwrap_or_else(|_| DEFAULT_MODEL.to_string());
|
||||
eprintln!("[admission] model: {model}");
|
||||
|
||||
let scratch = std::env::temp_dir().join(format!("buzz-mesh-admission-{}", std::process::id()));
|
||||
std::fs::create_dir_all(&scratch)?;
|
||||
let make_owner = |name: &str| -> anyhow::Result<(String, String)> {
|
||||
let keypair = OwnerKeypair::generate();
|
||||
let path = scratch.join(format!("{name}.keystore.json"));
|
||||
save_keystore(&path, &keypair, None, true)
|
||||
.map_err(|error| anyhow::anyhow!("saving {name} keystore: {error}"))?;
|
||||
Ok((path.display().to_string(), keypair.owner_id()))
|
||||
};
|
||||
let (serve_key, serve_owner) = make_owner("serve")?;
|
||||
let (trusted_key, trusted_owner) = make_owner("trusted")?;
|
||||
let (stranger_key, stranger_owner) = make_owner("stranger")?;
|
||||
eprintln!("[admission] owners — serve: {serve_owner}, trusted: {trusted_owner}, stranger: {stranger_owner}");
|
||||
|
||||
// Each role gets an isolated HOME: mesh-llm keeps its node endpoint key
|
||||
// and node-ownership.json under ~/.mesh-llm, so subprocesses sharing the
|
||||
// real HOME would share a node identity and clobber each other's
|
||||
// attestations. The native runtime cache must still point at the real
|
||||
// one (it resolves via HOME otherwise).
|
||||
let real_cache = std::env::var_os("MESH_LLM_NATIVE_RUNTIME_CACHE_DIR")
|
||||
.map(std::path::PathBuf::from)
|
||||
.unwrap_or(dirs_cache_dir()?.join("mesh-llm/native-runtimes"));
|
||||
let role_home = |name: &str| -> anyhow::Result<String> {
|
||||
let home = scratch.join(format!("{name}-home"));
|
||||
std::fs::create_dir_all(&home)?;
|
||||
Ok(home.display().to_string())
|
||||
};
|
||||
let serve_home = role_home("serve")?;
|
||||
let trusted_home = role_home("trusted")?;
|
||||
let stranger_home = role_home("stranger")?;
|
||||
|
||||
let exe = std::env::current_exe()?;
|
||||
|
||||
// 1. Serve child with allowlist {serve, trusted}.
|
||||
eprintln!("[admission] starting allowlist serve node (subprocess)...");
|
||||
let mut serve_child = Command::new(&exe)
|
||||
.env("MESH_ROLE", "serve")
|
||||
.env("MESH_SMOKE_MODEL", &model)
|
||||
.env("MESH_OWNER_KEY", &serve_key)
|
||||
.env("HOME", &serve_home)
|
||||
.env("MESH_LLM_NATIVE_RUNTIME_CACHE_DIR", &real_cache)
|
||||
.env(
|
||||
"MESH_TRUST_OWNERS",
|
||||
format!("{serve_owner},{trusted_owner}"),
|
||||
)
|
||||
.stdout(Stdio::piped())
|
||||
.stderr(Stdio::inherit())
|
||||
.spawn()?;
|
||||
let serve_guard = KillOnDrop(&mut serve_child);
|
||||
|
||||
let serve_stdout = serve_guard
|
||||
.0
|
||||
.stdout
|
||||
.take()
|
||||
.ok_or_else(|| anyhow::anyhow!("no serve stdout"))?;
|
||||
let mut serve_lines = std::io::BufReader::new(serve_stdout).lines();
|
||||
let invite = expect_line(&mut serve_lines, "INVITE:", Duration::from_secs(180))?;
|
||||
eprintln!("[admission] serve invite token acquired");
|
||||
let served_model = expect_line(&mut serve_lines, "READY:", Duration::from_secs(600))?;
|
||||
eprintln!("[admission] serve model ready: {served_model}");
|
||||
|
||||
// 2. Trusted client child: allowlisted owner + the invite token.
|
||||
eprintln!("[admission] starting TRUSTED client (allowlisted owner)...");
|
||||
let trusted_out = run_client(
|
||||
&exe,
|
||||
ClientRun {
|
||||
owner_key: &trusted_key,
|
||||
home: &trusted_home,
|
||||
cache_dir: &real_cache,
|
||||
invite: &invite,
|
||||
api_port: TRUSTED_API_PORT,
|
||||
console_port: TRUSTED_CONSOLE_PORT,
|
||||
window_secs: TRUSTED_WINDOW_SECS,
|
||||
},
|
||||
)?;
|
||||
let routed = match trusted_out.seen.as_deref() {
|
||||
Some(model) => model.to_string(),
|
||||
None => anyhow::bail!("ADMISSION FAIL: trusted (allowlisted) client never saw the model"),
|
||||
};
|
||||
eprintln!("[admission] PASS 1/3: trusted client admitted, sees routed model: {routed}");
|
||||
let content = match trusted_out.infer_ok.as_deref() {
|
||||
Some(content) => content.to_string(),
|
||||
None => anyhow::bail!(
|
||||
"ADMISSION FAIL: trusted client saw the model but inference did not route: {:?}",
|
||||
trusted_out.infer_fail
|
||||
),
|
||||
};
|
||||
eprintln!("[admission] PASS 2/3: trusted client inference routed over mesh: {content:?}");
|
||||
let _ = served_model; // serve-side id retained for logs only
|
||||
|
||||
// 4. Non-member child: same connection information, non-allowlisted owner key.
|
||||
eprintln!("[admission] starting NON-MEMBER client (owner is not allowlisted)...");
|
||||
let stranger_out = run_client(
|
||||
&exe,
|
||||
ClientRun {
|
||||
owner_key: &stranger_key,
|
||||
home: &stranger_home,
|
||||
cache_dir: &real_cache,
|
||||
invite: &invite,
|
||||
api_port: STRANGER_API_PORT,
|
||||
console_port: STRANGER_CONSOLE_PORT,
|
||||
window_secs: STRANGER_WINDOW_SECS,
|
||||
},
|
||||
)?;
|
||||
match (
|
||||
stranger_out.seen.as_deref(),
|
||||
stranger_out.infer_ok.as_deref(),
|
||||
stranger_out.infer_fail.as_deref(),
|
||||
) {
|
||||
(None, None, _) => eprintln!(
|
||||
"[admission] PASS 3/3: non-member saw no model ({STRANGER_WINDOW_SECS}s window)"
|
||||
),
|
||||
(Some(model), None, Some(error)) => eprintln!(
|
||||
"[admission] PASS 3/3: non-member saw gossip for {model} but inference was rejected: {error}"
|
||||
),
|
||||
(Some(model), Some(content), _) => anyhow::bail!(
|
||||
"ADMISSION FAIL: non-member reused the invite token and inferred through {model}: {content:?}"
|
||||
),
|
||||
(Some(model), None, None) => anyhow::bail!(
|
||||
"ADMISSION INCONCLUSIVE: non-member saw {model} but produced no inference verdict"
|
||||
),
|
||||
(None, Some(content), _) => anyhow::bail!(
|
||||
"ADMISSION FAIL: non-member inferred without model visibility: {content:?}"
|
||||
),
|
||||
}
|
||||
eprintln!("[admission] PASS: owner allowlist gates mesh membership and inference");
|
||||
|
||||
drop(serve_guard); // kills the serve child
|
||||
let _ = serve_child.wait();
|
||||
let _ = std::fs::remove_dir_all(&scratch);
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// One chat completion against a node's OpenAI endpoint; Ok(content) only if
|
||||
/// it really routed and produced non-empty output.
|
||||
async fn try_completion(
|
||||
http: &reqwest::Client,
|
||||
api_base: &str,
|
||||
model: &str,
|
||||
) -> anyhow::Result<String> {
|
||||
let resp = http
|
||||
.post(format!("{api_base}/chat/completions"))
|
||||
.timeout(Duration::from_secs(60))
|
||||
.json(&serde_json::json!({
|
||||
"model": model,
|
||||
"messages": [{"role": "user", "content": "Reply with exactly one word: PONG"}],
|
||||
"max_tokens": 16,
|
||||
"temperature": 0.0
|
||||
}))
|
||||
.send()
|
||||
.await?;
|
||||
let status = resp.status();
|
||||
let body = resp.text().await?;
|
||||
if !status.is_success() {
|
||||
anyhow::bail!("{status}: {body}");
|
||||
}
|
||||
let content = serde_json::from_str::<serde_json::Value>(&body)?["choices"][0]["message"]
|
||||
["content"]
|
||||
.as_str()
|
||||
.unwrap_or("")
|
||||
.to_string();
|
||||
if content.trim().is_empty() {
|
||||
anyhow::bail!("empty content");
|
||||
}
|
||||
Ok(content)
|
||||
}
|
||||
|
||||
/// The real user's OS cache dir (macOS: ~/Library/Caches), resolved before
|
||||
/// we override HOME for the child processes.
|
||||
fn dirs_cache_dir() -> anyhow::Result<std::path::PathBuf> {
|
||||
let home = std::env::var("HOME").map_err(|_| anyhow::anyhow!("HOME is not set"))?;
|
||||
#[cfg(target_os = "macos")]
|
||||
return Ok(std::path::PathBuf::from(home).join("Library/Caches"));
|
||||
#[cfg(not(target_os = "macos"))]
|
||||
return Ok(std::path::PathBuf::from(home).join(".cache"));
|
||||
}
|
||||
|
||||
struct ClientRun<'a> {
|
||||
owner_key: &'a str,
|
||||
home: &'a str,
|
||||
cache_dir: &'a std::path::Path,
|
||||
invite: &'a str,
|
||||
api_port: u16,
|
||||
console_port: u16,
|
||||
window_secs: u64,
|
||||
}
|
||||
|
||||
/// Spawn a client-role child and return its verdict line (`SEEN:…` / `NONE`).
|
||||
fn run_client(exe: &std::path::Path, run: ClientRun<'_>) -> anyhow::Result<ClientVerdict> {
|
||||
let output = Command::new(exe)
|
||||
.env("MESH_ROLE", "client")
|
||||
.env("MESH_OWNER_KEY", run.owner_key)
|
||||
.env("HOME", run.home)
|
||||
.env("MESH_LLM_NATIVE_RUNTIME_CACHE_DIR", run.cache_dir)
|
||||
.env("MESH_JOIN_TOKEN", run.invite)
|
||||
.env("MESH_API_PORT", run.api_port.to_string())
|
||||
.env("MESH_CONSOLE_PORT", run.console_port.to_string())
|
||||
.env("MESH_WINDOW_SECS", run.window_secs.to_string())
|
||||
.stderr(Stdio::inherit())
|
||||
.output()?;
|
||||
let stdout = String::from_utf8_lossy(&output.stdout);
|
||||
let mut verdict = ClientVerdict::default();
|
||||
let mut saw_any = false;
|
||||
for line in stdout.lines() {
|
||||
if let Some(model) = line.strip_prefix("SEEN:") {
|
||||
verdict.seen = Some(model.to_string());
|
||||
saw_any = true;
|
||||
} else if line == "NONE" {
|
||||
saw_any = true;
|
||||
} else if let Some(content) = line.strip_prefix("INFER_OK:") {
|
||||
verdict.infer_ok = Some(content.to_string());
|
||||
} else if let Some(error) = line.strip_prefix("INFER_FAIL:") {
|
||||
verdict.infer_fail = Some(error.to_string());
|
||||
}
|
||||
}
|
||||
if !saw_any {
|
||||
anyhow::bail!("client child produced no verdict; stdout: {stdout}");
|
||||
}
|
||||
Ok(verdict)
|
||||
}
|
||||
|
||||
/// What a client-role child reported on stdout.
|
||||
#[derive(Debug, Default)]
|
||||
struct ClientVerdict {
|
||||
/// Model id if the routed model became visible in the window.
|
||||
seen: Option<String>,
|
||||
/// Completion content if an inference actually routed over the mesh.
|
||||
infer_ok: Option<String>,
|
||||
/// Inference error if visibility existed but routing failed.
|
||||
infer_fail: Option<String>,
|
||||
}
|
||||
|
||||
/// Read serve-child stdout until a line with the given prefix appears.
|
||||
fn expect_line(
|
||||
lines: &mut std::io::Lines<std::io::BufReader<std::process::ChildStdout>>,
|
||||
prefix: &str,
|
||||
timeout: Duration,
|
||||
) -> anyhow::Result<String> {
|
||||
// BufReader::lines blocks; enforce the timeout coarsely via a deadline
|
||||
// check between lines (the child prints continuously enough in practice).
|
||||
let deadline = std::time::Instant::now() + timeout;
|
||||
for line in lines.by_ref() {
|
||||
let line = line?;
|
||||
if let Some(rest) = line.strip_prefix(prefix) {
|
||||
return Ok(rest.to_string());
|
||||
}
|
||||
if std::time::Instant::now() > deadline {
|
||||
break;
|
||||
}
|
||||
}
|
||||
anyhow::bail!("serve child ended or timed out before printing {prefix}")
|
||||
}
|
||||
|
||||
/// Kill the serve child on drop so a failed assertion never leaks a process.
|
||||
struct KillOnDrop<'a>(&'a mut Child);
|
||||
impl Drop for KillOnDrop<'_> {
|
||||
fn drop(&mut self) {
|
||||
let _ = self.0.kill();
|
||||
}
|
||||
}
|
||||
|
||||
/// Poll `/models` until a model id appears or the window closes.
|
||||
async fn wait_for_model(
|
||||
http: &reqwest::Client,
|
||||
api_base: &str,
|
||||
window: Duration,
|
||||
) -> anyhow::Result<Option<String>> {
|
||||
let url = format!("{api_base}/models");
|
||||
let deadline = std::time::Instant::now() + window;
|
||||
while std::time::Instant::now() < deadline {
|
||||
tokio::time::sleep(Duration::from_secs(3)).await;
|
||||
if let Ok(resp) = http.get(&url).send().await {
|
||||
let body = resp.text().await.unwrap_or_default();
|
||||
if let Ok(json) = serde_json::from_str::<serde_json::Value>(&body) {
|
||||
if let Some(id) = json["data"].get(0).and_then(|m| m["id"].as_str()) {
|
||||
return Ok(Some(id.to_string()));
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
Ok(None)
|
||||
}
|
||||
@@ -0,0 +1,423 @@
|
||||
//! End-to-end mesh + agent permutation harness — fully headless, no desktop
|
||||
//! app, no keychain. Proves the whole chain the UI exercises:
|
||||
//!
|
||||
//! share compute (serve node) → agent env preset → ACP agent → inference
|
||||
//!
|
||||
//! Permutations:
|
||||
//! P1 explicit-model chat — agent pinned to the served model id replies.
|
||||
//! P2 auto-model chat — agent sends `model: "auto"`; mesh router picks.
|
||||
//! P3 context-fit regression — an oversized output budget (150k tokens)
|
||||
//! must FAIL with the router's context error (proves the router's fit
|
||||
//! gate — the failure mode the 1024 preset cap protects against).
|
||||
//! P4 agentic tool use — agent + buzz-dev-mcp writes a file on disk.
|
||||
//!
|
||||
//! The serve node is the same `mesh_llm_sdk::serve` path Share-compute uses
|
||||
//! (publish off, mdns, loopback). The agent legs spawn the real
|
||||
//! `buzz-agent` binary with the exact env vars the relay-mesh preset ships.
|
||||
//!
|
||||
//! Hardware-gated, not CI. Run:
|
||||
//! cargo build --release -p buzz-agent -p buzz-dev-mcp
|
||||
//! cargo run -p buzz-relay --example mesh_agent_e2e
|
||||
//! Env: MESH_E2E_MODEL overrides the served model ref.
|
||||
use std::process::Stdio;
|
||||
use std::time::Duration;
|
||||
|
||||
use mesh_llm_sdk::{serve, MeshDiscoveryMode};
|
||||
use tokio::io::{AsyncBufReadExt, AsyncWriteExt, BufReader};
|
||||
use tokio::process::{Child, Command};
|
||||
|
||||
// Qwen3-8B: cached GGUF *and* complete layer package on this class of
|
||||
// machine, so the serve node starts in seconds. Qwen3-30B-A3B works too but
|
||||
// mesh-llm serves it from layer packages and will download them on first
|
||||
// serve (~7GB) — fine in the app (progress UI), too slow for a smoke.
|
||||
const DEFAULT_MODEL: &str = "unsloth/Qwen3-8B-GGUF:Q4_K_M";
|
||||
const API_PORT: u16 = 19437;
|
||||
const CONSOLE_PORT: u16 = 13231;
|
||||
|
||||
fn main() -> anyhow::Result<()> {
|
||||
tokio::runtime::Builder::new_multi_thread()
|
||||
.enable_all()
|
||||
// Same fix the desktop ships: mesh-llm futures need >2MiB stacks.
|
||||
.thread_stack_size(8 * 1024 * 1024)
|
||||
.build()?
|
||||
.block_on(run())
|
||||
}
|
||||
|
||||
async fn run() -> anyhow::Result<()> {
|
||||
let model = std::env::var("MESH_E2E_MODEL").unwrap_or_else(|_| DEFAULT_MODEL.to_string());
|
||||
mesh_llm_host_runtime::initialize_host_runtime()
|
||||
.await
|
||||
.map_err(|e| anyhow::anyhow!("host runtime init: {e}"))?;
|
||||
|
||||
eprintln!("[e2e] starting serve node with {model} (loading may take a minute)...");
|
||||
let cfg = serve::EmbeddedServeConfig::builder()
|
||||
.model(&model)
|
||||
.api_port(API_PORT)
|
||||
.console_port(CONSOLE_PORT)
|
||||
.publish(false)
|
||||
.auto_join(false)
|
||||
.discovery_mode(MeshDiscoveryMode::Mdns)
|
||||
.console_ui(true) // readiness poll needs the console bound
|
||||
.startup_timeout(Duration::from_secs(300))
|
||||
.build();
|
||||
let node = serve::start(cfg)
|
||||
.await
|
||||
.map_err(|e| anyhow::anyhow!("serve start: {e}"))?;
|
||||
let base = node.api_base_url().to_string();
|
||||
|
||||
// Wait for the model to be loaded + resolvable, capture its served id.
|
||||
let http = reqwest::Client::new();
|
||||
let mut served_id = String::new();
|
||||
for _ in 0..120 {
|
||||
if let Ok(resp) = http.get(format!("{base}/models")).send().await {
|
||||
if let Ok(json) = resp.json::<serde_json::Value>().await {
|
||||
if let Some(id) = json["data"].get(0).and_then(|m| m["id"].as_str()) {
|
||||
served_id = id.to_string();
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
tokio::time::sleep(Duration::from_secs(5)).await;
|
||||
}
|
||||
anyhow::ensure!(!served_id.is_empty(), "model never appeared in /models");
|
||||
eprintln!("[e2e] node up, served id = {served_id}");
|
||||
|
||||
let mut pass = 0usize;
|
||||
let mut fail = 0usize;
|
||||
let mut record = |name: &str, ok: bool, detail: String| {
|
||||
if ok {
|
||||
pass += 1;
|
||||
eprintln!("[e2e] PASS {name}: {detail}");
|
||||
} else {
|
||||
fail += 1;
|
||||
eprintln!("[e2e] FAIL {name}: {detail}");
|
||||
}
|
||||
};
|
||||
|
||||
// P1: explicit model id.
|
||||
let r = agent_chat(
|
||||
&base,
|
||||
&served_id,
|
||||
None,
|
||||
"Reply with exactly one word: PONG",
|
||||
&[],
|
||||
)
|
||||
.await;
|
||||
match r {
|
||||
Ok(text) => record(
|
||||
"P1 explicit-model chat",
|
||||
text.to_uppercase().contains("PONG"),
|
||||
text,
|
||||
),
|
||||
Err(e) => record("P1 explicit-model chat", false, e.to_string()),
|
||||
}
|
||||
|
||||
// P2: auto — router picks the model.
|
||||
let r = agent_chat(
|
||||
&base,
|
||||
"auto",
|
||||
None,
|
||||
"Reply with exactly one word: PONG",
|
||||
&[],
|
||||
)
|
||||
.await;
|
||||
match r {
|
||||
Ok(text) => record(
|
||||
"P2 auto-model chat",
|
||||
text.to_uppercase().contains("PONG"),
|
||||
text,
|
||||
),
|
||||
Err(e) => record("P2 auto-model chat", false, e.to_string()),
|
||||
}
|
||||
|
||||
// P3: regression — an output budget no served model's context can hold
|
||||
// must be rejected by the router with the context-fit error (the failure
|
||||
// mode that broke relay-mesh agents when buzz-agent's default 32768
|
||||
// budget met a 32k-context model). 150k output: passes buzz-agent's own
|
||||
// config validation (must stay under its 200k max_context_tokens) but
|
||||
// with the router's +25% margin overflows even 128k-context models like
|
||||
// GLM-4.7-Flash.
|
||||
let r = agent_chat(
|
||||
&base,
|
||||
&served_id,
|
||||
Some("150000"),
|
||||
"Reply with exactly one word: PONG",
|
||||
&[],
|
||||
)
|
||||
.await;
|
||||
match r {
|
||||
Ok(text) => record(
|
||||
"P3 oversized-budget must fail",
|
||||
false,
|
||||
format!("unexpectedly succeeded: {text}"),
|
||||
),
|
||||
Err(e) => {
|
||||
let msg = e.to_string();
|
||||
let is_context_503 = msg.contains("503")
|
||||
|| msg.contains("service_unavailable")
|
||||
|| msg.contains("context-compatible");
|
||||
record("P3 oversized-budget must fail", is_context_503, msg);
|
||||
}
|
||||
}
|
||||
|
||||
// P4: agentic tool use via buzz-dev-mcp — write a real file inside the
|
||||
// isolated ACP working directory. The MCP sandbox intentionally rejects
|
||||
// nonexistent absolute paths outside that root.
|
||||
let marker_name = format!("mesh-e2e-{}.txt", std::process::id());
|
||||
let prompt = format!(
|
||||
"Use your developer tools to create {marker_name} in the current working directory containing exactly the text BUZZ_OK (no quotes, no newline commentary). Then confirm."
|
||||
);
|
||||
let mcp = vec![("dev".to_string(), repo_bin("buzz-dev-mcp")?)];
|
||||
let (r, marker) =
|
||||
agent_chat_with_marker(&base, &served_id, None, &prompt, &mcp, &marker_name).await;
|
||||
let file_ok = std::fs::read_to_string(&marker)
|
||||
.map(|c| c.contains("BUZZ_OK"))
|
||||
.unwrap_or(false);
|
||||
match r {
|
||||
Ok(text) => record(
|
||||
"P4 agentic tool use",
|
||||
file_ok,
|
||||
if file_ok {
|
||||
format!("file written; agent said: {text}")
|
||||
} else {
|
||||
format!("no file at {}; agent said: {text}", marker.display())
|
||||
},
|
||||
),
|
||||
Err(e) => record("P4 agentic tool use", file_ok, format!("agent error: {e}")),
|
||||
}
|
||||
let _ = std::fs::remove_file(&marker);
|
||||
|
||||
eprintln!("[e2e] {pass} passed, {fail} failed");
|
||||
if fail > 0 {
|
||||
eprintln!("[e2e] FAIL: {fail} permutation(s) failed");
|
||||
exit_without_native_destructors(1);
|
||||
}
|
||||
eprintln!("[e2e] PASS: share-compute → agent → inference proven end to end");
|
||||
exit_without_native_destructors(0);
|
||||
}
|
||||
|
||||
/// Replace the process image so libc does not run llama.cpp's crashing Metal
|
||||
/// `atexit` handlers (mesh-console issue #8). `std::process::exit` is not enough:
|
||||
/// it skips Rust drops but still runs native C/C++ finalizers.
|
||||
fn exit_without_native_destructors(code: i32) -> ! {
|
||||
#[cfg(unix)]
|
||||
{
|
||||
use std::os::unix::process::CommandExt;
|
||||
let program = if code == 0 { "true" } else { "false" };
|
||||
let error = std::process::Command::new(program).exec();
|
||||
eprintln!("[e2e] failed to exec {program}: {error}");
|
||||
}
|
||||
std::process::exit(code)
|
||||
}
|
||||
|
||||
fn repo_bin(name: &str) -> anyhow::Result<String> {
|
||||
let path = std::env::current_dir()?.join("target/release").join(name);
|
||||
anyhow::ensure!(
|
||||
path.exists(),
|
||||
"{} missing — cargo build --release -p {name}",
|
||||
path.display()
|
||||
);
|
||||
Ok(path.to_string_lossy().into_owned())
|
||||
}
|
||||
|
||||
/// Spawn the real buzz-agent with relay-mesh preset env and drive one ACP
|
||||
/// session/prompt over stdio. Returns the concatenated agent message text,
|
||||
/// or Err carrying the agent's error message.
|
||||
async fn agent_chat(
|
||||
base: &str,
|
||||
model: &str,
|
||||
max_output_tokens: Option<&str>,
|
||||
prompt: &str,
|
||||
mcp_servers: &[(String, String)],
|
||||
) -> anyhow::Result<String> {
|
||||
let (result, _) =
|
||||
agent_chat_in_isolated_home(base, model, max_output_tokens, prompt, mcp_servers).await;
|
||||
result
|
||||
}
|
||||
|
||||
async fn agent_chat_with_marker(
|
||||
base: &str,
|
||||
model: &str,
|
||||
max_output_tokens: Option<&str>,
|
||||
prompt: &str,
|
||||
mcp_servers: &[(String, String)],
|
||||
marker_name: &str,
|
||||
) -> (anyhow::Result<String>, std::path::PathBuf) {
|
||||
let (result, home) =
|
||||
agent_chat_in_isolated_home(base, model, max_output_tokens, prompt, mcp_servers).await;
|
||||
(result, home.join(marker_name))
|
||||
}
|
||||
|
||||
async fn agent_chat_in_isolated_home(
|
||||
base: &str,
|
||||
model: &str,
|
||||
max_output_tokens: Option<&str>,
|
||||
prompt: &str,
|
||||
mcp_servers: &[(String, String)],
|
||||
) -> (anyhow::Result<String>, std::path::PathBuf) {
|
||||
let agent = match repo_bin("buzz-agent") {
|
||||
Ok(agent) => agent,
|
||||
Err(error) => return (Err(error), std::path::PathBuf::new()),
|
||||
};
|
||||
// Isolated HOME: no skills, no AGENTS.md chain, no keychain, tiny prompt.
|
||||
let home = std::env::temp_dir().join(format!("mesh-e2e-home-{}", std::process::id()));
|
||||
if let Err(error) = std::fs::create_dir_all(&home) {
|
||||
return (Err(error.into()), home);
|
||||
}
|
||||
|
||||
let mut command = Command::new(&agent);
|
||||
command
|
||||
.env_clear()
|
||||
.env("PATH", std::env::var("PATH").unwrap_or_default())
|
||||
.env("HOME", &home)
|
||||
// Exactly the environment apply_relay_mesh_env() supplies.
|
||||
.env("BUZZ_AGENT_PROVIDER", "openai")
|
||||
.env("BUZZ_AGENT_MODEL", model)
|
||||
.env("OPENAI_COMPAT_BASE_URL", base)
|
||||
.env("OPENAI_COMPAT_MODEL", model)
|
||||
.env("OPENAI_COMPAT_API_KEY", "buzz-mesh-local")
|
||||
.env("OPENAI_COMPAT_API", "chat")
|
||||
.env("BUZZ_AGENT_MAX_OUTPUT_TOKENS", "4096")
|
||||
// No BUZZ_AGENT_THINKING_EFFORT: apply_relay_mesh_env() deliberately
|
||||
// leaves it unset so each model's chat template picks its own default.
|
||||
// Pinning a value here would test a config the product does not ship.
|
||||
.stdin(Stdio::piped())
|
||||
.stdout(Stdio::piped())
|
||||
.stderr(Stdio::null());
|
||||
// P3 deliberately overrides the production default to exercise the
|
||||
// router's context-fit rejection. Normal and tool turns leave it unset,
|
||||
// matching the desktop provider path.
|
||||
if let Some(value) = max_output_tokens {
|
||||
command.env("BUZZ_AGENT_MAX_OUTPUT_TOKENS", value);
|
||||
}
|
||||
let mut child = match command.spawn() {
|
||||
Ok(child) => child,
|
||||
Err(error) => return (Err(error.into()), home),
|
||||
};
|
||||
|
||||
let result = drive_acp(&mut child, prompt, mcp_servers, &home).await;
|
||||
let _ = child.kill().await;
|
||||
(result, home)
|
||||
}
|
||||
|
||||
async fn drive_acp(
|
||||
child: &mut Child,
|
||||
prompt: &str,
|
||||
mcp_servers: &[(String, String)],
|
||||
cwd: &std::path::Path,
|
||||
) -> anyhow::Result<String> {
|
||||
let mut stdin = child
|
||||
.stdin
|
||||
.take()
|
||||
.ok_or_else(|| anyhow::anyhow!("no stdin"))?;
|
||||
let stdout = child
|
||||
.stdout
|
||||
.take()
|
||||
.ok_or_else(|| anyhow::anyhow!("no stdout"))?;
|
||||
let mut lines = BufReader::new(stdout).lines();
|
||||
|
||||
let mcp_json: Vec<serde_json::Value> = mcp_servers
|
||||
.iter()
|
||||
.map(|(name, command)| {
|
||||
serde_json::json!({ "name": name, "command": command, "args": [], "env": [] })
|
||||
})
|
||||
.collect();
|
||||
|
||||
let send = |v: serde_json::Value| format!("{v}\n");
|
||||
stdin
|
||||
.write_all(
|
||||
send(serde_json::json!({
|
||||
"jsonrpc": "2.0", "id": 1, "method": "initialize",
|
||||
"params": { "protocolVersion": 1, "clientCapabilities": {} }
|
||||
}))
|
||||
.as_bytes(),
|
||||
)
|
||||
.await?;
|
||||
stdin
|
||||
.write_all(
|
||||
send(serde_json::json!({
|
||||
"jsonrpc": "2.0", "id": 2, "method": "session/new",
|
||||
"params": {
|
||||
"cwd": cwd.to_string_lossy(),
|
||||
"mcpServers": mcp_json,
|
||||
"systemPrompt": "You are a terse test agent. Follow instructions exactly."
|
||||
}
|
||||
}))
|
||||
.as_bytes(),
|
||||
)
|
||||
.await?;
|
||||
|
||||
let mut session_id: Option<String> = None;
|
||||
let mut agent_text = String::new();
|
||||
let deadline = tokio::time::Instant::now() + Duration::from_secs(600);
|
||||
|
||||
loop {
|
||||
let line = tokio::time::timeout_at(deadline, lines.next_line())
|
||||
.await
|
||||
.map_err(|_| anyhow::anyhow!("agent timed out; text so far: {agent_text}"))??
|
||||
.ok_or_else(|| anyhow::anyhow!("agent closed stdout; text so far: {agent_text}"))?;
|
||||
let Ok(msg) = serde_json::from_str::<serde_json::Value>(&line) else {
|
||||
continue;
|
||||
};
|
||||
// Collect any streamed agent text from session/update notifications.
|
||||
if msg.get("method").and_then(|m| m.as_str()) == Some("session/update") {
|
||||
collect_text(&msg["params"]["update"], &mut agent_text);
|
||||
continue;
|
||||
}
|
||||
match msg.get("id").and_then(|i| i.as_i64()) {
|
||||
Some(2) => {
|
||||
if let Some(err) = msg.get("error") {
|
||||
anyhow::bail!("session/new failed: {err}");
|
||||
}
|
||||
let sid = msg["result"]["sessionId"]
|
||||
.as_str()
|
||||
.ok_or_else(|| anyhow::anyhow!("session/new: no sessionId: {msg}"))?
|
||||
.to_string();
|
||||
stdin
|
||||
.write_all(
|
||||
send(serde_json::json!({
|
||||
"jsonrpc": "2.0", "id": 3, "method": "session/prompt",
|
||||
"params": {
|
||||
"sessionId": sid,
|
||||
"prompt": [ { "type": "text", "text": prompt } ]
|
||||
}
|
||||
}))
|
||||
.as_bytes(),
|
||||
)
|
||||
.await?;
|
||||
session_id = Some(sid);
|
||||
}
|
||||
Some(3) => {
|
||||
anyhow::ensure!(session_id.is_some(), "prompt response before session");
|
||||
if let Some(err) = msg.get("error") {
|
||||
anyhow::bail!("session/prompt failed: {err}");
|
||||
}
|
||||
return Ok(agent_text.trim().to_string());
|
||||
}
|
||||
_ => {}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Recursively harvest "text" string fields out of a session/update payload.
|
||||
fn collect_text(value: &serde_json::Value, out: &mut String) {
|
||||
match value {
|
||||
serde_json::Value::Object(map) => {
|
||||
for (k, v) in map {
|
||||
if k == "text" {
|
||||
if let Some(s) = v.as_str() {
|
||||
out.push_str(s);
|
||||
}
|
||||
} else {
|
||||
collect_text(v, out);
|
||||
}
|
||||
}
|
||||
}
|
||||
serde_json::Value::Array(items) => {
|
||||
for item in items {
|
||||
collect_text(item, out);
|
||||
}
|
||||
}
|
||||
_ => {}
|
||||
}
|
||||
}
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,171 @@
|
||||
//! Local mesh serve→client→inference smoke test.
|
||||
//!
|
||||
//! Proves the full Buzz shared compute serve→consume path on a single box,
|
||||
//! without a relay or Nostr discovery:
|
||||
//!
|
||||
//! 1. Start a SERVE node hosting a GGUF (the `serve::start` path desktop
|
||||
//! uses in Share-compute mode), and read its mesh invite token.
|
||||
//! 2. Start a CLIENT node joined to that serve node via the invite token
|
||||
//! (the `client::start` path a Buzz shared compute agent uses), binding
|
||||
//! its own local OpenAI-compatible endpoint.
|
||||
//! 3. Drive one chat completion against the CLIENT endpoint and assert it
|
||||
//! routed through the mesh to the serve node and produced real output.
|
||||
//!
|
||||
//! Serve-only proves less than the PR claims (serve + client + routing), so
|
||||
//! this exercises the client hop end to end.
|
||||
//!
|
||||
//! This is hardware-gated and NOT a CI test — it loads a real model and runs
|
||||
//! inference. It lives as an example so CI never auto-runs it.
|
||||
//!
|
||||
//! Usage:
|
||||
//! # default model is a ~100MB instruct model, downloaded on first run:
|
||||
//! cargo run -p buzz-relay --example mesh_serve_client_smoke
|
||||
//!
|
||||
//! # or point at any local .gguf / hf model ref (e.g. the on-hardware 35B):
|
||||
//! MESH_SMOKE_MODEL=/path/to/model.gguf \
|
||||
//! cargo run -p buzz-relay --example mesh_serve_client_smoke
|
||||
use std::time::Duration;
|
||||
|
||||
use mesh_llm_sdk::{client, serve, MeshDiscoveryMode};
|
||||
|
||||
/// Small, real instruct model the mesh project itself uses for CI smoke.
|
||||
/// Downloaded on first run; ~100MB.
|
||||
const DEFAULT_MODEL: &str = "jc-builds/SmolLM2-135M-Instruct-Q4_K_M-GGUF:Q4_K_M";
|
||||
|
||||
const SERVE_API_PORT: u16 = 19337;
|
||||
const SERVE_CONSOLE_PORT: u16 = 13131;
|
||||
const CLIENT_API_PORT: u16 = 19338;
|
||||
const CLIENT_CONSOLE_PORT: u16 = 13132;
|
||||
|
||||
#[tokio::main]
|
||||
async fn main() -> anyhow::Result<()> {
|
||||
let model = std::env::var("MESH_SMOKE_MODEL").unwrap_or_else(|_| DEFAULT_MODEL.to_string());
|
||||
eprintln!("[smoke] model: {model}");
|
||||
let cache = mesh_llm_sdk::native_runtime::native_runtime_cache(None)?;
|
||||
let current = mesh_llm_sdk::native_runtime::CURRENT_MESH_VERSION;
|
||||
if !cache
|
||||
.installed()?
|
||||
.iter()
|
||||
.any(|runtime| runtime.mesh_version == current)
|
||||
{
|
||||
anyhow::bail!("MeshLLM native runtime for MeshLLM {current} is not installed; run `just mesh-e2e-hardware` to prepare it");
|
||||
}
|
||||
mesh_llm_host_runtime::initialize_host_runtime()
|
||||
.await
|
||||
.map_err(|error| anyhow::anyhow!("MeshLLM host runtime init failed: {error}"))?;
|
||||
eprintln!("[smoke] MeshLLM host runtime initialized");
|
||||
|
||||
let serve_cfg = serve::EmbeddedServeConfig::builder()
|
||||
.model(&model)
|
||||
.api_port(SERVE_API_PORT)
|
||||
.console_port(SERVE_CONSOLE_PORT)
|
||||
// publish so the client can join via the invite token; Mdns keeps
|
||||
// discovery local — no relay, no Nostr.
|
||||
.publish(true)
|
||||
.auto_join(false)
|
||||
.discovery_mode(MeshDiscoveryMode::Mdns)
|
||||
// console_ui(true) is required for readiness polling at rev bd16da4
|
||||
// (serve::start polls :console_port/api/status, which only binds when
|
||||
// !headless). See mesh_serve_smoke.rs for the full note.
|
||||
.console_ui(true)
|
||||
.build();
|
||||
|
||||
eprintln!("[smoke] starting serve node...");
|
||||
let serve_node = serve::start(serve_cfg).await?;
|
||||
let serve_base = serve_node.api_base_url().to_string();
|
||||
eprintln!("[smoke] serve up, api_base_url = {serve_base}");
|
||||
|
||||
let invite = serve_node
|
||||
.invite_token()
|
||||
.map(str::to_string)
|
||||
.ok_or_else(|| anyhow::anyhow!("serve node produced no invite token to join"))?;
|
||||
eprintln!("[smoke] serve invite token acquired (len {})", invite.len());
|
||||
|
||||
// Wait until the serve node reports the model loaded, so the client has
|
||||
// something to route to.
|
||||
let http = reqwest::Client::new();
|
||||
let serve_model_id = wait_for_model(&http, &serve_base).await?;
|
||||
eprintln!("[smoke] serve model ready: {serve_model_id}");
|
||||
|
||||
let client_cfg = client::EmbeddedClientConfig::builder()
|
||||
.api_port(CLIENT_API_PORT)
|
||||
.console_port(CLIENT_CONSOLE_PORT)
|
||||
.publish(false)
|
||||
.auto_join(false)
|
||||
.discovery_mode(MeshDiscoveryMode::Mdns)
|
||||
.join_token(&invite)
|
||||
.console_ui(true)
|
||||
.build();
|
||||
|
||||
eprintln!("[smoke] starting client node joined to serve...");
|
||||
let client_node = client::start(client_cfg).await?;
|
||||
let client_base = client_node.api_base_url().to_string();
|
||||
eprintln!("[smoke] client up, api_base_url = {client_base}");
|
||||
|
||||
// The served model must propagate across the mesh to the client's
|
||||
// /models view before we can route a completion through it.
|
||||
let routed_model_id = wait_for_model(&http, &client_base).await?;
|
||||
eprintln!("[smoke] client sees routed model: {routed_model_id}");
|
||||
|
||||
let chat_url = format!("{client_base}/chat/completions");
|
||||
let req = serde_json::json!({
|
||||
"model": routed_model_id,
|
||||
"messages": [{"role": "user", "content": "Reply with exactly one word: PONG"}],
|
||||
"max_tokens": 512,
|
||||
"temperature": 0.0
|
||||
});
|
||||
eprintln!("[smoke] POST {chat_url} (routes serve←client over mesh)");
|
||||
let resp = http.post(&chat_url).json(&req).send().await?;
|
||||
let status = resp.status();
|
||||
let body = resp.text().await?;
|
||||
println!("[smoke] completion status={status}");
|
||||
println!("[smoke] completion body={body}");
|
||||
|
||||
// Tear down both nodes before asserting, so a failed assert still cleans up.
|
||||
let _ = client_node.stop().await;
|
||||
let _ = serve_node.stop().await;
|
||||
|
||||
if !status.is_success() {
|
||||
anyhow::bail!("completion through client failed: {status}");
|
||||
}
|
||||
let json: serde_json::Value = serde_json::from_str(&body)?;
|
||||
let content = json["choices"][0]["message"]["content"]
|
||||
.as_str()
|
||||
.unwrap_or("");
|
||||
let finish = json["choices"][0]["finish_reason"].as_str().unwrap_or("");
|
||||
if content.trim().is_empty() {
|
||||
anyhow::bail!("completion routed but content was empty");
|
||||
}
|
||||
println!("[smoke] OK — routed completion finish_reason={finish:?} content={content:?}");
|
||||
eprintln!("[smoke] PASS: serve→client→inference proven over mesh");
|
||||
// Exit immediately, skipping remaining Rust drops (tokio runtime, iroh
|
||||
// endpoints). The embedded ggml Metal runtime can GGML_ASSERT inside C++
|
||||
// static destructors at process teardown when a node wasn't cleanly shut
|
||||
// down (observed here after a startup failure; mesh-console issue #8 is
|
||||
// the same crash — its fix is libc::_exit, which this repo's no-unsafe
|
||||
// rule rules out). Both nodes are stopped above, which is what keeps the
|
||||
// finalizers quiet on the success path.
|
||||
std::process::exit(0);
|
||||
}
|
||||
|
||||
/// Poll a node's `/models` until it reports a model, returning the served id
|
||||
/// (the node assigns its own id, e.g. `local-gguf/sha256-…`, not our ref).
|
||||
async fn wait_for_model(http: &reqwest::Client, api_base: &str) -> anyhow::Result<String> {
|
||||
let url = format!("{api_base}/models");
|
||||
for i in 0..120 {
|
||||
tokio::time::sleep(Duration::from_secs(5)).await;
|
||||
match http.get(&url).send().await {
|
||||
Ok(r) => {
|
||||
let body = r.text().await.unwrap_or_default();
|
||||
if let Ok(json) = serde_json::from_str::<serde_json::Value>(&body) {
|
||||
if let Some(id) = json["data"].get(0).and_then(|m| m["id"].as_str()) {
|
||||
return Ok(id.to_string());
|
||||
}
|
||||
}
|
||||
eprintln!("[smoke] waiting ({}s) {url} -> {body}", (i + 1) * 5);
|
||||
}
|
||||
Err(e) => eprintln!("[smoke] waiting ({}s) {url} err: {e}", (i + 1) * 5),
|
||||
}
|
||||
}
|
||||
anyhow::bail!("model never became visible at {api_base} within timeout")
|
||||
}
|
||||
@@ -0,0 +1,96 @@
|
||||
//! Local mesh-serve inference smoke test.
|
||||
//!
|
||||
//! Serves a GGUF model through the same `mesh_llm_sdk::serve` path Buzz
|
||||
//! desktop uses in "Share compute" mode, then drives one chat completion
|
||||
//! against the node's local OpenAI-compatible endpoint. No mesh publish, no
|
||||
//! auto-join, no Nostr discovery — pure single-node serve-and-self-consume,
|
||||
//! which is exactly the loopback variant we can prove on one box.
|
||||
//!
|
||||
//! Usage:
|
||||
//! cargo run -p buzz-relay --example mesh_serve_smoke -- <path-to.gguf>
|
||||
use std::time::Duration;
|
||||
|
||||
use mesh_llm_sdk::{serve, MeshDiscoveryMode};
|
||||
|
||||
#[tokio::main]
|
||||
async fn main() -> anyhow::Result<()> {
|
||||
let model = std::env::args()
|
||||
.nth(1)
|
||||
.ok_or_else(|| anyhow::anyhow!("usage: mesh_serve_smoke <path-to.gguf>"))?;
|
||||
eprintln!("[smoke] serving model: {model}");
|
||||
|
||||
let config = serve::EmbeddedServeConfig::builder()
|
||||
.model(&model)
|
||||
.api_port(19337)
|
||||
.console_port(13131)
|
||||
.publish(false)
|
||||
.auto_join(false)
|
||||
.discovery_mode(MeshDiscoveryMode::Mdns)
|
||||
// NOTE: console_ui(true) is required here, not cosmetic. In mesh rev
|
||||
// bd16da4, serve::start always polls `:console_port/api/status` to
|
||||
// confirm readiness, but the console HTTP server only binds when
|
||||
// !headless (i.e. console_ui == true). With console_ui(false) the
|
||||
// poll never succeeds and startup times out after 30s. Desktop's
|
||||
// Share-compute path sets console_ui(false) — likely hits the same
|
||||
// wall. Flagged to the team.
|
||||
.console_ui(true)
|
||||
.build();
|
||||
|
||||
let node = serve::start(config).await?;
|
||||
let base = node.api_base_url().to_string();
|
||||
eprintln!("[smoke] node up, api_base_url = {base}");
|
||||
|
||||
// Poll until the model reports loaded/ready (give it generous time — first
|
||||
// load of a 17GB GGUF into Metal can take a while).
|
||||
let http = reqwest::Client::new();
|
||||
let models_url = format!("{base}/models");
|
||||
let mut model_id = String::new();
|
||||
for i in 0..120 {
|
||||
tokio::time::sleep(Duration::from_secs(5)).await;
|
||||
match http.get(&models_url).send().await {
|
||||
Ok(r) => {
|
||||
let body = r.text().await.unwrap_or_default();
|
||||
// The serve node assigns its own id (e.g. local-gguf/sha256-…),
|
||||
// not the file path we passed — pull it from /models.
|
||||
if let Ok(json) = serde_json::from_str::<serde_json::Value>(&body) {
|
||||
if let Some(id) = json["data"].get(0).and_then(|m| m["id"].as_str()) {
|
||||
model_id = id.to_string();
|
||||
eprintln!(
|
||||
"[smoke] /models after {}s, served id = {model_id}",
|
||||
(i + 1) * 5
|
||||
);
|
||||
break;
|
||||
}
|
||||
}
|
||||
eprintln!("[smoke] waiting ({}s) /models -> {body}", (i + 1) * 5);
|
||||
}
|
||||
Err(e) => eprintln!("[smoke] waiting ({}s) /models err: {e}", (i + 1) * 5),
|
||||
}
|
||||
}
|
||||
if model_id.is_empty() {
|
||||
anyhow::bail!("model never became ready within timeout");
|
||||
}
|
||||
|
||||
// One real completion — use the server's own model id.
|
||||
let chat_url = format!("{base}/chat/completions");
|
||||
let req = serde_json::json!({
|
||||
"model": model_id,
|
||||
"messages": [
|
||||
{"role": "user", "content": "Reply with exactly one word: PONG"}
|
||||
],
|
||||
"max_tokens": 512,
|
||||
"temperature": 0.0
|
||||
});
|
||||
eprintln!("[smoke] POST {chat_url}");
|
||||
let resp = http.post(&chat_url).json(&req).send().await?;
|
||||
let status = resp.status();
|
||||
let body = resp.text().await?;
|
||||
println!("[smoke] completion status={status}");
|
||||
println!("[smoke] completion body={body}");
|
||||
|
||||
node.stop().await?;
|
||||
if !status.is_success() {
|
||||
anyhow::bail!("completion request failed: {status}");
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
@@ -0,0 +1,121 @@
|
||||
//! Tokio worker stack-size smoke — reproduces and verifies the fix for the
|
||||
//! mesh-llm model-download stack overflow (SIGABRT via stack guard).
|
||||
//!
|
||||
//! Crash report (2026-07-08, buzz-desktop 0.3.46): enabling Share compute
|
||||
//! aborted the app on a `tokio-rt-worker` thread inside
|
||||
//! `mesh_llm_host_runtime::models::resolve::download_model_ref_with_progress_details`
|
||||
//! — Rust's stack-overflow signal handler fired on tokio's default 2 MiB
|
||||
//! worker stack. Upstream mesh-llm runs its own binary on 8 MiB worker
|
||||
//! stacks for exactly this reason (`DEFAULT_WORKER_STACK_SIZE` in mesh-llm
|
||||
//! `main.rs`), as does mesh-console.
|
||||
//!
|
||||
//! This harness polls the same future as a spawned task (matching how Tauri
|
||||
//! polls command futures on worker threads) in two subprocess legs:
|
||||
//!
|
||||
//! 1. 2 MiB worker stacks (tokio default) — expected to DIE from the
|
||||
//! stack guard (signal, no exit code). Proves we reproduced the crash.
|
||||
//! 2. 8 MiB worker stacks (the fix installed in desktop `lib.rs` via
|
||||
//! `tauri::async_runtime::set`) — expected to complete the download.
|
||||
//!
|
||||
//! Each leg gets a fresh HF_HOME so the download really runs (a cache hit
|
||||
//! never reaches the deep code path). Network required; downloads a ~100 MB
|
||||
//! GGUF twice at most (leg 1 usually dies early). Not CI — run manually:
|
||||
//!
|
||||
//! cargo run -p buzz-relay --example mesh_stack_smoke
|
||||
use std::process::{Command, Stdio};
|
||||
|
||||
/// Small real model, same one the admission smoke uses.
|
||||
const MODEL: &str = "jc-builds/SmolLM2-135M-Instruct-Q4_K_M-GGUF:Q4_K_M";
|
||||
|
||||
const TOKIO_DEFAULT_STACK: usize = 2 * 1024 * 1024;
|
||||
/// Must match `buzz_lib::mesh_llm::MESH_WORKER_STACK_SIZE` (desktop crate is
|
||||
/// not a dependency of buzz-relay, so the value is duplicated here).
|
||||
const FIXED_STACK: usize = 8 * 1024 * 1024;
|
||||
|
||||
fn main() -> anyhow::Result<()> {
|
||||
match std::env::var("MESH_STACK_ROLE").ok().as_deref() {
|
||||
Some("download") => role_download(),
|
||||
_ => orchestrate(),
|
||||
}
|
||||
}
|
||||
|
||||
/// Subprocess: poll the download future as a spawned task on a worker
|
||||
/// thread with the requested stack size — the exact shape of a Tauri
|
||||
/// command future.
|
||||
fn role_download() -> anyhow::Result<()> {
|
||||
let stack: usize = std::env::var("MESH_STACK_SIZE")?.parse()?;
|
||||
let runtime = tokio::runtime::Builder::new_multi_thread()
|
||||
.enable_all()
|
||||
.thread_stack_size(stack)
|
||||
.build()?;
|
||||
runtime.block_on(async {
|
||||
tokio::spawn(async {
|
||||
mesh_llm_host_runtime::models::download_model_ref_with_progress_details(MODEL, true)
|
||||
.await
|
||||
.map(|_| ())
|
||||
.map_err(|error| anyhow::anyhow!("download failed: {error}"))
|
||||
})
|
||||
.await?
|
||||
})?;
|
||||
println!("DOWNLOAD_OK");
|
||||
// Skip destructors (ggml teardown abort, mesh-console issue #8).
|
||||
std::process::exit(0);
|
||||
}
|
||||
|
||||
fn run_leg(stack: usize, hf_home: &std::path::Path) -> anyhow::Result<(bool, Option<i32>)> {
|
||||
let exe = std::env::current_exe()?;
|
||||
let status = Command::new(exe)
|
||||
.env("MESH_STACK_ROLE", "download")
|
||||
.env("MESH_STACK_SIZE", stack.to_string())
|
||||
.env("HF_HOME", hf_home)
|
||||
.stdout(Stdio::inherit())
|
||||
.stderr(Stdio::inherit())
|
||||
.status()?;
|
||||
Ok((status.success(), status.code()))
|
||||
}
|
||||
|
||||
fn orchestrate() -> anyhow::Result<()> {
|
||||
let scratch = std::env::temp_dir().join(format!("mesh-stack-smoke-{}", std::process::id()));
|
||||
|
||||
println!(
|
||||
"=== leg 1: {} MiB worker stacks (tokio default) — expecting stack-guard death ===",
|
||||
TOKIO_DEFAULT_STACK / (1024 * 1024)
|
||||
);
|
||||
let hf1 = scratch.join("hf-2mb");
|
||||
std::fs::create_dir_all(&hf1)?;
|
||||
let (ok_2mb, code_2mb) = run_leg(TOKIO_DEFAULT_STACK, &hf1)?;
|
||||
|
||||
println!(
|
||||
"=== leg 2: {} MiB worker stacks (the fix) — expecting success ===",
|
||||
FIXED_STACK / (1024 * 1024)
|
||||
);
|
||||
let hf2 = scratch.join("hf-8mb");
|
||||
std::fs::create_dir_all(&hf2)?;
|
||||
let (ok_8mb, code_8mb) = run_leg(FIXED_STACK, &hf2)?;
|
||||
|
||||
let _ = std::fs::remove_dir_all(&scratch);
|
||||
|
||||
println!();
|
||||
println!(
|
||||
"leg 1 (2 MiB): success={ok_2mb} exit_code={code_2mb:?} (None = killed by signal, i.e. stack guard)"
|
||||
);
|
||||
println!("leg 2 (8 MiB): success={ok_8mb} exit_code={code_8mb:?}");
|
||||
|
||||
// Leg 2 is the hard gate: the fix must work.
|
||||
if !ok_8mb {
|
||||
anyhow::bail!("FAIL: download did not complete on 8 MiB worker stacks — fix is broken");
|
||||
}
|
||||
// Leg 1 documents the repro. If it *succeeds*, the overflow needs deeper
|
||||
// nesting than this harness provides — flag loudly but do not fail, the
|
||||
// fix leg is still proven.
|
||||
if ok_2mb {
|
||||
println!(
|
||||
"WARNING: 2 MiB leg did not crash here; overflow requires the app's extra \
|
||||
tauri/ipc nesting. Fix leg still verified."
|
||||
);
|
||||
} else {
|
||||
println!("repro confirmed: 2 MiB worker stack dies, matching the crash report");
|
||||
}
|
||||
println!("PASS");
|
||||
Ok(())
|
||||
}
|
||||
Reference in New Issue
Block a user