01The whole idea
One sentence, then the consequence.
M2 — distributor-pull.
A worker pod ships with only an environment.json (delivered in a Kubernetes ConfigMap).
At boot it pulls its .gtbundle from a distributor over the network,
digest-verifies the bytes, materializes the packs to disk,
activates the routed revision, and serves it over HTTP.
That is the difference between a toy and a cloud runtime. The image baked into the pod knows nothing about which packs it runs — that's decided by control-plane state and resolved at runtime. The same immutable image can serve any tenant, any revision, any bundle. Nothing about the workload is welded into the container.
Greentic's layering makes this clean. Each layer is a sealed artifact the one above composes:
The demo exercises the bottom-right of that chain: the Operator authors desired state,
the Runner (a greentic-start pod) pulls the Bundle and brings it to life.
02Why pull at boot, instead of baking the bundle in?
The constraint that shapes the entire design.
❌ Bake the bundle into the image
- A new image build for every revision of every tenant
- Registry bloat; slow rollouts
- The runtime image is no longer generic
- Secrets / packs entangled with the binary
✅ Pull the bundle at boot (M2)
- One immutable runtime image, forever
- Revisions change = ConfigMap changes, no rebuild
- Pod is configured by control-plane state, not by its image
- Integrity guaranteed by a pinned
sha256, not by trust in the registry
The cold state a pod boots from is deliberately minimal. The deployer materializes packs and
a runtime-config.json on the authoring host — but a cloud pod never receives those.
Its ConfigMap carries environment.json and nothing else. The demo reproduces this exactly
by deleting runtime-config.json and revisions/ before boot — byte-for-byte
the state a real pod wakes up in. That emptiness is what forces the pull.
03What happens when the worker boots
The core mechanism — greentic-start start --env local, step by step.
The pod's container command is just greentic-start start --env <id>. It reads the env store,
finds an empty runtime-config, and the boot seam takes over:
environment.json. There are no materialized packs on disk. The boot seam detects rc.revisions.is_empty().bundle_source_uri
bundle_source_uri says where to fetch the bundle from (HTTP / OCI / file).revision_pull::fetch_bundle_to_file
.gtbundle from the distributor. No early trust — just bytes on disk.bundle_digest
sha256 pinned at stage time is the integrity backstop. If the pulled bytes don't match, boot aborts — a tampered or wrong bundle can never activate.materialize_revision_from_bundle
runtime-config.json. Failure-atomic: an existing live revision is moved aside and rolled back on error, never half-bricked.runtime-config.json)./healthz → 200
serving probes only. That banner is the exact assertion the K8s E2E makes.
Why the digest matters more than the source. bundle_source_uri is an
opaque, caller-asserted reference — the worker will fetch whatever it points at.
Trust does not come from the URL; it comes from the bundle_digest pinned on the
revision at stage time. Breadth of the source ≠ injection risk, because the bytes are gated before
anything is unpacked.
04The commands — authoring the deployment
Every op verb the operator runs, in order, and what it records.
All authoring happens on the control-plane host with the deployer's op verbs.
Each verb writes to the env store (the thing that becomes the in-cluster ConfigMap). Nothing
touches a cluster until reconcile.
| Command | What it does |
|---|---|
env create |
Writes environment.json — the seed of the whole env. Becomes the gtc-env-store ConfigMap in-cluster. |
trust-root bootstrap |
Seeds the operator's Ed25519 signing key. bundles add refuses to run without it — packs are signed artifacts. |
bundles add |
Registers a deployment — a routing target. route_binding.path_prefixes:["/"] routes every request path to it. Returns a deployment_id. |
revisions stage M2 key step |
Records two things on the revision: • bundle_digest = sha256 of the local fixture → the integrity backstop• bundle_source_uri = where a remote worker will pull the bundle from (the M2 field). Returns a revision_id.
|
revisions warm |
Flips the revision lifecycle staged → ready. In-cluster this waits for the rollout to actually become healthy before proceeding. |
traffic set |
Routes traffic (weight_bps: 10000 = 100%). Only routed revisions get pulled, so this is what arms the boot-time fetch. Idempotency-keyed. |
env-packs add K8s only |
Binds the K8s deployer (greentic.deployer.k8s@1.0.0). This is what makes the env a Kubernetes env. Pins the runtime image for the rendered pods. |
env reconcile K8s only |
The only verb that touches the cluster. Server-side-applies all desired objects against the real API server (idempotent, converging). |
Authoring a pullable revision (Track 1 — loopback HTTP distributor)
# 1. env + trust root op env create # writes environment.json op trust-root bootstrap local # 2. a routing target op bundles add # route_binding.path_prefixes = ["/"] → deployment_id # 3. THE M2 STEP — stage a PULLABLE revision op revisions stage \ bundle_path=perf-smoke-bundle.gtbundle # → pins its sha256 as bundle_digest \ bundle_source_uri=http://127.0.0.1:8090/bundle.gtbundle # where to pull # 4. ready it, then send 100% of traffic (only routed revs are pulled) op revisions warm # staged → ready op traffic set # weight_bps = 10000 → revision_id
The fixture: perf-smoke-bundle.gtbundle (4 KiB, sha256:c5199a71…).
It's the one ready-made valid bundle in the workspace, and its pack carries no .wasm —
which is perfect here. It activates and serves /healthz, proving the
pull → materialize → activate chain, without needing a component runtime.
(A real request can't run a flow with it — that's the honest edge.)
05Two tracks — and what each one proves
The demo can run the same pull code in two places. ./demo.sh <cmd>.
| Track | What it runs | Real K8s? | Local status |
|---|---|---|---|
| 1 · host-pull | The M2 pull/serve code on the host, no cluster. Reproduces a pod's cold boot exactly — same pull → digest-gate → materialize → activate → serve code. | No (simulates the pod) | ✅ fully verified |
| 2 · kind-serve | A real worker POD boots start --env, reaches Ready, serves /healthz. Readiness probe = kubelet → pod. |
Yes | ✅ green locally |
| 2 · kind-pull | The full in-cluster pull: worker pod pulls (pod → pod) from an in-cluster busybox-httpd serving the bundle, logs the real-revision banner. | Yes | ⚠️ CI-only here |
The whole demo, end to end
cd /home/vampik/greenticai/my_demos/k8s-deploy-demo ./demo.sh host-pull # Track 1 — start here. ~10s, no Kubernetes needed. ./demo.sh host-stop # stop the worker it leaves running ./demo.sh kind-up # Track 2 — create a kind cluster + load the :develop runtime image ./demo.sh kind-serve # prove a worker POD boots + serves /healthz (local-green) ./demo.sh kind-pull # full in-cluster pull (CI-green) ./demo.sh kind-down # delete the cluster
Track 1 proves the code is correct
It runs the identical pull/materialize/activate/serve path, just outside a pod. The only thing it skips is the pod→pod transport. 100% reproducible on any machine.
Track 2 proves it works inside Kubernetes
Same objects, same pod boot command, same reconcile path as cloud (EKS/GKE/AKS). kind is just K8s running as Docker containers on your machine.
Why kind-pull is CI-only on this machine. The final hop is the worker pod pulling
pod → pod from an in-cluster httpd pod. On this host (CachyOS 7.0.11 + docker 29.5.3),
same-node pod-to-pod networking has 100% packet loss — a local kindnet/kernel defect, not a
code or test defect (kubelet → pod is fine, which is why kind-serve works).
The exact same test is green on CI's ubuntu-latest. Track 1's host-pull is the
local stand-in for precisely this path.
06Inside Kubernetes — control-plane vs data-plane
The split that holds true on real cloud too.
| What | Where it runs | Verb / command |
|---|---|---|
| Author the env create · env-packs add · bundles add · stage · warm · traffic |
Host / control-plane. You build the env store, which becomes a ConfigMap in-cluster. Nothing touches the cluster. | op … |
| Converge onto the cluster | Against the cluster. Server-side apply of every desired object onto the real API server. | op env reconcile |
| Boot, serve, pull | Inside the cluster. Worker / router pods boot start --env, serve /healthz, pull their bundles. |
the pod's container command |
Visually — what lives where during a K8s deploy:
CONTROL PLANE · your host
- env store on disk (
environment.json+ trust root) opverbs author desired stateop env reconciletalks to the API server- Holds the bundle
sha256+bundle_source_uri
- gtc-env-store ConfigMap — only
environment.json - worker pod boots
greentic-start start --env - Pulls bundle from
bundle_source_uri→ digest-gate → serve - Readiness probe hits the
/healthzit serves
The K8s-only delta vs the host track
# Bind the K8s deployer — THIS makes the env a Kubernetes env op env-packs add \ slot=deployer kind=greentic.deployer.k8s@1.0.0 pack_ref=builtin \ answers_ref=deployer-answers.json # pins runtime_image for the rendered pods # …author the revision exactly as Track 1… # The ONLY verb that touches the cluster: op env reconcile local # server-side apply → 14 objects # Watch a real Greentic worker come up inside Kubernetes: kubectl rollout status deployment/gtc-worker-<revid> -n gtc-local kubectl logs deployment/gtc-worker-<revid> -n gtc-local | grep "revision(s) for env" # SUCCESS: serving 1 revision(s) for env `local` (NOT "serving probes only")
The worker Deployment is named after the revision ULID: gtc-worker-<lowercased revision_id>.
Both router and worker pods boot the same start --env command.
07What reconcile applies — the 14 objects
12 env-level objects + 2 per warmed revision.
12 env-level objects
- Namespace
gtc-local - ConfigMap
gtc-env-store— carriesenvironment.json(with the routed revision + itsbundle_source_uri) - ConfigMap
gtc-runtime-config - Deployment
gtc-router - Service + PodDisruptionBudget for the router
- 6× NetworkPolicy — default-deny + scoped allows (incl. router-egress so the router can pull its bundle at boot)
+2 per warmed revision
- Deployment
gtc-worker-<revid>— bootsstart --env - Service for the worker
Archive the revision and re-reconcile → the worker pair is pruned back to the 12 env-level objects. Reconcile converges both directions; it's not append-only.
NetworkPolicy is load-bearing. The env's gtc-default-deny denies ingress to every pod,
and kindnet v1.31+ enforces it. That's why pulling needs an explicit allow — the router gets a
router-egress policy, and a worker pulling at boot needs a worker-egress allow. In real cloud the bundle
source is an external registry (outside the namespace, unaffected); in-cluster, the allow is the fixture's concern.
08How the local demo maps to real cloud
Every local stand-in, and its production counterpart.
| Demo (local) | Real cloud (EKS / GKE / AKS) |
|---|---|
kind create cluster |
The managed cluster you provisioned |
state/.greentic/…/environment.json |
The gtc-env-store ConfigMap mounted into the pod |
Stripping runtime-config.json + revisions/ |
The pod simply never receives them — the ConfigMap carries only environment.json |
python3 -m http.server on :8090 · in-cluster busybox-httpd |
The pack distributor / OCI registry the bundle_source_uri points at |
env-packs add binding greentic.deployer.k8s@1.0.0 |
The env declaring it deploys to Kubernetes |
op env reconcile local |
The operator / control-plane converging desired state onto the cluster |
Worker pod booting start --env |
Identical — the same container command |
The single production transport this machine can't exercise — a worker pod pulling over the cluster network — is exactly what CI covers, and what Track 1's host-pull substitutes for locally. Everything else in the chain runs unchanged from laptop to cloud.
09The honest edge — what a 200-with-an-error means
Where the fixture stops, and what comes next.
While the worker runs you can poke it. /status confirms it activated a real revision:
Probe the running worker
curl -s http://127.0.0.1:8080/status {"bundles_active":1,"deployments_routed":1,"env_id":"local","revisions_active":1,...} curl -s -X POST http://127.0.0.1:8080/ -H 'content-type: application/json' -d '{"text":"hi"}' # HTTP 200, body: [{"kind":{"kind":"custom","action":"response"}, "payload":{"metadata":{"error_kind":"flow_execution_failed", "error_message":"flow main has no start node",...}}}]
That POST is the honest edge of the fixture. The request reaches the
pulled-and-activated revision and is dispatched into its flow — the whole chain works —
but perf-smoke ships no real flow graph, so the flow itself fails with
flow main has no start node.
A useful 200 reply needs a bundle whose pack ships components (.wasm).
That is M3 — the next milestone. M2 proves the delivery; M3 proves the work.
The milestone ladder
/healthzstart --env, binds 0.0.0.0, reaches Ready. Probes-only. ✅ shipped..wasm components, so a flow can actually run and return a useful reply. ⏳ next.