I would like to contribute concurrent component provisioning, here is a proposal for discussion: https://github.com/cloudposse/atmos/pull/2159 Currently we are using our own github actions workflows that are patched to implement dependencies, but it is really ugly to do it via GH and matrix, Possibly it should be implemented after this one: https://github.com/cloudposse/atmos/pull/1891

yes, I'll get it to the finish line

sure I'll be happy to validate 1516, missed that one 👍

@shirkevich You can test 1516, by simply doing:

atmos terraform ... --use-version=1516

or by setting

ATMOS_USE_VERSION=1516

and atmos will automatically download and install it, then execute it.

https://github.com/cloudposse/atmos/pull/1516 was updated and merged to

main

Ok, @shirkevich please update your PRD based on what's now in main and we will take a closer look and the proposal so you can begin

After PR #1516, the routing on

origin/main

is still:

log.Debug("Routing to ExecuteTerraformQuery (multi-component)")
		err = e.ExecuteTerraformQuery(&info)

--all

,

--components

, and

--query

all enter through

isMultiComponentExecution

and land in

ExecuteTerraformQuery

, which walks stacks sequentially with no graph.

ExecuteTerraformAll

is never called from any command routing. The only references are its own definition, tests, and the PRD doc: 0b67828a4:internal/exec/terraform_all.go (definition) 0b67828a4:internal/exec/terraform_all_simple_test.go (test) 0b67828a4:internal/exec/terraform_all_test.go (test) 0b67828a4:docs/prd/terraform-dependency-order.md (docs) So

ExecuteTerraformAll

with its graph construction, dependency ordering, and

executeInDependencyOrder

loop is effectively dead code from the CLI's perspective -- it shipped as library code with tests but was never wired into the command tree. is it correct? so 1516 can't be tested yet with real codebase? or am I missing smth?

Apologies, this shouldn't have been merged. There was a misunderstanding on our end.

yep, I'll wait for those changes to be shipped (can help to test those) to fix properly proposal

@shirkevich apologies for misunderstanding we'll fix the missing functionality (from https://github.com/cloudposse/atmos/pull/1516), when will work with you on your proposal

https://github.com/cloudposse/atmos/pull/2159/changes#diff-3fb6d1b034115d50257da8a71370fca7a8813ada4ab8dac3f2f6bd544c3cdd5b proposal is ready to be reviewed. looks more like an epic from implementation pov. tried to split possible implementation to several phases to not make it humongous.

thank you

One thing I'd like to understand more is what packages would be developed to support this.

Don't have any opinion on how it should be packaged internally, need your guidance on that. So parallel execution should be universal to use afterwards also for helm/ansible/etc?

let me get back to you maybe by tomorrow with some more constructive feedback

@Erik Osterman (Cloud Posse) thank you, my PRD was so naive 🙈

Do you think this is still something you want to help us tackle?

sure, few question on design: A scheduler solves DAG ordering and concurrency, but it does not by itself solve: • stream isolation • signal forwarding / graceful shutdown • stdin policy for interactive commands • per-node log capture and labeling • child-process cleanup and exit handling Those are process-management concerns, and I think they deserve a separate reusable package boundary, something like pkg/process or pkg/taskrun. A structure like this seems cleaner and more reusable: pkg/scheduler • ready queue / worker pool • blocked/skipped/fail-fast semantics • aggregate result state pkg/process • spawn / wait / stop / kill • injected stdin/stdout/stderr • per-node log sinks • signal handling / cleanup tool adapters • Terraform, Helmfile, Ansible, etc. • prepare env/cwd/hooks/auth • define node completion semantics wdy?

package example

import (
	"context"
	"io"
	"time"
)

// Node is pure scheduling data.
// It does not know how to execute itself.
type Node struct {
	ID   string // unique DAG node ID: "terraform/tenant1-ue2-dev/vpc"
	Type string // "terraform", "packer", "ansible"
	Ref  any    // tool-specific payload prepared earlier
}

// Streams are injected by the orchestrator/process layer.
// This is what lets concurrent nodes avoid shared stdio.
type Streams struct {
	Stdin  io.Reader
	Stdout io.Writer
	Stderr io.Writer
}

// TaskSpec is the generic process contract.
// Adapters translate tool-specific nodes into this shape.
type TaskSpec struct {
	Command     []string
	Dir         string
	Env         []string
	Interactive bool
}

// Result is the generic execution result returned by the process layer.
type Result struct {
	ExitCode   int
	StartedAt  time.Time
	FinishedAt time.Time
	Err        error
}

// Runner owns subprocess lifecycle.
// This is the reusable process package, not Terraform-specific code.
type Runner interface {
	Run(ctx context.Context, spec TaskSpec, streams Streams) (*Result, error)
}

// Adapter owns tool-specific execution semantics.
// Scheduler uses adapters; adapters use the process layer.
type Adapter interface {
	Prepare(ctx context.Context, node *Node) (TaskSpec, error)
	Finalize(ctx context.Context, node *Node, result *Result) error
}

// Registry resolves adapters by node type.
type Registry interface {
	MustGet(nodeType string) Adapter
}

// Orchestrator is the generic DAG executor.
// In reality it would also own dependency tracking / ready queue / concurrency.
type Orchestrator struct {
	runner   Runner
	registry Registry
}

// ExecuteNode shows the intended boundary:
// scheduler/orchestrator drives execution,
// adapter translates tool semantics,
// process runner owns subprocess execution.
func (o *Orchestrator) ExecuteNode(ctx context.Context, node *Node, streams Streams) (*Result, error) {
	adapter := o.registry.MustGet(node.Type)

	// Tool-specific preparation:
	// auth/bootstrap, hooks, cwd/env resolution, command construction, etc.
	spec, err := adapter.Prepare(ctx, node)
	if err != nil {
		return nil, err
	}

	// Generic subprocess lifecycle:
	// stream isolation, signal forwarding, wait/kill, exit handling.
	result, err := o.runner.Run(ctx, spec, streams)
	if err != nil {
		return nil, err
	}

	// Tool-specific completion semantics:
	// hooks, store writes, planfile handling, post-processing, etc.
	if err := adapter.Finalize(ctx, node, result); err != nil {
		return nil, err
	}

	return result, nil
}

// -----------------------------------------------------------------------------
// Terraform adapter
// -----------------------------------------------------------------------------

type TerraformInfo struct {
	Dir string
	Env []string
	// plus stack/component/auth/workdir/hook metadata
}

type TerraformAdapter struct{}

func (a *TerraformAdapter) Prepare(ctx context.Context, node *Node) (TaskSpec, error) {
	info := node.Ref.(*TerraformInfo)

	// In real code this is where Terraform-specific prep would happen:
	// - validate workdir
	// - resolve auth context
	// - build atmos/terraform args
	// - decide whether interactive mode is allowed
	return TaskSpec{
		Command: []string{"terraform", "apply", "-auto-approve"},
		Dir:     info.Dir,
		Env:     info.Env,
	}, nil
}

func (a *TerraformAdapter) Finalize(ctx context.Context, node *Node, result *Result) error {
	// In real code:
	// - run after-apply hooks
	// - update stores
	// - publish plan/apply metadata
	return nil
}

// -----------------------------------------------------------------------------
// Packer adapter
// -----------------------------------------------------------------------------

type PackerInfo struct {
	Dir string
	Env []string
}

type PackerAdapter struct{}

func (a *PackerAdapter) Prepare(ctx context.Context, node *Node) (TaskSpec, error) {
	info := node.Ref.(*PackerInfo)

	return TaskSpec{
		Command: []string{"packer", "build", "."},
		Dir:     info.Dir,
		Env:     info.Env,
	}, nil
}

func (a *PackerAdapter) Finalize(ctx context.Context, node *Node, result *Result) error {
	return nil
}

// -----------------------------------------------------------------------------
// Ansible adapter
// -----------------------------------------------------------------------------

type AnsibleInfo struct {
	Dir string
	Env []string
}

type AnsibleAdapter struct{}

func (a *AnsibleAdapter) Prepare(ctx context.Context, node *Node) (TaskSpec, error) {
	info := node.Ref.(*AnsibleInfo)

	return TaskSpec{
		Command: []string{"ansible-playbook", "site.yml"},
		Dir:     info.Dir,
		Env:     info.Env,
	}, nil
}

func (a *AnsibleAdapter) Finalize(ctx context.Context, node *Node, result *Result) error {
	return nil
}

// -----------------------------------------------------------------------------
// Example nodes
// -----------------------------------------------------------------------------

func ExampleNodes() []*Node {
	return []*Node{
		{
			ID:   "terraform/tenant1-ue2-dev/vpc",
			Type: "terraform",
			Ref: &TerraformInfo{
				Dir: "/work/terraform/vpc",
				Env: []string{"AWS_PROFILE=dev"},
			},
		},
		{
			ID:   "packer/tenant1-ue2-dev/ami-builder",
			Type: "packer",
			Ref: &PackerInfo{
				Dir: "/work/packer/ami-builder",
				Env: []string{"AWS_PROFILE=dev"},
			},
		},
		{
			ID:   "ansible/tenant1-ue2-dev/playbook",
			Type: "ansible",
			Ref: &AnsibleInfo{
				Dir: "/work/ansible/playbook",
				Env: []string{"ANSIBLE_CONFIG=/work/ansible/ansible.cfg"},
			},
		},
	}
}

Sounds good to me. For stream isolation, hoping we can extend/reuse our current io package to assist with that, since it handles all the IO masking required for secrets handling.

https://github.com/cloudposse/atmos/pull/2079 what is the status of native ci integration? will it be shipped soon?

I'm pushing hard that we can merge this today.

ok, I can start with implementation of IO stuff while it is baking, I'll split this work to several PRs: 1. IO 2. Process isolation 3. Scheduler 4. Terraform adapter for scheduler 5. Packer/helm/ansible/etc - maybe pass it to someone who can better test it

sounds good!