https://www.puppet.com/community logo
Join Slack
Powered by
This message was deleted.
# puppet-enterprise
s
This message was deleted.
b
what exactly do you want to do? your Puppetfile in 
test
points to your modules with main branch and when main is updated you want to deploy 
test
?
m
Yes, exactly. For example here is a log snippet from my puppet primary server when the webhook is triggered:
Copy code
2023-06-20T17:12:51.351+02:00 DEBUG [qtp1737880586-273] [p.r.m.authentication] Authenticated subject someusername (c910c37b-1a11-4b26-b688-0399703f087d) via authentication token
2023-06-20T17:12:51.352+02:00 DEBUG [qtp1737880586-273] [p.c.core] Validating environment names: main
2023-06-20T17:12:51.491+02:00 INFO  [qtp1737880586-273] [p.c.app] Preparing to deploy: main
2023-06-20T17:12:51.491+02:00 INFO  [qtp1737880586-273] [p.c.app] Queuing deploy for main
2023-06-20T17:12:51.537+02:00 DEBUG [qtp1737880586-273] [p.c.shell-workers] Execute deploy
2023-06-20T17:12:51.538+02:00 DEBUG [deploy-pool-2-1] [p.c.worker-queue] deploy-pool-2-1 starting to deploy environment 'main' from work queue...
2023-06-20T17:12:51.538+02:00 INFO  [deploy-pool-2-1] [p.c.core] Attempting to deploy :environment 'main'...
2023-06-20T17:12:51.538+02:00 DEBUG [deploy-pool-2-1] [p.c.core] Invoking shell: /opt/puppetlabs/bin/r10k deploy --config /opt/puppetlabs/server/data/code-manager/r10k.yaml --verbose warn --cachedir /opt/puppetlabs/server/data/code-manager/worker-caches/deploy-pool-2-1 environment main --modules --incremental
2023-06-20T17:12:51.538+02:00 DEBUG [deploy-pool-2-1] [p.c.core] Shell command will time out after 600 seconds.
2023-06-20T17:12:51.705+02:00 DEBUG [cac4e4c2-e988-41f9-a8ce-f11fd2a77d70_QuartzSchedulerThread] [o.q.s.PropertySettingJobFactory] Producing instance of Job 'SCHEDULER_DEFAULT.1afc9c94c570-c6ec3265-1e52-4f66-8542-2db0fb7b2cad', class=puppetlabs.trapperkeeper.services.scheduler.job
2023-06-20T17:12:51.705+02:00 DEBUG [cac4e4c2-e988-41f9-a8ce-f11fd2a77d70_QuartzSchedulerThread] [o.q.c.QuartzSchedulerThread] batch acquisition of 1 triggers
2023-06-20T17:12:51.705+02:00 DEBUG [cac4e4c2-e988-41f9-a8ce-f11fd2a77d70_Worker-2] [o.q.c.JobRunShell] Calling execute on job SCHEDULER_DEFAULT.1afc9c94c570-c6ec3265-1e52-4f66-8542-2db0fb7b2cad
2023-06-20T17:12:51.706+02:00 DEBUG [cac4e4c2-e988-41f9-a8ce-f11fd2a77d70_QuartzSchedulerThread] [o.q.c.QuartzSchedulerThread] batch acquisition of 1 triggers
2023-06-20T17:12:51.902+02:00 DEBUG [cac4e4c2-e988-41f9-a8ce-f11fd2a77d70_QuartzSchedulerThread] [o.q.s.PropertySettingJobFactory] Producing instance of Job 'SCHEDULER_DEFAULT.1afc9c94c570-8a581e00-fcb6-4257-a22b-bb580e33b654', class=puppetlabs.trapperkeeper.services.scheduler.job
2023-06-20T17:12:51.902+02:00 DEBUG [cac4e4c2-e988-41f9-a8ce-f11fd2a77d70_QuartzSchedulerThread] [o.q.c.QuartzSchedulerThread] batch acquisition of 1 triggers
2023-06-20T17:12:51.902+02:00 DEBUG [cac4e4c2-e988-41f9-a8ce-f11fd2a77d70_Worker-7] [o.q.c.JobRunShell] Calling execute on job SCHEDULER_DEFAULT.1afc9c94c570-8a581e00-fcb6-4257-a22b-bb580e33b654
2023-06-20T17:12:51.902+02:00 DEBUG [cac4e4c2-e988-41f9-a8ce-f11fd2a77d70_QuartzSchedulerThread] [o.q.c.QuartzSchedulerThread] batch acquisition of 1 triggers
2023-06-20T17:12:52.461+02:00 DEBUG [deploy-pool-2-1] [p.c.core] r10k deploy process for environment 'main' completed.
2023-06-20T17:12:52.461+02:00 ERROR [deploy-pool-2-1] [p.c.core] Errors while deploying environment 'main' (exit code: 1):
/opt/puppetlabs/puppet/lib/ruby/gems/2.7.0/gems/rugged-0.27.7/lib/rugged/repository.rb:258: warning: Using the last argument as keyword parameters is deprecated
ERROR    -> Environment(s) 'main' cannot be found in any source and will not be deployed.
/opt/puppetlabs/puppet/lib/ruby/gems/2.7.0/gems/rugged-0.27.7/lib/rugged/repository.rb:258: warning: Using the last argument as keyword parameters is deprecated

2023-06-20T17:12:52.577+02:00 INFO  [deploy-pool-2-1] [p.c.core] Finished deploy attempt for :environment 'main'.
2023-06-20T17:12:52.577+02:00 DEBUG [deploy-pool-2-1] [p.c.worker-queue] deploy-pool-2-1 is ready for new work
2023-06-20T17:12:52.577+02:00 DEBUG [deploy-pool-2-1] [p.c.shell-workers] Execute deploy
2023-06-20T17:12:52.578+02:00 DEBUG [deploy-pool-2-1] [p.c.worker-queue] deploy-pool-2-1 found no work in the queue.
b
switch the default branch in your modules to 
test
and configure the webhook for all modules
m
In this example above I pushed a change in a module with a main branch.
Mhh not a fan of that
b
or rename your test branch in the control-repo to main
or setup a middleware that rewrites the webhook payload, like huginn
m
This isn't about a test branch for now, this is just the "basic" of setting up a Webhook to trigger a deployment of Puppet environments
From the docs there is a hint to use a prefix to actually solve this. how does come into play?
b
as an alternative, your pipelines for the modules can have a last step that triggers a code deploy
prefixes are for multiple control repos
m
mhh I guess I have to think about it more for a bit
b
if you've more questions let me know 🙂
m
It seems there is a gap in either my knowledge or the implementation/featureset where you have a Puppetfile in a control repo on a branch 
production
, 
nonprod
or 
test
and your own modules using 
main
where you can't ever find a way to map this to the given environment. This like a very common setup and I'm baffled this isn't… easier?! Maybe this is just me not fully understanding all the moving parts and how they interact.
The workaround would be a pipeline for each Bitbucket Repo of my modules to kick Jenkins to POST /v1/deploys
b
let's assume you've a control repo with 600 branches, not just 3. a branch of a module is updated. Do you expect that code-manager checks all 600 puppetfiles for that module and redploys all environments/just that module in the environments? That's quite resource intensive and partial deploys are an antipattern
and best practice is to version modules with git tags/forge releases and use branches only in test environments
m
We use that
b
and doing the POST to the deploy endpoint isn't a workaround, that's how it's supposed to be
if you move the logic to code-manager you eventuall ddos yourself or put environments in an unknown state
m
But why does the webhook exist in the first place when the docs say "it's the easiest way (but psst big asterisk when it comes to certain things)
b
the webhook is designed for your control repo
and it works perfectly fine there
and the limitation comes from the git server, not code-manager. your bitbucket/gitlab/github payload contain the updated branch ref, in your case main. that doesn't exist in the control-repo, so how should code-manager know what to deploy
m
because I thought the way you would do it is by telling code manager via a URI parameter how to map the branch name
b
no 🙂
m
As I said in the beginning: I am apparently not getting the functionality 🙂
So I can't and shouldn't use the webhook on a module but only on the control repo where the branch names match the puppet environment?
b
you can use it for modules but their branch names need to match the control-repo branch branches
(which is common)
m
(which we, the two of us, talked about a couple weeks ago and we are moving away from that)
b
ah 😄
m
😛
So our new setup is like this: production and nonprod (puppet environment) use git tagged modules on main and test (puppet environment) is using main "live". We want to add feature branches to the mix in pull requests but my colleague who's hacking on it hasn't gotten very far yet.
b
ah yeah
m
Yeah so now I thought "lete just quickly solve this problem of deploying code-manager automatically without a Jenkins" but apparently not
b
so you want to create with feature branches in your modules? and create a new temporary branch in the control-repo that maps to it?
m
Yup, but that's version 2 as I first want to solve deploying our environments first ^^
b
I suggest to setup code-manager for the control repo and a step in your CI to deploy test environment when main branch changed in the modules
m
yeah I guess we have to go that route… that makes sense and seems like the only way to go apparently
👍 1
Thanks for walking me through it and your time 🙂
b
your welcome 🙂
m
so you want to create with feature branches in your modules? and create a new temporary branch in the control-repo that maps to it?
Ah, sorry one more Q: Do you know of documentation or info on how to build this? Is there some blog where I could read?
b
Copy code
# Track control branch and fall-back to master if no matching branch.
mod 'hieradata',
     :git    => '<mailto:git@git.example.com|git@git.example.com>:organization/hieradata.git',
     :branch => :control_branch,
     :default_branch => 'main'
do something like that for all your modules
then code-manager will check during deployments of any environment if the module contains a matching branch. if that's not the case, 
main
will be deployed
so when you want to create a new environment and test a feature branch of a module, you only need to create a matching branch name in the control-repo, without editing the Puppetfile
m
Thanks!
And 
:branch => :control_branch,
will be the name of the feature branch for that module then?
b
:control_branch
is a pointer to the current environment name/branch name in the control repo
m
Ah! So the control repo needs to have a branch with the same branch name as the feature branch of the module I want to deploy
b
yes, as I said
👍 1
m
Sorry, just rephrasing as I didn't get it
b
sure, no problem
Open in Slack
PreviousNext
Powered by