http://coldfusion.com logo
Join Slack
Powered by
Has anyone had an automatic build fail because for...
# box-products
j
Has anyone had an automatic build fail because forgebox install fails because github blocked the download with a "too many requests" error? Any ways we can make our build more reliable by packaging forgebox dependancies into our local repo?
b
I've never seen that one, but to be clear, it's Github blocking you, not forgebox.
I would recommend the package maintainer use forgebox storage for their package artifacts, which comes from S3 and has no limit on requests
There's nothing we can do if a package maintainer decides to store their binaries on a server that restricts requests.
CommandBox could re-try failed requests 3 times or something
Can you put in a ticket for that idea?
j
Yes its github blocking, now our builds can't deploy. Is it possible to download the package zip directly and install it ourselves through box? like an offline type install?
I will raise an issue on the maintainer's github page
@bdw429s I think you should consider making forgebox cache external binaries for packages or provide a way users can do an offline install to make box install a more resilient process
b
@Jordan Clark The problem here is that forgebox doesn't ever touch or even see those binaries!
When you install this package, your PC directly makes the call to github (or whatever server) and forgebox is not involved
It's also not safe to assume forgebox even has access to them. A private package could have a location of
Copy code
<http://totalInternalSecretDomain.gov:1234/package.zip>
and it could be only accessible from that person's PC
j
when someone publishes the package to forgebox, in this case the box.json provided:
Copy code
"location" : "<https://github.com/cfsimplicity/spreadsheet-cfml/archive/v3.7.2.zip>",
you could download/cache a copy of the binary, and then when a user installs you provide the forgebox cache as the location, with the original as a backup
b
But, again, the package maintainer must use it ¯\_(ツ)_/¯
It is not a safe assumption for CommandBox to think the same URL will always yield the same binary
What about something like
Copy code
<http://s3-url.com/package/latest-bleeding-edge-snapshot.zip>
That zip file could change several times a day
which is why the cached-http endpoint is opt-in
j
I see your point
Flip side is packages should be consistent, so if zip files are changing randomly that could also affect builds, so a unique cache for each version of a package would prevent that
b
Not neccessarily
It's pretty common to have a snapshot or BE (bleeding edge) version that simply points to the latest code
Which I prefer much more than having 1 billion different snapshot builds (like Lucee does)
So if you want your build to not change between runs, the onus is on you not to use a BE or snapshot version
j
My workaround for now is I've downloaded the zip file, put in my own s3 bucket, and reinstalled so my box.json points our copy of the file:
Copy code
box install id="<https://hs-docker-assets.s3.us-east-1.amazonaws.com/cfmodules/spreadsheet-cfml-3.7.2.zip>" save=true
c
@bdw429s As the maintainer of this repo I'm happy to help here if I can, as long as the authoritative binaries remain on Github. The artifact caching sounds like it would easily sort things, but I'm not clear from the link you posted what I need to do. My box.json has this:
Copy code
"location" : "<https://github.com/cfsimplicity/spreadsheet-cfml/archive/v3.7.2.zip>",
Do I just add the 
+cached
to that URL?
b
Yeah, that should do it
👍 1
Copy code
"location" : "<https+cached://github.com/cfsimplicity/spreadsheet-cfml/archive/v3.7.2.zip>",
So, as long as Jordan has the file in his local artifacts cache, it won't actually download on subsequent installs
j
but this is happening in a clean pipeline build process, its never going to have the file in a local cache
b
Well, that's one of the many issues with builds pipelines, lol
There are ways to seed your artifacts. People do it a lot for this exact reason
j
So then I'm back to the same position, I'd have to download my own copy of the file from my s3 bucket and seed the cache
b
as long as the authoritative binaries remain on Github
@cfsimplicity Given you've chosen a place to host your artifacts that throttles traffic, is there a compelling reason not to use ForgeBox storage, which places the artifacts on S3?
That's not the _on_ly solution. Given your cloud pipeline, many of them have features to cache certain directories across builds
CommandBox also allows the artifact folder to be configured anywhere you like. https://commandbox.ortusbooks.com/package-management/artifacts#custom-artifacts-directory
j
@bdw429s can the artifacts directory read directly off an S3 location?
c
I'm hosting on Github coz that's where the repo is (and that's a fairly popular choice!) How would I go about hosting the binaries on Forgebox?
b
It depends a little on how your build works. If you manually public to forgebox via the web UI, you can just provide a zip file. Or if just using the
Copy code
publish
command, then it's all zipped up and stored for you
can the artifacts directory read directly off an S3 location?
CommandBox needs a local folder. Now, if you have some trick to create a folder on Linux or whatever that points to S3, then that would work too.
c
I use commandbox to publish releases to FB. So I instead need to make this change in 
box.json
?
Copy code
"location" : "forgeboxStorage",
b
Yes, by default, all the files in the project root will be zipped and sent to forgebox minus your gitignores and box.json ignores
👍 1
c
OK, @Jordan Clark I've followed Brad's instructions and published a new 3.7.3 release which will hopefully now be hosted on Forgebox.
b
Looks like it's working
Copy code
❯ install spreadsheet-cfml --verbose
 √ | Installing package [forgebox:spreadsheet-cfml]
   |-------------------------------------------------------------
   | Verifying package 'spreadsheet-cfml' in forgebox, please wait...
   | Installing version [3.7.3].
   | Verified entry in forgebox: 'spreadsheet-cfml'
   | Downloading entry from forgebox.
   | Deferring to [https] endpoint for forgebox entry [spreadsheet-cfml]...
   | Downloading [<HTTPS://ortus-forgebox-private.s3.us-east-1.amazonaws.com/cfsimplicity/spreadsheet-cfml/3.7.3.zip?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAJMDMWUSW7FUVMSFQ%2F2022112>
   | 2%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20221122T182747Z&X-Amz-Expires=300&X-Amz-SignedHeaders=host&X-Amz-Signature=bf17618725ed755590a5829d9861e2aa4a631bae9ec98970b6637457b3f5864a
   | Decompressing...
   | Storing download in artifact cache...
   | Done.
   | C:\Users\Brad\Desktop\sandbox\spreadsheet-test\/box.json updated with dependency.
   | Installing to: C:\Users\Brad\Desktop\sandbox\spreadsheet-test\/modules/spreadsheet-cfml
   | -> 75 File(s) Installed
   | -> 0 File(s) ignored
   | Eureka, 'spreadsheet-cfml' has been installed!
   |-------------------------------------------------------------

 4sec 783ms  ✓  CLI v1.2.3  12:27 PM  ~/Desktop/sandbox/spreadsheet-test/   (1.0.0) 
❯ ll
Nov 22,2022  12:11:51              <DIR>   modules/
Nov 22,2022  12:11:51             0.2 KB   box.json

 ✓  CLI v1.2.3  12:28 PM  ~/Desktop/sandbox/spreadsheet-test/   (1.0.0) 
❯ package show
{
    "dependencies":{
        "spreadsheet-cfml":"^3.7.3"
    },
    "installPaths":{
        "spreadsheet-cfml":"modules/spreadsheet-cfml/"
    }
}

 ✓  CLI v1.2.3  12:28 PM  ~/Desktop/sandbox/spreadsheet-test/   (1.0.0) 
❯ ll modules/spreadsheet-cfml/
Nov 22,2022  12:11:51              <DIR>   .github/
Nov 22,2022  12:11:51              <DIR>   helpers/
Nov 22,2022  12:11:51              <DIR>   javaLoader/
Nov 22,2022  12:11:51              <DIR>   lib/
Nov 22,2022  12:11:51              <DIR>   objects/
May 14,2021  17:05:22             0.2 KB   .gitattributes
Nov 22,2022  18:11:14            23.8 KB   CHANGELOG.md
Jan 15,2022  17:01:22             1.1 KB   LICENSE
Nov 22,2022  18:11:14             0.5 KB   ModuleConfig.cfc
Oct 12,2022  14:10:20            16.5 KB   README.md
Nov 22,2022  18:11:14            83.8 KB   Spreadsheet.cfc
Nov 22,2022  12:11:49             1.5 KB   box.json
Nov 14,2022  17:11:54            20.1 MB   lib-osgi.jar
Jan 20,2021  12:01:28             2.0 KB   osgiLoader.cfc
☝️ I asume those are the files expected when the package is installed
c
Yes, that looks fine.👍 Good work as always Brad on making this easy simple smile
👍 2
j
@bdw429s @cfsimplicity thanks so much guys for your attention and help!
👍 1
2 Views
Open in Slack
PreviousNext
Powered by