Is anyone using the Github Copilot Business plan w/CFML? I've searched the CFML Slack archives and found some mixed reviews. Just curious how well people have found it working with CFML. Since the amount of open source CFML is pretty limited (compared to other languages), I'd assume it's not great.

I've been using it and I think currently your assumptions are mostly correct - it doesn't really have enough code to draw good practice from (there have been some shockers I've seen that must have come from some public repos of people learning CF). However, it has really sped up my development, simply by virtue of filling in all the time-consuming stuff for me - so I can start writing a line and 8 times out of 10 it'll second-guess what I'm doing and finish it correctly for me. That's well worth the $10/month. And then of course I spend a lot of time on projects NOT doing just CF, but JS and HTML and CSS etc etc, and for those it's fantastic. Caveat: IF you ensure you take a stance of NOT trusting it one iota. ie - you still need to take the time to review what it comes out with as you would if a newb junior developer offered to write a function for you. So in summary, is it changing the way I write CF? Not one bit. Is it speeding my development up? Totally.

^^^ agreed I find it falls back to writing JS instead of CFScript too often to be helpful.

I think @gpickin and @Grant Copley have started playing with it too.

Just had a good example where I'm splitting the current cgi.server_name and it thought this was the solution to complete the line:

cgi.server_name.split(".");

whereas of course String.split() takes a regex so the above would give you a zero length array as the . needs to be escaped.

cgi.server_name.split("\.");

It's exactly stuff like that that means you totally cannot trust it an inch and have to review everything it suggests.

I find it to be very helpful with comments ;)

@Rich Wild obviously your tests will save you here right? I mean even if you got Co-pilot to write the test for that split operation you're doing, it'd be really clear from the test that it was thinking about things wrong. Or the test would just fail if it go the test right and the implementation wrong...

Totally, but that seems a bit cart before the horse to me.

Well... yer tests should always come before your implementation...

Sure, but you're going off on a different point, whereas I'm just trying to get across that general idea you can't just sit back drinking Piña Coladas whilst it does it all for you. Not yet, anyway.

I think we minimally covered "the point" with the various "no" and "cannot" responses. I was strategising further (or even just making the legit baseline observation) that simply relying on it nailing implementation code first time round (your

split

example), is only about as legit a gripe as for humans that only rely on one pass of the implementation code and that's it. It's not a way that one should be putting it (Copilot) to work. I'd say a human dev that works like that is failing at the job at hand as well. It doesn't seem like a reasonable measure of how good a wrench is if you only test it by hammering in nails. I'll also observe it's called "Copilot" not "Autopilot"... 😄 Now... you said you were getting me a pina colada, didn't you??

Just tried to stub a couple tests in chatGPT using Testbox and Mockbox and actually gave a good starting point without the normal cut and paste.

@Rich Wild Same experience here. Its ability to fill in or guess what I’m typing is pretty good and has been a great time saver. Definitely worth the $10/mo just for that alone. I agree with others here. It saves time but you still will need to carefully review the code it’s generating

"Sometimes it goes full-on stupid" but who doesn't at times? 😉

@spills Exactly 😂

Sometimes it goes full-on stupid

Maybe we're not thinking this through... maybe it's emulating a CFML dev really frickin well 😉

it only need to know 5 tags. 🤷‍♂️

I got the free plan because I maintain enough open source to qualify -- so I have Copilot installed on my laptop (where I do OSS) but not on my desktop (where I do paid work). I find it's mostly pretty crappy for writing production style code (i.e., source) but it's really pretty good at inventing tests and so that's mostly what I use it for, helping me writing test boilerplate and coming up with random variations of code for use in tests. I don't know whether it would be worth $10/month for work -- for CFML or Clojure -- but if I mostly wrote JS, I suspect it would be better. One big legal caveat: it may well suggest whole-cloth code from licensed OSS projects so you may inadvertently copy someone else's copyrighted code into your work projects that way. Be very careful about what you let it write for you.

I doubt they do, but I would expect that the copyright-checking (and responsibility thereof) ~burden ought to be on MS, not the dev. That would be what's fair.

There are already lawsuits pending so...

The other thing over time, will be how many projects move to private repos or self-hosted solutions to avoid being scanned? My employer moved all the repos to a self-hosted GitLab solution because of security concerns of proprietary code getting out there.

@spills Moved private GitHub repos over to self-hosted? Just curious

Yep, just didn't trust it, not my call.

Gotcha

If we put code out in a public repo, it's open source and we expect others to use it -- and we hope they obey the license but you can't trust people to obey your OSS license to be honest so the addition of programs scanning your code isn't much of an additional stretch.

Sean has just said what I was thinking. I also think most orgs that have actual code they need to keep secure... already have it secure. The ones that are suddenly going "ooh no, can't have MS stealing our stuff!" are most likely to have fairly ill-informed code bases, given their ill-informed'ness about how Copilot works. So in a way: yeah, good... get it out of everyone's sight! I realise there's been situations of ppl having their own code parroted back to them (more by chatgpt, but the same applies with Copilot's AI), but I wonder how much effort they put in to making that specific code come out to demonstrate their point? But... I just wonder, rather than having any basis for it. From what I can see, Copilot works on way too low a level for anyone's trade secrets to be compromised (and accordingly for any copyright claim to be upheld), and all this is more about ppl wanting to be naesayers or engaging in whataboutery than any reasonable concern. It's a force for good. It's more akin to 2023's version of auto-complete or code introspection than it is to Skynet.

There are genuine license-compliance concerns for Copilot but, yes, almost no OSS code out there represents "trade secrets" so this is mostly about the level of potential legal jeopardy for companies that use Copilot and then ship products derived from the Copilot-generated code. It's purely about licensing and copyright -- and the exact same risk is already present with a company's own developers who might decide to lift a function or two from an OSS project without attribution...

I don't agree with your take about copyright claim, if it is registered, then yeah might be issues for them. If you have your code registered you have attorney fees and statutory damages available.

Copyright isn't the protection most people think it is. That's why we have OSS licenses that have legal standing.

Agreed, but still is claim that can be made and again if it is a registered copyright then you probably don't want to be on defending side.

Most of the code that Copilot suggests is something "an average developer" would create on their own and so, if 100 developers would produce the same code independently, it doesn't really matter if that same code is also in 20 different OSS projects on GH already...

I have a paid sub and it does a decent job creating boilerplate code, in particular for tests.