Curious as to people's thoughts on building with C...
# ai-assisted-coding
g
Curious as to people's thoughts on building with CFML with AI when it's so easy now to build with alternative (and perhaps more AI-friendly?) languages? Has anyone started branching out now that the learning curve is vastly reduced? For me, I'm working on a monolith legacy CF app with very little tests, so rebuliding it from scratch in another language is not reasonable even with AI, but I am working through a modernisation effort screen by screen, building tests as I go, and AI has helped immensely with that. It occurred to me that when I'm in a better place automated-testing wise, it may not be a stretch to start building in Go/Java/Rust and avoid all the little runtime pains that CF can produce. In saying that, using AI I've found that those little mistakes tend to be vastly reduced or even non-existent (especially in the last 6 months or so) - perhaps the compile safety that those other languages offer isn't as important when the AI's CFML code quality is already pretty good?
g
I like the idea of using an existing app you already know - and having AI recreate it in language "X" - as an assistant to learning "X". I am not against learning anything new! My Default stance is however - for a public-facing application - where any dodginess - will effect company reputation / money / etc... I will always choose a language I genuinely know - so that I inherently know to enforce things to the AI - that while it may well work - this "other" option/way is better than what it first produced. Another good tip I use all the time - is asking the agent to read through all rules - Add new things it learned this session, remove rules that are superseded / etc. (Also, related) : I often get it to re-read all rules - as large context windows can see the agent forget things more often. so my tasks normally start with... Follow all rules. Do "X". If I find I write Do "X" , Do "Y", then do "Z" - a lot - I will adda rule that covers the "Y" and "Z". So I have rules like: Use a TDD approach. Create a Green test to guard against regressions. Create a Failing test for the new task - before any code. Complete the task - ensuring the test is now green. Re-run the regression test, ensuring it is green. I also find that if I am doing a LOT of related things - even though the regression test might be green - every now and again, some pre-existing test starts failing. So when I have a long list of tasks; (Think something like update all forms, one at a time to use React vXX) I will add in a rule that is after every nth form - run ALL unit tests. I think my job is likely to become AI harness engineer - rather than software engineer... but as I said - it still isn't at the level I want, all the time : so I personally make sure that I review all code before it is committed.
g
Yes, the job has become much more about reviewing code than writing it for me too. Especially in CFML where the refactoring tools aren't great, it's far easier to write a prompt than try to do it all by hand as well - another example where AI is filling the gaps in the language/tools compared to the more popular languages! In saying that, when I'm reviewing I tend to be looking at things from an architectural mindset and avoiding complexity, which is what I enjoy most anyway - reading things line-by-line is not usually that valuable as long as there is good code coverage in the tests (unless there are security implications or the like). I definitely agree with sticking to languages I understand for important stuff. I guess for me, these thoughts come from spending most of my working life in the Java-space, and missing that a bit (mostly the tools!) - but with the shift to AI, I'm not sure working with Java now will be much different to how I'm working with CF!
p
It depends how big the other app is, I'm supporting some mammoth re-writes now. Very good results. Starting fresh isn't what it used to be and now they have a much simpler architecture ( containers etc ). Cost is also significantly reduced. Key to rebuilding is working through extracting all the domain knowledge thats littered in code and peoples heads.....repository hygien is more important then ever now.
s
AI seems very good at creating regression tests around existing code (in any languageโ€”since it can use the existing source and any existing tests as templates for new tests). So, if you are contemplating a rewrite, at least start down the path of having AI write you a lot of tests for the existing code base. Then it can use those as a "spec" for any rewritten code and, if they are browser-level tests, as tests for the new version. That said, if AI can help maintain your existing code base, it probably matters less that it is in CFML in the first place since you don't really need new developers to be as fluent in CFML, just fluent "in AI" ๐Ÿ™‚ The language matters less.
b
At this point, I'm not certain there is a most AI-friendly language any more. I guess it used to be Python. With higher-end LLMs, they really don't break a sweat. You can litterally build anything you want. Lately, I've been building with RustCFML which has a few advantages. It runs faster than your typical JVM runtimes, compiles to a single executable, but most importanly (at least for me) is understanding the resulting code. In contrast, I've built using Golang, and it works well, but I have to think more when reading it. Then of course, there the thought "do I need to read it at all?". Just have agents do all the testing.
b
I've had great luck using agentic coding for CFML in the last year. I think the models have gotten quite comfortable with it. My rule of thumb is to never use AI for something you don't understand yourself. 15% of the time, it will make a terrible decision, change your design unannounced, remove functionality, or misunderstand the requirement. If you're letting it make a black box that you don't know anything about, you're screwed. Sean is totally correct about having good test coverage. The model doesn't have any institutional knowledge about the use cases of your code (unless you tell it) and works best in a red/green mode where it's very clear what test it needs to make pass IN ADDITION to an existing suite of tests to ensure no regressions.
b
Building a black box is probably not the issue anymore. IMHO. Letting agents to do the work after writing clear specifications seems to work fine. Generally, it is becoming less about code, and more about outcomes. I think what concerns people most is the lack of security.
b
If you're Ok with a black box being built, it's probably because you don't know what's in it ๐Ÿ˜‰ I can see the Java and CFML code copilot writes for me every day and it alternates between genius and pure lunacy. I would NEVER just blindly trust an agent to write something I don't know enough about to know if it's even doing correctly. Your immediate test may pass but that's not a long-term sustainable approach to software development IMO.
I know AI coding is improving at a fantastic rate, but I personally believe in a few years there will be a reckoning of the amount of pure spaghetti messes created by AI that no one wants to maintain or can even figure out. (Imagine one of those pictures of a server room with 10,000 Cat 5 cables all intertwined in a jumbled mess-- it may "work" but it may also be terribly maintained)
The other day I gave copilot a failing test and told it to make it pass and it just changed the test assertion to match the code. I don't know what the heck it was thinking, but if I just blindly trusted the "passing test", I'd be in for a bad surprise.
๐Ÿ‘๐Ÿผ 1
b
Brad, I'm saying you are wrong. I'm suggesting this is where things are going. It's a hard pill to swallow for sure.
b
Well, we'll certainly see in a few years, won't we ๐Ÿ˜‰
s
Given that "writing code is free" (essentially), it may be that if an AI-generated product gets unmaintainable, you could just update the specs and have AI regenerate the code from scratch. I've seen some people advocate that position: they only write specs, and if AI gets stuck trying to modify code, they just tell it to delete the code and start fresh with the updated spec. ๐Ÿค” There's a guy in the Clojure community who has been maintaining a library for years, and for v2 of the library, he wrote a detailed spec of everything, and has had AI generate versions in various languages. For each new release, he updates the spec, and generates a complete, from-scratch new version of the library in whatever language someone wants. Of course, now the problem becomes: can we write good specs? ๐Ÿ™‚
p
@bdw429s Your still on Copilot? Get on a better harnes and see the difference.
๐Ÿ‘ 1
@seancorfield spot on! If AI writes bad code, it's because you provided bad instructions.
s
@paul I've stayed with Copilot to be as "stock" as possible in a very fast-moving environment. VS Code has a full "agents" mode now, and Copilot is configured to plan first, then execute, using as many subagents as it needs/wants, and selecting from the available models as it sees fit. It's gotten pretty sophisticated these days.
p
Try Claude Code and see what you think.
๐Ÿ‘ 2
b
I've read that Copilot is fine for specfic coding tasks, but not not ideal for more robust reasoning and project planning. I'm with Paul here. With Claude Code, you open a terminal, cd to your working directory (even a blank directory), then ask Claude to write something. It does not mess with JS Code as Copilot does. It writes to its heart's content no matter what language, architecture, or patterns you ask it to follow.
b
I've never felt like copilot itself was holding me back. The reasoning, planning, and code are still provided by the models, right? The agent is just there to manage the tools and put it all together. Several people on the Ortus team like Claude Code, but I'm not aware of anything it does that Copilot doesn't. In fact, quite the opposite. I love that copilot can interact with vscode to run gradle tasks, run my junit tests via the vscode plugin so my UI reflects the results, and even interact with the other code plugins (where tools exist). I've used copilot with Github Models, Novita models, and local models I run in LM Studio and it's all quite easy. I can even switch back to previous chats and the context is stored. Perhaps Claude Code does this as well, dunno. But I'm not aware of anything I'm "missing" and I don't think the end models suddenly get smarter or "gooder" at writing code just because they're being called from one agent or another.
p
The model is used by multiple harness your correct in that, but the pre-prompts the harness provides ( transparently ) is where benefit arises and is why Claude Code is so popular. CC can be pointed to other models ( eg Kimi ) and it will perform very well due to that harness.. _"end models suddenly get smarter or "gooder" at writing code just because they're being called from one agent or another_" If what you were saying is true then people would not use a harness and interact directly with the inference endpoint for that model. Look at the Bench results between harneses and this is an indicator as to why CC is the preferred harnes.
b
I never said anything about not using a harness at all! I was saying that I think a given model is just as smart regardless of which harness/agent. You clearly thing CC's system prompts are better and that's fine. I was replying more specifically to the claim above that Copilot wasn't suitable for "robust reasoning and project planning" which doesn't make much sense to me. That task is largely falling on the models with the context/tools/info given to them by the harness and I have had great personal success planning in copilot. But geez, this is turning into the next religious war, lol. I'm not too interested in having a thumb wrestling match over whose harness's system prompts is better. ๐Ÿ™‚
p
Neither am I just making sure you understand how it all fits togther.
And your words literally state you think the outcome is not better depending on the harness and thats simply not true so I don't want others to get the wrong understanding.
b
What I'm saying is that I see more difference switching between models or changing what I ask than fiddling with the system prompts or harnesses. Are system prompts important? yes. Do they make a difference? yes. Should we make them as good as we can? yes. Do I think <insert model here> is suddenly better at reasoning in CC vs Copilot? No, not really-- at least not as a general rule. You do. Good for you. IMO success is more based on (totally made up percentages BTW) โ€ข 50% the model I'm using โ€ข 40% the prompt/specs I input โ€ข 10% the system prompts and tools provides by the harness I acknowledge there are differences and possible benefits to one harness or the other, even if it's subjective. What I'm not convinced is that there is one tool today which is vastly better and that everyone not using my favorite tool is wrong. Maybe soon, one will emerge, but the landscape is changing too rapidly right now.
b
I don't think Paul or I are suggesting the Claude Code is vastly better than Copilot... or any other tool for that matter. It's a different tool, that does different things. It's not perfect, but nothing is. What it has done is change how folks like Paul, Alex Skinner, and I think about the future. More so than any other tool.
๐Ÿ‘ 1
w
@Glen to speak to your original point, if you want a way out of the ruins, your first goal should be creating durable documentation that describes what each component or functional segment does. that's the language-agnostic pivot you need to have, whether you continue with cf modernization or reexpress in a different language. given a typical cf legacy app and all that that implies, you need sufficient descriptors outlining 'what it does'. if you have that, you have a sufficient foundation upon which to build irrespective of language. so focus on creating that collection first is my recommendation. can be just a folder of markdown docs produced from you directing the agent to analyze the application codebase. focus on one functional boundary then move to the next. once you've covered them all, you can have the ai distill a birds-eye view that is built on the boundary descriptions and their purpose
๐Ÿ‘ 1
you're less interested in HOW it does something, than the WHY and WHAT it does, which makes it the lingua franca of switching to something other than cf, should you wish that
s
To build on what Kevin said, one possibility is to use AI to produce a first draft of any documentation that is missing. Ask it to analyze the architecture of your codebase, identify subsystems, figure out data flows. You may have to give it some help with identifying groups of users and business roles, perhaps, but AI can be surprisingly good at filling in the (doc) gaps. At work, we were missing DB schema documentation. We had an initial domain definition doc but had slacked on documenting the concrete schema and how things were related. I installed a MySQL MCP server and let Copilot go to town with its choice of models, and it produced pretty comprehensive docs for our three primary databases (based on just a local dev copy of them). It figured out FK relationships based on conventions (we never use constraints, so it's
tableone.id
and
tabletwo.tableone_id
as a convention and it figured all of that out). It identified related groups of tables and, presumably based on the code which it also had access to, it documented how those groups of tables were used, and even lifted some of the test data out of the DB to use as examples in the new docs. A big help as we had several team changes that took away a lot of institutional knowledge that had not made it into our dev wiki!
๐Ÿ‘ 1
g
@seancorfield - AI does a surprisingly awesome job of matching DB structure to code. When you think about it - this makes perfect sense. (from a dev perspective) all the blog posts - all the apps - are all about storing/editing/retrieving data, processing, manipulating / etc. There is such an extravagant amount of information on the WWW - in a strictly black and white sense, I think we should be concerned if it wasn't GREAT at that task... The surprising part for me, at least is: "Oh yeah - I didn't even think to utilise AI for that task. It is such a great fit for XXX or YYY". And for me its those non-code-creation aspects that I'd like to get it doing more of - especially since it is so much better at being comprehensive/all encompassing, in those tasks, than any human could ever be (especially with competing priorities / time restrictions. Sure there are context limits to observe - but even an average context window can store more "stuff" at the same time - than I can. And un-uniquely, with respect to the space of software engineering - we just ARE, good at problem-solving, at seeing patterns, etc - so I think we are better than the general population, in content-processing tasks - yet AI is just so much better, again...
๐Ÿ‘๐Ÿป 1