Things relating to ethics and computers have changed a wee bit in the last 30 years!

In terms of content redistribution:

The training data for some of the models, as I understand it, clearly contains data that is not legal to redistribute.

My perspective of the history of free and open source development is that it was often an uphill battle against entrenched proprietary software interests who wielded copyright law as both a sword and shield.

That many of the companies circa Y2022-Y2026 with the largest market capitalisations seem to be able to evade those same laws — while individuals and independents are still bound by them — seems deeply unfair.

In terms of adjusting models:

I believe there’s a qualitative difference between being able to change the settings for a model — or to provide additional guidance during queries — compared to being able to fork the software, rebuild it bit-for-bit identically, and modify the source code and behaviour to fit any given use-case.

For important and widespread software, I think those are essential objectives – because otherwise we can’t comprehensively identify and address the biases that will certainly exist in the software. Without applying a scientific and engineering approach to that, we risk being continually led by amplified biases of LLMs that, even if marginal in-situ, are too difficult to identify and resolve.

My sense is that the framing of Open Weights is a defensive measure taken by newly entrenched large tech companies to deflect criticism and to garner the support of people who may understand some of the reasons why Open Source is so important — and yet, Open Weights does not, in my opinion, mean openness; it’s a misnomer. It may allow you to change the behaviour of the software, but fundamentally weights do not allow you to change the infrastructure and foundations of the software, or to disassemble it to its constituent parts.

That is to say: I don’t believe Open Weights is altruism; it’s a deliberate framing to attempt to delay and navigate a well-funded lie through the market.

I think that biases and blind spots already exist in the tech sector – and so I’m disinclined to click on the GeeksForGeeks or Oxide and Friends links; they may have insights that make the content feel edgy, unique or at the leading-edge, but I don’t think that they’ll contain outsider perspectives.

I also hear your argument, I think from a previous post, that opinions are only valid after trying to use LLMs.

I acknowledge that to some extent – but I also think that it’s an argument that softens (potentially important) resistance to LLM usage.

For example: I know that cars can travel fast, but I also know that in many urban centres, a bicycle can get people from A to B just as quickly (or sometimes more quickly) — especially as population density increases. I also know that 80% of the U.S. population resides in urban areas; and while I admit that some people will need to use automobiles, and that transport of large quantities of goods is aided by heavy vehicles, I don’t think that people should need to drive a car to have opinions about them.

In terms of learning and education:

Your idea that LLMs could make thinking time available for other tasks is reasonable. Even so: that’s not an argument for allowing inscrutable, non-scientific LLMs that cannot be replicated to become the norm.

Much of this mirrors the deployment of hardware, BIOS and firmware; a huge amount has been done to achieve reproducibility and determinism in software, but there are lower layers that were never as transparent. Just as we are getting close to being able to campaign effectively for those, an upper layer of LLMs has been added that lacks reproducibility and transparency.

Again I think it’s similar to cars: yes, maybe there could be a net increase in efficiency, in theory, from adoption of LLMs — but in my view, that’s not the direction that things are moving in, in practice.

I think that $500bn invested in clean energy and bicycles for Americans would have a more positive impact both for America and the wider world than $500bn on attempting to get the world to adopt nondeterministic and predominantly cloud-hosted software.

I could have probably engaged in better-faith, but this response attempts to hear some of your points while stating some that I feel are important.

I would argue that actually *most* LLMs are trained quite sustainably. There are very few that are not, like Grok. Most stories about coal-powered datacenters destroying water tables are just that – oft repeated stories of exceptions. There is a **ton** of LLM research happening in university datacenters that, as a rule, are powered by green electricity and often sifon the heat to the heating system of the campus.

And “disgust” of public to AI is a bubble. Ask your hairdresser and plumber and you will find out that they have had a good experience with ChatGPT or Claude or Gemini in the last week and will have another before the week end.

You’re right that training LLMs requires a lot of resources. I’m less convinced at the need for frequent retraining. I could see, for instance, that a model that likes to write Python 2 code might need retraining for Python 3. But the LLMs that I can run locally on my decidedly pre-AI GPU can’t cram enough into my 12GB of VRAM to have it be easily obsoleted, I suspect. There are certainly some things where this would happen. I’m also quite interested in RAG (retrieval-augmented generation) and there’s a lot of work being done to tie in LLMs with non-AI systems (LSPs would be an example relevant here) to let them ingest bite-sized bits of context that inform inference. I’m far from an expert here, so you might be right, or maybe the jury’s still out on this one.

Hi Joey, I’m with you most of the way here. I’m not sure there really is added value of having a human in a loop. As a guy that’s spent several Saturdays trying to fix up Rust and Go dependencies for my packages in Debian, I strongly dislike that work and I don’t see it as myself adding much value. Honestly, when you’ve got a dozen packages to work through, nobody is giving each of them a lot of attention.

“removing the incentive to fix it” is an interesting point. I can see that might happen, and having processes that are easy for someone with an LLM but difficult for someone without one would be a pattern we should seek to avoid.

The argument someone else would make would probably run like this: having those dependencies as separate packages makes copyright auditing easier, both for us and our users. It also reduces code duplication in the distro, and reduces the problem of a security fix from a source upload of dozens of packages to a source upload of one and binary rebuild of dozens.

That said, I’m not sure that’s actually desirable anyhow; it would probably be better to have a package’s source reflect the change made to close a security hole, rather than it being more dependent on what exact package it was built against. (These languages aren’t using dynamic linking like we are used to in C).

In any case, I’m with you in that this whole approach is broken in Debian and needs to be fixed.

Hi James,

I agree in part and disagree in part. “there are few LLMs” -> I don’t think that is true. Just look at the list from Ollama of ones that can be run locally: https://ollama.com/search Most of those entries actually represent a family of related models. “they are not DFSG” -> agreed. This is one of the interesting problems of LLMs, in that often the training corpus which might be considered source is so massive and the right to redistribute it unclear.

“we can’t adjust them” -> disagree. There are a lot of values that can be tweaked; for instance, https://www.geeksforgeeks.org/artificial-intelligence/what-are-llm-parameters/ . But moreover, a model can be a base for additional training, either directly or with techniques like retrieval-augmented generation (RAG).

“want levels of influence” -> for the big US names, sure. Or it might just be greed. Whatever it is, it’s not altruistic. I don’t necessarily know that is the same for the companies that are producing the open weight models. I also don’t think those motivations are necessarily different for traditional hardware companies (many of which are now also profiting from the AI space)

The conversation about learning: this is a most interesting one to me. I have a relative that is a college professor and also a child that is a college student. I’ve seen a lot of hype in this area, but I think it’s going to change how learning happens. Hopefully for the best, but I’m not convinced we’re there yet. Oxide and Friends had a fascinating episode about it here https://oxide-and-friends.transistor.fm/episodes/ai-in-higher-education-with-michael-littman . They discussed a laboratory class at Brown University in which CS students were trying LLMs and learned firsthand about their benefits and drawbacks.

We have long added layers of abstraction to computing. I may lament that many programmers today have never really used C, while someone from a prior generation may lament that people like me have never really written in assembly. (I had a class in it and also have used bits of it here and there, but in the scope of my career, it’s been quite brief.)

I think you are absolutely correct that things like vibe coding don’t lead to the kind of deep knowledge that we need. I’m less convinced that people with a CS background directing an agent will be prevented from learning. I can see the opposite; for instance, if the more mundane tasks of a job can be automated away, better research, architecture, or algorithms could be developed with the newfound free time.

That makes perfect sense to me. It might be a tool the human uses, but ultimately the human is the point of responsibility and communication.

A point, however: the models need to be continuously retrained to stay current, and it seems to me that the environmental concerns cannot be dismissed as, although running can be done with minimised harm (but we should ask whether that's the representative case) there's no such example for training.

jwz (@jwz@mastodon.social)

Papering over that by delegating grunt work to a LLM and relying on human review will a) probably remove most incentive to actually fix it and b) eliminate most of the actual added value having a human in the loop.

Similarly, many developers use multiple utilities and command-line tools to get things done; and each developer has their own preferences.

Because there are few LLMs and typically they are not DFSG, we can’t adjust them, and there’s a risk of them deciding how the project evolves, instead of those distinct individual human minds.

They and Wall Street are willing to invest a significant proportion of the U.S. GDP into becoming successful — and I don’t think that that’s because they’re likely to be successful, I think that it’s because they desperately want the levels of influence they could achieve if they are successful.

I wouldn’t discount the environmental effects so readily, either; sometimes it’s easier to grep, find and cat to figure something out, rather than to use a model that probably required megawatt or gigawatt hours to train. It may take you longer, but you’re more likely to learn. And we will always need people to learn and understand the systems, from the basics of packaging upwards.

I believe myself that an human should never be forced to interact with a LLM (for input or output). In Debian, we should consider that every choice (including code) has been made by an human with which we should be able to discuss.

If that choice was made with the help of a LLM, that’s not a problem (how could we even know). So, basically, it is accountability.

Guix packaging can do several things much more easily than Debian can.

Since they support multiple versions of a package a recipe can inherit from from a different recipe and you only have to customize what changed.

They have something like a super py2dsc that supports many many different packaging repositories.

https://guix.gnu.org/manual/1.5.0/en/html_node/Invoking-guix-import.html

And it mostly works.

They don’t have debian’s requirement for a strict copyright metadata file, so examining the project metadata is often enough to package something.

I also don’t understand what they did for rust packaging, but a they hid a whole bunch of rust crate dependencies from the rest of the packaging system.

It is vastly easier to do rust packaging for guix than debian.

packaging doesn’t necessarily need the full generality of an LLM, a regular program might be able to do much of it as well.

I‘m with you: I‘m not a big fan of AI. I do see the danger coming from untedtricted/unreflected use of AI, but I also do see the benefits when done right.

IMHO AI is a disruptive technology as the computer or the steam engine was in their time. You have to deal with and to learn proper use. If you don‘t: others will do and you will mostly fall behind and detached from development.

Debian should find a way to responsibly be able to use AI. Not banning but also overshoot the use of AI. Local LLMs should be preferred whenever possible snd sensible, alone for data protection reason.

Not sure if it helps when I write that opinion on -devel (not subscribed to -vote)…

Thanks for that. Separating the behavior of horrible corporations from the underlying technologies has been very slow and painful. This is an excellent start. Thanks.

Bill, thanks for dropping by! I was pointed to your blog post at https://www.billcatchings.com/2012/04/frog-from-my-past.html yesterday (and had previously found the CHM transcripts, but sadly not the video). Thanks for your contributions to those early days of Kermit! I happen to have the opportunity to visit CHM in the near future and hope they might be able to show me some of those items also.

Wow, all of this takes me back. I still remember taking the SuperBrain manual to my in-laws house over Christmas in 1980 to try and understand how to program it. Very cool to see people still using Kermit and helping to keep it going.

Bravo for you sir. However thinking of …

"Flon's Law: There is not now, and never will be, a language in which it is
the least bit difficult to write bad programs."

And an OSS emulator in C I recently looked at, I reminded that C can be a poster child for Flon's Law.

I am delighted that you left the campsite cleaner than you found it.

It wasn’t what I’d expected. I’ve worked with legacy C codebases before, and they were generally a massive mess. C-Kermit had already been modernized in the 10.x betas, to an extent. Everything supports ANSI C at least (gated behind ifdefs so the pre-ANSI compilers still work!)

The biggest issues were the race conditions. Kermit uses alarm() to handle timeouts, and, in some cases, siglongjmp() to deal with them. This caused a number of problems: calls that didn’t properly retry on EINTR, messed up context when jumping out of somewhere sensitive (especially when jumping out of OpenSSL frames), etc. Most of these were provoked by running the new test suite under very high load on multiple platforms. For instance, some might happen only on NetBSD or only on macOS, and only with 8 or 64 testing threads or something.

Kermit itself isn’t multithreaded; it predates pthreads by a long while. While we might today use a multithreaded approach, Kermit uses the Old Tools. They still work fine, but have some sharp edges (in fairness, so do threads).

The second class of challenge relates to security. There were the obvious uses of sprintf()/strcpy()/etc. There were some less-obvious things related to outdated design decisions. It was originally a feature to use kermit to log in to a remote, and then you could run kermit on the remote and it could cause your local machine to do stuff (the protocol can carry commands and files, bidirectionally). Most of the machinery to clamp down on that was already there; it was a matter of understanding the problem, thinking through the possible consequences of the problem and of the change, and making the change. Some other things lacked adequate protections, so I added them. scp went down a somewhat analogous path a few years ago, incidentally.

There are some areas of the code where Kermit rolls its own rather than using an existing solution. For instance, it has a HTTP client, FTP client, telnet client, and CLI interface. Some of these one might think of using an off-the-shelf library like libcurl or readline. Perhaps the HTTP and FTP clients will go that way some day (I honestly haven’t looked into this yet). But Kermit’s CLI is unique and rather special in its TOPS-20 sort of feel, and deeply inmeshed with the program, and I think ripping it out and replacing it with readline would be loss of functionality and not worth the substantial effort.

I happen to also be a Debian developer, so yes, you will be the first to get new versions :-)

I operate a mirror at https://kermit.mirrors.quux.org/ by permission. I also uploaded copes to the Internet Archive at https://archive.org/details/kermit-archives-2024-08-27 and https://archive.org/details/kermitmirror-2024-08-26 . I have periodically refreshed my mirror, so maybe it’s about time for another IA upload also.

I just looked carefully at my mail. We share something else. :-) (Beyond the blessing/curse of relatively unique last names that make mailbox searches unambiguous.)

And, yes, I see you back to 2005. /waves! :-)

Wow, small world. Thatnks for that help along the way!

And thank you for GPSBabel! I have probably been using it for all of those 25 years. Initially with some old Garmin GPS devices I barely remember the name of (GPS III and eMap maybe?), and now still to process and convert logs from flights in my little plane. I have a little script that does: gpsbabel -t -i kml -f “$1” -x simplify,count=2000 -o unicsv -F “$1.csv”

I, in fact, packaged up gpsbabel for Debian in 2004 and maintained it there until 2008.

https://github.com/OpenKermit/ckermit/blob/abe250a1db65fd4033adae08901af769d311ddaa/ckcmai.c#L453

Makes the 25 years on my own open source project, GPSBabel, seem childish.

I still have my HP48, but my college-age son has commandeered it for his classes, so I don’t get to use it very often. Though, through an emulator, it is my primary Android calculator still. Glad it helped!