AI is getting a much more widespread use than people with a technical background. So its application, namely in education but in all other non-CS disciplines will be through people with limited understanding of the biases. It is importing them to make them explicit, to underline that an LLM will produce the same biases it deduced from testing data and its loss function. But lots functions and test data are not public knowledge, studies need to be performed to understand how the coders’ own biases influenced the LLM scheme itself.

A photo has less bias because we know what it is representing: a photo only shows what can be seen. But the same understanding is not clear AI. Why showing a photo-realistic tree versus a biological diagram? Choices have been made, of which a broader audience needs to be aware of.

When the Generative Agents system was evaluated for how “believably human” the agents acted, researchers found the AI versions scored higher than actual human actors.

That's a neat finding. I feel like there's a lot to unpack there around how our expectations are formed.

So like... You ask the model about styles and it says 'diagrammatic' and you ask for an artistic but diagrammatic tree or whatever and that affects your worldview?

If people just ask for a tree and the issue is they didn't get what they expected, I don't care. They can learn to articulate their ideas and maybe, just maybe, appreciate that others exist who might describe their ideas differently.

But if the problem is the way your brain subtly restructures ideas to better fit queries then I'd agree it's going to have 'downstream' effects.

I really like that it talks about the ontological systems that are completely and utterly disregarded by the models. But then the article whiffed and forgot all about how those systems could inform models only to talk about how it constrains them. The reality is the models do NOT consider any ontological basis beyond what is encoded in the language used to train them. What needs to be done is to allow the LLMs to somehow tap into ontological models as part of the process for generating responses. Then you could plug in different ontologies to make specialized systems.

Wow, AI researchers are not only adopting philosophy jargon, but they're starting to cover some familiar territory. That is the difference between signifier (language) and signified (reality).

The problem is that spoken language is vague, colloquial, and subjective. Therefore spoken language can never produce something specific, universal, or objective.

Thus, a user receives an answer that has already undergone a filtering of sorts.

Wouldn't this be an expected trait of a system predicting next most likely token based on lossy compression of specific datasets and other lossy optimization?

In theory something similar could be done with enough training. Guess what that would cost. Does enough clean water and energy exist to train it? Probably best not to find out, but techbros will try.

I deep dived into AI research when the bubble first started with chatgpt 3.5. It turns out, most AI researchers are philosophers. Because thus far, there was very little tech wise elements to discuss. Neural networks and machine learning were very basic and a lot of proposals were theoretical. Generative AI as LLMs and image generators were philosophical proposals before real technological prototypes were built. A lot of it comes from epistemology analysis mixed in with neuroscience and devops. It's a relatively new trend that the wallstreet techbros have inserted themselves and dominated the space.

Or how we operationalize and interpret information from studies. You might think you're measuring something according to a narrow definition and operationalization of the measurement. But that doesn't guarantee that that's what you are actually getting. It's more an epistemological and philosophical issue. What is "believable human"? And how do you measure it? It's a rabbit hole in and of itself.

A photo has less bias because we know what it is representing: a photo only shows what can be seen.

i agree with you on ai but the above statement is ignoring what photography is and biases intrinsic to it.

You see, that understanding you expect to be developed for ai is not there for you for photography.

Depends. For an expert, that is self evident (even if it might not be clear which biases have been incorporated). But that is not how it has been marketed. Chatgpt and similar are perceived as answering “the truth” at all times, and that skews the user’s understanding of the answers. Researching how deeply the answers are affected by the coders’ bias is the focus of their research and a worthwhile undertaking to avoid overlooking something important

If you want, any work that does not encompass the whole world is applying a filter and therefore a bias of some sort. We don’t expect a photo to X-ray the roots of a tree, because we understand the physical constraints of photography. Sure, something could be just out of frame, something else could have been photoshopped out, you can create a different story by selecting different photos and so on. But we understand the “what” a photo represents. I doubt we have the dang understanding of “what” an LLM represents, what are the constraints of the possible answers, and we definitely don’t understand why a specific answer is chosen over the infinite other possibilities.

I don't think a logical system like an ontology is really capable of being represented in neural networks with any real fidelity.

Well it does great with completely illogical systems. I wonder if one can be used for a random seed?

For an expert, that is self evident

I am far from an expert, but it seemed obvious to ne.

I teach, nothing is evident to anyone