I'm very pro AI in general. I love using agentic engineering tools and use AI heavily to research many topics.
But I don't like having AI do any of my communication oriented writing. Unless it's technical documentation about something the AI wrote, but even then I usually am properly quoting the AI in my own writing. Not parading it's ideas as my own.
I feel like it defeats the purpose of me trying to communicate my ideas to people. My ideas then get tainted by the AI's knowledge when I use it to produce text for me. Also, I'm a very bad writer and want to improve on that front, so me writing more can help me improve.
You can prove more stuff with classical logic while intuitionistic logic restricts you.
For example given a real number x constructed in intuitionistic logic. You can't determine if x > 0 or x = 0 or x < 0. While you can in classical logic.
Also, more generally you can't prove existence statements in general without construction in intuitionistic logic.
So, there exists an x such that P(x) can be proven without actually finding x classically, but in intuitionistic logic I must provide a procedure for constructing x.
All this said, even though you can prove less statements in intuitionistic logic I find it's restrictions satisfying because it forces us to prove things by showing they exist via construction. Which to me is more satisfying than just showing that a construction exists.
Five stages of accepting constructive mathematics
by Andrej Bauer
"On the odd day, a mathematician might wonder what constructive mathematics is all about. They may have heard arguments in favor of constructivism but are not at all convinced by them, and in any case they may care little about philosophy. A typical introductory text about constructivism spends a great deal of time explaining the principles and contains only trivial mathematics, while advanced constructive texts are impenetrable, like all unfamiliar mathematics. How then can a mathematician find out what constructive mathematics feels like? What new and relevant ideas does constructive mathematics have to offer, if any? I shall attempt to answer these questions"
It looked to me like most of the raw pointers in the blog were const. Sometimes you don't want the baggage of smart pointers and getting a cheap easily copyable view of your data is nice, so you want to return a const T. Usually if an API returns a const T I assume lifetimes are handled for me and that the ptr is valid as long as it is not nullptr.
I think one interesting thing to point out is that the proof (disproof) was done by finding a counterexample of Erdős' original conjecture.
I agree with one of the mathematician's responses in the linked PDF that this is somewhat less interesting than proving the actual conjecture was true.
In my eyes proving the conjecture true requires a bit more theory crafting. You have to explain why the conjecture is correct by grounding it in a larger theory while with the counterexample the model has to just perform a more advanced form of search to find the correct construction.
Obviously this search is impressive not naive and requires many steps along the way to prove connections to the counterexample, but instead of developing new deep mathematics the model is still just connecting existing ideas.
Not to discount this monumental achievement. I think we're really getting somewhere! To me, and this is just vibes based, I think the models aren't far from being able to theory craft in such a way that they could prove more complicated conjectures that require developing new mathematics. I think that's just a matter of having them able to work on longer and longer time horizons.
Searching for a proof and disproof are sometimes not so different. In most cases, you nibble the borders to simplify the problem.
For example, to prove something is impossible let's say you first prove that there are only 5 families, and 4 of them are impossible. So now 80% of the problem is solved! :) If you are looking for counterexamples, the search is reduced 80% too. In both cases it may be useful
In counterexamples you can make guess and leaps and if it works it's fine. This is not possible for a proof.
On the other hand, once you have found a counterexample it's usual to hide the dead ends you discarded.
I agree there can be some theory crafting in the search for a counterexample, but in general I think it is easier to search for.
For proving a proposition P I have to show for all x P(x), but for contradiction I only have to show that there exists an x such that not P(x).
While I agree there could be a lot of theory crafting to reduce the search space of possible x's to find not P(x), but with for all x P(x) you have to be able to produce a larger framework that explains why no counter example exists.
Timothy Gowers said a proof (rather than disproof) would have been different and more impressive because it would have required new mathematical concepts.
> I think that's just a matter of having them able to work on longer and longer time horizons.
No this will never do the kind of math that humans did when coming up with complex numbers, or hell just regular numbers ex nihilo. No matter how long it's given to combine things in its training data.
I currently operate under the assumption that humans are at most as powerful as Turing Machines. And from what I understand these models internally are modeling increasingly harder and larger DFAs, so they're at least as powerful as regular languages.
Assuming humans are more powerful than regular languages I could maybe agree that these methods may not eventually yield entirely human like intelligence, but just better and better approximations.
The vibe I get though is that we aren't more powerful than regular languages, cause human beings feel computationally bounded. So I could see given enough "human signal" these things could learn to imitate us precisely.
Well yeah there is likely an equivalence between computability and epistemology, but I'm not sure it matters when comparing LLM intelligence to human intelligence. There is clearly a missing link that prevents the LLM from reaching beyond its training data the way humans do.
If you look at the life efforts and accomplishments of the ~100 billion humans who have ever lived, how many lifetimes would you discount as having "non-human intelligence" based on the lack of "novel" contributions to frontier of our species' scientific understanding according to the same high bar you apply to LLMs?
You're not quite addressing the question. More and more of the training data is now synthetic.
To be very specific - what novel things did the majority of the ~8 bil humans on Earth do say, yesterday, that you wouldn't otherwise dismiss as non-intelligent rehashing of the same tired patterns they always inhabit were those same actions attributed to LLMs?
What I'm getting at is that I think you're falling into the trap of thinking of the rare geniuses of human history, and furthermore their rare moments of accomplishment (relative to the long span of their lifetimes filled mostly without these accomplishments) when you think of "human intelligence", which is of course far overstating what actual human intelligence is.
Synthetic training data is carefully crafted by humans. The rare geniuses of human history use a different magnitude and configuration of the same kind of human intelligence that posted a dad joke on a site that got scraped into the training set and repeated, convincing people that it is intelligent like humans.
> that you wouldn't otherwise dismiss as non-intelligent rehashing of the same tired patterns they always inhabit were those same actions attributed to LLMs?
Regardless of whether something's been done before people still come up with them on their own without directly copying or amalgamating several copies. Pretty much every skilled profession includes figuring things out on the fly through the use of general reasoning that doesn't involve pattern matching against millions of examples.
> Synthetic training data is carefully crafted by humans.
Much, if not the majority of synthetic data is AI generated. Human experts then evaluate samples of the data, but nothing like the entire corpus which can be trillions of tokens of generated material.
> The rare geniuses of human history use a different magnitude and configuration of the same kind of human intelligence
I agree. What I don’t see any strong evidence for is that this intelligence is unique to humans. Nor do I see how it could ever be anything other than recombinations of existing data with random mutation. Where else would the building blocks for each invention come from, divine insight? We build on the shoulders of giants etc etc
Worth noting, as a sidebar, that we’re having this discussion on a post mentioning a novel breakthrough made by AI over a topic that many brilliant human mathematicians including Erdos himself failed to do.
> Regardless of whether something's been done before people still come up with them on their own without directly copying or amalgamating several copies.
I’m not even saying it in the “there’s nothing new under the sun” sense.
If you follow an average person’s day from beginning to end. Let’s say in Bangkok or NYC or Paris, at which part of the day are they not simply repeating a variation of something they’ve done many times before, or seen others around them do before, or read about others doing before, or heard about others doing before, watched others do before on TV etc etc
What you have left, how is it distinguishable, without reasoning backwards from the desired conclusion of human exceptionalism, from turning up the temperature on an LLM query?
How many data points does a human parse when they attempt to stand up as a toddler? Sight, sound, sensation from every limb and body part, inner ear, internal thought processes at the time conscious and unconscious related to the moment and attempting to interpret it in relation to all that it’s experienced to this point, including all prior attempts and whatever retained associated data, a hard to even comprehend stream of data, coming in continuously over however many minutes, hours, etc of attempts.
The stream of data the brain is processing from both external and internal sources from birth is incredibly rich, and if we attempted to represent the full depth of it it would far outweigh the size of any corpus models are being trained on now.
I think what may be genuinely missing from AI is the type of data that doesn’t translate completely into text. The audio and images/video we feed in are a totally incomplete slice of the POV of say even a single average human through their lifetime, and bereft of all the associated data a human has access to in the moment (sensory etc).
I think this tends more towards the world models that Yann Lecun et al are promoting as the key to more capable AI.
You seem to be missing their point (which I agree with). The type of intelligence we are equipped with allows us not to have the level of memory an LLM does and still complete tasks that are novel to us every single day. Like navigating a shopping cart through tricky coridors in a store, coming up with a dad joke as in sibling example, combining a set of tools to achieve something we have never seen before, etc.
LLMs approximate a lot of that very well by simply having seen it before.
Also watch kids develop language: they learn patterns with much less training data than LLMs.
I addressed much of this in my response to a sibling comment, but a few more here:
> novel to us every single day. Like navigating a shopping cart through tricky coridors in a store
We have been practicing navigating the physical world for something like 16hrs/day every day from the moment of our birth. All the sensory data passing through our brains during that time is far larger than any dataset an LLM is trained on.
Humans navigating a shopping cart at a store have likely navigated the physical world before, pushed a shopping cart before, and in combination have navigated stores while pushing shopping carts before. Nevertheless, many still bump into objects all along the way.
Them succeeding at successive variations of store layouts is not novel unless we expand the definition of novel to mean any recombination whatsoever of pre existing concepts.
I’m certain that with all the intense usage of AI by hundreds of millions of people, there have been countless collections of words passed to LLMs so far that have never before been uttered in exactly such a sequence, let alone in the dataset.
I’m equally certain the LLMs have responded to those words with collections of its own that have also never been uttered in that exact sequence, responding to their unique context.
It is trivial to produce an example of this now yourself if you’d like.
The LLM we’re talking about, mentioned in the OP, has never seen this solution to this problem in its dataset. A large number of brilliant mathematicians were not able to discover this solution. They are themselves expressing that this is a novel breakthrough and had this come from a human it would be treated as such.
If the response to that is “well it’s just recombining concepts it already knows until it finds a solution that works” I would ask how that differs from what humans do?
You missed the core of my point: humans operate, including in the real world, on much less training data. Give a human a shopping cart and ask them to push it backwards, and they'll figure it out in a few minutes even if they've never done it before.
This is the bit that's missing that LLMs do approximate amazingly well through sheer training set size, but in my opinion, it puts a cap on what novel things they can achieve in comparison with humans.
To me, I've thought about a related "invention space" before: with us creating software to solve many problems people are facing, why are there not any perfect solutions for any problem (running a cafe? a CNC machine? ...), and we always need more software built to cover one small (novel?) change for a particular owner?
The world space is just so large that you need whatever this intelligence is humans (and animals) have to navigate it successfully — but LLMs do not intrinsically.
Whether they can be so large that it does not matter in 99.99% of cases is to be seen.
> You missed the core of my point: humans operate, including in the real world, on much less training data.
I very specifically addressed this in my response to you. How much training data is contained in 16 waking hours of navigating the world fusing all sensory data, never mind data being simultaneously generated within the mind while this is all going on, from birth til death? From birth til pushing that shopping cart?
Far, far more than in all the training datasets being used for AI.
I also addressed this again in my reply to the sibling comment.
People tend to discount how much data humans have passing through their minds 24/7.
A human isn’t born in a vacuum as a fully formed adult and dropped into the shopping cart navigation problem.
A human has had far, far more training data fed into it that contains all the pieces necessary to translate to pushing a shopping cart when first seeing it, than a machine learning model which has been fed 1 million videos of a robot pushing a shopping cart.
Yeah I'm not sure what the exact context of the statement is.
I am absolutely certain that we have not already discovered let alone implemented the best possible learning algorithms. Humans have had more time to evolve, there's a great chance that we do learn more efficiently, and have developed specialized brains that are primed to learning things like how to navigate the physical world on planet Earth as bipeds.
That said, to say that we operate with less training data is just ignoring the reality of all the data we're training on at all times.
If we were to model in lossless fidelity what humans are capable of seeing, hearing, smelling, tasting, feeling, thinking consciously and subconsciously etc. essentially all the data flowing through our minds that we are constantly training on every moment of every day, even while we sleep/are unconscious, what sort of bitrate do you think would be required?
Modern LLMs train on datasets in the what, tens of terabytes in size? Let's call it 100 TB.
I would imagine that to losslessly reproduce the full suite of human sensory data (whatever that means for things like taste, touch, smell) would require a bitrate that hits that 100 TB total relatively quickly?
Let's stick to comparing language skills to language skills: at least in my experience with my two kids, they learn word formation patterns before they turn 2 — easy to notice because you see them make mistakes on exceptions.
LLMs needed how much training data to be able to do so?
FWIW, I still see them make up wrong words not following any grammatical pattern, esp in Serbian with less training data.
"...we're optimized for having not many experiences. You only live for about a billion seconds—that's assuming you don't learn anything after you're 30, which is pretty much true. So you live for about a billion seconds and you've got a 100 trillion connections. So [you've] got crazily more parameters than you have experiences. So our brains [are] optimized for making the best use of not very many experiences."
I think this is disingenuous comparison. When we read a book we can estimate the amount of data we're taking in based on the character count (each character being represented by some fixed amount of bits).
What you're suggesting on the other hand is something akin to counting the number of pixels on each page we look at. That's absurd overestimate of the amount of data a person reading is actually taking in.
I believe there is a point: we simulataneously ingest words, but also glyph shapes and learn acceptable variations between them (eg. serif vs non-serif, large x-height vs small, curlier or more elegant, playful letters...) — all of these contribute to our multi-faceted learning, but ultimately, we do seem to need less of the data to learn (how long it takes for us to learn to recognize letters vs OCR based on ML).
The act of discovery is usually associated with "abductive reasoning", i.e. finding a novel pattern in data.
Usually people point out that humans are more sample efficient: they might notice a novel pattern in a handful of samples, whereas training NN might require take millions.
However a claim that LLMs fundamentally cannot do abductive reasoning at all is not warranted - we don't see a clear cut, it just looks like the way LLMs do it is less efficient.
What did you say that added to the discussion? I wasn't wondering at all. More compute time won't create new mathematics. To believe otherwise is to misunderstand the technology and there is no amount of hackernews votes that will change that.
Unless someone eventually finds the consciousness center in the brain I will continue to hold the position that it is just another property of "things". I know consciousness must be real because it's the first thing I have access to without any sort of reasoning attached on top of it. Its realness is more visceral than atoms or any other physical theory because it is the way in which the world is conveyed to me, but I don't think I'm unique in any way for having it.
I feel like all systems, in a panpsychist sense, participate in consciousness, so in some way it's a property of matter or systems in our universe that we have somehow failed to account for in physics. We miss it because systems only exhibit consciousness internally like on top of having all the physical properties of rocks, rocks also have an internal state of being. That internal state of being for the most part is uninteresting cause it doesn't dictate the rocks actual form or function in the universe.
I'd argue human consciousness is the same. My conscious experience has nothing to do with the thoughts that are actually being produced. By this I mean there is no authorship of the thoughts and actions I perform by my consciousness. To me it seems more like a stage in which elements of my experience appear for brief moments before fading away, so much like the rock's internal experience my internal experience does not have any affect on the physical world.
Part of me then starts to worry why worry about consciousness at all if it's something that doesn't participate in the physical world because then what's the point of it all? Also, if all systems get to participate, then what stops things like basic logic gates on a PC from having consciousness as well. I tend to lean towards thinking that those feelings are similar to the same kinds of feelings humans used to have about thinking they were the center of the universe, but I'm not sure.
But I don't like having AI do any of my communication oriented writing. Unless it's technical documentation about something the AI wrote, but even then I usually am properly quoting the AI in my own writing. Not parading it's ideas as my own.
I feel like it defeats the purpose of me trying to communicate my ideas to people. My ideas then get tainted by the AI's knowledge when I use it to produce text for me. Also, I'm a very bad writer and want to improve on that front, so me writing more can help me improve.