"AI went from being very terrible to seemingly genuinely quite good at a professional level in a very short space of time..." says the Verge's AI reporter.
But AI systems "are still truly, truly terrible at some areas of math... even the days of the week."
If you look at academic math papers, a lot of the time you won't see numbers... So they're still terrible, but they're now also very good at this other part. As to why, at some point you reach a critical mass of what these systems can do. We saw it with writing, we've seen it with programming. They're very good at forging connections between different areas, applying old methods in new ways, those kinds of things. It appears that the newer models they're training have apparently reached that level where it clicks, and now it can do math...
There are areas now where it seems to be producing work that is on par with good mathematicians, alongside other parts where yeah, it can't count. All caught up in that is whether it's going to rewrite employment structures or funding structures... One would hope [math researchers] would branch out into all of these new exciting areas or pose new questions. But AI won't do that, and that's the concern. And then that would leave the field quite sterile, and it will have all of these things that have been done, and maybe nothing left to pursue... [A] lot are scared that it's closing off the field. So by definition, those breakthroughs that lead to something surprising and new that you can say, "Oh, this works here," may not be happening anymore...
There were some bleak responses from graduate students I saw in essays posted online. Where's their place in this as future researchers? Do they have a place in this? Is it as glorified AI proof checkers? That will be quite an unsatisfying career, I imagine. Or maybe not, I don't know. We will see. I think anything used properly will be a net boon.
They also had an interesting response when asked if AI democratizes access to high-level mathematical proofs:
A lot of the mathematicians I spoke to were almost quite weary of this, actually. They love the idea, in theory, of democratizing access. They're also quite fed up with AI-generated or -assisted papers that are flooding every publication imaginable, as well as the pre-print servers that they use in these fields. Some of those I spoke to said things like, "Oh, I got three emails this week alone with people being like, 'Hey, is this legit?'" Because they thought they'd solved something with ChatGPT or with Claude, and they also don't have the mathematical skills to check whether they've actually solved something.
On the flip side, there are parts where they said, "Well, we've got a talented undergrad who's done something that a talented undergrad would probably have never managed, and here they are doing grad-level work and they've produced a paper that is legit." And in the bigger scheme of things, a few I spoke to said, "Well yeah, a lot of these are in the ivory tower. Having access to this kind of thing globally could really boost access to the kind of things here." On the flip side, the cost. These things cost a lot to run.
But AI systems "are still truly, truly terrible at some areas of math... even the days of the week."
If you look at academic math papers, a lot of the time you won't see numbers... So they're still terrible, but they're now also very good at this other part. As to why, at some point you reach a critical mass of what these systems can do. We saw it with writing, we've seen it with programming. They're very good at forging connections between different areas, applying old methods in new ways, those kinds of things. It appears that the newer models they're training have apparently reached that level where it clicks, and now it can do math...
There are areas now where it seems to be producing work that is on par with good mathematicians, alongside other parts where yeah, it can't count. All caught up in that is whether it's going to rewrite employment structures or funding structures... One would hope [math researchers] would branch out into all of these new exciting areas or pose new questions. But AI won't do that, and that's the concern. And then that would leave the field quite sterile, and it will have all of these things that have been done, and maybe nothing left to pursue... [A] lot are scared that it's closing off the field. So by definition, those breakthroughs that lead to something surprising and new that you can say, "Oh, this works here," may not be happening anymore...
There were some bleak responses from graduate students I saw in essays posted online. Where's their place in this as future researchers? Do they have a place in this? Is it as glorified AI proof checkers? That will be quite an unsatisfying career, I imagine. Or maybe not, I don't know. We will see. I think anything used properly will be a net boon.
They also had an interesting response when asked if AI democratizes access to high-level mathematical proofs:
A lot of the mathematicians I spoke to were almost quite weary of this, actually. They love the idea, in theory, of democratizing access. They're also quite fed up with AI-generated or -assisted papers that are flooding every publication imaginable, as well as the pre-print servers that they use in these fields. Some of those I spoke to said things like, "Oh, I got three emails this week alone with people being like, 'Hey, is this legit?'" Because they thought they'd solved something with ChatGPT or with Claude, and they also don't have the mathematical skills to check whether they've actually solved something.
On the flip side, there are parts where they said, "Well, we've got a talented undergrad who's done something that a talented undergrad would probably have never managed, and here they are doing grad-level work and they've produced a paper that is legit." And in the bigger scheme of things, a few I spoke to said, "Well yeah, a lot of these are in the ivory tower. Having access to this kind of thing globally could really boost access to the kind of things here." On the flip side, the cost. These things cost a lot to run.
Read more of this story at Slashdot.
:format(jpeg):fill(f8f8f8,true)/s3/static.nrc.nl/taxonomy/6c7bb5e-Vorto_itemafbeelding.png)
:format(jpeg):fill(f8f8f8,true)/s3/static.nrc.nl/taxonomy/cc7ceaa-PreciesVier_itemafbeelding.png)
:format(jpeg):fill(f8f8f8,true)/s3/static.nrc.nl/taxonomy/e3dbc3a-Cinco_itemafbeelding.png)
:format(jpeg):fill(f8f8f8,true)/s3/static.nrc.nl/taxonomy/472f2b9-Woordzoeker_itemafbeelding.png)
:format(jpeg):fill(f8f8f8,true)/s3/static.nrc.nl/taxonomy/6cf24d7-Cijferblok_itemafbeelding.png)
:format(jpeg):fill(f8f8f8,true)/s3/static.nrc.nl/taxonomy/81f2d02-Koprol_itemafbeelding.png)
:format(jpeg):fill(f8f8f8,true)/s3/static.nrc.nl/taxonomy/c3f9335-Sudoku_itemafbeelding.png)
:format(jpeg):fill(f8f8f8,true)/s3/static.nrc.nl/taxonomy/06b57cb-AanZet_itemafbeelding.png)
