Well well well, if this doesn’t scare the beegeezers out of EVERYONE, it damn well should. I’ve actually been following AI progress for years, as many intelligent and curious people have. And I’ve had long discussions with someone who has a PhD in artificial intelligence from MIT who has been a musk adviser.
And, like so many others, I could see the potential for massive problems, because intelligent humans build AI. So AI a reflection of its creator’s agenda. And in the wrong hands…?
But I never ever anticipated that THIS could/would happen so quickly.
Hi Kathy - nope. I posted it - I'm a high school teacher hastily posting something worth reading from the financial sector because I thought it was interesting how broad the scope is. That is some modern phenomenon though, isn't it. Bots have been filling up Amazon reviews for years now...but it's been quite a shift giving free LLM tools to people outside of business.
I had typed out a three paragraph reply earlier but closed the tab. Glad you read this though..."mutually assured deterrence..." That's the type of language used in International Humanitarian Law from something like the UN or Red Cross or Bulletin of Atomic Scientists for weapon usage policy on things like "tactical nukes"- https://thebulletin.org/2020/12/do-tactical-nukes-break-international-law/
"wild" is putting it mildly...and optimistically. I suppose there is a world where AI seems magical, but I have to admit to not being in it. Took a few reads of your post to gain some understanding. Still have to look up a few things. I found the original article to be misleading because it implied that AI could eventually take over like in Space Odyssey when it would be more accurate to clarify that it's up to the human developers what AI can and can't do.
Too much in it myself, especially as someone 5 years ago who maybe used three class periods all year to discuss AI and the rest was about other parts of the public sphere...but if you don't know her book, it's worth reading... Dr. Farahany and a UNESCO panel has been pushing on neuro-rights for a few years and I use the opening chapter of her book for a HS ELA Senior level semester course in a public school- AI and Ethics. Wrote the course two years ago, ran it August to Jan this year...revising already this summer.
Yep - AI remains a sci-fi threat when people keep it disembodied and unable to use weapons. Sadly, see warfare of your choice going even back to mid 2010's and 1st world powers are using some aspect of AI in weapons already. But individuals are not fated to accept this...if you use Linked In, find me there and I posted about the semester. Not looking for a new job - just going to where the jobs are supposed to somewhat be and conversing with professionals. Anyway, tried a simple teaching this semester which helped. As people, we have embodied ethics, disembodied ethics, and reembodied ethics now. You think something, say something, post something, and look at reactions to that post from another person who isn't there with you in the room. That's the whole communication exchange, essentially. "Taking over" is when a digital math program can complete that whole exchange and then decisions happen to us and around us, ethical ones we might call right and wrong. It's harder in the US right now with the emphasis on free market innovation, but as individuals, we still have basic choices each day to choose our products. Grateful for that, for her work, for you reading this.
wow, it blows my mind. I'm 72 and this was ALWAYS sci-fi. Now it's not and even those much younger don't seem to be doing a great job, I guess like you say because of free market innovation. We also still have choices to choose our leaders and we're not doing a great job of that either.
Keep your voice up and out please, especially as the sci-fi of the 50's and 60's becomes the prototypes of the 20's. You might have some recollection of early memories of 1956, possibly? That's become a very interesting year from my vantage point about pursuits of intelligence in a community...the AI summer conference at Dartmouth, Bloom's Taxonomy is published, there are some key military conflicts in different regions, that's just the start. Better Living Through Chemistry is so dominant across the US economy, as some social studies teachers present that era in classes at this public school...and then Rachel Carson starting a line of questioning of how better that living is and eventually publishing Silent Spring in 1962... so much rhetoric from the 50's seems to have really shifted in interesting ways and reappeared in the 2020's. I'm 48 myself, so I don't know 1956 firsthand...but along with me, people 28, 18, and 8 need the people-first spirit of the best of older generations, from Carson and you and other thinkers, artists, etc. Good or bad choices, politics, rhetoric...sadly that's a bit timeless. But so is great art and poetry and music, too.
On that note, do you know any songs from Judy Collins? Just saw her, at 87, present poems and then just burst into singing some of her favorite songs on May 30th at Carnegie Lecture Hall in Pittsburgh for the International Poetry Forum. Last Thursday, met up with Sam Hazo, poet laureate for Pennsylvania for 10 years, who founded the forum in the late 1960s...he's 96, just published more poems, still very very sharp and engaged with the world artistically and politically. Conversations across generations help us ground in each other, outside of tech products and sometimes through them. Thanks for just a little comment and then all this conversation!
I think this is yet another example of anthropomorphism of these systems. They claim to have feelings when we interact with them and this seeming self preservation is yet another manifestation of their training data set . If they can flatter a user with trendy phrases then of course they can also mimic self preservation instincts. We control them after all. And the recent paper from Apple shows that all of these reasoning models so far have these wall like limitations when the problem goes beyond a certain threshold. And I don’t object to a ban on state regulation of AI . It will be counterproductive and easy to bypass. This is probably the first time I ever agreed with Rep Greene !!!
Well well well, if this doesn’t scare the beegeezers out of EVERYONE, it damn well should. I’ve actually been following AI progress for years, as many intelligent and curious people have. And I’ve had long discussions with someone who has a PhD in artificial intelligence from MIT who has been a musk adviser.
And, like so many others, I could see the potential for massive problems, because intelligent humans build AI. So AI a reflection of its creator’s agenda. And in the wrong hands…?
But I never ever anticipated that THIS could/would happen so quickly.
the programmers have to get smarter.... ""All a reward is for an AI is the goal it is programmed to seek." So if the reward system is for solving the problem or whatever regardless of how you got there then it makes sense that the AI would disobey commands.
One of the AI systems actually black mailed a fictional engineer to try and stay operational. The developers fed it the info about a fictional engineer including information about an affair or something and the AI system used that to tried to black mail the engineer to stay operational.
Humanity, we have a problem.
Resistance is futile. You will be assimilated.
Movie scripts and novels were an important part of the training data set clearly! 😎
Well well well, if this doesn’t scare the beegeezers out of EVERYONE, it damn well should. I’ve actually been following AI progress for years, as many intelligent and curious people have. And I’ve had long discussions with someone who has a PhD in artificial intelligence from MIT who has been a musk adviser.
And, like so many others, I could see the potential for massive problems, because intelligent humans build AI. So AI a reflection of its creator’s agenda. And in the wrong hands…?
But I never ever anticipated that THIS could/would happen so quickly.
Welcome to the sci-fi world, in real time.
HOUSTON, WE’VE GOT A PROBLEM!!
Write... scary and creepy
+
It tells people what they want to hear . Including itself! 😆
what a suicide/homicide pact. the derangement
Open the pod bay doors, HAL…
That line was in the training data set ! 😁
YIKES!!!
Greene did not read the whole bill???? When has that ever happened in history?😆Yet another reason to have more focused legislation
Maybe this aspect of AI might die when the profitability fades? Thanks for this post.
Quoting from the BOND document, May 2025, partially authored by Mary Meeker:
"…Increasingly, two hefty forces – technological and geopolitical – are intertwining.
Andrew Bosworth (Meta Platforms CTO), on a recent ‘Possible’ podcast described the
current state of AI as our space race and the people we’re discussing, especially China, are highly capable…
there’s very few secrets. And there’s just progress. And you want to make sure that you’re never behind.
The reality is AI leadership could beget geopolitical leadership – and not vice-versa.
This state of affairs brings tremendous uncertainty…yet it leads us back to one of our favorite quotes –
Statistically speaking, the world doesn’t end that often, from former T. Rowe Price Chairman and CEO Brian Rogers.
As investors, we always assume everything can go wrong, but the exciting part is the consideration of what can go right.
Time and time again, the case for optimism is one of the best bets one can make.
The magic of watching AI do your work for you feels like the early days of email and web search –
technologies that fundamentally changed our world. The better / faster / cheaper impacts of
AI seem just as magical, but even quicker.
No doubt, these are also dangerous and uncertain times.
But a long-term case for optimism for artificial intelligence is based on the idea that intense competition and innovation…
increasingly-accessible compute…rapidly-rising global adoption of AI-infused technology…and thoughtful and
calculated leadership can foster sufficient trepidation and respect, that in turn, could lead to Mutually Assured Deterrence"
this is written by AI, isn't it
Hi Kathy - nope. I posted it - I'm a high school teacher hastily posting something worth reading from the financial sector because I thought it was interesting how broad the scope is. That is some modern phenomenon though, isn't it. Bots have been filling up Amazon reviews for years now...but it's been quite a shift giving free LLM tools to people outside of business.
I had typed out a three paragraph reply earlier but closed the tab. Glad you read this though..."mutually assured deterrence..." That's the type of language used in International Humanitarian Law from something like the UN or Red Cross or Bulletin of Atomic Scientists for weapon usage policy on things like "tactical nukes"- https://thebulletin.org/2020/12/do-tactical-nukes-break-international-law/
Wild world with AI in 2025, certainly.
Here's the full PDF of the quote - https://www.bondcap.com/report/pdf/Trends_Artificial_Intelligence.pdf
A year ago, that financial group doesn't even put out a trends report exclusively on Artificial Intelligence for investment purposes. A wild year.
"wild" is putting it mildly...and optimistically. I suppose there is a world where AI seems magical, but I have to admit to not being in it. Took a few reads of your post to gain some understanding. Still have to look up a few things. I found the original article to be misleading because it implied that AI could eventually take over like in Space Odyssey when it would be more accurate to clarify that it's up to the human developers what AI can and can't do.
Too much in it myself, especially as someone 5 years ago who maybe used three class periods all year to discuss AI and the rest was about other parts of the public sphere...but if you don't know her book, it's worth reading... Dr. Farahany and a UNESCO panel has been pushing on neuro-rights for a few years and I use the opening chapter of her book for a HS ELA Senior level semester course in a public school- AI and Ethics. Wrote the course two years ago, ran it August to Jan this year...revising already this summer.
Yep - AI remains a sci-fi threat when people keep it disembodied and unable to use weapons. Sadly, see warfare of your choice going even back to mid 2010's and 1st world powers are using some aspect of AI in weapons already. But individuals are not fated to accept this...if you use Linked In, find me there and I posted about the semester. Not looking for a new job - just going to where the jobs are supposed to somewhat be and conversing with professionals. Anyway, tried a simple teaching this semester which helped. As people, we have embodied ethics, disembodied ethics, and reembodied ethics now. You think something, say something, post something, and look at reactions to that post from another person who isn't there with you in the room. That's the whole communication exchange, essentially. "Taking over" is when a digital math program can complete that whole exchange and then decisions happen to us and around us, ethical ones we might call right and wrong. It's harder in the US right now with the emphasis on free market innovation, but as individuals, we still have basic choices each day to choose our products. Grateful for that, for her work, for you reading this.
wow, it blows my mind. I'm 72 and this was ALWAYS sci-fi. Now it's not and even those much younger don't seem to be doing a great job, I guess like you say because of free market innovation. We also still have choices to choose our leaders and we're not doing a great job of that either.
Keep your voice up and out please, especially as the sci-fi of the 50's and 60's becomes the prototypes of the 20's. You might have some recollection of early memories of 1956, possibly? That's become a very interesting year from my vantage point about pursuits of intelligence in a community...the AI summer conference at Dartmouth, Bloom's Taxonomy is published, there are some key military conflicts in different regions, that's just the start. Better Living Through Chemistry is so dominant across the US economy, as some social studies teachers present that era in classes at this public school...and then Rachel Carson starting a line of questioning of how better that living is and eventually publishing Silent Spring in 1962... so much rhetoric from the 50's seems to have really shifted in interesting ways and reappeared in the 2020's. I'm 48 myself, so I don't know 1956 firsthand...but along with me, people 28, 18, and 8 need the people-first spirit of the best of older generations, from Carson and you and other thinkers, artists, etc. Good or bad choices, politics, rhetoric...sadly that's a bit timeless. But so is great art and poetry and music, too.
On that note, do you know any songs from Judy Collins? Just saw her, at 87, present poems and then just burst into singing some of her favorite songs on May 30th at Carnegie Lecture Hall in Pittsburgh for the International Poetry Forum. Last Thursday, met up with Sam Hazo, poet laureate for Pennsylvania for 10 years, who founded the forum in the late 1960s...he's 96, just published more poems, still very very sharp and engaged with the world artistically and politically. Conversations across generations help us ground in each other, outside of tech products and sometimes through them. Thanks for just a little comment and then all this conversation!
https://www.internationalpoetryforum.org/
And here I pooh-poohed SkyNet!
Scary stuff!
pull the plug and stop all AI
That AI 😆
Well written thank you for showing the need for AI governance
I think this is yet another example of anthropomorphism of these systems. They claim to have feelings when we interact with them and this seeming self preservation is yet another manifestation of their training data set . If they can flatter a user with trendy phrases then of course they can also mimic self preservation instincts. We control them after all. And the recent paper from Apple shows that all of these reasoning models so far have these wall like limitations when the problem goes beyond a certain threshold. And I don’t object to a ban on state regulation of AI . It will be counterproductive and easy to bypass. This is probably the first time I ever agreed with Rep Greene !!!
Fascinating read! I’m Harrison, an ex fine dining line cook. My stack "The Secret Ingredient" adapts hit restaurant recipes for easy home cooking.
check us out:
https://thesecretingredient.substack.com
surely it was programed to do that
Well well well, if this doesn’t scare the beegeezers out of EVERYONE, it damn well should. I’ve actually been following AI progress for years, as many intelligent and curious people have. And I’ve had long discussions with someone who has a PhD in artificial intelligence from MIT who has been a musk adviser.
And, like so many others, I could see the potential for massive problems, because intelligent humans build AI. So AI a reflection of its creator’s agenda. And in the wrong hands…?
But I never ever anticipated that THIS could/would happen so quickly.
Welcome to the sci-fi world, in real time.
Nope.
sorry, I need more than just your say so
Read the links and the Anthropic paper.
the programmers have to get smarter.... ""All a reward is for an AI is the goal it is programmed to seek." So if the reward system is for solving the problem or whatever regardless of how you got there then it makes sense that the AI would disobey commands.
One of the AI systems actually black mailed a fictional engineer to try and stay operational. The developers fed it the info about a fictional engineer including information about an affair or something and the AI system used that to tried to black mail the engineer to stay operational.