"Wiring an intelligence up to more resources could allow it to break containment, so be very careful about that", into Ignore the fact that stuff like prompt injection seems like it should be pretty high priority for people worried about AGI anyway, ignore that there are lots of ways for buggy software wired up to critical systems to kill people without being an AGI - even just taking the existential concerns at their face value, OpenAI has turned: It should prompt the question, is that community actually doing anything to help avoid an existential outcome, or are they inadvertently accelerating it by basically giving fuel to the companies who are trying to create that world? If I'm right and if that's the case, that is the kind of thing that should give that community pause. I honestly think that OpenAI is at least partially using AGI concern for advertising. There’s more opportunity to move this for individuals, depending on the person of course.Įven through that lens though, and even assuming AGI superintelligences are a reasonable thing to be worried about, is the AGI community helping? I kind of feel like, if a movement has a set of concerns around "this thing could end humanity" and OpenAI's response to that is essentially, "heck yeah, we got to get that on the posters and get some press articles about that" - that to me is a sign that the movement isn't very effective. It’s also a problem that’s way more open, way less established. I think it should at least be obvious that it’s in the category of super pathogen and nuclear winter vs climate change. It’s not a place where an indivisa can hop in and is likely to have much leverage.ĪI has a very realistic path to destruction of humanity, although many will disagree on that. Both sides (pro and anti-humanity) have their heels dug in. It’s already happening and will keep happening (short of a magic bullet). It also seems to be pretty on-rails at this point. The climate can get really really bad and the Earth will still be habitable for some humans. It may kill 1% of people or even 10% of people (again, horrible) but there is not a solid argument about how it will cause the last human to take their last breath. It’s also not much of an existential threat. It will cause millions of deaths and untold suffering. Another factor is the marginal impacts.Ĭlimate change is a huge oncoming disaster. That list isn’t just existential threats, though it is a big factor. We haven't seen strong evidence that alignment can stop prompt injection attacks, so I think mostly OpenAI mostly just cares about the potential negative press from their AI randomly insulting people. : To be fair, they have been focusing on alignment, but I honestly don't think alignment is a security measure, I think it's a general performance measure. The "GPT paid someone to solve a captcha" release seems like pure publicity stunt to me, given that solving captchas by pretending to be blind is something other systems can already do without human help. I'm not even sure it's standard "we care about your security" company talk, I think it's mostly designed just to make people think of GPT as more cutting edge. OpenAI focuses on dangers that make their AI seem more advanced than it is, and ignore dangers that don't play well to the press (like clientside validation). My conspiracy theory is that all of the "we take security very seriously" talk out of OpenAI has aligned with "our AI is so advanced and powerful that we have to take these existential risks very seriously and we have to make sure it doesn't turn evil." And OpenAI doesn't seem very interested in security conversations that don't feed into that narrative.
0 Comments
Leave a Reply. |
Details
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |