OpenAI know they have to move fast because DeepMind can throw a lot more hardware and expertise at the problem. 25,000 machines isnāt even stretching Googleās cloud. I think thatās less than the hardware in just one of their many, many datacentres.
However, looking at Bard I think I figured out where OpenAI have their advantage. You see, for all the years of NLP usage and data Google have, it was all about searches. And people frame a search very differently to the way they write a prompt for a chatbot. Itās a radically different kind of NLP that is needed, and I just have the feeling that Bard didnāt account for that, and that Googleās data actually made them go a little in the wrong direction.
Meanwhile, OpenAI have years of GPT data, the usage of Jasper, etc. So while they donāt have the hardware or expertise that Google have, they did have a much more applicable source of training for the right kind of NLP. But that advantage lessens every single day that Bard can be gathering usage data, and while DeepMind are actively working on a more chat-like, conversational kind of NLP to be using.
If you remember, Google actively chose their Dialog based LLM (LaMDA) over their more accurate PaLM-based LLM that I (and many others) had expected. Although Google have switched to more PaLM based systems according to recent news from the CEO. So while my guess may be wrong, it is not an entirely random or blind guess, and it does seem to fit some of what we knowā¦
Anyway, the main thing is that OpenAI moved first, and they need to keep that momentum going as long as they can. The more time passes, the greater the odds that Google will catch up and even overtakeā¦ Unless ChatGPT can leverage that first-mover advantage and gain the funding and resources to scale up and hope to even the odds while they still have the lead.
Iām pretty certain weāre nowhere close to AGI yet. I mean, for sure I think that GPT5 might be able to pass the Turing Test that used to be the benchmark for āproperā AI, but thatās because the test is based on chatting, and ChatGPT is a dedicated Chatbot. AGI has to be versatile, able to be intelligent in all sorts of applications, just as a person is. In other words, it has to go far beyond just writing like a human, or chatting to fool a human. It needs to be capable of actual reasoning, of figuring out knowledge it was never trained on - the dangerous stuff. If we were 6 months from that weād be hearing a lot more than just asking for a short pause - weād be hearing of major protests and calls for backtracking and turning off the AIsā¦
