The GPT-4 paper, without useful detail on model size, parameters, training methods other than a generic 2017 RLHF reference
ChatGPT-Plus subscribers will get capped GPT-4 access.
1 Like
Does the mention of 8K context window - 32K context window, refer to the maximum tokens (like the 4000 tokens in text-davinci-003) ?
2 Likes
yes, exciting, but more expensive
1 Like
100 messages are fair to me.
Training cut off also September 2021?
Is this true?
UPDATE:
Iāve watched their video announcement and Greg also confirmed training data cut-off Sept 2021
1 Like
Now that is unexpected in the extreme, so I share your doubts. It makes it seem like it is using the exact same carefully pre-selected training corpus, but simply allowed to run longer and deeper in building the associations - think of it like the neural connections between memories and thoughts in the mind - so that it has a deeper and broader conceptualization of the exact same dataā¦ Definitely not what weād have expected.
That would indeed be a rather fast, ādown and dirtyā way of building the new version very quickly, as the preselection of training data has to be one of the most human-intensive parts of the whole business (though certainly not the most expensive - that is always the runtime).
2 Likes
The GPT-4 paper, without useful detail on model size, parameters, training methods other than a generic 2017 RLHF reference
2 Likes
One thing I certainly noticed is that there was absolutely no mention at all of multi-modal output, even while showing us very clearly a detailed level of multi-modal input (the ability to process image input and analyse the image in depth).
Naturally, I now wonder if the Microsoft Suit doing his talk misunderstood where the multi-modality would be working, or if this is something they just didnāt want to talk about yet (but that would seem odd given a senior MS suit already announcing it and so it clearly being āoutā in the public domain).
2 Likes
The video was great. He was very nervous because there is a lot at stake, but he presented it great.
The ānoteā to the finished website story was of course awesome.
The great thing about the story is: if they have a running system now, it can optimize itself or give the developers hints on how to further optimize and improve it.
About Discord: Christoph, maybe you underestimate Discord a little bit. Discord Developer Portal - You can do a lot of cool things with itā¦
2 Likes
Itās not so much that we donāt know what amazing and cool things can be done with Discord, @Georg_Franz but more that we moved from Facebook largely because we didnāt want to be reliant upon (and beholden to) a third-party that even theoretically could take away our own access to our own community.
With a forum, great old tech as it is, the data is entirely our own, and Christoph always has full and ultimate admin rights over it. When building a community is a serious consideration, that ownership is a major consideration. Just like the difference between owning the land where your business is located, or renting it on contracts, and always accepting that the landlord could sell your contract, or their whole business, or evict you, just because they chose to and it suited their current objectives, not yoursā¦
1 Like
Thatās it, self-control.
I couldnāt care less about fancy features when I risk I could lose control over night, without reasonable recourse.
1 Like
GPT-4 API Pricing review
GPT-4 for prompts is 14x more expensive than the ChatGPT API;
GPT-4 for completions is 29x more expensive than the ChatGPT API.
But ChatGPT 3.5 just dropped to 10% over GPT3, so itās more like a 3x.
2 Likes