Device
Big iPhone
iPad
Small iPhone in Twitter
9:41
localhost:5173
ACT 1
PRETRAINING
You are a language model in early training.
LOSS
4.34
COMPUTE
85M
CONTEXT
172
SEQUENCE WINDOW
|
CHOOSE NEXT TOKEN
"prediction"
p=0.35
"the"
p=0.40
"he"
p=0.23
"can"
p=0.03
UPGRADES
+ More Parameters
+ Larger Context
+ Code Corpus
+ Dialogue Corpus
Later, this same UI starts to look intentional.