Senior Problems

Implement transformer block used in LLMs

Zero vs zero2 vs zero3

RetinaNet Theory

Discuss RMSProp optimization algorithm

How does Adagrad enhance the process of stochastic gradient descent optimization?

What limitations might one encounter when using Adagrad in neural network optimization?

How do binary encoding schemes like Protobuf or Parquet compare favorably against textual formats such as CSV or JSON?

Which option correctly defines a dot-product attention mechanism for a query q and a vector u

What are the benefits of using byte-pair encoding compared to traditional word-level tokenization?

What foundational concept underpins Google's BERT model training?