回看23年的llm学习

LLM核心论文23篇

1. Sentiment Neuron: Learning to Generate Reviews and Discovering Sentiment
2. GPT-1: Improving Language Understanding by Generative Pre-Training
3. Scaling Law: Scaling Laws for Neural Language Models
4. GPT-3: Language Models are Few-Shot Learners

价值对齐

1. InstructGPT: Training language models to follow instructions with human feedback
2. Constitutional AI: Harmlessness from AI Feedback

架构

1. Transformer: Attention is All You Need
2. T5: Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
3. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

轻量微调

1. Rethinking Efficient Tuning Methods from a Unified Perspective

推理时算法

1. Chain-of-Thought: Chain-of-Thought Prompting Elicits Reasoning in Large Language Models

榜单

1. MMLU: Measuring Massive Multitask Language Understanding
2. MATH: Measuring Mathematical Problem Solving With the MATH Dataset

多模态

1. Multimodal Few-Shot Learning with Frozen Language Models
2. CLIP: Learning Transferable Visual Models From Natural Language Supervision
3. Flamingo: a Visual Language Model for Few-Shot Learning

高观点

1. Pretrained Transformers as Universal Computation Engines
2. Large Language Models as General Pattern Machines
3. An Observation on Generalization

讲座

1. Ilya's talk at GTC2023
2. Alec Radford on LM (Youtube)

过去与未来

1. Learning Meaning in Natural Language Processing — The Semantics Mega-Thread
2. What will GPT-2030 look like

或许有25年的25篇 但是我还没整理 待更新