Glossary

0-9

1-shot learning 2-stage detector 3D convolution 4D data 5G + AI 6DoF pose estimation 7D representation 8-bit quantization 9-layer network 0-shot learning

A

Artificial Intelligence (AI)Algorithm Attention Autoencoder AGI / Artificial General Intelligence

B

BERT Boosting Backpropagation Batch Normalization Bias

C

Classifier / Classification Cross-Validation Clustering CNN / Convolutional Neural Network Chatbot

D

Discriminative Model Deterministic Model Deep Learning Data Augmentation Deepfake

E

Epoch Encoder Ensemble Learning Explainable AI (XAI)Embedding

F

Foundation Model Forward Propagation Fusion / Multimodal Fusion Feature Extraction Fine-tuning

G

GAN / Generative Adversarial Network Grounding Generative AI Gradient Descent Graph Neural Network (GNN)

H

Hierarchical Model Hyperparameter Hallucination Heuristic Hidden Layer

I

Interpretability Instruction tuning Imbalanced Data Instance / Sample Intelligence Amplification / Augmentation

J

JAX JSONL / JSON-lines Juxtaposition Jittering Joint Embedding

K

K-Shot Learning Kernel Trick KL Divergence (Kullback–Leibler Divergence)Knowledge Distillation K-means Clustering

L

LSTM / Long Short-Term Memory Large Language Model (LLM)Latent Variable Loss Function Learning Rate

M

Multimodal / Multimodality Multi-head Attention Machine Learning (ML)Model Meta-learning

N

NLU / Natural Language Understanding Normalization Neural Network Novelty Detection / Anomaly Detection NLP / Natural Language Processing

O

One-hot Encoding Overfitting Objective Function Optimizer Online Learning

P

Prompt Parameter Pretraining Policy / Reinforcement Learning Policy Pooling

Q

Q-learning Query Queue / Buffer Quantization Quality Estimation

R

RNN / Recurrent Neural Network Representation Learning Retrieval Augmented Generation (RAG)Reinforcement Learning (RL)Regularization

S

Softmax Sampling Supervised Learning Sequence Modeling Self-Supervised Learning

T

Tokenizer Transformer Tuning / Hyperparameter Tuning Transfer Learning Training Data

U

U-Net Uncertainty Estimation Underfitting Universal Approximation Theorem Unsupervised Learning

V

Vision Transformer (ViT)Variational Autoencoder (VAE)Vector Embedding Vanishing / Exploding Gradient Validation Set

W

Weight Decay Word Embedding Workflow Whitening / Whitening Transformation Weak Supervision

X

X-axis / feature axis XLM XLNet XAI / Explainable AI XOR problem

Y

Yoga of AI Y-transform / YUV Yield (model yield / throughput)Y-axis / feature axis YAGNI (You Aren't Gonna Need It)

Z

Z-score Normalization Zero-gradient phenomenon Zero-shot Learning / Zero-shot inference Zero-centric / Zero-bias initialization Zygosity in augmentation

什么是正则化

正则化是一种用于统计建模和机器学习的技术，旨在防止过拟合。过拟合是指模型在训练数据上表现良好，但在新数据上却无法泛化，导致预测不准确。通过引入额外的约束或惩罚项，正则化有助于简化模型，提高其在未见数据上的表现。

一方面，正则化通过增加一个惩罚项（如 L1 或 L2 范数）来抑制复杂模型的影响，促使模型学习更简单的结构，这通常能提高模型的泛化能力。常见的正则化方法包括岭回归（L2 正则化）和套索回归（L1 正则化）。这些方法在许多实际应用中都表现出色，例如在图像识别和自然语言处理任务中。

另一方面，虽然正则化有助于改善模型的稳定性和预测能力，但它也可能导致信息的丢失，尤其是在数据量较小的情况下。此外，选择合适的正则化参数也是一项挑战，过强的正则化可能会导致欠拟合。

未来，随着数据集的不断扩大和计算能力的提升，正则化技术也在不断演化。例如，新的正则化方法如 dropout 和 batch normalization 等正逐渐被广泛接受，显示出在深度学习中的重要性。总的来说，正则化是构建高效且稳健模型的关键手段，其重要性在机器学习的不断发展中只会愈加突出。