Glossary

0-9

1-shot learning 2-stage detector 3D convolution 4D data 5G + AI 6DoF pose estimation 7D representation 8-bit quantization 9-layer network 0-shot learning

A

Artificial Intelligence (AI)Algorithm Attention Autoencoder AGI / Artificial General Intelligence

B

BERT Boosting Backpropagation Batch Normalization Bias

C

Classifier / Classification Cross-Validation Clustering CNN / Convolutional Neural Network Chatbot

D

Discriminative Model Deterministic Model Deep Learning Data Augmentation Deepfake

E

Epoch Encoder Ensemble Learning Explainable AI (XAI)Embedding

F

Foundation Model Forward Propagation Fusion / Multimodal Fusion Feature Extraction Fine-tuning

G

GAN / Generative Adversarial Network Grounding Generative AI Gradient Descent Graph Neural Network (GNN)

H

Hierarchical Model Hyperparameter Hallucination Heuristic Hidden Layer

I

Interpretability Instruction tuning Imbalanced Data Instance / Sample Intelligence Amplification / Augmentation

J

JAX JSONL / JSON-lines Juxtaposition Jittering Joint Embedding

K

K-Shot Learning Kernel Trick KL Divergence (Kullback–Leibler Divergence)Knowledge Distillation K-means Clustering

L

LSTM / Long Short-Term Memory Large Language Model (LLM)Latent Variable Loss Function Learning Rate

M

Multimodal / Multimodality Multi-head Attention Machine Learning (ML)Model Meta-learning

N

NLU / Natural Language Understanding Normalization Neural Network Novelty Detection / Anomaly Detection NLP / Natural Language Processing

O

One-hot Encoding Overfitting Objective Function Optimizer Online Learning

P

Prompt Parameter Pretraining Policy / Reinforcement Learning Policy Pooling

Q

Q-learning Query Queue / Buffer Quantization Quality Estimation

R

RNN / Recurrent Neural Network Representation Learning Retrieval Augmented Generation (RAG)Reinforcement Learning (RL)Regularization

S

Softmax Sampling Supervised Learning Sequence Modeling Self-Supervised Learning

T

Tokenizer Transformer Tuning / Hyperparameter Tuning Transfer Learning Training Data

U

U-Net Uncertainty Estimation Underfitting Universal Approximation Theorem Unsupervised Learning

V

Vision Transformer (ViT)Variational Autoencoder (VAE)Vector Embedding Vanishing / Exploding Gradient Validation Set

W

Weight Decay Word Embedding Workflow Whitening / Whitening Transformation Weak Supervision

X

X-axis / feature axis XLM XLNet XAI / Explainable AI XOR problem

Y

Yoga of AI Y-transform / YUV Yield (model yield / throughput)Y-axis / feature axis YAGNI (You Aren't Gonna Need It)

Z

Z-score Normalization Zero-gradient phenomenon Zero-shot Learning / Zero-shot inference Zero-centric / Zero-bias initialization Zygosity in augmentation

什么是 KL 散度（Kullback–Leibler 散度）

Kullback-Leibler 散度（KL 散度）是信息论和统计学中的一个基本概念，用于量化两个概率分布之间的差异。它在机器学习、统计学和信息检索等领域得到广泛应用。KL 散度的数值越小，表示两个分布越相似；相反，数值越大则表示它们之间的差异越大。

KL 散度的公式定义为：
D_{KL}(P || Q) = ∑ P(i) log(P(i)/Q(i))，其中 P 和 Q 分别是两个概率分布。对于非负概率分布，KL 散度是非负的，且仅在 P 和 Q 完全相等时等于 0。KL 散度的一个显著特点是不可对称，即 D_{KL}(P || Q) 不等于 D_{KL}(Q || P)。

在实际应用中，KL 散度常用于模型评估、生成模型的训练和信息压缩。例如，在机器学习中，优化算法可能会通过最小化 KL 散度来调整模型的预测分布，使其更接近真实数据分布。

未来趋势方面，随着深度学习和大数据技术的发展，KL 散度可能会与其他信息度量方法结合，形成更复杂的模型，用于处理更高维度的数据。

KL 散度的优点在于其数学性质简单且易于计算，但缺点在于其对零概率事件敏感，可能会导致不稳定的结果。在使用时需注意确保输入的概率分布都是有效的。