Glossary

0-9

1-shot learning 2-stage detector 3D convolution 4D data 5G + AI 6DoF pose estimation 7D representation 8-bit quantization 9-layer network 0-shot learning

A

AGI / Artificial General Intelligence Artificial Intelligence (AI)Attention Algorithm Autoencoder

B

BERT Boosting Bias Backpropagation Batch Normalization

C

Cross-Validation Classifier / Classification CNN / Convolutional Neural Network Chatbot Clustering

D

Deepfake Discriminative Model Data Augmentation Deep Learning Deterministic Model

E

Epoch Explainable AI (XAI)Embedding Encoder Ensemble Learning

F

Forward Propagation Foundation Model Fine-tuning Feature Extraction Fusion / Multimodal Fusion

G

GAN / Generative Adversarial Network Grounding Graph Neural Network (GNN)Gradient Descent Generative AI

H

Hierarchical Model Hallucination Heuristic Hyperparameter Hidden Layer

I

Instruction tuning Imbalanced Data Instance / Sample Intelligence Amplification / Augmentation Interpretability

J

JAX JSONL / JSON-lines Juxtaposition Jittering Joint Embedding

K

K-Shot Learning Kernel Trick KL Divergence (Kullback–Leibler Divergence)K-means Clustering Knowledge Distillation

L

LSTM / Long Short-Term Memory Large Language Model (LLM)Learning Rate Loss Function Latent Variable

M

Meta-learning Multimodal / Multimodality Multi-head Attention Model Machine Learning (ML)

N

NLU / Natural Language Understanding Novelty Detection / Anomaly Detection Normalization Neural Network NLP / Natural Language Processing

O

One-hot Encoding Optimizer Objective Function Online Learning Overfitting

P

Prompt Pooling Parameter Policy / Reinforcement Learning Policy Pretraining

Q

Q-learning Quality Estimation Query Quantization Queue / Buffer

R

RNN / Recurrent Neural Network Reinforcement Learning (RL)Retrieval Augmented Generation (RAG)Regularization Representation Learning

S

Softmax Sampling Sequence Modeling Supervised Learning Self-Supervised Learning

T

Tokenizer Transformer Training Data Tuning / Hyperparameter Tuning Transfer Learning

U

U-Net Uncertainty Estimation Underfitting Unsupervised Learning Universal Approximation Theorem

V

Vision Transformer (ViT)Vector Embedding Vanishing / Exploding Gradient Variational Autoencoder (VAE)Validation Set

W

Weight Decay Workflow Weak Supervision Whitening / Whitening Transformation Word Embedding

X

X-axis / feature axis XLM XLNet XAI / Explainable AI XOR problem

Y

YAGNI (You Aren't Gonna Need It)Yoga of AI Y-transform / YUV Y-axis / feature axis Yield (model yield / throughput)

Z

Z-score Normalization Zygosity in augmentation Zero-centric / Zero-bias initialization Zero-gradient phenomenon Zero-shot Learning / Zero-shot inference

什麼是 KL 散度（Kullback–Leibler 散度）

Kullback-Leibler 散度（KL 散度）是信息論和統計學中的一個基本概念，用於量化兩個概率分布之間的差異。它在機器學習、統計學和信息檢索等領域得到廣泛應用。KL 散度的數值越小，表示兩個分布越相似；相反，數值越大則表示它們之間的差異越大。

KL 散度的公式定義為：
D_{KL}(P || Q) = ∑ P(i) log(P(i)/Q(i))，其中 P 和 Q 分別是兩個概率分布。對於非負概率分布，KL 散度是非負的，且僅在 P 和 Q 完全相等時等於 0。KL 散度的一個顯著特點是不可對稱，即 D_{KL}(P || Q) 不等於 D_{KL}(Q || P)。

在實際應用中，KL 散度常用於模型評估、生成模型的訓練和信息壓縮。例如，在機器學習中，優化算法可能會通過最小化 KL 散度來調整模型的預測分布，使其更接近真實數據分布。

未來趨勢方面，隨著深度學習和大數據技術的發展，KL 散度可能會與其他信息度量方法結合，形成更複雜的模型，用於處理更高維度的數據。

KL 散度的優點在於其數學性質簡單且易於計算，但缺點在於其對零概率事件敏感，可能會導致不穩定的結果。在使用時需注意確保輸入的概率分布都是有效的。