Glossary

0-9

1-shot learning 2-stage detector 3D convolution 4D data 5G + AI 6DoF pose estimation 7D representation 8-bit quantization 9-layer network 0-shot learning

A

AGI / Artificial General Intelligence Artificial Intelligence (AI)Attention Algorithm Autoencoder

B

BERT Boosting Bias Backpropagation Batch Normalization

C

Cross-Validation Classifier / Classification CNN / Convolutional Neural Network Chatbot Clustering

D

Deepfake Discriminative Model Data Augmentation Deep Learning Deterministic Model

E

Epoch Explainable AI (XAI)Embedding Encoder Ensemble Learning

F

Forward Propagation Foundation Model Fine-tuning Feature Extraction Fusion / Multimodal Fusion

G

GAN / Generative Adversarial Network Grounding Graph Neural Network (GNN)Gradient Descent Generative AI

H

Hierarchical Model Hallucination Heuristic Hyperparameter Hidden Layer

I

Instruction tuning Imbalanced Data Instance / Sample Intelligence Amplification / Augmentation Interpretability

J

JAX JSONL / JSON-lines Juxtaposition Jittering Joint Embedding

K

K-Shot Learning Kernel Trick KL Divergence (Kullback–Leibler Divergence)K-means Clustering Knowledge Distillation

L

LSTM / Long Short-Term Memory Large Language Model (LLM)Learning Rate Loss Function Latent Variable

M

Meta-learning Multimodal / Multimodality Multi-head Attention Model Machine Learning (ML)

N

NLU / Natural Language Understanding Novelty Detection / Anomaly Detection Normalization Neural Network NLP / Natural Language Processing

O

One-hot Encoding Optimizer Objective Function Online Learning Overfitting

P

Prompt Pooling Parameter Policy / Reinforcement Learning Policy Pretraining

Q

Q-learning Quality Estimation Query Quantization Queue / Buffer

R

RNN / Recurrent Neural Network Reinforcement Learning (RL)Retrieval Augmented Generation (RAG)Regularization Representation Learning

S

Softmax Sampling Sequence Modeling Supervised Learning Self-Supervised Learning

T

Tokenizer Transformer Training Data Tuning / Hyperparameter Tuning Transfer Learning

U

U-Net Uncertainty Estimation Underfitting Unsupervised Learning Universal Approximation Theorem

V

Vision Transformer (ViT)Vector Embedding Vanishing / Exploding Gradient Variational Autoencoder (VAE)Validation Set

W

Weight Decay Workflow Weak Supervision Whitening / Whitening Transformation Word Embedding

X

X-axis / feature axis XLM XLNet XAI / Explainable AI XOR problem

Y

YAGNI (You Aren't Gonna Need It)Yoga of AI Y-transform / YUV Y-axis / feature axis Yield (model yield / throughput)

Z

Z-score Normalization Zygosity in augmentation Zero-centric / Zero-bias initialization Zero-gradient phenomenon Zero-shot Learning / Zero-shot inference

什麼是多頭注意力

多頭注意力是一種在深度學習中被廣泛使用的機制，特別是在自然語言處理（NLP）和計算機視覺（CV）領域。它最初在Transformer模型中提出，變革了序列到序列的學習任務。多頭注意力的核心思想是將輸入特徵向量分成多個子空間，透過多個“頭”並行處理，從而捕捉輸入資料中的不同特徵和關係。

多頭注意力的運作方式是，首先將輸入資料通過線性變換分成多組，然後每組獨立地計算注意力權重並生成輸出。最後，這些輸出會被連接並通過另一線性變換合併。這種機制增強了模型的表達能力和效率。

在應用場景方面，多頭注意力被應用於機器翻譯、文本生成和圖像識別等任務中。由於其靈活性和高效性，它已經成為許多現代深度學習模型的核心組件之一。未來，隨著計算資源的增加和模型架構的持續演進，多頭注意力有望在更多領域得到應用。

然而，它也有一些缺點，例如在處理長序列時計算開銷較大，可能導致性能下降。因此，在設計模型時需要考慮到這些因素。