AI Glossary

0-9

3D Reconstruction What is 0-shot learning What is 1-shot learning What is 2-stage detector What is 3D convolution What is 4D data What is 5G + AI What is 6DoF pose estimation What is 7D representation What is 8-bit quantization What is 9-layer network

A

A/B Testing Accountability Accuracy Acoustic Modeling Activation Functions Active Learning Actor-Critic Methods Actuators AdaDelta AdaGrad Adam Optimizer Adjusted R-Squared Adversarial Attacks Affordance Learning Agent-Based Modeling Agentic AI / Autonomous Agents Agentic AI Frameworks Agglomerative Clustering AI Accelerators AI Act (EU)AI Agents AI Alignment AI and Bias AI and Sustainability AI APIs AI Art Generation AI Assistants AI Audit AI Auditing AI Bill of Rights (US Blueprint)AI Containment AI Democratization AI Ethics Boards AI Ethics Guidelines AI Feature Store AI for Climate Change AI Generated Content AI Governance Frameworks AI Guardrails AI Hallucinations AI in Healthcare Ethics AI in Warfare AI Legislation AI Literacy AI Marketplaces AI Model Governance AI Model Hub AI Model Registry AI Model Weights AI Music Generation AI Orchestration AI Policy AI Regulations AI Safety AI Security AI Singularity AI Transparency Report AI Watermarking AI Winter AI Workflow Automation AI-as-a-Service Alan Turing Algorithmic Accountability Algorithmic Bias Mitigation Algorithmic Discrimination Algorithmic Transparency Andrew Ng Anomaly Detection Anomaly Detection in Security Anthropic Apache Kafka API Development API Endpoints Apriori Algorithm Artificial General Intelligence (AGI)Artificial Neural Networks Artificial Superintelligence ASICs Association Rule Learning Asynchronous Advantage Actor-Critic Attention Mechanisms AUC Audio Classification Audio Signal Processing Augmented Reality Authentication Authorization Autoencoders Automated Reasoning Automatic Speech Recognition (ASR)Automation AutoML Autonomous Navigation Autoregressive Models What is AGI / Artificial General Intelligence What is Algorithm What is Artificial Intelligence (AI)What is Attention What is Autoencoder

B

C

Caffe Calculus Calibration California Consumer Privacy Act (CCPA)Canary Deployment Capsule Networks Carbon Footprint of AI Case-Based Reasoning Catastrophic Forgetting Central Limit Theorem Chain-of-Thought Chinese Room Argument Class Imbalance Classification Cloud AI Platforms Cloud Computing Clustering Algorithms Code Generation Models Cognitive Architectures Cognitive Computing Cohere Colab Notebooks Collaborative Filtering Color Spaces Complex Analysis Compliance Compliance Standards (ISO IEEE)Computational Complexity Computational Fluid Dynamics Computational Theory of Mind Compute-Optimal Models Concept Drift Conceptual Graphs Conditional Probability Confusion Matrix Consciousness in AI Consistency Models Constitutional AI Constraint Satisfaction Problems Containerization Content-Based Filtering Context Window Continual Learning Continuous Integration/Continuous Deployment (CI/CD)Control Systems Conversational AI Convolutional Neural Networks COPPA Coreference Resolution Correlation Correlation Matrix Cost-Sensitive Learning Cross-Entropy Loss Curriculum Learning Cyber Threat Intelligence Cybersecurity Regulations What is Chatbot What is Classifier / Classification What is Clustering What is CNN / Convolutional Neural Network What is Cross-Validation

D

DALL·E Data Annotation Data Catalog Data Centers Data Cleaning Data Drift Data Governance Data Ingestion Data Integration Data Labeling Data Lake Data Lakes Data Leakage Data Lineage Data Mining Data Pipeline Data Poisoning Data Preprocessing Data Privacy Data Protection Data Protection Laws Data Quality Data Security Data Sovereignty Data Transformation Data Versioning Data Visualization Data Visualization Techniques Data Warehousing Databases for AI Davies-Bouldin Index DBSCAN Decision Boundary Visualization Decision Trees Deep Belief Networks Deep Q-Networks Deep Reinforcement Learning Deepfakes DeepMind Demis Hassabis Dependency Parsing Depth Estimation Descriptive Statistics Dialogue Systems Differential Equations Differential Evolution Differential Privacy Diffusion Models Digital Divide Digital Provenance Digital Twins Dimensionality Reduction Direct Preference Optimization (DPO)Discourse Analysis Discrete Event Simulation Discrete Mathematics Disinformation Distributed Computing Distributed File Systems Distributed Training Docker Drones Dropout Dropout Regularization Dynamical Systems What is Data Augmentation What is Deep Learning What is Deepfake What is Deterministic Model What is Discriminative Model

E

F

G

H

I

J

John McCarthy Joint Probability Distribution Juergen Schmidhuber Jupyter Notebooks What is JAX What is Jittering What is Joint Embedding What is JSONL / JSON-lines What is Juxtaposition

K

K-Nearest Neighbors Kai-Fu Lee Kalman Filters Keras Knowledge Cutoff Knowledge Graphs Knowledge Representation Kubernetes What is K-means Clustering What is K-Shot Learning What is Kernel Trick What is KL Divergence (Kullback–Leibler Divergence)What is Knowledge Distillation

L

M

Machine Consciousness Machine Translation Markov Chain Models Markov Chain Monte Carlo Markov Decision Processes Markov Models Marvin Minsky Masked Language Models Master Data Management Matplotlib Matrix Decomposition MCP Mean Absolute Error Mean Squared Error Mechanistic Interpretability Mel-Frequency Cepstral Coefficients (MFCCs)Metadata Management Microservices Midjourney Mind Uploading Mini Tool Mini-Batch Gradient Descent Mixture of Experts (MoE)MLOps Mobile Robots Model Cards Model Compression Model Deployment Model Drift Model Explainability Tools Model Monitoring Model Serving Model Stealing Momentum Optimization Monitoring and Logging Monte Carlo Methods Monte Carlo Simulations Moral Machines Motion Detection Motion Planning Multi-Armed Bandit Problem Multimodal AI Music Information Retrieval MXNet What is Machine Learning (ML)What is Meta-learning What is Model What is Multi-head Attention What is Multimodal / Multimodality

N

O

Object Detection Object Tracking Ontologies OpenAI OpenAI GPT Optical Character Recognition Optimization Theory Out-of-Distribution (OOD) Data What is Objective Function What is One-hot Encoding What is Online Learning What is Optimizer What is Overfitting

P

Pandas Parallel Computing Parameter Count Parameter-Efficient Fine-Tuning (PEFT)Part-of-Speech Tagging Partial Dependence Plots Path Planning Pattern Recognition People also viewed Perception in AI Perceptron Perplexity Peter Norvig Philosophy of Mind Phonetics Pipelines Planning and Scheduling Plotly Policy Gradients Policy Optimization Pose Estimation Positional Encoding Pragmatics Precision Predictive Modeling Predictive Probability Preference Tuning Principal Component Analysis Privacy Privacy-Preserving Machine Learning Probability Density Functions Probability Theory Problem Solving Process Modeling Process-Based Supervision Prompt Chaining Prompt Engineering Prompt Injection Prompt Marketplace Prompt Templates Propositional Logic Proximal Policy Optimization Pruning PyTorch What is Parameter What is Policy / Reinforcement Learning Policy What is Pooling What is Pretraining What is Prompt

Q

QLoRA (Quantized Low-Rank Adaptation)Quantum Computing Quantum Machine Learning Question Answering Question Answering Systems What is Q-learning What is Quality Estimation What is Quantization What is Query What is Queue / Buffer

R

S

Saliency Maps SARSA Algorithm Scalable Oversight Scaling Laws Scatter Plot Scikit-Learn SciPy Seaborn Search Algorithms Secure Hardware Secure Multi-Party Computation Secure Protocols Self-Attention Self-Driving Cars Semantic Networks Semantic Parsing Semantic Role Labeling Semantic Segmentation Semantic Web Semi-Supervised Learning Sensors SentencePiece Sentiment Analysis Sequence Labeling Serverless Computing Serverless GPUs Set Theory SHAP Values Siamese Networks SIFT Silhouette Score Simulated Annealing Simulation Hypothesis Simulation-to-Real Transfer (Sim2Real)Simultaneous Localization and Mapping (SLAM)SMOTE Social Acceptance of AI Social Simulation SOTA (State of the Art)spaCy Spark Speaker Diarization Spectrogram Analysis Speech Enhancement Speech Recognition Speech Synthesis Spiking Neural Networks SQL Stable Diffusion Stacking State-Action Pairs Statistical Analysis Statistical Distributions Statistics Stemming Stochastic Gradient Descent Stochastic Modeling Stochastic Processes Stop Words Stream Processing Strong AI Strong vs. Weak AI Stuart Russell Style Transfer Subword Tokenization Support Vector Machines SURF Surveillance Swarm Intelligence Symbolic AI Synthetic Data Generation Synthetic Media System Dynamics System Prompt What is Sampling What is Self-Supervised Learning What is Sequence Modeling What is Softmax What is Supervised Learning

T

U

UMAP Unmanned Aerial Vehicles (UAVs)Unmanned Ground Vehicles What is U-Net What is Uncertainty Estimation What is Underfitting What is Universal Approximation Theorem What is Unsupervised Learning

V

Validation Curve Value Function Vector Database Version Control for Models Vibe code an AI Tool Video Generation Models Virtual Reality Simulations Voice Biometrics Voice Cloning Voice Conversion What is Validation Set What is Vanishing / Exploding Gradient What is Variational Autoencoder (VAE)What is Vector Embedding What is Vision Transformer (ViT)

W

Warmup Steps Weak AI What is Weak Supervision What is Weight Decay What is Whitening / Whitening Transformation What is Word Embedding What is Workflow Word Embeddings Word Sense Disambiguation WordPiece World Models

X

What is X-axis / feature axis What is XAI / Explainable AI What is XLM What is XLNet What is XOR problem

Y

What is Y-axis / feature axis What is Y-transform / YUV What is YAGNI (You Aren't Gonna Need It)?What is Yield (model yield / throughput)What is Yoga of AI Yann LeCun Yoshua Bengio

Z

What is Z-score Normalization What is Zero-centric / Zero-bias initialization What is Zero-gradient phenomenon What is Zero-shot Learning / Zero-shot inference What is Zygosity in augmentation Zero Trust Architecture

Markov Decision Processes

mɑrˈkɔv dɪˈsɪʒən ˈprɒsɛsɪz

Last updated: March 11, 2026

Markov Decision Processes (MDPs) are mathematical frameworks used to model decision-making in situations where outcomes are partly random and partly under the control of a decision maker. An MDP is defined by its states, actions, transition probabilities, and rewards. It provides a way to formalize the process of making a sequence of decisions to maximize some notion of cumulative reward. MDPs are widely used in various fields, including robotics, automated control, and economics, particularly where optimal decision-making is crucial under uncertainty.