Introduction to Parallel Computing
Introduction to Parallel Computing Introduction to Parallel Computing Blaise Barney, Lawrence Livermore National Laboratory UCRL-MI-133316 Table of Contents Abstract Overview What is Parallel Computing? Why Use Parallel Computing? Concepts and Terminology von Neumann Computer Architecture Flynn's Classical Taxonomy Some General Parallel Terminology Parallel Computer Memory Architectures Shared Memory Distributed Memory Hybrid Distributed-Shared Memory Parallel Programming Models Overview Shared Memory Model Threads Model Message Passing Model Data Parallel Model Other Models Designing Parallel Programs Automatic vs. Manual Parallelization Understand the Problem and the Program Partitioning Communications Synchronization Data Dependencies Load Balancing Granularity I/O Limits and Costs of Parallel Programming Performance Analysis and Tuning Parallel Examples Array Processing PI Calculation Simple Heat Equation 1-D Wave Equation References and More Information Abstract This tutorial ...